{"id":"batch-job-monitoring-dashboards-24028","slug":"batch-job-monitoring-dashboards-24028","title":"Watching 20 Batch Jobs at Once","description":"Everyone talks about batch API discounts, but nobody talks about the babysitting. When you're running a speech fine-tune across thousands of clips or a personal classifier, the real problem isn't the price — it's knowing which job finished, which one expired, and where the results went. We walk through the actual state of batch monitoring: which providers now offer webhooks (OpenAI, Gemini, OpenRouter, Bedrock), which still only do polling (Anthropic), and why a webhook is a doorbell, not a dashboard. Then we sort the tools that try to fill the gap into four piles — vendor consoles, spreadsheets with an AI column, eval runners, and workflow builders — and figure out which one to reach for when you're staring at twenty batches in flight.","excerpt":"Batch APIs give you a discount, then hand you a babysitting job. Here's what actually watches a batch run — and what doesn't.","pubDate":"2026-10-20T18:52:14.953Z","tags":["fine-tuning","api-integration","automation"],"heroImage":"https://files.myweirdprompts.com/covers/batch-job-monitoring-dashboards-24028.png","podcastAudioUrl":"https://episodes.myweirdprompts.com/audio/batch-job-monitoring-dashboards-24028.m4a","podcastDuration":"21:51","episodeNumber":6165,"transcript":"Corn: Everybody writes about batch as if it's a voucher. Half price, come back tomorrow, job done. Daniel isn't really after the voucher. He's been running a lot of batch lately, for fine-tuning a speech model and for a personal classifier, and what's wearing him down is the babysitting. Several jobs in flight, and somebody keeps asking what happened to that one. The API has already partly fixed that, because some providers will call you back when a job finishes, so the real gap is the screen that sits above all of them.\n\nHerman: The babysitting is the part nobody writes about, because it's dull. The discount is easy to explain. Watching a queue is not. And a speech fine-tune is thousands of short clips, every clip a request. That's the shape batch was made for, and it's also the shape where one lost job ruins your afternoon.\n\nCorn: Dull is one word for it. I'd call it an unpaid second job.\n\nHerman: We went through the discount side in the batch pricing episode, so I'll leave that where it is. The callbacks are the good news. OpenAI has webhook events for batch completed, failed, cancelled and expired. Gemini has webhooks for its batch jobs too. OpenRouter takes a callback address when you create the batch. Bedrock pushes job state changes into EventBridge, so you can wire one of those up to an email.\n\nCorn: And Anthropic?\n\nHerman: Anthropic's batch docs describe polling and nothing else. I haven't seen a webhook for them anywhere. Their Console is where you'd watch it, so the watching is on you. The Console is per workspace, so a batch run in another workspace isn't on the screen you're looking at.\n\nCorn: So the complaint still stands for one of the big names. Good. A webhook isn't a dashboard, though. It's a doorbell, and somebody still has to answer it.\n\nHerman: The OpenRouter doorbell rings with the batch ID and nothing else. Your receiver then has to go and fetch the results itself. And the setup is API work too. I haven't seen a screen in AI Studio for setting Gemini's endpoint, so as far as I can tell it's code all the way down.\n\nCorn: So it rings the bell and doesn't say who's there.\n\nHerman: Which is fine if you're a developer with a server. Corn, if you were watching twenty batches at once, what would you want on the screen?\n\nCorn: A sign that says done.\n\nHerman: That's the whole product, and I haven't seen anyone ship it. I haven't seen a vendor-agnostic batch dashboard that does the job. So the gap is a screen that sits above all the providers. What fills it today falls into four piles: the vendor consoles, spreadsheets with an AI column, eval runners that behave like batch runners, and workflow tools where you join boxes together until something runs.\n\nCorn: Which pile would you bet on for a fine-tune?\n\nHerman: None of them, on its own. The consoles see the jobs, the sheets see the rows, and as far as I can tell nothing sees both at once.\n\nCorn: Typical. Upload, define, track, download. Those are the four verbs you're after.\n\nHerman: They are. Upload is easy everywhere. Define is where the tools differ, because some want a prompt per row and some want a prompt per column. Track is the complaint. Download is where the expiry dates bite.\n\nCorn: Define is the one I'd look at first. A prompt per column sounds like a spreadsheet, and a prompt per row sounds like a form.\n\nHerman: It matters a lot once you change the prompt halfway through a run. Then you want to know which rows were labelled by which version.\n\nCorn: Which nobody ever writes down.\n\nHerman: I'd start with the consoles, since most people already have one. Anthropic's Console lists batches per workspace, and you can download results from there, if your organization's permissions allow it. The catch is that results last twenty-nine days from creation. A batch can hold a hundred thousand requests, or two hundred and fifty-six megabytes, so a classifier won't hit those limits unless it's enormous.\n\nCorn: Twenty-nine days. It's a library book, except the library doesn't send a reminder.\n\nHerman: Still, it's the most complete native screen on this list. It lists your batches, and it downloads the results. That's three of the four verbs without writing a line of code.\n\nCorn: And the fourth one is the verb with the expiry date.\n\nHerman: Google's Vertex AI has a Batch Inference page in the console. You point it at a file in Cloud Storage or a BigQuery table, and it writes the output back to a bucket, or to a table it creates if one isn't there. Azure's Foundry portal has batch jobs you can create and track, with a Global-Batch deployment type for that. A Microsoft Q and A thread from May describes jobs sitting in validating, and a mismatch where the model name in the file has to match the deployment name exactly.\n\nCorn: So before the console does anything, you've built a small data warehouse.\n\nHerman: Pretty much. And the Azure part is a reminder that the console is only as good as the setup behind it.\n\nCorn: Stuck in validating. That sounds like being on hold.\n\nHerman: It does. OpenAI has a batches page in its dashboard too, but I didn't confirm what it shows, so I won't describe it. I also didn't read how OpenAI's webhooks get configured or signed, so I can't tell you how painful the setup is. Assume it's code. From the API side, the statuses run validating, in progress, finalizing, completed, expired, cancelling, cancelled and failed. As far as I know, the output file is deleted thirty days after completion.\n\nCorn: So the results come with a use-by date. Which one would you keep open all day?\n\nHerman: The Anthropic one, for the listing and the download. Its only real problem is the calendar. And every console has the same blind spot. It shows its own batches, for its own models. A Gemini job and an Anthropic job never appear on one screen.\n\nCorn: So if your classifier runs on two providers, you've got two dashboards and a spreadsheet to tie them together.\n\nHerman: Well, that's the batch API landscape. And now, I believe, we have a phone call.\n\nDorothy: Hello dear, it is your mother. Time for my grim fact.\n\nDorothy: If you run a batch with one of those Claude machines, your results are only kept for twenty nine days, and then they are gone for good.\n\nDorothy: Anyway, I made a casserole, and I still have that shirt you left here in the airing cupboard. When are you coming to see me, sweetheart? Bye bye now.\n\nCorn: Mum. We're on air.\n\nHerman: Anyway, the spreadsheets. Hugging Face has an open source tool called AI Sheets, announced last year. You define columns with prompts, bring in a CSV or Parquet file, and each column runs on a model you choose, through Hugging Face's inference providers or a local model. You export the result to the Hub with a reusable config.\n\nCorn: I'd guess there's a limit, and it isn't a big one.\n\nHerman: A thousand rows in the interface, at launch. I'd check that, because I haven't seen an update since.\n\nCorn: A thousand rows is a sample, not a dataset.\n\nHerman: It's a good sample, though. You tune the column prompt on it, read the outputs, fix the prompt, and only then go big. For bigger runs, the write-up has you take that exported config and run it as a job on Hugging Face Jobs. I like that, because the config is the bit you want to keep. The catch is that Jobs bills by the minute while it's starting or running, and what's documented is mostly command line and API, not a screen.\n\nCorn: So the design is a GUI and the scale is a terminal.\n\nHerman: Pretty much the trade. Good design, no babysitting, and then you're back at a terminal for the heavy lifting.\n\nCorn: You're happy about a spreadsheet, Herman.\n\nHerman: I have a life, Corn. I DJ on weekends.\n\nCorn: Google will have its own version of this.\n\nHerman: It does. Fill with Gemini, announced in April, is a front end on the AI function in Google Sheets. You drag it down a column, or fill a range from a prompt. It's the easiest thing on this list to use, and it only speaks Gemini.\n\nCorn: The easiest to use, and it only knows one name.\n\nHerman: Dify is closer to what Daniel described. You build a workflow, publish it as a web app, and the app has a Run Batch tab. You download a CSV template, each row is one run, you upload the file, and you export the results as a CSV. It's provider-agnostic, which is rare on this list. The prompt lives in the workflow, so the file only carries the inputs.\n\nCorn: So you build the job, then the job gets its own front door.\n\nHerman: And the front door is the whole point. Corn, would you trust a CSV to hold ten thousand labels?\n\nCorn: I'd trust it over the label printer.\n\nHerman: That's not a no, and I'll take it.\n\nCorn: Then the eval tools are the other pile.\n\nHerman: Kiln AI is an open source desktop app. It generates synthetic data with schema validation, using your own API keys or Ollama, across a long list of providers, OpenRouter and Bedrock among them, and it runs on your machine. For a classifier I'd look at it first, because it's built for making the labelled examples in the first place.\n\nCorn: Making the labels with a machine. What could go wrong.\n\nHerman: The schema validation catches some of it. Langfuse runs experiments from its interface: you start from a dataset, pick a prompt, map the variables, and optionally add an LLM judge. Each item runs synchronously, so it's not batch in the discount sense, but the shape is a bulk job. A dataset goes in and results come out.\n\nCorn: A batch that doesn't wait. That's nearly a rebellion.\n\nHerman: Argilla is the piece I'd add to any labelling run. The distilabel pipeline can push records into the Argilla interface while the run is still going, so a person reviews the labels as they land. For classifier data, the review step matters more than the generation step. A model that's confidently wrong is much cheaper to catch on day one than after it's been trained on.\n\nCorn: Nobody budgets for the review step.\n\nHerman: I'd budget for it. The review is where you find out the model has been labelling every short clip as the same thing, which happens more often than the demo suggests.\n\nCorn: Argilla sounds like a lot of clicking.\n\nHerman: It's clicking with a purpose, and the clicks become the labels you train on.\n\nCorn: And if you only want to test, not label?\n\nHerman: Promptfoo is the nearest thing. It's similar in spirit: a browser setup, test cases from CSV on the command line, results exported as CSV or JSON, and a share link. I haven't confirmed whether its interface takes a CSV upload directly, so check that before you plan around it.\n\nCorn: And then the drag-the-boxes pile.\n\nHerman: N eight N's batch route runs through a community template for Anthropic batches. It uses HTTP request steps, plus an if-and-wait loop: submit the batch, check its status, then fetch the results. Kestra has an Anthropic message batch plugin and an OpenAI batch blueprint.\n\nCorn: So the workflow tool is a polling loop with a nicer picture of it.\n\nHerman: Your polling loop has a flow chart now, Corn.\n\nCorn: Mine has a nap in it.\n\nHerman: Seriously, though. Somebody still has to maintain that loop, and it's the loop you were trying to get away from.\n\nCorn: Then the doorbell's the better deal.\n\nHerman: Last, there's a handful of open source libraries that wrap several providers' batch APIs behind one call. batchata, batchlane, Mozilla's any-llm, and a few more. None of them, as far as I know, ships a dashboard. Vercel's AI gateway advertises batch processing, up to a thousand requests per call. I'd treat that as a changelog line until somebody has run it.\n\nCorn: So the library translates, and nothing watches the jobs.\n\nHerman: The translator is useful. You still have to watch it run. Now let's get practical, because Daniel has two jobs and they're different animals. The speech work first. OpenRouter's batch rejects audio on every provider. So the cheap, tidy route won't take a single audio file, and that's the one thing to write down.\n\nCorn: The tidy route won't take the file. Typical.\n\nHerman: For audio you need a job runner. Replicate has per-prediction webhooks, with filters for start, output, logs and completed. The catch is that input and output files are deleted after an hour, so your handler has to save each result the moment it arrives. That changes the design. You're not polling for results, you're catching them.\n\nCorn: An hour. I'd still be deciding whether to get up.\n\nHerman: Hugging Face Jobs is the other route. It bills per minute while it's starting or running, you need a positive credit balance, and the default timeout is thirty minutes. For long recordings you raise the timeout up front, or the job gets cut off partway. Would you rather have the output in a Hub dataset, or in a handler you wrote yourself?\n\nCorn: The dataset. Fewer things of mine to keep running.\n\nHerman: Thirty minutes, though. That's a long recording.\n\nCorn: Thirty minutes. That's one of my naps.\n\nHerman: If you'd rather drop audio into a box, there are hosted ones like TurboScribe, and self-hosted ones. There's a whisper web app with a queue and a SQLite database, and a desktop app called faster-whisper GUI. Those are the interfaces that actually take audio, which the batch APIs won't.\n\nCorn: So the batch APIs won't let the audio in the door, and the GUIs are the only ones who will. Which would you trust with a week of recordings?\n\nHerman: The one where a failed clip shows up on a screen, not the one where it shows up as a missing row three weeks later. With audio, a silent failure is much worse than a slow job. You'd rather wait for the timeout than find a gap in the training set by accident.\n\nCorn: So the cheap route is the risky one.\n\nHerman: The cheap route is the one that refuses your files, so it's not really a route at all. For the classifier, the text side has the most options. OpenRouter batch at half the standard price, with the callback landing in your receiver, which then fetches the labels. A spreadsheet, if the set's under a thousand rows. Dify's Run Batch, if you want upload, run and download in a browser. And Argilla in the middle, so a person sees the labels before they become training data.\n\nCorn: Those are three different answers to one question. How many rows, and who checks them.\n\nHerman: The rerun is the point. A classifier you'll retrain every month needs the labels back in a form you can version, and the rerun has to match the first run. That's where a template beats a clever dashboard. If the job is defined in a file, you can run it again in six months and know what changed.\n\nCorn: The part nobody budgets for, again.\n\nHerman: Again, and I still say budget for it. So the real problem is the job record, not the job.\n\nCorn: Which means somebody has to keep the record.\n\nHilbert: Royal Dansk. The blue tin, nineteen dollars, out of petty cash. Every job gets an index card. Job ID in the top corner, in pencil, because ink runs when the kettle's on. The card goes in the tin before I press the button, not after. A job you can't hold in your hand is a rumour. I've had jobs go to the other side of the building and come back without a name on them, and that's a rumour with a stamp on it.\n\nCorn: Go on, then. A rumour is a strong word for a vendor's dashboard.\n\nHilbert: The tin has eleven hundred and four cards in it. The Eastside Model Railway Society has a Cards Subcommittee, Thursday nights, minutes and all. The long-running dispute is whether the cards should be laminated. It's been in the minutes for eleven years. Maurice Dorrigan holds the only key to the cabinet, and he's against laminate on principle. He had one card laminated in the first year, by accident, and he has not forgiven the card.\n\nHerman: On principle.\n\nHilbert: Laminate won't take pencil. That's the principle. Last month I sent nine thousand and forty voice notes to a transcription service, one card to each note. The service wanted a purchase order. The purchase order went to the treasurer, who is Maurice, and he put it on the agenda. It is item six. Item six was adjourned to item nine, which is the cards. The minutes go on the Society's website, which Maurice built on a machine that has never been connected to the internet. Gwen types them up on a Tuesday and posts them from her phone, one line at a time, because the phone has a cracked corner that eats the full stops.\n\nCorn: So the job's been sitting in validating for a month, with a committee.\n\nHilbert: Nine thousand and forty-one. One was in my coat. I don't press go on a Thursday, because the Society meets Thursdays, and a job started on a Thursday is a job the Society wants a say in. I don't open a job in the rain either. The pencil swells. Four of the last six Thursdays were wet, so most of my Thursdays are spoken for by the weather and the minutes. Last Thursday a neighbour's son saw the button and pressed it. The Society has asked for his name in the minutes, and for a copy of his school report.\n\nCorn: So the Society has a veto on your Thursdays.\n\nHilbert: They do. The service sent the output back as a zip, nine thousand and forty-one files, every one called output. I wrote to them about the names. They wrote back in capitals that the names were the names. I have never had an answer to that letter that wasn't in capitals, and the letter's in the tin, under the car park cards. The car park cards are from a different committee. Don't ask me about the car park.\n\nHerman: Hang on. Every file is called output?\n\nHilbert: Every one. The zip had a readme as well, and the readme said, in full, see output. And here's the part you lot haven't got. The file goes off after a month, and the card stays. The card is the only record in the building that outlives the job. Maurice wanted the file names written on the card as well. The card would not take them. So the card's a card.\n\nHerman: Thirty days on OpenAI, and the file goes first. So the record has to live somewhere else.\n\nHilbert: Four seconds to find a job in the tin. I timed it at the Society's AGM, with Maurice holding the stopwatch and not trusting it. Forty minutes to find one on a dashboard, and I was on the right tab, and the right tab had a second tab inside it. Maurice timed the dashboard on his own phone, on the same afternoon, and got forty-one minutes. He has never once said sorry for the extra minute, and I'm not having it.\n\nCorn: Forty minutes is a long time to stare at a tab.\n\nHerman: I'd have said the dashboard wins on search. You've just told me the tin wins.\n\nHilbert: Hm. Maurice will want that in the minutes.\n\nCorn: So what should Daniel actually use?\n\nHerman: For the classifier, AI Sheets. Load the labelled sample, write the column prompt, check a few hundred rows by eye, then export the config to the Hub. Run the full set as a Hugging Face job from that config, so the big run repeats the test you already trusted. For the speech work, skip OpenRouter, since it won't take audio. Use Replicate with a webhook, and have the handler save every output the moment it arrives.\n\nCorn: And whatever he picks, the record has to outlive the file. A file has a shelf life. The record needs a longer one.\n\nHerman: Thanks to Hilbert Flumingtop for producing this one.\n\nCorn: This has been My Weird Prompts. Find us at my weird prompts dot com.\n\nHerman: See you tomorrow.\n","showNotes":"Batch APIs are usually pitched as a discount: half price, come back tomorrow, job done. But the discount is the easy part. The hard part is the babysitting — several jobs in flight, and someone keeps asking what happened to that one. Callbacks have started to close some of that gap. OpenAI has webhook events for batch completed, failed, cancelled, and expired. Gemini has webhooks for its batch jobs. OpenRouter takes a callback address at creation. Bedrock pushes state changes into EventBridge. Anthropic's batch docs describe polling and nothing else, and its Console is per workspace, so a run in another workspace isn't on the screen you're looking at.\n\nA webhook isn't a dashboard, though — it's a doorbell, and somebody still has to answer it. OpenRouter's callback carries the batch ID and nothing else, so your receiver has to fetch the results itself. There's no vendor-agnostic batch dashboard that does the job, so what exists falls into four piles: vendor consoles, spreadsheets with an AI column, eval runners that behave like batch runners, and workflow tools where you join boxes until something runs. The consoles see the jobs; the sheets see the rows; nothing sees both. Upload, define, track, download are the four verbs that matter, and track is the one everyone complains about.\n\nPractical notes matter here. Anthropic's Console is the most complete native screen — it lists batches and downloads results, with 29 days of retention from creation. Vertex AI's Batch Inference page reads from Cloud Storage or BigQuery and writes back to a bucket or table. Azure's Foundry portal has batch jobs and a Global-Batch deployment type. On the spreadsheet side, Hugging Face's AI Sheets caps at a thousand rows in the interface, with bigger runs exported as a config for Hugging Face Jobs. Google's Fill with Gemini is the easiest to use and only speaks Gemini. Dify publishes a workflow as a web app with a Run Batch tab. Kiln AI, Langfuse, Argilla, and Promptfoo round out the eval pile, and N8N and Kestra handle the drag-the-boxes approach with polling loops. One thing to write down: OpenRouter's batch rejects audio on every provider, so the tidy cheap route won't take a single audio file.","promptSanitized":"Daniel has been running a lot of batch inference lately, for fine-tuning an ASR model and for a personal classifier. He means batch as a bulk job, not a discount: upload a file, define the task, track it, download the results. OpenAI, Gemini and OpenRouter batches already send completion webhooks, and Bedrock can notify through EventBridge, but all of that is set up through the API and Anthropic's docs show polling only, so the gap is a GUI that sits above the providers. What are the best ways right now to run bulk AI jobs through a GUI, ideally vendor-agnostic and able to pipe outputs into a GitHub repo or a dataset, and how do vendor-native consoles compare?","transcriptUrl":"https://episodes.myweirdprompts.com/transcripts/batch-job-monitoring-dashboards-24028.md","url":"https://myweirdprompts.com/episode/batch-job-monitoring-dashboards-24028/","_links":{"self":"https://myweirdprompts.com/api/episodes/batch-job-monitoring-dashboards-24028.json","html":"https://myweirdprompts.com/episode/batch-job-monitoring-dashboards-24028/","audio":"https://episodes.myweirdprompts.com/audio/batch-job-monitoring-dashboards-24028.m4a"}}