Competitive intel and content ops teams often want a morning digest of competitor news and new listings in one place. The sources differ. Some paginate, and some require a login. The collection has to run the same way every day.
Claude summarizes well, once the content is in front of it. Fetching that content across sources is harder than it looks.
- Claude reads a single page and makes a sharp judgment call, but aggregating news and listings across dozens of sources is a repeated navigation problem, not a reading problem.
- Every fresh Claude or Computer Use run re-reasons the whole task from scratch, and that variance compounds once you're pulling from dozens of sources daily.
- Logins, pagination, and session state across many listing and news sources are exactly what a chat-based model cannot durably hold between requests.
- The fix is to compile the repeatable navigation once into a reliable agent and reserve the model for the parts that need judgment, like what counts as newsworthy or how to categorize a listing.
- Claude can stay in the loop as the orchestrator and summarizer while a compiled Airtop agent does the fetching, paging, and structuring.
Where Claude is helpful for aggregating news and listings
What Claude is genuinely good at
Give Claude a news article or a listing page and it does excellent work. It can tell you whether a headline matters to your market, whether a job posting matches a client's hiring pattern, or whether a property listing fits a buyer's criteria. That's judgment, and it's exactly what a reasoning model is built for.
Claude is also good at drafting the output: turning ten disparate items into a clean digest, grouping listings by category, or writing a summary paragraph that reads like a person wrote it. None of that changes when you move to aggregate news and listings work. The judgment layer holds up fine.
What aggregation requires
Aggregating news and listings is a different kind of task underneath the judgment. It requires visiting the same set of sources on a schedule, navigating past logins, clicking through pagination or infinite scroll, and returning the same shape of output every time. That's a repetition problem, and repetition is where a chat model's strengths stop mattering.
The job isn't "read this page well once." It's "read forty pages the same way, every morning, without babysitting it."
Why aggregating news and listings breaks
Re-reasoning every run introduces variance
Claude and Computer Use don't remember how they solved yesterday's version of the task. Each session reasons through the navigation again: where to click, what counts as the listing container, how to page forward. For a one-off task that flexibility is valuable. For a daily aggregation run, it's a liability, because even state-of-the-art Computer Use models produce unexpected or incorrect behavior 20 to 60 percent of the time depending on complexity. Airtop's own analysis of this problem points out that pulling the top items from a source like Product Hunt every morning is exactly the kind of repetitive task where full re-reasoning on each run wastes effort a compiled agent wouldn't spend twice. That's why extra agency helps the first run more than the hundredth, which is a big part of why teams eventually reason once, at build time, then run the compiled result — and daily news and listings aggregation is a hundredth-time task from day one.
Logins, pagination, and rate limits Claude can't hold state through
Most news and listings sources worth aggregating sit behind something: a login for a paywalled trade publication, an SSO wall for a vendor portal, infinite scroll on a listings marketplace, or a rate limit that kicks in after a burst of requests. A chat session has no durable place to keep a signed-in state between runs, so it either re-authenticates every time, which invites CAPTCHA and anti-bot friction, or it silently fails partway through and hands you a partial result that looks complete.
Much of this content is also rendered dynamically. When a listing only lives in the page itself, especially on JavaScript-heavy sites that load content as you scroll, there's no clean API shortcut. You need something that behaves like a browser, not a text fetcher.
Cost and speed compound across many sources run daily
Run this across twenty or thirty sources every day and small failure rates stop being small. If you do a task 1,000 times per day, even a small probability of error compounds, leading to a large number of attempts ending in failure. At aggregation scale, a 10% miss rate per source turns into a digest that's routinely missing whole categories, with no obvious signal that anything went wrong. It also compounds on the reasoning side: code-first agents need fewer reasoning steps than JSON-based tool calling, which matters once you multiply a single task across dozens of sources every day.
Can't we just use Computer Use?
The first demo of Computer Use handling a listings site or a news source often works, and it's easy to see why teams assume this is solved. The model clicks around, finds the content, and returns something usable. That first success is real, and it's worth taking seriously as a proof of concept.
The hidden cost shows up on the second and hundredth run. Computer Use loops are slow and expensive relative to a compiled path, and they carry the same 20 to 60 percent error range on complex flows. For a single news source checked once, that's tolerable. For an aggregation job spanning dozens of sources on a schedule, the errors stack, the run time stretches, and the token cost per run adds up in a way that's hard to justify against the value of a daily digest.
What works for aggregating news and listings
The practical split is to compile the repeatable navigation once and keep the model for judgment. Airtop's approach is to reason once, at build time, then run the compiled result: you describe the aggregation you want in plain language, Airtop turns the login flow, pagination, and extraction into reusable code, and the model only gets called for genuinely variable steps, like deciding whether an article is relevant or how to classify a listing.
For scale, Agent Builder lets you point an agent at thousands of URLs or rows and it works through them in one pass, on a schedule or a webhook trigger, across up to 100 simultaneous sessions, with credentials held in a vault so logins aren't a manual step each morning. That handles the part Claude alone can't: automate any website, including ones behind a login, the same way, every day.
Claude doesn't have to step out of the picture. You can orchestrate the aggregation from Claude Code and trigger the agent directly from your terminal, letting Claude plan the digest structure and write the summary while Airtop does the fetching. This pattern shows up in adjacent use cases too, where agents that monitor sources and deliver signals instead of a dashboard to check replace manual review entirely. If you'd rather skip the build step, start from a template instead of from scratch.
Aggregation use cases that deterministic agents can solve
Aggregation is many sources, one schema, on a clock. Claude is the editor. The fetch has to be compiled or the digest will not match yesterday's.
Combine several listing sources into one morning table
Each source paginates differently, and some sit behind logins. Claude can dedupe titles. It cannot hold four sessions and walk four grids unless that work is compiled.
Build a news digest from sites that are not RSS
Plenty of useful pages have no feed. A chat can summarize one URL you paste. A compiled agent can load the section, pull the headlines, and hand Claude a stable list.
Watch classifieds or marketplace listings for new matches
The query is fixed. The results are not. Re-teaching Computer Use the filters every day is the slow path. Compile the search. Let Claude rank what showed up.
Keep a research folder filling itself
Analysts want new source material without re-clicking ten sites. Airtop runs the sites. Claude writes the note when the folder gained something worth reading.
Aggregation is a build-once problem
Aggregating news and listings isn't a reasoning failure. It's a repetition problem wearing a reasoning costume. Ask Claude for the judgment calls: what's newsworthy, how to categorize a listing, how to phrase the summary. Let a compiled agent own the login, the pagination, and the daily fetch, so the shape of the output doesn't depend on how well a chat session re-guesses the navigation that morning. See how a compiled agent runs, then spin up your first agent in five minutes.
Try aggregation that doesn't break on day two
Stop re-running the same fragile prompt against a new page layout every morning. Build the login, pagination, and extraction once, keep Claude for the judgment calls, and let a compiled agent handle the rest. Spin up your first agent in five minutes and try it for free.
FAQs
Can Claude read RSS feeds or scrape listing sites directly?
Claude can read text you paste in or fetch from a simple API, but most listings sites and many news sources render content dynamically or sit behind pagination and logins that a chat model can't reliably click through on its own. That's when a listing only lives in the page itself, and it's the point where you need something that drives a browser instead of fetching raw text.
Why does my Claude-based news digest work once and then break the next day?
The first run usually works because the model reasons through the navigation fresh and gets lucky. The next day, a slightly different page layout, a login prompt, or a rate limit changes the path, and because even state-of-the-art Computer Use models produce unexpected behavior 20 to 60 percent of the time, that re-reasoning doesn't produce the same result twice. A compiled agent avoids this because the navigation is fixed at build time.
Can Airtop agents log into paywalled news sites or listing portals?
Yes. Airtop agents can automate any website, including ones behind a login, with credentials stored in a secure vault so authentication happens automatically on every scheduled run instead of requiring manual sign-in each morning.
How is this different from just using Computer Use with Claude?
Computer Use re-reasons the entire navigation on every run, which is fine once but produces incorrect behavior 20 to 60 percent of the time depending on complexity on repeat runs. Airtop instead lets you reason once, at build time, then run the compiled result, so the same navigation runs the same way every time, with the model reserved for actual judgment calls.
Can I trigger an aggregation agent from Claude Code?
Yes. You can orchestrate the aggregation from Claude Code and trigger the agent directly from your terminal, letting Claude Code kick off the browser-based agent and receive structured results back in the same conversation.
Can one agent handle dozens of sources, or do I need to build one per site?
A single Agent Builder workflow can point an agent at thousands of URLs or rows and it works through them in one pass, running on a schedule across many simultaneous sessions, so you're not maintaining a separate script per source.
What should Claude still be responsible for in this workflow?
Keep Claude for judgment: deciding what's newsworthy, how to categorize a listing, and how to phrase the final summary. Those are exactly the variable decisions that don't compile down to fixed code, unlike the repeated login-and-navigate steps a compiled agent handles.
