Why a failed scrape should cost you nothing

Every data API bills one of two ways. Per request: you pay when you ask. Per result: you pay for what comes back. On a pricing page the difference looks like a detail. On an invoice it is often the largest single factor, and it gets larger the harder your targets are.

Say you are pulling profiles for ten thousand handles from a list somebody exported months ago. In practice a chunk of that list is dead: accounts deleted, gone private, renamed, or region-locked where you are asking from. Call it seventy per cent that resolve — which is a normal fill rate for a real list and an optimistic one for an old one. Priced per request at $1.00 per thousand, you pay $10.00. You get seven thousand profiles, so your true cost is $1.43 per thousand profiles. Priced per result at $1.00 per thousand, you pay $7.00, and your true cost is exactly $1.00 per thousand — because it cannot be anything else. The advertised rate was identical. The real one differs by 43%, and the gap is entirely your fill rate.

Fill rate is not a constant. It drops when you go after harder targets, when a platform tightens something, when your list ages, and when you expand into a region where more accounts are restricted. Under per-request billing, every one of those events raises your cost per useful row while your usage graph looks unchanged. That is the uncomfortable property: the cost increase is invisible in the place you would look for it. Your request count is flat, your bill is flat, and your cost per delivered row has quietly gone up by a third.

It is not always the worse deal. On a paged endpoint with a high fill rate — comments on posts you already know are busy, followers of large accounts — one request can return hundreds of rows, and per-request billing makes that page nearly free. If your workload is overwhelmingly of that shape, and your targets are hand-picked rather than exported from something, per-request can come out ahead. The question is not which model is better in the abstract but which one matches the distribution of what you are actually asking for.

Not always obvious from the pricing page, because the words are used loosely. "Pay per success" usually means per request that returned a 200, which is not the same as per row delivered — a 200 with an empty array is a success by that definition and costs you full price. The question that settles it: if I ask for a hundred and forty come back, what do I pay? If the answer is "a hundred", you are on per-request billing whatever it is called.

We bill per result delivered. A run that returns forty of the hundred rows you asked for costs forty rows. A run that returns nothing costs nothing, and that is not a goodwill gesture — it falls out of the unit being the row rather than the attempt. There is a rate card behind that with two bands, because a run that produces one object costs us about as much to carry out as one that produces four hundred rows, and pretending otherwise would mean subsidising one kind of caller with another.