for await...of: Paginated APIs as a Simple Loop

Every paginated API integration grows the same scaffolding: a while loop, a cursor variable, a hasMore flag, and results.push(...page) until memory or patience runs out. Async generators move all of that behind a function boundary, permanently.

async function* allOrders(baseUrl) {
    let cursor = null;
    do {
        const url = cursor ? `${baseUrl}?cursor=${cursor}` : baseUrl;
        const page = await (await fetch(url)).json();

        yield* page.items;          // hand out items one at a time
        cursor = page.nextCursor;   // remember where we are between yields
    } while (cursor);
}

Consumption looks like the API was never paginated at all:

for await (const order of allOrders('/api/orders')) {
    process(order);
    if (order.createdAt < cutoffDate) break;   // stop early - no more fetches happen
}

Why this beats the accumulate-everything loop

Laziness. Pages are fetched only when the loop needs more items. That break statement doesn't just exit - it means page 3 through 200 are never requested. Searching for one item in a big collection stops network traffic the moment it's found.

Constant memory. Nothing accumulates unless the caller chooses to. Processing 100k records streams them; the old pattern held them all. (Backend readers will recognize this as the same shape as PHP generators for big CSVs - identical idea, different runtime.)

The bookkeeping is written once. Cursor handling, page-size quirks, retry logic - it all lives inside the generator. Five call sites, zero duplicated pagination code.

Composability: generators wrap generators

async function* take(iter, n) {
    let i = 0;
    for await (const item of iter) {
        if (i++ >= n) return;
        yield item;
    }
}

async function* filter(iter, fn) {
    for await (const item of iter) if (fn(item)) yield item;
}

// First 50 refunds, fetched as lazily as possible:
for await (const o of take(filter(allOrders(url), o => o.status === 'refunded'), 50)) {
    report(o);
}

The caveat worth knowing

for await processes items sequentially - it won't parallelize your handling. That's usually right (ordered processing, API rate limits are respected naturally), but if each item needs an expensive independent operation, batch them: collect chunks of 10 from the generator, then Promise.allSettled the chunk (combinator guide here). Generators for fetching, combinators for fanning out - each tool where it's strong.

Support: ES2018, so everywhere that matters, including Node since forever. The next time you see a while(hasMore) loop, you know what it wants to become.