Pagination

The envelope#

A paged list answers:

{
  "data": [
    "…"
  ],
  "pagination": {
    "has_more": true,
    "next_cursor": "eyJzIjoic3Vic2lkaWVzOnVwZGF0ZWQiLCJrIjoi…",
    "limit": 50
  },
  "meta": {
    "request_id": "req_…",
    "warnings": []
  }
}

• has_more is a fact, not a guess: the API reads one row more than you asked for. There is no total count.
• next_cursor is set only when has_more is true. Pass it back as cursor, with the same parameters, for the next page. The change feed is the exception: it returns a cursor on every page, the last included, so you can poll on from there (see following the feed).
• limit is the page size that was used.

$ curl "https://api.subsido.be/v1/subsidies?status=open&sort=updated&limit=100&cursor=eyJzIjoic3Vic2lkaWVzOnVwZGF0ZWQiLCJrIjoi…" \  -H "Authorization: Bearer sb_live_..."

Page sizes#

limit defaults to 50. Above your plan’s maximum it is an error, 400 invalid_parameter, never a silent truncation: quietly serving 20 rows to someone who asked for 100 is how a client comes to believe it has read a whole list when it has read its first page.

The same figures are in max_page_size of GET /v1/plans and GET /v1/key. A key whose account has no plan gets 402 subscription_required on every list, whatever the limit.

Cursors#

• A cursor carries the last row’s sort key and id. Treat it as opaque: its format can change.
• A cursor belongs to the listing and sort order it came from. Replayed against another (a different sort, another route) it is 400 invalid_cursor, and you start again without one; the API never silently restarts at page one.
• Keep the filters the same while you page. The cursor does not remember them, so changing them mid-way pages through a different list from where the old one stopped.
• Page one request at a time. Parallel page fetches do not make a cursor walk faster and do spend your per-second rate limit.

Sort orders#

Which lists page#

Walking a whole list#

import os, requests

def all_open_measures():
    params = {"status": "open,continuous,forthcoming", "sort": "updated", "limit": 100}
    headers = {"Authorization": f"Bearer {os.environ['SUBSIDY_API_KEY']}"}
    while True:
        r = requests.get("https://api.subsido.be/v1/subsidies", params=params, headers=headers, timeout=30)
        r.raise_for_status()
        page = r.json()
        yield from page["data"]
        if not page["pagination"]["has_more"]:
            return
        params["cursor"] = page["pagination"]["next_cursor"]