Understanding 5 Types of Web API Pagination

Pagination is especially important in a world where performance is everything, yet we’re producing and consuming more data than ever before. Pagination is an easy and surefire way to prevent data dumps, sluggish performance, and app breakage.

If you’re new to API pagination, we recommend reading our guide here. But first, let’s start with a quick definition.

According to sitechecker.pro, pagination is “an ordinal numbering of pages, which is usually located at the top or bottom of the site pages.” It’s best practice to implement pagination with APIs since some calls might yield billions of results, depending on the data you’re working with.

There are multiple ways to implement API pagination in your development projects. However, knowing which one will be best for your application is not always easy. So, we’ve put together a thorough guide to the four types of API pagination you’re most likely to encounter so you can decide which approach will be best for your project.

1. Cursor API Pagination

A cursor is a piece of data that points to an element with the info to retrieve the previous and next items. With cursor API pagination, the server will usually return the element to retrieve the next page. This element is usually hidden, also, so users can’t manipulate the results. Some APIs will send a cursor for each element so that they can link elements together.

Cursor API pagination can be implemented in a variety of ways. Some APIs return the cursor in the payload. Others will return it as a Header, especially Link Headers. Cursors sometimes contain data that can be used to filter other parameters.

If you’re going to use Cursor API pagination, you should consider adding an expiration date for the token and for cursors sent through API requests, as recommended by Google.

To illustrate, imagine that you’ve got a list of results in a JSON object.

[
  { "type": "examples", "id": "1" },
  { "type": "examples", "id": "5" },
  { "type": "examples", "id": "7" },
  { "type": "examples", "id": "8" },
  { "type": "examples", "id": "9" }
]

Imagine a cursor labeled abcde falls on "id": "5". Now imagine a second cursor, xxx, lands on "id" "9". An example of a query using Cursor API pagination might look like:

GET /example-data?page[after]=abcde&page[size]=2

The returned results might look like:

{
  "links": {
    "prev": "/example-data?page[before]=yyy&page[size]=2",
    "next": "/example-data?page[after]=zzz&page[size]=2"
  },
  "data": [
    // the pagination item metadata is optional below.
    { "type": "examples", "id": "7", "meta": { "page": { "cursor": "yyy" } } },
    { "type": "examples", "id": "8", "meta": { "page": { "cursor": "zzz" } }  }
  ]
}

On the other hand, if the query looked like this:

GET /example-data?page[before]=xxx&page[size]=3

The response might be:

{
  "links": {
    "prev": "/example-data?page[before]=abcde&page[size]=3",
    "next": "/example-data?page[after]=zzz&page[size]=3"
  },
  "data": [
    // again, optional pagination item metadata is allowed for each item here.
    { "type": "examples", "id": "5" },
    { "type": "examples", "id": "7" },
    { "type": "examples", "id": "8" }
  ]
}

Advantages of Cursor API Pagination

Doesn’t need to be updated following an API update if the cursor is opaque
It’s fast
No problems if a record is updated

Disadvantages of Cursor API Pagination

No skipping pages
No parallel requests
Semi-complex implementation
Difficult to debug
Items can be missed if they’re on previous pages

2. Keyset API Pagination

Keyset-based API pagination is when an API provides a key parameter that delineates the query results. One example is if an API is sorted by ID, one key parameter could be since_id. Some other possibilities could include since_updated_at or since_created_at.

These examples would only work for datasets sorted by ID, though. It also requires the since_ID parameter to be sent as part of the query, which is often the value of the last ID element in the request.

This is relatively easy to implement in code, though. An example of Keyset-based API pagination in an SQL query could look like:

Select *
FROM products
WHERE ID > <since_id_param>
ORDER BY ID ASC
LIMIT 100

Advantages of Keyset-Based API Pagination:

Efficient, especially when using a Where command.
Limits redundancies, as items from previous pages won’t be replicated on the current page.

Disadvantages of Keyset-Based API Pagination

Depends upon rigid IDs
Difficult navigation
Unable to send parallel requests
API exposes multiple key parameters
User needs to know the key-value
Possible missing items if they’re outside the query’s scope.

3. Offset API Pagination

Offset API pagination is the most common form of API pagination, particularly if you’re using a SQL database. It’s also sometimes called page-based pagination. An API endpoint accepts a parameter for a page number and then returns that page.

Commonly, queries will use LIMIT and OFFSET. LIMIT sets page length, and then OFFSET is the number of records already returned.

Here’s an example of what Offset API pagination might look like in code:

Select *
FROM products
ORDER BY Id
LIMIT 20
OFFSET 200;

Advantages of Offset API Pagination

Easy navigation
Supports parallel requests
Stateless on server-side
Easily understood and debugged
Platform agnostic

Disadvantages of Offset API Pagination

Poor performance for large offset

4. Seek API Pagination

Seek API pagination is another version of keyset API pagination. Using atter_ID and start_ID removes the tight coupling between filters and pages. It also reduces some of the issues with low cardinality field like state enums or category enums.

The main downside of seek API pagination is that it’s hard to implement custom sort orders.

An example of seek API pagination in code:

SELECT
    *
FROM
    Items
WHERE
  Id > 20
LIMIT 20

This example is perfectly adequate if you’re sorting by Id. What do you do if you want to sort by email address, though? In that case, the email address matches the after_id. If they match, a WHERE query is performed.

Advantages of Seek API Pagination

Uncoupled pagination from filter
Consistent ordering
Stable performance

Disadvantages of Seek API Pagination

Complex implementation on the backend
Deleted items can break the start_id

5. Time-Based API Pagination

You can also configure your API to return pages based on when they were created. Time-based API pagination uses a CREATED command to return items created in a certain order. This can be useful if you want a list of all items created before a certain item, for example.

An example of time-based API pagination might look like:

curl https://api.mywebsite.com/items?created_before_timestamp=1505086265160&limit=50

Time-based API pagination removes the possibility of items being skipped or overlooked. Items aren’t tightly coupled to a certain order, either, which reduces the chance of breakage.

Advantages of Time-Based API Pagination

Uncoupled from itemized indexing
Entries won’t be skipped or overlooked
Can return a specific date range

Disadvantages of Time-Based API Pagination

Not all items have a time-stamp

Final Thoughts on API Pagination

Considering how much data you regularly work with when using APIs, it’s vital to at least think about API pagination as one potential way to control how your results are returned. Accidentally invoking 500 pages of returned results can have disastrous consequences, especially if you’re paying for data usage.

As developers, it’s important to keep these considerations in mind as well. Having query results returned with no restrictions runs the risk of making your API sluggish as a best-case scenario. API pagination is a simple, elegant way to make sure your customers and API consumers get the best possible service and results when using your API.