RESTful API Pagination can seem daunting to many developers. Perhaps it’s because the name is evocative of something much more complex than it actually is. But once you understand how pagination works and how to implement it, it can grant wonderful utility to almost any API.

Below, we’ll look at some concrete examples of REST API pagination in production. We’ll briefly cover what pagination actually is, and look at how each of these APIs handles pagination in their unique way.

What is Pagination in RESTful API?

Pagination, in simple terms, is breaking a complex output into simplistic sets to better serve the user, conserve resources, and aid in navigation. If you’ve ever searched for a term and seen 100 or more pages as results, you’ve witnessed an API pagination example first-hand. When a query results in many results, pagination can break these results into chunks that are easier to navigate and parse.

So, how does an API typically go about paginating results? Well, there are a few common approaches to RESTful API pagination: offset pagination, basic keyset pagination, and seek pagination. Let’s briefly review these pagination types before digging into real-world examples.

Offset Pagination

Offset pagination is the simplest to implement. In this sort of pagination, results are delivered as discrete, limited chunks, with a specified offset amount noted to provide pagination sizes for navigation beyond the first paginated result. An API request using limit and offset would look like this:

GET /items?limit=20&offset=100

The result would show 20 items at a time with an offset of 100. This is easy to implement and requires little coding. Unfortunately, it also carries with it significant downsides. The most obvious is that, by using an offset, the entire data set is navigated to generate the paginated result. This means that for extremely large data sets, the entire result would be scanned and then broken into discrete chunks. The only efficiency boost would be in the user’s view, and the resource cost to the application would still be incurred.

Basic Keyset Pagination

Keyset pagination is a slightly more complicated form of pagination but is still relatively common. In this approach, the filter values of the previous page are used to choose the next set of items that will be paginated. In essence, it depends on the existing pagination steps to determine what the actual pagination results should be.

Such a request may look like this:

GET /items?limit=20

This simple query would result in a filter for the next page that would state when the initial query was made, creating a divider based upon a unique variable — in many cases, the session ID or the UTC of the initial request.

Keyset pagination is obviously far more efficient than the offset approach in that only a small subset is ever pulled from the results, and additional results are contingent upon the former pagination. A significant benefit to this approach is that the results are set in order, which means that, unlike the sorting solution, additional input or data changes do not necessarily affect the paginated results.

Advanced Keyset Pagination: AKA Seek Pagination

Seek pagination is a more complex form of pagination, and, accordingly, takes additional forethought to effectively implement. In this approach, developers can automatically create after_id and other filters to delineate results based upon an incremental value of previous responses. For example, review the following initial request:

GET /items?limit=20

This request behaves similarly to our existing outputs. But what about the next page? A pagination link can be created to automatically increment based upon the limit amount as follows:

GET /items?limit=20&after_id=20

As long as the increment value is appropriately set and the internal logic is consistent, further pagination should alter the id by the stated limit. For example:

GET /items?limit=20&after_id=20
GET /items?limit=20&after_id=40
GET /items?limit=20&after_id=60
GET /items?limit=20&after_id=80

Four Examples of API Pagination Implementations

With this brief crash course out of the way, let’s take a look at some API pagination examples in the wild.

1. SmartBear QAComplete API

The SmartBear QAComplete API utilizes offset pagination to aid in pagination through a list of defects. According to the documentation, GET operations return the first 25 results, but additional results are possible through the use of the limit and offset parameters.

The following GET request will return the first 30 defects:

GET /defects?limit=30

What if we wanted to grab them in batches of 30? This is easily achieved by simply adding an additional variable in the form of an offset. The offset will offset the results by the specified number, beginning a new batch at the offset number as if it were the first result in the original ordered list.

GET /defects?limit=30&offset=30

In this case, 30 defects will be shown starting at the 31st result. And additional pages can easily be generated by simply incrementing the offset. Typically, this would lead to some issues with ordered lists, but the QAComplete API gets around this by serving the metadata for the complete set to give users informed context for their pagination:

{
   "metadata": {
      "result_set": {
         "count": 25,
         "offset": 0,
         "limit": 25,
         "total": 77
      },

By providing this data through the metadata result, applications can appropriately paginate based upon the entire data set instead of just the last paginated result.

2. Twitter API

There are few websites with as much user-generated content as Twitter, so the need to paginate this output is incredibly important. Twitter has implemented a keyset pagination approach utilizing a generated token. Let’s take a look at an example from the Twitter developer documentation on pagination.

In this scenario, a request is made to the Twitter API to deliver tweets from a specific user within a certain time range. The user ID is “2244994945”. First, we make the actual call to the API:

https://api.twitter.com/2/users/2244994945/tweets?tweet.fields=created_at&max_results=100&start_time=2019-01-01T17:00:00Z&end_time=2020-12-12T01:00:00Z

As a result, we receive the following output:

{
  "data": [
    {
      "created_at": "2020-12-11T20:44:52.000Z",
      "id": "1337498609819021312",
      "text": "Thanks to everyone who tuned in today..."
    },
    .
    .
    .
   {
      "created_at": "2020-05-06T17:24:31.000Z",
      "id": "1258085245091368960",
      "text": "It's now easier to understand Tweet impact..."
    }
  ],
  "meta": {
    "oldest_id": "1258085245091368960",
    "newest_id": "1337498609819021312",
    "result_count": 100,
    "next_token": "7140w"
  }
}

What is important to note are the contents of the meta section of that response. In the output, we have a value called next_token — this response forms the keyset by which all responses are paginated. To get the next page of contents, we can adjust the request slightly to pass the next_token value:

https://api.twitter.com/2/users/2244994945/tweets?tweet.fields=created_at&max_results=100&start_time=2019-01-01T17:00:00Z&end_time=2020-12-12T01:00:00Z&pagination_token=7140w

The output of this request will be as follows:

{
  "data": [
    {
      "created_at": "2020-04-29T17:01:44.000Z",
      "id": "1255542797765013504",
      "text": "Our developer community is full of inspiring ideas..."
    },
    .
    .
    .
    {
      "created_at": "2019-11-21T16:17:23.000Z",
      "id": "1197549579035496449",
      "text": "Soon, we'll be releasing tools in..."
    }
  ],
  "meta": {
    "oldest_id": "1197549579035496449",
    "newest_id": "1255542797765013504",
    "result_count": 100,
    "next_token": "7140k9",
    "previous_token": "77qp8"
  }
}

Note that, in addition to the new batch of tweets, we also have a new token — 77qp8 — which denotes the next page of our results. This approach avoids the issue of unordered addition that is common to keyset pagination. It does so by basing the pagination token not on an externality such as date or time, but instead on an auto-generated token separated from the internal logic and data set.

3. Spotify API

The Spotify API utilizes a simple offset and limit parameter set to allow pagination. This is accomplished using a zero-based default pagination approach, wherein the omission of the offset parameter simply returns the first of a total set as dictated by the limit parameter.

An example endpoint, as provided in the documentation, shows how this works:

https://api.spotify.com/v1/artists/1vCWHaC5f2uS3yhpwWbIA6/albums?album_type=SINGLE&offset=20&limit=10

This request would simply result in a list of 50 singles from a specific artist. The offset, in this case, 20, would start the result on the twentieth single and would then limit the additional results in pagination to the next 10 in the set. With each new pagination step, this offset would be incremented to deliver the next batch.

It should be noted that Spotify is an excellent example of a hybrid model being appropriate. Spotify approaches their paginating with a multi-solution implementation depending on what data is being returned. When the item endpoint being polled would return an array of items as opposed to a simple ordered list, Spotify defaults to using a paging object to deliver pagination rather than this offset method.

4. Twitch API

The Twitch API uses cursor-based pagination, a subtype of keyset pagination. In this approach, the Twitch API uses three parameters to control paging through results. The “after” parameter is used to move to the next page of the result, the “before” parameter for the previous, and the “first” parameter to delineate how many items should be included per page. The Twitch API itself responds with the cursor in question upon search. If there are additional results to be had, the output will include a “cursor” field with a specific object:

   "pagination": {
      "cursor": "iJma9"
    }

From here, you can use the after or before parameters to specify the direction of movement relative to the cursor. For example, the following query will display the 40 entities before the cursor.

https://api.twitch.tv/helix/streams?first=40&before=iJma9' \

Likewise, after can be used to specify the 40 entities after the specific cursor as follows:

https://api.twitch.tv/helix/streams?first=40&after=iJma9' \

5. Salesforce GraphQL API

The Salesforce Platform APIs continually rank as some of the most popular APIs, year by year. And Salesforce now provides a GraphQL API that allows developers to query records that correspond to objects like Accounts or Contacts. By default, the API responds with the first 10 results. But, Salesforce provides several other ways to work on this data.

For example, The RecordQuery type contains fields that enable more fine-grained pagination capabilities. first allows you to specify precisely how many results to return, and after enables you to return results after the given cursor. upperBound could also be handy for pagination, as it allows you to specify a limit on the number of records you wish the request to return.

Here is the RecordQuery Specification, as found in the Query Objects documentation:

type RecordQuery {
  Account( # request the "Account" field to query "Account" objects
    after: String
    first: Int
    orderBy: Account_OrderBy
    scope: ACCOUNT_SCOPE
    upperBound: Int
    where: Account_Filter
  ): AccountConnection
}

Conclusion

Ultimately, pagination sounds more daunting than it actually is. And as we’ve seen, many APIs in production demonstrate how effective pagination can be in filtering data. Not only does proper pagination deliver a heightened developer experience, but it also leads to a more understandable, digestible, and usable API in the long run.

What do you think are some of the best examples of pagination? Let us know in the comments below!