Introduction

In Denodo we have four different pagination methods for configuring DF, JSON and XML data sources to retrieve data from paginated REST APIs. In this document we provide an overview of these methods using examples coming from different popular APIs.

The pagination methods available for data source configured with the  HTTP Path Data Route :

• Next page URL from a HTTP header on the response
• Next page URL on the response
• Token continuation
• Paging indices

We will use examples of these methods with REST APIs that return JSON responses. These APIs are: Youtube API, Google Search API, Shopify API, Spotify API and Denodo Scheduler API.

Pagination methods

Next page URL from a HTTP header on the response

This pagination method is used to connect to REST services that include the URL of the “next page” in an HTTP header of the response. We can see this type of pagination in the Shopify API.

If we see the response header of this API we can see that there is a header called link that has the URL to the next page. You can see how to configure this API and pagination type in the KB document How to integrate the Shopify API in Denodo.

Next page URL on the response

This pagination method is used to connect to REST services that include the URL to the “next page” in the response. We can see this type of pagination in the Spotify API.

If we take a look at the JSON response of the Spotify API, we can see that the URL of the next page is in the field ”next” and the path to that field is tracks/next. You can see how to configure it in Denodo by reading the KB document How to integrate the Spotify API in Denodo.

Token continuation

This pagination method is used to connect to REST services that include a token in the response that has to be added to the URL to obtain the “next page”. We can see this pagination method with the Youtube API.

Seeing the following image, the parameter in the URL for the next token is pageToken and the path to the next token in the JSON response is nextPageToken.The parameter in the URL for the next token is a parameter that will be added to the URL with the token of the next page. For more details check the KB document How to integrate the Youtube API in Denodo.

https://www.googleapis.com/youtube/v3/search?&q=@query&type=@type&key=@apikey&part=snippet&nextpageToken

Paging indices

This pagination method is used to connect to REST services that support:

• Indicating the page number as a parameter on the URL.
• Indicating the range of rows as parameters on the URL (for example, obtain the records 50 to 100).

In this pagination method we need to introduce two parameters: one parameter for the page size and another for the index of the first record. We also have to specify the parameter offset which indicates the offset for the next requests.

We can see how to configure this pagination method by seeing the KB documents: Creating a dashboard on Denodo Scheduler and How to integrate the Google Search API in Denodo.

References

Creating a dashboard on Denodo Scheduler

How to integrate the Youtube API in Denodo

How to integrate the Shopify API in Denodo.

How to integrate the Google Search API in Denodo.

How to integrate the Spotify API in Denodo