You can translate the question and the replies:

how to do pagination to retrieve json API datasource second page elements

JSON contains 15000 elements but the baseview returning results as 2 pages and each page contains 10000 elements...it displaying only 10000 elements only after flatten the baseview and not showing the second page 5000 elements in the results. How to do the pagination for this requirement in the datasource?
user
02-02-2024 04:49:25 -0500
code

1 Answer

Hi, my understanding of your issue is that you have a JSON data source with pagination and you reach to the point where you have a flattened view where you can only see the results of the first page, but not the second. In trying to replicate this behavior, I use a REST service that delivers a total of 1400 rows with pagination settings of 1000 rows per page, therefore with two pages of results. Following the information on the [JSON Sources of the Virtual DataPort Administration guide](https://community.denodo.com/docs/html/browse/6.0/vdp/administration/creating_views/importing_data_sources_and_creating_base_views/json_sources/), I create a JSON data source and leave pagination configuration disabled. When creating the base view, I keep "use pagination options of the data source". And I also keep "JSON Root" selected as the Tuple Root value. This base view is not flattened, but allows me to see how many elements I get. In this form, if I execute the view and select "Retrieve all rows", I get back 1 row with a column that contains an array with 1000 elements (which is what the Web service is delivering as the first page). This means, since I am not configuring any pagination in data source and base view, I only get the first page that the webservice is delivering. Now I take one step further and configure the JSON data source in Denodo with pagination, (page size 1000, $count as page size parameter, $start_index as page number parameter, 1000 as offset for next requests, and 2 as max number of requests). I create the base view with "use pagination options from the source", and again keep "JSON Root" selected as Tuple Root value. Executing the base view gives me two rows, one with an array of 1000 elements, and another one with the rest 400. This means, once I get the pagination right in the data source, I get all the data, Denodo is handling the necessary requests to the web service. I take another step where I flatten this content so we see 1400 rows and not two with aggregates. To do that, I create a base view based on the paginated data source, and on the JSON Root, I reference a path for the Tuple Root that only picks certain elements of the aggregate. In executing this one with the "retrieve all rows" option, I now get the desired 1400, because as previously, the pagination is handled internally by Denodo to retrieve all the data. In summary, if the view does not show all the results, something in the data source pagination configuration might need adjustment. Furthermore, if you have a valid support user and you need more help, I would recommend to contact the Denodo Support Team through [this link](https://support.denodo.com/cases/add) as they will be better able to assist you.
Denodo Team
06-02-2024 11:14:21 -0500
code
You must sign in to add an answer. If you do not have an account, you can register here