You can translate the question and the replies:

Joins with Paginated Data

If we have a schenario where results from two paginated APIs are to be JOINED. e.g. We have 1 million customers, and an API that returns 100 at a time. And another API that reutrn All Customer Orders, again 100 at a time, with no guarantee about what pages(s) a customer's record would be present in. If we apply a join on these two views, will Denodo automatically fetch all "Pages" from both APIs to satisfy join conditions and if so is there a performance implication?
user
28-04-2021 13:05:21 -0400

1 Answer

Hi, Denodo includes built-in [Pagination](https://community.denodo.com/docs/html/browse/latest/en/vdp/administration/creating_data_sources_and_base_views/path_types_in_virtual_dataport/http_path#pagination) functionality to handle APIs that return data on multiple pages. Denodo fetches data from all the pages when configured accordingly. In Denodo, while connecting a Data Source of HTTP connection, there is a tab to configure paginated Data. To enable this option, go to the tab Pagination, select Check Box The service returns paginated data, and select the pagination method of this service. When configured, it returns all the rows of the requests transparently. After retrieving the data from both the data sources you can [join these two views](https://community.denodo.com/docs/html/browse/latest/en/vdp/administration/creating_derived_views/creating_join_views/creating_join_views) using graphical interface. Also, you can review the performance using the [execution trace](https://community.denodo.com/docs/html/browse/latest/en/vdp/administration/creating_derived_views/querying_views/execution_trace_of_a_statement) and if there are any bottlenecks present you can identify the dominant factor and solve them by considering [caching, creating summaries, using Denodo query optimizer](https://community.denodo.com/kb/view/document/Best%20Practices%20to%20Maximize%20Performance%20IV:%20Detecting%20Bottlenecks%20in%20a%20Query?tag=Best+Practices#h.r53zo0gze7a9) etc. I find this to be extremely helpful when I am trying to gather more information about how Denodo is performing while handling any data. For more information, you can refer to the [HTTP Path](https://community.denodo.com/docs/html/browse/latest/en/vdp/administration/creating_data_sources_and_base_views/path_types_in_virtual_dataport/http_path), [Pagination](https://community.denodo.com/docs/html/browse/latest/en/vdp/administration/creating_data_sources_and_base_views/path_types_in_virtual_dataport/http_path#pagination) and [Detecting Bottlenecks in a Query](https://community.denodo.com/kb/view/document/Best%20Practices%20to%20Maximize%20Performance%20IV:%20Detecting%20Bottlenecks%20in%20a%20Query?tag=Best+Practices#h.r53zo0gze7a9) section of the Virtual DataPort VQL Guide. Hope this helps!
Denodo Team
29-04-2021 13:48:58 -0400
You must sign in to add an answer. If you do not have an account, you can register here