You can translate the question and the replies:

How to configure the Json data source for VDP-Data Catalog - Synchronize?

Hi Team, I am refering link "https://community.denodo.com/answers/question/details?questionId=9060g0000004FVuAAM&title=Data+Catalog+-+Synchronize+comments" for the Data Catalog - VDP - Code Synchronize. I am trying to create JSON data source calls the Denodo Data Catalog REST API and schedule the execution of this (JSON) view. But When I creating the data source referring to "https://community.denodo.com/docs/html/browse/latest/vdp/data_catalog/appendix/rest_api/rest_api#synchronize-elements-with-the-server" I found the HTTPURL it is provided is like "https://denodo-server.acme.com:9090/denodo-data-catalog/apirest/synchronize", I try to replace "denodo-server.acme.com:9090" with our server name and port, And I found the HTTPSURL cannot access. Actually our Datacatalog link is "http://", but in the document example it is "https://denodo-server.acme.com....." So what should I define my Json data source? Please see my configuration of my data-catalog Json data source, anything wrong: Data routh: HTTP Client In configuration: HTTP method: POST URL: https://our-server-host.compamy.com:9090/denodo-data-catalog/apirest/synchronize chosse: Post body { { "allServers": "true" | "serverName": "<name of the Virtual DataPort server>" } , "type": { "all" | "view" | "ws" } , "priority": {"server_with_local_changes" | "local" | "server" } } COntent type: application/json Authentication: Basic Login and passwrod: fill in out credentials here for username and pwd. Above not work, how can I configure it?
user
12-11-2020 02:35:58 -0500

9 Answers

Hi, Since your Data Catalog URL does not use HTTPS, you should use HTTP when specifying the URL in the data source. For example: ``` http://our-server-host.company.com:9090/denodo-data-catalog/apirest/synchronize ``` The POST body request to send will depend on your requirements. However, make sure to escape all of the "@", "{", "}", and "\" characters. For example, this POST request will synchronize its metadata with all of the Virtual DataPort servers registered in Data Catalog for web services and views and, in case of conflicts, prioritize changes in Virtual DataPort but preserve any edits made in Data Catalog: ``` \{ "allServers": "true", "type": "all", "priority": "server_with_local_changes" \} ``` Also, I noticed a small typo in your HTTP header, which should be "Content-Type". As for authentication, the [REST API of Data Catalog](https://community.denodo.com/docs/html/browse/latest/vdp/data_catalog/appendix/rest_api/rest_api#rest-api) supports two authentication methods: * HTTP Basic * Kerberos The ["Synchronize Elements with the Server"](https://community.denodo.com/docs/html/browse/latest/vdp/data_catalog/appendix/rest_api/rest_api#synchronize-elements-with-the-server) section of the Data Catalog Guide provides more information on the Data Catalog REST API's synchronize operation, such as the POST body elements and HTTP header. You may also find it helpful to refer to the following resources: * ["JSON Sources"](https://community.denodo.com/docs/html/browse/latest/vdp/administration/creating_data_sources_and_base_views/json_sources/json_sources#json-sources) in the Virtual DataPort Administration Guide * ["Error creating JSON data source with "Post Body" parameters"](https://community.denodo.com/kb/view/document/Error%20creating%20JSON%20data%20source%20with%20%22Post%20Body%22%20parameters?category=Common+Errors) Knowledge Base article Hope this helps!
Denodo Team
16-11-2020 15:34:51 -0500
Hi team, Thanks for your response, after changing to use HTTP when specifying the URL in the data source, and change the post body, my data source can be tested successfully. Then I created a base views on top of it. When I execute the view, I can see the views returned data, but I expand the returned result, I see in the last level of result,** it is shown "ERROR", and the message is "Incorrect number of updated tuples".** I also tried to run the cur command directly, like below: curl -v --request POST --user "username:pwd" --header "Content-Type: application/json" --data '{ "allServers": "true", "priority": "server" }' "http://host.company.com:9090/denodo-data-catalog/apirest/synchronize", it is also shown the same error message: * Connection #0 to host host.company.com left intact {"result":"OK","data":{"responses":[{"serverName":"Fine-UU-TEST","serverUrl":"//host.comapny.com:9999/Finance","httpStatusCode":500,"jsonResponse":{**"result":"ERROR","data":null,"message":"Incorrect number of updated tuples"**}}]},"message":null} Why it happend?
user
30-11-2020 03:17:36 -0500
some typo above, anyway, here is m curl command and return result, can you help check: command: curl --request POST --user "username:password" --header "Content-Type: application/json" --data '{ "allServers": "true", "priority": "server" }' "http://denodo_server_host:9090/denodo-data-catalog/apirest/synchronize" Result: {"result":"OK","data":{"responses":[{"serverName":"denodo_servername","serverUrl":"//denodo_server_host:9999/Finance","httpStatusCode":500,"jsonResponse":{"result":"ERROR","data":null,"message":"Incorrect number of updated tuples"}}]},"message":null} The httpStatusCode is 500, not 200. Can you please help check why? thanks.
user
30-11-2020 03:37:04 -0500
Hi team, any suggestions about above, why the status code is 500. not 200. Anything wrong of my curl command.
user
04-12-2020 03:50:06 -0500
Hi, The HTTP 500 error caused by "Incorrect number of updated tuples" can usually be resolved by recreating the Virtual DataPort server in Data Catalog. This is how you can do it: 1. In Data Catalog, navigate to the ["Administration > Set-Up > Server"](https://community.denodo.com/docs/html/browse/latest/vdp/data_catalog/administration/servers_configuration/servers_configuration#servers-configuration) section. 2. Here, rename the existing server to something else (e.g. "ServerNameOld"). 3. Create a new server (e.g. "ServerName") pointing to the same URL as "ServerNameOld". After recreating the server in Data Catalog as "ServerName", try synchronizing the Data Catalog metadata again. If the synchronization is successful, you can keep "ServerName" and delete "ServerNameOld". Hope this helps!
Denodo Team
07-12-2020 13:49:40 -0500
Ok, I use your way to create a new server, and sync it, and then I keep using below command, and I can see the auto-syncronization works: curl –request POST –user “username:password” –header “Content-Type: application/json” –data ‘{ “allServers”: “true”, “priority”: “server” }’ “http://denodo_server_host:9090/denodo-data-catalog/apirest/synchronize“ Questions: 1). Is it '500' status code because some previous syncronization failed before? 2). If I want to just sync the code from VDP for one specific database, and not sync all the metadata for all databases, how can we make it by this auto way or command? Thank you.
user
09-12-2020 00:17:29 -0500
Also, if I configured two Virtual DataPort Servers in Data catalog, Can I use the curl command to define only refresh the metadata for one specific server? Thanks.
user
09-12-2020 03:57:42 -0500
Hi team, any update?
user
14-12-2020 09:35:33 -0500
Hi, The 500 error caused by "Incorrect number of updated tuples" is usually returned when there is an issue with the metadata, which is why recreating the server in Data Catalog typically resolves the issue. If you want to just synchronize the metadata for one server, then instead of: ``` "allServers": "true" ``` you could use: ``` "serverName": "<name of the Virtual DataPort server>" ``` With "serverName", only the metadata of the specified server will be synchronized with Data Catalog. To learn more about the different elements in the request body, you can check out the ["Synchronize Elements with the Server"](https://community.denodo.com/docs/html/browse/latest/vdp/data_catalog/appendix/rest_api/rest_api#synchronize-elements-with-the-server) section of the Data Catalog Guide. Hope this helps!
Denodo Team
15-12-2020 21:43:31 -0500
You must sign in to add an answer. If you do not have an account, you can register here