REST API

The Data Catalog provides a REST API to perform programmatically the following tasks:

Consider this:

  • The requests and responses of this API are in JSON format.
  • All the operations are stateless. That is, the client applications have to send the user credentials on every request.

Authentication with the REST API of the Data Catalog

The REST API of the Data Catalogs supports two authentication methods:

  1. HTTP Basic authentication
  2. Kerberos authentication

For Kerberos authentication, add the header:

Authorization: <Kerberos token encoded in base 64>

If the user credentials included in the request are incorrect, the API returns the HTTP code 403 (Forbidden).

Synchronize Elements with the Server

Operation to synchronize the metadata defined in the Data Catalog with the metadata from the Virtual DataPort server:

  • URL: /apirest/synchronize

  • Method: POST

  • Header: Content-Type: application/json

  • Syntax of the body of the request:

    {
        { "allServers": "true" | "serverName": "<name of the Virtual DataPort server>" }
        , "type": { "all" | "view" | "ws" }
        , "priority": {"server_with_local_changes" | "local" | "server" }
    }
    

    The body of the request is a JSON document with these elements:

    • "allServers": "true" or serverName: with allServer, the Data Catalog synchronizes its metadata with the metadata of all the Virtual DataPort servers registered on the Data Catalog. With serverName, it synchronizes its metadata with the metadata of that server.
    • type (optional): elements whose metadata will be synchronized. The possible values are:
      1. all (default value): web services and views
      2. view: only views
      3. ws: only web services
    • priority (optional): decision in case of conflicts. That is, action to do if the metadata of an element (e.g. the description of a view) has changed in Virtual DataPort and in the Data Catalog since the last time the metadata was synchronized. The possible values are:
      1. server_with_local_changes (default value): Virtual DataPort server but keeping the changes in the Data Catalog.
      2. local: changes in the Data Catalog have a priority over changes in Virtual DataPort.
      3. server: changes in Virtual DataPort override changes on the Data Catalog.

If the request succeeds, the operation returns the HTTP code 200 and the body of the response contains a JSON document with the list of elements (views and/or web services) that have been synchronized (i.e. inserted, removed and modified).

Format of the response of the operation “synchronize”
 {
     "result": "OK",
     "data": {
         "view": {
             "inserted": [ <list_of_inserted_views> ],
             "removed": [ <list_of_removed_views> ],
             "modified": [ <list_of_modified_views> ]
         },
         "ws": {
             "inserted": [ <list_of_inserted_web_services> ],
             "removed": [ <list_of_removed_web_services> ],
             "modified": [ <list_of_modified_web_services> ]
         }
     },
     "message": "OK"
 }

Example

This example uses the tool “curl” to send a request to synchronize the metadata:

Example of a request to the operation “synchronize” using “curl”
curl --request POST --user "jsmith:my_password" --header "Content-Type: application/json" --data '{ "allServers": "true", "priority": "server_with_local_changes" }' "https://denodo-server.acme.com:9090/denodo-data-catalog/apirest/synchronize"

Consider this:

  • The element "allServers": "true" of the body of the request indicates the Data Catalog to synchronize the metadata of all the Virtual DataPort servers. This is the most common value since usually you will only have one Virtual DataPort registered on the Data Catalog.
  • The element "priority": "server_with_local_changes" of the body of the request indicates the Data Catalog what to do in case of conflict.

The API returns a JSON document like this:

Example of a response of the operation “synchronize”
{
    "result": "OK",
    "data": {
        "view": {
            "inserted": ["customer360.invoice", "customer360.invoice_line"],
            "removed": [],
            "modified": []
        },
        "ws": {
            "inserted": [],
            "removed": [],
            "modified": []
        }
    },
    "message": "OK"
}

In this example, Virtual DataPort contains two new views: “invoice” and “invoice_line” of the database “customer360”.

Compute the Usage Statistics

Operation to compute and update the usage statistics of the Data Catalog.

  • URL: /apirest/fetching-statistics

  • Method: POST

  • Header: Content-Type: application/json

  • Body of the request:

    {
        "serverName": <text> | "allServers": "true"
    }
    

If the request succeeds, the operation returns the HTTP code 200 and the body of the response contains information about this operation, for each Virtual DataPort server registered in the Data Catalog.

Example

This example uses the tool “curl” to update the usage statistics of the Data Catalog:

Example of a request to the operation “fetching-statistics” using “curl”
curl --request POST --user "jsmith:my_password" --data  '{ "allServers": true }' --header "Content-Type: application/json" "https://denodo-server.acme.com:9090/denodo-data-catalog/apirest/fetching-statistics"
Example of a response of the operation “fetching-statistics”
 {
     "result": "OK",
     "data": {
         "responses": [
             {
                 "serverName": "localhost",
                 "serverUrl": "//localhost:9999/admin",
                 "httpStatusCode": 200,
                 "jsonResponse": {
                     "result": "OK",
                     "data": null,
                     "message": null
                 }
             },
             {
                 "serverName": "remote",
                 "serverUrl": "//remoteHost:19999/admin",
                 "httpStatusCode": 500,
                 "jsonResponse": {
                     "result": "ERROR",
                     "data": null,
                     "message": "None of periods was chosen for statistics"
                 }
             }
         ]
     },
     "message": null
 }

In this example, the statistics from server “localhost” were successfully computed but there was an error when doing the same for the server “localhost2”.

Export Metadata

Operation to export the Data Catalog’s configuration and metadata (categories, tags, saved queries, etc.) as a collection of JSON files compressed in a single zip file.

  • URL: /apirest/admin/export
  • Method: POST
  • URL parameters (all of them are optional and can be combined):
    • contentSearch=on: add it to export Content Search settings.
    • personalization=on: add it to export Personalization settings.
    • kerberos=on: add it to export Kerberos settings.
    • serverName: Virtual DataPort server. You have to specify this if there is more than one server registered with the Data Catalog. Otherwise, it is optional.

If the request succeeds, the operation returns the HTTP code 200 and a zip file with the result.

Example

This example uses the tool “curl” to export the Content Search settings and the Personalization settings:

Example of a request to the operation “export” using “curl”
curl --request POST --user "jsmith:my_password" --output "export-data-catalog.zip" "https://denodo-server.acme.com:9090/denodo-data-catalog/apirest/admin/export?contentSearch=on&personalization=on"

Import Metadata

Operation to import the Data Catalog’s configuration and metadata (categories, tags, saved queries, etc.). The input is a zip file with the files like the ones generated by the operation “export”.

A zip file exported from the web interface (Administration / Export menu) can be loaded from this REST API service.

  • URL: /apirest/admin/import
  • Method: POST
  • Content-Type: multipart/form-data
  • URL parameters (all of them are optional and can be combined):
    • override=true: add it to overwrite the existing metadata and configuration parameters.
    • serverName: Virtual DataPort server. You have to specify this if there is more than one server registered with the Data Catalog. Otherwise, it is optional.
  • Body of the request. Form-data parameter:
    • file: Name of the zip file to import. Absolute or relative file paths are not recognized, the file must be located in the same directory as the import/export scripts.

If the request has been successfully executed, the operation returns the HTTP code 200 and a JSON document with the result.

Example

POST localhost:9090/denodo-data-catalog/apirest/admin/import?override=true
Body parameter: file: admin-metadata-isst-1568129517057.zip
Example of a response of the operation “export”
 {
     "Result": "OK",
     "Object": null,
     "Message": "Metadata successfully imported",
     "Code": null
 }