Limit the Maximum Number of Rows Returned¶

You can limit the maximum number of rows returned by the RESTful web service. That is, if this limit is enabled, when a client queries a view, the number of rows in the response will never exceed this limit.

This feature is useful to make sure that there is not any client who puts too much strain on the Virtual DataPort server or its data sources.

To set this limit, follow these steps:

1. Stop all the Denodo servers. The goal is to make sure the web container is stopped.

2. Edit the file other_settings.xml of the folder <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/

3. Set the value of the property maximumRowsPerPage to the limit you want to set.

4. Start the Denodo servers.

Changes to this file will be preserved even when an update of the Denodo Platform is installed.

Design Studio provides graphical support to configure this limit on REST web services (the ones that only publish a set of views). The section Settings Tab (REST) explains how to do this.

Limit the Memory Used When Generating Response¶

Some configurations applied to the web service may require to store the result in memory until the full response is generated. This is the case for returning pagination information as HTTP headers and for enabling RFC 7807 errors while paginating results. The main reason is that HTTP headers and status code should be setted before the response is being sent to the client so no rows streaming can be done in the previous scenarios. To prevent this features to use more memory than expected you can enable a memory limit by following these steps:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Add the following env-entry section

   <env-entry>
       <env-entry-name>resultMemoryLimit</env-entry-name>
       <env-entry-type>java.lang.Integer</env-entry-type>
       <env-entry-value>100</env-entry-value>
   </env-entry>
   

3. Set a value in MB for the maximum size for each HTTP response generation. A value of 0 will disable the limit

SET 'com.denodo.wsgenerator.rest.memoryLimit'='<value in MB>';

SET 'com.denodo.wsgenerator.rest.enableMemoryLimit'='false';

Cross-origin Resource Sharing (CORS)¶

The RESTful web service provides support for Cross-origin resource sharing (Cross-Origin Resource Sharing (CORS)). The section Cross-origin Resource Sharing (CORS) (settings of published REST web services) explains in more details what CORS is.

To enable the CORS support on the global RESTful web service, follow these steps:

1. Do the steps of the section Initial Set-Up of CORS (you have done this already if you published a Denodo REST web service with CORS enabled).

2. Before starting any server of this Denodo installation, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/cors_settings.xml

3. Uncomment the contents of the file.

4. If the value of the property is *, the Service will allow CORS requests received from any domain.

   To limit the domains from which CORS requests are allowed, change the value of the property cors.allowOrigin; replace * with the list of allowed URLs. Separate each URL by a comma.

   For example, https://foo.com, http://foo.com, https://foo.bar.com

   CORS requests from any other origin will be denied with the HTTP code 403 (Forbidden)

   Important

   For each URL, enter its protocol as well. URLs that not contain the protocol are invalid. E.g. foo.com is invalid.

5. Start the Denodo servers.

Changes to this file will be preserved even when an update of the Denodo Platform is installed.

This change only affects the global RESTful web service. If you want to publish REST web services with CORS support, see the section Cross-origin Resource Sharing (CORS) (settings of published REST web services).

Customizing the Look & Feel of the RESTful Web Service¶

The RESTful web service and the REST web service show the logo of the Denodo Platform at the top.

To replace the Denodo logo with another logo, follow these steps:

1. Copy the new logo to the directory <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/resources/.

2. Edit the file stylesheet-customizable.css of the directory <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/resources/

3. Uncomment the CSS definition div#page-header.

4. In this definition, replace the URL of the attribute background-image with the URL of the new logo.

   You should end up with a CSS definition similar to this one:

   div#page-header {
      background-image: url('/denodo-restfulws/resources/new_logo.png');
      background-position: top left;
      background-repeat: no-repeat;
      height: 45px;
      padding-bottom: 25px;
      position: relative;
      width: 85%;
      height: 80px;
   }
   

These changes are effective immediately; you do not need to restart. You may need to refresh the cache of the browsers.

Changes to this file will be preserved even when an update of the Denodo Platform is installed.

After following the steps above, all the REST web services that you deploy or redeploy and that do not have its CSS definition customized, will use this new CSS definition. Therefore, they will display the new icon instead of the Denodo one.

Customizing RESTful Web Service to Return Pagination Information as HTTP Headers¶

By default the RESTful Web Service provides pagination information in the response by adding the links to previous/next page. This allows VDP to stream results from the beginning and add the pagination information when the result has been processed.

For receiving the pagination information as HTTP Headers configure the following:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Locate or add the following env-entry section

   <env-entry>
       <env-entry-name>enablePaginationHeaders</env-entry-name>
       <env-entry-type>java.lang.Boolean</env-entry-type>
       <env-entry-value>FALSE</env-entry-value>
   </env-entry>
   

3. Update the value to TRUE

SET 'com.denodo.wsgenerator.restws.enablePaginationHeaders'='true';

This configuration will return the following HTTP headers when using pagination on RESTful Web Services:

1. pagination-index: The index of the first result returned on this page.

2. pagination-count: The requested number of results on this page.

3. pagination-page-results: The actual number of results returned in this page (the value would be equals to pagination-count except on the last page that may be lower).

There is also the possibility to rename the headers using a JSON mapping by configuring the following:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Locate or add the following env-entry section

   <env-entry>
       <env-entry-name>paginationHeadersMapping</env-entry-name>
       <env-entry-type>java.lang.String</env-entry-type>
       <env-entry-value>{}</env-entry-value>
   </env-entry>
   

3. Update the value to specify the mapping of names in JSON format. For example: {"pagination-page-results":"pageresults", "pagination-index":"pageindex", "pagination-count":"pagecount"}

   <env-entry>
       <env-entry-name>paginationHeadersMapping</env-entry-name>
       <env-entry-type>java.lang.String</env-entry-type>
       <env-entry-value>{"pagination-page-results":"pageresults", "pagination-index":"pageindex", "pagination-count":"pagecount"}</env-entry-value>
   </env-entry>
   

SET 'com.denodo.wsgenerator.restws.paginationHeadersMapping'='{<mapping JSON>}';

Follow RFC 7807 Specification on HTTP Errors¶

By default the RESTful web service and deployed web services returning an error do generate a custom error message. The response Content-Type is also the same as when an HTTP 200 status is returned (application/json for example).

In RFC 7807 there is a definition of a standard way to return problems in HTTP REST responses for JSON and XML messages. This RFC describes how the response should contain an application/problem+json or application/problem+xml as Content-Type. It does also provide requirements and recommendations about the error message structure.

You can enable the RESTFul web service to follow RFC 7807 specification by doing the following:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Locate or add the following env-entry section

   <env-entry>
    <env-entry-name>restfulErrorsAsApplicationProblem</env-entry-name>
    <env-entry-type>java.lang.Boolean</env-entry-type>
    <env-entry-value>FALSE</env-entry-value>
   </env-entry>
   

3. Update the value to TRUE

SET 'com.denodo.wsgenerator.restws.restfulErrorsAsApplicationProblem'='true';

Once this setting is enabled the response error messages will change its structure and this will be reflected also in the OpenAPI specification. Check the following samples on how the error is being formatted:

Authentication error as application/problem+json:

{
   "type":"http://www.denodo.com/restful/error/http/401/vdp/-1",
   "title":"Unauthorized",
   "status":401,
   "code":-1,
   "detail":"You are not authorized to access this resource",
   "__errors__":[
      {
         "message":"You are not authorized to access this resource"
      }
   ]
}

Authentication error as application/problem+xml:

<problem
    xmlns="urn:ietf:rfc:7807">
    <type>http://www.denodo.com/restful/error/http/401/vdp/-1</type>
    <title>unauthorized</title>
    <status>401</status>
    <code>-1</code>
    <detail>you are not authorized to access this resource</detail>
    <__errors__>
        <error message="you are not authorized to access this resource"></error>
    </__errors__>
</problem>

Customizing Request Parameter Names in RESTFul Web service¶

By default the RESTful web service and deployed web services require a set of predefined input parameters defined in the table Parameters supported by the Denodo RESTful Web service and published REST Web services.

You can define a mapping to rename the query parameters in the RESTFul web service by doing the following:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Locate or add the following env-entry section

   <env-entry>
    <env-entry-name>restfulParameterNamesMapping</env-entry-name>
    <env-entry-type>java.lang.String</env-entry-type>
    <env-entry-value>{}</env-entry-value>
   </env-entry>
   

3. Update the value to specify the mapping of names in JSON format. To rename $format to format and $count to number_of_results you need to add the following value: {"$format":"format", "$count":"number_of_results"}

   <env-entry>
    <env-entry-name>restfulParameterNamesMapping</env-entry-name>
    <env-entry-type>java.lang.String</env-entry-type>
    <env-entry-value>{"$format":"format", "$count":"number_of_results"}</env-entry-value>
   </env-entry>
   

By doing this you can use the mapped names in the invocation query. The OpenAPI specification and pagination links will also reflect the new names configured.

SET 'com.denodo.wsgenerator.restws.restfulParameterNamesMapping'='{<mapping JSON>}';

Modifying the Connection Parameters Between the RESTful Web Service and Virtual DataPort¶

As any client of Virtual DataPort, the RESTful web service has its own connection parameters. In most situations, the default connection parameters of the service are fine, but there may be scenarios where you need to change them. For example, let us say that you are going to query views that take a long time to respond. The default query timeout of this service is fifteen minutes. If you think that some queries may take longer, increase it by following these steps:

1. Stop all the Denodo servers. The goal is to make sure the web container is stopped.

2. Edit the file connection_settings.xml of the folder <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/.

3. Increase the value of the property queryTimeout. Note the value of this property is in milliseconds.

   You can also modify other properties of this file if you need to. Each property has a comment above explaining it.

4. Start the Denodo servers.

Changes to this file will be preserved even when an update of the Denodo Platform is installed.

This change only affects the global RESTful web service. Each REST web service has its own connection parameters.

Modify the RESTful Web Service to Return Only the Result set on XML/JSON Response¶

By default the RESTful Web Service returns execution results grouping metadata information like view name, data and errors. In some environments it may be required to get only data as an array without additional information.

To enable this behavior you must configure the following:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Locate or add the following env-entry section

   <env-entry>
       <env-entry-name>hideRestfulMetadataOnResult</env-entry-name>
       <env-entry-type>java.lang.Boolean</env-entry-type>
       <env-entry-value>FALSE</env-entry-value>
   </env-entry>
   

3. Update the value to TRUE

SET 'com.denodo.wsgenerator.restws.hideRestfulMetadataOnResult'='true';

Examples:

• XML example:

 {
    "name": "testfail",
    "elements": [
      {
        "field":"value"
      },
      {
        "field":"value"
      },
    ]
}

Will change to:

[
     {
       "field":"value"
     },
     {
       "field":"value"
     },
]

• XML example:

<denodo:view name="<view_name>" xmlns="...">
    <view_name>
        <rowdata/>
    </view_name>
    <view_name>
        <rowdata/>
    </view_name>
</denodo:view>

Will change to:

<denodo:view xmlns="...">
    <item>
        <rowdata/>
    </item>
    <item>
        <rowdata/>
    </item>
</denodo:view>

Mark View Columns as Mandatory, Remove From Input or Remove From Output¶

In the published web services you can set columns as mandatory and remove it from the input or the output. However, in the generic RESTful Web Service is necessary to define a mapping between those actions and a tag name.

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml

2. Add the following env-entry section

   <env-entry>
       <env-entry-name>tagNamesMapping</env-entry-name>
       <env-entry-type>java.lang.String</env-entry-type>
       <env-entry-value>{"mandatory_field_tag": "<tag_name>", "remove_field_from_input_tag": "<tag_name>", "remove_field_from_output_tag": "<tag_name>"}</env-entry-value>
   </env-entry>
   

3. Update the value to specify the mapping of tag names in JSON format. For example: {"mandatory_field_tag": "mandatory_tag", "remove_field_from_input_tag": "remove_input_tag", "remove_field_from_output_tag": "remove_output_tag"}

   <env-entry>
       <env-entry-name>tagNamesMapping</env-entry-name>
       <env-entry-type>java.lang.String</env-entry-type>
       <env-entry-value>{"mandatory_field_tag": "mandatory_tag", "remove_field_from_input_tag": "remove_input_tag", "remove_field_from_output_tag": "remove_output_tag"}</env-entry-value>
   </env-entry>
   

4. Start the web container.

5. Using Design Studio, create the three tags specified before: “mandatory_tag”, “remove_input_tag” and “remove_output_tag” (see Tags Management).

6. Tag the columns of a view with the previous tags to apply the corresponding action when accessing the view from the generic RESTful service. For example, there’s a view that has three fields “iinc_id”, “summary” and “ttime”:

   • To set the “iinc_id” field as mandatory when querying the view from the RESTful service, tag the column “iinc_id” with “mandatory_tag”.

   • To remove the field “summary” from the input fields when querying the view from the RESTful service, tag the column “summary” with “remove_input_tag”.

   • To remove the field “ttime” from the output fields when querying the view from the RESTful service, tag the column “ttime” with “remove_output_tag”.

Rename the “__errors__” tag of response error messages¶

By default, response error messages from web services have the following structure:

{
   "__errors__": [
         {
            "message": "You are not authorized to access this resource"
         }
   ]
}

You can change the “__errors__” tag by another name by doing the following:

1. With the web container stopped, edit the file <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-restfulws/WEB-INF/other_settings.xml and add the following property:

   <env-entry>
       <env-entry-name>responseErrorsFieldName</env-entry-name>
       <env-entry-type>java.lang.String</env-entry-type>
       <env-entry-value>error_custom_tag</env-entry-value>
   </env-entry>
   

2. Replace error_custom_tag with the tag you need to use

SET 'com.denodo.wsgenerator.restws.responseErrorsFieldName' = 'error_custom_tag';

Once this setting is enabled the response error messages will change the “__errors__” tag by the name specified in the property. For the previous example:

{
    "error_custom_tag": [
        {
            "message": "You are not authorized to access this resource"
        }
    ]
}

<?xml version="1.0" encoding="UTF-8"?>
<error_custom_tag xmlns="http://www.denodo.com/restful">
    <error message="You are not authorized to access this resource"></error>
</error_custom_tag>

Configure Multiple Authentication Methods¶

The following authentication methods are supported: HTTP Basic with VDP, OAUTH 2.0 and Kerberos. To enable one or multiple of these methods add the property com.denodo.restfulws.supportedAuthentications in <DENODO_HOME>/conf/vdp/VDBConfiguration.properties. The allowed values are AUTH_HTTP_BASIC_VDP, AUTH_HTTP_OAUTH2 and AUTH_HTTP_SPNEGO separated by commas. For instance, to enable all supported authentications:

SET 'com.denodo.restfulws.supportedAuthentications' = 'AUTH_HTTP_BASIC_VDP,AUTH_HTTP_OAUTH2,AUTH_HTTP_SPNEGO';

Configure the OpenAPI 3 Endpoint¶

SET 'com.denodo.restfulws.openapiv3.enabled' = 'true';

The OpenAPI document fields Title, Description and Version can be customized by setting the following property:

SET 'com.denodo.restfulws.openapiv3.info' = '{"title":"<title>", "description":"<description>", "version":"<version>"}';

The retrieved results can be limited by tags, databases and number of views by setting the following property:

SET 'com.denodo.restfulws.openapiv3.limitViews' = '{"allowedTags":["<tag1>", "<tag2>"], "allowedDatabases":["<db1>", "<db2>"], "viewsLimit": <numberOfViews>}';

The request parameters database and tag overwrite the property values. For instance: https://denodo-server.acme.com:9443/denodo-restfulws/OpenAPIv3/openapi.yaml?database=db1&database=db2&tag=tag1&tag=tag2.

Important

A default limit of retrieved views is set to 10.000. The generated OpenAPI document includes a boolean specification extension called x-viewsLimitReached in the info field which indicates if the specified views limit has been reached.

Send Addition Information in the Authentication Process Using HTTP Headers¶

To configure the desired headers execute the following:

SET 'com.denodo.restfulws.customAuthHeaders'='["header1", "header2", "header3"]';

and restart the server.

Configure Page Size Based on View Tags¶

To configure the page sizes (maximum and default) for each desired tag set the following property:

SET 'com.denodo.restfulws.tagPageConfiguration' = '[{"tagName": "<tagName1>", "maxPage": <maxPageNumber>, "defaultPage": <defaultPageNumber>}, {"tagName": "<tagName2>", ...}]';

and restart the server.

Access Workspaces¶

To access a workspace add the parameter $workspace, for example: https://<host>:<port>/denodo-restfulws?$workspace=<workspace>. Alternatively, the header X-workspace can be set. If both are present the parameter has preference.