USER MANUALS

Prevent Load Balancers From Closing Connections of Data Catalog and Design Studio

Some proxies and load balancers close connections that have been inactive for a while; that is, no data flows through them. This affects operations that are not immediate. For example, if in Design Studio you run a query like “obtain the top 10 customers grouped by country”, this query returns only a few rows but may take several minutes to return the result. While the server is executing the query, the load balancer may consider that the connection between the browser of the user and Design Studio is inactive and close it, even though the user is still waiting for the query to finish.

The Data Catalog and the Design Studio have a feature to keep alive these connections between the browser of the user and the application (i.e. the web container of the Denodo Platform).

When this feature is enabled and the user runs an operation that may take several minutes to return data, when the application has not returned data through this connection for several minutes, it returns a “no-op” message to keep the connection alive.

When you enable this feature in Data Catalog, these operations benefit from this feature:

  1. Metadata synchronization.

  2. Query execution.

  3. Query export.

  4. VQL Shell: execution of queries, query export and download the execution trace.

When you enable this feature in Design Studio, these operations benefit from this feature:

  1. Queries executed from the Query panel of views and stored procedures, including Execute and download.

  2. VQL Shell: execution of any statement and “Execute and Download”

Note

The “Execute and Download” feature using ping connection may run slightly slower when the query returns a large number of rows.

Enabling this Feature

This feature is disabled by default in Data Catalog and Design Studio. To enable it, follow these steps:

  1. To enable this in Data Catalog, edit the file <DENODO_HOME>/conf/data-catalog/DataCatalogBackend.properties and add these properties:

# This property
connection-ping.enabled=true

# Every time this interval passes (in milliseconds) without sending data,
# the application sends a "noop" message to keep the connection alive.
#
# In this example, the value is two minutes
connection-ping.polling-interval=120000

connection-ping.core-pool-size=20
connection-ping.max-pool-size=40
connection-ping.queue-capacity=50

  1. To enable this in Design Studio, edit the file <SOLUTION_MANAGER_HOME>/conf/design-studio/DesignStudioBackend.properties and add the properties above.

  2. Restart Design Studio of the Solution Manager.

  3. Consider enabling this feature in the Design Studio of the installations of Denodo Platform if you use them.

This feature uses a pool of threads; the properties above control this pool; the default values suggested above should be valid for any use case:

  • If the number of threads is less than connection-ping.core-pool-size, the application will create a new thread to run a new task.

  • If the number of threads is equal (or greater than) connection-ping.core-pool-size, the application will put the task into the queue.

  • If the queue is full, and the number of threads is less than connection-ping.max-pool-size, the application will create a new thread to run tasks in.

  • If the queue is full, and the number of threads is greater than or equal to connection-ping.max-pool-size, the application will reject the task.

Add feedback