This document explains how to configure Apache JMeter (http://jmeter.apache.org/index.html) to carry out performance tests on the Denodo Platform.
Apache JMeter is an open source performance testing tool. Any application that works on request/response model can be load tested with JMeter. It offers modules to test databases using JDBC, web services, web pages and many more. It can be used to simulate a heavy load on a server to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server under heavy concurrent load.
Therefore, it is a perfect tool to test a Denodo installation simulating access using JDBC or published web services, and tune the configuration parameters of the server accordingly. It is also a very useful tool to perform capacity analysis on a production server and draw the red lines for the current deployment.
JMeter can be downloaded from http://jmeter.apache.org/download_jmeter.cgi.
No special installation is required, just decompress the file. JMeter is a Java based application, so you will need to have the java executable in your OS path.
In order to use a JDBC connection to Denodo, the Denodo JDBC driver has to be loaded in the JMeter classpath. To do so, just copy Denodo’s JDBC Driver to <JMeter>/lib folder.
Denodo’s JDBC driver can be found under <DENODO_HOME>/tools/client-drivers/jdbc/denodo-vdp-jdbcdriver.jar (For Denodo versions previous to 6.0 it will be located under <DENODO_HOME>/lib/vdp-jdbcdriver-core/denodo-vdp-jdbcdriver.jar)
To launch JMeter GUI, just execute the file <JMeter>/bin/jmeter.bat or jmeter.sh.
This guide is not intended to be an exhaustive list of all the features of JMeter. For detailed information please refer to the Apache JMeter manual, that can be found online in http://jmeter.apache.org/usermanual/index.html.
In JMeter a test plan is defined as a number of concurrent users that execute the defined operations, in our case JDBC or web services requests. To construct the test plan, we will need the following elements:
The first step in a JMeter Test Plan is to add a Thread Group element. The Thread Group tells JMeter the number of users you want to simulate, how often the users should send requests, and the how many requests they should send. To add the Thread Group element, select the Test Plan, click your right mouse button to get the Add menu, and then select Add > Thread(Users) > Thread Group.
In the section Thread Properties, configure the following parameters:
Now that we have defined our users, it is time to define the tasks that they will be performing. In this case, JDBC requests to the Denodo server.
Before setting up the requests we will need to configure the JDBC Driver and the connection pool. Begin by selecting the Users element. Click your right mouse button to get the Add menu, and then select Add > Config Element > JDBC Connection Configuration. Then, select this new element to view its Control Panel.
Configure the fields as follows:
Select the Users element again. Click your right mouse button to get the Add menu, and then select Add > Sampler > JDBC Request.
You can define several requests for your test plan. JMeter will send the requests in the order you add them to the tree, and will execute them for each user. It is usually a good idea to choose a set of queries from a previous analysis based on the monitoring of the current production scenario.
You will have to define the following parameters:
If you want to test the access to Denodo via Web Services, you will need to set up an HTTP Request sampler or a Web Service (SOAP) Request, depending on your case. The configuration is very similar to the JDBC Requests, although you don’t need to configure any driver in this case.
You can also mix JDBC and web service requests in the same test case to simulate a scenario with mixed load. Just keep in mind that the requests will be sent in the same order they are defined in the test plan tree.
To add a HTTP request select the Users element again. Click your right mouse button to get the Add menu, and then select Add > Sampler > JDBC Request.
The final element you need to add to your Test Plan is a Listener. This element is responsible for storing all of the results of your requests in a file and presenting a visual model of the data.
Select the Users element and add a Summary Report listener (Add > Listener > Summary Report).
Important Note: The Apache JMeter documentation recommends avoid using the GUI for load testing. The graphical interface mode does have a limitation which slows down the CPU utilization while running the recorded script.
It is recommended to use the command line interface option instead of the GUI mode for running the load tests defined above.
Once the plan is set up, you will have the tests stored in .jmx file. For instance: test.jmx file.
You can execute the plan using a command line and execute a command like the following:
<JMeter>/bin/jmeter.bat -n -t test.jmx -l testresults.jtl
The command will generate a testresults.jtl file in the <JMeter>/bin folder.
After generating the testresults.jtl file you can view the results following these steps:
Once the test plan is set up, you can run it clicking the “Play” button in the toolbar, or using the menu option Run > Start.
Select the Summary Report element in the tree to see the report. In the report you will find useful metrics like the execution times (min, max and average), the percentage of errors and the throughput (queries per second).
In combination with the report generated by JMeter, it is also very useful to monitor the behavior of the Denodo server in real time with tools like the Denodo Diagnostic & Monitoring tool or VisualVM.
This will allow you to monitor not only the results of the queries, but also a number of interesting parameters like:
Denodo Diagnostic & Monitoring tool monitoring different tests launched from JMeter
Once the test is completed, it is time to analyze the results. If there are no errors in the execution, we can increase the number of users (in the configuration screen of the users element in JMeter) to test the server under a heavier load.
An analysis of several parameters in execution time (usually via JMX or using the Diagnostic & Monitoring tool) and the analysis of the logs of the server (found in <DENODO_HOME>/logs/vdp/vdp.log) will help in the tuning process.
The following parameters are usually involved in the behavior of the system and may impact the result of the tests.
The Denodo server is a Java process, and as such, it needs a memory configuration adequate to its needs. In high load scenarios, the following parameters are recommended:
When the server runs out of memory, it will write Heap Space errors in the log file.
For for information about the recommended settings for the JVM, you can refer to the following Knowledge Base article: Denodo Admin and Development Best Practices
If the client application is connected to Denodo using a connection pool, their parameters must be set accordingly. In the case of JMeter, this is configured in the JDBC configuration screen, as explained in previous sections. This also applies to the web services deployed in the embedded web service container that can be configured from the Edit Web service screen:
JDBC and ODBC sources use a connection pool for the queries issued to these types of sources. The values of these parameters can limit the number of outbound connections, leaving new incoming queries waiting for an available connection. These parameters can be modified in the “Configuration” screen for the data source, clicking on “Connection>Connection pool configuration”.
The status of the connection pools during the execution of the test can be accessed via JMX, using the properties numActive and numIdle in the beans com.denodo.vd.management.mbeans.DataSourceManagementInfo.<db>.(jdbc|odbc).<data source>
This information can also be checked using the Denodo Diagnostic & Monitoring tool in the Data sources tab for JDBC and ODBC sources.
An error raised by Denodo that shows that an outbound connection pool is exhausted is:
Error getting JDBC connection java.util.NoSuchElementException: Timeout waiting for idle object
Denodo’s cache is just another JDBC source, so its connection pool should be configured accordingly as well. The cache settings can be modified from the Administration tool menu, Administration > Server Configuration > Cache.
Denodo uses an internal threads pool, which is used by the threads of the execution engine. The parameter “Max threads” will limit the number of threads available for the execution engine. If the number of concurrent requests is too high, then this may lead to exhausting the threads pool. In that case, the max threads parameter should be increased. Other parameters to bear in mind are the thread timeout and checkout time. High concurrency scenarios may slow down the average execution times, producing timeouts in execution or waiting queues. These settings can be modified from the Administration tool menu, Administration > Server Configuration > Threads Pool. More information on these parameters can be found in the Virtual DataPort Administration Guide, section “Threads Pool”.
The errors raised in Denodo when the threads pool is exhausted will look like:
com.denodo.vdb.engine.thread.ThreadPool  - addWork - error requesting thread: Timeout while waiting for an object (checkout)
It is possible to limit the number of requests that the Denodo will accept concurrently. When that limit is reached, the new requests will be queued and executed according to their arrival order. Limiting the number of concurrent requests is useful in high load environments, since it avoids performance degradation issues when there is a peak load. These settings can be modified from the Administration tool menu, Administration > Server Configuration > Concurrent Requests. For more information on these parameters, please go to the Virtual DataPort Administration Guide, section “Limiting the Number of Concurrent Requests”.
When Denodo issues a high number of queries to one particular source, the performance of that source can also be affected or even be the main bottleneck of the execution.
Web services deployed by Denodo are small client applications running in a web applications container. Denodo’s embedded web container is an Apache Tomcat. Considerations on its performance must be taken into account during stress tests as well. Adequate memory configuration and connections pools for the web services with the adequate size and timeouts are important parameters. When the embedded Tomcat is used, it must also be clear that the Apache Tomcat process will compete with the Denodo VDP server for HW resources, especially CPU and memory.
By default, Denodo’s embedded web container is configured to handle a maximum of 150 simultaneous threads. If the number of concurrent requests we want to test is bigger than that, this setting must be changed accordingly.
<!-- Define a non-SSL HTTP/1.1 Connector →