You can translate the question and the replies:

Denodo VDP server memory utilization is constantly high

Using the latest Denodo 8.0 with August patch. We have recently tweaked the file-max parameter to increase the swap file size in Denodo VDP server(linux) after facing the 'Too many open files' error. The soft limit is set as 1024 and hard limit is set as 16384. Since then, we are seeing a constant usage of memory in Denodo DMT tool and also the server utlization goes above 80% at times. Does it have any negative impact to memory because of this config change? Could you please suggest?
user
25-10-2022 17:19:39 -0400
code

3 Answers

Hi, In Denodo, ‘**Too many open files**’ error occurs when you are executing costly operations with large views that use swapping mechanism and the swapping has exceeded the maximum number of open files allowed by your operating system. You could avoid this error by reducing the number of files created by the swapping mechanism by modifying the Memory Usage settings of the Virtual DataPortserver. To achieve this, you could perform the following steps: 1. Navigate to "Administration > Server configuration > Memory usage" in the Web Design Studio. * Increase the value of "Maximum size in memory of each node" parameter. * You could also increase the "Maximum size of blocks stored in swap" parameter to create larger files. By creating files of larger size you can reduce the number of files created and thereby prevent the error. 2. Increase the maximum number of open files allowed by your Linux operating system. You can take a look at the [Linux Installation](https://community.denodo.com/kb/en/view/document/Denodo%20Admin%20and%20Development%20Best%20Practices#h.m4hlbwqltlfy-1:~:text=or%20data%20files.-,Linux%20Installation,-The%20recommended%20folder) section of the Denodo Admin and Development Best Practices Knowledge Base article. You could take a look at the [**Configuring the Memory Usage and Swapping Policy**](https://community.denodo.com/docs/html/browse/latest/en/vdp/administration/server_configuration/configuring_the_memory_usage_and_swapping_policy/configuring_the_memory_usage_and_swapping_policy) section of the Virtual DataPort Administration Guide for more information on Memory usage settings. If the issue persists and if you are a valid support user, you could create a support case on Denodo Support Site so that our team will assist you. Hope this helps!
Denodo Team
26-10-2022 06:32:14 -0400
code
Denodo, The question is after implementing both the solution options (Change in VDP client and also increasing the file limits in OS) that we could see a constant memory usage. Does it have to be one or the other option or will both help in improving this situation that was the primary ask? Current config: Amazon EC2 Linux - RHEL OS We have 8GB reserved for Heap and 4 core in a VDP server. For swap files, soft limit - 1024 , hard limit - 16384 Increased the memory in node to be - 100MB Increased the block size of swap file - 300KB Can you pls advise?
user
26-10-2022 12:16:52 -0400
Hi, The peaks in Memory utilization may occur due to a long-running query or resource-intensive query which is being executed in the Virtual DataPort server. In case I encounter this issue, I would load the log files of Denodo Monitor into the Diagnostic and Monitoring Tool and I will check for the queries which has lead to this high memory consumption. I would also ensure to align the JVM configuration with the [Recommended settings for the JVM in case of memory-demanding scenarios](https://community.denodo.com/kb/en/view/document/Denodo%20Admin%20and%20Development%20Best%20Practices#h.b9hulw4bhvja:~:text=easier%20to%20understand.-,JVM%20Configuration,-Denodo%20Virtual%20DataPort). For further assistance, you could create a support case on Denodo Support Site so that our team will help you. Hope this helps!
Denodo Team
27-10-2022 09:10:24 -0400
code
You must sign in to add an answer. If you do not have an account, you can register here