Applies to:
Denodo 8.0
,
Denodo 7.0
Last modified on: 21 Oct 2020
Tags:
Administration
Kubernetes
Performance
In a traditional environment, Denodo is usually installed on servers with a capacity matching what the license allows. However, sometimes we are not free to define the capacity of the Denodo servers and we need to use a server that has more cores than what we are allowed to use by contract. In those cases, we need to limit the number of processors used by Denodo. Applying the limitation will depend mainly on the Operating System used, in general, this will be achieved with taskset on Linux and setting affinity on Windows.
In Kubernetes environments, we also might face a licensing issue because the node where the pod runs has more cores than the Denodo license. In these scenarios, the architecture is more complex given the dynamic nature of containers and because the worker nodes might be running several pods, and although the previous solution to assign specific cores to servers should work, you have other options available specific to Kubernetes environments.
By default, the pods running in Kubernetes are able to view as available all the cores from the node where the pod is running. If you are using a Denodo license with core restrictions, you might find that a server cannot start because the cores available in the pod exceeds the license restriction.
You can check the number of cores in your nodes with the following command:
kubectl get nodes -o custom-columns=NODE:.metadata.name,CPU:.status.capacity.cpu |
So, if we are running one Denodo pod per node, then the easier option will be to create worker nodes with the same number of processors we are allowed to run, but if this option is not valid, then we can also make use of the static policy of the CPU Manager.
The Kubernetes CPU Manager allows two different policies: none and static.
In order to enable the static policy, we must connect to each worker node with SSH and apply the CPU Manager configuration into the kubelet.service file:
$ sudo vi /etc/systemd/system/kubelet.service |
In the kubelet.service, add the parameters --cpu-manager-policy=static and do a reservation of CPUs greater than 0 for --kube-reserved and --system-reserved:
... --cpu-manager-policy=static \ --kube-reserved=cpu=1,memory=200Mi,ephemeral-storage=1Gi \ --system-reserved=cpu=1,memory=200Mi,ephemeral-storage=1Gi \ ... |
Then, reset the CPU Manager by deleting the folder /var/lib/kubelet/cpu_manager_state and restarting the kubelet:
$ sudo rm -rf /var/lib/kubelet/cpu_manager_state $ sudo systemctl daemon-reload $ sudo systemctl stop kubelet $ sudo systemctl start kubelet |
After these steps, you can deploy your Denodo pods in Guaranteed mode and request full cores in the pod definition. For instance, you can append the following to the pod YAML to request 1 exclusive core:
... resources: limits: memory: "2000Mi" cpu: "1" ... |
Denodo checks the number of cores in use by querying the Java Runtime. You can also use a Java call to the Java Runtime to test that the CPU Manager is working as expected with the following Java class:
public class cores { public static void main(String[] args){ Runtime runtime = Runtime.getRuntime(); int num_cores = runtime.availableProcessors(); System.out.println(num_cores); } } |
If the number of cores printed matches the resource limits configured, this means that the static policy is being taken into account.