How to limit the number of CPUs in Kubernetes

Applies to: Denodo 8.0 , Denodo 7.0
Last modified on: 21 Oct 2020
Tags: Administration Kubernetes Performance

Download document

You can translate the document:

Overview

In a traditional environment, Denodo is usually installed on servers with a capacity matching what the license allows. However, sometimes we are not free to define the capacity of the Denodo servers and we need to use a server that has more cores than what we are allowed to use by contract. In those cases, we need to limit the number of processors used by Denodo. Applying the limitation will depend mainly on the Operating System used, in general, this will be achieved with taskset on Linux and setting affinity on Windows.

In Kubernetes environments, we also might face a licensing issue because the node where the pod runs has more cores than the Denodo license. In these scenarios, the architecture is more complex given the dynamic nature of containers and because the worker nodes might be running several pods, and although the previous solution to assign specific cores to servers should work, you have other options available specific to Kubernetes environments.

By default, the pods running in Kubernetes are able to view as available all the cores from the node where the pod is running. If you are using a Denodo license with core restrictions, you might find that a server cannot start because the cores available in the pod exceeds the license restriction.

You can check the number of cores in your nodes with the following command:

kubectl get nodes -o custom-columns=NODE:.metadata.name,CPU:.status.capacity.cpu

So, if we are running one Denodo pod per node, then the easier option will be to create worker nodes with the same number of processors we are allowed to run, but if this option is not valid, then we can also make use of the static policy of the CPU Manager.

The Static Policy of the CPU Manager

The Kubernetes CPU Manager allows two different policies: none and static.

  • On the one hand the default policy “none does not really provide any affinity to the pods, so the restrictions applied in the Pod definition are enforced using CFS quota.
  • On the other hand, the static policy allows to assign dedicated CPUs on the node to a pod, and that CPU will be removed from the shared pool and not available for other processes. For this, the pods must run in Guaranteed QoS mode and request full cores (not fractions). The other pods running in Burstable mode or Guaranteed but with fractional CPU requests run with the shared pool of CPUs.

In order to enable the static policy, we must connect to each worker node with SSH and apply the CPU Manager configuration into the kubelet.service file:

$ sudo vi /etc/systemd/system/kubelet.service

In the kubelet.service, add the parameters --cpu-manager-policy=static and do a reservation of CPUs greater than 0 for --kube-reserved and --system-reserved:

    ...

    --cpu-manager-policy=static \

    --kube-reserved=cpu=1,memory=200Mi,ephemeral-storage=1Gi \

    --system-reserved=cpu=1,memory=200Mi,ephemeral-storage=1Gi \

    ...

Then, reset the CPU Manager by deleting the folder /var/lib/kubelet/cpu_manager_state and restarting the kubelet:

$ sudo rm -rf /var/lib/kubelet/cpu_manager_state

$ sudo systemctl daemon-reload

$ sudo systemctl stop kubelet

$ sudo systemctl start kubelet

After these steps, you can deploy your Denodo pods in Guaranteed mode and request full cores in the pod definition. For instance, you can append the following to the pod YAML to request 1 exclusive core:

    ...

    resources:

      limits:

        memory: "2000Mi"

        cpu: "1"

    ...

Testing the CPU Manager

Denodo checks the number of cores in use by querying the Java Runtime. You can also use a Java call to the Java Runtime to test that the CPU Manager is working as expected with the following Java class:

public class cores {

    public static void main(String[] args){

        Runtime runtime = Runtime.getRuntime();

        int num_cores = runtime.availableProcessors();

        System.out.println(num_cores);

    }

}

If the number of cores printed matches the resource limits configured, this means that the static policy is being taken into account.

Questions

Ask a question
You must sign in to ask a question. If you do not have an account, you can register here

Featured content

DENODO TRAINING

Ready for more? Great! We offer a comprehensive set of training courses, taught by our technical instructors in small, private groups for getting a full, in-depth guided training in the usage of the Denodo Platform. Check out our training courses.

Training