How to create your own Docker images for Denodo Platform Containers

Applies to: Denodo 8.0 , Denodo 7.0
Last modified on: 03 Dec 2020
Tags: Administration Docker

Download document

You can translate the document:

Introduction

This article’s aim is to explain how to build your own docker images. This is not an introductory document to Containers so we assume that the reader has some knowledge of Docker.

Building the Docker images

In the following sections, we are going to describe how to build your own Docker images with Denodo Platform installed.

Building an image for Denodo Platform involves:

  1. Generating a Denodo Platform auto-installation XML file. You will need to download a Denodo Platform installer compatible with your OS.
  2. Creating a customized Dockerfile with the instructions to build your image using the Denodo Platform auto-installation file.
  3. Preparing a build folder with the auto-installation XML file, an unzipped Denodo Platform installer for Linux, and the Dockerfile.
  4. Executing the docker build command in the build folder.

Generating an auto-installation file

To generate an auto-installation file you must download and unzip the Denodo Platform installer for the OS (Linux, Windows) that you have installed on your local machine, independently of the OS file system you will use as the base in the image you intend to build. Follow the instructions to generate the auto-installation file depending on your installed OS.  The generated file is supposed to be named install.xml in the subsequent sections.

Notice that the installer sets in the auto-installation file the Denodo Platform installation path according to the OS it was generated, so remember to edit the auto-installation file to change the installation path to be coherent with the path syntax of the Linux base image that we will use to generate the Container image. We assume in the rest of the document that the Denodo Platform will be installed in the path /opt/denodo

Creating a Dockerfile

The following Dockerfile is a skeleton that may serve you to build your own customized one. Using the Debian:stable image as the base, it copies the Denodo Platform Linux installer that you have already downloaded and the generated auto-installation file from your host to the image file system to install Denodo Platform on a container.

FROM debian:stable as builder

ENV DENODO_HOME=/opt/denodo

ENV PATH="${DENODO_HOME}/jre/bin:${PATH}"

# The Denodo installation requires the uncompressed denodo installer and

# an install.xml file for unattended installation.

COPY ./denodo-install-<version_name>-linux64/denodo-install-<version_name> /opt/denodo-install

COPY ./install.xml /opt/denodo-install/install.xml

WORKDIR /opt/denodo-install

RUN chmod +x installer_cli.sh

RUN ./installer_cli.sh install --autoinstaller install.xml

# Modify the vqlserver start script to wait for vqlserver to exit (so

# the container does not die upon start)

RUN sed -i "s/exit 0/wait/g" ${DENODO_HOME}/bin/vqlserver.sh

## Multi-step built to reduce the size of the final image

FROM debian:stable

ENV DENODO_HOME=/opt/denodo

ENV PATH="${DENODO_HOME}/jre/bin:${PATH}"

# Copy Denodo Platform from the builder image to the end image

COPY --from=builder $DENODO_HOME $DENODO_HOME

# Sets the workdir for the rest of the script

WORKDIR $DENODO_HOME

# The following ports used by VDP are exposed:

# 9999 - Denodo (JDBC, AdminTool, ...)

# 9997 & 9995 RMI/JMX (Both are JMX ports for Denodo 8.0. 9997 is JDBC for Denodo 7.0. Only expose 9995 incase of Denodo 8.0)

# 9996 - ODBC

# 9090 - Web Services

EXPOSE 9999 9997 9996 9090 9995

CMD ${DENODO_HOME}/bin/vqlserver.sh startup

# Example docker run command (Include 9995 for Denodo 8.0 only):

# docker run -d -h localhost -p 9999:9999 -p 9997:9997 -p 9996:9996 -p 9995:9995 -p  # 9090:9090 -v <path>:/opt/denodo/conf/denodo.lic denodo-image

Make sure to make the changes to include the actual version name for the file path on the above file content. This Dockerfile changes the vqlserver.sh script, which is used to start the VDP server, so it waits for the VDP server to stop (instead of the default exit behavior) in order to prevent the container from automatically stopping after executing the script. This change should always be present at your customized Dockerfile, so in case you need to customize the settings of the Denodo Platform, ensure that you do this change as one of the last steps.

The last part of the Dockerfile just exposes the ports that are typically used by the VDP server and defines the default command to execute.

If you want to start multiple denodo modules in the same container you will need to create your own wrapper script to invoke all the individual startup scripts and modify the CMD line in the Dockerfile. You should keep the vqlserver.sh as the last command, as it will keep the script waiting until VDP dies, marking the container as dead for Docker.  

Starting several services

In case you want to start several Denodo Platform services when running your container you must create a custom start script and add it to the Container that launches all the desired services. Take into account that some services must be started in a specific order, for example, the data catalog requires the VDP server to be started previously.

As an alternative, you can manually start these servers with the docker exec command after the Denodo Platform container has been started. For example, you can start the data catalog in a running denodo container named “denodo” with the command:

$ docker exec -t denodo ./bin/datacatalog_startup.sh

Denodo Licenses

Denodo Platform requires a license to run. If you intend to use a license managed by the License Manager you can create an image containing a SolutionManager.properties configuration file placed on the path $DENODO_HOME/conf that defines the License Manager location.

Alternatively, you can map this configuration file or a Denodo standalone license (like the  Evaluation or Denodo Express licenses) from your host to a container using the Docker volume mount parameters. The previous Dockerfile assumes that you will map the license this way.

Preparing the build directory

If your local machine has installed an OS whose file system is different from the one that your base image will use (Linux), it is necessary to download and unzip the suitable Denodo Platform installer too. In the following example, we are going to use a Debian image as a base so the Denodo Platform for Linux is the required one independently of your local OS.

You need to create a temporary folder and add the following files to it:

  • Your customized build file, named Dockerfile
  • The auto-installation file, named install.xml
  • The unzipped Denodo Platform installer (and optionally the jar file corresponding to any Denodo Update), in the folder with name:
    denodo-install-<version_name>-linux64

Executing the docker build command

To generate your new Denodo Platform image you need to open a command line on the recently created build directory and run the following command:

$ docker build -t denodo-image:latest .

Please note the space period at the end of the build command. Check that the execution of this command has occurred without errors.

Deploying in Red Hat OpenShift

The Dockerfile template provided before uses a Debian image as the base system. However, if you want to deploy the container in Red Hat OpenShift you must base your container image on Red Hat Universal Base Image (UBI).

It is possible to create the Denodo container based on a different image base by just replacing the text debian:stable in the Dockerfile with your preferred UBI image.

For example, if you want to use ubi8 you can replace the line in the template that indicates:

FROM debian:stable

with:

FROM registry.access.redhat.com/ubi8/ubi:latest

Another requirement for deploying in OpenShift is that the processes do not run as root. This is an additional security measure, but it is typically more practical to allow the container to run their processes as root.

In order to run Denodo as a different user you need to replace the line:

COPY --from=builder $DENODO_HOME $DENODO_HOME

with:

RUN addgroup denodo && \

    adduser --disabled-password --gecos '' \

            --ingroup denodo denodo

COPY --from=builder --chown=denodo:denodo \

     ${DENODO_HOME} ${DENODO_HOME}

USER denodo

Testing the image

You can test the image you have just built with the docker run command (Include 9995 for Denodo 8.0 only):

$ docker run -d --name denodo -h localhost -p 9999:9999 -p 9996:9996 -p 9995:9995 -p 9090:9090 -v <path>:/opt/denodo/conf/denodo.lic denodo-image

Where:

-v <path>:/opt/denodo/conf/denodo.lic

Specifies the Standalone Denodo License file to use by the Denodo Server, where:

  • <path> - Path for the license file in the host OS, for example: /home/denodo/denodo.lic
  • /opt/denodo/conf/denodo.lic - Path for the license file in the container (Do not change this)

Installing a new update

Denodo delivers new updates regularly to improve the user experience of the Denodo Platform. When a new update is released, the Denodo container that is available to download from the Denodo Support Site gets updated to the new version, so there is always a container with the latest version of Denodo ready to download.

However, there might be situations where you would like to build your own container using a specific Denodo version that does not match that version available in the Support Site.

In those cases, you can use the following Dockerfile as the template for building a new image with the installation of the desired update:

FROM denodo-platform:<version> as builder

# Update installation

COPY denodo-<update-version>.jar /opt/hotfix/denodo-<update-version>.jar

RUN ${DENODO_HOME}/jre/bin/java -jar /opt/hotfix/denodo-v80-update-xxx.jar ${DENODO_HOME} -c

# Using multi-stage builds to reduce the size of the image

FROM debian:stable

ENV DENODO_HOME=/opt/denodo

ENV PATH="${DENODO_HOME}/jre/bin:${PATH}"

# Copy Denodo Platform from the builder image to the end image

COPY --from=builder $DENODO_HOME $DENODO_HOME

# Sets the workdir for the rest of the script

WORKDIR $DENODO_HOME

# The following ports used by VDP are exposed:

# 9999 - Denodo (JDBC, AdminTool, ...)

# 9997 & 9995 (Both are JMX ports for Denodo 8.0. 9997 is JDBC for Denodo 7.0. Only expose 9995 incase of Denodo 8.0)

# 9996 - ODBC

# 9090 - Web Services

EXPOSE 9999 9997 9996 9995 9090

 

CMD ${DENODO_HOME}/bin/vqlserver.sh startup

After setting the values for the image version and the name of the update file in the placeholders of the template, you can build and test the image by using the same commands already provided in the previous sections of the article.

Questions

Ask a question
You must sign in to ask a question. If you do not have an account, you can register here

Featured content

DENODO TRAINING

Ready for more? Great! We offer a comprehensive set of training courses, taught by our technical instructors in small, private groups for getting a full, in-depth guided training in the usage of the Denodo Platform. Check out our training courses.

Training