...
This is a generic project to create Docker images to run Yale applications. It should be possible to build an image on the developer’s machine for testing, then check the project into git.yale.edu and run the Jenkins job to built the image for the Yale container runtime environment.The requirements are:
Requirements
The Jenkins job builds an image using just the Dockerfile with no custom parameters. Therefore, the Dockerfile all by itself must build and image for the Yale environment, while files in this project override build args and provide additional tags only on the developer’s desktop.
The exact method of doing the equivalent of a “docker build” on the desktop is defined by the developer in a profile. It is possible to use a local docker command, or a remote ssh docker command, or a substitute for Docker like Podman.
Each user provides a profile that defines the Sandbox environment (the local substitute for Artifactory and Harbor for example) and the personal choices (the use of a local docker, remote docker, or substitute).
A script file is included in every project and is run only on the developer desktop to do that type of build. The project specific script file includes project specific parameters while the generic code shared by all projects built by this developer are in the developer’s profile script.
The developer may run a copy of Harbor on his machine, or may choose to scan images and share images across VMs using a different method. All images are tagged on the developer machine with the same prefixes they will have in the Yale Harbor server.
The general approach is:
The Dockerfile has ARG statements assigning default values appropriate for the Yale Jenkins build of the image to store it in the Yale Harbor server. The Jenkins job just does a plain “docker build” on the project directory with the plain Dockerfile with all its default values.
...
project to define any project specific overrides. For example, the Dockerfile may contain a reference to a Released artifact version such as 1.0.19 while the script may contain the developer’s override version 1.0.20-SANDBOX.
ARGs
Whenever there is a value in the Dockerfile that should be different in the Sandbox from the Jenkins build value, create an ARG and set the default to match the value in Jenkins. Then you can override this ARG value by adding a --build-arg parameter to the “docker build”.
Examples:
ARG ARTIFACT_SERVER=https://repository.its.yale.edu/artifactory/libs-releases-local
ARG YaleArtifactVersion=1.0.48
he ARG ARTIFACT_SERVER would typically not be in the build.json file because it is a permanent part of the developer desktop configuration. If the user provides a local artifact server URL, it will be set in the user’s $HOME\sandboxProfile.ps1 file:
$SANDBOX_ARTIFACT_SERVER="http://repository.yale.sandbox/artifactory"
Putting the project specific --build-arg overrides in the script means that every script file is unnecessarily unique. Instead, the override values are stored in a build.json file in the project that is only used on the developer desktop. Minimally it contains values that are coded into the Jenkins job when someone builds it. Minimally this includes the image tags which are not part of the Dockerfile but must be provided as parameters of the “docker build” command itself and which are different for each project.
In cases where the developer is building an artifact on the desktop and then including it into the image, the developer will want to replace the URL for the Artifactory server with a URL that fetches the newly built artifact from the local Maven repository. Since this will be the same for all projects, it is part of the profile.
While the user can do a “docker.exe build” on the desktop, there are alternate approaches to do a “wsl docker build” (run it in WSL), or “ssh userid@vm docker build” (run it on a VM), or “multipass exec vm docker build” (special version of ssh for Multipass), or “wsl podman build” (run podman replacement for docker in WSL), or “nerdctl build” (Rancher desktop). There are lots of ways to build an image. The profile provides a generic command which can be rewritten to actually use any of the above methodswhich is read by code in the profile script and automatically generates the override parameters.
| Code Block |
|---|
{
"new_image_tag": "iiq:8.1p1-yale-1.0.48",
"build_arg": {
"YaleArtifactVersion": "1.0.48-SNAPSHOT",
"SailpointVersion": "8.1p1"
}
} |
The only parameter required in the build.json file is the “new_image_tag” which is unique for each project and cannot be specified in the Dockerfile (but must be a -t on the “docker build” command).
The project specific build.ps1 script defines variables, includes the Profile script, and then calls the build-image command generated by the profile script. Most build.ps1 files are minimal and identical with this common content.
| Code Block |
|---|
if (test-path $HOME\SandboxProfile.ps1) {
. $HOME\SandboxProfile.ps1
build-image
} else {
Write-Host "You have no $HOME/SandboxProfile.ps1 file. Get if from https://git.yale.edu/gilbert/SandboxImageBuild"
} |
Remember, the build.ps1 and build.json files are checked into the Git project but are not used by the Jenkins build. While they contain information for you, they are mostly intended to help the next developers who works on this project, providing an example of things that they may want to customize.
Remove Bad Optimization
Docker has two default behaviors that over optimize the build process and generate mistakes. These should be suppressed by adding parameters to turn them off.
--no-cache disables a Docker optimization which caches the results of each line in the Dockerfile and in a subsequent run of the same file (or any file that starts out the same), uses the saved results from the previous run instead of re-executing the statement. While this may save a few seconds, it produces unexpected behavior and is counter intuitive. The problem occurs with statements that get files or packages from the network. The Dockerfile statement is expected to get the most current files and not some files downloaded and saved months ago. This can happen with all sorts of statements, but the one example that makes the problem clear is the commonly used expression to apply the most recent bugfixes to the system:
“RUN apt update && apt upgrade -Y”
Since this statement itself doesn’t change, the default behavior of Docker is to download fixes the first time you build the image, and then never apply any new patches to any future build. Specifying --no-cache runs the entire Dockerfile fresh every time.
--pull is a special version of --no-cache that applies to the image in the FROM statement. A DockerHub image like “tomcat:latest” is periodically updated with the latest version and fixes. A more reasonable choice for a base image specifies a particular version of Java and Tomcat, but then allows the latest dot release with bug fixes. The default Docker behavior is to download a base image and then never check DockerHub for a newer version of the image as long as the tag doesn’t change. Adding --pull causes Docker to check for a more recent version of the base image and pull one if it is available.
Base Choice
The official Tomcat images offer versions of OpenJDK from various sources (Amazon, Eclipse, etc.) and various releases of Debian and Ubuntu.
Plain OpenJDK and the most recent release of Debian seem to be good choices, and Debian has a special package repository that is quickly updated whenever a vulnerability is announced.