Creating Your First Image
Base Image
Let’s start with a simple base image. This image will pull an opensuse docker image and update the packages. To setup your environment do something like this.
TUTORIAL_DIR=/tmp/velocity
mkdir -p $TUTORIAL_DIR/images
export VELOCITY_IMAGE_PATH=$TUTORIAL_DIR/images
export VELOCITY_BUILD_DIR=$TUTORIAL_DIR/build
export VELOCITY_DISTRO=opensuse
# all subsequent commands in this tutorial were run from the TUTORIAL_DIR, but they don't have to be
Note
You will also need Apptainer installed on your system.
Next create a directory in the images directory called opensuse. In this directory create a file called
specs.yaml and a directory called templates with
a file named default.vtmp. Your image directory and files should now look like this.
opensuse
├── specs.yaml
└── templates
└── default.vtmp
versions:
- spec: 15.4
when: distro=opensuse
@from
docker.io/opensuse/leap:{{ __version__ }}
@run
zypper --non-interactive refresh
zypper --non-interactive update
zypper clean --all
Now if you run velocity avail you should get the following.
$ velocity avail
==> opensuse
15.4
Now build the image.
$ velocity build opensuse
==> Build Order:
opensuse@15.4-a1913cd
==> a1913cd: BUILD opensuse@15.4 ...
==> a1913cd: GENERATING SCRIPT ...
==> a1913cd: BUILDING ...
==> a1913cd: IMAGE /tmp/velocity/build/opensuse-15.4-a1913cd/a1913cd.sif (opensuse@15.4) BUILT [0:01:03]
==> BUILT: /tmp/velocity/opensuse-15.4__frontier-opensuse.sif
If you wish to see more output you can add the -v flag to the build command.
Adding Different Versions
So now we have a base opensuse image. That’s great but before we move on let’s make some different versions of the image
so that we have more options for building later. Edit the opensuse specs.yaml and add some versions.
versions:
- spec:
- 15.4
- 15.5
- 15.6
when: distro=opensuse
$ velocity avail
==> opensuse
15.4
15.5
15.6
Specifying Version
When building an image Velocity will default to the latest image. To specify a version use <image>@<version> e.g.
opensuse@15.6. Versions take the form <major>.<minor>.<patch>-<suffix>. You can also specify greater than, less
than, and in-between via <image>@<version>:, <image>@:<version> and <image>@<version>:<version> respectively.
Hello World!
Now let’s get a little more complicated. Let’s create an image that runs a python script which prints Hello, World!. You
can give it whatever version you want:
opensuse
├── specs.yaml
└── templates
└── default.vtmp
hello-world
├── files
│ └── hello_world.py
├── specs.yaml
└── templates
└── default.vtmp
Notice that now there is a new folder called files with a python script in it.
#!/usr/bin/env python3
print("Hello, World!")
versions:
- spec: 1.0
dependencies:
- spec: opensuse
when: distro=opensuse
files:
- name: hello_world.py
@from
{{ __base__ }}
@copy
hello_world.py /hello_world
@run
zypper --non-interactive install python3
chmod +x /hello_world
@entry
/hello_world
$ velocity avail
==> hello-world
1.0
==> opensuse
15.4
15.5
15.6
$ velocity build hello-world -v
==> Build Order:
opensuse@15.6-59edd44
hello-world@1.0-a167014
==> 59edd44: BUILD opensuse@15.6 ...
==> 59edd44: GENERATING SCRIPT ...
==> 59edd44: BUILDING ...
==> 59edd44: IMAGE /tmp/velocity/build/opensuse-15.6-59edd44/59edd44.sif (opensuse@15.6) BUILT [0:00:36]
==> a167014: BUILD hello-world@1.0 ...
==> a167014: COPYING FILES ...
==> a167014: GENERATING SCRIPT ...
==> a167014: BUILDING ...
==> a167014: IMAGE /tmp/velocity/build/hello-world-1.0-a167014/a167014.sif (hello-world@1.0) BUILT [0:00:27]
==> BUILT: /tmp/velocity/hello-world-1.0_opensuse-15.6__frontier-opensuse.sif
Our hello-world image has been built!
$ ls -al
total 207020
drwxr-xr-x 4 xxx xxx 120 Sep 25 10:44 .
drwxrwxrwt 33550 root root 761680 Sep 25 10:45 ..
drwxr-xr-x 5 xxx xxx 100 Sep 25 10:44 build
-rwxr-xr-x 1 xxx xxx 167243776 Sep 25 10:44 hello-world-1.0_opensuse-15.6__frontier-opensuse.sif
drwxr-xr-x 4 xxx xxx 80 Sep 25 10:42 images
-rwxr-xr-x 1 xxx xxx 44744704 Sep 25 10:39 opensuse-15.4__frontier-opensuse.sif
Now you can run the image!
$ apptainer run hello-world-*-opensuse.sif # replace * with the specifics of your build
Hello, World!
OLCF Images
Let’s extend what we have done so far and explore some more features of Velocity using a base set of image definitions provided at https://github.com/olcf/velocity-images. Clone the repository and run:
Note
The opensuse image in the git repository will override the opensuse image you just created because velocity
selects conflicting images by their order in VELOCITY_IMAGE_PATH.
export VELOCITY_IMAGE_PATH=<path to the cloned repo>:$VELOCITY_IMAGE_PATH
Let’s check what images are available now.
Note
Due to updates to https://github.com/olcf/velocity-images the output shown below may be different for you.
$ velocity avail
==> gcc
12.3.0
13.2.0
14.1.0
==> llvm
17.0.0
17.0.6
==> mpich
3.4.3
==> opensuse
15.4
15.5
15.6
==> rocm
5.7.1
6.0.1
6.1.3
If you were to look at the contents of https://github.com/olcf/velocity-images you would notice that there is a
folder in it defining an ubuntu image. Why does that image not show up? At the beginning of this tutorial
we set export VELOCITY_DISTRO=opensuse. In the ubuntu specs.yaml file you would see:
versions:
- spec:
- 20.04
- 22.04
- 24.04
when: distro=ubuntu
The when: distro=ubuntu means that the defined versions will not show up unless the distro is set to ubuntu.
Run the following command and compare the difference.
$ velocity -d ubuntu avail
==> gcc
12.3.0
13.2.0
14.1.0
==> hello-world
1.0
==> llvm
17.0.0
17.0.6
==> mpich
3.4.3
==> rocm
5.7.1
6.0.1
6.1.3
==> ubuntu
20.04
22.04
24.04
Important
This is important because it keeps us from trying to build a container with two distros, but it may catch you off guard by hiding images you thought you had defined.
Now let try building our hello-world image on an ubuntu base. In the current state the build will fail but let’s
run it anyway and trouble shoot it.
$ velocity -d ubuntu build hello-world
==> Build Order:
hello-world@1.0-7562a9e
==> 7562a9e: BUILD hello-world@1.0 ...
==> 7562a9e: COPYING FILES ...
==> 7562a9e: GENERATING SCRIPT ...
Traceback (most recent call last):
File "/ccs/home/xxx/.conda/envs/main_x86_64/lib/python3.11/site-packages/velocity/_backends.py", line 131, in generate_script
if len(sections["@from"]) != 1:
~~~~~~~~^^^^^^^^^
KeyError: '@from'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/ccs/home/xxx/.conda/envs/main_x86_64/lib/python3.11/site-packages/velocity/__main__.py", line 111, in <module>
builder.build()
File "/ccs/home/xxx/.conda/envs/main_x86_64/lib/python3.11/site-packages/velocity/_build.py", line 128, in build
self._build_image(u, last, name)
File "/ccs/home/xxx/.conda/envs/main_x86_64/lib/python3.11/site-packages/velocity/_build.py", line 223, in _build_image
script = self.backend_engine.generate_script(unit, script_variables)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ccs/home/xxx/.conda/envs/main_x86_64/lib/python3.11/site-packages/velocity/_backends.py", line 140, in generate_script
raise TemplateSyntaxError("You must have an @from section in your template!")
velocity._exceptions.TemplateSyntaxError: You must have an @from section in your template!
We see that an error occurred in the GENERATING SCRIPT section. But if we look under ==> Build Order at the top
we will notice the real cause. The ubuntu image is not being built. This causes the error in script generation
because in our default.vtmp for hello-world we have {{ __base__ }} defined in our @from section which
looks for a previous image to build on. Let’s edit our hello-world specs.yaml. It should look like this.
versions:
- spec: 1.0
dependencies:
- spec: opensuse
when: distro=opensuse
- spec: ubuntu
when: distro=ubuntu
files:
- name: hello_world.py
Under the dependencies section we added the ubuntu image, but we specified it should only be a dependency when
our distro is set to ubuntu. You can test that Velocity is now adding ubuntu as a dependency by running:
$ velocity -d ubuntu spec hello-world
> hello-world@1.0-f6bfef8
^ubuntu@24.04-ce71495
Now we can try to build again, but it will fail with a new error.
$ velocity -d ubuntu build hello-world
==> Build Order:
ubuntu@24.04-043b176
hello-world@1.0-9dcbb36
==> 043b176: BUILD ubuntu@24.04 ...
==> 043b176: GENERATING SCRIPT ...
==> 043b176: BUILDING ...
==> 043b176: IMAGE /tmp/velocity/build/ubuntu-24.04-043b176/043b176.sif (ubuntu@24.04) BUILT [0:00:11]
==> 9dcbb36: BUILD hello-world@1.0 ...
==> 9dcbb36: COPYING FILES ...
==> 9dcbb36: GENERATING SCRIPT ...
==> 9dcbb36: BUILDING ...
INFO: User not listed in /etc/subuid, trying root-mapped namespace
INFO: The %post section will be run under fakeroot
INFO: Starting build...
INFO: Verifying bootstrap image /tmp/velocity/build/ubuntu-24.04-043b176/043b176.sif
INFO: Copying hello_world.py to /hello_world
INFO: Running post scriptlet
+ zypper --non-interactive install python3
/.post.script: 1: zypper: not found
FATAL: While performing build: while running engine: exit status 127
Velocity prints out the error from the build zypper: not found. Lets look back at the vtmp
script we wrote for hello-world. Under the @run section we had:
@run
zypper --non-interactive install python3
chmod +x /hello_world
We used zypper to install python because it is not installed in the opensuse docker image by default. We need to edit this
script to support ubuntu. Change the @run section to:
@run
?? distro=opensuse |> zypper --non-interactive install python3 ??
?? distro=ubuntu |> apt -y install python3 ??
chmod +x /hello_world
Now we can test by doing a verbose dry-run for opensuse and ubuntu.
$ velocity build hello-world -dv
==> Build Order:
opensuse@15.6-90ac66d
hello-world@1.0-5fa2515
==> 90ac66d: BUILD opensuse@15.6 --DRY-RUN ...
==> 90ac66d: GENERATING SCRIPT ...
SCRIPT: /tmp/velocity/build/opensuse-15.6-90ac66d/script
Bootstrap: docker
From: docker.io/opensuse/leap:15.6
%post
zypper --non-interactive refresh
zypper --non-interactive update
zypper clean --all
==> 90ac66d: BUILDING ...
#!/usr/bin/env bash
apptainer build --disable-cache /tmp/velocity/build/opensuse-15.6-90ac66d/90ac66d.sif /tmp/velocity/build/opensuse-15.6-90ac66d/script;
==> 90ac66d: IMAGE /tmp/velocity/build/opensuse-15.6-90ac66d/90ac66d.sif (opensuse@15.6) BUILT [0:00:00]
==> 5fa2515: BUILD hello-world@1.0 --DRY-RUN ...
==> 5fa2515: COPYING FILES ...
FILE: /tmp/velocity/images/hello-world/files/hello_world.py -> /tmp/velocity/build/hello-world-1.0-5fa2515/hello_world.py
==> 5fa2515: GENERATING SCRIPT ...
SCRIPT: /tmp/velocity/build/hello-world-1.0-5fa2515/script
Bootstrap: localimage
From: /tmp/velocity/build/opensuse-15.6-90ac66d/90ac66d.sif
%files
hello_world.py /hello_world
%post
zypper --non-interactive install python3
chmod +x /hello_world
%runscript
/hello_world
==> 5fa2515: BUILDING ...
#!/usr/bin/env bash
apptainer build --disable-cache /tmp/velocity/build/hello-world-1.0-5fa2515/5fa2515.sif /tmp/velocity/build/hello-world-1.0-5fa2515/script;
==> 5fa2515: IMAGE /tmp/velocity/build/hello-world-1.0-5fa2515/5fa2515.sif (hello-world@1.0) BUILT [0:00:00]
==> BUILT: /tmp/velocity/hello-world-1.0_opensuse-15.6__frontier-opensuse.sif
$ velocity -d ubuntu build hello-world -dv
==> Build Order:
ubuntu@24.04-ce71495
hello-world@1.0-b03891d
==> ce71495: BUILD ubuntu@24.04 --DRY-RUN ...
==> ce71495: GENERATING SCRIPT ...
SCRIPT: /tmp/velocity/build/ubuntu-24.04-ce71495/script
Bootstrap: docker
From: docker.io/ubuntu:24.04
%post
export DEBIAN_FRONTEND="noninteractive"
apt -y update
apt -y upgrade
apt clean
%environment
export DEBIAN_FRONTEND="noninteractive"
==> ce71495: BUILDING ...
#!/usr/bin/env bash
apptainer build --disable-cache /tmp/velocity/build/ubuntu-24.04-ce71495/ce71495.sif /tmp/velocity/build/ubuntu-24.04-ce71495/script;
==> ce71495: IMAGE /tmp/velocity/build/ubuntu-24.04-ce71495/ce71495.sif (ubuntu@24.04) BUILT [0:00:00]
==> b03891d: BUILD hello-world@1.0 --DRY-RUN ...
==> b03891d: COPYING FILES ...
FILE: /tmp/velocity/images/hello-world/files/hello_world.py -> /tmp/velocity/build/hello-world-1.0-b03891d/hello_world.py
==> b03891d: GENERATING SCRIPT ...
SCRIPT: /tmp/velocity/build/hello-world-1.0-b03891d/script
Bootstrap: localimage
From: /tmp/velocity/build/ubuntu-24.04-ce71495/ce71495.sif
%files
hello_world.py /hello_world
%post
apt -y install python3
chmod +x /hello_world
%runscript
/hello_world
==> b03891d: BUILDING ...
#!/usr/bin/env bash
apptainer build --disable-cache /tmp/velocity/build/hello-world-1.0-b03891d/b03891d.sif /tmp/velocity/build/hello-world-1.0-b03891d/script;
==> b03891d: IMAGE /tmp/velocity/build/hello-world-1.0-b03891d/b03891d.sif (hello-world@1.0) BUILT [0:00:00]
==> BUILT: /tmp/velocity/hello-world-1.0_ubuntu-24.04__x86_64-ubuntu.sif
We can see that each build uses the correct command to install python. Now we can actually build the image.
$ velocity -d ubuntu build hello-world
==> Build Order:
ubuntu@24.04-ce71495
hello-world@1.0-b03891d
==> ce71495: BUILD ubuntu@24.04 ...
==> ce71495: GENERATING SCRIPT ...
==> ce71495: BUILDING ...
==> ce71495: IMAGE /tmp/velocity/build/ubuntu-24.04-ce71495/ce71495.sif (ubuntu@24.04) BUILT [0:00:00]
==> b03891d: BUILD hello-world@1.0 ...
==> b03891d: COPYING FILES ...
==> b03891d: GENERATING SCRIPT ...
==> b03891d: BUILDING ...
==> b03891d: IMAGE /tmp/velocity/build/hello-world-1.0-b03891d/b03891d.sif (hello-world@1.0) BUILT [0:00:09]
==> BUILT: /tmp/velocity/hello-world-1.0_ubuntu-24.04__x86_64-ubuntu.sif
This example is a demonstration of one of the major strengths of Velocity. The hello-world image can now be built
on any version of opensuse or ubuntu, but instead of having a separate script for each combination of version and distro we have
just three. One for opensuse, one for ubuntu and one for hello-world. This may not seem like a big win for an
example like hello-world; however, this becomes a big win for images like the gcc image in
https://github.com/olcf/velocity-images. If you look at the gcc image default.vtmp script you will see that it can
build practically any version of gcc on ubuntu, opensuse and rockylinux.
The last thing we need to look at for this tutorial is Velocity’s support for multiple container backends. Let’s look at
a dry-run example of the opensuse image that we have been building with apptainer.
$ velocity build opensuse -vd
==> Build Order:
opensuse@15.6-01205e8
==> 01205e8: BUILD opensuse@15.6 --DRY-RUN ...
==> 01205e8: GENERATING SCRIPT ...
SCRIPT: /tmp/xxx/velocity/opensuse-15.6-01205e8/script
Bootstrap: docker
From: docker.io/opensuse/leap:15.6
%post
zypper --non-interactive refresh
zypper --non-interactive update
zypper clean --all
==> 01205e8: BUILDING ...
#!/usr/bin/env bash
apptainer build --disable-cache /tmp/xxx/velocity/opensuse-15.6-01205e8/01205e8.sif /tmp/xxx/velocity/opensuse-15.6-01205e8/script;
==> 01205e8: IMAGE /tmp/xxx/velocity/opensuse-15.6-01205e8/01205e8.sif (opensuse@15.6) BUILT [0:00:00]
==> BUILT: /tmp/opensuse-15.6__x86_64-opensuse.sif
Next let’s look at the same thing but with the backend set to podman.
Warning
If Podman is not installed this will fail. Conversely, if you are on a system that does not have Apptainer
installed, build commands using Apptainer will fail.
$ velocity -b podman build opensuse -vd
==> Build Order:
opensuse@15.6-dd91fe3
==> dd91fe3: BUILD opensuse@15.6 --DRY-RUN ...
==> dd91fe3: GENERATING SCRIPT ...
SCRIPT: /tmp/xxx/velocity/opensuse-15.6-dd91fe3/script
FROM docker.io/opensuse/leap:15.6
RUN zypper --non-interactive refresh && \
zypper --non-interactive update && \
zypper clean --all
==> dd91fe3: BUILDING ...
#!/usr/bin/env bash
podman build -f /tmp/xxx/velocity/opensuse-15.6-dd91fe3/script -t localhost/dd91fe3:latest .;
==> dd91fe3: IMAGE localhost/dd91fe3:latest (opensuse@15.6) BUILT [0:00:00]
==> BUILT: localhost/opensuse-15.6__x86_64-opensuse:latest
As you can see Velocity automatically renders the scripts to the correct format and changes the build commands to use Podman. Amazing!!!
One last note about debugging builds with Velocity. We set VELOCITY_BUILD_DIR at the beginning of this tutorial.
If you look in the directory that it points to you will find a folder for each image that was built. Each folder
contains the rendered script, build log, files generated by the build (e.g SIF files), build commands, and any files
that were needed for the build (e.g. hello_world.py). All of these can be very useful for debugging a build.
One very helpful feature is that the build of an image can be run manually by running the build script in a folder.
.
├── hello-world-1.0-0ff1ff7
│ ├── 0ff1ff7.sif
│ ├── build
│ ├── hello_world.py
│ ├── log
│ └── script
...