Containerization & Docker / Apptainer#

This project is built following the 12-Factor App methodology, meaning its configuration is strictly separated from the code. This makes it incredibly easy to containerize and run using Docker or Apptainer.

Prerequisites#

Ensure you have Docker or Apptainer installed and running on your system.

1. Building the Container (Docker)#

The project includes an optimized Dockerfile using uv at the root of the repository. To build the container image, run the following command from the project root:

docker build -t hello-world-job .

This command will:

Use a lightweight Python 3.13 base image.
Copy the blazing-fast uv package manager into the container.
Install the mypkgs library and its dependencies directly into the system Python.
Set up a secure, non-root user.

2. Running with Docker#

Because the application relies on environment variables for its configuration (like the config file path and the output directory), we use Docker’s --env-file flag to inject these variables at runtime.

Basic Run#

To run the container using your local .env file:

docker run --env-file .env hello-world-job

Warning

Ephemeral File Systems By default, Docker containers have an ephemeral file system. If the script successfully runs and writes data to the OUTPUT_DIR inside the container, that data will be lost as soon as the container shuts down!

Running with Volume Mounts (Recommended)#

To save the output files generated by the script to your local host machine, you must map a local directory to the container’s output directory using a volume mount (-v).

Assuming your .env specifies OUTPUT_DIR=/app/results (which is the working directory inside the container), you can run:

docker run \
  --env-file .env \
  -v $(pwd)/results:/app/results \
  hello-world-job

3. Running with Apptainer (HPC Environments)#

If you are running this project on a university cluster or High-Performance Computing (HPC) environment, you will likely use Apptainer instead of Docker. Fortunately, Apptainer can natively run Docker images.

Pulling or Building the Image#

Apptainer uses .sif (Singularity Image Format) files. You can generate this file in two ways:

Option A: Build from your local Docker daemon (if you just ran docker build locally):

apptainer build hello-world-job.sif docker-daemon://hello-world-job:latest

Option B: Pull directly from the GitHub Container Registry (if you are using the CI/CD workflow):

apptainer pull hello-world-job.sif docker://ghcr.io/j-i-l/pythonproject:latest

Running the Apptainer Image#

Apptainer supports environment variable files and volume binding just like Docker, though the flags are slightly different (--bind instead of -v).

Note

Strict Read-Only File Systems Unlike Docker, Apptainer containers are strictly read-only by default. This makes volume binding absolutely mandatory if your script needs to write to the OUTPUT_DIR.

To run the job via Apptainer, passing your .env file and binding your local results folder:

apptainer run \
  --env-file .env \
  --bind $(pwd)/results:/app/results \
  hello-world-job.sif

Note on Apptainer Environment Variables#

If you prefer not to use an --env-file, Apptainer allows you to pass environment variables directly from your host shell by prefixing them with APPTAINERENV_.

For example:

export APPTAINERENV_CONFIG_PATH=/app/config/settings.json
export APPTAINERENV_OUTPUT_DIR=/app/results
apptainer run --bind $(pwd)/results:/app/results hello-world-job.sif

The `.dockerignore` File#

To keep your Docker images small and secure, it is highly recommended to include a .dockerignore file in the root of your repository. This file works exactly like a .gitignore file, preventing unnecessary local files—such as your .venv directory, __pycache__, or local results folders—from being copied into the container during the COPY . /app step.

```{important} Do NOT ignore the .git folder! Because this project uses dynamic versioning (via hatch-vcs or setuptools_scm), the package manager needs to read the repository’s Git history to determine the correct version number during installation.

If you add .git to your .dockerignore file, the build process will crash with a setuptools_scm error stating it cannot detect the version. Ensure your .git folder is available to the Docker build context.