# Containerization & Docker / Apptainer This project is built following the [12-Factor App methodology](https://12factor.net/), meaning its configuration is strictly separated from the code. This makes it incredibly easy to containerize and run using Docker or Apptainer. ## Prerequisites Ensure you have Docker or Apptainer installed and running on your system. ## 1. Building the Container (Docker) The project includes an optimized `Dockerfile` using `uv` at the root of the repository. To build the container image, run the following command from the project root: ```bash docker build -t hello-world-job . ``` This command will: 1. Use a lightweight Python 3.13 base image. 2. Copy the blazing-fast `uv` package manager into the container. 3. Install the `mypkgs` library and its dependencies directly into the system Python. 4. Set up a secure, non-root user. ## 2. Running with Docker Because the application relies on environment variables for its configuration (like the config file path and the output directory), we use Docker's `--env-file` flag to inject these variables at runtime. ### Basic Run To run the container using your local `.env` file: ```bash docker run --env-file .env hello-world-job ``` ```{warning} Ephemeral File Systems By default, Docker containers have an ephemeral file system. If the script successfully runs and writes data to the `OUTPUT_DIR` inside the container, that data will be lost as soon as the container shuts down! ``` ### Running with Volume Mounts (Recommended) To save the output files generated by the script to your local host machine, you must map a local directory to the container's output directory using a volume mount (`-v`). Assuming your `.env` specifies `OUTPUT_DIR=/app/results` (which is the working directory inside the container), you can run: ```bash docker run \ --env-file .env \ -v $(pwd)/results:/app/results \ hello-world-job ``` --- ## 3. Running with Apptainer (HPC Environments) If you are running this project on a university cluster or High-Performance Computing (HPC) environment, you will likely use Apptainer instead of Docker. Fortunately, Apptainer can natively run Docker images. ### Pulling or Building the Image Apptainer uses `.sif` (Singularity Image Format) files. You can generate this file in two ways: **Option A: Build from your local Docker daemon** (if you just ran `docker build` locally): ```bash apptainer build hello-world-job.sif docker-daemon://hello-world-job:latest ``` **Option B: Pull directly from the GitHub Container Registry** (if you are using the CI/CD workflow): ```bash apptainer pull hello-world-job.sif docker://ghcr.io/j-i-l/pythonproject:latest ``` ### Running the Apptainer Image Apptainer supports environment variable files and volume binding just like Docker, though the flags are slightly different (`--bind` instead of `-v`). ```{note} Strict Read-Only File Systems Unlike Docker, Apptainer containers are **strictly read-only** by default. This makes volume binding absolutely mandatory if your script needs to write to the `OUTPUT_DIR`. ``` To run the job via Apptainer, passing your `.env` file and binding your local results folder: ```bash apptainer run \ --env-file .env \ --bind $(pwd)/results:/app/results \ hello-world-job.sif ``` ### Note on Apptainer Environment Variables If you prefer not to use an `--env-file`, Apptainer allows you to pass environment variables directly from your host shell by prefixing them with `APPTAINERENV_`. For example: ```bash export APPTAINERENV_CONFIG_PATH=/app/config/settings.json export APPTAINERENV_OUTPUT_DIR=/app/results apptainer run --bind $(pwd)/results:/app/results hello-world-job.sif ``` ## The `.dockerignore` File To keep your Docker images small and secure, it is highly recommended to include a `.dockerignore` file in the root of your repository. This file works exactly like a `.gitignore` file, preventing unnecessary local files—such as your `.venv` directory, `__pycache__`, or local `results` folders—from being copied into the container during the `COPY . /app` step. ```{important} Do NOT ignore the `.git` folder! Because this project uses dynamic versioning (via `hatch-vcs` or `setuptools_scm`), the package manager needs to read the repository's Git history to determine the correct version number during installation. If you add `.git` to your `.dockerignore` file, the build process will crash with a `setuptools_scm` error stating it cannot detect the version. Ensure your `.git` folder is available to the Docker build context. ```