Artifact Management #

An artifact is the output of the build process ready to deploy — a Docker image, Python package, compiled binary, or tarball archive. Managing artifacts well means: artifacts can be traced to the commit that produced them, stored in a persistent place accessible to all environments, their versions are immutable (can’t change after creation), and unused ones are cleaned from storage. Ansible can automate this entire lifecycle. This article discusses comprehensive artifact management strategies — from immutable labeling, registry selection, distribution with checksum verification, atomic symlink deployment, to signing and automatic cleanup.

Principles of Good Artifacts #

Before diving into implementation, understand the four principles that distinguish professional artifact management from merely “putting files on a server”:

1. Immutable
   Once an artifact is tagged v2.1.0, its contents never change.
   If there's a bugfix, create a new artifact with tag v2.1.1.

2. Traceable
   From the artifact, it can be traced: which code commit made it,
   who ran the build, when, and from which branch.

3. Verified
   Every artifact has a checksum (SHA256) verifiable before
   deployment — ensuring the artifact isn't corrupted or tampered with.

4. Lifecycle-Managed
   Old unneeded artifacts are automatically cleaned up
   to save storage.

These four principles are interrelated. Immutable enables verified (SHA256 checks happen against something that doesn’t change). Traceable supports auditing and debugging when a deployment fails. Lifecycle-managed prevents storage costs from ballooning and ensures old artifacts aren’t misused. Without these four principles, we’ll soon find confusion like: “which artifact is running in production?”, “who deployed this version?”, “when was this release made?”

The Artifact Lifecycle Flow #

Artifacts have a clear lifecycle from source code to cleanup. A flowchart visualization helps us see where each tool and team plays a role:

flowchart LR
    A["Source Code"] -->|"git push"| B["CI Build Job"]
    B -->|"compile & test"| C["Artifact Production"]
    C -->|"tag v2.1.0 + SHA"| D["Registry / Object Storage"]
    D -->|"pull by tag"| E["Deployment Job"]
    E -->|"health check pass"| F["Production Running"]
    D -->|"garbage collect"| G["Cleanup: delete old v2.0.x"]
    F -. "bug found" .-> H["Trigger Rollback"]
    H -->|"re-deploy v2.0.5"| F

Notice: there are two “exit” paths from D (Registry). The first is distribution to production through the deployment job. The second is cleanup for old artifacts. A third path (rollback) can re-pull from the registry to return to a previous version. This is why the registry must be persistent — it must not disappear just because a deployment finished.

Docker images in the registry are the “single source of truth” for deployment. Never rebuild images when deploying — always pull the image already built and tagged in CI. The same image must be used in staging, production, and rollback. If we rebuild per environment, we can’t guarantee staging accurately represents production.

Docker Images as Artifacts #

A Docker registry is the most common way to distribute container-based artifacts. Ansible, through the community.docker modules, can automate building, tagging, and pushing images to registries like Docker Hub, GitHub Container Registry (GHCR), AWS ECR, or self-hosted options like Harbor.

# playbooks/build-artifact.yml
---
- name: Build and push the Docker image artifact
  hosts: localhost
  vars:
    registry: registry.company.com
    image_name: myapp
    version: "{{ version | mandatory }}"
    git_sha: "{{ lookup('pipe', 'git rev-parse --short HEAD') }}"
    build_date: "{{ ansible_date_time.iso8601 }}"

  tasks:
    - name: Login to the registry
      community.docker.docker_login:
        registry_url: "{{ registry }}"
        username: "{{ registry_username }}"
        password: "{{ vault_registry_password }}"
      no_log: true

    - name: Build the image with traceability labels
      community.docker.docker_image:
        name: "{{ registry }}/{{ image_name }}"
        tag: "{{ version }}"
        source: build
        build:
          path: "{{ playbook_dir }}/.."
          labels:
            version: "{{ version }}"
            git.sha: "{{ git_sha }}"
            build.date: "{{ build_date }}"
            build.pipeline: "{{ lookup('env', 'CI_PIPELINE_ID') | default('local') }}"
          args:
            APP_VERSION: "{{ version }}"
        state: present

    - name: Push the image to the registry
      community.docker.docker_image:
        name: "{{ registry }}/{{ image_name }}"
        tag: "{{ version }}"
        push: true
        source: local

    - name: Generate the checksum manifest
      command: >
        docker inspect
        --format="{{ '{{' }}index .RepoDigests 0{{ '}}' }}"
        {{ registry }}/{{ image_name }}:{{ version }}        
      register: image_digest
      changed_when: false

    - name: Save the artifact manifest
      copy:
        content: |
          image={{ registry }}/{{ image_name }}:{{ version }}
          digest={{ image_digest.stdout }}
          version={{ version }}
          git_sha={{ git_sha }}
          build_date={{ build_date }}          
        dest: "{{ playbook_dir }}/artifact-manifest.txt"
      delegate_to: localhost

Docker labels (org.opencontainers.image.version, org.opencontainers.image.revision, org.opencontainers.image.created) are the industry standard for traceability. Tools like dive, crane, or skopeo can read these labels without deep image inspection. With consistent labels, we can run queries like: “which image was made from commit abc123?” just by running docker images --filter "label=org.opencontainers.image.revision=abc123".

ANTI-PATTERN: Committing Binaries to Git vs Artifact Registries #

One of the most common mistakes in early projects is storing build binaries directly in the Git repository. This looks pragmatic (“just clone and you already have the binaries!”), but brings big problems.

# ANTI-PATTERN: commit binaries to Git
git add dist/myapp-2.1.0.tar.gz   # 50MB binary
git add build/myapp-2.1.0.jar    # 80MB binary
git add target/release/myapp     # 120MB binary
git commit -m "build: add v2.1.0 binaries"
git push origin main

The problems are immediately felt: the Git repository balloons quickly. Every fresh clone downloads the entire history, including all now-irrelevant binaries. CI/CD becomes slow because it must check out all binaries just to reach the latest source code. Worse, binaries committed to Git have no build metadata (who built them, from which commit, with which dependency versions), aren’t immutable (anyone can amend and force-push), and are hard to clean up (we’d need filter-branch or BFG Repo-Cleaner to remove them).

# CORRECT: artifacts in the registry, Git only contains source code
# .gitignore
dist/
build/
target/
*.jar
*.tar.gz
*.zip
*.exe

# CI/CD pipeline: build → push to the registry, never to Git
# .github/workflows/build.yml
- name: Build the artifact
  run: |
    docker build -t registry.company.com/myapp:${{ github.sha }} .
    docker push registry.company.com/myapp:${{ github.sha }}    

The Git repository only contains source code, build configuration, and tests. Physical artifacts (binaries, images, packages) live in the registry. Result: repo clones are fast, Git history is clean, artifacts can be purged independently when unused, and traceability is clear — in Git we see code, in the registry we see the artifacts produced by that code.

A Git repository storing binaries can balloon to tens of GB in months. Migration to a registry afterwards is very painful — everyone must re-clone their repos, GitHub/GitLab charges extra storage, and we must rewrite history to remove the binaries. Prevention is far easier: enforce .gitignore and .dockerignore from the start, and educate the team that binaries aren’t part of source code.

Distributing Artifacts to Managed Nodes #

After the build, artifacts must be distributed to target servers. For Docker, this means pulling the image from the registry. For binaries (tarballs, packages), we download from object storage. The pattern used stays the same: fetch the artifact, verify the checksum, extract, and switch the symlink.

# playbooks/distribute-artifact.yml
---
- name: Distribute the artifact to managed nodes
  hosts: appservers
  vars:
    artifact_url: "https://artifacts.company.com/releases/{{ app_name }}/{{ version }}/{{ app_name }}-{{ version }}.tar.gz"
    artifact_checksum: "sha256:{{ artifact_sha256 }}"

  tasks:
    - name: Create the artifact directory
      file:
        path: /opt/releases/{{ version }}
        state: directory
        owner: deployer
        mode: '0755'

    - name: Download the artifact with checksum verification
      get_url:
        url: "{{ artifact_url }}"
        dest: "/opt/releases/{{ version }}/app.tar.gz"
        checksum: "{{ artifact_checksum }}"   # Ansible automatically verifies after download
        owner: deployer
        mode: '0644'
      register: artifact_download

    - name: Extract the artifact
      unarchive:
        src: "/opt/releases/{{ version }}/app.tar.gz"
        dest: "/opt/releases/{{ version }}/"
        remote_src: true
      when: artifact_download.changed

    - name: Atomic symlink switch to the new version
      file:
        src: "/opt/releases/{{ version }}"
        dest: /opt/app/current
        state: link
        force: true   # Overwrite the existing symlink

    - name: Clean up old releases (keep the last 3)
      shell: |
        ls -dt /opt/releases/*/ | tail -n +4 | xargs rm -rf        
      args:
        warn: false
      changed_when: false

The checksum: parameter in the get_url module is an Ansible feature that automatically downloads the SHA256SUM, verifies it, and fails the task if the checksum doesn’t match. This is the first defense against corrupted downloads or artifacts tampered with in transit (man-in-the-middle attacks). Without checksum verification, we have no guarantee that the binary on the server is the binary we uploaded.

The atomic symlink pattern is an elegant way to deploy without downtime. The /opt/app/current symlink is atomically updated after all files are successfully extracted. Running services reading from /opt/app/current will always point to a complete, consistent release directory. There’s no time window where a service reads a half-extracted directory.

Atomic Symlinks for Zero-Downtime Deployment #

sequenceDiagram
    participant CI as "CI Pipeline"
    participant FS as "Filesystem"
    participant App as "Application (running)"
    CI->>FS: "extract to /opt/releases/v2.1.0/"
    CI->>FS: "symlink /opt/app/current → /opt/releases/v2.1.0/"
    Note over FS,App: "The symlink switch is atomic at the filesystem level"
    App->>FS: "read /opt/app/current/file.txt"
    FS-->>App: "file from v2.1.0"
    CI->>App: "SIGHUP reload config"
    App->>FS: "open new file"
    CI->>FS: "delete /opt/releases/v2.0.9/ (cleanup)"

This sequence diagram shows how elegant atomic symlinks are: extract the new directory outside the path the application is currently reading, then switch the symlink — an atomic operation at the filesystem level. A running application never reads a half-finished directory — when the symlink switches, all subsequent requests immediately read from the new directory.

On Linux, symlinks are atomic renames. The command ln -sfn target linkname is a single syscall replacing the symlink target instantly. There’s no window where the symlink points to an invalid or half-finished target. This is what makes the atomic symlink pattern so safe for zero-downtime deployments.

Artifacts in Object Storage (S3/MinIO) #

For non-container artifacts like binaries or packages, object storage is the right choice. S3 (or compatible: MinIO, Wasabi, Backblaze B2) offers 99.999999999% durability at per-GB costs far cheaper than block storage.

# Upload the artifact to S3 after the build
- name: Upload the artifact to S3
  amazon.aws.s3_object:
    bucket: company-artifacts
    object: "releases/{{ app_name }}/{{ version }}/{{ app_name }}-{{ version }}.tar.gz"
    src: "/tmp/build/{{ app_name }}-{{ version }}.tar.gz"
    mode: put
    metadata:
      version: "{{ version }}"
      git_sha: "{{ git_sha }}"
      build_date: "{{ ansible_date_time.iso8601 }}"
    region: ap-southeast-1

# Upload the checksum file
- name: Upload the SHA256 checksum
  amazon.aws.s3_object:
    bucket: company-artifacts
    object: "releases/{{ app_name }}/{{ version }}/{{ app_name }}-{{ version }}.tar.gz.sha256"
    content: "{{ artifact_sha256 }}  {{ app_name }}-{{ version }}.tar.gz"
    mode: put
    region: ap-southeast-1

S3 metadata (x-amz-meta-version, x-amz-meta-git-sha) makes artifacts self-describing. When listing objects in S3, this metadata can be seen without downloading the artifact. This helps operations like “find the artifact for commit abc123” without manually scanning artifact contents.

ANTI-PATTERN: No Versioning vs Semver + Digest #

Without consistent versioning, we’ll soon lose track of: “which artifact is running in production?”, “when did this bug appear?”, or “roll back to which version?”.

# ANTI-PATTERN: latest tag, no versioning
docker tag myapp:latest registry.company.com/myapp:latest
docker push registry.company.com/myapp:latest

# On the server, run:
docker run -d registry.company.com/myapp:latest

The fatal problem: the latest tag always changes. Today latest is v2.0.5, tomorrow it could be v2.0.6 without us realizing. When deploying with the latest tag, we don’t know exactly which version is running. When a bug appears, we can’t “freeze” production for investigation because the latest target keeps moving.

# CORRECT: tag with semver + SHA256 digest
VERSION="2.1.0"
GIT_SHA=$(git rev-parse --short HEAD)
TAG="${VERSION}-${GIT_SHA}"
DIGEST=$(docker build -t registry.company.com/myapp:${TAG} . | tee /dev/stderr | grep -oP 'sha256:[a-f0-9]{64}')

# Push with multiple tags
docker push registry.company.com/myapp:${TAG}
docker push registry.company.com/myapp:${VERSION}  # Major.minor pointer

# On the server, run with an explicit tag + digest
- name: Deploy with an explicit tag
  community.docker.docker_container:
    name: myapp
    image: "registry.company.com/myapp:2.1.0-abc123@{{ image_digest }}"
    state: started

With this pattern, production deployments always have three identifiers: an explicit tag (2.1.0-abc123), a SHA256 digest, and manifest metadata. No ambiguity. Rolling back to v2.0.5 is just swapping the tag, with no guesswork.

Always include the SHA256 digest when deploying to production, not just a tag. Tags can be moved (even though they shouldn’t be), but a SHA256 digest is a cryptographic identifier of the image contents. When we deploy with image: myapp@sha256:abc..., we’re guaranteed to get exactly the same image we verified before. This is the strongest guarantee in the artifact supply chain.

Comparing Artifact Storage #

The artifact storage choice affects cost, features, and operational complexity. This table compares the popular options:

For teams that are full-Docker and already on AWS, ECR is the simplest choice — IAM integration, built-in lifecycle policies, and no egress costs within the region. For polyglot teams (Docker + npm + Maven + PyPI), Artifactory OSS or Nexus OSS provides a single registry for all formats. For teams focused on low cost and full control, MinIO self-hosted is a solid choice.

Storage Backend Decision Tree #

Choosing the right storage backend depends on artifact format, scale, and our ecosystem. This decision tree helps determine the choice:

flowchart TD
    A["Artifact format?"] -->|"Container image"| B["Container Registry"]
    A -->|"Tarball / binary"| C["Object Storage"]
    A -->|"Polyglot packages"| D["Universal Repo Manager"]
    B --> E{"Cloud or Self-host?"}
    E -->|"Cloud-native AWS"| F["ECR"]
    E -->|"Cloud-native GCP"| G["GAR"]
    E -->|"Cloud-native Azure"| H["ACR"]
    E -->|"GitHub project"| I["GHCR"]
    E -->|"Self-host"| J["Harbor / Quay"]
    D --> K{"Scale?"}
    K -->|"Large enterprise"| L["Artifactory Pro"]
    K -->|"Small-medium team"| M["Nexus OSS"]
    C -->|"AWS"| N["S3"]
    C -->|"Self-host"| O["MinIO"]
    C -->|"Multi-cloud"| P["Cloudflare R2 / Wasabi"]

This decision tree isn’t a rigid rule — it’s a heuristic. For small projects, we can use GHCR for everything including tarballs. For enterprises with strict compliance, Artifactory Pro might be the only option meeting audit requirements. The important thing: choose based on real needs, not on “best practices” read without understanding the context.

Artifact Lifecycle Management #

Old artifacts piling up will consume expensive storage. Clean up periodically with a clear retention policy.

# playbooks/cleanup-artifacts.yml
---
- name: Clean up unused Docker artifacts
  hosts: localhost
  vars:
    registry: registry.company.com
    image_name: myapp
    keep_versions: 10    # Keep the last 10 versions

  tasks:
    - name: Fetch all image tags from the registry
      uri:
        url: "https://{{ registry }}/v2/{{ image_name }}/tags/list"
        headers:
          Authorization: "Bearer {{ vault_registry_token }}"
        return_content: true
      register: image_tags
      no_log: true

    - name: Sort the tags and pick which to delete
      set_fact:
        tags_to_delete: >-
          {{ image_tags.json.tags
             | sort
             | list
             | difference(['latest'])
             | list
             | reverse
             | list
             | skip(keep_versions) }}          

    - name: Delete the old tags from the registry
      uri:
        url: "https://{{ registry }}/v2/{{ image_name }}/manifests/{{ item }}"
        method: DELETE
        headers:
          Authorization: "Bearer {{ vault_registry_token }}"
        status_code: [202, 404]
      loop: "{{ tags_to_delete }}"
      loop_control:
        label: "Deleting tag: {{ item }}"

Rules of thumb: keep the last N versions in the registry (N=10 is a good default), keep everything currently running in production, and don’t keep unused ones longer than 90 days. For cloud container registries like ECR, use the built-in lifecycle policy which is more reliable than a custom playbook.

Artifact Signing with Cosign #

In the era of supply chain attacks (compromised CI, dangerous dependencies), verifying that an artifact was truly made by our team — not an outsider impersonating us — becomes very important. Cosign (part of Sigstore) enables cryptographic signing for container images.

# Sign the image after the build
COSIGN_EXPERIMENTAL=1 cosign sign registry.company.com/myapp:v2.1.0-abc123

# Verify the signature during deployment
cosign verify registry.company.com/myapp:v2.1.0-abc123 \
  --certificate-identity-regexp "https://github.com/company/myapp" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com"

# playbooks/deploy-with-verification.yml
- name: Verify the artifact signature before deploying
  hosts: appservers
  tasks:
    - name: Pull the image
      community.docker.docker_image:
        name: "registry.company.com/myapp:{{ deploy_version }}"
        source: pull

    - name: Verify the signature with cosign
      command: >
        cosign verify registry.company.com/myapp:{{ deploy_version }}
        --certificate-identity-regexp "https://github.com/company/myapp"
        --certificate-oidc-issuer "https://token.actions.githubusercontent.com"        
      register: signature_check
      failed_when: signature_check.rc != 0
      changed_when: false

    - name: Deploy only if the signature is valid
      community.docker.docker_container:
        name: myapp
        image: "registry.company.com/myapp:{{ deploy_version }}"
        state: started
      when: signature_check.rc == 0

Signing adds a security layer: even if the registry is compromised and an attacker swaps the image, signature verification fails and the deployment is automatically rejected. This is cheap defense-in-depth for high-value deployments.

Cosign is free and open source, part of the Sigstore project sponsored by the Linux Foundation. For most teams, initial signing + verification setup takes less than 1 working day, but its value in supply chain security is enormous. Start with teams that have compliance requirements (finance, healthcare), or that release applications publicly.

Summary #

• Artifacts must be immutable — once created with a specific tag, their contents never change. A bugfix = a new artifact with a new tag.
• Add labels/metadata to artifacts at build time: version, git SHA, build date, pipeline ID — enabling full traceability from artifact to source code.
• Use checksum: in get_url to verify artifact integrity after download — detects corrupted downloads or tampered artifacts.
• Atomic symlinks (/opt/app/current → /opt/releases/v2.1.0) are an elegant deployment pattern — version switching is atomic, rollback is as easy as swapping the symlink.
• Store artifact checksums in a place separate from the artifact itself — ideally in different object storage or cryptographically signed.
• Lifecycle management: automatically clean up old artifacts — keep only the last N versions to control storage costs.
• Don’t commit binaries to Git — binaries balloon the repository and have no build metadata. Use a registry, not Git, for artifacts.
• Tag with semver + git SHA, plus a SHA256 digest when deploying to production. Avoid the latest tag in production.
• Choose the registry based on the ecosystem: ECR for AWS-native, GHCR for GitHub projects, Artifactory/Nexus for polyglot, S3/MinIO for large tarballs.
• Sign artifacts with cosign for supply chain security — verify the signature in the deployment job before starting the container.
• S3 metadata (x-amz-meta-version, x-amz-meta-git-sha) makes artifacts self-describing for querying without downloads.
• Retention policies must be explicit: keep the last N versions, keep everything running in production, clean up after 90 days.

← Previous: Rollback Strategy Next: Notification & Reporting →