Testing #

In the modern Infrastructure as Code (IaC) paradigm, our infrastructure code must be treated exactly like any other software application code. We must not let playbooks be deployed directly to production servers without going through rigorous testing. Changing firewall configurations, restarting database clusters, or updating library packages without automated testing guarantees is a huge technical debt. Small errors in variable writing or OS version mismatches can easily cause total service failure in production environments. This article comprehensively discusses Ansible automated testing strategies to ensure every code change is safe before being applied.

Why Infrastructure Code Testing Strategies Matter #

Testing infrastructure automation code has its own challenges because it directly interacts with target OS system states. If we test web application code, we can easily mock the database. However, to test Ansible code, we must ensure it truly can modify Linux configurations, configure users, and validate services running on the real file system.

For easier understanding, we can divide Ansible testing levels into three main layers (testing pyramid):

1. Syntax and Style Layer (Static Analysis): Verifies YAML syntax correctness and best practice compliance without running playbooks.
2. Simulation Layer (Dry-Run): Runs the playbook in simulation check mode to see the impact of changes before they truly happen on managed nodes.
3. Integration Testing Layer: Builds mock containers or virtual machines, executes the playbook for real, verifies the server’s final state, and tests idempotency capabilities.

Static Analysis with ansible-lint #

ansible-lint is a command-line static analysis tool (linter) specifically designed to analyze Ansible playbooks, roles, and variables. This linter acts as our first line of defense in detecting potential bugs, bad writing styles, security violations (like credential leaks), and non-compliance with modern standards.

Important ansible-lint Detection Rules #

• name[play]: Ensures every play and task has a clear name description so execution logs are easy to read.
• command-instead-of-module: Detects if we use raw shell/command modules for tasks that actually already have idempotent built-in Ansible modules (for example calling shell: apt-get install instead of the apt module).
• no-changed-when: Warns if we use command/shell modules without including the changed_when attribute, because those modules always report “changed” status (not idempotent) by default.

Configuring ansible-lint #

We can customize this linter’s behavior in detail using a .ansible-lint configuration file placed at our project’s root directory:

# .ansible-lint
---
# Setting a strict production-level profile
profile: production

# Ignoring certain rules that don't match our team's conventions
skip_list:
  - yaml[line-length]      # Skip YAML line length checks
  - name[template]         # Ignore warnings about variable use in task names

# Determining directory paths to skip from linting scans
exclude_paths:
  - .cache/
  - molecule/
  - tests/

To run the linter on the entire project, we just type the following command in the terminal:

# CORRECT: Running automatic linting before committing code
ansible-lint .

Syntax Verification and Dry-Run Testing #

Before targeting active production servers, Ansible provides very useful built-in parameters to minimize execution error risks: --syntax-check and --check (Dry-run).

Syntax Verification via --syntax-check #

This command reads and parses all YAML files, loads all related roles and templates, then verifies whether there are typos or invalid YAML indentation structures. This command runs fast because it doesn’t make SSH connections to target servers.

# CORRECT: Checking the validity of playbook configuration file syntax
ansible-playbook site.yml --syntax-check

Change Simulation via Dry-Run (--check) #

Running the playbook with the -C or --check parameter puts Ansible in simulation mode. Ansible tries to contact target servers, reads the current system state, and reports what changes the modules would make without actually modifying anything on the servers. If we combine it with the --diff option, we can see line-by-line text comparisons (diff output) of the configuration templates that would be installed.

# CORRECT: Running a dry-run simulation with configuration difference visualization
ansible-playbook site.yml --check --diff

Overcoming Check Mode Limitations #

Some tasks depending on output from previous tasks that haven’t executed will trigger fatal errors when run in check mode. For example, if the first task downloads an installer file, and the second task extracts that file, the second task fails in --check mode because the physical installer file never exists on disk.

To solve this problem, we can use the check_mode: false directive to force specific tasks to still run for real even if the playbook runs with the --check parameter.

# ANTI-PATTERN: Letting status check tasks die during simulation
- name: Download the SSL certificate file
  ansible.builtin.get_url:
    url: "https://our.internal/cert.pem"
    dest: /etc/ssl/certs/app.pem

- name: Validate the certificate expiration date
  ansible.builtin.command: openssl x509 -enddate -noout -in /etc/ssl/certs/app.pem
  register: cert_check
  # DON'T: This task crashes in check mode because the /etc/ssl/certs/app.pem file isn't really downloaded.

# CORRECT: Forcing preparation tasks to still execute with check_mode: false
- name: Download the SSL certificate file for real for smooth simulation
  ansible.builtin.get_url:
    url: "https://our.internal/cert.pem"
    dest: /etc/ssl/certs/app.pem
  check_mode: false # ✓ Still run this task for real even in simulation mode

- name: Validate the certificate expiration date safely
  ansible.builtin.command: openssl x509 -enddate -noout -in /etc/ssl/certs/app.pem
  register: cert_check
  failed_when: cert_check.rc != 0
  changed_when: false

Molecule: Integration Testing Framework for Roles #

Molecule is the most popular and recommended testing framework in the Ansible ecosystem. It’s designed to simplify the process of creating isolated integration tests for each Ansible Role modularly.

Molecule works with an automatic workflow: it triggers Docker container (or VM) infrastructure creation, runs (converges) our role inside that container, runs verification assertions to ensure the system is configured correctly, tests idempotency (running the role a second time to ensure no additional status changes), and destroys the test container again after finishing.

This Molecule test lifecycle flow is visualized sequentially through the following diagram:

flowchart TD
    Start["Start: molecule test"] --> DestroyOld["1. Destroy (Clean Up Old Containers)"]
    DestroyOld --> Dependency["2. Dependency (Download Other Roles if Needed)"]
    Dependency --> Syntax["3. Syntax (Check Syntax)"]
    Syntax --> Create["4. Create (Start New Test Container)"]
    Create --> Converge["5. Converge (Run the Playbook/Role for Real)"]
    Converge --> Idempotence["6. Idempotence (Run the Second Time: Must Be 'changed=0')"]
    Idempotence --> Verify["7. Verify (Run Test Assertions)"]
    Verify --> DestroyNew["8. Destroy (Destroy the Test Container)"]
    DestroyNew --> End["Done: Success"]

Molecule Test Directory Structure #

When we initialize a Molecule scenario inside a role, it creates the following test directory structure:

roles/mysql_server/
  ├── tasks/
  │   └── main.yml
  └── molecule/
      └── default/
          ├── molecule.yml       # Driver and platforms configuration file
          ├── converge.yml       # Playbook to apply the role
          └── verify.yml         # Test assertion playbook

Here’s an example of the main molecule.yml configuration file using Docker as our test driver:

# roles/mysql_server/molecule/default/molecule.yml
---
dependency:
  name: galaxy

driver:
  name: docker # ✓ Using Docker containers for very fast test performance

platforms:
  # We can test the role on various target OS variations in parallel
  - name: ubuntu-22-node
    image: geerlingguy/docker-ubuntu2204-ansible:latest
    privileged: true
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
    cgroupns: host
    command: /lib/systemd/systemd

provisioner:
  name: ansible
  playbooks:
    converge: converge.yml
    verify: verify.yml

verifier:
  name: ansible # Using Ansible assertion playbooks for status verification

The converge.yml file is tasked with calling the role we want to test:

# roles/mysql_server/molecule/default/converge.yml
---
- name: Run the MySQL Server Role Application
  hosts: all
  become: true
  roles:
    - role: mysql_server

Writing Verification Scenarios Using the Assert Module #

After Molecule successfully applies the role (converge), we must validate whether the target server has truly reached the state we expect. We write these test assertions in the verify.yml file using the ansible.builtin.assert module and other assertion modules.

Here’s an example of writing a comprehensive verification assertion scenario to test the validity of a MySQL database server configuration:

# roles/mysql_server/molecule/default/verify.yml
---
- name: Run MySQL Configuration Verification Assertions
  hosts: all
  gather_facts: false
  tasks:
    - name: 1. Verify the mysql-server package is installed
      ansible.builtin.package_facts:
        manager: auto

    - name: Ensure mysql-server is in the installed packages list
      ansible.builtin.assert:
        that:
          - "'mysql-server' in ansible_facts.packages"
        fail_msg: "The mysql-server package is not installed on the system!"

    - name: 2. Verify the mysql service is running and set to auto-start
      ansible.builtin.service_facts:

    - name: Ensure the mysql daemon is active
      ansible.builtin.assert:
        that:
          - "ansible_facts.services['mysql.service'].state == 'running'"
          - "ansible_facts.services['mysql.service'].status == 'enabled'"
        fail_msg: "The mysql daemon service is not running or not enabled!"

    - name: 3. Verify the default database port (3306) is listening for connections
      ansible.builtin.wait_for:
        port: 3306
        timeout: 5
        state: started
      register: port_check

    - name: Ensure the port successfully responds to connections
      ansible.builtin.assert:
        that:
          - "port_check.state == 'started'"
        fail_msg: "The MySQL port 3306 is closed!"

    - name: 4. Verify the main configuration file is created with safe access rights
      ansible.builtin.stat:
        path: /etc/mysql/mysql.conf.d/mysqld.cnf
      register: config_file

    - name: Ensure the configuration file exists and is mode 0640
      ansible.builtin.assert:
        that:
          - "config_file.stat.exists"
          - "config_file.stat.mode == '0640'"
          - "config_file.stat.pw_name == 'root'"
        fail_msg: "The mysql configuration file doesn't exist or its access rights are too loose!"

With the assertions above, we test the real state in detail. If there’s a file permission configuration error or a dead port, the Molecule integration test immediately shows a red failure mark automatically.

Advanced Testing Using testinfra #

For operations teams already familiar with the Python programming language ecosystem, using Ansible playbooks to verify servers sometimes feels less flexible. We can use an alternative verifier called testinfra.

Testinfra is a Python module based on the pytest testing framework specifically designed to verify server configurations. Writing tests with Testinfra feels very expressive, clean, and easy for Python developers to read.

Here’s an example of writing a /tests/test_mysql.py test script using Testinfra:

# CORRECT: Writing structured infrastructure tests using pytest-testinfra
import pytest

def test_mysql_package_installed(host):
    """Ensure the mysql-server package is properly installed on the operating system"""
    mysql = host.package("mysql-server")
    assert mysql.is_installed

def test_mysql_service_is_running_and_enabled(host):
    """Ensure the mysql service is actively running and set to auto-start"""
    mysql_service = host.service("mysql")
    assert mysql_service.is_running
    assert mysql_service.is_enabled

def test_mysql_port_is_listening(host):
    """Ensure the default port 3306 is open for tcp network traffic"""
    socket = host.socket("tcp://127.0.0.1:3306")
    assert socket.is_listening

def test_mysql_config_file_permissions(host):
    """Ensure the mysqld.cnf configuration file exists and is protected with root access rights"""
    config = host.file("/etc/mysql/mysql.conf.d/mysqld.cnf")
    assert config.exists
    assert config.user == "root"
    assert config.group == "root"
    assert config.mode == 0o640

To run the Python test script against the target machine, we can trigger pytest directly from our terminal cli:

# CORRECT: Running testinfra using pytest via a target SSH connection
pytest -v --hosts="ssh://[email protected]" tests/test_mysql.py

CI/CD Pipeline Integration (GitHub Actions) #

An automated testing strategy won’t deliver maximum benefit if it isn’t run routinely on every code change. We must integrate ansible-lint and Molecule testing into the CI/CD (Continuous Integration) workflow on our Git repository server.

Here’s a complete workflow configuration for GitHub Actions that automatically triggers the testing process every time a team member creates a new Pull Request:

# .github/workflows/ansible-ci.yml
---
name: Ansible Quality Control CI

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  static_analysis:
    name: 1. Static Analysis (Linter)
    runs-on: ubuntu-latest
    steps:
      - name: Pull the repository code
        uses: actions/checkout@v4

      - name: Setup the Python environment
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install linting dependencies
        run: |
          pip install --upgrade pip
          pip install ansible ansible-lint          

      - name: Run the ansible-lint code analysis
        run: ansible-lint .

  integration_test:
    name: 2. Integration Test (Molecule)
    needs: static_analysis
    runs-on: ubuntu-latest
    steps:
      - name: Pull the repository code
        uses: actions/checkout@v4

      - name: Setup the Python environment
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install the Docker daemon
        uses: docker/setup-buildx-action@v3

      - name: Install Molecule testing dependencies
        run: |
          pip install --upgrade pip
          pip install ansible molecule molecule-plugins[docker]          

      - name: Run the Molecule test on all scenarios
        run: |
          cd roles/mysql_server
          molecule test          
        env:
          PY_COLORS: "1"
          ANSIBLE_FORCE_COLOR: "1"

With the automation pipeline above, our main repository is fully protected from bugs. Pull Requests can’t be merged if the linter detects wrong code writing styles, or if Molecule assertions fail when tested in Docker containers.

Summary #

• ansible-lint is used to analyze syntax and best practice (IaC best practice) compliance statically without needing to connect to target servers.
• Use --syntax-check and --check (Dry-run) to test YAML file validity and simulate configuration change impacts before executing in production.
• check_mode: false is important on preparation tasks (like package downloads) so the dry-run simulation process doesn’t experience false-positive failures.
• Molecule is Ansible’s standard integration testing framework for testing Role functionality in isolation inside instant Docker container environments.
• The Molecule testing process covers a complete lifecycle: Create -> Converge -> Idempotence -> Verify -> Destroy.
• Use the ansible.builtin.assert module inside the verify.yml file to validate open ports, service daemon active status, and file integrity.
• testinfra offers highly expressive, developer-friendly Python pytest-based test script writing for verifying real infrastructure.
• Integrate the entire testing suite into the CI/CD pipeline (like GitHub Actions) to automatically validate every code change when creating Pull Requests.

← Previous: Performance Next: Logging →