Debugging #

Writing clean infrastructure automation code is often accompanied by troubleshooting. One of the most common sources of playbook execution failures is variable value mismatch. You might find a configuration template producing empty lines, a when conditional task mysteriously always skipped, or a playbook running smoothly on development servers but failing completely when applied to production servers.

When faced with variable values that don’t match expectations, you need a systematic approach and the right toolkit to analyze Ansible’s memory contents in real time. Ansible provides very reliable built-in modules for runtime investigation, prerequisite validation, and tracing variable precedence override origins. This article thoroughly covers variable debugging techniques using the debug module, enforcing fail-fast with assert and fail, tracing variable precedence in sequence, debugging interactive input (vars_prompt), and safely testing Jinja2 template rendering.

The debug Module: The Main Investigation Tool #

The debug module is the Swiss Army knife for Ansible developers. Its main job is printing variable values, text messages, or complex data structures to the terminal screen (stdout) during playbook execution.

There are two main parameters we often use in the debug module:

1. var: Used to display the variable contents and its data structure literally (including list or dictionary data types).
2. msg: Used to print custom text strings combined with variable interpolation using Jinja2 expressions.

# Example of basic debug module usage
- name: Runtime Variable Investigation
  hosts: webservers
  tasks:
    - name: Display the database variable data structure literally
      debug:
        var: db_configuration
      # ✓ The var parameter doesn't need curly braces {{ }}.
      # This module prints the entire db_configuration dictionary structure neatly.

    - name: Display a custom interpolated message
      debug:
        msg: "This server has IP {{ ansible_default_ipv4.address }} with {{ ansible_memtotal_mb }} MB of RAM"
      # ✓ The msg parameter needs curly braces {{ }} for string interpolation.

Embedding Permanent Debug with the verbosity Parameter #

Inserting debug modules in the middle of tasks often clutters the terminal output when the playbook runs in normal mode by the operations team. However, removing the debug task is also a loss because you’ll need it again when an error occurs in the future.

To bridge this need, you can add the verbosity option to the debug module. The verbosity parameter accepts an integer value (usually 1 to 3) representing the CLI verbosity level. Debug tasks with the verbosity parameter only print to the terminal if you run the ansible-playbook command with the -v, -vv, or -vvv flags.

# Applying a hidden debug module (only active in verbose mode)
- name: Process the SSL configuration update
  template:
    src: ssl.conf.j2
    dest: /etc/nginx/conf.d/ssl.conf
  notify: Reload Nginx

- name: Print the secret SSL parameters for debugging
  debug:
    msg: "SSL Key Path: {{ ssl_private_key_path }}, SSL Cert: {{ ssl_certificate_path }}"
    verbosity: 2  # ✓ Only appears when run with the -vv flag or higher

Here’s a table representing the effect of CLI verbose flags on the execution of the debug task above:

Prerequisite Validation: Fail-Fast Strategies with assert and fail #

In production infrastructure automation, letting a playbook run halfway before eventually failing due to an invalid variable is a dangerous action. For example, your playbook successfully deletes the old application folder, but fails when downloading the new binary because the download URL variable turned out empty. This condition leaves your server in a broken state (downtime).

The best strategy is applying the Fail-Fast concept (fail as early as possible) at the start of a play using the assert or fail modules.

1. Validating Variables with the assert Module #

The assert module checks whether a set of conditional statements is true. If even one condition is false, Ansible immediately stops the playbook right there and displays the custom error message you defined.

You can also use built-in Ansible filters like ipaddr (for IP format validation) or regex to ensure input validity.

# CORRECT: Checking variable validity before starting installation
- name: Initial Deployment Variable Validation
  hosts: appservers
  tasks:
    - name: Enforce mandatory variable boundary conditions
      assert:
        that:
          - app_version is defined
          - app_version | length > 0
          - app_port | int >= 1024
          - app_port | int <= 65535
          - env in ['development', 'staging', 'production']
          - db_host_ip | ipaddr
        fail_msg: >
          Automation stopped! Invalid variables:
          app_version={{ app_version | default('EMPTY') }},
          app_port={{ app_port | default('MISSING') }},
          db_host_ip={{ db_host_ip | default('INVALID') }}          
        success_msg: "All parameters are valid. Starting deployment..."
      # ✓ If any 'that' condition fails, the playbook stops at the first second.

2. Stopping Execution with the fail Module #

If you want to do more procedural logic checks (for example based on the execution result of a previous shell command) and want to stop the play with a dynamic error message, you can combine the fail module with the when parameter.

# Stop the playbook if free disk storage capacity is critical
- name: Check the free disk storage space
  command: df -BG /var/www --output=avail
  register: disk_info
  changed_when: false

- name: Validate a minimum of 10 Gigabytes of disk space
  fail:
    msg: >
      Insufficient server storage capacity!
      Available: {{ disk_info.stdout_lines[1] | trim }}
      Minimum: 10GB required to decompress the assets.      
  when: disk_info.stdout_lines[1] | trim | replace('G', '') | int < 10
  # ✓ Explicitly stops the play with clear troubleshooting instructions.

Debugging User Input in vars_prompt #

When a playbook is designed to request direct user input (interactive prompt), input failures (like password typos or empty values) often mess up subsequent tasks.

You can safely audit input testing by controlling the private parameter. By default, private: true hides the input characters on screen. However, for debugging purposes (non-sensitive), you can change it to private: false and use assert to validate it before use.

# Using vars_prompt with instant assertion validation
- name: Interactive Deployment Playbook
  hosts: all
  vars_prompt:
    - name: target_release_tag
      prompt: "Enter the application release tag you want to deploy"
      private: false  # Show the characters so users can confirm the spelling
      default: "latest"

  tasks:
    - name: Validate the release tag format (must start with the letter v)
      assert:
        that:
          - target_release_tag is defined
          - target_release_tag is match('^v[0-9]+\\.[0-9]+\\.[0-9]+$') or target_release_tag == 'latest'
        fail_msg: "Wrong release tag format! Example of a correct format: v1.0.2 or 'latest'"

Debugging Diagnosis Flow: Tracing Variable Origins #

When a variable has a wrong value, your job is finding which location defined that value and overrode it based on the precedence hierarchy.

Here’s the decision flow diagram for diagnosing undefined or wrongly-valued variables in Ansible:

flowchart TD
    Start(["Start Diagnosing Variable X"]) --> CheckDefined{"Is Variable X Defined?"}
    
    CheckDefined -- "No" --> TraceMissing["Check: defaults/main.yml file in the Role, or inventory group_vars/all"]
    TraceMissing --> AddVar["Define Variable X"]
    
    CheckDefined -- "Yes" --> CheckVal{"Does X's Value Match Expectations?"}
    
    CheckVal -- "Yes" --> Done(["Variable X Successfully Debugged"])
    
    CheckVal -- "No (Wrong Value)" --> RunCLIQuery["Run a CLI Query: ansible -m debug to trace the runtime"]
    RunCLIQuery --> CheckExtraVars{"Is the Value Overridden by Extra Vars -e?"}
    
    CheckExtraVars -- "Yes" --> FixExtraVars["Correct the -e parameter when running the CLI"]
    CheckExtraVars -- "No" --> CheckSetFact{"Is the Value Overridden by set_fact during the Play?"}
    
    CheckSetFact -- "Yes" --> FixSetFact["Fix the set_fact evaluation logic in tasks"]
    CheckSetFact -- "No" --> TracePrecedenceTree["Trace files from highest to lowest priority: vars/main.yml -> group_vars -> host_vars"]
    
    TracePrecedenceTree --> CorrectFile["Adjust the value in the file with the right priority"]
    FixExtraVars --> Done
    FixSetFact --> Done
    CorrectFile --> Done
    AddVar --> Done

CLI Command Guide for Precedence Debugging #

You can run a series of terminal commands to see variable snapshots from various system angles:

# 1. See the actual active variable value at runtime on the target server
ansible -i inventory/ hosts.ini web-prod-01 -m debug -a "var=app_port"

# 2. Display all variables bound to the target host (filter with grep)
ansible -i inventory/ hosts.ini web-prod-01 -m debug -a "var=hostvars['web-prod-01']" | grep app_port

# 3. Check variable values stored at the inventory level (without running a playbook)
ansible-inventory -i inventory/ --host web-prod-01 --list | grep app_port

Creating Dynamic Runtime Variables: The set_fact Module #

The set_fact module is your way to create new variables or change existing variable values dynamically mid-playbook. Variables created with set_fact act in the host scope and register at priority level 19. This makes set_fact variables very powerful, able to override almost all static variables written in inventory or roles.

# Creating dynamic variables based on target software availability
- name: Check whether the docker binary is installed
  command: which docker
  register: docker_check
  failed_when: false
  changed_when: false

- name: Set the docker status flag dynamically
  set_fact:
    is_docker_available: "{{ docker_check.rc == 0 }}"
    docker_runtime_version: "{{ 'Not Detected' if docker_check.rc != 0 else 'Active' }}"
  # ✓ The 'is_docker_available' variable is now available to all subsequent tasks in this play.

- name: Install the docker monitoring module if docker is detected
  pip:
    name: docker
  when: is_docker_available

Safe Jinja2 Template Debugging (Localhost Preview) #

Writing Jinja2 templates for complex configuration files is often prone to looping syntax errors or empty variables. If you directly apply the template to target servers, a render error can kill active server services.

You can use the localhost rendering preview technique (evaluating the template locally on the Control Node) to do a visual audit of the render result before the file is distributed to destination servers.

# Rendering the template locally for inspection
- name: Audit Nginx Configuration Rendering
  hosts: webservers
  tasks:
    - name: Render the configuration to a local preview directory
      template:
        src: nginx.conf.j2
        dest: "/tmp/preview-nginx-{{ inventory_hostname }}.conf"
      delegate_to: localhost
      # ✓ The render action is fully redirected to our local Control Node
      
      check_mode: no
      # ✓ Ensures this task still runs even when executing the playbook with the --check flag

    - name: Visually inspect the rendered file on the Control Node
      command: cat "/tmp/preview-nginx-{{ inventory_hostname }}.conf"
      delegate_to: localhost
      register: nginx_preview
      verbosity: 1
      # ✓ The file is printed to the Control Node terminal for developer validation

Debugging Encrypted Variables (Ansible Vault) #

When working with sensitive data like database passwords or API keys, you use Ansible Vault to encrypt those variables. However, this triggers a debugging challenge: you can’t read those encrypted variables directly in the Git repository files.

If you want to verify whether Ansible successfully decrypts those variables correctly at runtime without accidentally leaking their values to the global terminal logs, you can use a debug task with a high verbosity parameter combined with the assert module.

# Checking whether the vault decryption succeeded and the value isn't empty
- name: Validate Database Password Decryption
  assert:
    that:
      - db_password is defined
      - db_password | length > 0
      - not db_password.startswith('$ANSIBLE_VAULT;')
    fail_msg: "Failed to decrypt the database password! Check our vault key file."
    success_msg: "Sensitive variables were successfully decrypted safely in memory."

This way, you can make sure the decryption process ran successfully without having to print the sensitive password string to the common terminal log screen.

Summary #

• Runtime inspection: The debug module with the var parameter (for object dumps) and msg (for string interpolation) is the main variable troubleshooting tool.
• Permanent Debug: Use the verbosity: 2 or 3 option to embed debug tasks that only appear when the playbook runs with verbose flags (-vv).
• Fail-Fast Assertion: Use the assert module to validate variable existence, length, and ranges at the start of a play to avoid mid-way failures.
• Fail Halts: The fail module combined with when enables forced playbook stops with informative error instructions.
• set_fact Priority: The set_fact module registers dynamic variables at runtime at priority level 19 (very strong, overrides inventory and role vars).
• Template Audit: Redirect template rendering using delegate_to: localhost to visually inspect files on the Control Node before deploying to target servers.

← Previous: Facts Next: What is a Role? →