Performance Anti Pattern #

Execution speed and efficiency are the keys to successful large-scale infrastructure automation. When we first write Ansible playbooks, our main focus is usually just logic correctness — making sure servers are configured properly. However, as the number of managed nodes grows, playbook efficiency starts being tested. A playbook that initially finished in 2 minutes for 3 servers can balloon to 45 minutes when run against 100 servers. Slow execution is often not caused by insufficient server capacity, but by playbook design decisions triggering repeated, inefficient operations.

A slow feedback loop slows the development cycle, hinders emergency deployment processes, and widens the failure risk window (maintenance window). Most Ansible bottlenecks can be easily identified and fixed without changing the final system logic. By understanding how Ansible interacts with remote servers behind the scenes, we can optimize playbooks to run many times faster. In this article, we’ll explore various performance anti-patterns, why those patterns slow us down, and how to apply the right optimization solutions.

1. Enabling Fact Gathering Without Using It #

By default, Ansible runs the setup module at the start of every play to collect facts about target servers, such as operating system details, IP addresses, memory allocation, and disk status. This process is known as fact gathering.

Why Is This Dangerous? #

The fact collection process takes about 1 to 3 seconds per host. Ansible must do an SSH handshake, transfer the fact collection module, execute it on the remote server, and return very large JSON data to the control node. If our playbook has many plays managing hundreds of servers, but we never use Ansible’s built-in variables (ansible_*) in our task logic, this process wastes enormous time and computing resources for nothing.

# ANTI-PATTERN: leaving fact gathering on in a play that doesn't need it
- name: Restart the Web Application Service
  hosts: webservers
  # gather_facts defaults to true if not defined!
  tasks:
    - name: Restart the nginx systemd service
      systemd:
        name: nginx
        state: restarted
      become: true
    # This playbook runs slowly on hundreds of hosts because it collects hardware data
    # that the systemd module above never reads.

The fundamental solution is disabling gather_facts on plays that only do operational tasks or static configuration not dependent on remote server hardware conditions.

# CORRECT: Disabling fact gathering for simple operational tasks
- name: Restart the Web Application Service Quickly
  hosts: webservers
  gather_facts: false   # ✓ CORRECT: Saves the initial SSH connection bootstrap time
  tasks:
    - name: Restart the nginx systemd service
      systemd:
        name: nginx
        state: restarted
      become: true

If our playbook really needs server fact information but it’s spread across several different plays, we can enable the fact caching system. This way, server facts are only collected once at the start, stored in a local cache (like a JSON file or Redis database), and reused in subsequent play executions without re-querying the remote server.

# CORRECT: Configuring Fact Caching in ansible.cfg
# File: ansible.cfg
[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible_fact_cache
fact_caching_timeout = 86400  # Cache valid for 24 hours

2. Using Loops for Tasks That Support Batch Operations #

Ansible provides the loop directive (or with_items in old versions) to run one task repeatedly with different input parameters.

Why Is This Dangerous? #

If we use loop on system package manager modules like apt, yum, or pip, Ansible executes those modules individually as many times as there are items in the loop. That means if there are 10 packages to install, Ansible calls the package manager command 10 separate times. Each call requires a new SSH round-trip, module initialization, package database locking (like the apt lock), and dependency checking. This makes installation time very slow.

# ANTI-PATTERN: using a loop to install packages one by one
- name: Install Maintenance System Packages
  apt:
    name: "{{ item }}"
    state: present
  loop:
    - nginx
    - postgresql-client
    - redis-tools
    - python3-pip
    - git
  become: true
  # ✗ DON'T: Ansible calls the apt-get install command 5 separate times!

Instead, we should send the package list directly to the module parameter. Most built-in Ansible modules like apt, yum, package, and pip are optimized to accept lists and execute them in a single batch operation.

# CORRECT: Batch installation in a single module call
- name: Install Maintenance System Packages in Batch
  apt:
    name:
      - nginx
      - postgresql-client
      - redis-tools
      - python3-pip
      - git
    state: present
    update_cache: true
  become: true
  # ✓ CORRECT: Only one apt-get install command call. Much faster and more efficient.

[!TIP] This batch optimization pattern also applies to other modules. For example, when copying many static files, instead of using loop with the copy module which eats repeated SSH time, we can use the synchronize module which leverages the rsync utility behind the scenes to transfer files in bulk over a single connection.

3. Leaving the Forks Value at the Default Configuration (Default forks = 5) #

By default, Ansible is configured to process tasks in parallel on at most 5 servers at once. This parameter is controlled by the forks option in the ansible.cfg configuration.

Why Is This Dangerous? #

If our inventory contains 100 remote servers, Ansible only executes the task on the first 5 servers, waits for all to finish, then continues to the next 5 servers. To complete one task across all hosts, Ansible needs 20 queue batches (100 / 5 = 20). If one task takes 10 seconds, the total task execution time becomes 200 seconds. This creates a very long bottleneck queue even though the CPU and memory resources on our control node are still very free.

# ANTI-PATTERN: letting the default forks limit restrict parallelization
# File: ansible.cfg (Or with no custom configuration at all)
[defaults]
# forks = 5   # ✗ DON'T: The built-in limit is too low for modern infrastructure

We must raise the forks value according to the control node’s hardware capacity (CPU and RAM) and our network bandwidth. As a rule of thumb, we can raise forks to 20 to 50 to optimize parallelization without overloading the control node.

# CORRECT: Increasing the forks value in ansible.cfg
# File: ansible.cfg
[defaults]
forks = 30   # ✓ CORRECT: Processes 30 servers simultaneously, speeding up execution up to 600%

To determine the optimal forks number for our infrastructure, do gradual testing while monitoring control node system resource usage with the following command:

# Monitor control node CPU and memory resource utilization while the playbook runs
ansible-playbook -i inventory/production/ site.yml &
watch -n1 'ps aux | grep ansible | head -5; echo "---"; uptime'

4. Ignoring SSH Connection Optimization (Pipelining and Multiplexing) #

By default, the Ansible task execution process on target hosts through the SSH protocol involves several stages: Ansible generates a temporary python script file, wraps it, transfers the file to a temporary directory on the target host using SFTP/SCP, grants execution permissions, runs the file, and finally deletes the file when done.

Why Is This Dangerous? #

This copy-and-run script file cycle happens for every task on every host. This creates very high file transfer and SSH connection initialization overhead, especially if our playbook has dozens of short tasks. Without SSH connection optimization, most of our playbook execution time is spent negotiating connection encryption and transferring small script files.

Here’s a comparison diagram between the default execution process (without optimization) versus the execution process optimized with Pipelining and SSH Multiplexing (ControlPersist):

sequenceDiagram
    autonumber
    participant C as "Control Node (Ansible)"
    participant S as "Managed Node (Server)"
    Note over C,S: Without SSH Optimization (Default)
    C->>S: "Initiate TCP & SSH Cryptographic Handshake"
    S-->>C: "SSH Session Opened"
    C->>S: "Transfer python module script file (.py)"
    S-->>C: "Transfer complete"
    C->>S: "Run the remote python script"
    S-->>C: "Module execution result"
    C->>S: "Close SSH Connection"
    Note over C,S: With Optimization (Pipelining & ControlPersist)
    C->>S: "Initiate TCP & SSH Cryptographic Handshake (Only once)"
    S-->>C: "Multiplexing Session opened on a local socket"
    C->>S: "Execute directly via stdin (Pipelined)"
    S-->>C: "Module execution result"
    C->>S: "Execute the next task via the same socket"
    S-->>C: "Module execution result"
    Note over C,S: The socket stays open during ControlPersist (e.g. 60s)

To solve this connection overhead problem, we must enable two important features in the Ansible SSH configuration: Pipelining and SSH Multiplexing (ControlPersist).

# CORRECT: Enabling SSH Pipelining and Multiplexing in ansible.cfg
# File: ansible.cfg
[defaults]
forks = 20

[ssh_connection]
pipelining = True
# ✓ CORRECT: Pipelining executes python modules directly through SSH stdin without transferring files.
# This can reduce task execution time by 30% - 50%.

ssh_args = -o ControlMaster=auto -o ControlPersist=120s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r
# ✓ CORRECT: Enables SSH multiplexing. The initial cryptographic connection to the remote server is saved
# as a socket file in /tmp/. Subsequent tasks reuse the existing SSH tunnel without re-handshaking.
# ControlPersist=120s keeps the tunnel open for 2 minutes after the last task finishes.

[!WARNING] When enabling pipelining = True, make sure the security configuration on managed nodes (remote servers) doesn’t have the requiretty option in the /etc/sudoers file. If that option is active, privilege escalation execution via sudo will fail because pipelining doesn’t provide a pseudo-TTY. Disable requiretty for the ansible user to take advantage of this feature.

5. Repeatedly Running Local Tasks Without run_once #

There are times when we need to run tasks interacting with the control node (localhost), like fetching credentials from an external API (e.g. HashiCorp Vault), creating local backup directories, or logging notifications to Slack.

Why Is This Dangerous? #

If we use the delegate_to: localhost directive without including run_once: true, Ansible executes that task repeatedly as many times as the number of target hosts in our play. If we manage 50 servers, Ansible sends 50 identical API requests to Vault or Slack. This can trigger request rate limiting from the target API, waste bandwidth, and drastically slow down the playbook due to unnecessary queue processing.

# ANTI-PATTERN: running a local delegation task repeatedly for every host
- name: Fetch the API Authorization Token from the Central Server
  hosts: appservers
  tasks:
    - name: Get a temporary access token
      uri:
        url: "https://vault.company.internal/v1/auth/token"
        method: POST
        body_format: json
        body:
          role_id: "app-role"
          secret_id: "app-secret"
      register: vault_response
      delegate_to: localhost
      # ✗ DON'T: This task is called repeatedly for every host in appservers!
      # If there are 100 hosts, 100 HTTP requests are sent to Vault with exactly the same result.

The solution is adding the run_once: true parameter to that local delegation task. Ansible only executes the task once, then shares the registered result variable to all other hosts in the play.

# CORRECT: Using run_once to limit a single local task execution
- name: Fetch the API Authorization Token Efficiently
  hosts: appservers
  tasks:
    - name: Get a temporary access token (Only once is enough)
      uri:
        url: "https://vault.company.internal/v1/auth/token"
        method: POST
        body_format: json
        body:
          role_id: "app-role"
          secret_id: "app-secret"
      register: vault_response
      delegate_to: localhost
      run_once: true              # ✓ CORRECT: Executed only once for the whole play
      no_log: true

    - name: Apply the token to the application configuration file
      template:
        src: "config.json.j2"
        dest: "/etc/myapp/config.json"
      # The 'vault_response' result from the first host is automatically accessible to other hosts

6. Serial Scheduling Without Task Parallelization (Serial vs Free Strategy) #

By default, Ansible uses the linear execution strategy. In this strategy, Ansible finishes the first task across all active hosts before moving to the second task simultaneously.

Why Is This Dangerous? #

If one host experiences a long delay running a task (e.g. due to disk performance issues or slow network connections), the whole playbook queue is held up. Other servers that already finished the first task faster are forced to wait for the slow server before continuing to the next task.

We can optimize this workflow by adjusting the execution strategy or using the serial parameter wisely.

# ALTERNATIVE: Using the Free Strategy for Independent Tasks
- name: Log Collection and System Analysis
  hosts: database_servers
  strategy: free   # ✓ CORRECT: Each host executes all tasks as fast as possible
  # without waiting for other hosts to finish the same task. Suitable for plays
  # doing data collection or independent status verification.
  tasks:
    - name: Collect disk statistics
      command: df -h
    - name: Collect memory status
      command: free -m

[!IMPORTANT] Don’t use strategy: free if the tasks in our playbook have strict order dependencies between different hosts (e.g. turning off the load balancer -> updating app servers -> turning the load balancer back on). For zero-downtime application deployment scenarios (rolling updates), use the serial option with a balanced percentage proportion so it doesn’t slow the release but still limits the failure blast radius.

# CORRECT: Rolling Update with a Balanced Serial Proportion
- name: Deploy the Web Application Cluster
  hosts: webservers
  serial: "25%"  # ✓ CORRECT: Processes 25% of servers at once in one batch.
  # If the first batch succeeds, continue to the next batch. This provides safety
  # without sacrificing execution performance like 'serial: 1' does.
  tasks:
    - name: Update the source code
      git:
        repo: "https://github.com/app/web.git"
        dest: "/var/www/html"

7. Ignoring Task Profiling to Analyze Bottlenecks #

Optimizing playbook performance without measuring first is a speculative action that often misses the target. We might spend hours restructuring a task that actually only takes 2 seconds out of a 15-minute total execution time.

Why Is This Dangerous? #

Without duration data visualization, we don’t know where the real bottleneck is. Is it the package manager repository update process? Or is it a template module slow at processing Jinja2 logic? We need an automatic profiling system to measure each task’s duration.

We can enable Ansible’s built-in callback plugins to profile task execution times in detail without adding third-party tools.

# CORRECT: Enabling the Profiling Callback in ansible.cfg
# File: ansible.cfg
[defaults]
# Enable the built-in callbacks for task, role, and total timer profiling
callbacks_enabled = profile_tasks, profile_roles, timer

After the callback is enabled, every time we run a playbook, Ansible shows a summary of the longest task durations at the end of the terminal output.

# Example Terminal Profiling Output after the Playbook Finishes:
Thursday 18 June 2026  15:45:10 +0700 (0:00:03.245)       0:08:42.123
===================================================================
Install all packages in batch ------------------------------- 185.34s  ← Main Optimization Target!
Gathering Facts ---------------------------------------------- 72.15s  ← Enable Fact Caching!
Compile application assets from source ----------------------- 54.12s
Verify database migration state ------------------------------ 12.04s
Template configuration file ----------------------------------- 8.11s

With this data, we know exactly which task to optimize first. Focus our optimization efforts on the top tasks consuming the largest percentage of time.

Summary #

• Turn off gather_facts: false — For all operational plays that don’t read target host hardware properties. Use fact caching if facts are still needed across several plays.
• Avoid Package Manager Loops — Send the package list directly as a list to the module argument (apt, yum, pip) to execute them in a single batch, instead of looping one by one.
• Raise the forks Limit — Don’t let the default forks configuration of 5 restrict parallelization on large-scale infrastructure. Set a minimum of 20 to 50.
• Enable Pipelining and ControlPersist — Configure the SSH options in ansible.cfg to minimize cryptographic handshake overhead and temporary script file transfer processes.
• Limit Local Delegation with run_once — Always include run_once: true on tasks using delegate_to: localhost to prevent sending repeated identical requests to external APIs.
• Choose the Right Execution Strategy — Use strategy: free for independent audit or data collection tasks, and apply percentage-based serial proportions for efficient rolling updates.
• Profile Durations Regularly — Enable the profile_tasks and profile_roles callbacks in ansible.cfg to get duration data visualization guiding performance optimization priorities.

← Previous: Security Anti Pattern Next: CI/CD Anti Pattern →