Project Structure #

A well-organized project structure is the main foundation of infrastructure automation success. A brilliantly written playbook still becomes a maintenance burden if team members struggle to find the relevant configuration files. When our Ansible project grows from managing a few servers to hundreds of servers across regions, an ad-hoc directory structure quickly turns into “ansible spaghetti”. We need to apply disciplined directory layout conventions, variable management, and environment separation so the project stays easy to understand, secure, and scalable without obstacles.

1. Standard Directory Layout for Enterprise Scale #

In building Infrastructure as Code, we need directory layout standardization. This ensures every engineer joining the team can immediately understand where playbooks, roles, inventories, and variables live without exhausting manual searches.

Here’s the most complete directory structure we recommend for medium to large-scale Ansible projects:

ansible-infrastructure/
├── ansible.cfg                   # Default Ansible configuration specific to this project
├── requirements.yml              # Collection and role dependency definitions from Ansible Galaxy
├── README.md                     # Quick orientation guide and playbook run documentation
├── .gitignore                    # File to exclude local files, caches, and secrets from Git
│
├── inventory/                    # Main directory for all environment configurations
│   ├── development/              # Development Environment
│   │   ├── hosts.ini             # Host list for dev
│   │   └── group_vars/           # Group-specific variables in the dev environment
│   │       ├── all.yml           # Global dev variables
│   │       ├── webservers.yml    # Dev webservers group-specific variables
│   │       └── vault.yml         # Dev secret variables (encrypted)
│   ├── staging/                  # Staging Environment
│   │   ├── hosts.ini             # Host list for staging
│   │   └── group_vars/           # Group-specific variables in the staging environment
│   │       ├── all.yml
│   │       ├── webservers.yml
│   │       └── vault.yml
│   └── production/               # Production Environment
│       ├── hosts.ini             # Host list for prod
│       └── group_vars/           # Group-specific variables in the prod environment
│           ├── all.yml
│           ├── webservers.yml
│           └── vault.yml
│
├── group_vars/                   # Global variables shared across all environments (shared)
│   └── all.yml                   # Contains common non-sensitive variables like time zones
│
├── playbooks/                    # All main playbook files live here
│   ├── site.yml                  # Master playbook orchestrating the entire infrastructure
│   ├── provision.yml             # Playbook for provisioning new OS/VM instances
│   ├── deploy.yml                # Playbook for deploying applications to servers
│   ├── patch.yml                 # Playbook for OS security patch updates
│   └── rollback.yml              # Emergency playbook to restore application versions
│
├── roles/                        # Directory for our internal custom roles
│   ├── common/                   # Base role required on every host
│   ├── nginx/                    # Role for Nginx installation and configuration
│   ├── postgresql/               # Role for the PostgreSQL database
│   └── myapp/                    # Role specific to our application module deployment
│
├── collections/                  # Local directory for collection installation (ignored by Git)
│   └── ansible_collections/
│
├── tests/                        # Playbook and role automated testing
│   ├── test-vars.yml             # Dummy variables for testing
│   └── integration/              # Integration testing scenarios
│
├── scripts/                      # Operational helper scripts
│   ├── encrypt-vault.sh          # Helper to quickly encrypt secret files
│   └── rotate-vault-password.sh  # Vault password rotation automation script
│
└── .github/                      # Continuous integration (CI/CD) pipelines
    └── workflows/
        ├── validate.yml          # Syntax and linting validation pipeline
        └── deploy.yml            # Automatic deployment pipeline

By separating files into functional directories like the above, we isolate change risk. For example, if we only want to change staging configuration variables, we know for certain that the change is limited to the inventory/staging/ directory and won’t touch the main program code in roles/.

2. Environment Isolation Through Inventory Structure #

One fatal mistake often found in the field is mixing hosts from various environments (dev, staging, prod) in a single inventory file. This risks mis-targeted deployment incidents, where a playbook meant for development servers is executed against production servers due to a target parameter typo.

We must apply absolute isolation using separate inventory directories for each working environment.

A. Hosts.ini File Structure #

In each environment directory, we place a host file only referencing servers in that environment.

Example inventory/production/hosts.ini:

[webservers]
prod-web-01 ansible_host=10.10.10.11
prod-web-02 ansible_host=10.10.10.12

[dbservers]
prod-db-01 ansible_host=10.10.10.20
prod-db-02 ansible_host=10.10.10.21

[load_balancers]
prod-lb-01 ansible_host=10.10.10.5

[production:children]
webservers
dbservers
load_balancers

B. Variable Merging Mechanism #

Ansible has a very strict variable evaluation hierarchy. By arranging group_vars/ directories under each environment, we can ensure variables with the same name hold different values according to the environment context being run.

Here’s a flow diagram of how Ansible evaluates and merges variables from various directory levels when we execute a deployment command:

flowchart TD
    A["Start Playbook Execution"] --> B["Evaluate Global group_vars/all.yml"]
    B --> C["Evaluate group_vars/all.yml in the Environment Inventory"]
    C --> D["Evaluate Environment group_vars/[group_name].yml"]
    D --> E["Evaluate Environment host_vars/[host_name].yml"]
    E --> F["Evaluate Role Variables (defaults/main.yml)"]
    F --> G["Evaluate Role Variables (vars/main.yml)"]
    G --> H["Evaluate Playbook Variables (vars or vars_files)"]
    H --> I["Evaluate Extra Vars (--extra-vars)"]
    I --> J["Final Variables Formed"]

With this model, we put safe global default values in group_vars/all.yml at the project root (e.g. app_port: 8080), then override those values specifically for production in inventory/production/group_vars/all.yml (e.g. app_port: 443) without breaking the development environment configuration.

3. Standardizing ansible.cfg Configuration at the Project Root #

By default, Ansible looks for global configuration files at /etc/ansible/ansible.cfg or ~/.ansible.cfg. Relying on this global configuration is a dangerous anti-pattern because every developer on our team might have different global configuration versions. This can cause different execution behavior across developer machines.

We must include a local ansible.cfg file at the project root directory. Ansible automatically reads this file when run from that directory.

Here’s the optimal, safe ansible.cfg configuration for daily operational needs:

# ansible.cfg
[defaults]
# Point the default inventory location to our management directory
inventory          = inventory/

# Set the role search path, both local and global roles
roles_path         = roles:~/.ansible/roles

# Set the installed collection search path
collections_paths  = collections:~/.ansible/collections

# Default user used for SSH connections to target hosts
remote_user        = ansible-deploy

# SSH private key location for automatic authentication
private_key_file   = ~/.ssh/ansible_deploy_key

# Reject connections if the target server's host key changes (man-in-the-middle security)
host_key_checking  = true

# Disable creating .retry files that litter our working directory
retry_files_enabled = false

# Change terminal output format to YAML for easier reading than single-line JSON
stdout_callback    = yaml

# Enable callback plugins to analyze task execution times
callbacks_enabled  = profile_tasks, timer

# Limit the number of parallel forks to save bandwidth and controller RAM
forks              = 20

# Optimize smart system facts gathering
gathering          = smart
fact_caching       = jsonfile
fact_caching_connection = /tmp/ansible_fact_cache
fact_caching_timeout    = 3600

[ssh_connection]
# Enable pipelining to speed up task execution by minimizing new SSH connections
pipelining         = true

# Additional SSH parameters to keep connections alive and use ControlMaster
ssh_args           = -o ControlMaster=auto -o ControlPersist=60s

[vault]
# Automatically set the Ansible Vault secret key map per environment
vault_identity_list = [email protected]_pass_dev, [email protected]_pass_staging, [email protected]_pass_prod

Key Parameter Explanations: #

1. pipelining = true: By default, Ansible transfers python modules to target servers, runs them, then deletes them through a separate SSH connection per task. Pipelining minimizes this overhead by sending modules directly through SSH stdin, increasing execution speed up to 4x.
2. stdout_callback = yaml: Changes the output from the messy default form to a very clean indented YAML format, so we can detect errors quickly.
3. vault_identity_list: Lets us use different vault passwords for each environment without manually typing the --vault-password-file flag every time we run a playbook.

4. Consistent Naming Conventions #

Naming conventions are the shared language uniting a team. Without clear naming rules, we’ll see confusing name variations like setupNginx, install-nginx, and nginx_install in the same project.

We apply the following standard naming rules across the entire project:

A. Variable Namespacing Patterns #

Variable collision is a very hard-to-track bug in Ansible because variables are global in scope during playbook runtime. If the nginx role and the postgresql role both define a variable named port, one will unintentionally overwrite the other’s value.

# ANTI-PATTERN: Variables without namespaces, prone to collisions
port: 80
version: 1.14.2
config_dir: /etc/nginx

# CORRECT: Using the role name prefix as the namespace
nginx_port: 80
nginx_version: 1.14.2
nginx_config_dir: /etc/nginx

B. Sensitive Variable Separation (Vault) #

Never mix regular variables with sensitive variables in one file. We must isolate all secrets into a dedicated file named vault.yml and reference them using bridge variables.

# inventory/production/group_vars/all/vars.yml
# CORRECT: Non-sensitive variables declared in a regular file
db_user: myapp_admin
db_host: prod-db-01.internal
db_password: "{{ vault_db_password }}" # References the encrypted variable in vault.yml

# inventory/production/group_vars/all/vault.yml
# CORRECT: This file is fully encrypted using ansible-vault
vault_db_password: "VerySecretAndLong123!"

5. site.yml as the Master Entry Point #

As infrastructure grows, we’ll have dozens of playbooks for various purposes. Without one clear main entry point, new operators will be confused about which playbook to run to set up the entire infrastructure from scratch.

We must create a file named site.yml inside the playbooks/ directory (or linked to the project root). This file acts as the highest-level orchestrator calling other playbooks or roles in sequence.

Here’s an example playbooks/site.yml implementation:

# playbooks/site.yml
# Master Playbook - Living documentation of our infrastructure architecture
---
- name: Apply basic configuration to all servers
  hosts: all
  gather_facts: true
  become: true
  roles:
    - common
    - security-hardening

- name: Configure and cluster the PostgreSQL database
  hosts: db_servers
  become: true
  roles:
    - postgresql
    - postgresql-backup

- name: Set up Web Servers and the Node.js Application
  hosts: web_servers
  become: true
  roles:
    - nodejs
    - myapp

- name: Configure the outer Nginx Load Balancer
  hosts: load_balancers
  become: true
  roles:
    - nginx

Why Is This Approach Very Powerful? #

1. Infrastructure Documentation: Just by reading the site.yml file, anyone can immediately understand our infrastructure topology — what server types we have and which roles run on them.
2. Partial Execution Capability: If we only want to update database servers without touching load balancers, we can use Ansible’s built-in limit flag:

   ansible-playbook -i inventory/production/ playbooks/site.yml --limit db_servers
   

   This is much safer than managing many separate small playbooks without a central orchestrator.

6. Actionable README.md #

README.md is often neglected and only contains a one-sentence description that doesn’t help. A bad README.md forces new developers into time-wasting trial-and-error that risks breaking systems.

We must write an action-oriented (actionable) README.md containing concrete copy-paste commands to start working.

Here’s the standard README.md template that every Ansible project root should have:

# Ansible Infrastructure Codebase

This repository manages all provisioning, configuration, and deployment automation for our cloud infrastructure.

## 1. System Prerequisites

Before running playbooks, make sure our local machine has:
- **Python**: Version 3.11 or higher
- **Ansible**: Version 2.15 or higher
- **SSH Key**: Private key registered at `~/.ssh/ansible_deploy_key` with sudo access on target servers.

## 2. Initial Setup

Run the following commands in the root directory to install external dependencies and prepare local vault encryption keys:

```bash
# Install external collection and role dependencies
ansible-galaxy install -r requirements.yml

# Create a local vault password file for the development environment (don't commit this file!)
echo "our_dev_password" > .vault_pass_dev
chmod 600 .vault_pass_dev
```

## 3. Daily Operational Guide

### A. Code Validation (Dry-run)
Always run syntax checks and change simulations before doing real deployments:

```bash
# Check playbook syntax
ansible-playbook -i inventory/staging/ playbooks/site.yml --syntax-check

# Simulate changes (Check mode with visual diff)
ansible-playbook -i inventory/staging/ playbooks/site.yml --check --diff
```

### B. Application Deployment to Staging
```bash
ansible-playbook -i inventory/staging/ playbooks/deploy.yml -e "app_version=v2.4.0"
```

### C. Full Infrastructure Deployment to Production
```bash
ansible-playbook -i inventory/production/ playbooks/site.yml
```

## 4. Secret Management (Ansible Vault)

We lock sensitive variables per environment. To edit the production vault file:

```bash
ansible-vault edit inventory/production/group_vars/all/vault.yml --vault-id [email protected]_pass_prod
```

With a README.md structured like this, new developer onboarding time can be cut from days to just a few minutes.

7. Structure Patterns for Small vs Large-Scale Projects #

As the number of playbooks and roles grows, one monorepo can become too dense and hard to manage because too many teams modify the same files. We must know when to keep a simple architecture and when to split into a distributed architecture.

Here’s a decision tree diagram guiding us to choose the most efficient project architecture pattern based on team scale and server count:

flowchart TD
    A["Start Project Scale Evaluation"] --> B{"How many target hosts?"}
    B -->|< 50 Servers| C{"How many teams manage it?"}
    B -->|> 50 Servers| D["Use the Structured Monorepo Pattern"]
    
    C -->|"One Centralized Team"| E["Use the Single Repository Pattern (Simple)"]
    C -->|"Multi-Team Across Divisions"| D
    
    D --> F{"Are roles often shared with other projects?"}
    F -->|"Yes"| G["Split Roles into Separate Ansible Collections"]
    F -->|"No"| H["Keep the Monorepo with Strict CI/CD"]

A. Small Project Pattern (< 50 Servers) #

For small scale, we don’t need to split the inventory into overly deep sub-folders if our servers are very dynamic. Just use one inventory file with clear host group separation inside it. However, make sure variables stay neatly organized.

B. Enterprise Scale Pattern (> 50 Servers / Multi-Team) #

At this scale, we recommend splitting repositories. Main program code (generic roles like nginx or mysql) should be pulled out of the main infrastructure repository and published as an Ansible Collection or standalone role repository. The main infrastructure repository then only acts as the “glue” importing those roles through the requirements.yml file and managing their variable configuration.

Example requirements.yml:

# requirements.yml
---
roles:
  # Import open-source roles from Ansible Galaxy with pinned versions
  - name: geerlingguy.nginx
    version: 3.1.1
  - name: geerlingguy.postgresql
    version: 3.0.0

collections:
  # Import cloud provider collections for provisioning
  - name: amazon.aws
    version: 6.0.0
  - name: community.general
    version: 8.2.0

This keeps our main repository clean, lightweight, and minimizes Git commit conflicts between team members from different divisions.

Anti-Patterns to Avoid #

In structuring Ansible projects, there are several bad practices (anti-patterns) commonly done but very harmful in the long run:

1. Storing SSH Keys or Plaintext Vault Passwords in Git #

This is the most critical security gap. Writing raw passwords or putting SSH private keys in the repository exposes them to everyone with Git read access, increasing data leak risk.

# ANTI-PATTERN: Storing raw passwords in YAML files
ansible_ssh_pass: "VeryStrongSecret123"
db_root_password: "super_secret_db_pass"

# CORRECT: Referencing securely encrypted vault variables
ansible_ssh_pass: "{{ vault_ansible_ssh_pass }}"
db_root_password: "{{ vault_db_root_password }}"

2. Using Non-Standard Ad-Hoc Library Directories #

Putting custom python modules in random places makes Ansible fail to detect those modules on other developer machines.

# ANTI-PATTERN: Random module file locations
ansible-infrastructure/
├── my_custom_module.py
└── playbooks/
    └── deploy.yml

# CORRECT: Put modules in the built-in library folder at the root so they're auto-loaded
ansible-infrastructure/
├── library/
│   └── my_custom_module.py
└── playbooks/
    └── deploy.yml

Project Structure Review Checklist #

We must verify our folder architecture compliance using the following checklist criteria before merging code to the main branch:

DIRECTORIES & STRUCTURE:
  □ ansible.cfg is at the project root and doesn't rely on the global /etc/ansible/ configuration.
  □ External dependencies are declared in writing in requirements.yml with pinned versions.
  □ The roles/ directory only contains internal roles, while external roles are installed to a separate external directory.
  □ The tests/ folder contains minimal playbooks validating all roles independently.

INVENTORY & VARIABLES:
  □ Every environment has its own physically isolated inventory directory.
  □ No sensitive global variables placed in the root group_vars/all.yml without encryption.
  □ Sensitive variables are in vault.yml files and encrypted with ansible-vault.
  □ All variable names use the role name prefix (namespace) to prevent name collisions.

DOCUMENTATION & ENTRY POINTS:
  □ A site.yml file exists at the root or in the playbooks folder as the main orchestrator.
  □ The README.md file contains clear copy-paste commands for initial setup and playbook execution.
  □ The .gitignore file excludes local caches (.ansible-lint, .cache, .tmp) and vault password files.

Summary #

• Separate Inventory Folders — We must separate inventory files per environment (dev, staging, prod) into their own sub-folders to prevent fatal mis-targeted playbook executions against production.
• Local ansible.cfg Configuration — Always include an ansible.cfg file at the project root so all team members and CI/CD pipelines execute Ansible with identical behavior parameters.
• Variable Namespacing — Avoid variable collisions by always prefixing every variable we create with the role name (e.g. nginx_port instead of port).
• Secret Separation — Separate sensitive variables into encrypted vault.yml files and use bridge variables in regular configuration files to reference their values.
• site.yml as the Main Map — Use the master site.yml playbook as the single orchestrator entry point to document and run the configuration of all our servers in an orderly way.
• Action-Oriented README.md — Create concrete installation and execution instructions in README.md so new developers can contribute directly without technical obstacles.
• Pin Dependency Versions — Always lock role and collection versions in requirements.yml so our system avoids bugs from future library version changes.

← Previous: CI/CD Anti Pattern Next: Code Quality →