From 171cbfbb329e46e52dbaca931f4062cfa9d36812 Mon Sep 17 00:00:00 2001 From: Pieter Date: Sat, 27 Dec 2025 14:00:22 +0100 Subject: [PATCH] WIP: Ansible base configuration - common role (#2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Progress on Issue #2: Ansible Base Configuration Completed: - ✅ Ansible installed via pipx (isolated Python environment) - ✅ Hetzner Cloud dynamic inventory configured - ✅ Ansible configuration (ansible.cfg) - ✅ Common role for base system hardening: - SSH hardening (key-only, no root password) - UFW firewall configuration - Fail2ban for SSH protection - Automatic security updates - Timezone and system packages - ✅ Comprehensive Ansible README with setup guide Architecture Updates: - Added Decision #15: pipx for isolated Python environments - Updated ADR changelog with pipx adoption Still TODO for #2: - Docker role - Traefik role - Setup playbook - Deploy playbook - Testing against live server Files added: - ansible/README.md - Complete Ansible guide - ansible/ansible.cfg - Ansible configuration - ansible/hcloud.yml - Hetzner dynamic inventory - ansible/roles/common/* - Base hardening role Partial progress on #2 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- ansible/README.md | 285 ++++++++++++++++++ ansible/ansible.cfg | 39 +++ ansible/hcloud.yml | 37 +++ ansible/roles/common/defaults/main.yml | 42 +++ ansible/roles/common/handlers/main.yml | 17 ++ ansible/roles/common/tasks/fail2ban.yml | 20 ++ ansible/roles/common/tasks/firewall.yml | 28 ++ ansible/roles/common/tasks/main.yml | 30 ++ ansible/roles/common/tasks/ssh.yml | 23 ++ ansible/roles/common/tasks/updates.yml | 23 ++ .../roles/common/templates/20auto-upgrades.j2 | 7 + .../common/templates/50unattended-upgrades.j2 | 28 ++ ansible/roles/common/templates/jail.local.j2 | 18 ++ docs/architecture-decisions.md | 42 +++ 14 files changed, 639 insertions(+) create mode 100644 ansible/README.md create mode 100644 ansible/ansible.cfg create mode 100644 ansible/hcloud.yml create mode 100644 ansible/roles/common/defaults/main.yml create mode 100644 ansible/roles/common/handlers/main.yml create mode 100644 ansible/roles/common/tasks/fail2ban.yml create mode 100644 ansible/roles/common/tasks/firewall.yml create mode 100644 ansible/roles/common/tasks/main.yml create mode 100644 ansible/roles/common/tasks/ssh.yml create mode 100644 ansible/roles/common/tasks/updates.yml create mode 100644 ansible/roles/common/templates/20auto-upgrades.j2 create mode 100644 ansible/roles/common/templates/50unattended-upgrades.j2 create mode 100644 ansible/roles/common/templates/jail.local.j2 diff --git a/ansible/README.md b/ansible/README.md new file mode 100644 index 0000000..251c743 --- /dev/null +++ b/ansible/README.md @@ -0,0 +1,285 @@ +# Ansible Configuration Management + +Ansible playbooks and roles for configuring and managing the multi-tenant VPS infrastructure. + +## Prerequisites + +### 1. Install Ansible (via pipx - isolated environment) + +**Why pipx?** Isolates Ansible in its own Python environment, preventing conflicts. + +```bash +# Install pipx +brew install pipx +pipx ensurepath + +# Install Ansible +pipx install --include-deps ansible + +# Install required dependencies +pipx inject ansible requests python-dateutil + +# Verify installation +ansible --version +``` + +### 2. Install Ansible Collections + +```bash +ansible-galaxy collection install hetzner.hcloud community.sops community.general +``` + +### 3. Set Hetzner Cloud API Token + +```bash +export HCLOUD_TOKEN="your-hetzner-cloud-api-token" +``` + +Or add to your shell profile (`~/.zshrc` or `~/.bashrc`): +```bash +export HCLOUD_TOKEN="your-token-here" +``` + +## Quick Start + +### Test Dynamic Inventory + +```bash +cd ansible +ansible-inventory --graph +``` + +You should see your servers grouped by labels. + +### Ping All Servers + +```bash +ansible all -m ping +``` + +### Run Setup Playbook + +```bash +# Full setup (common + docker + traefik) +ansible-playbook playbooks/setup.yml + +# Specific server +ansible-playbook playbooks/setup.yml --limit test + +# Dry run (check mode) +ansible-playbook playbooks/setup.yml --check +``` + +## Directory Structure + +``` +ansible/ +├── ansible.cfg # Ansible configuration +├── hcloud.yml # Hetzner Cloud dynamic inventory +├── playbooks/ # Playbook definitions +│ ├── setup.yml # Initial server setup +│ ├── deploy.yml # Deploy/update applications +│ └── upgrade.yml # System upgrades +├── roles/ # Role definitions +│ ├── common/ # Base system hardening +│ ├── docker/ # Docker + Docker Compose +│ ├── traefik/ # Reverse proxy +│ ├── zitadel/ # Identity provider +│ ├── nextcloud/ # File sync/share +│ └── backup/ # Restic backup +└── group_vars/ # Group variables + └── all.yml # Variables for all hosts +``` + +## Roles + +### common +Base system configuration and security hardening: +- SSH hardening (key-only auth, no root password) +- UFW firewall configuration +- Fail2ban for SSH protection +- Automatic security updates +- Timezone and locale setup + +**Variables** (`roles/common/defaults/main.yml`): +- `common_timezone`: System timezone (default: `Europe/Amsterdam`) +- `common_ssh_port`: SSH port (default: `22`) +- `common_ufw_allowed_ports`: List of allowed firewall ports + +### docker +Docker and Docker Compose installation: +- Latest Docker Engine from official repository +- Docker Compose V2 +- Docker daemon configuration +- User permissions for Docker + +### traefik +Reverse proxy with automatic SSL: +- Traefik v3 with Docker provider +- Let's Encrypt automatic certificate generation +- HTTP to HTTPS redirection +- Dashboard (optional) + +### zitadel +Identity provider deployment (see Zitadel Agent for details) + +### nextcloud +File sync/share deployment (see Nextcloud Agent for details) + +### backup +Restic backup configuration to Hetzner Storage Box + +## Playbooks + +### setup.yml +Initial server provisioning and configuration: +```bash +ansible-playbook playbooks/setup.yml +``` + +Runs roles in order: +1. `common` - Base hardening +2. `docker` - Container platform +3. `traefik` - Reverse proxy + +### deploy.yml +Deploy or update applications: +```bash +ansible-playbook playbooks/deploy.yml +``` + +Runs application-specific roles based on server labels. + +## Dynamic Inventory + +The `hcloud.yml` inventory automatically queries Hetzner Cloud API for servers. + +**Server Grouping:** +- By client: `client_test`, `client_alpha` +- By role: `role_app_server` +- By location: `location_fsn1`, `location_nbg1` + +**View inventory:** +```bash +ansible-inventory --graph +ansible-inventory --list +ansible-inventory --host test +``` + +## Common Tasks + +### Check Server Connectivity +```bash +ansible all -m ping +``` + +### Run Ad-hoc Command +```bash +ansible all -a "uptime" +ansible all -a "df -h" +``` + +### Update All Packages +```bash +ansible all -m apt -a "update_cache=yes upgrade=dist" +``` + +### Restart Service +```bash +ansible all -m service -a "name=docker state=restarted" +``` + +### Limit to Specific Hosts +```bash +# Single host +ansible-playbook playbooks/setup.yml --limit test + +# Multiple hosts +ansible-playbook playbooks/setup.yml --limit "test,alpha" + +# Group +ansible-playbook playbooks/setup.yml --limit client_test +``` + +## Development Workflow + +### Creating a New Role + +```bash +cd ansible/roles +mkdir -p newrole/{tasks,handlers,templates,defaults,files} +``` + +Minimum structure: +- `defaults/main.yml` - Default variables +- `tasks/main.yml` - Main task list +- `handlers/main.yml` - Service handlers (optional) +- `templates/` - Jinja2 templates (optional) + +### Testing Changes + +```bash +# Syntax check +ansible-playbook playbooks/setup.yml --syntax-check + +# Dry run (no changes) +ansible-playbook playbooks/setup.yml --check + +# Limit to test server +ansible-playbook playbooks/setup.yml --limit test + +# Verbose output +ansible-playbook playbooks/setup.yml -v +ansible-playbook playbooks/setup.yml -vvv # Very verbose +``` + +## Troubleshooting + +### "No inventory was parsed" +- Ensure `HCLOUD_TOKEN` environment variable is set +- Verify token has read access +- Check `hcloud.yml` syntax + +### "Failed to connect to host" +- Verify server is running: `tofu show` +- Check SSH key is correct: `ssh -i ~/.ssh/ptt_infrastructure root@` +- Verify firewall allows SSH from your IP + +### "Permission denied (publickey)" +- Ensure `~/.ssh/ptt_infrastructure` private key exists +- Check `ansible.cfg` points to correct key +- Verify public key was added to server via OpenTofu + +### "Module not found" +- Install missing Ansible collection: + ```bash + ansible-galaxy collection install + ``` + +### Ansible is slow +- Enable SSH pipelining (already configured in `ansible.cfg`) +- Use `--forks` to increase parallelism: `ansible-playbook playbooks/setup.yml --forks 20` +- Enable fact caching (already configured) + +## Security Notes + +- Ansible connects as `root` user via SSH key +- No passwords are used anywhere +- SSH hardening applied automatically via `common` role +- UFW firewall enabled by default +- Fail2ban protects SSH +- Automatic security updates enabled + +## Next Steps + +After initial setup: +1. Deploy Zitadel: Follow Zitadel Agent instructions +2. Deploy Nextcloud: Follow Nextcloud Agent instructions +3. Configure backups: Use `backup` role +4. Set up monitoring: Configure Uptime Kuma + +## Resources + +- [Ansible Documentation](https://docs.ansible.com/) +- [Hetzner Cloud Ansible Collection](https://github.com/ansible-collections/hetzner.hcloud) +- [Ansible Best Practices](https://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html) diff --git a/ansible/ansible.cfg b/ansible/ansible.cfg new file mode 100644 index 0000000..f787527 --- /dev/null +++ b/ansible/ansible.cfg @@ -0,0 +1,39 @@ +[defaults] +# Inventory configuration +inventory = hcloud.yml +host_key_checking = False +interpreter_python = auto_silent + +# Performance +forks = 10 +gathering = smart +fact_caching = jsonfile +fact_caching_connection = /tmp/ansible_facts +fact_caching_timeout = 86400 + +# Output +stdout_callback = yaml +bin_ansible_callbacks = True +display_skipped_hosts = False + +# SSH +remote_user = root +private_key_file = ~/.ssh/ptt_infrastructure +timeout = 30 + +# Roles +roles_path = ./roles + +[inventory] +# Enable Hetzner Cloud dynamic inventory plugin +enable_plugins = hetzner.hcloud.hcloud + +[privilege_escalation] +become = True +become_method = sudo +become_user = root +become_ask_pass = False + +[ssh_connection] +pipelining = True +ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no diff --git a/ansible/hcloud.yml b/ansible/hcloud.yml new file mode 100644 index 0000000..3f960b0 --- /dev/null +++ b/ansible/hcloud.yml @@ -0,0 +1,37 @@ +# Hetzner Cloud Dynamic Inventory Configuration +# Queries Hetzner Cloud API for server list at runtime +# Requires: HCLOUD_TOKEN environment variable + +plugin: hetzner.hcloud.hcloud + +# Optional: Filter by label +# label_selector: role=app-server + +# Group servers by labels +keyed_groups: + # Group by client label + - key: labels.client + prefix: client + separator: _ + + # Group by role label + - key: labels.role + prefix: role + separator: _ + + # Group by location + - key: location + prefix: location + separator: _ + +# Compose custom variables +compose: + ansible_host: ipv4_address + server_id: id + server_type: server_type + datacenter: datacenter + +# Create groups for all servers +groups: + # All Hetzner Cloud servers + hetzner_cloud: True diff --git a/ansible/roles/common/defaults/main.yml b/ansible/roles/common/defaults/main.yml new file mode 100644 index 0000000..ed6a0b6 --- /dev/null +++ b/ansible/roles/common/defaults/main.yml @@ -0,0 +1,42 @@ +--- +# Default variables for common role + +# Timezone +common_timezone: "Europe/Amsterdam" + +# SSH Configuration +common_ssh_port: 22 +common_ssh_permit_root_login: "prohibit-password" +common_ssh_password_authentication: "no" +common_ssh_pubkey_authentication: "yes" + +# UFW Firewall +common_ufw_default_incoming: "deny" +common_ufw_default_outgoing: "allow" +common_ufw_allowed_ports: + - { port: "22", proto: "tcp", comment: "SSH" } + - { port: "80", proto: "tcp", comment: "HTTP" } + - { port: "443", proto: "tcp", comment: "HTTPS" } + +# Automatic Updates +common_unattended_upgrades: true +common_auto_reboot: false # Manual control over reboots + +# Fail2ban +common_fail2ban_enabled: true +common_fail2ban_bantime: 3600 # 1 hour +common_fail2ban_findtime: 600 # 10 minutes +common_fail2ban_maxretry: 5 + +# System packages +common_packages: + - curl + - wget + - git + - vim + - htop + - net-tools + - ufw + - fail2ban + - unattended-upgrades + - apt-listchanges diff --git a/ansible/roles/common/handlers/main.yml b/ansible/roles/common/handlers/main.yml new file mode 100644 index 0000000..5ab7286 --- /dev/null +++ b/ansible/roles/common/handlers/main.yml @@ -0,0 +1,17 @@ +--- +# Handlers for common role + +- name: Restart SSH + service: + name: ssh + state: restarted + +- name: Restart unattended-upgrades + service: + name: unattended-upgrades + state: restarted + +- name: Restart fail2ban + service: + name: fail2ban + state: restarted diff --git a/ansible/roles/common/tasks/fail2ban.yml b/ansible/roles/common/tasks/fail2ban.yml new file mode 100644 index 0000000..5d725e9 --- /dev/null +++ b/ansible/roles/common/tasks/fail2ban.yml @@ -0,0 +1,20 @@ +--- +# Fail2ban configuration for SSH protection + +- name: Install fail2ban + apt: + name: fail2ban + state: present + +- name: Configure fail2ban defaults + template: + src: jail.local.j2 + dest: /etc/fail2ban/jail.local + mode: '0644' + notify: Restart fail2ban + +- name: Ensure fail2ban is running and enabled + service: + name: fail2ban + state: started + enabled: yes diff --git a/ansible/roles/common/tasks/firewall.yml b/ansible/roles/common/tasks/firewall.yml new file mode 100644 index 0000000..a3737f2 --- /dev/null +++ b/ansible/roles/common/tasks/firewall.yml @@ -0,0 +1,28 @@ +--- +# UFW firewall configuration + +- name: Reset UFW to default state + community.general.ufw: + state: reset + when: ansible_facts['distribution'] == 'Ubuntu' + +- name: Set UFW default policies + community.general.ufw: + direction: "{{ item.direction }}" + policy: "{{ item.policy }}" + loop: + - { direction: 'incoming', policy: '{{ common_ufw_default_incoming }}' } + - { direction: 'outgoing', policy: '{{ common_ufw_default_outgoing }}' } + +- name: Allow specified ports through UFW + community.general.ufw: + rule: allow + port: "{{ item.port }}" + proto: "{{ item.proto }}" + comment: "{{ item.comment }}" + loop: "{{ common_ufw_allowed_ports }}" + +- name: Enable UFW + community.general.ufw: + state: enabled + logging: 'on' diff --git a/ansible/roles/common/tasks/main.yml b/ansible/roles/common/tasks/main.yml new file mode 100644 index 0000000..f3584d9 --- /dev/null +++ b/ansible/roles/common/tasks/main.yml @@ -0,0 +1,30 @@ +--- +# Main tasks for common role - base system setup and hardening + +- name: Update apt cache + apt: + update_cache: yes + cache_valid_time: 3600 + +- name: Install common packages + apt: + name: "{{ common_packages }}" + state: present + +- name: Set timezone + community.general.timezone: + name: "{{ common_timezone }}" + +- name: Configure SSH hardening + include_tasks: ssh.yml + +- name: Configure UFW firewall + include_tasks: firewall.yml + +- name: Configure automatic updates + include_tasks: updates.yml + when: common_unattended_upgrades + +- name: Configure fail2ban + include_tasks: fail2ban.yml + when: common_fail2ban_enabled diff --git a/ansible/roles/common/tasks/ssh.yml b/ansible/roles/common/tasks/ssh.yml new file mode 100644 index 0000000..036924f --- /dev/null +++ b/ansible/roles/common/tasks/ssh.yml @@ -0,0 +1,23 @@ +--- +# SSH hardening configuration + +- name: Configure SSH daemon + lineinfile: + path: /etc/ssh/sshd_config + regexp: "{{ item.regexp }}" + line: "{{ item.line }}" + state: present + with_items: + - { regexp: '^#?PermitRootLogin', line: 'PermitRootLogin {{ common_ssh_permit_root_login }}' } + - { regexp: '^#?PasswordAuthentication', line: 'PasswordAuthentication {{ common_ssh_password_authentication }}' } + - { regexp: '^#?PubkeyAuthentication', line: 'PubkeyAuthentication {{ common_ssh_pubkey_authentication }}' } + - { regexp: '^#?PermitEmptyPasswords', line: 'PermitEmptyPasswords no' } + - { regexp: '^#?X11Forwarding', line: 'X11Forwarding no' } + - { regexp: '^#?MaxAuthTries', line: 'MaxAuthTries 3' } + notify: Restart SSH + +- name: Ensure SSH is running and enabled + service: + name: ssh + state: started + enabled: yes diff --git a/ansible/roles/common/tasks/updates.yml b/ansible/roles/common/tasks/updates.yml new file mode 100644 index 0000000..90ce22e --- /dev/null +++ b/ansible/roles/common/tasks/updates.yml @@ -0,0 +1,23 @@ +--- +# Automatic security updates configuration + +- name: Install unattended-upgrades + apt: + name: + - unattended-upgrades + - apt-listchanges + state: present + +- name: Configure unattended-upgrades + template: + src: 50unattended-upgrades.j2 + dest: /etc/apt/apt.conf.d/50unattended-upgrades + mode: '0644' + notify: Restart unattended-upgrades + +- name: Enable automatic updates + template: + src: 20auto-upgrades.j2 + dest: /etc/apt/apt.conf.d/20auto-upgrades + mode: '0644' + notify: Restart unattended-upgrades diff --git a/ansible/roles/common/templates/20auto-upgrades.j2 b/ansible/roles/common/templates/20auto-upgrades.j2 new file mode 100644 index 0000000..ea9891c --- /dev/null +++ b/ansible/roles/common/templates/20auto-upgrades.j2 @@ -0,0 +1,7 @@ +// Enable automatic package updates +// Managed by Ansible - do not edit manually + +APT::Periodic::Update-Package-Lists "1"; +APT::Periodic::Download-Upgradeable-Packages "1"; +APT::Periodic::AutocleanInterval "7"; +APT::Periodic::Unattended-Upgrade "1"; diff --git a/ansible/roles/common/templates/50unattended-upgrades.j2 b/ansible/roles/common/templates/50unattended-upgrades.j2 new file mode 100644 index 0000000..3f08683 --- /dev/null +++ b/ansible/roles/common/templates/50unattended-upgrades.j2 @@ -0,0 +1,28 @@ +// Automatic upgrade configuration +// Managed by Ansible - do not edit manually + +Unattended-Upgrade::Allowed-Origins { + "${distro_id}:${distro_codename}"; + "${distro_id}:${distro_codename}-security"; + "${distro_id}ESMApps:${distro_codename}-apps-security"; + "${distro_id}ESM:${distro_codename}-infra-security"; +}; + +// List of packages to not update +Unattended-Upgrade::Package-Blacklist { +}; + +// Automatically reboot if needed +Unattended-Upgrade::Automatic-Reboot "{{ 'true' if common_auto_reboot else 'false' }}"; + +// Reboot time if automatic reboot is enabled +Unattended-Upgrade::Automatic-Reboot-Time "03:00"; + +// Email notification (disabled by default) +// Unattended-Upgrade::Mail "root"; + +// Remove unused dependencies +Unattended-Upgrade::Remove-Unused-Dependencies "true"; + +// Automatically cleanup old kernels +Unattended-Upgrade::Remove-New-Unused-Dependencies "true"; diff --git a/ansible/roles/common/templates/jail.local.j2 b/ansible/roles/common/templates/jail.local.j2 new file mode 100644 index 0000000..9a29cef --- /dev/null +++ b/ansible/roles/common/templates/jail.local.j2 @@ -0,0 +1,18 @@ +# Fail2ban configuration +# Managed by Ansible - do not edit manually + +[DEFAULT] +bantime = {{ common_fail2ban_bantime }} +findtime = {{ common_fail2ban_findtime }} +maxretry = {{ common_fail2ban_maxretry }} + +# Email notifications (disabled by default) +# destemail = root@localhost +# sendername = Fail2Ban +# mta = sendmail + +[sshd] +enabled = true +port = {{ common_ssh_port }} +logpath = %(sshd_log)s +backend = %(sshd_backend)s diff --git a/docs/architecture-decisions.md b/docs/architecture-decisions.md index baa55ac..eb2a85b 100644 --- a/docs/architecture-decisions.md +++ b/docs/architecture-decisions.md @@ -798,6 +798,47 @@ infrastructure/ --- +## 15. Development Environment and Tooling + +### Decision: Isolated Python Environments with pipx + +**Choice:** Use `pipx` for installing Python CLI tools (Ansible) in isolated virtual environments. + +**Why pipx:** +- Prevents dependency conflicts between tools +- Each tool has its own Python environment +- No interference with system Python packages +- Easy to upgrade/rollback individual tools +- Modern best practice for Python CLI tools + +**Implementation:** +```bash +# Install pipx +brew install pipx +pipx ensurepath + +# Install Ansible in isolation +pipx install --include-deps ansible + +# Inject additional dependencies as needed +pipx inject ansible requests python-dateutil +``` + +**Benefits:** +| Aspect | Benefit | +|--------|---------| +| Isolation | No conflicts with other Python tools | +| Reproducibility | Each team member gets same isolated environment | +| Maintainability | Easy to upgrade Ansible without breaking other tools | +| Clean system | No pollution of system Python packages | + +**Alternatives Considered:** +- **Homebrew Ansible** - Rejected: Can conflict with system Python, harder to manage dependencies +- **System pip install** - Rejected: Pollutes global Python environment +- **Manual venv** - Rejected: More manual work, pipx automates this + +--- + ## Changelog | Date | Change | Author | @@ -807,4 +848,5 @@ infrastructure/ | 2024-12 | Switched from Terraform to OpenTofu (licensing concerns) | Pieter / Claude | | 2024-12 | Switched from HashiCorp Vault to SOPS + Age (simplicity, open source) | Pieter / Claude | | 2024-12 | Switched from Keycloak to Zitadel (Swiss company, GDPR jurisdiction) | Pieter / Claude | +| 2024-12 | Adopted pipx for isolated Python tool environments (Ansible) | Pieter / Claude | ``` \ No newline at end of file