Aller au contenu

MaDada infrastructure description

"Infrastructure" is a bit of a pompous name for it, as it is fairly simple and limited :)

TL;DR;

Software platform

Servers

Our servers are all VPSs provided by Gandi. We use 2 servers:

  • a production machine (the one behind https://madada.fr) which hosts all the production services:

  • the application itself (alaveteli)

  • web server (nginx)
  • mail server (postfix)
  • pop server (dovecot) for passing incoming emails to alaveteli
  • database (postgresql)

  • a staging server (https://dadastaging.okfn.fr) which duplicates the production server's configuration as much as possible (same software versions, same file structure, etc...). The only differences are secrets, server names and network settings.

Deployment

We deploy the service using an ansible playbook.

This page won't try to explain ansible in totality, their docs should help you with this if you're not familiar with the system. Instead here are only some of our specifics:

  • The main setup playbook is in a role called alaveteli, under ansible/roles/alaveteli, and reading through ansible/roles/alaveteli/tasks/main.yml should give you a step by step view of how we do our deployment.
  • The role relies on 2 other roles:
  • anxs.postgresql to setup the database
  • geerlingguy.swap to setup a small swap file to help avoid memory issues during some deployment steps (search for that name in the playbook to see more details)

The goal of our ansible setup is to run from our gitlab CI pipeline, so that nobody should need to mess with the servers directly, nor deploy from their own machines.

The code in this repo (under the ansible folder) should hold all the info needed to rebuild the server from scratch.

Our goal is to avoid touching the servers directly, and instead change config entirely through the playbooks, which act as documentation of our work, helps with traceabililty, and reproducibility.

Secrets management

Secrets are stored in 2 different ways: - ansible vault files for most of them, which are under ansible/group_vars/*/vault.yml - gitlab CI environment variables for the ones that are needed before decrypting the vault files (eg. the vault password)

All secret variables in the playbook are prefixed with vault_ to help identify them. To make things easier to follow, we also copy them to a non-prefixed variable with the same name (eg. vault_somepassword -> somepassword) in the main.yml config file where all variables are set. This has two consequences:

  • there are no vault_* variables floating around the code,
  • all variables are set in one place, making it easier to find the relevant ones.

DNS

All DNS configuration is done via Gandi's DNS servers.

Backups

Backups are done for two elements:

  • postgresql database, using wal-g. It is setup to do Point-In-Time-Restore (PITR), meaning the database can be restored to any point in its history, with a resolution of 1 minute.
  • raw emails are saved as files and backed up using restic.

All backups are done off-site, to an S3 compatible block storage hosted by Exoscale in Geneva (the servers are in Paris).

All configuration details are in this repository, and most easily found by searching for "walg" and "restic" respectively.

As an overview, both are configured using an envdir, which converts files into environment variables for the target program. The name of the file determines the env var's name, and the file's content is the value of the resulting variable. All of these are defined in the playbook. For instance, search for db_backup_walg_envdir. The actual values are defined in the playbook variables.


Dernière mise à jour: 2022-06-21