Backups with Borg#

system status mariadb status postgres status userdata status

Le système et les données utilisateur sont sauvegardées une fois par jour à 5h. Les sauvegardes de CLUB1 sont basées sur Borg et Borgmatic. Elles sont ainsi stockées de manière dédupliquée et chiffrée à plusieurs endroits dont un dépôt off-site offert par Etienne Le Louët (serveur Debian auto-hébergé à Paris).

These are incremental backups with the following retention policy:

  • 7 daily backups

  • 4 weekly backups

The oldest backup kept is therefore about 1 month old and each day of the last week is accessible. In case of hardware failure, a maximum of 24 hours of data will be lost. The successful completion of backups is verified using the Healthchecks.io service

Backups are divided into four groups:

  1. System for server configuration and application files.

  2. MariaDB for MariaDB databases.

  3. Postgres for PostgreSQL databases.

  4. Userdata for the data of the personal space (home folder).

See also

The journal article: Sauvegardes (in French)

Restoration process#

This section describes the total restoration of the system by reinstalling all software from zero.

1. Installation du système#

Language      : English
Location      : Europe/France
Locale        : en-US
Hostname      : club1.fr
Root password :
First user    : Any username/password
Partitionning : Guided (All files in one partition) + 16Go Swap
Mirrors       : deb.debian.org
Popcon        : yes
Software      : SSH server, Standard system utilities

2. Étapes préparatoires#

This phase must be done with the user root.

2.1. General preparations#

dpkg-reconfigure locales

Then select locals en-GB, en-US and fr-FR in UTF-8. Mainly for not having error messages in the following steps.

apt install vim borgbackup screen
mkdir ~/.ssh
echo "nas.jeanpierre.moe ecdsa-sha2-nistp256 \
AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdzQyGgLl9YgPVGey\
CxRnOdswraNagx6xSFROaxjTjrM2WwXlFhxLXOxounGKUYBA2l5McCgNh8cneJjKOunmQ=" \
     >> ~/.ssh/known_hosts

2.2. Pre-restauration of specific files#

borg list club1@nas.jeanpierre.moe:backups/system

Then replace ARCHIVE with the desired archive identifier from the above list. The most recent one is at the bottom left.

cd / && borg extract club1@nas.jeanpierre.moe:backups/system::ARCHIVE \
    etc/shadow etc/passwd etc/group etc/gshadow root/apt-packages.txt \
    etc/borgmatic.d/system.yaml root/.ssh

Two passwords will be requested: that of SSH then that of the archive.

3. Installation des paquets#

This phase must be done with the user root.

mkdir /etc/systemd/system-preset
echo 'disable *' > /etc/systemd/system-preset/disable-all.preset
apt install extrepo
extrepo enable grafana
extrepo enable jellyfin
extrepo enable matrix
apt update
# screen (~= 20min)
cat /root/apt-packages.txt | DEBIAN_FRONTEND=noninteractive xargs apt install --no-install-recommends -y

4. Restauration totale#

This phase must be done with the user root.

4.1. system archive and etc folder#

cp /etc/fstab /root

# screen (~= 40min)
borgmatic extract --archive latest --repository=club1@nas.jeanpierre.moe:backups/system --destination /

cd /etc
git add . && git reset --hard master
etckeeper init
cp /root/fstab /etc
git add -p && git commit

systemctl daemon-reload

4.2. Some special files#

mkdir /var/spool/postfix/opendkim && chown opendkim:postfix /var/spool/postfix/opendkim
ln -s /usr/local/src/hydrian/TTRSS-Auth-LDAP/plugins/auth_ldap /usr/share/tt-rss/www/plugins/

4.3. LDAP database#

systemctl stop slapd
slapadd -l /root/ldap_data.ldif && chown -R openldap:openldap /var/lib/ldap
systemctl restart slapd nslcd

4.4. email aliases database#

newaliases
postmap hash:/etc/aliases_senders
chmod g+w /etc/aliases*.db

4.5. MariaDB databases#

systemctl start mariadb
# screen (~= 20min)
borgmatic restore --archive latest --repository=club1@nas.jeanpierre.moe:backups/mariadb
mariadb < /root/mariadb-users.sql

4.6. Postgres databases#

systemctl start postgresql
sudo -u postgres psql -c 'create role root with superuser login;'
sudo -u postgres psql -c 'create database root with owner root;'
# screen (~= 6h)
borgmatic restore --archive latest --repository=club1@nas.jeanpierre.moe:backups/postgres

4.7. Finalization#

rm -rf /home/*
reboot

Software#

Borg#

Incremental backups software in CLI, based on deduplication. It also compresses and encrypts saved data. — Site, Sources

Borgmatic#

CLI software to automate backup management with Borg. In particular, it is able to backup Postgres and MariaDB databases. — Site, Sources