Linux

Backup? What?

i do backups of my home folder with Vorta, tha uses borg in the backend. I never tried restic, but borg is the first incremental backup utility i tried that doesnt increase the backup size when i move or rename a file. I was using backintime before to backup 500gb on a 750gb drive and if I moved 300gb to a different folder, it would try to copy those 300gb again onto the backup drive and fail for lack of storage, while borg handles it beautifully.

as an offsite solution, i use syncthing to mirror my files to a pc at my fathers house that is turned on just once in a while to save power and disc longevity.

My systems are all on btrfs, so I make use of subvolumes and use brkbk to backup snapshots to other locations.

This looks a bit like borgbackup. It is also versioned and stores everything deduplicated, supports encryption and can be mounted using fuse.

Borg Backup, whenever I feel like it - usually monthly.

Since most of the machines I need to backup are VMs, I do it by the means of hypervisor. I'd use borg scheduled in crontab for physical ones.

I've found that the easiest and most effective way to backup is with an rsync cron job. It's super easy to setup (I had no prior experience with either rsync or cron and it took me 10 minutes) and to configure. The only drawback is that it doesn't create differential backups, but the full task takes less than a minute every day so I don't consider that a problem. But do note that I only backup my home folder, not the full system.

For reference, this is the full line I use: sync -rau --delete --exclude-from='/home//.rsync-exclude' /home/ /mnt/Data/Safety/rsync-myhome

".rsync-exclude" is a file that lists all files and directories I don't want to backup, such as temp or cache folders.

(Edit: two stupid errors.)

My conclusion after researching this a while ago is that the good options are Borg and Restic. Both give you incremental backups with cheap timewise snapshots. They are quite similar to each other, and I don't know of a compelling reason to pick one over the other.

• Offline Backup on 2 separate HDD/SSD
• Backup on HDD within my desktop pc
• Backup offsite with restic to Hetzner Storage Box

I'm curious, is there a reason why noone uses deja-dup? I use it with an external SSD on Ubuntu and (receently) Mint, where it comes pre-installed, and did not encounter Problems.

Borg to a NAS.

500GB of that NAS is "special" so I then rsync that to a 500GB old laptop hdd, of which is is duplicated again to another 500GB old laptop hdd.

Same 500GB rsync'd to Cloud Server.

My KVM hosts use “virsh backup begin” to make full backups nightly.

All machines, including the KVM hosts and laptops, use rsync with --link-dest to create daily incremental versioned backups on my main backup server.

The main backup server pushes client-side encrypted backups which include the latest daily snapshot for every system to rsync.net via Borg.

I also have 2 DASs with 2 22TB encrypted drives in each. One of these is plugged into the backup server while the other one sits powered off in a drawer in my desk at work. The main backup server pushes all backups to this DAS weekly and I swap the two DASs ~monthly so the one in my desk at work is never more than a month or so out of date.

I recently switched to Kopia for my offsite backup solution.

It's apparently one of the faster options, and it can be set up so that the files of the differential backups are handled by a repository server on the offsite end, so file management doesn't need to happen over the network at a snails pace.

The result is a way to maintain frequent full backups of my nextcloud instance, with almost no downtime.

Nextcloud only goes into maintenance mode for the duration of a postgres database dump, after which the actual file system backup occurs using a temporary btrfs snapshot, containing a frozen filesystem at the time of the database dump.

I keep all of my documents on a local server so all that is on any of my computers is software. So if I need to reinstall Linux I cab just do it without wording about losing anything.

I have a server with a RAID-1 array, that makes daily, weekly, and monthly read only btrfs snapshots. The whole thing (sans snapshots) is sync'd with syncthing to two rPi's in two different geographic locations.

I know neither raid nor syncthing are "real" backup solutions, but with so many copies of the files living in so many locations (in addition to my phone, laptop, etc.) I'm reasonably confident its a decent solution.

I use external drive for my important data and if my system is borked (which never happen to me) I just reinstall the OS

I use Duplicacy to backup to my TrueNAS server. Crucial data like documents are backed up a second time to my GDrive, also using Duplicacy. Sadly it's a paid solution, but it works great for me.

Borg daily to the local drive then copied across to a USB drive, then weekly to cloud storage. Script is triggered by daily runs of topgrade before I do any updates

I created a script that I dropped into /etc/cron.hourly which does the following:

1. Use rsync to mirror my root partition to a btrfs partition on another hard drive (which only updates modified files).
2. Use btrfs subvolume snapshot to create a snapshot of that mirror (which only uses additional storage for modified files).
3. Moves "old" snapshots into a trash directory so I can delete them later if I want to save space.

It is as follows:

#!/usr/bin/env python
from datetime import datetime, timedelta
import os
import pathlib
import shutil
import subprocess
import sys

import portalocker

DATETIME_FORMAT = '%Y-%m-%d-%H%M'
BACKUP_DIRECTORY = pathlib.Path('/backups/internal')
MIRROR_DIRECTORY = BACKUP_DIRECTORY / 'mirror'
SNAPSHOT_DIRECTORY = BACKUP_DIRECTORY / 'snapshots'
TRASH_DIRECTORY = BACKUP_DIRECTORY / 'trash'

EXCLUDED = [
    '/backups',
    '/dev',
    '/media',
    '/lost+found',
    '/mnt',
    '/nix',
    '/proc',
    '/run',
    '/sys',
    '/tmp',
    '/var',

    '/home/*/.cache',
    '/home/*/.local/share/flatpak',
    '/home/*/.local/share/Trash',
    '/home/*/.steam',
    '/home/*/Downloads',
    '/home/*/Trash',
]

OPTIONS = [
    '-avAXH',
    '--delete',
    '--delete-excluded',
    '--numeric-ids',
    '--relative',
    '--progress',
]

def execute(command, *options):
    print('>', command, *options)
    subprocess.run((command,) + options).check_returncode()

execute(
    '/usr/bin/mount',
    '-o', 'rw,remount',
    BACKUP_DIRECTORY,
)

try:
    with portalocker.Lock(os.path.join(BACKUP_DIRECTORY,'lock')):
        execute(
            '/usr/bin/rsync',
            '/',
            MIRROR_DIRECTORY,
            *(
                OPTIONS
                +
                [f'--exclude={excluded_path}' for excluded_path in EXCLUDED]
            )
        )

        execute(
            '/usr/bin/btrfs',
            'subvolume',
            'snapshot',
            '-r',
            MIRROR_DIRECTORY,
            SNAPSHOT_DIRECTORY / datetime.now().strftime(DATETIME_FORMAT),
        )

        snapshot_datetimes = sorted(
            (
                datetime.strptime(filename, DATETIME_FORMAT)
                for filename in os.listdir(SNAPSHOT_DIRECTORY)
            ),
        )

        # Keep the last 24 hours of snapshot_datetimes
        one_day_ago = datetime.now() - timedelta(days=1)
        while snapshot_datetimes and snapshot_datetimes[-1] >= one_day_ago:
            snapshot_datetimes.pop()

        # Helper function for selecting all of the snapshot_datetimes for a given day/month
        def prune_all_with(get_metric):
            this = get_metric(snapshot_datetimes[-1])
            snapshot_datetimes.pop()
            while snapshot_datetimes and get_metric(snapshot_datetimes[-1]) == this:
                snapshot = SNAPSHOT_DIRECTORY / snapshot_datetimes[-1].strftime(DATETIME_FORMAT)
                snapshot_datetimes.pop()
                execute('/usr/bin/btrfs', 'property', 'set', '-ts', snapshot, 'ro', 'false')
                shutil.move(snapshot, TRASH_DIRECTORY)

        # Keep daily snapshot_datetimes for the last month
        last_daily_to_keep = datetime.now().date() - timedelta(days=30)
        while snapshot_datetimes and snapshot_datetimes[-1].date() >= last_daily_to_keep:
            prune_all_with(lambda x: x.date())

        # Keep weekly snapshot_datetimes for the last three month
        last_weekly_to_keep = datetime.now().date() - timedelta(days=90)
        while snapshot_datetimes and snapshot_datetimes[-1].date() >= last_weekly_to_keep:
            prune_all_with(lambda x: x.date().isocalendar().week)

        # Keep monthly snapshot_datetimes forever
        while snapshot_datetimes:
            prune_all_with(lambda x: x.date().month)
except portalocker.AlreadyLocked:
    sys.exit('Backup already in progress.')
finally:
    execute(
        '/usr/bin/mount',
        '-o', 'ro,remount',
        BACKUP_DIRECTORY,
    )

I use Pika Backup (GUI that uses Borg Backup on the backend) to back up my desktop to my home server daily, then overnight that server has a daily backup using Borg to a Hetzner Storage Box. It's easy to set it and forget it (other than maybe verifying the backups every once in a while), and having that off site back up gives me peace of mind.

Most of my data are on 2x16TB HDDs running an mdraid1 and then I backup it all to a usb drive with Borg Backup.
The os.qcow2 files live on my m.2 NVMe and are manually backuped to the mdraid1 before running the borg backup.
I should automate the borg backup but currently I just do it manually a few times a month.
Would also like to have two usb drives and keep one offline in another part of the house but that's another future project.

I recently bought a storagebox from Hatzner and set up my server to run borgmatic every day to backup to it.

I've also discovered that Pika Backup works really well as a "read only" graphical browser for borg repos.

My work flow is pretty similar to yours:

For my desktop and laptops: systemd timer and service that backups every 15 minutes using restic to my NAS.

For my NAS : daily backup using restic + ZFS snapshots.

All restic backups are then uploaded daily to Backblaze B2.

Timeshift for snapshots and deja backups for files

Nice try, mister ransonware attacker hacker!

I currently use rclone with encryption to iDrive e2. I’m considering switching to Backrest, though.

I originally tried Backblaze b2, but exceeded their API quotas in their free tier and iDrive has “free” API calls, so I recently bought a year’s worth. I still have a 2 year Proton subscription and tried rclone with Proton drive, but it was too slow.

There’s nothing saved on my system I couldn’t afford to lose. All my work stuff is saved in Google Drive for better or worse. I have a few small files in a personal Proton Drive that I backup manually. I wipe my own system a few times a year and I rarely ever save anything first. Honestly very refreshing to live your life like that. Other than my cat, pretty much all my possessions could disappear tomorrow and I’d get over it pretty quickly.

I use borg the same way you describe. Part of my nixos config builds a systemd unit that starts a backup on various directories on my machine at midnight every day. I have 2 repos: one to store locally and on a cloud backup provider (borgbase) and another thats just stored locally. That is, another computer in my house. That local only is for all my home media. I havent yet put the large dataset of photos and videos on the cloud or offsite.

for my server I use proxmox backup server to an external HDD for my containers, and I back up media monthly to an encrypted cold drive.

For my desktop? I use a mix of syncthing (which goes to the server) and windows file history(if I logged into the windows partition) and I want to get timeshift working I just have so much data that it's hard to manage so currently I'll just shed some tears if my Linux system fails

rsync 😪

I rsync ~/ to a USB nub. A no brainer.