My new daily backup script, pg_dump with zstd compression level 19.

docker exec \ ak-postgres-1 \ pg_dump -U umeyashiki umeyashiki_akkoma | \ nice -n 19 \ ionice -c 3 \ chrt --idle 0 \ zstd -T0 -19 --rsyncable -q > "$BACKUP_DIR/db_latest.sql.zst";

Today I learned something new about the scheduling priority. Since zstd compression is very CPU-intensive, set it to low priority so it doesn’t slow the entire system down during compression.

Commands that precede zstd here are:

  • nice -n 19 [cmd]
  • ionice -c 3 [cmd]
  • chrt --idle 0 [cmd]

By chaining nice, ionice, and chrt together before the zstd command, the script forces the compression process to run with the absolute lowest possible priority for both the CPU and the disk.

  • nice 19 is the lowest priority CPU priority.
  • ionice with class 3 means idle. A program running with idle I/O priority will only get disk time when no other program has asked for disk I/O for a defined grace period.
  • chrt --idle 0: Set scheduling policy to SCHED_IDLE (scheduling very low priority jobs).

References:

  • man 1 chrt
  • man 1 nice
  • man 1 ionice

#linux #docker #zstd #postgresql #sched

@yuka I may suggest you to dump it in custom format as it also retains the index. It’s also compressed.
@yonle my pg_dump already preserves the indexes. Did I miss something?

@yuka i mean the index data itself.

still though, on irc the maintainer suggests using custom. It will save you more space & time to dump

@yonle Sorry for a stupid question. But what is index data?
@yuka  can you just try dump in custom format and see it yourself
@yonle Can you share your DB backup command?
@yuka pg_dump -U postgres -d akkoma --format custom -f akkoma-DD-MM-YYYY.pgdump
@yonle @yuka The custom format does not contain the index data, only the CREATE INDEX statement.