yopp1d ago
Oh fuck #postgres. In 18 they made a change in data path from `/var/lib/postgresql/data` to `/var/lib/postgresql/`.

EDIT:

I was wrong, it's not postgres change, it's change made in "official" docker image, which in fact is not official and maintained by "the PostgreSQL Docker Community". My apologies to pg people.

They "the PostgreSQL Docker Community" (not affiliated with Postgres in any way) did exactly zero checks so users will shoot themselves in a leg in two ways:

  • kinda benign: after 17 → 18 upgrade it won’t start
  • fatal: user mounted docker volume at the old pathContainer restarts and data gone (that’s me)

    • What the actual fuck. Why on earth you couldn’t do a simple fucking check that old directory exists and has data, but new directory is empty, and then start with directory with data spitting warnings that default has been changed and you need to do something about that.

    This is utter incompetence and giving exactly zero fucks about your user base

    Show threadAndreas Scherbaum
    @alex What is your environment or OS?
    Mar 23 at 11:56amWeb
    Show threadTomas Vondra1d ago
    @ascherbaum @alex Yeah. I wonder what packages are being used?
    Show threadyopp1d ago

    @tomasv @ascherbaum nothing special: dockerized environment, official docker image. I’ve deployed one of the services with pg18 but used default mount path to data. So I was unplesantly surprised to find empty database after 18.2 to 18.3 image upgrade. Old container was gc’ed because both pg and app healthchecks passed. Which is technically correct: app rolled bootstrap migration and went up in blank state and pg was indeed available. But data is gone. It was expected that there could be issues with 18, so it was first deployment with not that important data.

    Another another lesson that observability as useful as your risk assessment. Also lesson in trust in vendors: no matter how consistent in their changes they were in past, they absolutelly will do stupid things. That on me. I could’ve paid more attention that’s also on me.

    But doing change of this magnitude so careless and without any proper discussion of what can go wrong for downstream users is on the people who made this damn pull request merged.

    Worst part this change is purely about aesthetics and addresses exactly zero issues: pg sill gives no shit about operational complexity of major version upgrades. While I respect people around pg, database itself is an example of software that is plan hostile to users. It’s never joy, it always different from of pain.

    Show threadAndreas Scherbaum1d ago

    @alex @tomasv What does "official docker image" mean?

    If you refer to this:

    https://hub.docker.com/_/postgres

    This is kind of the official "Docker Postgres Image", but this is not maintained by the PG project.

    Blaming Postgres for this is the wrong audience.

    postgres - Official Image | Docker Hub

    The PostgreSQL object-relational database system provides reliability and data integrity.

    Show threadyopp1d ago

    @ascherbaum @tomasv

    🤦 thanks for pointing this out.

    My apologies for pg people. I stand corrected, it's maintained by "docker community" (what the fuck "official" means then)