kaesebrotI've been having some huge performance problems with my Matrix Synapse server and I think that might be related to me joining the Synapse Admins room (15k members). I already tried leaving the room again through FluffyChat (which runs into a "Oops something went wrong" error) and then via matrix-nio for python calling the API, that call goes through successfully but returns a 504 Gateway Timeout...what else can I do? Can I just delete the room from the DB? #matrix #matrixsynapse #synapse
Har-Har Links@das_kaesebrot @element in my experience
- patience and wait for it to sort itself out, possibly crashing a couple times
- scale the hardware vertically
- scale horizontally, i.e. decouple the federation logic running the (uncancelable?) room join into a separate worker process, unblocking the main process/other workers
- patience and wait for it to sort itself out, possibly crashing a couple times
- scale the hardware vertically
- scale horizontally, i.e. decouple the federation logic running the (uncancelable?) room join into a separate worker process, unblocking the main process/other workers
@HarHarLinks @element
- I've been waiting for 6 months on and off 😅
- Imho the hardware should be more than sufficient (4 cores, 8 GiB of RAM for the DB, 4 cores, 12 GiB for Synapse, all running on NVMe storage)
- I might try that, thanks
- I've been waiting for 6 months on and off 😅
- Imho the hardware should be more than sufficient (4 cores, 8 GiB of RAM for the DB, 4 cores, 12 GiB for Synapse, all running on NVMe storage)
- I might try that, thanks
@das_kaesebrot @element ah, 1. wasn't quite clear. Server logs might help?
Element@das_kaesebrot 15k user room should generally be fine. what db and server spec are you using?
@element Hi, thanks for your reply. Here's my setup:
Synapse server:
- podman container inside an unprivlieged LXC container
- 12 GiB DDR4 memory, 12 GiB swap
- filesystem on an NVMe SSD
- presence disabled
- retention: using the example config from https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#retention
- version: ghcr.io/element-hq/synapse:v1.139.0
- cp_min: 3, cp_max: 5
- reverse proxy for SSL termination in front of the server
- 4 cores of an AMD Ryzen 2700
(1/2)
@element
Database server:
- dedicated KVM-based virtual machine
- same physical host
- PostgreSQL 17
- shared_buffers: 1GB
- max_wal_size = 2GB
- min_wal_size = 80MB
- NVMe SSD as underlying filesystem
- connection Synapse -> PSQL via TCP/IP through an isolated network
- 4 cores of an AMD Ryzen 7 2700
I am the only user on my server instance. Starting a client leads to an extremely high load on my database (IP redacted):
@element It usually takes a few minutes until all states are synced for the chats I am part of. I am only part of a few small chats aside from the Synapse Admins one (#synapse:matrix.org)
I just want to leave that chat so I can finally use my server in peace but even using the API client I'm only running into a timeout (please see the original toot)