Masto.hostFeb 23
There is currently an incident, and database servers 11, 22 and 23 stopped responding. Instances using those servers are down. I apologize for the disruption. I'm trying to gather more information and will update as soon as I know more.
Show threadMasto.hostFeb 23
OK, the problem has been identified in the data center: https://bare-metal-servers.status-ovhcloud.com/incidents/dgksqhl6lswj
[GRA4][Dedicated Servers] - Rooms GRA0404 - GRA0406 incident notification

Bare Metal Cloud's Status Page - [GRA4][Dedicated Servers] - Rooms GRA0404 - GRA0406 incident notification.

Show threadMasto.hostFeb 23
Database server 23 is responding again.
Database servers 11 and 22 still down.
Show threadMasto.hostFeb 23
Database server 11 is now also back online. Only database 22 still down.
Show threadNoelleFeb 23
@mastohost Should the checkmarks on https://status.masto.host/ be updating? They all show green despite the outage
Show threadMasto.host
@noelle the servers are still sending out the Heartbeat Monitor that's why it's green. Ideally, they shouldn't be green, but https://360monitoring.com/ reports them as up as long as there is a heartbeat being sent.
Sites & Servers Monitoring - 360 Monitoring

Sites & servers monitoring solutions by 360 Monitoring. Keep track of your online business assets with confidence, supported by world-class internal and external system monitoring.

360 Monitoring
Feb 23 at 4:34pmWeb
Show threadsoaprootFeb 23
@mastohost Ah yes, sometimes I have built a health check which includes a call to database, redis, and anything else essential. No idea about the details of doing that with Mastodon and 360monitoring, though. @noelle