Codeberg.orgJul 3
Codeberg is currently unreachable, apparently due to a network outage at our provider. We'll hit you up as soon as we know more.
Show threadCodeberg.orgJul 3

From a quick ping on various services, it seems the issue is not with our volunteer-driven rackspace provider, but affecting the whole datacenter we're located in.

Which might be good news, because it then doesn't rely on volunteers to fix the issue, but on professional staff 😉.

Show threadCodeberg.orgJul 3
We're back. It looks like all servers (all datacenter?) lost power, but we have no confirmed information on the background yet.
Show threadSirWumpus 👾🍁Jul 3
@Codeberg That's not a good sign for a data centre you'd expect to provide 5-9s uptime. No industrial UPS, no generator? If a professional data centre looses power such that client machines shutdown (unless you're in the charity / staff rack that isn't covered) that's a worry.
Show threadCodeberg.orgJul 3
@sirwumpus The DC claims to have redundant power feeds into the building, UPS and generators. However, it's not the first time I have seen UPS systems fail, even in large datacenters. Often during maintenance and human error :) ~f
Show threadSirWumpus 👾🍁

@Codeberg True, though when I was with TASFrance they would run the generator monthly and check the UPS charges and holds a charge. An unplanned outage during maintenance seems odd - has to be human error (like cooking toast or falling onto power mains terminals).

Thank you for the informative updates.

Jul 3 at 10:50amWeb
Show threadCodeberg.orgJul 3
@sirwumpus The outage immediately rang a bell for this German blog article from uberspace https://blog.uberspace.de/2021/06/ausfall-am-29-06-2021/ that was also once affected by a power outage. In short, during maintenance of the UPS, some component broke and resulted in a short interruption of 70ms, which was enough to cause headache recovering all network equipment for several hours.
Ausfall am 29.06.2021

Show threadCodeberg.orgJul 3
@sirwumpus
From the uptime report of our servers, they rebooted immediately after the power was lost, so the outage was probably very short (a few minutes at most, probably much less). But they were not reachable, probably because it took so long until all network equipment etc recovered.
Show threadSirWumpus 👾🍁Jul 3
@Codeberg Maybe someone was sucking all the power 🔋 out of the room 😁
Show threadCodeberg.orgJul 3
@sirwumpus Still no official information from the provider, but BCIX has the outage reported as well https://status.bcix.net/maintenance-windows/52472
BCIX Status - [BCIX] Incident at BCIX site A36s Speedbone

SUMMARY: We would like to inform you about the following Incident: Incident: 2025-6715 SEVERITY : high Device : edge-a36s-21.bcix.de, edge-a36s-31.bcix.de Location : Alboinstrasse 36 (Speedbone) Start Time : 2025-07-03 09:30:00 UTC (2025-07-03 11:30:00 Europe/Berlin) End Time : 2025-07-03 09:40:00 UTC (2025-07-03 11:40:00 Europe/Berlin) AFFECTED NETWORKS: ASN IPv4 IPv6 PEER ========================================================================= 112 193.178.185.112 2001:7f8:19:1::70:1 www.as112.net 250 193.178.185.23 2001:7f8:19:1::fa:1 AS250 917 193.178.185.165 2001:7f8:19:1::395:1 Misaka.io 9002 193.178.185.64 2001:7f8:19:1::232a:1 RETN 10310 193.178.185.110 2001:7f8:19:1::2846:1 Yahoo 10310 193.178.185.111 2001:7f8:19:1::2846:2 Yahoo 12732 193.178.185.97 2001:7f8:19:1::31bc:3 dhosting 12741 193.178.185.46 2001:7f8:19:1::31c5:1 netia 13020 193.178.185.124 2001:7f8:19:1::32dc:1 CCC Event 20562 193.178.185.147 2001:7f8:19:1::5052:1 Open Peering 20676 193.178.185.60 2001:7f8:19:1::50c4:1 Plusnet 29670 193.178.185.50 2001:7f8:19:1::73e6:1 INBERLIN 29670 193.178.185.51 2001:7f8:19:1::73e6:2 INBERLIN 31078 193.178.185.142 2001:7f8:19:1::7966:1 ase-network 34758 193.178.185.180 2001:7f8:19:1::87c6:1 Axera 39614 193.178.185.85 2001:7f8:19:1::9abe:1 Dawico 44194 193.178.185.82 2001:7f8:19:1::aca2:2 Freifunk 50472 193.178.185.105 2001:7f8:19:1::c528:1 ccc 198225 193.178.185.131 2001:7f8:19:1:0:3:651:1 MDDSL GmbH 201701 193.178.185.74 2001:7f8:19:1:0:3:13e5:1 ffrl 206313 193.178.185.81 2001:7f8:19:1:0:3:25e9:1 Freifunk Nordwest 206813 193.178.185.13 2001:7f8:19:1:0:3:27dd:1 Freifunk Guetersloh 211286 193.178.185.176 2001:7f8:19:1:0:3:3956:1 measurement.network 215250 - 2001:7f8:19:1:0:3:48d2:1 V4LESS DESCRIPTION: A power failure occurred, leading to disruptions across multiple systems. This incident affected peering services, resulting in downtime. Immediate investigation and remediation efforts are underway to restore functionality If you have any questions regarding this incident, please don't hesitate to contact us. Thank you -- BCIX NOC BCIX Management GmbH Albrechtstr. 110 12103 Berlin +49 30 2332195 55 noc@bcix.de https://www.bcix.de/

BCIX Status