We currently are experiencing a network issue at our new Colo DC and as a result #Universeodon is currently offline.
I'm waiting on a reply from the on-site remote hands to see if they can resolve it, given I only replaced two of the switches last week I'm starting to wonder if our router itself is at fault so I'll have to contact the manufacturer and see if they can arrange a replacement.
I haven't had any updates as of yet, I have submitted a request with the routers manufacturer for them to ship a replacement so I can swap it out and hopefully minimise any downtime when I then need to do that.
Apologies, I'm not actually feeling very well at the moment, I'm pretty shattered and full of cold so the idea of driving for nearly 2 hours to go and restart some shit isn't thrilling me very much today. I'm hoping the remote hands can take a look shortly and we can at least get things back to normal and I can then schedule a visit when I'm not feeling quite so crappy to replace the device as well as perform some other upgrades while I'm there.
Just chased our colo provider and they've confirmed they have an engineer en-route to the site now to take a look for me (I haven't got access myself anyway so even if I do drive up there one of their folks has to be there to give me a fob).
I'm hoping to get an update from them relatively shortly and get the networking operational and to see what's gone wrong. I have a few guesses as to what might be the problem and am going to have to book a half day to go up and make some changes to get things running smoother. I've also ordered an additional switch which I need to get installed at some point either way as it'll increase the network bandwidth between our storage server and our app servers which is something we're going to start to run into problems with anyway in the next few weeks.
We are now back up and running, The team at the colo have restarted the networking gear and I've now restarted all the servers to fix the network storage connectivity.
I've requested a replacement device from the manufacturer (Ubiquiti) as it's pretty much exactly a week since the last set of issues and a different dead device.
I've also got some new hardware now on order which I'm hoping will mean even if this device does fail in the future the impact should be somewhat less hopefully.
My apologies again for the disruption here, I appreciate it's a pain.
Ryan Wild
Onj 🎶
Djembro, RO, supports 🇺🇦🇬🇪
Bill Denneen