Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.

I am still trying a few things, but I am not very optimistic that I can get access.

The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.

As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable 😓

I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing 😥

Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.

you are viewing a single comment's thread
view the rest of the comments

[–] PirateFrog@lemmy.dbzer0.com 16 points 3 days ago* (last edited 3 days ago)

It usually can be reset, it's entirely unexpected that a component that enables that functionality would randomly break. You're essentially implying that any admin who self hosts cannot ever leave their house for any length of time without being irresponsible because they cannot ensure with 100% certainly that a computer component won't fail while their gone, or a lightning storm won't fry something, or a flood won't harm the servers.

Like, dude, c'mon. We're volunteers, and poVoq is running this stuff for free. We're not an enterprise business with multiple staff on hand to swap out failed components 24/7 so that our 'customers' don't lose millions of dollars.

If anything, this only proves the strength of decentralized media, as we're able to swap to a functioning node and continue on our merry way while the other is being repaired. Like what an incredible and resilient system considering none of this is making anyone any money.