Damus
Stefano Marinelli profile picture
Stefano Marinelli
@Stefano Marinelli

BSD.cafe and illumos Cafe "Barista", Founder and System Administrator, Unix enthusiast ( #FreeBSD, #OpenBSD, #NetBSD, #DragonflyBSD, #Illumos and #Linux ), with a keen eye for everything happening in this world and the fascinating beings that populate it. I enjoy #music, #photography, and, of course, #technology.

Most of my posts will self-destruct after 6 months.

"I Solve Problems" - https://it-notes.dragas.net/2024/10/03/i-solve-problems-eurobsdcon/

Relays (1)
  • wss://relay.ditto.pub – read & write

Recent Notes

Stefano Marinelli profile picture
One of the positive things that happened yesterday was reconnecting with a group of colleagues I hadn't spoken to in a long time.

They said something interesting:

"For years, many of us thought OpenBSD was a bit paranoid. That they were going too far. Then came the flood of CVEs, RCEs, and all the rest, and OpenBSD has been holding up remarkably well.

They weren't overreacting. They had simply understood a few things better than the rest of us."

That last sentence brought a satisfied smile to my face.

#OpenBSD #RunBSD #IT #SysAdmin #Security
Stefano Marinelli profile picture
Once again, the drain was full of non-compostable paper towels. Every time they wash their hands, they probably throw the paper towels into the toilet.

The result is a blockage in the building's main sewer outlet, which causes everything to come rushing back up through the bathroom closest to the end of the line.

And guess whose bathroom happens to be closest to the end of the line?

For now, they've cleared the blockage, but we'll need to address the root cause.

The suspects are either the university students living on the top floor or a very elderly person who moved into the building about a year ago and who, apparently, occasionally experiences moments of mental confusion.

Neither scenario will be easy to deal with. And then there's the cleaning to take care of. Again.

I need lasagna. Definitely.

#Crapturday
Stefano Marinelli profile picture
End of the day. I head back into the fog, stop by the office for the first time in a week and... here we go again. Another sewage backup.

Not as bad as in January, but still quite a mess.

The building manager got straight on it, and they might even come to clear the drains tomorrow.

The prime suspects, judging by what we found on the office floor, are the students on the second floor. Again. They've already been warned, but apparently they still haven't quite grasped what you can and cannot flush down the drains.

And so, once again, it's time to bring back that dear old hashtag:

#Crapturday
Stefano Marinelli profile picture
Highlight of the day: yet another WordPress site left to rot since 2024.

I had pointed out the problem months ago, but the client had simply replied with a "thank you."

Today, an employee emails me saying he "wanted to install some updates, but PHP isn't up to date," followed by a whole explanation of how PHP should be updated. Systemd was involved, obviously.

Spoiler: PHP was already up to date, running the latest 8.3 release, inside a dedicated FreeBSD jail for that client's website.

I take a look, and the site is compromised. WordPress 6.6.x. I immediately tell them to contact the web agency responsible for maintaining it, otherwise I'll have to take the site offline.

Shortly afterwards, they reply that the web agency says *I'm* supposed to handle it.

I've never touched that website, apart from providing the hosting. It's full of completely unnecessary plugins, and it's basically just a brochure website.

I inform the client. I wait.

The client is desperate. He asks me, "Please, help us. Our reputation is at stake."

That's when I have one of my five-minute bursts of fury.

I clone the entire site into a static version using an old plugin that still worked (something I had developed years ago with another developer). I make a few adjustments and show it to the client.

"Thank you."

From today onwards, static only for those kinds of sites. And whenever they need changes, they'll ask me.

In today's world, we simply cannot afford to compromise on security anymore, even if that means coming across as overly strict or categorical.

#SysAdmin #IT #WebDevelopment #Webdev
Stefano Marinelli profile picture
"Of course it was DNS!"

But this time, it was my fault. Again.

I upgraded the BSD Cafe instances to Ruby 4.0.10 (installed via rbenv). Everything seemed fine, until a friendly message arrived: "Hey, the media aren't working properly!"

I checked, and @nprofile1q... was right: HTTP 503.

I started investigating, but couldn't really figure out WHY it wasn't working.

And yes, it was DNS.

For media delivery, we have several remote Varnish caches around the world, to improve latency and reduce the load on the origin SeaweedFS server.

On the Mastodon servers, I also have an /etc/hosts entry pointing media.bsd.cafe directly to SeaweedFS, bypassing those caches.

Except... I'd only added the IPv4 address!

Apparently, after the Ruby upgrade, connections started using IPv6 instead, resolving media.bsd.cafe to one of the remote Varnish caches. And that's where things went wrong.

I fixed the mapping, and... we're back!

Sometimes the problem isn't DNS itself. It's the person who configured it.

That would be me. 😉

#IT #SysAdmin #TIL
Stefano Marinelli profile picture
Me: “Static websites are great for security! No CMS, no database, no PHP, no plugins. Just a few innocent HTML files. What could possibly go wrong?”

nginx: “Hold my CVE.”

Me: “Well, technically, I said the website was secure. I never said anything about the web server.”

And this, kids, is why we can’t have nice things. Not even an index.html.

#nginx #SysAdmin #Security #StaticSites
1😂1😱1