Mastodawn

Lemmy world will be upgraded to 0.18.3 today at 20:00 CET

https://lemmy.world/post/2370025

[Done] Lemmy world was upgraded to 0.18.3 today (2023-07-30) - Lemmy.world

Update The upgrade was done, DB migrations took around 5 minutes. We’ll keep an eye out for (new) issues but for now it seems to be OK. Original message We will upgrade lemmy.world to 0.18.3 today at 20:00 UTC+2 (Check what this isn in your timezone [https://www.timeanddate.com/worldclock/fixedtime.html?msg=Lemmy.world+upgrade+to+lemmy+18.3&iso=20230730T20&p1=16]). Expect the site to be down for a few minutes. ““Edit”” I was warned it could be more than a few minutes. The database update might even take 30 minutes or longer. Release notes for 0.18.3 can be found here: https://github.com/LemmyNet/lemmy/blob/main/RELEASES.md [https://github.com/LemmyNet/lemmy/blob/main/RELEASES.md] (This is unrelated to the downtimes we experienced lately, those are caused by attacks that we’re still looking into mitigating. Sorry for those)

afoutopatisa Jul 30, 2023

Any idea if 18.3 will be able to provide more security?

woelkchen Jul 30, 2023

github.com/LemmyNet/lemmy/blob/main/RELEASES.md#m…

lemmy/RELEASES.md at main · LemmyNet/lemmy

🐀 A link aggregator and forum for the fediverse. Contribute to LemmyNet/lemmy development by creating an account on GitHub.

GitHub

Nato Boram Jul 30, 2023

This version brings major optimizations to the database queries, which significantly reduces CPU usage. There is also a change to the way federation activities are stored, which reduces database size by around 80%. Special thanks to @phiresky for their work on DB optimizations.

The federation code now includes a check for dead instances which is used when sending activities. This helps to reduce the amount of outgoing POST requests, and also reduce server load.

In terms of security, Lemmy now performs HTML sanitization on all messages which are submitted through the API or received via federation. Together with the tightened content-security-policy from 0.18.2, cross-site scripting attacks are now much more difficult.

phoenix591 Jul 30, 2023

in some ways: they now sanitize input so things like the xss attack a while ago is much much harder.

will it solve the attacks they’re experiencing? nope.

Redjard Jul 30, 2023

Doesn’t it drastically improve performance? I’d say that will help with ddos, more requests can slip through before it affects performance

Antik 👾Jul 30, 2023

Release notes for 0.18.3 can be found here: github.com/LemmyNet/lemmy/blob/main/RELEASES.md

lemmy/RELEASES.md at main · LemmyNet/lemmy

🐀 A link aggregator and forum for the fediverse. Contribute to LemmyNet/lemmy development by creating an account on GitHub.

GitHub

Ruud Jul 30, 2023

Thanks, added to the post

curryandbeans Jul 30, 2023

Thanks for all the fine work, dudes

ren (a they/them)Jul 30, 2023

Thank you for the heads up!!

2pm EST for us silly Freedum Unit folks.

cybervseas Jul 30, 2023

EDT actually, not EST.

ren (a they/them)Jul 30, 2023

oh yeah, forgot about that. Could’ve just put ET.

Ruud Jul 30, 2023

Phone Home

ren (a they/them)Jul 30, 2023

Lol. Word is the update gives a boost to smaller communities. Looking forward to this!

Flat Pluto Society Jul 30, 2023

2PM Entertainment Tonight

woelkchen Jul 30, 2023

Will calckey.world be renamed as well, as upstream abandoned the calckey name?

Coelacanth Jul 30, 2023

I’m curious about this as well. I’ve looked into Firefish a bit and it looks interesting, but I can’t seem to follow Lemmy communities from the flagship instance Firefish.Social.

Works on calckey.world, though.

Ruud Jul 30, 2023

Calckey.world also runs Firefish 1.0

Coelacanth Jul 30, 2023

Firefish.social was upgraded to 1.0.something yesterday I believe. Maybe something broke? On Firefish.Social trying to follow a Lemmy community ends up in just an endless spinning wheel and “processing” while it works normally on calckey.world.

Curious.

Ruud Jul 30, 2023

Good to know, I’ll skip this update for now…

Coelacanth Jul 30, 2023

Where is the best place to report this to the Devs? I don’t have a GitHub account.

Ruud Jul 30, 2023

I’m in contact with him, I’ll let him know.

Coelacanth Jul 30, 2023

Great! It’s a very interesting project, I’m much more excited by it than Mastodon.

Blaze (he/him)Jul 30, 2023

It looks much nicer to be honest

Coelacanth Jul 30, 2023

It does, and the Antenna feature is really clever and useful.

Ruud Jul 30, 2023

Not renamed. A new server will be started where ppl can migrate to. Yet to determine a new name as firefish.world is taken…

zkfcfbzr Jul 30, 2023

Add controversial ranking
Change logic for determining comment default language
Add infinite scroll user option

Neat. I hope the comment language logic change means the default won’t be “Undetermined” anymore.

Ulu-Mulu-no-die Jul 30, 2023

Sorry for those

It’s not your fault :) We know you admins are working really hard to keep the server as stable as possible.

JuvenoiaAgent Jul 30, 2023

Don’t humanize them, they’ll expect us to always treat them with respect! /s (obvs 🙄)

ghariksforge Jul 30, 2023

good luck!

Godnroc Jul 30, 2023

A website which should let you check what time that is for your local time zone; although I used Central Europe Summer Time (CEST) instead of Central Europe Time (CET) and I’m not sure if that’s right.

8:00 PM 20:00 CEST to Local Time -- TimeBie

8 PM ( 20:00 ) Central European Summer Time to Your Local Time and Worldwide Time Conversions

RustedSwitch Jul 30, 2023

see what time this is in your time zone

Event Time Announcer - Lemmy.world upgrade to lemmy 18.3

Event Time Announcer shows time for Lemmy.world upgrade to lemmy 18.3 in locations all over the world. In Amsterdam it happens on Sonntag, 30. Juli 2023, 20:00:00.

Ruud Jul 30, 2023

Thanks, will put this in the post

quaddo Jul 30, 2023

Fwiw, it can be helpful to call out the date for such changes. Preferably in YYYY-MM-DD (ISO 8601).

While it’s helpful to link to an off-site timezone converter tool (thanks for that, btw), “today” can be a different date, depending on where in the world you are. For example, Japan, Australia, and New Zealand.

Ruud Jul 30, 2023

Good point

TheBananaKing Jul 30, 2023

Even better is “when this post is N hours old” :)

quaddo Jul 30, 2023

As someone who has had to grind through heaps of logs over the years, from systems in various timezones, from products that disagreed on the ‘best’ datetime format, I’ve become a fan of adopting ISO 8601 as much as possible. For personal systems such as a laptop, that’s a different story. But if I’m spinning up an EC2 instance in us-west-2 or a VM in Central Europe, I avoid the whole “err, what TZ is this in, or should even be in?” decision-making process and just run with WHO CARES IT’S SET TO UTC NOW LET’S MOVE ON ALREADY 😀

And not that anyone here is likely to care, but here’s a quick shout out to lnav - The Logfile Navigator for grinding on system logs (for systems where something like Prometheus or whatever hasn’t been proactively set up).

The Logfile Navigator

The Logfile Navigator, lnav for short, is an advanced log file viewer for the small-scale.

The Logfile Navigator

Rai Jul 30, 2023

ISO8601 is the only way to go.

GONADS125 Jul 30, 2023

I’m sure the attacks are just as annoying for you guys as they are to us, if not more. Appreciate the update.

cpo Jul 30, 2023

Great work. I prefer some downtime to twAtter anyday.

cerberus Jul 30, 2023

Great! Thanks for keeping things up to date and running!

Kokesh Jul 30, 2023

Is there way to finally fix the bug causing user stats to reset every time?

Ruud Jul 30, 2023

Those stats are not a Lemmy feature, they are a feature of the app you use. (If I understand correctly).

AlmightySnoo 🐢🇮🇱🇺🇦Jul 30, 2023

This version brings major optimizations to the database queries, which significantly reduces CPU usage. There is also a change to the way federation activities are stored, which reduces database size by around 80%.

Is it me or is the 80% figure just insane? Are there any benchmarks to see how fast this has become versus say Lemmy 0.18.2 on a very large instance?

miket Jul 30, 2023

Is it me or is the 80% figure just insane?

Not really, you’d be surprised how often systems are bloated all because of a single option, character, etc. Most developers don’t start optimizing until much later in the software’s lifecycle. Often enough, it is easily overlooked. That’s why code reviews are needed often with fresh pair of eyes.

Just to set the expectations, reducing database size or CPU usage does not necessarily mean it is faster but it does mean there’s more free capacity on the servers with handle more users at the same speed. More importantly; they may reduce costs on the smaller indie instances that doesn’t need to buy larger server instances.

Hopefully, we’ll continue to see more of these optimizations.

AlmightySnoo 🐢🇮🇱🇺🇦Jul 30, 2023

I believe if the backend doesn’t have to write as much data then you’ll have less I/O operations so it should IMO have an impact on the overall speed of Lemmy (unless all of those operations are done asynchronously). Same for the reduced CPU usage, it could allow for more stuff in parallel.

miket Jul 30, 2023

Speed/pref and capacity are two separate things. I/O has nothing to do with the size of the database. You can write 100TB per second into the database and choose to only store 1TB of content. That does not mean the app is writing 1TB per second, it is still writing 100TBps.

They said they changed how the activities data is stored by 80%, here’s one of the changes they made: github.com/LemmyNet/lemmy/issues/3103

As you can see, the issue here is that they were storing a lot of data in the activities table that is not needed, it was only meant for debug purposes. So, they split up the data into two and not store the other data as it isn’t needed; they’re still writing these data the same as before. One part is used to ensure they don’t re-process the same data but this is the same thing they were doing before this change.

In addition, they’ve limited how long the data is retained for 3 months, which is a separate job they run to remove data.

All of this has zero impact on the users using the app right now. The main benefit is for instance admins with limited storage. One might say the system slows down if there’s not enough space but that is still the same case here with this MR, it will still slow down.

Configurable Activity Cleanup Duration · Issue #3103 · LemmyNet/lemmy

Did you check to see if this issue already exists? Is this only a single feature request? Do not put multiple feature requests in one issue. Is this a question or discussion? Don't use this, use ht...

GitHub

2014MU69 Jul 30, 2023

Wait does that mean posts older than 3 months gets automatically deleted? Isn’t that kinda bad? Being able to find years old posts is an important part of Reddit and pretty much all social media.

miket Jul 31, 2023

Nobody said anything about posts. Activities could mean extra metadata like “someone joined”, “someone changed”, posted, edited, etc.

Most of the time, these isn’t required after a short while.

miket Jul 30, 2023

Funnily enough, this is the feature that can speed up the performance by doing less calls:

The federation code now includes a check for dead instances which is used when sending activities. This helps to reduce the amount of outgoing POST requests, and also reduce server load.

randint Jul 30, 2023

Minor suggestion: write your time zones like UTC+2 and not CEST. I’m pretty sure most people outside Europe don’t know what the time zone CEST is. Yes, you provided a link that helpfully converts the time to the users’ local time zone, but sometimes it’s nicer to be able to know something without having to click into a link.

Ruud Jul 30, 2023

OK, updating the post

randint Jul 30, 2023

Wow, you actually changed it! I was kind of shocked when I came back to my feed seeing UTC+2 in the title. Thanks! :D

Ruud Jul 30, 2023

Yep, I like good tips like these. :-) Thanks

ydieb Jul 30, 2023

Super nitpick. Can we just use 1800 utc? Then everyone can convert to their zone directly. *signed me in a CEST zone.

tburkhol Jul 30, 2023

I keep one of my spare clocks set to UTC for exactly this.

💡dim Jul 30, 2023

yeh, the actual utc time is much more straightforward than +2

Bappity Jul 30, 2023

it would be nice if Lemmy had support for formatting Unix timestamps in whatever timezone whoever is looking at it is in, like discord

Poplar?Jul 30, 2023

The problem with open-source for me is reading ideas like this and having to deal with the frustration of not knowing how to write it myself.

sep Jul 30, 2023

www.youtube.com/watch?v=-5wpm-gesOY

Enjoy ;)

The Problem with Time & Timezones - Computerphile

A web app that works out how many seconds ago something happened. How hard can coding that be? Tom Scott explains how time twists and turns like a twisty-tur...

YouTube

zahel Jul 30, 2023

Nice! Just updated my own self-hosted instance, went super smooth and quick.