Just put the site behind a cache, like Cloudflare, and set your cache control headers properly?
They mention that they are already using Cloudflare. I’m confused about what is actually causing the load. They don’t mention any technical details, but it does kinda sound like their cache control headers are not set properly. I’m too lazy to check for myself though…
deleted by creator
If caching is properly configured, the cache (Cloudflare) will see thousands of requests, but the VPS should only see one request.
This should be front and center, caching won’t be able to make up for that…
Of course it will, cloudflare is in front of it, they can definitely handje this traffic as long as itsfoss bothers to set correct caching headers for cloudflare to use. That’s the entire point of cloudflare…
deleted by creator
I think they just advertised how trivial it would be to take their website down…
I always downvote posts with titles like this. Here’s Why -
same. read more to find out!
Direct link to article:
https://news.itsfoss.com/mastodon-link-problem/
TL;DR:
When you share a link on Mastodon, a link preview is generated for it, right?
With Mastodon being a federated platform (a part of the Fediverse), the request to generate a link preview is not generated by just one Mastodon instance. There are many instances connected to it who also initiate requests for the content almost immediately.
And, this “fediverse effect” increases the load on the website’s server in a big way.
Does Lemmy not cause this issue? Other federated software was not mentioned in the article at all.
So the preview should be federated as well?
How many requests are we actually talking about here, though? Is that better or worse than everyone clicking the link?
They say it’s fediversal in the comments on Mastodon.
Gotta respect boycotting Cloudflare on principle… but also, why?
It’s an interesting and frustrating problem. I think there are three potential ways forward, but they’re both flawed:
-
Quasi-Centralization: a project like Mastodon or a vetted Non-Profit entity operates a high-concurrency server whose sole purpose is to cache link metadata and Images. Servers initially pull preview data from that, instead of the direct page.
-
We find a way to do this in some zero-trust peer-to-peer way, where multiple servers compare their copies of the same data. Whatever doesn’t match ends up not being used.
-
Servers cache link metadata and previews locally with a minimal amount of requests; any boost or reshare only reflects a proxied local preview of that link. Instead of doing this on a per-view or per-user basis, it’s simply per-instance.
I honestly think the third option might be the least destructive, even if it’s not as efficient as it could be.
As I understand it, 3) already happens. What causes the load is that each connected instance is also loading and caching the preview.
-
So why doesn’t a random follower posting a link on Mastodon cause server load issues, but a popular follower does?
i mean it’s solid training but they do realise it’s not limited to mastodon, right?
the slashdot effect has been around for years
There’s another reason I don’t share “It’s FOSS” links anywhere: this should have been a github issue but it’s turned into a clickbaity headline. Every othe article coming out of “It’s FOSS” is either low effort, sensationalist, or both.
