I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site’s servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.
I have experienced this on my personal server as well (DDOS), but at the time didn’t know why, exactly.
This isn’t the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.
Random back off is an ancient and very well understood algorithm. This is not exactly rocket science.
No one said anything about random.
Nobody said the delay had to be fixed instead of random either…
Yes that’s the problem…?
“Random back off” is the name of a technique to avoid this exact problem.
Each instance generates a random number of (say) seconds between (say 15 to 120) and schedules the fetch for then. It is extremely unlikely that any two instances select the same second.
Yes I understand. I just don’t appreciate the demeaning tone this information was delivered in.