cross-posted from: https://lemmy.management/post/665809
I made this tool to help self-hosters, new admins, or smaller instances have more global and updated content on their instances.
This is the similar to Lemmy Community Seeder but is designed to be run periodically to capture new communities, and include EVERYTHING by default.
You don’t understand how federated replication works. It doesn’t occur on-demand when you read a post, it occurs when the instance hosting the community gets a post, comment, or vote. The federation load you place on other servers has nothing to do with how many users are on your instance or how much they read… it has everything to do with how many communities they subscribe to. This script is literally signing you up to proactively receive the firehose of every post and comment in the lemmyverse, without regard for what you actually look at.
I completely understand the idea of the app, and your confusion about how much load it generates is exactly why it’s such an irresponsible idea. If you want to fill the timeline of your small instance, do so by subscribing to specific communities you’re interested in until your timeline becomes active enough for you. Subscribing to 100 communities you care about will result in a very lively feed of stuff that is interesting to you, while generating a tiny percentage of the federation load this approach does. Carpet bombing the entire lemmyverse with subscriptions you cannot read is madness. It’s like writing a reddit app that downloads everything ever posted to reddit to your phone to save you the trouble of picking subreddits to follow. It’s bad for reddit, bad for your phone, bad for your isp, and a bad idea all around. If I were running a large instance, I’d defederate with any tiny instance I observed subscribing indiscriminately via this script. It’s abuse.
I hear ya and I’m open to learning, but your way I have to still use another instance to be exposed to new communities. In essence I can’t have a Reddit “all” page… Is that correct?
There’s no such thing as an “all” page in Lemmy or on Reddit.
lemmy.worldwe don’t see everything in the “all” feed. Beehaw defederated with us, and some new/small communities haven’t had anyone subscribe to them. These are missing from the “all” feed.On a private Lemmy instance you are the admin and must curate your own frontpage. Community discovery on Lemmy does kind of suck right now, hopefully it will get better over time. For now, Lemmyverse.net is a good place to discover new communities, and you can browse the incomplete “all” feed of a major instance without an account there. Another responder in this thread suggested creating a dummy account to subscribe to stuff you want to see in your all feed but not in your subscribed feed… and it’s fine to do that kind of thing liberally. But it should be a human selected list of finite length. Indiscriminate subscription is bad all around.
You’re not the first person to want an /r/all, https://lemmy.directory/ tried. You can still see the announcement at https://lemmy.world/post/21875. It is now, less than one month later… broken. I’ll leave you to speculate on why and how you plan to avoid whatever problem shut them down.
You should write something that detects indiscriminate subscribing and automatically defederates with them.
It was a lot of work, but this is what I did (not 100 communities, but enough). I browser all on lemmy.world for days, finding communities I wanted to subscribe to. Then I subscribed to those on my account on my private instance. When I want more communities, I go back to lemmy.world and find more.