This site is currently struggling to handle the amount of new users. I have already upgraded the server, but it will go down regardless if half of Reddit tries to join.
However Lemmy is federated software, meaning you can interact seamlessly with communities on other instances like beehaw.org or lemmy.one. The documentation explains in more detail how this works. Use the instance list to find one where you can register. Then use the Community Browser to find interesting communities. Paste the community url into the search field to follow it.
You can help other Reddit refugees by inviting them to the same Lemmy instance where you joined. This way we can spread the load across many different servers. And users with similar interests will end up together on the same instances. Others on the same instance can also automatically see posts from all the communities that you follow.
Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements. That way the server will only go down sooner.
You must log in or # to comment.
Sadly, I feel like the Fediverse, based on ActivityPub, was fundamentally designed wrong for scaling potential. I do like Fedi and I like ActivityPub, but I think instances should not have to be responsible for all of this:
- Owning user accounts
- Exclusively host communities
- Serving local and remote users webpages and media
- Never going down, as this results in users and content becoming unavailable
Because servers “own” the user accounts and communities it’s not trivial for users to switch to a different instance, and as instances scale their costs go up slightly exponentially.
I wish the Fediverse from the beginning was a truly distributed content replication platform, usenet-style or Matrix-style, and every instance would add additional capacity to the network instead of hosting specific communities or users.
I guess it’s a bit too late for a redesign now… Perhaps decentralized identifiers will take us there in some form in the future.
Not sure why you reference Matrix, which has even worse scaling issues as it indeed tries to replicate nearly the entire network on every server.
The Fediverse is really just working how the general web does, just with some standardized API for websites to interact. It’s not perfect, but it works and has proven to be relatively scalable.
It sounds a bit like you had a bit too much of the Bluesky cool-aid, which indeed replicates nearly all of the mistakes of Matrix and makes it impossible to scale via small community owned servers instead of big company owned data-centers (which might be by design?).
Well yeah, point taken that replicating everything everywhere and forever might be impossible. But I do believe at a minimum my identity should be portable and accessing Fedi (ie. in microblogging: posting and viewing a feed of the latest posts of my follows) should be decoupled from which instance I pick to access the Fediverse.
I don’t particularly like how owners of instances which grew are now essentially locked in to having to spend 100s or 1000s of dollars a month keeping their now expensive instances running and providing service. This is a bad place to be for a platform ran by volunteers. Letting instance owners scale their service down as well as up would be ideal. But this requires at least decentralized identity, and at best some form of content hosting redundancy…
It’s easy to say the current architecture of Fedi works when it’s still small. Your instance has 139 users… That’s not intended as a slight. Hosting instances is good and I applaud you for it! But I wish it were easier to more equally share the load once the platform becomes more popular.
Is there any group of devs that work on this issue that you know of? I’d be interested in looking into it.
No. And I think it’s a really hard problem. poVoq was right to call me out on full replication being a bad move, because duplicating all content on every server is obviously inefficient. But a solution in-between, with decentralization and redundancy, is probably a very complex challenge. Doesn’t seem impossible, but very complex network protocols rarely seem to succeed.
Edit: Sorry I was still thinking about some fabled perfect protocol. But if you’re looking into decentralized identifiers, W3 is working on one approach. It’s not something I have seen used anywhere or integrated with ActivityPub yet, but that could be the future I’m hoping for. Probably.
It almost sounds like you’re describing RAID 5 of content across fediverse servers.
Something like that. But also with fully decentralized identity. So all content is signed by a keypair which is local to the user, and can be used to access Fedi through arbitrary instances. Probably I am too wishful.
I like the idea of decentralizing identity. One of the oddest things about the current fediverse is how closely tied accounts are to servers that host specific content. From the server’s perspective it would be like everything’s posted anonymously except all the messages are pgp signed.
But how would the system handle user customization settings? Things like blocked users or subscribed topics. Would that all need to be stored locally in your browser and parsed by the arbitrary instance you’re using?
And what if some instances want to refuse hosting certain content on the network. Maybe there’s some way defederating instances could account for that.
I could envision a 2nd class of server, running something like OAuth/OIDC, which handles the authentication into any Lemmy instance (or better yet, any ActivityPub based instance).
This server would also be self-hostable, and provide only authentication services, so it would be rather lightweight. But would help reduce the load on the content servers.
I feel like for decentralized identity maybe a page could be taken out of Blockchain. Ethereum ledger is duplicated in its entirety on every host, and there are L2s that help spread the load but roll up to the L1. If identity could be attached to something like that, each person has a key to identify themselves. Identity I think would be best separated from all content related to the identity, the user could choose a server to host that data, as well as back it up with like a shared user data backup agreement between a few servers in case one dies. It’d be very similar to raid but the data only needs to be on 2-3 servers. I suppose community data could be the same. I can envision when a new server joins the federation, it could be auto assigned to share with 2-3 similarly sized communities with algorithms making sure there aren’t any closed groups of sharing (a->b&c, b->a&c, c->a&b) wouldn’t ever want that. They all seems to be the most reasonable solution in my head.
While it might not be too late for that update, it would require some reconciliation to happen. There’s the potential for multiple users and communities of the same name across servers that would need to be considered.
Following a decentralized identifier would essentially be like following a unique public key. Their screen name is just some text which can be anything. The bigger problem is the overall infrastructure of Fedi which is very much based on lists of user@domain … this no longer trivially works if the “user” you follow could be posting from anywhere. It doesn’t seem unsolvable, just kind of difficult to imagine with the momentum behind Fedi as it is right now.
Yeah, the fact that the user auth and permission models were intentionally left out of the W3C spec initially really ended up locking ActivityPub into particular dialects and patterns that are now proving problematic for scaling.
I’m not sure it’s 100% too late for a redesign, though. The committee is still active and the Fediverse could still theoretically grow by an order of magnitude or two. Does that seem likely right this minute? No, but sometimes that kind of vision is what an ecosystem needs.
Thank you for posting this, as you seem to have more knowledge of the underlying protocols than i have. Had some visions of server takedowns and meltdowns. I have little idea how the protocol works but i’d imagine that account migration/replication would not be such a big deal to implement, and communities are already being replicated, no? As a work-around.
I applied for a few other instances but this one came through first. Your downfall is being too good compared to the competition.
That’s how I wound up here too.
I’ll be honest. I only applied for this one but that was because I had (still have) no idea what I’m doing lol
First post for me!
Sorry, I applied and got approved here. Still waiting to hear back from beehaw…
I’m really digging this UI compared to Reddit, but I am 99.9% a mobile user via the native Reddit app (don’t @ me!)
I am very tempted to setup my own instance. Wondering what resource usage looks like for an instance.
Yeah, I think I have two accounts (I registrated in a community and then came here and had to create another one because I couldn’t log in). It’s kind of confusing for people who are not as tech savy as myself.
Well, my understanding is your user exists on whatever instance you signed up on. You could technically create users on every single instance, but that is not necessary. You only need one user to exist somewhere, and then you can subscribe to, and post to communities on other instances.
For example: from lemmy.ml, if you search for
!gaming@beehaw.orgyou can then open the sidebar and subscribe to, and post to, the gaming community on beehaw.org with your lemmy.ml user.!gaming@beehaw.org is not the same community as !gaming@lemmy.ml
For example: from lemmy.ml, if you search for !gaming@beehaw.org you can then open the sidebar and subscribe to, and post to, the gaming community on beehaw.org with your lemmy.ml user.
This was really helpful, thank you!
An easy way to understand this is that instances are like email providers. You can sign up on Gmail, but still email someone using Outlook or something else.
That analogy makes a lot of sense. Very helpful to new users
This is one of the biggest hurdles to get into Lemmy. I consider myself quite tech savvy but I am at a stage of my life that I cannot read hundreds of page of documentation just to use a forum.
There need to be a way to seamlessly move people from instance to another without them having to do it themselves or at the least a way way shorter documentation that goes to the point in one page.
I’ve only been here a day or so, but having to search for the community doesn’t seem that bad? It’s almost exactly like searching for a subreddit to join.
/r/subredditturns into!community@address, like !gaming@beehaw.orgAnd once you are on that community you can open the sidebar and subscribe (join in Reddit terms).
I just created https://lemmy.film if that would be useful for anyone.
Sorry for contributing towards this by registering but I’m very appreciative of the work being done to facilitate this community. I hope to see Lemmy grow with the negative direction other platforms are taking.
IMHO, selecting an instance is definitely the biggest user experience problem Lemmy has at the moment. New users who are unfamiliar with the platform are going to pick the biggest instances, and that’s going to create performance problems.
We’ll need to prioritize work on instance browsing. Lemmy has outgrown the experience over at join-lemmy.org. If I could wave a magic wand, instance browsing and onboarding would have a way to show instance capacity / performance, a way to categorize and filter instances, and a way to recommend instances based upon interests. That would probably help to spread people out more evenly.
No problem just make a pull request with your changes.
You probably don’t want my code if you want a stable platform. ;)
That said, I dig what y’all are doing, and I’m veteran experience / interaction designer who’s been around the block for a few decades. So I might be able to find some time to mockup some experience concepts and or help to run user tests with audiences that your curious about.
I’m more of a backender myself, but I think some UX mockup would go a long way in getting this improved.
There’s a website I highly recommend called fediverse observer, it doesn’t really go based on interest, but it has some other factors it uses and I really like it.
I think that there should be some meaningful way to “preview” aspects of one instance that may make it more attractive than another instance to a new user. I just joined lemmy.world today simply because it seemed the most generic. Onboarding process could use some work; https://lemmy.world/post/37906 is great at explaining it but people will only really see it for the first time once they join…
Also I have no clue if that second link works. ¯\(ツ)/¯
deleted by creator
if you’re a registered user on a server, when you click [Communities], there you can see
- [local] communities - Those created on your server
- [all] communities - Those local and also those already federated to your server
You can subscribe to a community of any server which your server can federate with. The list of connected servers you can find via the /instances link at the bottom of the page.
There’s an easy to use community search tool here https://browse.feddit.de/
If you’ve found a community you like to follow, translate the original URL to a federated URL You do this by putting the community URL of the original server in the search bar; e.g.
(This search functionality is available in the web interface, but not yet available in the Jerboa app)
The result will list the federated URL. A federated URL has the form:
https://<your server>/c/<community-id>@<other server>Visiting the federated link, and clicking [Subscribe] will make that community be federated to your server from now on. Your subscribed community will now also be listed under the [all] communities listing on your server.
Is there a newbie guide written up already somewhere? This is exactly the sort of things it needs, which should be linked to new signups.
Documentation etc can be found from their github page
Sure it can be found there, but pointing newbies to github would be too demanding.
At the bottom of every page, there you find a link to the documentation pages.
Sure, its a bit wordy… lacking in images with step by step instructions. If I had time I’d put something together with screenshots and figure out git and submit a pull request… maybe eventually.
Thank you, this is suitable for pointing people here who have this veryFAQ. Could you please add a line, that this search functionality is not yet available in the Jerboa app?
Ah ok I see, thanks for the reply.
Maybe a developer is better suited to answer that question. @nutomic@lemmy.ml i’d assume there’s a difference in load in the backend and load on the frontend.
No. I am on my own little single-user instance and I can follow, vote, post and reply anywhere from here. It’s just a little awkward sometimes because you have to learn how to paste URLs in the search box, and until you subscribe there will be some missing content. But once you get past that, everything works.
Nope. You can subscribe/post/comment on any community on any instance. There is one small seam though: if you’re the first person to subscribe from your instance, you need to put in the full URL of the community (https://lemmy.ml/c/gaming, for example) to pull it into your instance.
After that, everybody on the same instance as you will see it when searching for communities just like it was local.
EDIT: Oh, forgot to mention: make sure the search is set to “All”, not “Communities” when you do this.
deleted by creator
-
Technically yes, but it’s not even vaguely in the same ballpark. If I’ve understood the devs talking about the optimization issues (I could be wrong! Just my limited understanding) the big performance hit is in the local feed. That means being on another instance takes a gigantic amount of the load off, even if you’re still accessing the same community.
-
If lemmy.ml is down, so are all the communities hosted there. All communities not on lemmy.ml would still be up.
-
Correct the performance problem now is all from local users (visiting lemmy.ml in their browser or app).
-
If lemmy.ml goes down, other instances still have full mirrors of them. Users there can interact with their local mirror as usual, and other users can see those interactions. However these would not be federated to other instances (lemmy.ml is responsible for announcing community posts to followers). However federated actions are retried a few times so it might federate later.
-
-
I would be happy to use another instance but my account is on this one. Is there a way to migrate an account, or perhaps “link” accounts on multiple instances somehow?
AFAICT no. There is an open issue on the Lemmy GitHub repo. In general, all ActivityPub services I’ve used have this same account stratification problem.
I think this is key. Have the possibility to move an account to another instance or have it spread out somehow. This would also secure the account in case an instance dies for some reason.
I would appreciate this as well. Besides the flood of users issue, this server’s theme (Marxist-Leninist) doesn’t mesh with my politics. I created my account in the early days of Lemmy, so I have an extensive history that I am loath to sacrifice.
Wait isn’t lemmy.ml a general purpose server?
It is, but unlike elsewhere, ML folks are usually not banned or defederated here, only if they break rules (like insulting other users).
is it possible to move an existing profile to a new server, like on Mastodon? or I need to create a new one and “start over”?
Right now, there is no import/export. It’s a known useful feature, but the devs have no time to work on it (I’ve been following all the optimization work they’ve been doing on github, I don’t know if they sleep). You’ll have to start over atm, sorry.
thanks for the quick answer!
That’s a bummer but it is good to know.
Currently you have to start over.
Is scaling the server a largely financial issue, or not? @nutomic@lemmy.ml
could you reasonably confidently say that you could 10x the amount of users for something like 1000$/mo on liberapay?
If so, would you mind setting a “goalpost” for the community to help lift the financial burden?I think they said they’re at the highest tier of their provider. May need to migrate to a different provider and get a beefier setup.
Do Lemmy instances not scale horizontally?
In theory, they can. But it depends on how it’s deployed.
From my cursory look at the deployment docs, Lemmy’s default deployment option is via docker. It relies on a postgreSQL server, which may or may not scale horizontally depending on the admin’s choice of implementation. For example, a deployment on AWS using Aurora would theoretically utilize auto-scaling.
I haven’t personally deployed an instance so, grain of salt.
They do, but I’m not sure how well, I’m not a dev, and have no programming knowledge, so looking at the documentation looks like arcane hieroglyphs.
I’m pretty sure I read a comment about it from one of the devs, but can’t recall the fine details of the conversation.
I’m going to set up a general purpose instance tomorrow with the intention of handling a relatively large number of users. The main problem is choosing a domain!
Naming things is one of the two most difficult issues in IT, alongside cache validation and off-by-one errors.
I’m getting the following error reading this post: “item at index 2 does not exist”
Should I post this on stack overflow or some other Lemmy help community?
Which frontend are you using?
choosing the name for my instance was easy. programming related? programming.dev it is!
There are only two hard things in CS: naming things, caching, and off-by-one errors.
I already said that
Look at that, you sure did. I missed the “two hard things”. Wasn’t even drunk. 🤷
I name everything as var1 var2 etc.
var37.social incoming
I was also contemplating setting up a new instance for this. I have 100s of gigs of unused ram, CPUs on idle and a 10gbit connection looking for something to do. The only issue I couldn’t figure out was the name. I own itjust.works was thinking of something clever subdomain to use with it. I’m glad I’m not the only one with this issue
sh.itjust.works
I did it! https://sh.itjust.works
Credits go to you for the naming
Lol awesomesauce. I just made an account, I’ll use it as my main instance for a while. Let’s hope we can survive reddit hug of death 2.0 in July!
@autisticaudioguy lol same, just signed up today.
Dude killer url, nice one! Question for all, I clicked their link and went there and it’s an instance, surely. I tried to comment on their post, but was required to sign in… I’m already signed in over here, I gotta sign in there, too? Anyhow I tried to sign in with my lemmy.ml creds but that didn’t work. How can I interact with posts there?
When you open a post it should show you instructions on the right: use the search bar in lemmy.ml to search for
!main@sh.itjust.works
This is a great one! Might use it
Keep it simple with
lemmy.itjust.works.If you get this going or need a hand then let me know.
can’t wait for fedd.itjust.works to go online!
Do you also have a few million dollars under your mattress? 😁
I’d like to tell myself that if it got to the point where it started to cost a few million that i would be able to have it pay for itself!
It’s a week later, but I did get this done finally. I’ve set up https://lem.monster/ . Still doing some tweaking, but it’s open.
You might wanna consider temporarily closing sign-up requests on
lemmy.mlsimilarly to howmastodon.socialdid it during its large influx. Making a sign-up request and just receiving an infinite loading icon is a very frustrating experience.Similarly, you want to make it as easy as possible to financially contribute to lemmy, even if it means using proprietary platforms like Patreon.
Overall, the current Reddit API change is probably one of the largest opportunities for lemmy right now, so smoothing over the user experience as fast as possible in the coming days will be of atmost importance if we want lemmy to become a viable Reddit alternative…
Saldy it’s very common to have this influx towards the “main server” as people that are not used to the federated aspect come to the platform.
Either way, it would be interesting to collect this information and later post some metrics about the exodus from Reddit, kind of like how Fosstodon and other Mastodon instances did when Twitter had their issues.
I don’t think you can expect the bulk of reddit (not technical) user base to care or know what federation is.
If people need to hunt out servers they will stay on something centralized until an alternative takes over.
Hence why Mastedon is dead in the water.
I know it probably won’t be fun for you hosting, but this makes me happy! Hopefully Lemmy will grow a lot!
