I know that Lemmy is open source and it can only get better from here on out, but I do wonder if any experts can weigh in whether the foundation is well written? Or are we building on top of 4 years worth of tech debt?

  • redcalcium@c.calciumlabs.com
    link
    fedilink
    arrow-up
    40
    ·
    edit-2
    1 year ago

    As long as the backend is stateless, it can be scaled to handle huge amount of users, at least in theory. IMO the main issue right now with Lemmy deployment is pictrs not being stateless. It uses a filesystem-based internal database called sled. Not only this make pictrs not stateless, you can’t even run multiple replica of pictrs in the same host because sled would crash if the database file lock is already acquired by another replica. Someone with some rust skill should consider donating their time to add postgresql support to pictrs soon, which will greatly help making Lemmy scalable. Too bad I know nothing about rust.

    • Minus Zero@lemmy.world
      link
      fedilink
      arrow-up
      12
      ·
      1 year ago

      An interesting choice of solution there for image hosting… I would have thought they would have gone with a simple proxy through to an object store like S3, GCS, Wasabi, insert other clone here. Or even picked an off the shelf BLOB capable system for self hosting like Mongo or Cassandra. Then your image hosting becomes stateless as you just give each image a flake ID, pop it in the storage system and give back a shortened URL. I’m sure they had their reasons though :-)

    • ccunix@lemmy.world
      link
      fedilink
      arrow-up
      1
      ·
      1 year ago

      I think pictrs has a not very well documented object storage option. I’m struggling to find the link right now though.