I do think the other home server implementations gaining parity (production-ready) with the reference home server would go a long way. I haven’t run a home server but I’ve heard from those that have that it really has a hard time scaling. (Though this serves as impetus to give it a try over spring break)
Which brings me to the caveats of the protocol, I personally don’t think the design is ideal, it’s more described as a distributed message bus, what I’ve read of the spec it’s over engineered, it made good decisions wrt using modern web technologies (JSON, WebRTC), but it didn’t scope itself to the particular task.
That said, I haven’t written a federated protocol, and they have.
But if I was going to, I’d really want to look at Discord and see how to copy a lot of that model, but break parts of it out to facilitate federation:
I originally wrote a huge hypothetical design here that I speculated would fare better, but honestly the specifics become less relevant, point is that the shared state of rooms is a real challenge, and one out of scope for just a federated instant messaging system, and I’m no longer certain it’s viable.
I’m personally more interested in P2P protocols than federated, so that’s the stuff I build in my spare time.
So instead of something like Lemmy or Matrix, I’d have something like BitTorrent or Tor, so nodes just add capacity instead of hosting specific content. You could configure your node(s) to pin specific content (e.g. for backups or latency), but your data would also be distributed to other peoples’ computers.
This provides data redundancy, permanency of the service (no centralization whatsoever), and ease of scaling (every client could store and seed data), but comes with complexity. I think it’s workable though.
Matrix is probably something worth looking at, at least from an intellectual standpoint, for you.
It uses shared message state and a DAG, plus some fancy perfect forward secrecy (using Signal’s Double Ratchet algorithm), which is at least interesting.
There’s also Tox (chat/protocol) if you want totally distributed chat.
Personally, I really like distributed models from a theoretical standpoint; but for end-user applications they pose very difficult constraints, we live in a world with ⪅50% publicly routed IP for one, they fundamentally require immense data replication, latency in peer-finding, bandwidth constraints, and ultimately sub-par UX. I thought IPFS with a way to pay nodes to pin content was a really neat idea, but hasn’t caught on, for example. Not to discourage you, if you think it’s workable then have at it, but I think it at least explains the current state of things.
Yeah, I’ll have to look into Matrix’s design. I know some of the basics, but the devil is in the details.
we live in a world with ⪅50% publicly routed IP for one
STUN servers bring access up dramatically, and there’s always TURN for the stragglers. Things seem to be getting better with more and more people getting IPv6. I still don’t have it though, and I’m also behind CGNAT, so I know the pain.
But I don’t think bandwidth is really a problem. You should use similar bandwidth to a centralized service, provided the application does appropriate caching and there’s some form of cooperative querying to reduce wire transfer.
I’m following Iroh development (started as a faster replacement for IPFS), and this video was particularly instructive for reducing wire overhead (fancy bloom filter application). I’m trying to build something like Lemmy with it, and I’m interested to see if a similar approach could work for something like Discord.
But yeah, given how much I’m struggling with it certainly explains why it’s not very popular. I could build my project as a centralized service in much less time because synchronizing between clients is very straightforward, and something I do every day at work. But we already have that, and the Fediverse just takes that idea and connects nodes together, so I wouldn’t be adding anything. However, I think longer term, something like this (probably not my specific project) is going to be really important, and I think it’ll be “fast enough.”
pay nodes to pin content
Yeah, I don’t think that’s going to work, because you’d have to pay a sufficient number of nodes such that one node losing interest won’t screw you. And then you need some form of contract (smart contracts on a blockchain?) to ensure you are getting what you’re paying for instead of just getting screwed by someone tossing the data and never telling you.
I’m trying to design my system such that everyone participates in some small way in supporting the network. If you’re on a phone, you store a lot less than if you’re on a PC, but more than if you’re in a web browser. Maybe we could have a distributed fund for rewarding people for supplying more resources than necessary, idk, but I’m honestly not interested in the money part, I just want to build something cool that can’t be taken down because someone got bought out, lost interest, or their government disconnected from the internet. In fact, my system should “just work” if you have people traveling w/ data between isolated networks, provided they have a list of relays within each network that they can connect to peers with.
I do think the other home server implementations gaining parity (production-ready) with the reference home server would go a long way. I haven’t run a home server but I’ve heard from those that have that it really has a hard time scaling. (Though this serves as impetus to give it a try over spring break)
Which brings me to the caveats of the protocol, I personally don’t think the design is ideal, it’s more described as a distributed message bus, what I’ve read of the spec it’s over engineered, it made good decisions wrt using modern web technologies (JSON, WebRTC), but it didn’t scope itself to the particular task.
That said, I haven’t written a federated protocol, and they have. But if I was going to, I’d really want to look at Discord and see how to copy a lot of that model, but break parts of it out to facilitate federation:
I originally wrote a huge hypothetical design here that I speculated would fare better, but honestly the specifics become less relevant, point is that the shared state of rooms is a real challenge, and one out of scope for just a federated instant messaging system, and I’m no longer certain it’s viable.
I’m personally more interested in P2P protocols than federated, so that’s the stuff I build in my spare time.
So instead of something like Lemmy or Matrix, I’d have something like BitTorrent or Tor, so nodes just add capacity instead of hosting specific content. You could configure your node(s) to pin specific content (e.g. for backups or latency), but your data would also be distributed to other peoples’ computers.
This provides data redundancy, permanency of the service (no centralization whatsoever), and ease of scaling (every client could store and seed data), but comes with complexity. I think it’s workable though.
Matrix is probably something worth looking at, at least from an intellectual standpoint, for you. It uses shared message state and a DAG, plus some fancy perfect forward secrecy (using Signal’s Double Ratchet algorithm), which is at least interesting. There’s also Tox (chat/protocol) if you want totally distributed chat.
Personally, I really like distributed models from a theoretical standpoint; but for end-user applications they pose very difficult constraints, we live in a world with ⪅50% publicly routed IP for one, they fundamentally require immense data replication, latency in peer-finding, bandwidth constraints, and ultimately sub-par UX. I thought IPFS with a way to pay nodes to pin content was a really neat idea, but hasn’t caught on, for example. Not to discourage you, if you think it’s workable then have at it, but I think it at least explains the current state of things.
Yeah, I’ll have to look into Matrix’s design. I know some of the basics, but the devil is in the details.
STUN servers bring access up dramatically, and there’s always TURN for the stragglers. Things seem to be getting better with more and more people getting IPv6. I still don’t have it though, and I’m also behind CGNAT, so I know the pain.
But I don’t think bandwidth is really a problem. You should use similar bandwidth to a centralized service, provided the application does appropriate caching and there’s some form of cooperative querying to reduce wire transfer.
I’m following Iroh development (started as a faster replacement for IPFS), and this video was particularly instructive for reducing wire overhead (fancy bloom filter application). I’m trying to build something like Lemmy with it, and I’m interested to see if a similar approach could work for something like Discord.
But yeah, given how much I’m struggling with it certainly explains why it’s not very popular. I could build my project as a centralized service in much less time because synchronizing between clients is very straightforward, and something I do every day at work. But we already have that, and the Fediverse just takes that idea and connects nodes together, so I wouldn’t be adding anything. However, I think longer term, something like this (probably not my specific project) is going to be really important, and I think it’ll be “fast enough.”
Yeah, I don’t think that’s going to work, because you’d have to pay a sufficient number of nodes such that one node losing interest won’t screw you. And then you need some form of contract (smart contracts on a blockchain?) to ensure you are getting what you’re paying for instead of just getting screwed by someone tossing the data and never telling you.
I’m trying to design my system such that everyone participates in some small way in supporting the network. If you’re on a phone, you store a lot less than if you’re on a PC, but more than if you’re in a web browser. Maybe we could have a distributed fund for rewarding people for supplying more resources than necessary, idk, but I’m honestly not interested in the money part, I just want to build something cool that can’t be taken down because someone got bought out, lost interest, or their government disconnected from the internet. In fact, my system should “just work” if you have people traveling w/ data between isolated networks, provided they have a list of relays within each network that they can connect to peers with.