[v1] We are going inter-planetary!

[rennokki]

Readme is WIP

P2P connectivity

Libp2p offers great capability of out-of-the-box, P2P networking for any environment. This will allow better implementation and less point of failures, with minimal centralization: no more bottlenecks like PubSub and networking, whether it is Kubernetes or your localhost, it just works.

Tasks

• Pusher: Adapt old code to P2P
• Libp2p: Use Peer Content Routing to discover (and mesh with) instances that are sharing the same app/namespace (preparing for future Pusher impl)
• Libp2p: Metrics with libp2p's Prometheus driver (https://github.com/libp2p/js-libp2p/tree/master/packages/metrics-prometheus)
• Libp2p: Add custom configurations for the Server instances (libp2p-itself and stubs)
• Soketi: Bring back the Routing for custom WS/HTTP routing
• Maybe: AWS Gateway API compatibility?

Demo

With Docker, you can spin up three Soketi instances that are able to discover each other over the network with mDNS.

docker-compose up -d

With a WS client, you got now 6001, 6002 and 6003 as open ports for Soketi.

The WS URL format is the following:

ws://<host>:<port>/<room_id>

Using your WS client (i.e. ws-cat), connect to at least two:

wscat -c ws://host.docker.internal:6001/room-1

wscat -c ws://host.docker.internal:6002/room-1

Messages are now transported via the P2P Protocol:

❯ wscat -c ws://host.docker.internal:6002/room-1
Connected (press CTRL+C to quit)
> {"message": "IPFS Rocks!"}
> 

❯ wscat -c ws://host.docker.internal:6001/room-1
Connected (press CTRL+C to quit)
< "{\"message\":\"IPFS Rocks!\"}"
> 

Publishing different packages using Aegir

Since there will be multiple implementations for the same structure, different packages will make them more maintenable. Meet Aegir. Aegir will help with publishing multiple packages at the same time.

Development

@soketi/impl

This package contains the very base of components needed to build the servers (explained below).

Brain

Brain is the codename for the memory storage. It resembles some simple Redis-like KV (put, get, evictions, etc.) for easy implementation of multiple providers.

Gossiper

Gossipers are instances that help with cross-host comms. Since we leverage just Libp2p (for now), the Gossiper is an overlay for the underlying PubSub.

Gossipers are not meant to be discovering other instances necessarily. For better encapsulation, the Prospector is the one that has this mere task.

Gossipers are not for the request-response architecture, instead they are for the send-and-forget messaging, like PUBLISH on Redis or Kafka. For req-res, Prospectors do that. (We plan to break it down even further with an Ambassator component that will communicate with other instances, but it's too much for now)

Prospector

Prospector is a codename for the discoverer of other instances. In a normal Redis setup, you would find the number of subscribed instances. Here, the Prospector ensures direct communication between hosts (i.e. getting the total amount of connections across a specific namespace by ensuring to call only the nodes that have connections in that namespace).

This might be replaced by a distributed, libp2p-protocol based database, but more research is needed.

Prospector is still work in progress.

@soketi/impl-interfaces

This contains the types and interfaces for Typescript.

@soketi/servers

WIP