LibreSocial · Documentation

Overlay (Pastry)

Purpose

The overlay provides the structured peer-to-peer substrate the whole LibreSocial framework sits on: a 160-bit identifier space with proximity-aware DHT routing, node identity derived from the user's elliptic-curve public key, join/bootstrapping with per-ring membership, and the transport plumbing that carries messaging, PAST storage, SCRIBE and monitoring traffic. The code contains

Capacity/load-balancing mechanisms from the later publications (Adaptive Chord, capacity-aware storage indirection, CapSearch, capacity management) are not present in this code base as of 2026: the overlay is still the homogeneous Pastry DHT of the 2001 papers, with only the hooks those designs need (attribute export, id conversion, ring bookkeeping) — see "Design options".

Current implementation (as of 2026)

Key classes and packages (real paths)

Path Role
Source/rice.freepastry/src/rice/pastry/Id.java 160-bit node/object ids (IdBitLength = 160, nlen = 5 ints), digit-wise distance getDistance(...), Id.build(byte[])
.../rice/pastry/routing/RoutingTable.java Pastry routing table; rows = IdBitLength / idBaseBitLength, cols = 1 << idBaseBitLength, maxEntries per slot, RouteSet observable as NodeSetEventSource
.../rice/pastry/leafset/LeafSet.java, SimilarSet.java leaf set with merge(...) and half-set (SimilarSet) bookkeeping
.../rice/pastry/routing/Router.java, .../rice/pastry/standard/StandardRouter.java, RapidRerouter forwarding with re-routing on failure
.../rice/pastry/join/JoinProtocol.java, JoinRequest.java, InitiateJoin.java and .../rice/pastry/standard/ConsistentJoinProtocol.java join (JoinRequest state flags such as HAS_LEAFSET) and consistent-join verification with re-sends to failed peers
.../rice/pastry/transport/TransportPastryNodeFactory.java assembles one node: new RoutingTable(localhandle, rtMax, rtBase, pn), new LeafSet(localhandle, lSetSize, routeTable), RapidRerouter, maintenance frequencies
.../rice/pastry/PastryNodeFactory.java reads pastry_rtMax, pastry_rtBaseBitLength, pastry_lSetSize from the environment
.../rice/pastry/socket/** socket transport, RendezvousSocketPastryNodeFactory/RendezvousSocketNodeHandle (firewalled peers via rendezvous), LivenesSourceRouteForwardStrategy
.../org/mpisws/p2p/transport/wire/ringidentification/RingIDTransportLayer.java, RingIDServiceProvider.java LibreSocial addition: transport wrapper that stamps/probes a per-ring UUID so two overlays on the same wire protocol can be told apart (PastryNode.getRingIDServiceProvider())
.../rice/p2p/multiring/** FreePastry multi-ring experiment (RingId, RingCertificate, MultiringNode) — in the fork, p2p_multiring_base = 2 configured, not used by the framework
Source/com.p2pframework/src/com/p2pframework/network/freepastry/PastryNetwork.java the Network implementation (886 lines): creates/boots the Pastry node, exposes monitoring attributes
…/network/freepastry/InternetPastryNodeFactory.java deployment factory (666 lines): NAT policy, external-address probing through the bootstrap, UPnP port opening, reachability check
…/network/freepastry/MonitoredSocketPastryNodeFactory.java attaches traffic/message counters and the back-off layer
…/network/freepastry/IDFactory.java, PastryID.java, PastryIP.java, IPFactory.java com.p2pframework.network.ID ↔ Pastry Id mapping plus a BigDecimal converter used by the monitoring/SkyEye id space
…/network/freepastry/SecureJoinProtocol.java, SecureInitiateJoin.java, SecureJoinRequest.java, AuthorisationCheckMessage.java, AuthorisationProofMessage.java, EncryptPRawMessage.java join-time authorisation/proof exchange (present, not wired — see below)
…/network/freepastry/backofflayer/** BackOffLayerImpl, BackOffMessageWrapper, BackOffLayerConfig: re-queueing/back-off of outgoing messages to prevent loss under congestion
Source/com.p2pframework/src/com/p2pframework/ring/Ring.java, PastryRingManager.java, RingManager.java, RingDispatcher.java, PastryRingDispatcher.java, RingNetworkSettings.java, RingStatus.java multi-ring bookkeeping, bootstrap cache, XML persistence
…/ring/lifeness/PastryRingLifenessChecker.java, RingConnectionVerifier.java, RingLifenessChecker.java out-of-band TCP/UDP reachability and ring-id probing

Wiring

Framework owns PastryNetwork, RingDispatcher and StorageDispatcher. Boot sequence (PastryNetwork.initPastryNodeAndBootstrapAddress() → bootPastryNode()):

  1. ECAsymmetricKey(username, password) → ECPublicKey.toByteArray() → Id.build(bytes); the resulting Pastry Id is the node id (deterministic per account); the password reference is nulled right after.
  2. The Ring comes from RingDispatcher.getCurrentRing(); the debug ring (Ring.DEBUG_RING_ID = cdefaf90-66e8-11e3-949a-0800200c9a66) is rejected in non-debug mode.
  3. Bind address: the ring's RingNetworkSettings.localInterface/localPort; for a non-single-node ring the local address is auto-detected by opening a client Socket to the bootstrap and reading getLocalAddress(); -Dlibresocial.bindLoopback=true rewrites bind address and bootstrap host to 127.0.0.1 (commit adfbf158c, 2026-07-07, for multi-node tests on one host).
  4. Optional port forwarding: getExternalMostAddress(...) through the bootstrap, then UPnP with nat_handler_class = rice.pastry.socket.nat.sbbi.SBBINatHandler (findFireWall, findAvailableFireWallPort(bindport, pfPort, 5, "LibreSocial"), openFireWallPort(...)); the result is published as external_address.
  5. Factory chain MonitoredSocketPastryNodeFactory → InternetPastryNodeFactory → RendezvousSocketPastryNodeFactory → SocketPastryNodeFactory → TransportPastryNodeFactory. The per-node transport stack is wire layer → MagicNumberTransportLayer → RingIDTransportLayer → source-routing layer → PriorityTransportLayer → BackOffLayerImpl; the byte counter is attached in getMagicNumberTransportLayer(), the message counter and the back-off wrapper in getPriorityTransportLayer(), which returns new BackOffLayerImpl(priorityTransportLayer, 1000, 5, 30, 30, liveness, env).
  6. node.addLeafSetListener(ring) / node.addRouteSetListener(ring): Ring is a NodeSetListener and continuously refreshes the bootstrap-address cache.
  7. node.getRingIDServiceProvider().setLocalUUID(ring.getId()), then node.boot(bootAddress); the caller blocks on node.isReady() / node.joinFailed() with a 650 000 ms timeout (node.wait(1000) loop) and records joinNetworkDuration.

Ring.nodeSetUpdate(...) accepts only RendezvousSocketNodeHandles reporting canContactDirect(), so firewalled (rendezvous-only) peers are never advertised as bootstraps.

Notable parameters

Defaults live in Source/rice.freepastry/config/freepastry.params; the unit-test bundle ships a variant at Source/com.p2pframework.unittests/resources/freepastry.params (pastry_factory_bootsInParallel = 1, pastry_socket_scm_max_open_sockets = 300, pastry_socket_scm_num_ping_tries = 5, scm_backoff_limit = 5, p2p_past_messageTimeout = 30000).

Design options and decision

(The publications document the alternatives discussed at the time; where a link to this specific code is not stated in the sources, the rationale is reconstructed and marked as such.)

Technical details

Identifier space

Routing and node state

Join, membership and ring lifecycle

Ring identification on the wire

RingIDTransportLayer frames messages with the local ring UUID and answers requestRemoteRingID(addr, callback) probes; RingConnectionVerifier is the client side, used by the lifeness checker and by InternetPastryNodeFactory.isReachableFromExternalNode. This keeps a node from joining a different LibreSocial ring that happens to speak the same FreePastry protocol on the same port.

NAT traversal and firewall handling

InternetPastryNodeFactory ("designed for real deployments (with NATs)", modified by project group 2013) provides NAT-policy constants (ALWAYS/PREFIX_MATCH/NEVER, OVERWRITE/USE_DIFFERENT_PORT/FAIL/RENDEZVOUS), external-address discovery by asking externally reachable nodes (findExternalAddress, one address per NAT hop), reachability verification through a bootstrap, and UPnP port opening. Firewalled nodes still participate via rendezvous handles but are excluded from the bootstrap cache.

Integration points

Present but not active (verified 2026)

Several overlay-adjacent mechanisms exist without being wired into a running node — relevant when reading older descriptions of the system:

Evolution and supersession

Overlay and storage substrate

Ring management, NAT traversal and the framework (project reports)

Monitoring over the overlay

Load balancing and capacity (research above this overlay)

Platform and current state