LibreSocial · Documentation

PAST storage

Persistent, replicated content storage on the Pastry overlay: content ids, insert/lookup protocol, leaf-set replication, capacity-aware redirection, caching hooks.

Purpose

PAST storage is the only remote persistence mechanism in LibreSocial. Every byte a peer shares — profile data, wall entries, photos, login items, group keys, DDS buckets — is wrapped in a StorageItem, addressed by a hash of an application key, and stored on the peers that are responsible for that key in the Pastry keyspace, replicated replicationFactor times. The layer provides:

Explicitly out of scope here: set/list/tree semantics on top of stored objects (DDS), the plugin-facing facade and social caching (StorageDispatcher), and access-control policy (Security, Access control).

Current implementation (as of 2026)

Naming note. The component index refers to this package as com.p2pframework.storage.past, which reads like an OSGi bundle name. There is no bundle of that name: the code lives in the bundle Source/com.p2pframework/, package com.p2pframework.storage.past; the modified FreePastry PAST implementation lives in the bundle Source/rice.freepastry/, package rice.p2p.past.

Class Path Size Role
StorageDispatcher (interface) Source/com.p2pframework/src/com/p2pframework/storage/StorageDispatcher.java 148 L framework-wide store/retrieve/delete/lookup API, extends Sensor
PastryStorageDispatcher Source/com.p2pframework/src/com/p2pframework/storage/past/PastryStorageDispatcher.java 1155 L the implementation: PAST client, local caches, monitoring, lifecycle
PastryStorageItem …/storage/past/PastryStorageItem.java 554 L PastContent wrapper: serialized item, EC public key, nonce, change signature
KryoPersistentStorage …/storage/past/KryoPersistentStorage.java 2233 L fork of FreePastry PersistentStorage, Kryo instead of XML
KryoMemoryStorage, KryoLRUCache, StoreCallbackContinuation, RetrieveCallbackContinuation …/storage/past/ 40/73/43/52 L in-memory caches with Kryo size accounting; bridges PAST callbacks ↔ framework continuations
item model Source/com.p2pframework/src/com/p2pframework/storage/ — AbstractStorageItem, GenericStorageItem<T>, SecureStorageItem, SecureGroupDataItem, SymmetricEncryptedStorageItem, AccessControlList, StoreContinuation/RetrieveContinuation, storage exceptions
PAST engine (fork) Source/rice.freepastry/src/rice/p2p/past/PastImpl.java 2876 L insert/lookup/fetch state machine + heterogeneity/redirect extension
replication Source/rice.freepastry/src/rice/p2p/replication/ — ReplicationManagerImpl, ReplicationImpl (replica set = endpoint.range(handle, replicationFactor, …))
capacity messages / model Source/rice.freepastry/src/rice/p2p/past/messaging/, CapacityCalculator.java, CapacityPrioQueue.java, HeterogeneityNode.java — project-added CanYouProvideMessage, RedirectedLookupMessage, AckMessage, BandwidthNotificationMessage, MessageLostMessage; per-node capacity estimate and provider priority queue

Wiring

Framework.getStorageDispatcher()                       (Framework.java:150)
  └─ PastryClassFactory.createStorageDispatcher()      (PastryClassFactory.java:51)
       └─ new PastryStorageDispatcher(framework)  →  rice.p2p.past.PastImpl over the Pastry node
bundle com.p2pframework.storagedispatcher:
   SignStorageDispatcher @Component(service = { IStorageDispatcher.class, ILocalStorage.class })
     — OSGi facade for plugins; delegate via @Reference FRAMEWORK → framework.getStorageDispatcher()

PastryStorageDispatcher is created once per framework instance and (re)initialised in initStorage():

Notable parameters (2026 defaults)

From Source/org.lifesocial.core.utilities/resources/config/LibreSocialConfig.txt:

Parameter Value Meaning
replicationFactor 4 PAST replicas; comment in the config: "#strongNodes >= replicationFactor+1 in case weak nodes are present"
enableHeterogeneityAwareness 1 capacity-weighted provider selection and bandwidth notification enabled
heterogeneityCacheSize 50 (MB) cap of the PAST content cache; further clamped to maxMemory/10 (second cache: maxMemory/20, half the configured size)
CanYouProvideTimeOut 7000 ms how long the responsible node waits for provider offers
providerListLength, randomSelectionModeEnabled 1, 0 providers kept per object id; 0 = capacity-weighted choice, 1 = uniform random provider
objectSizeCalculationEnabled 1 measure object size for bandwidth estimation
autoDeleteStorage 0 on reconnect, wipe the storage dir instead of reusing it
measuringData, socialMetricCollection 1 measurement of store/retrieve and social-caching metrics

Verified current-state caveats

Design options and decision

The storage design was fixed early (FreePastry/PAST) and then revised twice: first to make storage capacity-aware, later to make retrieval cache-aware. Where a publication does not state the alternatives it weighed, the alternatives below are reconstructed and marked as such.

  1. Structured DHT + PAST instead of a custom store or gossip-based replication. Pastry gives deterministic keyspace routing with low maintenance cost; PAST adds versioned, replicated content with a checkInsert() authorization hook and a backup cache for non-responsible objects (Pastry (2001), PAST (2001), Storage Management and Caching in PAST (2001)). Decision: adopted, then forked. Consequences still visible in 2026: deletion is an overwrite, since PAST defines no remove — deleteItem(hash) stores new GenericStorageItem<String>("") as a tombstone and retrieveItem maps empty-String content back to ItemNotFoundException; and the cache is sized from the configuration (heterogeneityCacheSize) and filled on insert as well as on lookup (insert() → doInsert() → local cache(obj, …) in PastImpl).

  2. Capacity-aware indirection instead of mirrors on strong peers — the load-balancing decision. Explored in Adaptives Loadbalancing für strukturierte P2P-Netzwerke (2011) and Adding Capacity-Aware Storage Indirection to Homogeneous DHTs (2013): Adaptive-Chord decouples responsibility from serving by keeping mirrors on strong peers (evaluated on Chord in simulation, with Re-Chord as a churn-tolerant variant); the LibreSocial variant leaves the DHT alone and redirects lookups to a peer that already holds a copy and has enough capacity. Trade-off as reported there: mirrors cost storage and maintenance on strong nodes but keep hop count at 1; redirection costs an extra round trip plus provider probing, needs no extra storage, and degrades gracefully when providers are weak. Decision: redirection shipped (the code in PastImpl plus five new PAST messages), Adaptive-Chord stayed a simulation result. (The papers do not name the rejected variant for the product; that attribution is reconstructed from the shipped code, which contains no mirror placement.)

  3. Structure-aware caching (2016) vs. social caching (2020). For collective retrieval of DDS structures, Distributed Data Structures Improvement for Collective Retrieval Time (2016) proposes a Structure Cache at the root object — IStructureCache with CacheStructure(depth) / BuildStructureFromCache(), replicated with the root — turning N·S retrievals into ≈2·S (190 s → 40 s and 165 s → 30 s in PeerfactSim.KOM, Chord, 1000 nodes, KAD churn), at the price of a slower single-item retrieval (1.1–1.4 s → 18.4 s) and (C−1)·SizeOf(OverlayKey) cache maintenance. Caching Structures for Distributed Data Management (2020) instead pushes content to friends proactively. Decision as visible in the 2026 tree: social caching shipped (com.p2pframework.storagedispatcher, SocialCache, strategies RANDOM/TOTAL/TREND/SOCIAL_SCORE, socialCacheUpdater = socialscore); Structure Cache never landed — those class names do not exist in the source. (Neither paper names the alternative it rejected; the trade-off above is reconstructed from the reported numbers and the state of the code.)

  4. Kryo instead of FreePastry's XML serialization — decided in the commit history, not in a paper: 2015-02-19/20 "Changed to kryo as persistence storage instead of slow xml" and "Changed of object size measuring from xml to kryo", 2015-02-26 "KryoMemorystorage as replacement for normal MemoryStorage with slow XML Serialization". Trade-off: much faster and smaller, but the object graph must survive Kryo rewrites — hence explicit registrations (PastryStorageItem, byte[], rice.pastry.Id, int[]) and the class-loading/instantiation workarounds in KryoPersistentStorage. The cost reappeared in 2026 on the Kryo upgrade (commits 71e397bf0, 39163847b, d5ca53614).

Technical details

Content ids

PastryIdFactory (bundle rice.freepastry) digests the key string with SHA-1 into a rice.pastry.Id; Id.IdBitLength = 160, so content keys and node keys share one 160-bit space. Application keys come from org.lifesocial.core.main/src/org/lifesocial/core/data/utils/HashKeyBuilder.java as concatenations such as <username><LoginPlugin><LoginItemToken>; StorageKeyImpl keeps the username separately, which lets the social cache tell foreign keys from own ones. The string handed down here is StorageKey.getKeyBasis(), converted by pastryIDFactory.buildId(hash).

Stored object format (PastryStorageItem)

PastryStorageItem extends ContentHashPastContent implements PastContent, PastryContent and carries:

checkInsert(oldPastContent) is where write authorization belongs: verify the old content's public key over the bytes-to-sign of the old nonce, reject PastException("not authorized, no signature") / "not authorized, writer does not own object", and for DistributedDataBucket refuse to overwrite a non-empty bucket — but with disableSecurity = true it returns this immediately.

Insert / retrieve / delete flow and replication

Store: storeItem(item, hash, className [, ID | StoreContinuation | pubKey, signKey]) → past.lookup(id, false, …) first; if an item exists, signChange(existing.getNonceForSign(), signKey); then past.insert(new PastryStorageItem(...), new StoreCallbackContinuation(cont, …)); group variants (storeItemGroup, retrieveItemGroup) encrypt via GroupKeysRetriever/KeyContainer into SecureGroupDataItem; each operation is reported to the measuring subsystem (m.collect(b, className, "send", storageItem)).

Retrieve: retrieveItem(String|ID, RetrieveContinuation) → past.lookup/fetch → RetrieveCallbackContinuation decrypts SecureGroupDataItem, and treats a tombstone (empty String payload) as ItemNotFoundException.

Delete: deleteItem(hash [, StoreContinuation]) → tombstone overwrite. Handles: lookupHandles(hash) runs past.lookupHandles(id, past.getReplicationFactor() + 1, …) on a joined thread — how callers discover which peers hold a copy (capacity-based search, DDS consolidation).

Standard FreePastry replication underneath: ReplicationManagerImpl/ReplicationImpl keep the replica set as endpoint.range(handle, replicationFactor, handle.getId(), true) — the replicationFactor peers closest to the key inside the leaf set — and re-issue inserts when membership changes. GCPastImpl (rice.p2p.past.gc) exists in the fork, but the framework instantiates plain PastImpl.

Capacity-aware redirection (the project's PAST extension)

Protocol (all messages under rice.p2p.past.messaging, all state in PastImpl):

  1. lookup(id, cache, cont): check local storage first; otherwise send LookupMessage (timeout p2p_past_lookupMessageTimeout).
  2. A responsible node that holds the object answers directly only if the requester is itself or no provider is known; otherwise it probes its provider list with CanYouProvideMessage (timeout CanYouProvideTimeOut = 7000 ms). A provider answers with CapacityCalculator.getCapacity() only if heterogeneityCache.exists(id).
  3. getSuitableProvider(Queue) picks the provider: with enableHeterogeneityAwareness = 1 and randomSelectionModeEnabled = 0, a capacity-weighted roulette over CapacityPrioQueue.getTotalCapacity() (the node itself is included); with randomSelectionModeEnabled = 1, uniform random.
  4. The requester receives a NodeHandle instead of the content and sends a RedirectedLookupMessage to the chosen provider. It measures elapsed time and object size → bandwidthKbPerS, feeds CapacityCalculator.newUploadBandwidthMeasured(...) (EWMA: value = old*0.9 + d*0.1), caches the object into the heterogeneity cache, and informs the network: AckMessage(id, capacity, -1) to the responsible node, plus BandwidthNotificationMessage to the provider when awareness is on.
  5. The responsible node records the acked node in providerList: HashMap<Id, Queue<HeterogeneityNode>> and trims the queue to providerListLength (currently 1, i.e. one provider per object).

getHeterogeneityStatistics() exposes totalNumOfProviders, avgLookupTime, numberOfRedirections, numberOfSentLookups, ANSWERED_REDIRECTED_LOOKUP_COUNT, ANSWERED_OBJECT_LOOKUP_REQUESTS, ANSWERED_REDIRECTED_LOOKUP_OBJECT_SIZE, ANSWERED_OBJECT_LOOKUP_REQUESTS_SIZE, MESSAGE_HOPCOUNT, NUMBEROFSTOREDELEMENTS, RECEIVED_LOOKUPS, NUMBEROFREPLICATIONS. Capacity inputs are Bandwidth (measured upload), Space (usable MB) and Memory (freeMemory). FreePastry timeouts stay configurable through the environment: p2p_past_messageTimeout, p2p_past_lookupMessageTimeout, p2p_past_successfulInsertThreshold.

Local caching hooks (three caches, plus one that never shipped)

Cache Where Size Notes
heterogeneityCache PastImpl (created by PastryStorageDispatcher.initStorage()) min(maxMemory/10, 50 MB) Kryo LRU; source of provider offers; filled on lookup and insert
backup cache StorageManagerImpl(pastryIDFactory, stor, cache) min(maxMemory/20, 25 MB) FreePastry "cache what you route" mechanism, Kryo variant
SocialCache + LRU bundle com.p2pframework.storagedispatcher CACHE_SIZE = 200 entries, CACHE_DEFAULT_CACHE_TIME = 20000 ms, CACHE_MIN_CACHE_TIME = 10000 ms; login items 600000 ms push to friends over MessageChannel; lookup order: social cache → LRU → PAST; counters SOCIAL_CACHE_COUNT, NORMAL_CACHE_COUNT, PASTRY_STORAGE_COUNT, NULL_VALUE_COUNT
Structure Cache — — 2016 proposal, not implemented in this tree

Local persistence (KryoPersistentStorage)

Fork of FreePastry PersistentStorage with Kryo instead of XML. Layout and constants: root .../FreePastry-Storage-Root/, metadata.cache (ReverseTreeMap, synced every METADATA_SYNC_TIME = 300000 ms), PERSISTENCE_MAGIC_NUMBER = 8038844221L, MAX_FILES = 256 per directory, MAX_DIRECTORIES = 32, ZERO_LENGTH_NAME = "!", plus a dirty-directory purge timer. Two OSGi/Java-specific workarounds are load-bearing: enterKryoClassLoader()/leaveKryoClassLoader() swap the thread-context class loader (fixes "Unable to find class: …PastryStorageItem" inside Equinox), and new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()) (Objenesis) instantiates classes without a no-arg constructor.

Lifecycle

As a framework EventListener, PastryStorageDispatcher reacts to ring changes:

Monitoring

StorageDispatcher extends Sensor, and getAttributes() publishes store/retrieve times, stored, retrieved and deleted item counts, used/total size and element counts from KryoPersistentStorage (STORAGEDISPATCHER_USED_SIZE, STORAGEDISPATCHER_TOTAL_SIZE, STORAGEDISPATCHER_NUMINSTORAGE_ELEMENTS), plus per-plugin message rates from RateBuffers. Measured PAST figures: revival-scale (store 0.048 s, lookup latency 11.50 ms, hop count 1 on a 2-node overlay) in Revival & build, simulated large-scale numbers in Evaluation & benchmarks.

Integration points

Evolution and supersession

Dates are from the libresocial Git history of the storage paths (the history of the current repository starts 2011-07-08; an upstream port is recorded as "ported initial commit", 2020-07-13).

Open points, stated honestly: the disabled signature checks, the unmaintained cache attributes, and providerListLength = 1 (one provider per object, so redirection has no fallback if that peer leaves) are current-code facts that no publication resolves — candidates for follow-up work.