PAST storage
Persistent, replicated content storage on the Pastry overlay: content ids, insert/lookup protocol, leaf-set replication, capacity-aware redirection, caching hooks.
Purpose
PAST storage is the only remote persistence mechanism in LibreSocial. Every byte a peer
shares — profile data, wall entries, photos, login items, group keys, DDS buckets — is wrapped
in a StorageItem, addressed by a hash of an application key, and stored on the peers that are
responsible for that key in the Pastry keyspace, replicated replicationFactor times. The layer
provides:
- Key space mapping — map an application key string to a 160-bit overlay key
(
rice.pastry.Id), so that a lookup of a key converges on the same set of peers for every user. - Replication and durability — keep
replicationFactorcopies on the overlay leaf set closest to the key, re-replicated as the ring changes; locally, responsibility survives restarts throughKryoPersistentStorageon disk plus in-memory LRU caches. - Versioned writes — PAST has no delete; an object is overwritten by a newer
PastContentwhosecheckInsert()authorizes the change (nonce + EC signature chain). - Heterogeneity handling — weak peers must not become the bottleneck of the network, so the responsible node may redirect a retrieval to a stronger peer that already holds a copy.
Explicitly out of scope here: set/list/tree semantics on top of stored objects (DDS), the plugin-facing facade and social caching (StorageDispatcher), and access-control policy (Security, Access control).
Current implementation (as of 2026)
Naming note. The component index refers to this package as com.p2pframework.storage.past,
which reads like an OSGi bundle name. There is no bundle of that name: the code lives in the
bundle Source/com.p2pframework/, package com.p2pframework.storage.past; the modified FreePastry
PAST implementation lives in the bundle Source/rice.freepastry/, package rice.p2p.past.
| Class | Path | Size | Role |
|---|---|---|---|
StorageDispatcher (interface) |
Source/com.p2pframework/src/com/p2pframework/storage/StorageDispatcher.java |
148 L | framework-wide store/retrieve/delete/lookup API, extends Sensor |
PastryStorageDispatcher |
Source/com.p2pframework/src/com/p2pframework/storage/past/PastryStorageDispatcher.java |
1155 L | the implementation: PAST client, local caches, monitoring, lifecycle |
PastryStorageItem |
…/storage/past/PastryStorageItem.java |
554 L | PastContent wrapper: serialized item, EC public key, nonce, change signature |
KryoPersistentStorage |
…/storage/past/KryoPersistentStorage.java |
2233 L | fork of FreePastry PersistentStorage, Kryo instead of XML |
KryoMemoryStorage, KryoLRUCache, StoreCallbackContinuation, RetrieveCallbackContinuation |
…/storage/past/ |
40/73/43/52 L | in-memory caches with Kryo size accounting; bridges PAST callbacks ↔ framework continuations |
| item model | Source/com.p2pframework/src/com/p2pframework/storage/ |
— | AbstractStorageItem, GenericStorageItem<T>, SecureStorageItem, SecureGroupDataItem, SymmetricEncryptedStorageItem, AccessControlList, StoreContinuation/RetrieveContinuation, storage exceptions |
| PAST engine (fork) | Source/rice.freepastry/src/rice/p2p/past/PastImpl.java |
2876 L | insert/lookup/fetch state machine + heterogeneity/redirect extension |
| replication | Source/rice.freepastry/src/rice/p2p/replication/ |
— | ReplicationManagerImpl, ReplicationImpl (replica set = endpoint.range(handle, replicationFactor, …)) |
| capacity messages / model | Source/rice.freepastry/src/rice/p2p/past/messaging/, CapacityCalculator.java, CapacityPrioQueue.java, HeterogeneityNode.java |
— | project-added CanYouProvideMessage, RedirectedLookupMessage, AckMessage, BandwidthNotificationMessage, MessageLostMessage; per-node capacity estimate and provider priority queue |
Wiring
Framework.getStorageDispatcher() (Framework.java:150)
└─ PastryClassFactory.createStorageDispatcher() (PastryClassFactory.java:51)
└─ new PastryStorageDispatcher(framework) → rice.p2p.past.PastImpl over the Pastry node
bundle com.p2pframework.storagedispatcher:
SignStorageDispatcher @Component(service = { IStorageDispatcher.class, ILocalStorage.class })
— OSGi facade for plugins; delegate via @Reference FRAMEWORK → framework.getStorageDispatcher()
PastryStorageDispatcher is created once per framework instance and (re)initialised in
initStorage():
pastryIDFactory = new rice.pastry.commonapi.PastryIdFactory(env);storageDirectory = framework.getConfiguration().getStorageDirectory() + "/" + network.getUsername()—Configuration.getStorageDirectory()is<applicationDirectory>/<STORAGE_DIR>/<ring id>, so each ring and each user gets its own storage root;stor = new KryoPersistentStorage(pastryIDFactory, storageDirectory, 0, env)andcache = new KryoLRUCache(new KryoMemoryStorage(pastryIDFactory), secondCacheSize, env);past = new PastImpl(ownPastryNode, new StorageManagerImpl(pastryIDFactory, stor, cache), ConfigLoader.getIntConfigProperty("replicationFactor"), "", true, heterogeneityCache).
Notable parameters (2026 defaults)
From Source/org.lifesocial.core.utilities/resources/config/LibreSocialConfig.txt:
| Parameter | Value | Meaning |
|---|---|---|
replicationFactor |
4 |
PAST replicas; comment in the config: "#strongNodes >= replicationFactor+1 in case weak nodes are present" |
enableHeterogeneityAwareness |
1 |
capacity-weighted provider selection and bandwidth notification enabled |
heterogeneityCacheSize |
50 (MB) |
cap of the PAST content cache; further clamped to maxMemory/10 (second cache: maxMemory/20, half the configured size) |
CanYouProvideTimeOut |
7000 ms |
how long the responsible node waits for provider offers |
providerListLength, randomSelectionModeEnabled |
1, 0 |
providers kept per object id; 0 = capacity-weighted choice, 1 = uniform random provider |
objectSizeCalculationEnabled |
1 |
measure object size for bandwidth estimation |
autoDeleteStorage |
0 |
on reconnect, wipe the storage dir instead of reusing it |
measuringData, socialMetricCollection |
1 |
measurement of store/retrieve and social-caching metrics |
Verified current-state caveats
- Write authorization is disabled in the shipped code.
PastryStorageItemcontainsprivate static final boolean disableSecurity = true; // WARNING: THIS COMPLETELY DISABLES SIGNATURE CHECKS, andcheckInsert()returnsthisimmediately when set — the EC signature/nonce machinery and theDistributedDataBucketownership checks exist but are not enforced. Commitfa17217a1"Disabled Signature Checks!" (2015-03-27). - Storage caches are no longer monitored. The
CACHE_USED_SIZE,CACHE_TOTAL_SIZEandCACHE_NUM_ELEMENTSattribute constants exist, but their publication ingetAttributes()is commented out with the note "Cache is not used anymore in LifeSocial". - Two cache generations coexist. The PAST-level caches (heterogeneity cache,
StorageManagerImplbackup cache) are live; the social cache is a newer, separate mechanism in thecom.p2pframework.storagedispatcherbundle (socialCacheEnabled = 1,normalCacheEnabled = 0). - Structure Cache is not in the tree. A search for
StructureCache/IStructureCache/CacheStructure(across all Java sources inSource/returns no code matches (see below). - Kryo persistence is a maintained fork, patched during the 2026 revival (Java 17, OSGi class
loading):
KryoMemoryStorage/KryoLRUCachecarry the header "Modified by Lukas Mueller 2026 … Replaced obsolete Kryo UnsafeOutput/UnsafeInput with standard Output/Input for Java 17 compatibility".
Design options and decision
The storage design was fixed early (FreePastry/PAST) and then revised twice: first to make storage capacity-aware, later to make retrieval cache-aware. Where a publication does not state the alternatives it weighed, the alternatives below are reconstructed and marked as such.
-
Structured DHT + PAST instead of a custom store or gossip-based replication. Pastry gives deterministic keyspace routing with low maintenance cost; PAST adds versioned, replicated content with a
checkInsert()authorization hook and a backup cache for non-responsible objects (Pastry (2001), PAST (2001), Storage Management and Caching in PAST (2001)). Decision: adopted, then forked. Consequences still visible in 2026: deletion is an overwrite, since PAST defines no remove —deleteItem(hash)storesnew GenericStorageItem<String>("")as a tombstone andretrieveItemmaps empty-Stringcontent back toItemNotFoundException; and the cache is sized from the configuration (heterogeneityCacheSize) and filled on insert as well as on lookup (insert()→doInsert()→ localcache(obj, …)inPastImpl). -
Capacity-aware indirection instead of mirrors on strong peers — the load-balancing decision. Explored in Adaptives Loadbalancing für strukturierte P2P-Netzwerke (2011) and Adding Capacity-Aware Storage Indirection to Homogeneous DHTs (2013): Adaptive-Chord decouples responsibility from serving by keeping mirrors on strong peers (evaluated on Chord in simulation, with Re-Chord as a churn-tolerant variant); the LibreSocial variant leaves the DHT alone and redirects lookups to a peer that already holds a copy and has enough capacity. Trade-off as reported there: mirrors cost storage and maintenance on strong nodes but keep hop count at 1; redirection costs an extra round trip plus provider probing, needs no extra storage, and degrades gracefully when providers are weak. Decision: redirection shipped (the code in
PastImplplus five new PAST messages), Adaptive-Chord stayed a simulation result. (The papers do not name the rejected variant for the product; that attribution is reconstructed from the shipped code, which contains no mirror placement.) -
Structure-aware caching (2016) vs. social caching (2020). For collective retrieval of DDS structures, Distributed Data Structures Improvement for Collective Retrieval Time (2016) proposes a Structure Cache at the root object —
IStructureCachewithCacheStructure(depth)/BuildStructureFromCache(), replicated with the root — turningN·Sretrievals into ≈2·S(190 s → 40 s and 165 s → 30 s in PeerfactSim.KOM, Chord, 1000 nodes, KAD churn), at the price of a slower single-item retrieval (1.1–1.4 s → 18.4 s) and(C−1)·SizeOf(OverlayKey)cache maintenance. Caching Structures for Distributed Data Management (2020) instead pushes content to friends proactively. Decision as visible in the 2026 tree: social caching shipped (com.p2pframework.storagedispatcher,SocialCache, strategiesRANDOM/TOTAL/TREND/SOCIAL_SCORE,socialCacheUpdater = socialscore); Structure Cache never landed — those class names do not exist in the source. (Neither paper names the alternative it rejected; the trade-off above is reconstructed from the reported numbers and the state of the code.) -
Kryo instead of FreePastry's XML serialization — decided in the commit history, not in a paper: 2015-02-19/20 "Changed to kryo as persistence storage instead of slow xml" and "Changed of object size measuring from xml to kryo", 2015-02-26 "KryoMemorystorage as replacement for normal MemoryStorage with slow XML Serialization". Trade-off: much faster and smaller, but the object graph must survive Kryo rewrites — hence explicit registrations (
PastryStorageItem,byte[],rice.pastry.Id,int[]) and the class-loading/instantiation workarounds inKryoPersistentStorage. The cost reappeared in 2026 on the Kryo upgrade (commits71e397bf0,39163847b,d5ca53614).
Technical details
Content ids
PastryIdFactory (bundle rice.freepastry) digests the key string with SHA-1 into a
rice.pastry.Id; Id.IdBitLength = 160, so content keys and node keys share one 160-bit space.
Application keys come from org.lifesocial.core.main/src/org/lifesocial/core/data/utils/HashKeyBuilder.java
as concatenations such as <username><LoginPlugin><LoginItemToken>; StorageKeyImpl keeps the
username separately, which lets the social cache tell foreign keys from own ones. The string handed
down here is StorageKey.getKeyBasis(), converted by pastryIDFactory.buildId(hash).
Stored object format (PastryStorageItem)
PastryStorageItem extends ContentHashPastContent implements PastContent, PastryContent and carries:
byte[] serializedStorageItem— theStorageItemwritten with Java serialization;byte[] pubKey,int keyVersion— the EC public key of the writer;UUID nonce(+serializedNonce) andbyte[] changeSignature— the version/authorization chain;getBytesToSign()andsignChange(oldNonce, signKey)used by the dispatcher before every insert.
checkInsert(oldPastContent) is where write authorization belongs: verify the old content's public key
over the bytes-to-sign of the old nonce, reject PastException("not authorized, no signature") /
"not authorized, writer does not own object", and for DistributedDataBucket refuse to overwrite a
non-empty bucket — but with disableSecurity = true it returns this immediately.
Insert / retrieve / delete flow and replication
Store: storeItem(item, hash, className [, ID | StoreContinuation | pubKey, signKey]) →
past.lookup(id, false, …) first; if an item exists, signChange(existing.getNonceForSign(), signKey);
then past.insert(new PastryStorageItem(...), new StoreCallbackContinuation(cont, …)); group variants
(storeItemGroup, retrieveItemGroup) encrypt via GroupKeysRetriever/KeyContainer into
SecureGroupDataItem; each operation is reported to the measuring subsystem
(m.collect(b, className, "send", storageItem)).
Retrieve: retrieveItem(String|ID, RetrieveContinuation) → past.lookup/fetch →
RetrieveCallbackContinuation decrypts SecureGroupDataItem, and treats a tombstone (empty String
payload) as ItemNotFoundException.
Delete: deleteItem(hash [, StoreContinuation]) → tombstone overwrite. Handles:
lookupHandles(hash) runs past.lookupHandles(id, past.getReplicationFactor() + 1, …) on a joined thread —
how callers discover which peers hold a copy (capacity-based search, DDS consolidation).
Standard FreePastry replication underneath: ReplicationManagerImpl/ReplicationImpl keep the replica
set as endpoint.range(handle, replicationFactor, handle.getId(), true) — the replicationFactor peers
closest to the key inside the leaf set — and re-issue inserts when membership changes. GCPastImpl
(rice.p2p.past.gc) exists in the fork, but the framework instantiates plain PastImpl.
Capacity-aware redirection (the project's PAST extension)
Protocol (all messages under rice.p2p.past.messaging, all state in PastImpl):
lookup(id, cache, cont): check local storage first; otherwise sendLookupMessage(timeoutp2p_past_lookupMessageTimeout).- A responsible node that holds the object answers directly only if the requester is itself or no
provider is known; otherwise it probes its provider list with
CanYouProvideMessage(timeoutCanYouProvideTimeOut = 7000 ms). A provider answers withCapacityCalculator.getCapacity()only ifheterogeneityCache.exists(id). getSuitableProvider(Queue)picks the provider: withenableHeterogeneityAwareness = 1andrandomSelectionModeEnabled = 0, a capacity-weighted roulette overCapacityPrioQueue.getTotalCapacity()(the node itself is included); withrandomSelectionModeEnabled = 1, uniform random.- The requester receives a
NodeHandleinstead of the content and sends aRedirectedLookupMessageto the chosen provider. It measures elapsed time and object size →bandwidthKbPerS, feedsCapacityCalculator.newUploadBandwidthMeasured(...)(EWMA:value = old*0.9 + d*0.1), caches the object into the heterogeneity cache, and informs the network:AckMessage(id, capacity, -1)to the responsible node, plusBandwidthNotificationMessageto the provider when awareness is on. - The responsible node records the acked node in
providerList: HashMap<Id, Queue<HeterogeneityNode>>and trims the queue toproviderListLength(currently 1, i.e. one provider per object).
getHeterogeneityStatistics() exposes totalNumOfProviders, avgLookupTime, numberOfRedirections,
numberOfSentLookups, ANSWERED_REDIRECTED_LOOKUP_COUNT, ANSWERED_OBJECT_LOOKUP_REQUESTS,
ANSWERED_REDIRECTED_LOOKUP_OBJECT_SIZE, ANSWERED_OBJECT_LOOKUP_REQUESTS_SIZE, MESSAGE_HOPCOUNT,
NUMBEROFSTOREDELEMENTS, RECEIVED_LOOKUPS, NUMBEROFREPLICATIONS. Capacity inputs are Bandwidth
(measured upload), Space (usable MB) and Memory (freeMemory). FreePastry timeouts stay configurable
through the environment: p2p_past_messageTimeout, p2p_past_lookupMessageTimeout,
p2p_past_successfulInsertThreshold.
Local caching hooks (three caches, plus one that never shipped)
| Cache | Where | Size | Notes |
|---|---|---|---|
heterogeneityCache |
PastImpl (created by PastryStorageDispatcher.initStorage()) |
min(maxMemory/10, 50 MB) |
Kryo LRU; source of provider offers; filled on lookup and insert |
| backup cache | StorageManagerImpl(pastryIDFactory, stor, cache) |
min(maxMemory/20, 25 MB) |
FreePastry "cache what you route" mechanism, Kryo variant |
SocialCache + LRU |
bundle com.p2pframework.storagedispatcher |
CACHE_SIZE = 200 entries, CACHE_DEFAULT_CACHE_TIME = 20000 ms, CACHE_MIN_CACHE_TIME = 10000 ms; login items 600000 ms |
push to friends over MessageChannel; lookup order: social cache → LRU → PAST; counters SOCIAL_CACHE_COUNT, NORMAL_CACHE_COUNT, PASTRY_STORAGE_COUNT, NULL_VALUE_COUNT |
| Structure Cache | — | — | 2016 proposal, not implemented in this tree |
Local persistence (KryoPersistentStorage)
Fork of FreePastry PersistentStorage with Kryo instead of XML. Layout and constants: root
.../FreePastry-Storage-Root/, metadata.cache (ReverseTreeMap, synced every
METADATA_SYNC_TIME = 300000 ms), PERSISTENCE_MAGIC_NUMBER = 8038844221L, MAX_FILES = 256 per
directory, MAX_DIRECTORIES = 32, ZERO_LENGTH_NAME = "!", plus a dirty-directory purge timer. Two
OSGi/Java-specific workarounds are load-bearing: enterKryoClassLoader()/leaveKryoClassLoader() swap the
thread-context class loader (fixes "Unable to find class: …PastryStorageItem" inside Equinox), and
new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()) (Objenesis) instantiates classes without
a no-arg constructor.
Lifecycle
As a framework EventListener, PastryStorageDispatcher reacts to ring changes:
NETWORK_PRECONNECT→ ifautoDeleteStorage == 1,deleteOldStorage(), theninitStorage()ifpast == null;NETWORK_DISCONNECT→((Destructable) past).destroy(),past = null, record the logout time; on the next connect, if the last logout is older thanLOGOUT_DELETE_TIME = 300000 msthe whole storage directory of that user is deleted (FileUtils.delete) instead of being reused — stale responsibility sets after a long offline period are treated as invalid.
Monitoring
StorageDispatcher extends Sensor, and getAttributes() publishes store/retrieve times, stored, retrieved
and deleted item counts, used/total size and element counts from KryoPersistentStorage
(STORAGEDISPATCHER_USED_SIZE, STORAGEDISPATCHER_TOTAL_SIZE,
STORAGEDISPATCHER_NUMINSTORAGE_ELEMENTS), plus per-plugin message rates from RateBuffers. Measured PAST
figures: revival-scale (store 0.048 s, lookup latency 11.50 ms, hop count 1 on a 2-node overlay) in
Revival & build, simulated large-scale numbers in
Evaluation & benchmarks.
Integration points
- Framework:
Framework.getStorageDispatcher(), built byPastryClassFactory; plugins only seeIStorageDispatcher/ILocalStoragefrom thecom.p2pframework.storagedispatcherbundle, whoseInformationCacheis wired back to the facade. - DDS:
com.p2pframework.DDS.DataBackendservers (DistributedLinkedServer,PrefixHashTreeServer,GroupServer, …) callstorage.retrieveItem(msg.positionHash, …)/storeItemon buckets and list nodes; bucket splitting followsdds_split_size(Configuration.java:396,-1= automatic). See DDS and DDS internals. - Overlay:
rice.freepastry(FreePastry merged with the maintained las2peer fork). Unit tests:Source/com.p2pframework.unittests/src/com/p2pframework/network/freepastry/past/replication/Replication.java.
Evolution and supersession
Dates are from the libresocial Git history of the storage paths (the history of the current
repository starts 2011-07-08; an upstream port is recorded as "ported initial commit", 2020-07-13).
- 2007–2010 — base. FreePastry PAST as storage;
StorageDispatcherabstraction; per-user storage directories. Content security (nonce + EC signature incheckInsert) is specified in Practical Security in P2P-based Social Networks (2009) and consolidated in the dissertation (2010), which also defines the monitoring design behind this component's attributes (LifeSocial.KOM: A Secure and P2P-based Solution (2011)). - 2012–2013 — heterogeneity extension.
CapacityMessagescreated 2012-09-13;CanYouProvideMessage,AckMessage,CapacityCalculatoradded by 2012-10-12; redirect/provider tuning through early 2013 (providerListLength, random vs. capacity-weighted selection). The plain-PAST replication description in the 2010 dissertation is no longer complete — Adaptives Loadbalancing für strukturierte Peer-to-Peer-Netzwerke (2011) and Capacity-Aware Storage Indirection (2013) are the current references for who serves an object. - 2014–2015 — secure DDS and serialization. Stored objects become the substrate of secure DDS:
group-key encryption, per-element signatures, tombstone deletion with consolidation
(Secure Distributed Data Structures (2014),
Sets, Lists and Trees (2016)).
In February 2015 the whole persistence path switched from XML to Kryo; in March 2015 signature
enforcement was switched off (
disableSecurity = true), which is still the state in 2026 — the papers' authorization guarantees describe the intended mechanism, not the enforced one. - 2016 — structure cache proposed, not shipped. Structure Cache (2016) is a design contribution only; the 2020 caching work supersedes it as the approach actually integrated.
- 2019–2020 — social caching.
SocialCacheand the strategies (random,total,trend,social-score) were project-group work merged 2019-10-28 ("Social cache changes by Moritz Kanzler …"), described in Caching Structures … (2020) and the dissertation (2020), which are the reference for the current parameters (socialCacheUpdater = socialscore,socialCacheUpdateLimit = 2,socialCacheMUCLimit = 150); Systematic Evaluation of LibreSocial (2020) supplies the measured storage behaviour at simulated scale. - 2025–2026 — revival, and the storage path is where it hurt. The project had not been compiled since about 2016; Reviving LibreSocial (2025) and the setup guide (2025) document the first successful runs. The 2026 Gradle modernization then had to repair the storage stack itself: Kryo added as a bundle dependency (2026-05-13), workarounds for regressions from the Kryo upgrade (2026-06-17), fixes for "Kryo throwing exceptions for PastryStorageItem" (2026-07-06), and a "Quick fix for Kryo exceptions stemming from version upgrade" (2026-10-12); catalogued in Revival & build.
Open points, stated honestly: the disabled signature checks, the unmaintained cache attributes, and
providerListLength = 1 (one provider per object, so redirection has no fallback if that peer leaves) are
current-code facts that no publication resolves — candidates for follow-up work.
Related publications
- Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems (2001)
- PAST: A large-scale, persistent peer-to-peer storage utility (2001)
- Storage Management and Caching in PAST, a Large-Scale, Persistent Peer-to-Peer Storage Utility (2001)
- Practical Security in P2P-based Social Networks (IEEE LCN 2009) (2009)
- Monitoring and Management of Peer-to-Peer Systems (2010)
- Adaptives Loadbalancing für strukturierte Peer-to-Peer-Netzwerke (2011)
- LifeSocial.KOM: A Secure and P2P-based Solution for Online Social Networks (2011)
- Adding Capacity-Aware Storage Indirection to Homogeneous Distributed Hash Tables (2013)
- Secure Distributed Data Structures for Peer-to-peer-based Social Networks (2014)
- Sets, Lists and Trees: Distributed Data Structures on Distributed Hash Tables (2016)
- Distributed Data Structures Improvement for Collective Retrieval Time (Structure Cache) (2016)
- Caching Structures for Distributed Data Management in P2P-based Social Networks (2020)
- Peer-to-Peer Mechanisms for Fully Decentralized, Secure and Scalable Online Social Networks (2020)
- Systematic Evaluation of LibreSocial — A Peer-to-Peer Framework for Online Social Networks (2020)
- LibreSocial: A peer-to-peer framework for online social networks (2020)
- Reviving LibreSocial: Error Fixes and a Baseline Evaluation on Modern Systems (2025)
- Running LibreSocial: A Legacy Java Project Setup Guide (2025)