LibreSocial · Documentation

Testing Methodology

Deep-dive page: how LibreSocial has been tested from 2011 to 2026 — the Tester plugin, test-case design, run discipline, and what changed over time.

Supersession note: the description of the current testing bed is Imam 2025 (Reviving LibreSocial) plus the 2026 work items; the method of automated Tester-driven evaluation is documented best by Peng & Balabhadruni 2014 and Khitman 2020; the metrics vocabulary was introduced by Liu 2013 and corrected in 2014 (see below) — the 2014 naming supersedes the 2013 naming.

The Tester plugin (com.p2pframework.tester)

LibreSocial ships an integrated system-test framework, not unit tests:

Maintenance debt (the recurring finding)

The catalogue repeatedly rotted when the framework changed:

Year State found Evidence
2014 only 7 of 26 test cases still worked after the 2013 framework redesign; 19 were deleted/rewritten Peng 2014
2020 28 of 56 (50 %) broken and repaired Khitman 2020
2025 catalogue dormant during dormancy; revival re-enabled core messaging/wall flows Imam 2025
2026 rebuild of automated system tests is an open goal blog: test safety net, tester component

Conclusion drawn in the docs repeatedly: test cases must be independent (no shared global state in ParameterGenerator) and must be part of every framework refactor.

Metrics model (SkyEye-backed)

Tests are evaluated through the monitoring over the overlay itself (SkyEye, monitoring component):

Run discipline (2025/2026 rules, codified from earlier pain)

From the revival runbook (Imam 2025):

Scenario families used since 2013

  1. Synthesis test — fully scheduled test plan; validates every feature works.
  2. Random test — randomized user behavior (later: algorithm synthesized from a >35,000-user study); with/without churn (logout). Churn is where defects historically appeared (friend-request hangs 2013; message loss 2014; randomization bug 2020).
  3. Long-term test — stability over days; 2013: ~25 h until FreePastry queue overflow; 2014: 5,300 min (3.67 days) with the same final cause (see bug catalogue).
  4. Scale/replication sweeps — see evaluation & benchmarks.
  5. Alpha test with real users — 2013: documented deployment (10 VMs, Xvfb) but failed for social reasons (2 accounts); re-planned as Docker-cluster user studies in 2026 (blog).

Sources