845d6b8dcd
## Summary - Introduce a standalone Python roadmap validator with a CLI entry point, modular validation pipeline, and GitHub Actions wiring so roadmap content can be linted locally and in CI. - Provide reusable validation primitives for path resolution, front-matter parsing, identity checks, task parsing, catalog enforcement, and template adherence. - Document usage, configuration, and workflow behaviour to make the validator approachable for contributors. ## Validator Details - **Core tooling** - Added the `tools/roadmap_validator/` package with `validate.py` (CLI), `validator.py` (orchestration), and helper modules (`tasks.py`, `identity.py`, `paths.py`, `constants.py`, `issues.py`). - CLI supports directory/file targets, skips default filenames, emits GitHub annotations, and integrates optional substring filtering - README explains features, environment variables, and development guidance. - **Catalog and template enforcement** - `catalog.py` verifies each allowed content unit has `index.md` and `preview.md`, confirms roadmap entries appear under the proper quarter/area, and flags stale or missing links. - `templates.py` enforces template basics: front matter completeness, `## Description` ordering/content, template placeholder cleanup, and task section detection. - **Task validation** - `tasks.py` checks required metadata (`owner`, `status`, `start-date`, `end-date`), date formats, populated descriptions/deliverables, TODO markers, tangible deliverable heuristics, and `fully-qualified-name` prefixes. - **Workflow integration** - `.github/workflows/roadmap-validator.yml` runs the validator on pushes and manual dispatch, installs dependencies, scopes validation to changed Markdown, and surfaces findings via GitHub annotations. ## Existing Roadmap Updates - Normalised 2025q4 commitments across Web, DST, QA, SC, and other units by filling in missing descriptions, deliverables, schedule notes, recurring task statuses, and maintenance tasks. - Added tasks where absent, removed remaining template placeholders, aligned fully qualified names, and ensured roadmap files conform to the new validator checks. ## Testing ```bash python tools/roadmap_validator/validate.py *2025q4* ``` CI: `Roadmap Validator` workflow runs automatically on pushes/dispatch. --------- Co-authored-by: kaiserd <1684595+kaiserd@users.noreply.github.com>
4.6 KiB
title, tags, draft, description
| title | tags | draft | description | |||
|---|---|---|---|---|---|---|
| Libp2p Evaluation |
|
false | Test libp2p on each new version or feature and look for regressions, learn scaling properties and run scaling studies. |
vac:dst:vac:2025q4-libp2p-evaluation
Description
Test libp2p on each new version or requested feature and look for regressions, learn scaling properties and run scaling studies, understand the limits of Waku and its behaviour. Deliver reports and actionable insights. Do this monthly, reliably, with documentation of findings.
The scope of this commitment depends on the P2P team work and improvements, and it is subjected to change.
Background
We want to learn specific, actionable information about libp2p's behaviour and how it is evolving over time with each new release and with each thing we are specifically asked to check and test.
We will use a combination of real world testing, theoretical analysis and simulation to determine and measure the success, side effects and other factors of libp2p and its evolution.
Narrative
We will support the Conduit of Expertise narrative directly by analysing and evaluating new libp2p releases and their features, both with regards to features they have today and with regards to how that compares to past behaviour.
Additionally, these efforts will contribute to the Premier Research destination narrative by improving and strengthening our relationship with the libp2p team and thus increasing the reach and influence of the IFT, and improving the chances that we successfully grow our ecosystem's products and collaborations and especially those we want to work with externally.
Additional info
Task list
Regression testing (recurring)
- fully qualified name:
vac:dst:vac:2025q4-libp2p-evaluation:regression-testing - owner: Alberto
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Run different scenarios and collect evidence and data of libp2p's behaviour.
Test for known regressions that have occurred in the past and ensure they don't happen again.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Analysis:
Universal connectivity
- fully qualified name:
vac:dst:vac:2025q4-libp2p-evaluation:universal-connectivity - owner: Alberto
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Help p2p team running scenarios with universal connectivity app and checking functionality with other libp2p implementations.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- PRs:
- Report:
Quic-vs-tcp
- fully qualified name:
vac:dst:vac:2025q4-libp2p-evaluation:quic-vs-tcp - owner: TBD
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Perform a comparison between nim-libp2p using quic and waku using mplex and yamux.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Reports:
- Related PRs if apply:
Interop framework
- fully qualified name:
vac:dst:vac:2025q4-libp2p-evaluation:interop-framework - owner: TBD
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Perform a comparison between nim-libp2p using quic and waku using mplex and yamux.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Reports:
- Related PRs if apply:
Mix in nim-libp2p node
- fully qualified name:
vac:dst:vac:2025q4-libp2p-evaluation:mix-in-nim-libp2p-node - owner: Farooq
- status: 90%
- start-date: 2025/10/06
- end-date: 2025/10/17
Description
After refactor was done in vac:dst:ift:2025q3-dst-tooling:nim-libp2p-dst-node-refactor,
include mix to the configuration options of the node.
Deliverables
- PRs:
- Related Documents:
Go-libp2p node
- fully qualified name:
vac:dst:vac:2025q4-libp2p-evaluation:go-libp2p-node - owner: Farooq
- status: 60%
- start-date: 2025/10/06
- end-date: 2025/10/24
Description
Include go-libp2p node to the dst node repository. It should behave the same than the nim-libp2p node. Create a small deployment with both implementations using the same publisher. Report results of a single scenario (fixed parameters) using K8s and shadow.
Deliverables
- PRs:
- Related Documents: