845d6b8dcd
## Summary - Introduce a standalone Python roadmap validator with a CLI entry point, modular validation pipeline, and GitHub Actions wiring so roadmap content can be linted locally and in CI. - Provide reusable validation primitives for path resolution, front-matter parsing, identity checks, task parsing, catalog enforcement, and template adherence. - Document usage, configuration, and workflow behaviour to make the validator approachable for contributors. ## Validator Details - **Core tooling** - Added the `tools/roadmap_validator/` package with `validate.py` (CLI), `validator.py` (orchestration), and helper modules (`tasks.py`, `identity.py`, `paths.py`, `constants.py`, `issues.py`). - CLI supports directory/file targets, skips default filenames, emits GitHub annotations, and integrates optional substring filtering - README explains features, environment variables, and development guidance. - **Catalog and template enforcement** - `catalog.py` verifies each allowed content unit has `index.md` and `preview.md`, confirms roadmap entries appear under the proper quarter/area, and flags stale or missing links. - `templates.py` enforces template basics: front matter completeness, `## Description` ordering/content, template placeholder cleanup, and task section detection. - **Task validation** - `tasks.py` checks required metadata (`owner`, `status`, `start-date`, `end-date`), date formats, populated descriptions/deliverables, TODO markers, tangible deliverable heuristics, and `fully-qualified-name` prefixes. - **Workflow integration** - `.github/workflows/roadmap-validator.yml` runs the validator on pushes and manual dispatch, installs dependencies, scopes validation to changed Markdown, and surfaces findings via GitHub annotations. ## Existing Roadmap Updates - Normalised 2025q4 commitments across Web, DST, QA, SC, and other units by filling in missing descriptions, deliverables, schedule notes, recurring task statuses, and maintenance tasks. - Added tasks where absent, removed remaining template placeholders, aligned fully qualified names, and ensured roadmap files conform to the new validator checks. ## Testing ```bash python tools/roadmap_validator/validate.py *2025q4* ``` CI: `Roadmap Validator` workflow runs automatically on pushes/dispatch. --------- Co-authored-by: kaiserd <1684595+kaiserd@users.noreply.github.com>
3.3 KiB
title, tags, draft, description
| title | tags | draft | description | |||
|---|---|---|---|---|---|---|
| Waku Evaluation |
|
false | Test Waku on each new version or requested feature and look for regressions, learn scaling properties and run scaling studies. |
vac:dst:waku:2025q4-waku-evaluation
Description
Test Waku on each new version or requested feature and look for regressions, learn scaling properties and run scaling studies, understand the limits of Waku and its behaviour. Deliver reports and actionable insights. Do this monthly, reliably, with documentation of findings.
Background
We want to learn specific, actionable information about Waku's behaviour and how it is evolving over time with each new release and with each thing we are specifically asked to check and test.
We will use a combination of real world testing, theoretical analysis and simulation to determine and measure the success, side effects and other factors of Waku and its evolution.
Narratives
We will support the Conduit of Expertise narrative directly by analysing and evaluating new Waku releases and their features, both with regards to features they have today and with regards to how that compares to past behaviour.
We will:
- Enable improvements to Waku by allowing for repeatable, measureable and real world insights into Waku, all the way from theory to practice and back.
- Reduce the risk of a Waku regression making it into a new release of Waku.
Additionally, these efforts will contribute to the Premier Research destination narrative by:
- Improving and strengthening our relationship with the Waku team and improving the quality and reputation of IFT's work, inside and outside of Waku.
Additional info
Task list
Regression testing (recurring)
- fully qualified name:
vac:dst:waku:2025q4-waku-evaluation:regression-testing - owner: Alberto
- status: in progress (50%)
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Run different scenarios and collect evidence and data of Waku's behaviour.
Test for known regressions that have occurred in the past and ensure they don't happen again.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Reports:
- Related PRs if apply:
Quic-vs-tcp
- fully qualified name:
vac:dst:waku:2025q4-waku-evaluation:quic-vs-tcp - owner: Alberto
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Perform a comparison between Waku using quic and waku using mplex and yamux.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Reports:
- Related PRs if apply:
nWaku API docker image
- fully qualified name:
vac:dst:waku:2025q4-waku-evaluation:nwaku-api-image - owner: TBD
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Merge all utilities from DST/nWaku into a single docker image. This involves images that are used to interact with store, filter and lightpush protocols. It should be investigated if adding the Waku publisher here is also a useful idea. Documentation and code clarity is required, as this could be used by the waku team, or other projects.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Reports:
- Related PRs if apply: