Compare commits

...

9 Commits

Author SHA1 Message Date
Jimmy Debe
b33e8edc9d Update manifest.md 2025-12-11 09:19:10 -05:00
Jimmy Debe
fafd2b19cf Updates 2025-11-05 10:15:38 -05:00
Jimmy Debe
ae244b7374 Update Codex manifest 2025-11-05 09:46:23 -05:00
Jimmy Debe
512b631f21 Update codex/raw/manifest.md
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-11-05 09:26:09 -05:00
Jimmy Debe
0b5bd120e1 Update codex/raw/manifest.md
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-11-05 09:25:35 -05:00
Jimmy Debe
b1ba2e3b2c Update manifest.md 2025-10-24 08:15:57 -04:00
Jimmy Debe
751b7f7f10 Update manifest.md 2025-10-16 17:00:39 -04:00
Jimmy Debe
27bc607455 Update manifest.md 2025-09-25 17:25:45 -04:00
Jimmy Debe
2d9155a33b Add manifest 2025-09-25 17:15:32 -04:00

95
codex/raw/manifest.md Normal file
View File

@@ -0,0 +1,95 @@
---
title: CODEX-MANIFEST
name: Codex Manifest
status: raw
category: Standards Track
tags: codex
editor:
contributors:
- Jimmy Debe <jimmy@status.im>
---
## Abstract
This specification defines the attributes of the Codex manifest.
## Rationale
The Codex manifest provides the description of the metadata uploaded to the Codex network.
It is in many ways similar to the BitTorrent metainfo file also known as .torrent files,
(for more information see, [BEP3](http://bittorrent.org/beps/bep_0003.html) from BitTorrent Enhancement Proposals (BEPs).
While the BitTorrent metainfo files are generally distributed out-of-band,
Codex manifest receives its own content identifier, [CIDv1](https://github.com/multiformats/cid#cidv1), that is announced on the Codex DHT, also
see the [CODEX-DHT specification](./dht.md).
In version 1 of the BitTorrent protocol a user wants to upload (seed) some content to the BitTorrent network,
the client chunks the content into pieces.
For each piece, a hash is computed and
included in the pieces attribute of the info dictionary in the BitTorrent metainfo file.
In Codex,
instead of hashes of individual pieces,
we create a Merkle Tree computed over the blocks in the dataset.
We then include the CID of the root of this Merkle Tree as treeCid attribute in the Codex Manifest file.
See [CODEX-MERKLE-TREE](./merkle-tree.md) for more information.
In version 2 of the BitTorrent protocol also uses Merkle Trees and
includes the root of the tree in the info dictionary for each .torrent file.
The Codex manifest, CID in particular,
has the ability to uniquely identify the content and
be able to retrieve that content from any single Codex client.
## Semantics
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
The manifest is encoded in multibase base58btc encoding and
has corresponding `CID`, the manifest `CID`.
In Codex, manifest `CID` SHOULD be announced on the [CODEX-DHT](./dht.md), so
the nodes storing the corresponding manifest block can be found by other clients requesting to download the corresponding dataset.
From the manifest,
providers storing relevant blocks SHOULD be identified using the `treeCid` attribute.
The manifest `CID` in Codex is similar to the `info_hash` from BitTorrent.
The `filename` of the original dataset MAY be included in the manifest.
The `mimetype` of the original dataset MAY be included in the manifest.
### Manifest Attributes
```protobuf
message Manifest {
optional bytes treeCid = 1; // cid (root) of the tree
optional uint32 blockSize = 2; // size of a single block
optional uint64 datasetSize = 3; // size of the dataset
optional uint32 codec = 4; // Dataset codec
optional uint32 hcodec = 5; // Multihash codec
optional uint32 version = 6; // Cid version
optional string filename = 7; // original filename
optional string mimetype = 8; // original mimetype
}
```
| attribute | type | description |
|-----------|------|-------------|
| `treeCid` | bytes | A hash based on [CIDv1](https://github.com/multiformats/cid#cidv1) of the root of the [Codex Tree], which is a form of a Merkle Tree corresponding to the dataset described by the manifest. Its multicodec is (codex-root, 0xCD03) |
| `blockSize` | integer | The size of each block for the given dataset. The default block size used in Codex is 64KiB. |
| `datasetSize` | integer | The total size of all blocks for the original dataset. |
| `codec` | [MultiCodec](https://github.com/multiformats/multicodec) | The multicodec used for the CIDs of the dataset blocks. Codex uses (codex-block, 0xCD02) |
| `hcodec` | [MultiCodec](https://github.com/multiformats/multicodec) | Multicodec used for computing of the multihash used in blocks CIDs. Codex uses (sha2-256, 0x12). |
| `version` | integer | The version of CID used for the dataset blocks. |
| `filename` | string | When provided, it can be used by the client as a file name while downloading the content. |
| `mimetype` | string | When provided, it can be used by the client to set a content type of the downloaded content. |
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [CODEX-MERKLE-TREE](./merkle-tree.md)
- [BEP3](http://bittorrent.org/beps/bep_0003.html)
- [CODEX-DHT specification](./dht.md)
- [MultiCodec](https://github.com/multiformats/multicodec)