23 Commits

Author	SHA1	Message	Date
Bjerg	92a7818512	feat: bodies stage (#190) ... * chore: clean up `.gitignore` * fix: make RO cursors `Send + Sync` * feat(wip): bodies stage * driveby: improve docs * chore: don't panic if we're the first stage * chore: use `Vec` for ommers * feat: error handling in bodies downloader * chore: remove stale comment * chore: pascal-case stage id * refactor: remove unused new fns * refactor: distinguish downloaders with prefix * refactor: move downloader errs to own module * refactor: `stream_bodies` -> `bodies_stream` * test: fix borked imports in header stage * test: clean up header tests * test: add basic body stage tests * test: add 2 more body stage test skeletons * test: move generator test utils to own module * refactor: move proof functions to primitives crate * feat: add block generator test utils * test: more body stage tests * chore: fix typo (`Cannonical*` -> `Canonical`) * docs: document `bodies_to_download` * test: more body stage tests * test: more body stage tests * refactor: clean up body stage tests a bit * test: fix broken tests * refactor: clean up body stage tests * test: more body stage tests	2022-11-15 08:49:11 +01:00
rakita	7ecbe01741	feat(provider): Add StorageProvider impl, table changes (#172) ... * feat(provider): Add StorageProvider impl, table changes * unwind header numbers by walker (#174) * readme, fmt * fix tests * Update crates/interfaces/src/provider/db_provider/storage.rs Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com> * Update crates/interfaces/src/provider/db_provider/storage.rs Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com> Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>	2022-11-08 15:55:45 +01:00
rakita	86ffb4756d	feat(bin): simple reth node command (#158) ... * feat(bin): simple reth node command * use silent flag * small chore	2022-11-07 12:10:09 -08:00
rakita	699db03187	feat(db): db error handling and Provider scaffolding (#154) ... * feat(db): db error handling and Provider scaffolding * remove one map_err * fix after merge	2022-11-03 17:00:58 +01:00
joshieDo	becceb2802	fix(db): add traits Encode/Decode for keys and Compress/Uncompress for values (#151) ... * add UncompressedUint * add more UncompressedUint keys * add docs * split key/value to encode/decode and compress/uncompress traits * reveert into * clippy * rm whitespaces * remove TODO * Remove scale encode/decode traits * decompress * clippy Co-authored-by: rakita <dragan0rakita@gmail.com>	2022-11-03 00:02:19 -07:00
joshieDo	ee99c29c8f	feat(interfaces): add ShardedKey (#142) ... * add ShardedKey * Update crates/interfaces/src/db/models/sharded_key.rs Co-authored-by: rakita <rakita@users.noreply.github.com> Co-authored-by: rakita <rakita@users.noreply.github.com>	2022-10-27 20:46:22 -07:00
joshieDo	ee41dfac35	feat(db): set-upChangeSet tables (#135) ... * add AccountChangeSet * add StorageChangeSet * add tests to TxNumberAddress * minor fixes	2022-10-25 03:47:45 -07:00
rakita	8ac5214fc6	chore(libmbx): fmt, clippy and deny list updated (#134) ... * feat(db): Add mdbx-rs apache licenced code 55e234 * feat(db): replace mdbx with reth-mdbx, metadata changes * chore(db): bump mdbx-sys to 0.12.1 * remove libmdbx from cargo deny * cargo fmt * cargo clippy * one more clippy error	2022-10-25 03:18:51 -07:00
rakita	5c55c57c5e	feat(db): switch to reth-libmdbx-rs, bump reth-mdbx-sys to 0.12.1 (#133) ... * feat(db): Add mdbx-rs apache licenced code 55e234 * feat(db): replace mdbx with reth-mdbx, metadata changes * chore(db): bump mdbx-sys to 0.12.1	2022-10-25 02:50:02 -07:00
rakita	1fe0affa4b	chore(db): comment for cursor, cleanup unneeded fn (#131)	2022-10-25 10:34:30 +02:00
joshieDo	2a8189d7c9	chore(db): Add tests and fuzz target to BlockNumHash (#130) ... * add tests and fuzz target for BlockNumHash * use block 1 for test_block_num_hash * new clippy stds * fmt	2022-10-25 01:23:10 -07:00
joshieDo	3c5b235455	feat(db): Add DupSort PlainStorageState table (#121) ... * add dupsort table and test * use env.update instead Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>	2022-10-24 11:23:28 -07:00
rakita	4eca851429	feat(interface): implicit trait bound for DB cursors (#122) ... * feat(interface): implicit trait bound for DB cursors * test cursor * walking fixed for RO * impl for Walker for DupCursor	2022-10-24 11:20:14 -07:00
Georgios Konstantopoulos	a4e505132c	headers(part1) feat(interfaces): introduce implicit trait bounds (#117) ... * feat(ifaces): modify DB GAT to have implicit bounds https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats\#the-better-gats * test(ifaces): impl new trait for mock * feat(ifaces): impl db container we use that downstream to avoid consuming the db transactions when committing a stage * fix(test): explicitly define lifetime * test: ensure can spawn * test: pls review * chore: lints/warnings * feat(db): impl the new gats * test(db): try to make real db work with gats like in stages * test(db): make Stage trait take DBContainer otherwise we cannot call db.commit() gst * feat(stages): impl new traits * chore(db): cleanup tests * chore: remove unused imports	2022-10-23 23:10:59 -07:00
rakita	483bcdf9ab	feat(db): Database Abstraction (#101) ... * database tx traits * wip build passes * Db and tx results * nightly and db GAT * Impl tx, wip cursor * Move Decode to Table Key, working cursor trait * wip dupsort * build all Cursor abstraction * cleanup * wip cleanup * old stages * codecs moved o interface,stages wip * resolve db ref, it builds * Add tx commit after execution * fmt * Remove sync send restriction * Add missing rw cursor functions * Cleanup, added missing cursor fn. rust toolchain * fmt * add nightly to ci * deny dead_code, remove unwrap * rm printfn, stages fix, bench fix	2022-10-20 08:32:13 -07:00
joshieDo	630baf5d70	feat(db): fuzzing & benchmark (#86) ... * fuzz with test-fuzz * move fuzzing to db/codecs * add criterion & iai * print encoded size sum on criterion benchmark * fix BlockNumHash encode * add gh action for benchmarks * don't sum results * test ci values * Revert "test ci values" This reverts commit cc47fd8053. * specify criterion version * add docs * remove benchmark job	2022-10-19 13:08:07 -07:00
joshieDo	063b444792	feat(db): codec encoding/decoding (#51) ... * wip * add table macro * add simple put get test with Address * add Env.view and Env.update * docs * slightly change the test * add initial table initialization and placeholders * lint & some * replace String with str * add error.rs * add docs to encode * add docs * clamp * add source on libmdbx_max_page_size * add BlockNumer_BlockHash * add scale * set header filed to bytes Bytes * remove unwrap * restrict scale to chosen types * into bytes * add postcard * changed to BlockNumHash * add proc_macro_attribute codecs * fix feature flagging * set a version for postcard * cleanup * seal ScaleOnly * remove unnecessary dependencies * properly encode/decode blocknumhash * change Account codec to scale * add missing feature to scale * add codec to a couple more types * silence clippy * add docs about table encoding * move and add reth-codecs * clippy * make proc-macro visible * add README.md	2022-10-17 01:04:57 -07:00
Roman Krasiuk	08d8a7f030	cursor put (#54)	2022-10-12 17:16:59 +02:00
Roman Krasiuk	bbb46216ea	chore: test utils (#48)	2022-10-11 20:09:18 +02:00
Roman Krasiuk	5ebfc01338	chore: migrate stages to new db abstractions (#43) ... * migrate * set block as u64 * linter * BNum -> BlockNumber * stage id alias	2022-10-11 19:54:05 +02:00
Georgios Konstantopoulos	c41c6b99a6	db: supply table as Type not as fn param (#32)	2022-10-10 12:52:42 -07:00
joshieDo	60d3c64410	feat(db): mdbx integration & table skeletons (#15) ... * wip * add table macro * add simple put get test with Address * add Env.view and Env.update * docs * slightly change the test * add initial table initialization and placeholders * lint & some * replace String with str * add error.rs * add docs to encode * add docs * clamp * add source on libmdbx_max_page_size	2022-10-10 12:35:35 -07:00
Bjerg	849e8ae518	feat: add db crate (#9)	2022-10-03 16:13:42 +02:00