diff --git a/404.html b/404.html index ce63405b..aeeb2c34 100644 --- a/404.html +++ b/404.html @@ -1,26 +1,23 @@ - + - -Page Not Found | Vac Research + +Vac Research - - - + + + -
-
Skip to main content

Page Not Found

We could not find what you were looking for.
Please contact the owner of the site that linked you to the original URL and let them know their link is broken.

- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}
Skip to main content

Page Not Found

We could not find what you were looking for.

Please contact the owner of the site that linked you to the original URL and let them know their link is broken.

\ No newline at end of file diff --git a/_og/09a58c601b223babe5c8d17b041d0985df39c2f5.png b/_og/09a58c601b223babe5c8d17b041d0985df39c2f5.png new file mode 100644 index 00000000..5c9fe965 Binary files /dev/null and b/_og/09a58c601b223babe5c8d17b041d0985df39c2f5.png differ diff --git a/_og/146849b08b6f48a21e7891db4ba81663317a351c.png b/_og/146849b08b6f48a21e7891db4ba81663317a351c.png deleted file mode 100644 index 49fcc485..00000000 Binary files a/_og/146849b08b6f48a21e7891db4ba81663317a351c.png and /dev/null differ diff --git a/_og/1d76e97062a4e71c2a3a86e411201b549cfa5c11.png b/_og/1d76e97062a4e71c2a3a86e411201b549cfa5c11.png new file mode 100644 index 00000000..7a00174a Binary files /dev/null and b/_og/1d76e97062a4e71c2a3a86e411201b549cfa5c11.png differ diff --git a/_og/2036b5d864453e6ef3671525ac88ef88eb774eb2.png b/_og/2036b5d864453e6ef3671525ac88ef88eb774eb2.png new file mode 100644 index 00000000..6cf40d9c Binary files /dev/null and b/_og/2036b5d864453e6ef3671525ac88ef88eb774eb2.png differ diff --git a/_og/20ce1dfd1cb5720107a286391bd0e2ed1cfc975c.png b/_og/20ce1dfd1cb5720107a286391bd0e2ed1cfc975c.png deleted file mode 100644 index 2e03c97f..00000000 Binary files a/_og/20ce1dfd1cb5720107a286391bd0e2ed1cfc975c.png and /dev/null differ diff --git a/_og/243d28332ccf02f5949101e52fe2842f5cca9eae.png b/_og/243d28332ccf02f5949101e52fe2842f5cca9eae.png deleted file mode 100644 index 28fe42b0..00000000 Binary files a/_og/243d28332ccf02f5949101e52fe2842f5cca9eae.png and /dev/null differ diff --git a/_og/29170e3a19469e33dd7f34677672c8907a5c17e9.png b/_og/29170e3a19469e33dd7f34677672c8907a5c17e9.png new file mode 100644 index 00000000..523abbfb Binary files /dev/null and b/_og/29170e3a19469e33dd7f34677672c8907a5c17e9.png differ diff --git a/_og/2a59fd69ac557f8bf52d825c02cb26e3d7890b08.png b/_og/2a59fd69ac557f8bf52d825c02cb26e3d7890b08.png deleted file mode 100644 index ad3a0602..00000000 Binary files a/_og/2a59fd69ac557f8bf52d825c02cb26e3d7890b08.png and /dev/null differ diff --git a/_og/2c679d5298be6cb0fc3fb02f96d735947ded15ec.png b/_og/2c679d5298be6cb0fc3fb02f96d735947ded15ec.png new file mode 100644 index 00000000..f9f4ebcd Binary files /dev/null and b/_og/2c679d5298be6cb0fc3fb02f96d735947ded15ec.png differ diff --git a/_og/2d99bdeaec237fcf4fd949b2e477c781d49a7519.png b/_og/2d99bdeaec237fcf4fd949b2e477c781d49a7519.png deleted file mode 100644 index 0db9339e..00000000 Binary files a/_og/2d99bdeaec237fcf4fd949b2e477c781d49a7519.png and /dev/null differ diff --git a/_og/2e96130036606efdef25482a03152731d45dcffd.png b/_og/2e96130036606efdef25482a03152731d45dcffd.png deleted file mode 100644 index 687404d9..00000000 Binary files a/_og/2e96130036606efdef25482a03152731d45dcffd.png and /dev/null differ diff --git a/_og/2f143dc8e139f7218879ecc0caeabcceaca9eea3.png b/_og/2f143dc8e139f7218879ecc0caeabcceaca9eea3.png deleted file mode 100644 index aeb94c34..00000000 Binary files a/_og/2f143dc8e139f7218879ecc0caeabcceaca9eea3.png and /dev/null differ diff --git a/_og/2f550d2e360bfcc02b797171d6b8f7b85b0483b6.png b/_og/2f550d2e360bfcc02b797171d6b8f7b85b0483b6.png deleted file mode 100644 index 2292707b..00000000 Binary files a/_og/2f550d2e360bfcc02b797171d6b8f7b85b0483b6.png and /dev/null differ diff --git a/_og/33c21345f05e5cf44cce9cc3b84d7a784acd9c6b.png b/_og/33c21345f05e5cf44cce9cc3b84d7a784acd9c6b.png new file mode 100644 index 00000000..f47b6b4d Binary files /dev/null and b/_og/33c21345f05e5cf44cce9cc3b84d7a784acd9c6b.png differ diff --git a/_og/345c35ee9f63c19554552ba686c7a289bea985df.png b/_og/345c35ee9f63c19554552ba686c7a289bea985df.png new file mode 100644 index 00000000..58e63ca1 Binary files /dev/null and b/_og/345c35ee9f63c19554552ba686c7a289bea985df.png differ diff --git a/_og/36ae1883b1ed0b2a15ae158f9cd93aaa9f5d7c69.png b/_og/36ae1883b1ed0b2a15ae158f9cd93aaa9f5d7c69.png new file mode 100644 index 00000000..bd354023 Binary files /dev/null and b/_og/36ae1883b1ed0b2a15ae158f9cd93aaa9f5d7c69.png differ diff --git a/_og/3e66368dcf852275bdef19ede4f4b64b30beae85.png b/_og/3e66368dcf852275bdef19ede4f4b64b30beae85.png new file mode 100644 index 00000000..604f15e9 Binary files /dev/null and b/_og/3e66368dcf852275bdef19ede4f4b64b30beae85.png differ diff --git a/_og/3ef9132dff3526ac9748fdf93b437ed276abf3e7.png b/_og/3ef9132dff3526ac9748fdf93b437ed276abf3e7.png new file mode 100644 index 00000000..92b0e243 Binary files /dev/null and b/_og/3ef9132dff3526ac9748fdf93b437ed276abf3e7.png differ diff --git a/_og/3fc6d4687b9bc295fe7ac535b76ad99560a9d570.png b/_og/3fc6d4687b9bc295fe7ac535b76ad99560a9d570.png deleted file mode 100644 index a045fdd0..00000000 Binary files a/_og/3fc6d4687b9bc295fe7ac535b76ad99560a9d570.png and /dev/null differ diff --git a/_og/439d7244245c416c881c8565b92257c8769ba808.png b/_og/439d7244245c416c881c8565b92257c8769ba808.png deleted file mode 100644 index e38f3299..00000000 Binary files a/_og/439d7244245c416c881c8565b92257c8769ba808.png and /dev/null differ diff --git a/_og/444ed6ae948e7aeab44ee754d239543dd00e0ecc.png b/_og/444ed6ae948e7aeab44ee754d239543dd00e0ecc.png new file mode 100644 index 00000000..30ac1b40 Binary files /dev/null and b/_og/444ed6ae948e7aeab44ee754d239543dd00e0ecc.png differ diff --git a/_og/44526c4a7f0afa1dbfa1abc1380a674fae156eaf.png b/_og/44526c4a7f0afa1dbfa1abc1380a674fae156eaf.png new file mode 100644 index 00000000..234d536b Binary files /dev/null and b/_og/44526c4a7f0afa1dbfa1abc1380a674fae156eaf.png differ diff --git a/_og/48145b063be6b05adcecc3c973bd7ae8dc353090.png b/_og/48145b063be6b05adcecc3c973bd7ae8dc353090.png deleted file mode 100644 index fec4958c..00000000 Binary files a/_og/48145b063be6b05adcecc3c973bd7ae8dc353090.png and /dev/null differ diff --git a/_og/48306cace9f72742f960a47314853b31cad1494e.png b/_og/48306cace9f72742f960a47314853b31cad1494e.png new file mode 100644 index 00000000..5e280ede Binary files /dev/null and b/_og/48306cace9f72742f960a47314853b31cad1494e.png differ diff --git a/_og/5b4d0998a5ee97473506bc460b87d66e2baf5980.png b/_og/4925de9d10caa0e9f7b9ee0ddb1bba6163e08169.png similarity index 64% rename from _og/5b4d0998a5ee97473506bc460b87d66e2baf5980.png rename to _og/4925de9d10caa0e9f7b9ee0ddb1bba6163e08169.png index 5be24f22..578d35b7 100644 Binary files a/_og/5b4d0998a5ee97473506bc460b87d66e2baf5980.png and b/_og/4925de9d10caa0e9f7b9ee0ddb1bba6163e08169.png differ diff --git a/_og/4c430d9c145e087cbe8aa9e1d5d0aabc8d3b463c.png b/_og/4c430d9c145e087cbe8aa9e1d5d0aabc8d3b463c.png deleted file mode 100644 index 765095dc..00000000 Binary files a/_og/4c430d9c145e087cbe8aa9e1d5d0aabc8d3b463c.png and /dev/null differ diff --git a/_og/4c4b54fb8be20f83fd7b8a39972878b7f421e417.png b/_og/4c4b54fb8be20f83fd7b8a39972878b7f421e417.png new file mode 100644 index 00000000..0f4bdb79 Binary files /dev/null and b/_og/4c4b54fb8be20f83fd7b8a39972878b7f421e417.png differ diff --git a/_og/4d6e4b871fcef4875c7bf138f739528f74dc957d.png b/_og/4d6e4b871fcef4875c7bf138f739528f74dc957d.png deleted file mode 100644 index 334030cb..00000000 Binary files a/_og/4d6e4b871fcef4875c7bf138f739528f74dc957d.png and /dev/null differ diff --git a/_og/4e84d86959a90ae0dd4012a0ca2ec4485d514a99.png b/_og/4e84d86959a90ae0dd4012a0ca2ec4485d514a99.png new file mode 100644 index 00000000..2a228362 Binary files /dev/null and b/_og/4e84d86959a90ae0dd4012a0ca2ec4485d514a99.png differ diff --git a/_og/4f9b11e84e8d585e56892719c0f44e73b925535e.png b/_og/4f9b11e84e8d585e56892719c0f44e73b925535e.png new file mode 100644 index 00000000..6c42ef28 Binary files /dev/null and b/_og/4f9b11e84e8d585e56892719c0f44e73b925535e.png differ diff --git a/_og/4fb0f69300a9c3471222cb2331fff338fdee1324.png b/_og/4fb0f69300a9c3471222cb2331fff338fdee1324.png deleted file mode 100644 index 5995bf75..00000000 Binary files a/_og/4fb0f69300a9c3471222cb2331fff338fdee1324.png and /dev/null differ diff --git a/_og/5013eb600c6b7d4cf7450a8586491fb0899993f6.png b/_og/5013eb600c6b7d4cf7450a8586491fb0899993f6.png deleted file mode 100644 index 83c705e1..00000000 Binary files a/_og/5013eb600c6b7d4cf7450a8586491fb0899993f6.png and /dev/null differ diff --git a/_og/51136dee01a8b71773787c81d0f5480a5e93659c.png b/_og/51136dee01a8b71773787c81d0f5480a5e93659c.png new file mode 100644 index 00000000..12e97ba1 Binary files /dev/null and b/_og/51136dee01a8b71773787c81d0f5480a5e93659c.png differ diff --git a/_og/51595438632ff1fe81b95e9d31f37b68163cfe74.png b/_og/51595438632ff1fe81b95e9d31f37b68163cfe74.png deleted file mode 100644 index 9b5679aa..00000000 Binary files a/_og/51595438632ff1fe81b95e9d31f37b68163cfe74.png and /dev/null differ diff --git a/_og/51d2c61b6930034c507d3b20892b375fb749d601.png b/_og/51d2c61b6930034c507d3b20892b375fb749d601.png new file mode 100644 index 00000000..0ed2ccd3 Binary files /dev/null and b/_og/51d2c61b6930034c507d3b20892b375fb749d601.png differ diff --git a/_og/5277bcb9b4ce26c094a69b8680b82f95b2a1861c.png b/_og/5277bcb9b4ce26c094a69b8680b82f95b2a1861c.png deleted file mode 100644 index 8e7d2302..00000000 Binary files a/_og/5277bcb9b4ce26c094a69b8680b82f95b2a1861c.png and /dev/null differ diff --git a/_og/577a6e310259b1fc0f0a2d70be15d6b5f11ce160.png b/_og/577a6e310259b1fc0f0a2d70be15d6b5f11ce160.png deleted file mode 100644 index bcaa17a4..00000000 Binary files a/_og/577a6e310259b1fc0f0a2d70be15d6b5f11ce160.png and /dev/null differ diff --git a/_og/5a3c3ca861ae8316af5d7abc9bdd33b1d414f4e7.png b/_og/5a3c3ca861ae8316af5d7abc9bdd33b1d414f4e7.png new file mode 100644 index 00000000..3e0fa0a8 Binary files /dev/null and b/_og/5a3c3ca861ae8316af5d7abc9bdd33b1d414f4e7.png differ diff --git a/_og/5e8213de01e59b7eda0331ad469a4d15a77f86c8.png b/_og/5e8213de01e59b7eda0331ad469a4d15a77f86c8.png new file mode 100644 index 00000000..2b215ab0 Binary files /dev/null and b/_og/5e8213de01e59b7eda0331ad469a4d15a77f86c8.png differ diff --git a/_og/604e1aeb0aab31550ecc61f28cb813ee40b3e3f2.png b/_og/604e1aeb0aab31550ecc61f28cb813ee40b3e3f2.png deleted file mode 100644 index cbad007d..00000000 Binary files a/_og/604e1aeb0aab31550ecc61f28cb813ee40b3e3f2.png and /dev/null differ diff --git a/_og/64128c84268e996fdd357fc9469a4719ff1f252c.png b/_og/64128c84268e996fdd357fc9469a4719ff1f252c.png deleted file mode 100644 index 881e8565..00000000 Binary files a/_og/64128c84268e996fdd357fc9469a4719ff1f252c.png and /dev/null differ diff --git a/_og/64fcbaf9e44b194b0ba00dc2b1ba0f08b288ad6a.png b/_og/64fcbaf9e44b194b0ba00dc2b1ba0f08b288ad6a.png deleted file mode 100644 index 08aaf768..00000000 Binary files a/_og/64fcbaf9e44b194b0ba00dc2b1ba0f08b288ad6a.png and /dev/null differ diff --git a/_og/6643efafa8660d30fe0d9edfe33c31ce917deffe.png b/_og/6643efafa8660d30fe0d9edfe33c31ce917deffe.png new file mode 100644 index 00000000..791ba5c9 Binary files /dev/null and b/_og/6643efafa8660d30fe0d9edfe33c31ce917deffe.png differ diff --git a/_og/68bd9208336196f3869fde4bb15688e89b800c75.png b/_og/68bd9208336196f3869fde4bb15688e89b800c75.png deleted file mode 100644 index 0a1ce0f8..00000000 Binary files a/_og/68bd9208336196f3869fde4bb15688e89b800c75.png and /dev/null differ diff --git a/_og/719e4ba5a4ed9ba85f34f968395fc2bffd666795.png b/_og/719e4ba5a4ed9ba85f34f968395fc2bffd666795.png deleted file mode 100644 index 807912ea..00000000 Binary files a/_og/719e4ba5a4ed9ba85f34f968395fc2bffd666795.png and /dev/null differ diff --git a/_og/c05f83f42c7cf8c705e42288377b3e1268e41c4c.png b/_og/73a10ef7afa81ea32fd6c203fba6a559f2b14b97.png similarity index 72% rename from _og/c05f83f42c7cf8c705e42288377b3e1268e41c4c.png rename to _og/73a10ef7afa81ea32fd6c203fba6a559f2b14b97.png index d4c2f506..00629986 100644 Binary files a/_og/c05f83f42c7cf8c705e42288377b3e1268e41c4c.png and b/_og/73a10ef7afa81ea32fd6c203fba6a559f2b14b97.png differ diff --git a/_og/759fd40efa5df89d67147c46bbbb354ceadb7981.png b/_og/759fd40efa5df89d67147c46bbbb354ceadb7981.png deleted file mode 100644 index 7188f847..00000000 Binary files a/_og/759fd40efa5df89d67147c46bbbb354ceadb7981.png and /dev/null differ diff --git a/_og/78309406172987d2a2e49f052c1abbe7dbe60e65.png b/_og/78309406172987d2a2e49f052c1abbe7dbe60e65.png deleted file mode 100644 index d5565bab..00000000 Binary files a/_og/78309406172987d2a2e49f052c1abbe7dbe60e65.png and /dev/null differ diff --git a/_og/79522d94e33272659ad417eb39a9b2f294deea99.png b/_og/79522d94e33272659ad417eb39a9b2f294deea99.png new file mode 100644 index 00000000..c2e7b361 Binary files /dev/null and b/_og/79522d94e33272659ad417eb39a9b2f294deea99.png differ diff --git a/_og/7ddaa8f631b24ffb30b1d5f0033364d1cac60d47.png b/_og/7ddaa8f631b24ffb30b1d5f0033364d1cac60d47.png deleted file mode 100644 index 54dbec7c..00000000 Binary files a/_og/7ddaa8f631b24ffb30b1d5f0033364d1cac60d47.png and /dev/null differ diff --git a/_og/7f403f30f8417cd0aca7cbf2cf215a5dfbeda316.png b/_og/7f403f30f8417cd0aca7cbf2cf215a5dfbeda316.png new file mode 100644 index 00000000..5841f658 Binary files /dev/null and b/_og/7f403f30f8417cd0aca7cbf2cf215a5dfbeda316.png differ diff --git a/_og/81ea0699a3ee1c3a1c423440819e3802f214d545.png b/_og/81ea0699a3ee1c3a1c423440819e3802f214d545.png new file mode 100644 index 00000000..c3b24f44 Binary files /dev/null and b/_og/81ea0699a3ee1c3a1c423440819e3802f214d545.png differ diff --git a/_og/83663c72c74de2d37f06d27aef012fd6df389d41.png b/_og/83663c72c74de2d37f06d27aef012fd6df389d41.png new file mode 100644 index 00000000..d67f38db Binary files /dev/null and b/_og/83663c72c74de2d37f06d27aef012fd6df389d41.png differ diff --git a/_og/84bf3d4f0b89c5927cae2d71b0af32e50b90e8d9.png b/_og/84bf3d4f0b89c5927cae2d71b0af32e50b90e8d9.png deleted file mode 100644 index 53719428..00000000 Binary files a/_og/84bf3d4f0b89c5927cae2d71b0af32e50b90e8d9.png and /dev/null differ diff --git a/_og/8745a87c3b2163cb01fbea416f21e1e0c3e0209a.png b/_og/8745a87c3b2163cb01fbea416f21e1e0c3e0209a.png new file mode 100644 index 00000000..e4561eba Binary files /dev/null and b/_og/8745a87c3b2163cb01fbea416f21e1e0c3e0209a.png differ diff --git a/_og/87908e9c6aad50f213f1637c6ad96b75d7861a61.png b/_og/87908e9c6aad50f213f1637c6ad96b75d7861a61.png new file mode 100644 index 00000000..06c4e3d4 Binary files /dev/null and b/_og/87908e9c6aad50f213f1637c6ad96b75d7861a61.png differ diff --git a/_og/87f3d6931e6bc62797c7ef32398316194c1bd923.png b/_og/87f3d6931e6bc62797c7ef32398316194c1bd923.png deleted file mode 100644 index 141e6ea5..00000000 Binary files a/_og/87f3d6931e6bc62797c7ef32398316194c1bd923.png and /dev/null differ diff --git a/_og/884ae2508f8fc33b7f55b4dfd2584afb50e52436.png b/_og/884ae2508f8fc33b7f55b4dfd2584afb50e52436.png new file mode 100644 index 00000000..f4828163 Binary files /dev/null and b/_og/884ae2508f8fc33b7f55b4dfd2584afb50e52436.png differ diff --git a/_og/8a7292c4f736c1537aadb877afdccf17241ee14b.png b/_og/8a7292c4f736c1537aadb877afdccf17241ee14b.png deleted file mode 100644 index 8c18ca41..00000000 Binary files a/_og/8a7292c4f736c1537aadb877afdccf17241ee14b.png and /dev/null differ diff --git a/_og/8fd4c9f856349ecb65ab78ec974a776127d6f653.png b/_og/8fd4c9f856349ecb65ab78ec974a776127d6f653.png new file mode 100644 index 00000000..3b29e058 Binary files /dev/null and b/_og/8fd4c9f856349ecb65ab78ec974a776127d6f653.png differ diff --git a/_og/949d357a1c94f3e9eaee169995734dafe9777777.png b/_og/949d357a1c94f3e9eaee169995734dafe9777777.png new file mode 100644 index 00000000..c3de72e5 Binary files /dev/null and b/_og/949d357a1c94f3e9eaee169995734dafe9777777.png differ diff --git a/_og/96384e501123c5045cd44e97827e443a28520ed9.png b/_og/96384e501123c5045cd44e97827e443a28520ed9.png deleted file mode 100644 index e7ae0a73..00000000 Binary files a/_og/96384e501123c5045cd44e97827e443a28520ed9.png and /dev/null differ diff --git a/_og/c54198631ecabf6cc6b0ed357a7e7761c1ad7ba5.png b/_og/99781e1caaed616723b669c8734da3ccc4891c9c.png similarity index 51% rename from _og/c54198631ecabf6cc6b0ed357a7e7761c1ad7ba5.png rename to _og/99781e1caaed616723b669c8734da3ccc4891c9c.png index ab4c79f1..0037fcc9 100644 Binary files a/_og/c54198631ecabf6cc6b0ed357a7e7761c1ad7ba5.png and b/_og/99781e1caaed616723b669c8734da3ccc4891c9c.png differ diff --git a/_og/9b3aa806bd938ee41b4cbbac5a100bab16d21c00.png b/_og/9b3aa806bd938ee41b4cbbac5a100bab16d21c00.png new file mode 100644 index 00000000..0a0138c1 Binary files /dev/null and b/_og/9b3aa806bd938ee41b4cbbac5a100bab16d21c00.png differ diff --git a/_og/9d82bd34b4da2d2ef8b0c71a989a5b9e79d1e49c.png b/_og/9d82bd34b4da2d2ef8b0c71a989a5b9e79d1e49c.png deleted file mode 100644 index b56762d6..00000000 Binary files a/_og/9d82bd34b4da2d2ef8b0c71a989a5b9e79d1e49c.png and /dev/null differ diff --git a/_og/c4bd246dc4b3b867d7c0a2d4651a36dec5b7d1ff.png b/_og/9d86b91535e8b8d04c131771d925b627121a3e66.png similarity index 59% rename from _og/c4bd246dc4b3b867d7c0a2d4651a36dec5b7d1ff.png rename to _og/9d86b91535e8b8d04c131771d925b627121a3e66.png index a9e36eb6..2a2162b6 100644 Binary files a/_og/c4bd246dc4b3b867d7c0a2d4651a36dec5b7d1ff.png and b/_og/9d86b91535e8b8d04c131771d925b627121a3e66.png differ diff --git a/_og/a550bf108de1372a53c81ee91f565ce31dc7bb2c.png b/_og/a550bf108de1372a53c81ee91f565ce31dc7bb2c.png new file mode 100644 index 00000000..de9b83dc Binary files /dev/null and b/_og/a550bf108de1372a53c81ee91f565ce31dc7bb2c.png differ diff --git a/_og/a829e7e452b1228aad936bde704f6d9c99e49f02.png b/_og/a829e7e452b1228aad936bde704f6d9c99e49f02.png deleted file mode 100644 index ef530ed5..00000000 Binary files a/_og/a829e7e452b1228aad936bde704f6d9c99e49f02.png and /dev/null differ diff --git a/_og/a8afa27462fbf8882303eb6bbf0759d273fb9b31.png b/_og/a8afa27462fbf8882303eb6bbf0759d273fb9b31.png deleted file mode 100644 index 37307829..00000000 Binary files a/_og/a8afa27462fbf8882303eb6bbf0759d273fb9b31.png and /dev/null differ diff --git a/_og/aa00755b197054c79e81aa6faa4ffa7965148956.png b/_og/aa00755b197054c79e81aa6faa4ffa7965148956.png new file mode 100644 index 00000000..fecd281d Binary files /dev/null and b/_og/aa00755b197054c79e81aa6faa4ffa7965148956.png differ diff --git a/_og/aaf6724587fed1991765b02883623dc52dd2ceb8.png b/_og/aaf6724587fed1991765b02883623dc52dd2ceb8.png deleted file mode 100644 index 3f6cedaf..00000000 Binary files a/_og/aaf6724587fed1991765b02883623dc52dd2ceb8.png and /dev/null differ diff --git a/_og/ab6119a81316093d616e89b83d9720ba0009eec4.png b/_og/ab6119a81316093d616e89b83d9720ba0009eec4.png deleted file mode 100644 index 9cd365d5..00000000 Binary files a/_og/ab6119a81316093d616e89b83d9720ba0009eec4.png and /dev/null differ diff --git a/_og/abda591d2e64342c6f7f0e0db8aa21c3efa4114b.png b/_og/abda591d2e64342c6f7f0e0db8aa21c3efa4114b.png new file mode 100644 index 00000000..560c1a28 Binary files /dev/null and b/_og/abda591d2e64342c6f7f0e0db8aa21c3efa4114b.png differ diff --git a/_og/aca0aa4197ad1a1adebb40cd3e25f40f1f44a118.png b/_og/aca0aa4197ad1a1adebb40cd3e25f40f1f44a118.png deleted file mode 100644 index e699bfbf..00000000 Binary files a/_og/aca0aa4197ad1a1adebb40cd3e25f40f1f44a118.png and /dev/null differ diff --git a/_og/b390ad99a1b62447645b5f6d241475c553cc60cd.png b/_og/b390ad99a1b62447645b5f6d241475c553cc60cd.png deleted file mode 100644 index b14117e2..00000000 Binary files a/_og/b390ad99a1b62447645b5f6d241475c553cc60cd.png and /dev/null differ diff --git a/_og/b61e1e14e1c90c20c11ca42120e1439898e1528c.png b/_og/b61e1e14e1c90c20c11ca42120e1439898e1528c.png deleted file mode 100644 index 7094a8b4..00000000 Binary files a/_og/b61e1e14e1c90c20c11ca42120e1439898e1528c.png and /dev/null differ diff --git a/_og/b6c2046892f6cddb0f524177fbe40c9b6e8a4d08.png b/_og/b6c2046892f6cddb0f524177fbe40c9b6e8a4d08.png new file mode 100644 index 00000000..c61e53e7 Binary files /dev/null and b/_og/b6c2046892f6cddb0f524177fbe40c9b6e8a4d08.png differ diff --git a/_og/bbd016263935bc5c85a1d436e9a9eb9d6c1e5d68.png b/_og/bbd016263935bc5c85a1d436e9a9eb9d6c1e5d68.png new file mode 100644 index 00000000..f52f11d4 Binary files /dev/null and b/_og/bbd016263935bc5c85a1d436e9a9eb9d6c1e5d68.png differ diff --git a/_og/be5a002db19d7c195865aa03bdd739737d75738b.png b/_og/be5a002db19d7c195865aa03bdd739737d75738b.png deleted file mode 100644 index 64d1c394..00000000 Binary files a/_og/be5a002db19d7c195865aa03bdd739737d75738b.png and /dev/null differ diff --git a/_og/c08cef4416253a611e6bc05977477f8aa48f40a0.png b/_og/c08cef4416253a611e6bc05977477f8aa48f40a0.png new file mode 100644 index 00000000..66128c7b Binary files /dev/null and b/_og/c08cef4416253a611e6bc05977477f8aa48f40a0.png differ diff --git a/_og/c214e96ddb2c5d0adf295755d455548ecff35d71.png b/_og/c214e96ddb2c5d0adf295755d455548ecff35d71.png new file mode 100644 index 00000000..777eb8b0 Binary files /dev/null and b/_og/c214e96ddb2c5d0adf295755d455548ecff35d71.png differ diff --git a/_og/c6fff028b8a9e9d2a38236cb5be7c10fe82a98b1.png b/_og/c6fff028b8a9e9d2a38236cb5be7c10fe82a98b1.png new file mode 100644 index 00000000..63ae446b Binary files /dev/null and b/_og/c6fff028b8a9e9d2a38236cb5be7c10fe82a98b1.png differ diff --git a/_og/c77f05f87dd61f5055bf9ba9ce2369d245adc53a.png b/_og/c77f05f87dd61f5055bf9ba9ce2369d245adc53a.png new file mode 100644 index 00000000..68af1243 Binary files /dev/null and b/_og/c77f05f87dd61f5055bf9ba9ce2369d245adc53a.png differ diff --git a/_og/c9352e62215b7e6f121502ec97633791ded290b2.png b/_og/c9352e62215b7e6f121502ec97633791ded290b2.png new file mode 100644 index 00000000..67bcdb50 Binary files /dev/null and b/_og/c9352e62215b7e6f121502ec97633791ded290b2.png differ diff --git a/_og/c9b48cb9beba3cc3c60952e3bae6dc9d11bd6f47.png b/_og/c9b48cb9beba3cc3c60952e3bae6dc9d11bd6f47.png deleted file mode 100644 index c093c6c3..00000000 Binary files a/_og/c9b48cb9beba3cc3c60952e3bae6dc9d11bd6f47.png and /dev/null differ diff --git a/_og/ca9681bfcd20e3ff820cf66bd0dc8ed070d33449.png b/_og/ca9681bfcd20e3ff820cf66bd0dc8ed070d33449.png new file mode 100644 index 00000000..4aa1c429 Binary files /dev/null and b/_og/ca9681bfcd20e3ff820cf66bd0dc8ed070d33449.png differ diff --git a/_og/cb1af6f3575e50b0ea643b07c65a37ace2eb5e25.png b/_og/cb1af6f3575e50b0ea643b07c65a37ace2eb5e25.png new file mode 100644 index 00000000..319b033b Binary files /dev/null and b/_og/cb1af6f3575e50b0ea643b07c65a37ace2eb5e25.png differ diff --git a/_og/ccbfee797a8a7cf0995378a7326e8a930a83380e.png b/_og/ccbfee797a8a7cf0995378a7326e8a930a83380e.png new file mode 100644 index 00000000..ec2005bb Binary files /dev/null and b/_og/ccbfee797a8a7cf0995378a7326e8a930a83380e.png differ diff --git a/_og/cf2dfc037f7802cd0346bcedb59cd89a99eafbc2.png b/_og/cf2dfc037f7802cd0346bcedb59cd89a99eafbc2.png deleted file mode 100644 index ae7b5698..00000000 Binary files a/_og/cf2dfc037f7802cd0346bcedb59cd89a99eafbc2.png and /dev/null differ diff --git a/_og/d1714eb3683c946e88a49742733c17cc41a053ce.png b/_og/d1714eb3683c946e88a49742733c17cc41a053ce.png new file mode 100644 index 00000000..4403ef76 Binary files /dev/null and b/_og/d1714eb3683c946e88a49742733c17cc41a053ce.png differ diff --git a/_og/d3fd3ae1b37acc120b4705d4f375252f0800e288.png b/_og/d3fd3ae1b37acc120b4705d4f375252f0800e288.png deleted file mode 100644 index 6bd82ec6..00000000 Binary files a/_og/d3fd3ae1b37acc120b4705d4f375252f0800e288.png and /dev/null differ diff --git a/_og/d79cf273f9899be2eed536095869952c6912fb37.png b/_og/d79cf273f9899be2eed536095869952c6912fb37.png deleted file mode 100644 index 04a24339..00000000 Binary files a/_og/d79cf273f9899be2eed536095869952c6912fb37.png and /dev/null differ diff --git a/_og/d7f914eaf3a6c0341ec9bd219e2394926cbe5e87.png b/_og/d7f914eaf3a6c0341ec9bd219e2394926cbe5e87.png new file mode 100644 index 00000000..4590be38 Binary files /dev/null and b/_og/d7f914eaf3a6c0341ec9bd219e2394926cbe5e87.png differ diff --git a/_og/de7aa5a50391f3705a51426be591e76b17a02d75.png b/_og/de7aa5a50391f3705a51426be591e76b17a02d75.png new file mode 100644 index 00000000..b1457893 Binary files /dev/null and b/_og/de7aa5a50391f3705a51426be591e76b17a02d75.png differ diff --git a/_og/dfd624431b8e708ff5ddaf2d96623a866cdd9d86.png b/_og/dfd624431b8e708ff5ddaf2d96623a866cdd9d86.png new file mode 100644 index 00000000..62e24587 Binary files /dev/null and b/_og/dfd624431b8e708ff5ddaf2d96623a866cdd9d86.png differ diff --git a/_og/e094c497f8e757d812d62787c142bdb50be3f1c2.png b/_og/e094c497f8e757d812d62787c142bdb50be3f1c2.png new file mode 100644 index 00000000..13b29c41 Binary files /dev/null and b/_og/e094c497f8e757d812d62787c142bdb50be3f1c2.png differ diff --git a/_og/e40218f393902a10518de47edfc810879e937ace.png b/_og/e40218f393902a10518de47edfc810879e937ace.png deleted file mode 100644 index 0532ba8b..00000000 Binary files a/_og/e40218f393902a10518de47edfc810879e937ace.png and /dev/null differ diff --git a/_og/e5bc60f1045ecf004ed9c230feb5326d2eddcd26.png b/_og/e5bc60f1045ecf004ed9c230feb5326d2eddcd26.png deleted file mode 100644 index 61381e41..00000000 Binary files a/_og/e5bc60f1045ecf004ed9c230feb5326d2eddcd26.png and /dev/null differ diff --git a/_og/e7bab32753259058ce7c9174a0a4f39b25ab9752.png b/_og/e7bab32753259058ce7c9174a0a4f39b25ab9752.png deleted file mode 100644 index c5c13867..00000000 Binary files a/_og/e7bab32753259058ce7c9174a0a4f39b25ab9752.png and /dev/null differ diff --git a/_og/eaaf36083bf7ce7be7a04999f3318810e3d216c3.png b/_og/eaaf36083bf7ce7be7a04999f3318810e3d216c3.png deleted file mode 100644 index 4e89d892..00000000 Binary files a/_og/eaaf36083bf7ce7be7a04999f3318810e3d216c3.png and /dev/null differ diff --git a/_og/eb3da81173286446c2e0940a695d829a165383a1.png b/_og/eb3da81173286446c2e0940a695d829a165383a1.png deleted file mode 100644 index 81f82159..00000000 Binary files a/_og/eb3da81173286446c2e0940a695d829a165383a1.png and /dev/null differ diff --git a/_og/eb6b9c0c7794ffaa0bb87b6e65785f9abad171c6.png b/_og/eb6b9c0c7794ffaa0bb87b6e65785f9abad171c6.png new file mode 100644 index 00000000..179710a8 Binary files /dev/null and b/_og/eb6b9c0c7794ffaa0bb87b6e65785f9abad171c6.png differ diff --git a/_og/eb966ccf0596fbd106b817b8ab3fe74bb0242de4.png b/_og/eb966ccf0596fbd106b817b8ab3fe74bb0242de4.png new file mode 100644 index 00000000..aabb665f Binary files /dev/null and b/_og/eb966ccf0596fbd106b817b8ab3fe74bb0242de4.png differ diff --git a/_og/f0dc2f9c7351165f486821509b7c281775d6e174.png b/_og/f0dc2f9c7351165f486821509b7c281775d6e174.png deleted file mode 100644 index 5000ab78..00000000 Binary files a/_og/f0dc2f9c7351165f486821509b7c281775d6e174.png and /dev/null differ diff --git a/_og/f1e2bde483785d549e76773dc99fef3d853a1c0e.png b/_og/f1e2bde483785d549e76773dc99fef3d853a1c0e.png new file mode 100644 index 00000000..b5f154c6 Binary files /dev/null and b/_og/f1e2bde483785d549e76773dc99fef3d853a1c0e.png differ diff --git a/_og/f2227d898f062931e7f5df073cbac35f111078db.png b/_og/f2227d898f062931e7f5df073cbac35f111078db.png new file mode 100644 index 00000000..15cb7a32 Binary files /dev/null and b/_og/f2227d898f062931e7f5df073cbac35f111078db.png differ diff --git a/_og/f278fab3c548f92716b3878a00bcf8c9b707fa60.png b/_og/f278fab3c548f92716b3878a00bcf8c9b707fa60.png deleted file mode 100644 index 41731e99..00000000 Binary files a/_og/f278fab3c548f92716b3878a00bcf8c9b707fa60.png and /dev/null differ diff --git a/_og/f56275d6f463a5db79d44003d33ab32143f55ab3.png b/_og/f56275d6f463a5db79d44003d33ab32143f55ab3.png deleted file mode 100644 index e67993ad..00000000 Binary files a/_og/f56275d6f463a5db79d44003d33ab32143f55ab3.png and /dev/null differ diff --git a/_og/fc5c70c77ce4bc4423198438d20cd073587a12f2.png b/_og/fc5c70c77ce4bc4423198438d20cd073587a12f2.png deleted file mode 100644 index 734ccc61..00000000 Binary files a/_og/fc5c70c77ce4bc4423198438d20cd073587a12f2.png and /dev/null differ diff --git a/_og/fc6becd739c2e826a04a7dd5aef0c610158cb0cb.png b/_og/fc6becd739c2e826a04a7dd5aef0c610158cb0cb.png deleted file mode 100644 index a92ef4f7..00000000 Binary files a/_og/fc6becd739c2e826a04a7dd5aef0c610158cb0cb.png and /dev/null differ diff --git a/assets/css/styles.2f71fd91.css b/assets/css/styles.2f71fd91.css deleted file mode 100644 index ff87256d..00000000 --- a/assets/css/styles.2f71fd91.css +++ /dev/null @@ -1 +0,0 @@ -.col,.container{padding:0 var(--ifm-spacing-horizontal);width:100%}.markdown>h2,.markdown>h3,.markdown>h4,.markdown>h5,.markdown>h6{margin-bottom:calc(var(--ifm-heading-vertical-rhythm-bottom)*var(--ifm-leading))}.markdown li,body{word-wrap:break-word}body,ol ol,ol ul,ul ol,ul ul{margin:0}blockquote,pre{margin:0 0 var(--ifm-spacing-vertical)}.breadcrumbs__link,.button{transition-timing-function:var(--ifm-transition-timing-default)}.button,code{vertical-align:middle}.button--outline.button--active,.button--outline:active,.button--outline:hover,:root{--ifm-button-color:var(--ifm-font-color-base-inverse)}.menu__link:hover,a{transition:color var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.navbar--dark,:root{--ifm-navbar-link-hover-color:var(--ifm-color-primary)}:root,html[data-theme=dark]{--ifm-color-emphasis-500:var(--ifm-color-gray-500)}[data-theme=dark],html[data-theme=dark]{--ifm-color-scheme:dark}:root,[data-theme=light]{--ifm-color-scheme:light}.menu,.navbar-sidebar,body,html{overflow-x:hidden}code,pre code{--lsd-typography-generic-font-family:monospace}.events .theme-doc-breadcrumbs .breadcrumbs__link,.text--capitalize{text-transform:capitalize}.button,.dropdown__link,.footer__item,.text--truncate{white-space:nowrap}.toggleButton_dl49,html{-webkit-tap-highlight-color:transparent}.mdx-erc__title,.mdx-event-card__title{-webkit-line-clamp:2;-webkit-box-orient:vertical}.mdx-cta-section .mdx-cta-section--align-center .mdx-cta-button,.mdx-video-section .mdx-video-section--align-center .mdx-cta-button{margin-inline:auto}.mdx-cta-section--full-width .mdx-cta-section__link,.mdx-video-section--full-width .mdx-video-section__link{grid-area:2/2/3/3}*,.buttonGroup_Qu4e button,.dropdownNavbarItem_o23I,.events .theme-doc-breadcrumbs .breadcrumbs__item--active,.mdx-event-info__container,.mdx-news-card__external-link-icon,.mdx-numbers-view-number,.root_xZfc,.toggle_K23S{box-sizing:border-box}.clean-list,.containsTaskList_mC6p,.dropdown__menu,.menu__list,.root_N57j ul li{list-style:none}:root{--ifm-dark-value:10%;--ifm-darker-value:15%;--ifm-darkest-value:30%;--ifm-light-value:15%;--ifm-lighter-value:30%;--ifm-lightest-value:50%;--ifm-contrast-background-value:90%;--ifm-contrast-foreground-value:70%;--ifm-contrast-background-dark-value:70%;--ifm-contrast-foreground-dark-value:90%;--ifm-color-primary:#3578e5;--ifm-color-secondary:#ebedf0;--ifm-color-success:#00a400;--ifm-color-info:#54c7ec;--ifm-color-warning:#ffba00;--ifm-color-danger:#fa383e;--ifm-color-primary-dark:#306cce;--ifm-color-primary-darker:#2d66c3;--ifm-color-primary-darkest:#2554a0;--ifm-color-primary-light:#538ce9;--ifm-color-primary-lighter:#72a1ed;--ifm-color-primary-lightest:#9abcf2;--ifm-color-primary-contrast-background:#ebf2fc;--ifm-color-primary-contrast-foreground:#102445;--ifm-color-secondary-dark:#d4d5d8;--ifm-color-secondary-darker:#c8c9cc;--ifm-color-secondary-darkest:#a4a6a8;--ifm-color-secondary-light:#eef0f2;--ifm-color-secondary-lighter:#f1f2f5;--ifm-color-secondary-lightest:#f5f6f8;--ifm-color-secondary-contrast-background:#fdfdfe;--ifm-color-secondary-contrast-foreground:#474748;--ifm-color-success-dark:#009400;--ifm-color-success-darker:#008b00;--ifm-color-success-darkest:#007300;--ifm-color-success-light:#26b226;--ifm-color-success-lighter:#4dbf4d;--ifm-color-success-lightest:#80d280;--ifm-color-success-contrast-background:#e6f6e6;--ifm-color-success-contrast-foreground:#003100;--ifm-color-info-dark:#4cb3d4;--ifm-color-info-darker:#47a9c9;--ifm-color-info-darkest:#3b8ba5;--ifm-color-info-light:#6ecfef;--ifm-color-info-lighter:#87d8f2;--ifm-color-info-lightest:#aae3f6;--ifm-color-info-contrast-background:#eef9fd;--ifm-color-info-contrast-foreground:#193c47;--ifm-color-warning-dark:#e6a700;--ifm-color-warning-darker:#d99e00;--ifm-color-warning-darkest:#b38200;--ifm-color-warning-light:#ffc426;--ifm-color-warning-lighter:#ffcf4d;--ifm-color-warning-lightest:#ffdd80;--ifm-color-warning-contrast-background:#fff8e6;--ifm-color-warning-contrast-foreground:#4d3800;--ifm-color-danger-dark:#e13238;--ifm-color-danger-darker:#d53035;--ifm-color-danger-darkest:#af272b;--ifm-color-danger-light:#fb565b;--ifm-color-danger-lighter:#fb7478;--ifm-color-danger-lightest:#fd9c9f;--ifm-color-danger-contrast-background:#ffebec;--ifm-color-danger-contrast-foreground:#4b1113;--ifm-color-gray-0:var(--ifm-color-white);--ifm-color-gray-100:#f5f6f7;--ifm-color-gray-200:#ebedf0;--ifm-color-gray-300:#dadde1;--ifm-color-gray-400:#ccd0d5;--ifm-color-gray-500:#bec3c9;--ifm-color-gray-600:#8d949e;--ifm-color-gray-700:#606770;--ifm-color-gray-800:#444950;--ifm-color-gray-900:#1c1e21;--ifm-color-emphasis-0:var(--ifm-color-gray-0);--ifm-color-emphasis-100:var(--ifm-color-gray-100);--ifm-color-emphasis-200:var(--ifm-color-gray-200);--ifm-color-emphasis-300:var(--ifm-color-gray-300);--ifm-color-emphasis-400:var(--ifm-color-gray-400);--ifm-color-emphasis-600:var(--ifm-color-gray-600);--ifm-color-emphasis-700:var(--ifm-color-gray-700);--ifm-color-emphasis-800:var(--ifm-color-gray-800);--ifm-color-emphasis-900:var(--ifm-color-gray-900);--ifm-color-emphasis-1000:var(--ifm-color-gray-1000);--ifm-color-content:var(--ifm-color-emphasis-900);--ifm-color-content-inverse:var(--ifm-color-emphasis-0);--ifm-color-content-secondary:#525860;--ifm-background-color:#0000;--ifm-background-surface-color:var(--ifm-color-content-inverse);--ifm-global-radius:0.4rem;--ifm-font-color-base:var(--ifm-color-content);--ifm-font-color-base-inverse:var(--ifm-color-content-inverse);--ifm-font-color-secondary:var(--ifm-color-content-secondary);--ifm-font-family-base:system-ui,-apple-system,Segoe UI,Roboto,Ubuntu,Cantarell,Noto Sans,sans-serif,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";--ifm-font-weight-light:300;--ifm-font-weight-semibold:500;--ifm-font-weight-bold:700;--ifm-line-height-base:1.65;--ifm-global-spacing:1rem;--ifm-spacing-horizontal:var(--ifm-global-spacing);--ifm-transition-fast:200ms;--ifm-transition-slow:400ms;--ifm-transition-timing-default:cubic-bezier(0.08,0.52,0.52,1);--ifm-global-shadow-lw:0 1px 2px 0 #0000001a;--ifm-global-shadow-md:0 5px 40px #0003;--ifm-global-shadow-tl:0 12px 28px 0 #0003,0 2px 4px 0 #0000001a;--ifm-z-index-dropdown:100;--ifm-z-index-fixed:200;--ifm-z-index-overlay:400;--ifm-container-width:1140px;--ifm-container-width-xl:1320px;--ifm-code-background:#f6f7f8;--ifm-code-border-radius:var(--ifm-global-radius);--ifm-code-font-size:90%;--ifm-code-padding-horizontal:0.1rem;--ifm-code-padding-vertical:0.1rem;--ifm-pre-background:var(--ifm-code-background);--ifm-pre-border-radius:var(--ifm-code-border-radius);--ifm-pre-color:inherit;--ifm-pre-line-height:1.45;--ifm-pre-padding:1rem;--ifm-heading-color:inherit;--ifm-heading-margin-top:0;--ifm-heading-margin-bottom:var(--ifm-spacing-vertical);--ifm-heading-font-family:var(--ifm-font-family-base);--ifm-heading-font-weight:var(--ifm-font-weight-bold);--ifm-heading-line-height:1.25;--ifm-h1-font-size:2rem;--ifm-h2-font-size:1.5rem;--ifm-h3-font-size:1.25rem;--ifm-h4-font-size:1rem;--ifm-h5-font-size:0.875rem;--ifm-h6-font-size:0.85rem;--ifm-leading:calc(var(--ifm-leading-desktop)*1rem);--ifm-table-cell-padding:0.75rem;--ifm-table-background:#0000;--ifm-table-stripe-background:#00000008;--ifm-table-border-width:1px;--ifm-table-border-color:var(--ifm-color-emphasis-300);--ifm-table-head-background:inherit;--ifm-table-head-color:inherit;--ifm-table-head-font-weight:var(--ifm-font-weight-bold);--ifm-table-cell-color:inherit;--ifm-link-color:var(--ifm-color-primary);--ifm-link-hover-color:var(--ifm-link-color);--ifm-link-hover-decoration:underline;--ifm-blockquote-font-size:var(--ifm-font-size-base);--ifm-blockquote-border-left-width:2px;--ifm-blockquote-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-blockquote-padding-vertical:0;--ifm-blockquote-color:var(--ifm-color-emphasis-800);--ifm-blockquote-border-color:var(--ifm-color-emphasis-300);--ifm-hr-background-color:var(--ifm-color-emphasis-500);--ifm-scrollbar-track-background-color:#f1f1f1;--ifm-scrollbar-thumb-background-color:silver;--ifm-scrollbar-thumb-hover-background-color:#a7a7a7;--ifm-alert-background-color:inherit;--ifm-alert-border-color:inherit;--ifm-alert-border-radius:var(--ifm-global-radius);--ifm-alert-color:var(--ifm-font-color-base);--ifm-badge-background-color:inherit;--ifm-badge-border-color:inherit;--ifm-badge-color:var(--ifm-color-white);--ifm-breadcrumb-color-active:var(--ifm-color-primary);--ifm-breadcrumb-item-background-active:var(--ifm-hover-overlay);--ifm-button-background-color:inherit;--ifm-button-border-color:var(--ifm-button-background-color);--ifm-button-border-width:var(--ifm-global-border-width);--ifm-button-font-weight:var(--ifm-font-weight-bold);--ifm-button-padding-horizontal:1.5rem;--ifm-button-padding-vertical:0.375rem;--ifm-card-background-color:var(--ifm-background-surface-color);--ifm-card-border-radius:calc(var(--ifm-global-radius)*2);--ifm-card-horizontal-spacing:var(--ifm-global-spacing);--ifm-card-vertical-spacing:var(--ifm-global-spacing);--ifm-toc-border-color:var(--ifm-color-emphasis-300);--ifm-toc-link-color:var(--ifm-color-content-secondary);--ifm-toc-padding-vertical:0.5rem;--ifm-toc-padding-horizontal:0.5rem;--ifm-dropdown-background-color:var(--ifm-background-surface-color);--ifm-dropdown-font-weight:var(--ifm-font-weight-semibold);--ifm-dropdown-link-color:var(--ifm-font-color-base);--ifm-dropdown-hover-background-color:var(--ifm-hover-overlay);--ifm-footer-background-color:var(--ifm-color-emphasis-100);--ifm-footer-color:inherit;--ifm-footer-link-color:var(--ifm-color-emphasis-700);--ifm-footer-link-hover-color:var(--ifm-color-primary);--ifm-footer-link-horizontal-spacing:0.5rem;--ifm-footer-padding-horizontal:calc(var(--ifm-spacing-horizontal)*2);--ifm-footer-padding-vertical:calc(var(--ifm-spacing-vertical)*2);--ifm-footer-title-color:inherit;--ifm-footer-logo-max-width:min(30rem,90vw);--ifm-hero-background-color:var(--ifm-background-surface-color);--ifm-hero-text-color:var(--ifm-color-emphasis-800);--ifm-menu-color:var(--ifm-color-emphasis-700);--ifm-menu-color-active:var(--ifm-color-primary);--ifm-menu-color-background-active:var(--ifm-hover-overlay);--ifm-menu-color-background-hover:var(--ifm-hover-overlay);--ifm-menu-link-padding-horizontal:0.75rem;--ifm-menu-link-padding-vertical:0.375rem;--ifm-navbar-background-color:var(--ifm-background-surface-color);--ifm-navbar-height:3.75rem;--ifm-navbar-link-color:var(--ifm-font-color-base);--ifm-navbar-link-active-color:var(--ifm-link-color);--ifm-navbar-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-navbar-padding-vertical:calc(var(--ifm-spacing-vertical)*0.5);--ifm-navbar-search-input-background-color:var(--ifm-color-emphasis-200);--ifm-navbar-search-input-color:var(--ifm-color-emphasis-800);--ifm-navbar-search-input-placeholder-color:var(--ifm-color-emphasis-500);--ifm-navbar-sidebar-width:83vw;--ifm-pagination-color-active:var(--ifm-color-primary);--ifm-pagination-nav-color-hover:var(--ifm-color-primary);--ifm-pills-color-active:var(--ifm-color-primary);--ifm-pills-color-background-active:var(--ifm-hover-overlay);--ifm-tabs-color:var(--ifm-font-color-secondary);--ifm-tabs-color-active:var(--ifm-color-primary);--ifm-tabs-color-active-border:var(--ifm-tabs-color-active)}.badge--danger,.badge--info,.badge--primary,.badge--secondary,.badge--success,.badge--warning{--ifm-badge-border-color:var(--ifm-badge-background-color)}.button--link,.button--outline{--ifm-button-background-color:#0000}html{-webkit-font-smoothing:antialiased;-webkit-text-size-adjust:100%;text-size-adjust:100%;background-color:var(--ifm-background-color);color:var(--ifm-font-color-base);color-scheme:var(--ifm-color-scheme);font:var(--ifm-font-size-base)/var(--ifm-line-height-base) var(--ifm-font-family-base);text-rendering:optimizelegibility}body{background-color:rgb(var(--lsd-surface-primary));height:auto!important}iframe{border:0;color-scheme:auto}.container{margin:0 auto;max-width:var(--ifm-container-width)}.container--fluid{max-width:inherit}.row{display:flex;flex-wrap:wrap;margin:0 calc(var(--ifm-spacing-horizontal)*-1)}.row--no-gutters{margin-left:0;margin-right:0}.margin-horiz--none,.margin-right--none{margin-right:0!important}.row--no-gutters>.col{padding-left:0;padding-right:0}.row--align-top{align-items:flex-start}.row--align-bottom{align-items:flex-end}.row--align-center{align-items:center}.row--align-stretch{align-items:stretch}.row--align-baseline{align-items:baseline}.col{--ifm-col-width:100%;flex:1 0;margin-left:0;max-width:var(--ifm-col-width)}.padding-bottom--none,.padding-vert--none{padding-bottom:0!important}.padding-top--none,.padding-vert--none{padding-top:0!important}.padding-horiz--none,.padding-left--none{padding-left:0!important}.padding-horiz--none,.padding-right--none{padding-right:0!important}.col[class*=col--]{flex:0 0 var(--ifm-col-width)}.col--1{--ifm-col-width:8.33333%}.col--offset-1{margin-left:8.33333%}.col--2{--ifm-col-width:16.66667%}.col--offset-2{margin-left:16.66667%}.col--3{--ifm-col-width:25%}.col--offset-3{margin-left:25%}.col--4{--ifm-col-width:33.33333%}.col--offset-4{margin-left:33.33333%}.col--5{--ifm-col-width:41.66667%}.col--offset-5{margin-left:41.66667%}.col--6{--ifm-col-width:50%}.col--offset-6{margin-left:50%}.col--7{--ifm-col-width:58.33333%}.col--offset-7{margin-left:58.33333%}.col--8{--ifm-col-width:66.66667%}.col--offset-8{margin-left:66.66667%}.col--9{--ifm-col-width:75%}.col--offset-9{margin-left:75%}.col--10{--ifm-col-width:83.33333%}.col--offset-10{margin-left:83.33333%}.col--11{--ifm-col-width:91.66667%}.col--offset-11{margin-left:91.66667%}.col--12{--ifm-col-width:100%}.col--offset-12{margin-left:100%}.blog-wrapper main,.margin-horiz--none,.margin-left--none{margin-left:0!important}.margin--none{margin:0!important}.blog-archive-page .main-wrapper main>section,.margin-top--none,.mdx-grid>.mdx-grid__content .mdx-news-card .mdx-news-card__title{margin-top:0!important}.margin-bottom--none,.markdown>:last-child{margin-bottom:0!important}.main-wrapper>.container,.margin-vert--none{margin-bottom:0!important;margin-top:0!important}.margin-bottom--xs,.margin-vert--xs{margin-bottom:.25rem!important}.margin-top--xs,.margin-vert--xs{margin-top:.25rem!important}.margin-horiz--xs,.margin-left--xs{margin-left:.25rem!important}.margin-horiz--xs,.margin-right--xs{margin-right:.25rem!important}.margin--xs{margin:.25rem!important}.margin-bottom--sm,.margin-vert--sm{margin-bottom:.5rem!important}.margin-top--sm,.margin-vert--sm{margin-top:.5rem!important}.margin-horiz--sm,.margin-left--sm{margin-left:.5rem!important}.margin-horiz--sm,.margin-right--sm{margin-right:.5rem!important}.margin--sm{margin:.5rem!important}.margin-bottom--md,.margin-vert--md{margin-bottom:1rem!important}.margin-top--md,.margin-vert--md{margin-top:1rem!important}.margin-horiz--md,.margin-left--md{margin-left:1rem!important}.margin-horiz--md,.margin-right--md{margin-right:1rem!important}.margin--md{margin:1rem!important}.margin-bottom--lg,.margin-vert--lg{margin-bottom:2rem!important}.margin-top--lg,.margin-vert--lg{margin-top:2rem!important}.margin-horiz--lg,.margin-left--lg{margin-left:2rem!important}.margin-horiz--lg,.margin-right--lg{margin-right:2rem!important}.margin--lg{margin:2rem!important}.margin-bottom--xl,.margin-vert--xl{margin-bottom:5rem!important}.margin-top--xl,.margin-vert--xl{margin-top:5rem!important}.margin-horiz--xl,.margin-left--xl{margin-left:5rem!important}.margin-horiz--xl,.margin-right--xl{margin-right:5rem!important}.margin--xl{margin:5rem!important}.padding--none{padding:0!important}.padding-bottom--xs,.padding-vert--xs{padding-bottom:.25rem!important}.padding-top--xs,.padding-vert--xs{padding-top:.25rem!important}.padding-horiz--xs,.padding-left--xs{padding-left:.25rem!important}.padding-horiz--xs,.padding-right--xs{padding-right:.25rem!important}.padding--xs{padding:.25rem!important}.padding-bottom--sm,.padding-vert--sm{padding-bottom:.5rem!important}.padding-top--sm,.padding-vert--sm{padding-top:.5rem!important}.padding-horiz--sm,.padding-left--sm{padding-left:.5rem!important}.padding-horiz--sm,.padding-right--sm{padding-right:.5rem!important}.padding--sm{padding:.5rem!important}.padding-bottom--md,.padding-vert--md{padding-bottom:1rem!important}.padding-top--md,.padding-vert--md{padding-top:1rem!important}.padding-horiz--md,.padding-left--md{padding-left:1rem!important}.padding-horiz--md,.padding-right--md{padding-right:1rem!important}.padding--md{padding:1rem!important}.padding-bottom--lg,.padding-vert--lg{padding-bottom:2rem!important}.padding-top--lg,.padding-vert--lg{padding-top:2rem!important}.padding-horiz--lg,.padding-left--lg{padding-left:2rem!important}.padding-horiz--lg,.padding-right--lg{padding-right:2rem!important}.padding--lg{padding:2rem!important}.padding-bottom--xl,.padding-vert--xl{padding-bottom:5rem!important}.padding-top--xl,.padding-vert--xl{padding-top:5rem!important}.padding-horiz--xl,.padding-left--xl{padding-left:5rem!important}.padding-horiz--xl,.padding-right--xl{padding-right:5rem!important}.padding--xl{padding:5rem!important}code{background-color:var(--ifm-code-background);border:.1rem solid #0000001a;border-radius:var(--ifm-code-border-radius);font-family:var(--ifm-font-family-monospace);font-size:var(--ifm-code-font-size);padding:var(--ifm-code-padding-vertical) var(--ifm-code-padding-horizontal)}a code{color:inherit;color:rgb(var(--lsd-text-primary))}pre{background-color:var(--ifm-pre-background);border-radius:var(--ifm-pre-border-radius);color:var(--ifm-pre-color);font:var(--ifm-code-font-size)/var(--ifm-pre-line-height) var(--ifm-font-family-monospace);overflow:auto;padding:var(--ifm-pre-padding)}pre code{background-color:initial;border:none;font-size:100%;line-height:inherit;padding:0}kbd{background-color:var(--ifm-color-emphasis-0);border:1px solid var(--ifm-color-emphasis-400);border-radius:.2rem;box-shadow:inset 0 -1px 0 var(--ifm-color-emphasis-400);color:var(--ifm-color-emphasis-800);font:80% var(--ifm-font-family-monospace);padding:.15rem .3rem}h1,h2,h3,h4,h5,h6{color:var(--ifm-heading-color);font-family:var(--ifm-heading-font-family);font-weight:var(--ifm-heading-font-weight);line-height:var(--ifm-heading-line-height);margin:var(--ifm-heading-margin-top) 0 var(--ifm-heading-margin-bottom) 0}.sub1,h1{font-size:var(--ifm-h1-font-size)}.sub2,h2{font-size:var(--ifm-h2-font-size)}.sub3,h3{font-size:var(--ifm-h3-font-size)}.sub4,h4{font-size:var(--ifm-h4-font-size)}.sub5,h5{font-size:var(--ifm-h5-font-size)}.sub6,h6,small{font-size:var(--ifm-h6-font-size)}.container_lyt7,.container_lyt7>svg,img{max-width:100%}img[align=right]{padding-left:var(--image-alignment-padding)}img[align=left]{padding-right:var(--image-alignment-padding)}.markdown{--ifm-h1-vertical-rhythm-top:3;--ifm-h2-vertical-rhythm-top:2;--ifm-h3-vertical-rhythm-top:1.5;--ifm-heading-vertical-rhythm-top:1.25;--ifm-h1-vertical-rhythm-bottom:1.25;--ifm-heading-vertical-rhythm-bottom:1}.markdown:after,.markdown:before{content:"";display:table}.markdown:after{clear:both}.markdown h1:first-child{--ifm-h1-font-size:3rem;margin-bottom:calc(var(--ifm-h1-vertical-rhythm-bottom)*var(--ifm-leading))}.markdown>h2{--ifm-h2-font-size:2rem;margin-top:calc(var(--ifm-h2-vertical-rhythm-top)*var(--ifm-leading));letter-spacing:-1.5px}.markdown>h3{--ifm-h3-font-size:1.5rem;margin-top:calc(var(--ifm-h3-vertical-rhythm-top)*var(--ifm-leading))}.markdown>h4,.markdown>h5,.markdown>h6{margin-top:calc(var(--ifm-heading-vertical-rhythm-top)*var(--ifm-leading))}.markdown>p,.markdown>pre,.markdown>ul{margin-bottom:var(--ifm-leading)}.markdown li>p{margin-top:var(--ifm-list-paragraph-margin)}.markdown li+li{margin-top:var(--ifm-list-item-margin)}ol,ul{margin:0 0 var(--ifm-list-margin);padding-left:var(--ifm-list-left-padding)}ol ol,ul ol{list-style-type:lower-roman}ol ol ol,ol ul ol,ul ol ol,ul ul ol{list-style-type:lower-alpha}table thead tr{border-bottom:2px solid var(--ifm-table-border-color)}table thead,table tr:nth-child(2n){background-color:var(--ifm-table-stripe-background)}table tr{background-color:var(--ifm-table-background);border-top:var(--ifm-table-border-width) solid var(--ifm-table-border-color)}table td,table th{border:var(--ifm-table-border-width) solid var(--ifm-table-border-color);padding:var(--ifm-table-cell-padding)}table th{background-color:var(--ifm-table-head-background);color:var(--ifm-table-head-color);font-weight:var(--ifm-table-head-font-weight)}strong{font-weight:var(--ifm-font-weight-bold)}a{color:var(--ifm-link-color);text-decoration:var(--ifm-link-decoration)}a:hover{color:var(--ifm-link-hover-color);text-decoration:var(--ifm-link-hover-decoration)}.button:hover,.text--no-decoration,.text--no-decoration:hover,a:not([href]){text-decoration:none}p{margin:0 0 var(--ifm-paragraph-margin-bottom)}blockquote{border-left:var(--ifm-blockquote-border-left-width) solid var(--ifm-blockquote-border-color);box-shadow:var(--ifm-blockquote-shadow);color:var(--ifm-blockquote-color);font-size:var(--ifm-blockquote-font-size);padding:var(--ifm-blockquote-padding-vertical) var(--ifm-blockquote-padding-horizontal)}blockquote>:first-child{margin-top:0}blockquote>:last-child{margin-bottom:0}hr{background-color:var(--ifm-hr-background-color);border:0;height:var(--ifm-hr-height);margin:var(--ifm-hr-margin-vertical) 0}.shadow--lw{box-shadow:var(--ifm-global-shadow-lw)!important}.shadow--md{box-shadow:var(--ifm-global-shadow-md)!important}.shadow--tl{box-shadow:var(--ifm-global-shadow-tl)!important}.text--primary,.wordWrapButtonEnabled_EoeP .wordWrapButtonIcon_Bwma{color:var(--ifm-color-primary)}.text--secondary,[data-theme=dark] .root_JWD1>div>div{color:var(--ifm-color-secondary)}.text--success{color:var(--ifm-color-success)}.text--info{color:var(--ifm-color-info)}.text--warning{color:var(--ifm-color-warning)}.text--danger{color:var(--ifm-color-danger)}.text--center{text-align:center}.text--left{text-align:left}.text--justify{text-align:justify}.text--right{text-align:right}.text--lowercase{text-transform:lowercase}.alert__heading,.mdx-hero-title--uppercase,.text--uppercase,table td strong,table th{text-transform:uppercase}.text--light{font-weight:var(--ifm-font-weight-light)}.text--normal{font-weight:var(--ifm-font-weight-normal)}.text--semibold{font-weight:var(--ifm-font-weight-semibold)}.text--bold{font-weight:var(--ifm-font-weight-bold)}.text--italic{font-style:italic}.text--truncate{overflow:hidden;text-overflow:ellipsis}.text--break{word-wrap:break-word!important;word-break:break-word!important}.clean-btn{background:none;border:none;color:inherit;cursor:pointer;font-family:inherit;padding:0}.alert,.alert .close{color:var(--ifm-alert-foreground-color)}.clean-list{padding-left:0}.alert--primary{--ifm-alert-background-color:var(--ifm-color-primary-contrast-background);--ifm-alert-background-color-highlight:#3578e526;--ifm-alert-foreground-color:var(--ifm-color-primary-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-primary-dark)}.alert--secondary{--ifm-alert-background-color:var(--ifm-color-secondary-contrast-background);--ifm-alert-background-color-highlight:#ebedf026;--ifm-alert-foreground-color:var(--ifm-color-secondary-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-secondary-dark)}.alert--success{--ifm-alert-background-color:var(--ifm-color-success-contrast-background);--ifm-alert-background-color-highlight:#00a40026;--ifm-alert-foreground-color:var(--ifm-color-success-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-success-dark)}.alert--info{--ifm-alert-background-color:var(--ifm-color-info-contrast-background);--ifm-alert-background-color-highlight:#54c7ec26;--ifm-alert-foreground-color:var(--ifm-color-info-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-info-dark)}.alert--warning{--ifm-alert-background-color:var(--ifm-color-warning-contrast-background);--ifm-alert-background-color-highlight:#ffba0026;--ifm-alert-foreground-color:var(--ifm-color-warning-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-warning-dark)}.alert--danger{--ifm-alert-background-color:var(--ifm-color-danger-contrast-background);--ifm-alert-background-color-highlight:#fa383e26;--ifm-alert-foreground-color:var(--ifm-color-danger-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-danger-dark)}.alert{--ifm-code-background:var(--ifm-alert-background-color-highlight);--ifm-link-color:var(--ifm-alert-foreground-color);--ifm-link-hover-color:var(--ifm-alert-foreground-color);--ifm-link-decoration:underline;--ifm-tabs-color:var(--ifm-alert-foreground-color);--ifm-tabs-color-active:var(--ifm-alert-foreground-color);--ifm-tabs-color-active-border:var(--ifm-alert-border-color);background-color:var(--ifm-alert-background-color);border:var(--ifm-alert-border-width) solid var(--ifm-alert-border-color);border-left-width:var(--ifm-alert-border-left-width);border-radius:var(--ifm-alert-border-radius);box-shadow:var(--ifm-alert-shadow);padding:var(--ifm-alert-padding-vertical) var(--ifm-alert-padding-horizontal)}.alert__heading{align-items:center;display:flex;font:700 var(--ifm-h5-font-size)/var(--ifm-heading-line-height) var(--ifm-heading-font-family);margin-bottom:.5rem}.alert__icon{display:inline-flex;margin-right:.4em}.alert__icon svg{fill:var(--ifm-alert-foreground-color);stroke:var(--ifm-alert-foreground-color);stroke-width:0}.alert .close{margin:calc(var(--ifm-alert-padding-vertical)*-1) calc(var(--ifm-alert-padding-horizontal)*-1) 0 0;opacity:.75}.alert .close:focus,.alert .close:hover{opacity:1}.alert a{text-decoration-color:var(--ifm-alert-border-color)}.alert a:hover{text-decoration-thickness:2px}.avatar{column-gap:var(--ifm-avatar-intro-margin);display:flex}.avatar__photo{border-radius:50%;display:block;height:var(--ifm-avatar-photo-size);overflow:hidden;width:var(--ifm-avatar-photo-size)}.card--full-height,.navbar__logo img{height:100%}.avatar__photo--sm{--ifm-avatar-photo-size:2rem}.avatar__photo--lg{--ifm-avatar-photo-size:4rem}.avatar__photo--xl{--ifm-avatar-photo-size:6rem}.avatar__intro{display:flex;flex:1 1;flex-direction:column;justify-content:center;text-align:var(--ifm-avatar-intro-alignment)}.badge,.breadcrumbs__item,.breadcrumbs__link,.button{display:inline-block}.avatar__name{font:700 var(--ifm-h4-font-size)/var(--ifm-heading-line-height) var(--ifm-font-family-base)}.avatar__subtitle{margin-top:.25rem}.avatar--vertical{--ifm-avatar-intro-alignment:center;--ifm-avatar-intro-margin:0.5rem;align-items:center;flex-direction:column}.badge{background-color:var(--ifm-badge-background-color);border:var(--ifm-badge-border-width) solid var(--ifm-badge-border-color);border-radius:var(--ifm-badge-border-radius);color:var(--ifm-badge-color);font-size:75%;font-weight:var(--ifm-font-weight-bold);line-height:1;padding:var(--ifm-badge-padding-vertical) var(--ifm-badge-padding-horizontal)}.badge--primary{--ifm-badge-background-color:var(--ifm-color-primary)}.badge--secondary{--ifm-badge-background-color:var(--ifm-color-secondary);color:var(--ifm-color-black)}.breadcrumbs__link,.button.button--secondary.button--outline:not(.button--active):not(:hover){color:var(--ifm-font-color-base)}.badge--success{--ifm-badge-background-color:var(--ifm-color-success)}.badge--info{--ifm-badge-background-color:var(--ifm-color-info)}.badge--warning{--ifm-badge-background-color:var(--ifm-color-warning)}.badge--danger{--ifm-badge-background-color:var(--ifm-color-danger)}.breadcrumbs{margin-bottom:0;padding-left:0}.breadcrumbs__item:not(:last-child):after{background:var(--ifm-breadcrumb-separator) center;content:" ";display:inline-block;filter:var(--ifm-breadcrumb-separator-filter);height:calc(var(--ifm-breadcrumb-separator-size)*var(--ifm-breadcrumb-size-multiplier)*var(--ifm-breadcrumb-separator-size-multiplier));margin:0 var(--ifm-breadcrumb-spacing);opacity:.5;width:calc(var(--ifm-breadcrumb-separator-size)*var(--ifm-breadcrumb-size-multiplier)*var(--ifm-breadcrumb-separator-size-multiplier))}.breadcrumbs__item--active .breadcrumbs__link{background:var(--ifm-breadcrumb-item-background-active);color:var(--ifm-breadcrumb-color-active)}.breadcrumbs__link{border-radius:var(--ifm-breadcrumb-border-radius);font-size:calc(1rem*var(--ifm-breadcrumb-size-multiplier));padding:calc(var(--ifm-breadcrumb-padding-vertical)*var(--ifm-breadcrumb-size-multiplier)) calc(var(--ifm-breadcrumb-padding-horizontal)*var(--ifm-breadcrumb-size-multiplier));transition-duration:var(--ifm-transition-fast);transition-property:background,color}.breadcrumbs__link:any-link:hover,.breadcrumbs__link:link:hover,.breadcrumbs__link:visited:hover,area[href].breadcrumbs__link:hover{background:var(--ifm-breadcrumb-item-background-active);text-decoration:none}.breadcrumbs--sm{--ifm-breadcrumb-size-multiplier:0.8}.breadcrumbs--lg{--ifm-breadcrumb-size-multiplier:1.2}.button{background-color:var(--ifm-button-background-color);border:var(--ifm-button-border-width) solid var(--ifm-button-border-color);border-radius:var(--ifm-button-border-radius);cursor:pointer;font-size:calc(.875rem*var(--ifm-button-size-multiplier));font-weight:var(--ifm-button-font-weight);line-height:1.5;padding:calc(var(--ifm-button-padding-vertical)*var(--ifm-button-size-multiplier)) calc(var(--ifm-button-padding-horizontal)*var(--ifm-button-size-multiplier));text-align:center;transition-duration:var(--ifm-button-transition-duration);transition-property:color,background,border-color;-webkit-user-select:none;user-select:none}.button,.button:hover{color:var(--ifm-button-color)}.button--outline{--ifm-button-color:var(--ifm-button-border-color)}.button--outline:hover{--ifm-button-background-color:var(--ifm-button-border-color)}.button--link{--ifm-button-border-color:#0000;color:var(--ifm-link-color);text-decoration:var(--ifm-link-decoration)}.button--link.button--active,.button--link:active,.button--link:hover{color:var(--ifm-link-hover-color);text-decoration:var(--ifm-link-hover-decoration)}.button.disabled,.button:disabled,.button[disabled]{opacity:.65;pointer-events:none}.button--sm{--ifm-button-size-multiplier:0.8}.button--lg{--ifm-button-size-multiplier:1.35}.button--block{display:block;width:100%}.button.button--secondary{color:var(--ifm-color-gray-900)}:where(.button--primary){--ifm-button-background-color:var(--ifm-color-primary);--ifm-button-border-color:var(--ifm-color-primary)}:where(.button--primary):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-primary-dark);--ifm-button-border-color:var(--ifm-color-primary-dark)}.button--primary.button--active,.button--primary:active{--ifm-button-background-color:var(--ifm-color-primary-darker);--ifm-button-border-color:var(--ifm-color-primary-darker)}:where(.button--secondary){--ifm-button-background-color:var(--ifm-color-secondary);--ifm-button-border-color:var(--ifm-color-secondary)}:where(.button--secondary):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-secondary-dark);--ifm-button-border-color:var(--ifm-color-secondary-dark)}.button--secondary.button--active,.button--secondary:active{--ifm-button-background-color:var(--ifm-color-secondary-darker);--ifm-button-border-color:var(--ifm-color-secondary-darker)}:where(.button--success){--ifm-button-background-color:var(--ifm-color-success);--ifm-button-border-color:var(--ifm-color-success)}:where(.button--success):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-success-dark);--ifm-button-border-color:var(--ifm-color-success-dark)}.button--success.button--active,.button--success:active{--ifm-button-background-color:var(--ifm-color-success-darker);--ifm-button-border-color:var(--ifm-color-success-darker)}:where(.button--info){--ifm-button-background-color:var(--ifm-color-info);--ifm-button-border-color:var(--ifm-color-info)}:where(.button--info):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-info-dark);--ifm-button-border-color:var(--ifm-color-info-dark)}.button--info.button--active,.button--info:active{--ifm-button-background-color:var(--ifm-color-info-darker);--ifm-button-border-color:var(--ifm-color-info-darker)}:where(.button--warning){--ifm-button-background-color:var(--ifm-color-warning);--ifm-button-border-color:var(--ifm-color-warning)}:where(.button--warning):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-warning-dark);--ifm-button-border-color:var(--ifm-color-warning-dark)}.button--warning.button--active,.button--warning:active{--ifm-button-background-color:var(--ifm-color-warning-darker);--ifm-button-border-color:var(--ifm-color-warning-darker)}:where(.button--danger){--ifm-button-background-color:var(--ifm-color-danger);--ifm-button-border-color:var(--ifm-color-danger)}:where(.button--danger):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-danger-dark);--ifm-button-border-color:var(--ifm-color-danger-dark)}.button--danger.button--active,.button--danger:active{--ifm-button-background-color:var(--ifm-color-danger-darker);--ifm-button-border-color:var(--ifm-color-danger-darker)}.button-group{display:inline-flex;gap:var(--ifm-button-group-spacing)}.button-group>.button:not(:first-child){border-bottom-left-radius:0;border-top-left-radius:0}.button-group>.button:not(:last-child){border-bottom-right-radius:0;border-top-right-radius:0}.button-group--block{display:flex;justify-content:stretch}.button-group--block>.button{flex-grow:1}.card{background-color:var(--ifm-card-background-color);border-radius:var(--ifm-card-border-radius);box-shadow:var(--ifm-global-shadow-lw);display:flex;flex-direction:column;overflow:hidden}.card__image{padding-top:var(--ifm-card-vertical-spacing)}.card__image:first-child{padding-top:0}.card__body,.card__footer,.card__header{padding:var(--ifm-card-vertical-spacing) var(--ifm-card-horizontal-spacing)}.card__body:not(:last-child),.card__footer:not(:last-child),.card__header:not(:last-child){padding-bottom:0}.card__body>:last-child,.card__footer>:last-child,.card__header>:last-child{margin-bottom:0}.card__footer{margin-top:auto}.table-of-contents{font-size:.8rem;margin-bottom:0;padding:var(--ifm-toc-padding-vertical) 0}.table-of-contents,.table-of-contents ul{list-style:none;padding-left:var(--ifm-toc-padding-horizontal)}.table-of-contents li{margin:var(--ifm-toc-padding-vertical) var(--ifm-toc-padding-horizontal)}.table-of-contents__left-border{border-left:1px solid var(--ifm-toc-border-color)}.table-of-contents__link{color:var(--ifm-toc-link-color);display:block}.table-of-contents__link--active,.table-of-contents__link--active code,.table-of-contents__link:hover,.table-of-contents__link:hover code{color:var(--ifm-color-primary);text-decoration:none}.close{color:var(--ifm-color-black);float:right;font-size:1.5rem;font-weight:var(--ifm-font-weight-bold);line-height:1;opacity:.5;padding:1rem;transition:opacity var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.breadcrumbs__item,.close:hover,.table-of-contents__link--active code{opacity:.7}.close:focus,.theme-code-block-highlighted-line .codeLineNumber_Tfdd:before{opacity:.8}.dropdown{display:inline-flex;font-weight:var(--ifm-dropdown-font-weight);position:relative;vertical-align:top}.admonitionIcon_BGV6,.root_JWD1>span svg{vertical-align:middle}.dropdown--hoverable:hover .dropdown__menu,.dropdown--show .dropdown__menu{opacity:1;pointer-events:all;transform:translateY(-1px);visibility:visible}#nprogress,.dropdown__menu,.navWrapper_ybYI,.navbar__item.dropdown .navbar__link:not([href]){pointer-events:none}.dropdown--right .dropdown__menu{left:inherit;right:0}.dropdown--nocaret .navbar__link:after{content:none!important}.dropdown__menu{background-color:var(--ifm-dropdown-background-color);border-radius:var(--ifm-global-radius);box-shadow:var(--ifm-global-shadow-md);left:0;max-height:80vh;min-width:10rem;opacity:0;overflow-y:auto;position:absolute;top:calc(100% - var(--ifm-navbar-item-padding-vertical) + .3rem);transform:translateY(-.625rem);transition-duration:var(--ifm-transition-fast);transition-property:opacity,transform,visibility;transition-timing-function:var(--ifm-transition-timing-default);visibility:hidden;z-index:var(--ifm-z-index-dropdown)}.menu__caret,.menu__link,.menu__list-item-collapsible{border-radius:.25rem;transition:background var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.dropdown__link{color:var(--ifm-dropdown-link-color);display:block;font-size:.875rem}.dropdown__link--active,.dropdown__link:hover{background-color:var(--ifm-dropdown-hover-background-color);color:var(--ifm-dropdown-link-color);text-decoration:none}.dropdown__link--active,.dropdown__link--active:hover{--ifm-dropdown-link-color:var(--ifm-link-color)}.dropdown>.navbar__link:after{border-color:currentcolor #0000;border-style:solid;border-width:.4em .4em 0;content:"";margin-left:.3em;position:relative;top:2px;transform:translateY(-50%);display:none}.menu__list-item--collapsed .menu__caret:before,.menu__list-item--collapsed .menu__link--sublist:after,.menu__list-item-collapsible>a[aria-expanded=false]>svg{transform:rotate(90deg)}.footer{background-color:var(--ifm-footer-background-color);color:var(--ifm-footer-color);padding:var(--ifm-footer-padding-vertical) var(--ifm-footer-padding-horizontal)}.footer--dark{--ifm-footer-background-color:#303846;--ifm-footer-color:var(--ifm-footer-link-color);--ifm-footer-link-color:var(--ifm-color-secondary);--ifm-footer-title-color:var(--ifm-color-white)}.footer__links{margin-bottom:1rem}.footer__link-item{color:var(--ifm-footer-link-color);line-height:2}.footer__link-item:hover{color:var(--ifm-footer-link-hover-color)}.footer__link-separator{margin:0 var(--ifm-footer-link-horizontal-spacing)}.footer__title{color:var(--ifm-footer-title-color);font:700 var(--ifm-h4-font-size)/var(--ifm-heading-line-height) var(--ifm-font-family-base);margin-bottom:var(--ifm-heading-margin-bottom)}.menu,.navbar__link{font-weight:var(--ifm-font-weight-semibold)}.docItemContainer_hrrU article>:first-child,.docItemContainer_hrrU header+*,.footer__item,.mdx-roadmap .mdx-cta-button{margin-top:0}.admonitionContent_iozl>:last-child,.footer__items{margin-bottom:0}.main-wrapper>main,.table-of-contents,[type=checkbox]{padding:0}.hero{align-items:center;background-color:var(--ifm-hero-background-color);color:var(--ifm-hero-text-color);display:flex;padding:4rem 2rem}.hero--primary{--ifm-hero-background-color:var(--ifm-color-primary);--ifm-hero-text-color:var(--ifm-font-color-base-inverse)}.hero--dark{--ifm-hero-background-color:#303846;--ifm-hero-text-color:var(--ifm-color-white)}.hero__title,.title_f1Hy{font-size:3rem}.hero__subtitle{font-size:1.5rem}.menu__list{margin:0;padding-left:0}.menu__caret,.menu__link{padding:var(--ifm-menu-link-padding-vertical) var(--ifm-menu-link-padding-horizontal)}.menu__list .menu__list{flex:0 0 100%;margin-top:.25rem;padding-left:var(--ifm-menu-link-padding-horizontal)}.menu__list-item:not(:first-child){margin-top:.25rem}.menu__list-item--collapsed .menu__list{height:0;overflow:hidden}.menu__list-item-collapsible{display:flex;flex-wrap:wrap;position:relative}.menu__caret:hover,.menu__link:hover,.menu__list-item-collapsible--active,.menu__list-item-collapsible:hover{background:var(--ifm-menu-color-background-hover)}.menu__list-item-collapsible .menu__link--active,.menu__list-item-collapsible .menu__link:hover{background:none!important}.menu__caret,.menu__link{align-items:center;display:flex}.navbar-sidebar,.navbar-sidebar__backdrop{bottom:0;opacity:0;top:0;transition-duration:var(--ifm-transition-fast);transition-timing-function:ease-in-out;left:0;visibility:hidden}.menu__link{color:var(--ifm-menu-color);flex:1;line-height:1.25}.menu__link:hover{color:var(--ifm-menu-color);text-decoration:none}.menu__caret:before,.menu__link--sublist-caret:after{content:"";filter:var(--ifm-menu-link-sublist-icon-filter);height:1.25rem;transform:rotate(180deg);transition:transform var(--ifm-transition-fast) linear;width:1.25rem}.menu__link--sublist-caret:after{background:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem;margin-left:auto;min-width:1.25rem}.menu__link--active,.menu__link--active:hover{color:var(--ifm-menu-color-active)}.navbar__brand,.navbar__link{color:var(--ifm-navbar-link-color)}.menu__link--active:not(.menu__link--sublist){background-color:var(--ifm-menu-color-background-active)}.menu__caret:before{background:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem}.navbar--dark,html[data-theme=dark]{--ifm-menu-link-sublist-icon-filter:invert(100%) sepia(94%) saturate(17%) hue-rotate(223deg) brightness(104%) contrast(98%)}.navbar{background-color:var(--ifm-navbar-background-color);box-shadow:var(--ifm-navbar-shadow);padding:var(--ifm-navbar-padding-vertical) var(--ifm-navbar-padding-horizontal)}.icon_S7Kx,.navbar,.navbar>.container,.navbar>.container-fluid{display:flex}.navbar--fixed-top{position:sticky;top:0;z-index:var(--ifm-z-index-fixed)}.navbar__inner{display:flex;flex-wrap:wrap;justify-content:space-between;width:100%}.navbar__brand{align-items:center;display:flex;margin-right:1rem;min-width:0}.navbar__brand:hover{color:var(--ifm-navbar-link-hover-color);text-decoration:none}.announcementBarContent_xLdY,.navbar__title{flex:1 1 auto}.navbar__toggle{display:none;margin-right:.5rem}.navbar__logo{flex:0 0 auto;margin-right:.5rem}.navbar__items{align-items:center;display:flex;flex:1;min-width:0}.navbar__items--center{flex:0 0 auto}.blog-wrapper>.container>.row>aside>nav>ul li,.navbar__items--center .navbar__brand{margin:0}.navbar__items--center+.navbar__items--right{flex:1}.navbar__items--right{flex:0 0 auto;justify-content:flex-end}.navbar__items--right>:last-child{padding-right:0}.navbar__item{display:inline-block;padding:var(--ifm-navbar-item-padding-vertical) var(--ifm-navbar-item-padding-horizontal)}.navbar__link--active,.navbar__link:hover{color:var(--ifm-navbar-link-hover-color);text-decoration:none}.navbar--dark,.navbar--primary{--ifm-menu-color:var(--ifm-color-gray-300);--ifm-navbar-link-color:var(--ifm-color-gray-100);--ifm-navbar-search-input-background-color:#ffffff1a;--ifm-navbar-search-input-placeholder-color:#ffffff80;color:var(--ifm-color-white)}.navbar--dark{--ifm-navbar-background-color:#242526;--ifm-menu-color-background-active:#ffffff0d;--ifm-navbar-search-input-color:var(--ifm-color-white)}.navbar--primary{--ifm-navbar-background-color:var(--ifm-color-primary);--ifm-navbar-link-hover-color:var(--ifm-color-white);--ifm-menu-color-active:var(--ifm-color-white);--ifm-navbar-search-input-color:var(--ifm-color-emphasis-500)}.navbar__search-input{-webkit-appearance:none;appearance:none;background:var(--ifm-navbar-search-input-background-color) var(--ifm-navbar-search-input-icon) no-repeat .75rem center/1rem 1rem;border:none;border-radius:2rem;color:var(--ifm-navbar-search-input-color);cursor:text;display:inline-block;font-size:.9rem;height:2rem;padding:0 .5rem 0 2.25rem;width:12.5rem}.cursor-pointer,.pills__item,.tabs__item{cursor:pointer}.navbar__search-input::placeholder{color:var(--ifm-navbar-search-input-placeholder-color)}.navbar-sidebar{background-color:var(--ifm-navbar-background-color);box-shadow:var(--ifm-global-shadow-md);position:fixed;transform:translate3d(-100%,0,0);transition-property:opacity,visibility,transform;width:var(--ifm-navbar-sidebar-width)}.navbar-sidebar--show .navbar-sidebar,.navbar-sidebar__items{transform:translateZ(0)}.navbar-sidebar--show .navbar-sidebar,.navbar-sidebar--show .navbar-sidebar__backdrop{opacity:1;visibility:visible}.navbar-sidebar__backdrop{background-color:#0009;position:fixed;right:0;transition-property:opacity,visibility}.footer,.row{position:relative}.navbar-sidebar__brand{align-items:center;box-shadow:var(--ifm-navbar-shadow);display:flex;flex:1;height:var(--ifm-navbar-height);padding:var(--ifm-navbar-padding-vertical) var(--ifm-navbar-padding-horizontal)}.navbar-sidebar__items{display:flex;height:calc(100% - var(--ifm-navbar-height));transition:transform var(--ifm-transition-fast) ease-in-out}.navbar-sidebar__items--show-secondary{transform:translate3d(calc((var(--ifm-navbar-sidebar-width))*-1),0,0)}.navbar-sidebar__item{flex-shrink:0;padding:.5rem;width:calc(var(--ifm-navbar-sidebar-width))}.navbar-sidebar__back{background:var(--ifm-menu-color-background-active);font-size:15px;font-weight:var(--ifm-button-font-weight);position:relative;text-align:left;top:-.5rem}.navbar-sidebar__close{display:flex;margin-left:auto}.pagination{column-gap:var(--ifm-pagination-page-spacing);display:flex;font-size:var(--ifm-pagination-font-size);padding-left:0}.pagination--sm{--ifm-pagination-font-size:0.8rem;--ifm-pagination-padding-horizontal:0.8rem;--ifm-pagination-padding-vertical:0.2rem}.pagination--lg{--ifm-pagination-font-size:1.2rem;--ifm-pagination-padding-horizontal:1.2rem;--ifm-pagination-padding-vertical:0.3rem}.pagination__item{display:inline-flex}.pagination__item>span{padding:var(--ifm-pagination-padding-vertical)}.pagination__item--active .pagination__link{color:var(--ifm-pagination-color-active)}.pagination__item--active .pagination__link,.pagination__item:not(.pagination__item--active):hover .pagination__link{background:var(--ifm-pagination-item-active-background)}.pagination__item--disabled,.pagination__item[disabled]{opacity:.25;pointer-events:none}.pagination__link{border-radius:var(--ifm-pagination-border-radius);color:var(--ifm-font-color-base);display:inline-block;padding:var(--ifm-pagination-padding-vertical) var(--ifm-pagination-padding-horizontal);transition:background var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.pagination__link:hover{text-decoration:none}.pagination-nav{grid-gap:var(--ifm-spacing-horizontal);display:grid;gap:var(--ifm-spacing-horizontal);grid-template-columns:repeat(2,1fr)}.footer>div,.grid{grid-template-columns:repeat(24,1fr)}.pagination-nav__link{border:1px solid var(--ifm-color-emphasis-300);border-radius:var(--ifm-pagination-nav-border-radius);display:block;height:100%;line-height:var(--ifm-heading-line-height);padding:var(--ifm-global-spacing);transition:border-color var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.pagination-nav__link:hover{border-color:var(--ifm-pagination-nav-color-hover);text-decoration:none}.blog-wrapper>.container>.row>aside>nav>div:first-child,.blog-wrapper>.container>.row>aside>nav>ul li a{border-left:1px solid #0000;font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important}.dropdown__link:hover,.footer__item a{text-decoration:underline}.pagination-nav__label{font-size:var(--ifm-h4-font-size);font-weight:var(--ifm-heading-font-weight);word-break:break-word}.pagination-nav__link--prev .pagination-nav__label:before{content:"« "}.pagination-nav__link--next .pagination-nav__label:after{content:" »"}.pagination-nav__sublabel{color:var(--ifm-color-content-secondary);font-size:var(--ifm-h5-font-size);font-weight:var(--ifm-font-weight-semibold);margin-bottom:.25rem}.pills__item,.tabs{font-weight:var(--ifm-font-weight-bold)}.pills{display:flex;gap:var(--ifm-pills-spacing);padding-left:0}.pills__item{border-radius:.5rem;display:inline-block;padding:.25rem 1rem;transition:background var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.navbar__left-items>.navbar__item:first-of-type,.tabs,:not(.containsTaskList_mC6p>li)>.containsTaskList_mC6p{padding-left:0}.pills__item--active{color:var(--ifm-pills-color-active)}.pills__item--active,.pills__item:not(.pills__item--active):hover{background:var(--ifm-pills-color-background-active)}.pills--block{justify-content:stretch}.pills--block .pills__item{flex-grow:1;text-align:center}.tabs{color:var(--ifm-tabs-color);display:flex;margin-bottom:0;overflow-x:auto}.tabs__item{border-bottom:3px solid #0000;border-radius:var(--ifm-global-radius);display:inline-flex;padding:var(--ifm-tabs-padding-vertical) var(--ifm-tabs-padding-horizontal);transition:background-color var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.tabs__item--active{border-bottom-color:var(--ifm-tabs-color-active-border);border-bottom-left-radius:0;border-bottom-right-radius:0;color:var(--ifm-tabs-color-active)}.tabs__item:hover{background-color:var(--ifm-hover-overlay)}.tabs--block{justify-content:stretch}.tabs--block .tabs__item{flex-grow:1;justify-content:center}html[data-theme=dark]{--ifm-color-emphasis-0:var(--ifm-color-gray-1000);--ifm-color-emphasis-100:var(--ifm-color-gray-900);--ifm-color-emphasis-200:var(--ifm-color-gray-800);--ifm-color-emphasis-300:var(--ifm-color-gray-700);--ifm-color-emphasis-400:var(--ifm-color-gray-600);--ifm-color-emphasis-600:var(--ifm-color-gray-400);--ifm-color-emphasis-700:var(--ifm-color-gray-300);--ifm-color-emphasis-800:var(--ifm-color-gray-200);--ifm-color-emphasis-900:var(--ifm-color-gray-100);--ifm-color-emphasis-1000:var(--ifm-color-gray-0);--ifm-background-color:#1b1b1d;--ifm-background-surface-color:#242526;--ifm-hover-overlay:#ffffff0d;--ifm-color-content:#e3e3e3;--ifm-color-content-secondary:#fff;--ifm-breadcrumb-separator-filter:invert(64%) sepia(11%) saturate(0%) hue-rotate(149deg) brightness(99%) contrast(95%);--ifm-code-background:#ffffff1a;--ifm-scrollbar-track-background-color:#444;--ifm-scrollbar-thumb-background-color:#686868;--ifm-scrollbar-thumb-hover-background-color:#7a7a7a;--ifm-table-stripe-background:#ffffff12;--ifm-toc-border-color:var(--ifm-color-emphasis-200);--ifm-color-primary-contrast-background:#102445;--ifm-color-primary-contrast-foreground:#ebf2fc;--ifm-color-secondary-contrast-background:#474748;--ifm-color-secondary-contrast-foreground:#fdfdfe;--ifm-color-success-contrast-background:#003100;--ifm-color-success-contrast-foreground:#e6f6e6;--ifm-color-info-contrast-background:#193c47;--ifm-color-info-contrast-foreground:#eef9fd;--ifm-color-warning-contrast-background:#4d3800;--ifm-color-warning-contrast-foreground:#fff8e6;--ifm-color-danger-contrast-background:#4b1113;--ifm-color-danger-contrast-foreground:#ffebec}:root{--docusaurus-progress-bar-color:var(--ifm-color-primary);--content-padding:16px;--container-max-width:1072px;--pagination-nav-margin-top:96px;--ifm-navbar-link-color:rgb(var(--lsd-text-primary));--ifm-navbar-background-color:rgb(var(--lsd-surface-primary));--ifm-dropdown-link-color:rgb(var(--lsd-text-primary));--ifm-dropdown-hover-background-color:rgb(var(--lsd-surface-secondary));--ifm-background-color:rgb(var(--lsd-surface-primary));--ifm-color-white:#fff;--ifm-color-black:#000;--ifm-color-primary:rgb(var(--lsd-theme-primary));--ifm-color-secondary:rgb(var(--lsd-theme-secondary));--prism-background-color:rgb(var(--lsd-theme-primary));--ifm-color-gray-0:rgb(var(--lsd-surface-primary));--ifm-color-gray-100:#f8f8fa;--ifm-color-gray-200:#f7f7f8;--ifm-color-gray-300:#eeeef0;--ifm-color-gray-400:#c0c0c2;--ifm-color-gray-500:#828285;--ifm-color-gray-600:#474747;--ifm-color-gray-700:#373738;--ifm-color-gray-800:#1a1a1a;--ifm-color-gray-900:#151515;--ifm-color-gray-1000:var(--ifm-color-black);--ifm-font-color-base-inverse:var(--ifm-color-black);--docusaurus-highlighted-code-line-bg:#ffffff26;--ifm-card-background-color:var(--ifm-color-gray-100)!important;--ifm-alert-foreground-color:var(--ifm-color-gray-700);--ifm-button-background-color:rgb(var(--lsd-surface-primary));--ifm-global-border-width:1px;--ifm-global-radius:0.6rem;--ifm-hover-overlay:#0000000d;--ifm-font-family-base:var(--lsd-typography-generic-font-family);--ifm-font-family-monospace:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;--lgs-font-family-secondary:Georgia,"Times New Roman",serif;--ifm-font-size-base:100%;--ifm-line-height-base:1.625;--ifm-font-size-secondary:100%;--ifm-line-height-secondary:1.5;--ifm-font-weight-light:400;--ifm-font-weight-normal:400;--ifm-font-weight-semibold:400;--ifm-font-weight-bold:600;--ifm-font-weight-base:var(--ifm-font-weight-normal);--ifm-h1-font-size:var(--lsd-h1-fontSize);--ifm-h2-font-size:var(--lsd-h2-fontSize);--ifm-h3-font-size:var(--lsd-h3-fontSize);--ifm-h4-font-size:var(--lsd-h4-fontSize);--ifm-h5-font-size:var(--lsd-h5-fontSize);--ifm-h6-font-size:var(--lsd-h6-fontSize);--ifm-global-spacing:24px;--ifm-spacing-vertical:var(--ifm-global-spacing);--ifm-spacing-horizontal:0.75rem;--ifm-image-alignment-padding:1.25rem;--ifm-leading-desktop:1.25;--ifm-leading:24px;--ifm-list-left-padding:2rem;--ifm-list-margin:1rem;--ifm-list-item-margin:0.25rem;--ifm-list-paragraph-margin:1rem;--ifm-code-background:rgb(var(--lsd-text-primary))!important;--ifm-pre-padding:1.125rem;--ifm-link-decoration:none;--ifm-paragraph-margin-bottom:var(--ifm-leading);--ifm-blockquote-border-left-width:3px;--ifm-blockquote-shadow:none;--ifm-hr-height:1px;--ifm-hr-margin-vertical:1.5rem;--ifm-scrollbar-size:7px;--ifm-alert-border-width:0px;--ifm-alert-border-left-width:5px;--ifm-alert-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-alert-padding-vertical:var(--ifm-spacing-vertical);--ifm-alert-shadow:var(--ifm-global-shadow-lw);--ifm-avatar-intro-margin:1rem;--ifm-avatar-intro-alignment:inherit;--ifm-avatar-photo-size:3rem;--ifm-badge-border-radius:var(--ifm-global-radius);--ifm-badge-border-width:var(--ifm-global-border-width);--ifm-badge-padding-horizontal:calc(var(--ifm-spacing-horizontal)*0.5);--ifm-badge-padding-vertical:calc(var(--ifm-spacing-vertical)*0.25);--ifm-breadcrumb-border-radius:1.5rem;--ifm-breadcrumb-spacing:0.5rem;--ifm-breadcrumb-padding-horizontal:0.8rem;--ifm-breadcrumb-padding-vertical:0.4rem;--ifm-breadcrumb-size-multiplier:1;--ifm-breadcrumb-separator:url('data:image/svg+xml;utf8,');--ifm-breadcrumb-separator-filter:none;--ifm-breadcrumb-separator-size:0.5rem;--ifm-breadcrumb-separator-size-multiplier:1.25;--ifm-button-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-button-padding-vertical:0.4rem;--ifm-button-size-multiplier:1;--ifm-button-transition-duration:var(--ifm-transition-fast);--ifm-button-border-radius:calc(var(--ifm-global-radius)*var(--ifm-button-size-multiplier));--ifm-button-group-spacing:2px;--ifm-card-border-radius:var(--ifm-global-radius);--ifm-card-horizontal-spacing:var(--ifm-spacing-horizontal);--ifm-card-vertical-spacing:var(--ifm-spacing-vertical);--ifm-footer-link-horizontal-spacing:0;--ifm-footer-padding-horizontal:var(--content-padding);--ifm-footer-padding-vertical:24px;--ifm-footer-background-color:#0000;--ifm-menu-link-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-menu-link-padding-vertical:0.25rem;--ifm-menu-color:rgb(var(--lsd-text-primary),0.6);--ifm-menu-color-background-active:#0000;--ifm-menu-link-sublist-icon:url('data:image/svg+xml;utf8,');--ifm-menu-link-sublist-icon-filter:none;--ifm-navbar-height:64px;--ifm-navbar-item-padding-horizontal:0.75rem;--ifm-navbar-item-padding-vertical:0.25rem;--ifm-navbar-padding-horizontal:var(--content-padding);--ifm-navbar-padding-vertical:calc(var(--ifm-spacing-vertical)/3);--ifm-navbar-shadow:var(--ifm-global-shadow-lw);--ifm-navbar-search-input-icon:url('data:image/svg+xml;utf8,');--ifm-navbar-sidebar-width:100vw;--ifm-pagination-border-radius:var(--ifm-global-radius);--ifm-pagination-font-size:1rem;--ifm-pagination-item-active-background:var(--ifm-hover-overlay);--ifm-pagination-page-spacing:0.2em;--ifm-pagination-padding-horizontal:calc(var(--ifm-spacing-horizontal)*1);--ifm-pagination-padding-vertical:calc(var(--ifm-spacing-vertical)*0.25);--ifm-pagination-nav-border-radius:var(--ifm-global-radius);--ifm-pills-spacing:0.125rem;--ifm-tabs-padding-horizontal:1rem;--ifm-tabs-padding-vertical:1rem;--ifm-alert-background-color:#0000;--ifm-color-secondary-contrast-background:#0000;--ifm-color-success-contrast-background:#0000;--ifm-color-info-contrast-background:#0000;--ifm-color-warning-contrast-background:#0000;--ifm-color-danger-contrast-background:#0000;--ifm-alert-border-radius:none;--note:#d4d5d8;--tip:#6ace4b;--caution:#ddaa39;--danger:#e46967;--info:#68b1d0;--ifm-color-secondary-dark:var(--note);--ifm-color-secondary-light:var(--note);--ifm-color-success-dark:var(--tip);--ifm-color-success-light:var(--tip);--ifm-color-info-dark:var(--info);--ifm-color-info-light:var(--info);--ifm-color-warning-dark:var(--caution);--ifm-color-warning-light:var(--caution);--ifm-color-danger-dark:var(--danger);--ifm-color-danger-light:var(--danger);--ifm-footer-link-color:rgb(var(--lsd-text-primary));--ifm-color-content-secondary:rgb(var(--lsd-text-primary));--ifm-hero-background-color:#f8f8fa;--ifm-hero-text-color:var(--ifm-color-black);--desktop-footer-gap:200px;--mobile-footer-gap:144px;--doc-grid-gap:16px;--ifm-table-background:#0000!important;--ifm-table-stripe-background:#0000!important;--ifm-pre-border-radius:0!important;--ifm-code-font-size:100%!important;background-color:#0000!important;--doc-sidebar-width:16.66vw!important;--doc-sidebar-max-width:320px;--ifm-navbar-item-padding-vertical:8px;--ifm-navbar-item-padding-horizontal:12px;--docusaurus-announcement-bar-height:auto;--doc-sidebar-width:300px;--doc-sidebar-hidden-width:30px;--mobile-width:100%;--mobile-height:176px;--docusaurus-tag-list-border:var(--ifm-color-emphasis-300)}#nprogress .bar{background:var(--docusaurus-progress-bar-color);height:2px;left:0;position:fixed;top:0;width:100%;z-index:1031}#nprogress .peg{box-shadow:0 0 10px var(--docusaurus-progress-bar-color),0 0 5px var(--docusaurus-progress-bar-color);height:100%;opacity:1;position:absolute;right:0;transform:rotate(3deg) translateY(-4px);width:100px}body,html{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;height:100%}main{min-height:calc(100vh - var(--ifm-navbar-height) - var(--content-padding))}svg *{fill:rgb(var(--lsd-text-primary))}.grid{display:grid;gap:16px}.table-of-contents__link--active,.table-of-contents__link:hover{color:rgb(var(--lsd-text-primary))!important}h1{font-size:var(--lsd-h1-fontSize);font-weight:var(--lsd-h1-fontWeight);line-height:var(--lsd-h1-lineHeight)}h1,h2,h3,h4{font-family:var(--lsd-typography-generic-font-family)}h2{font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight)}h3{font-size:var(--lsd-h3-fontSize);font-weight:var(--lsd-h3-fontWeight);line-height:var(--lsd-h3-lineHeight)}h4{font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight)}code{background:#ffffff26;border-radius:0;color:rgb(var(--lsd-text-primary));background:rgba(var(--lsd-theme-primary),.1)}pre code{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-body1-fontSize)!important;font-weight:var(--lsd-body1-fontWeight)!important;line-height:var(--lsd-body1-lineHeight)!important}.theme-doc-footer-edit-meta-row div[class*=lastUpdated_] b,table td strong,table th{font-weight:400!important}.mdx-page body{overflow-x:hidden!important}.mdx-page h2{margin-top:unset!important}.alert,.card,.theme-code-block{box-shadow:none!important}.menu__list-item-collapsible>a,.navbar{box-shadow:none}.sans{font-family:var(--lgs-font-family-secondary);line-height:var(--ifm-line-height-secondary)}.blog-wrapper--index [itemprop=blogPost] .markdown,.external-link,.footer__bottom,.footer__copyright,.menu__list-item .menu__link,.theme-doc-footer-edit-meta-row div[class*=lastUpdated_],table{font-size:var(--lsd-body2-fontSize)!important;font-weight:var(--lsd-body2-fontWeight)!important;line-height:var(--lsd-body2-lineHeight)!important;font-family:var(--lsd-typography-generic-font-family)!important}.markdown a,.markdown a:hover,article a,article a:hover{text-decoration:underline;text-decoration-color:inherit;text-decoration-color:rgba(var(--lsd-text-primary),.6);text-underline-offset:.3rem}.card{padding:var(--ifm-spacing-vertical) var(--ifm-spacing-horizontal);padding:var(--ifm-alert-padding-vertical) var(--ifm-alert-padding-horizontal)}.mdx-cta-section--no-border,.mdx-cta-section--no-border>*,.mdx-feature-list--border-none,.mdx-video-section--no-border,.mdx-video-section--no-border>*,.table-of-contents__left-border{border:none!important}.menu__link{align-items:center;display:flex;justify-content:space-between}.menu__link span{color:rgba(var(--lsd-text-primary),.6)}.theme-edit-this-page{align-items:center;display:flex;gap:12px}.menuExternalLink_NmtK,.navbar__inner,.navbar__item{align-items:center}.dropdown__link:hover{color:rgb(var(--lsd-text-secondary))}.blog-wrapper>.container>.row>aside>nav>ul li a:hover,.breadcrumbs__item--active,.footerLogoLink_BH7S:hover,.hash-link:focus,.theme-code-block:hover .buttonGroup_Qu4e button,:hover>.hash-link{opacity:1}.breadcrumbs__item:not(:last-child):after{background:none;content:"/";opacity:1}.breadcrumbs__link{background:#0000!important;padding:0}.navbar{height:var(--ifm-navbar-height)}.navbar__logo,a[class^=sidebarLogo_] img{height:40px}a[class^=sidebarLogo_]{margin-left:-16px!important;margin:0}.header-github-link:hover,.mdx-event-card__label,.mdx-event-info__label{opacity:.6}.header-github-link:before{background:url("data:image/svg+xml;charset=utf-8,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E") no-repeat;content:"";display:flex;height:16px;width:16px}html[data-theme=dark] .header-github-link:before{background:url("data:image/svg+xml;charset=utf-8,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath fill='%23fff' d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E") no-repeat}.dropdown__link--active{color:rgb(var(--lsd-text-secondary))!important;background-color:#0000}.row{margin:0}.theme-doc-version-banner *,.theme-doc-version-banner:hover *{color:#fff;text-decoration-color:#fff!important}.blog-wrapper main>section article ul li .lsd-tag>span,.blog-wrapper main>section article ul li .lsd-tag>span>span,.menu__link--active span,.theme-doc-version-badge{color:rgb(var(--lsd-text-primary))}.navbar__left-items>a:not(:last-child){margin-right:1.25rem}.navbar__left-items>a:not(:last-child)[href^="/"]{margin-right:1.75rem}.menu__link--sublist-caret:after{background:none;content:url(data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTYiIGhlaWdodD0iMTYiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHBhdGggZD0ibTQgOS41My45NC45NEw4IDcuNDE3bDMuMDYgMy4wNTMuOTQtLjk0LTQtNC00IDRaIiBmaWxsPSIjZmZmIi8+PC9zdmc+);filter:unset;height:unset;min-width:unset;width:unset}.menu__list-item--collapsed>.menu__link--sublist-caret:after{transform:rotate(0)!important}.menu__link--active{border-radius:0}.dropdown__menu,.pagination-nav__link{border:1px solid rgb(var(--lsd-border-primary))}.table-of-contents a{padding-left:16px}.table-of-contents__link--active{border-left:1px solid rgb(var(--lsd-border-primary));padding-left:16px}.navbar__item{display:inline-flex;padding:0}.mdx-accordion-item,.navbar__link--active{border-bottom:1px solid rgb(var(--lsd-border-primary))}.dropdown__menu{background:rgb(var(--lsd-surface-primary));border-radius:0;margin-right:-1px;margin-top:6px;padding:0;transition-duration:0ms}.dropdown__link{background-color:#0000;border-radius:0;margin:0;padding:5px 11px;text-decoration:none}.dropdown__link *{text-decoration:inherit!important}.dropdown__link:hover{background-color:#0000}.blog-wrapper .breadcrumbs__item,.blog-wrapper--index h2[itemprop=headline]>a,.events .breadcrumbs__item,.events .breadcrumbs__link{text-decoration:none}.margin-left-8,.mdx-jpd__external-link-icon{margin-left:8px}.pagination-nav{margin-top:var(--pagination-nav-margin-top)!important}.overflow-hidden,.pagination-nav a{overflow:hidden}.theme-admonition,table{overflow:auto}.pagination-nav span{-webkit-line-clamp:1;-webkit-box-orient:vertical;display:-webkit-box;overflow:hidden;text-overflow:ellipsis;width:70%;word-break:break-word}.pagination-nav__link{align-items:center;border-radius:0;display:flex;gap:18px;padding:10px 14px}.theme-doc-version-badge,table td,table th,table thead>tr{border:1px solid rgb(var(--lsd-border-primary))}.pagination-nav__link--next{grid-column:2/3;text-align:right;justify-content:flex-end}.footer__item a svg,.menu__caret,.pagination-nav__label:after,.pagination-nav__label:before,.theme-doc-footer-edit-meta-row div[class*=lastUpdated_] small{display:none}.no-breadcrumbs .theme-doc-breadcrumbs,.theme-back-to-top-button{display:none!important}.authors_dZ4g,.container_PkUo>div,.footer__link-item,.tag_NdBD{display:inline-block}.lsd-collapse__content{padding:24px}.theme-doc-version-badge{background:#0000;margin-bottom:16px;padding:4px 8px}.navbar__right-items{align-items:center;display:flex}table{border-collapse:collapse;margin-bottom:var(--ifm-spacing-vertical);display:table;table-layout:fixed;width:100%}table th{text-align:left}table td{color:var(--ifm-table-cell-color)}.theme-doc-footer{margin-bottom:96px}.theme-doc-footer-edit-meta-row div[class*=lastUpdated_]{font-style:normal;opacity:.7}.navbar-sidebar__item{padding-block:0;padding-inline:var(--content-padding)}.navbar-sidebar__back{align-items:center;display:flex;gap:16px;margin:24px 0!important;padding:0;width:100%!important}.theme-doc-sidebar-menu .menu__link{padding-left:var(--content-padding);padding-right:0}.menu__list-item-collapsible>a[aria-expanded=true]>svg{transform:rotate(180deg)}.events .pagination-nav,.events aside,.menu__list-item-collapsible>a>svg,.news .theme-doc-sidebar-container,.news header h1{display:none}main[itemtype*=Blog]{margin-left:0}.mdx-ghc-subheader-text,main[itemtype*=Blog] .theme-doc-breadcrumbs{margin-bottom:40px}main[itemtype*=Blog] .blog-divider{background:rgb(var(--lsd-theme-primary));margin-block:40px}.events .theme-doc-markdown>h1{margin-bottom:16px!important}.events .theme-doc-breadcrumbs{display:block}.full-width>div .main-wrapper main{grid-column:1/17!important}.news .news-title{display:flex;font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h1-fontSize);font-weight:var(--lsd-h1-fontWeight);gap:16px;line-height:var(--lsd-h1-lineHeight);margin-bottom:0!important}.news .news-title span{opacity:.4}.footer__links>div:first-of-type,.footer__links>div:nth-of-type(3),.mdx-event-profile__title{margin-bottom:8px}.footer__links>div:nth-of-type(2){margin-bottom:34px}.container{padding-inline:var(--content-padding)}.main-wrapper>main,.main-wrapper>main>.row,.main-wrapper>main>.row>.col{--ifm-spacing-horizontal:var(--content-padding);flex-basis:100%!important;flex:unset;margin:0;max-width:unset;width:100%!important}.footer{margin-bottom:18px;width:100%!important}.footer>div{display:grid;gap:1rem;max-width:unset;padding:24px 0 0}.footer__bottom{grid-column:1/12;grid-row:1/1;text-align:unset}.content_knG7,.mdx-cta-section .mdx-cta-section--align-center *,.root_EFVO{text-align:center}#__docusaurus-base-url-issue-banner-container,.blog-archive-page .main-wrapper header .container p,.blog-wrapper main>section article ul li a:after,.blog-wrapper main>section article ul li a:before,.docSidebarContainer_b6E3,.footer__bottom>div:first-of-type,.groups_p1lF .divider_eGUz:last-child,.mdx-accordion-item>input,.modal_kLVz .closeButton_Rr0e,.sidebarLogo_Ydl9,.themedImage_kfRS,.wrapper_SWrM .fullscreenButton_Bocn,[data-hidden-doc-sidebar=true] .theme-doc-sidebar-container,[data-theme=dark] .lightToggleIcon_K4TL,[data-theme=light] .darkToggleIcon_lKkA,div[class^=blogHeader] span,html[data-announcement-bar-initially-dismissed=true] .announcementBar_mb4j{display:none}.footer__title{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-label1-fontSize)!important;font-weight:var(--lsd-label1-fontWeight)!important;font-weight:var(--ifm-font-weight-normal);line-height:var(--lsd-label1-lineHeight)!important;margin-bottom:4px}.blog-wrapper main>section article ul li .lsd-tag>span,.blog-wrapper main>section article ul li .lsd-tag>span>span,.footer__item,.footer__item a{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-label1-fontSize)!important;font-weight:var(--lsd-label1-fontWeight)!important;line-height:var(--lsd-label1-lineHeight)!important}.footer>div:first-child>.footer__links,.footer>div:nth-child(2)>.footer__links{display:grid;grid-column:13/25;grid-row:1/1;margin-bottom:0}.footer>div:nth-child(2)>.footer__links{gap:2rem 1rem;grid-template-columns:repeat(2,1fr)}.footer__col{grid-template-columns:span 1}.blog-wrapper main>section article ul li a,.codeBlockStandalone_BRZX,.codeBlockStandalone_MEMb,.col.footer__col,.modal_kLVz>div>div:first-child,.noResults_mD2O,.root_N57j ul,.root_ONDG.empty_a7qb{padding:0}.footer__items{display:flex;flex-direction:row;flex-wrap:wrap;gap:.5rem}.footer__item{display:inline-flex}.footer__item:not(:last-child):after{content:"•";display:inline-block;margin-left:8px;text-decoration:none}.footer__logo{max-width:var(--ifm-footer-logo-max-width);margin:0!important}.navbar__inner{flex-direction:row}.hero--primary{--ifm-hero-background-color:rgb(var(--lsd-surface-primary));--ifm-hero-text-color:rgb(var(--lsd-text-primary))}.docusaurus-mermaid-container{margin-block:40px}.hidden-scrollbar{-ms-overflow-style:none;scrollbar-width:none}.hidden-scrollbar::-webkit-scrollbar{display:none;width:0}[class^=mdx-]{--lsd-h1-fontWeight:100;--lsd-h2-fontWeight:300;--lsd-h3-fontWeight:300;--lsd-h4-fontWeight:300;--lsd-h5-fontWeight:300;--lsd-h6-fontWeight:300;--lsd-subtitle1-fontWeight:300}.hidden{opacity:0;visibility:hidden}.blog-wrapper h1[itemprop=headline]{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h1-fontSize)!important;font-weight:var(--lsd-h1-fontWeight)!important;line-height:var(--lsd-h1-lineHeight)!important}.blog-wrapper>.container{display:flex;margin:0;max-width:unset;padding-inline:0;width:100%}.mdx-cta-section .mdx-cta-section--align-right .mdx-cta-button,.mdx-logo-carousel .mdx-section-header__extra,.mdx-video-section .mdx-video-section--align-right .mdx-cta-button{margin-left:auto}.blog-wrapper>.container>.row,.mdx-roadmap,.mdx-scroll-buttons{width:100%}.blog-wrapper main>header h1{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important;margin-bottom:.5rem}.blog-wrapper>.container>.row>aside{max-width:unset}.blog-wrapper>.container>.row>aside>nav{padding-top:16px;top:var(--ifm-navbar-height)}.blog-wrapper>.container>.row>aside>nav>div:first-child{font-family:var(--lsd-typography-generic-font-family)!important;margin-bottom:0!important;padding:4px 0 4px 15px}.blog-wrapper>.container>.row>aside>nav>ul li a{box-sizing:border-box;display:block;font-family:var(--lsd-typography-generic-font-family)!important;opacity:.6;padding:4px 0 4px 2rem}.blog-wrapper--index>.container>.row>aside>nav>div:first-child,.blog-wrapper>.container>.row>aside>nav>ul li a[class*=sidebarItemLinkActive_]{border-color:rgb(var(--lsd-border-primary))}.blog-wrapper main article,.mdx-asset-card:not(.mdx-asset-card--downloadable) .mdx-asset-card__inner,.mdx-erc--with-preview .mdx-erc__preview-image{border-bottom:1px solid rgb(var(--lsd-border-primary))}.blog-wrapper--index h2[itemprop=headline]{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.content_knG7 a,.footerLink_sh7M,.linkContent__x3v:hover,.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>a:hover,.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>a:hover{text-decoration:underline}.blog-wrapper>.container>.row>main{margin-top:16px;max-width:unset}.blog-archive-page .main-wrapper header .container h1,.blog-wrapper article header h2,.blog-wrapper main>section article h2{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.blog-wrapper article footer{align-items:center;display:flex;flex-direction:row;justify-content:space-between;margin-top:24px!important}.blog-wrapper main article{margin-bottom:34px!important;padding-bottom:34px}.blog-wrapper main article:last-of-type{border-bottom:none!important}.blog-archive-page .main-wrapper{padding:2rem var(--content-padding)}.blog-archive-page .main-wrapper header{border-bottom:1px solid var(--ifm-hero-text-color);margin:0;padding:0 0 1rem}.blog-archive-page .main-wrapper header .container{margin:0;padding:0!important}.blog-archive-page .main-wrapper main>section .container{margin:0;max-width:unset;padding:0}.blog-archive-page .main-wrapper main>section .container>.row,.mdx-logo-carousel-buttons,.mdx-roadmap__timeline{margin-top:64px}.blog-archive-page .main-wrapper main>section .container>.row>.col{margin:0!important;max-width:unset;padding:0}.blog-archive-page .main-wrapper main>section .container>.row>.col h3{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h5-fontSize)!important;font-weight:var(--lsd-h5-fontWeight)!important;line-height:var(--lsd-h5-lineHeight)!important}.blog-archive-page .main-wrapper main>section .container>.row>.col a{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-body1-fontSize)!important;font-weight:var(--lsd-body1-fontWeight)!important;line-height:var(--lsd-body1-lineHeight)!important}.blog-archive-page .main-wrapper main>section .container>.row>.col ul{margin:1rem 0 0;padding:0 0 0 1.5rem}.blog-wrapper main>section article{border-bottom:none;padding:0}.blog-wrapper main>section article ul,.mdx-app-card__name{margin-top:1rem}.nav_cMpg,.root_QACb{margin:0 auto;max-width:var(--container-max-width)}.blog-wrapper main>section article ul li{margin:0 .5rem 0 0!important;padding:0!important}.blog-wrapper main>section article ul li .lsd-tag{height:28px;padding:3px 11px}.errorBoundaryError_a6uf{color:red;white-space:pre-wrap}body:not(.navigation-with-keyboard) :not(input):focus{outline:0}[data-theme=dark] .themedImage--dark_OvIx,[data-theme=light] .themedImage--light_BL8e,html:not([data-theme]) .themedImage--light_BL8e{display:initial}.icon_S7Kx.s_AZDZ svg{width:10px}.icon_S7Kx.m_thRi svg{width:20px}.icon_S7Kx.l_WHPt svg{width:40px}[data-theme=dark] .icon_S7Kx.fill_hNhN,[data-theme=dark] .icon_S7Kx.fill_hNhN *{fill:var(--ifm-color-white)}[data-theme=dark] .icon_S7Kx.stroke_N8dm svg,[data-theme=dark] .icon_S7Kx.stroke_N8dm svg *{stroke:var(--ifm-color-white)}[data-theme=light] .icon_S7Kx.fill_hNhN,[data-theme=light] .icon_S7Kx.fill_hNhN *{fill:var(--ifm-color-black)}[data-theme=light] .icon_S7Kx.stroke_N8dm svg,[data-theme=light] .icon_S7Kx.stroke_N8dm svg *{stroke:var(--ifm-color-black)}.backdrop_Ifvc{background:rgb(var(--lsd-surface-primary));z-index:201}.backdrop_Ifvc,.navWrapper_ybYI{height:100%;left:0;opacity:0;position:fixed;top:0;visibility:hidden;width:100%}.navWrapper_ybYI{background:#0000;z-index:203}.navWrapper_ybYI>*{pointer-events:auto}.nav_cMpg{align-items:center;background:rgb(var(--lsd-surface-primary));display:flex;flex-direction:row;height:var(--ifm-navbar-height);justify-content:space-between;padding:var(--ifm-navbar-padding-vertical) var(--ifm-navbar-padding-horizontal)}.visible_ynAX{opacity:1!important;visibility:visible!important}.codeBlockContent_ugSV,.wrapper_SWrM{position:relative}.wrapper_SWrM:not(.active_qZD5) .fullscreenButton_Bocn{background:rgb(var(--lsd-surface-primary));bottom:8px;position:absolute;right:8px;transition:.3s}.root_QACb{padding:0;width:100%}.skipToContent_fXgn{background-color:var(--ifm-background-surface-color);color:var(--ifm-color-emphasis-900);left:100%;padding:calc(var(--ifm-global-spacing)/2) var(--ifm-global-spacing);position:fixed;top:1rem;z-index:calc(var(--ifm-z-index-fixed) + 1)}.skipToContent_fXgn:focus{box-shadow:var(--ifm-global-shadow-md);left:1rem}.closeButton_CVFx{line-height:0;padding:0}.content_knG7{font-size:85%;padding:5px 0}.content_knG7 a{color:inherit}.announcementBar_mb4j{align-items:center;background-color:var(--ifm-color-white);border-bottom:1px solid var(--ifm-color-emphasis-100);color:var(--ifm-color-black);display:flex;height:var(--docusaurus-announcement-bar-height)}.announcementBarPlaceholder_vyr4{flex:0 0 10px}.announcementBarClose_gvF7{align-self:stretch;flex:0 0 30px}.iconExternalLink_nPIU{margin-left:.3rem}.footerLogoLink_BH7S{opacity:.5;transition:opacity var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.firstRow_ar1q{border-top:1px solid rgb(var(--lsd-text-primary));margin-bottom:96px!important}.mdx-cta-section,.mdx-cta-section--list .mdx-cta-section__list>:not(:first-child),.mdx-video-section,.secondRow__ww3{border-top:1px solid rgb(var(--lsd-border-primary))}.firstRow_ar1q,.secondRow__ww3{display:flex;flex-direction:row}.secondRow__ww3{padding-bottom:16px!important}.backToTop_wDfN{bottom:16px;position:absolute}.l-modal,.l-modal__backdrop{left:0;position:fixed;top:0}.backToTop_wDfN,.footerLink_sh7M{width:-moz-fit-content;width:fit-content}.footerLink_sh7M{height:-moz-fit-content;height:fit-content}.toggle_K23S{border:1px solid rgb(var(--lsd-border-primary));height:32px;width:32px}.toggle_K23S svg{height:14px;width:14px}.toggleButton_dl49{align-items:center;border-radius:50%;display:flex;height:100%;justify-content:center;transition:background var(--ifm-transition-fast);width:100%}.dropdownNavbarItem_o23I,.l-modal__content{border:1px solid rgb(var(--lsd-border-primary))}.toggleButton_dl49:hover{background:var(--ifm-color-emphasis-200)}.toggleButtonDisabled_AAS_{cursor:not-allowed}.darkNavbarColorModeToggle_X3D1:hover{background:var(--ifm-color-gray-800)}.l-modal{height:100vh;opacity:0;transition:.3s;visibility:hidden;width:100%;z-index:9999}.l-modal__container{display:grid;gap:1rem;grid-template-columns:repeat(24,1fr);margin:0 auto;max-width:1376px;overflow:auto}.l-modal--open{opacity:1;visibility:visible}.backToTopButton_sjWU,.mdx-hero-video .mdx-hero-video__placeholder{visibility:hidden;opacity:0}.l-modal__content{background:rgb(var(--lsd-surface-primary));grid-column:8/19;height:-moz-fit-content;height:fit-content;margin:64px 0;max-height:80vh;padding:7px;z-index:9998}.groups_p1lF,.root_ONDG{max-height:60vh;overflow-y:auto}.l-modal__backdrop{background-color:rgba(var(--lsd-surface-primary),.6);height:100%;width:100%;z-index:9997}.modal_kLVz .header_QwCa{align-items:center;display:flex;flex-direction:row;gap:.5rem}.root_EFVO{padding:32px}.root_ONDG{padding:24px}.title_uwS_{font-size:.75rem!important}.item_RHYF{align-items:center;display:flex;flex-direction:row;margin-top:1rem}.item_RHYF .itemTitle_jtAv{flex-grow:1;overflow:hidden;padding-left:1rem;text-overflow:ellipsis;white-space:nowrap}.linkContent__x3v,.root_JWD1{align-items:center;display:flex}.item_RHYF .itemTitle_jtAv mark,.root_JWD1 mark{background:none;color:inherit}.root_xZfc{padding:8px 8px 0;position:relative;width:100%}.collapse_LWBd button,.textField_af43{width:100%!important}.textField_af43 button svg{height:auto;max-height:100%!important;width:14px}.root_N57j>div:first-child{color:#828285;padding:0 24px}.root_N57j>div:first-child,.root_N57j>div:first-child *{font-size:.75rem}.root_N57j ul li{margin-top:1rem}.root_JWD1{padding:0 24px}.root_JWD1>div{margin-left:1rem}.root_Ooa0{text-decoration:none!important}.root_Ooa0.level1_fpUf{display:block;padding-left:32px}.icon_sZn2{height:auto;width:16px}.icon_sZn2.fill_fkan,.icon_sZn2.fill_fkan *{fill:currentColor}.icon_sZn2.stroke_j3wH,.icon_sZn2.stroke_j3wH *{stroke:currentColor}[data-theme=light] .icon_sZn2{color:#000}[data-theme=dark] .icon_sZn2{color:#fff}.root_ltHz{padding:24px 0}.groups_p1lF{display:flex;flex-direction:column;gap:16px}[data-theme=dark] .divider_eGUz>div{opacity:.1}[data-theme=dark] .topBar_Dtew button{background-color:var(--ifm-button-color)}.linkContent__x3v{gap:6px;text-decoration:none}.linkContent__x3v:hover{text-decoration-color:rgba(var(--lsd-text-primary),.6);text-underline-offset:.3rem}.mdx-erc:hover .mdx-erc__title,.mdx-ghc__issue-title-link:hover,.mdx-ghc__view-on-github-link:hover{text-decoration:underline!important}.externalLinkIcon_Qfue{width:8px}.externalLinkIcon_Qfue svg,.root_TTsz.fullWidth_HBtH{height:auto;width:100%}.dropdownNavbarItem_o23I{height:28px;padding:6px 12px}.navbarHideable_OoEf{transition:transform var(--ifm-transition-fast) ease}.navbarHidden_zoxl{transform:translate3d(0,calc(-100% - 2px),0)}.rightSection_CLeF{align-items:center;display:flex;justify-content:flex-end;white-space:nowrap}.input-cta-section__description,.mdx-event-card__info *,.mdx-event-info__content,.mdx-news-card__info *{white-space:pre-wrap}.iconButtonGroup_ktNv button:not(:last-child){border-right:1px solid!important;margin-right:-1px!important}.mainWrapper_z2l0{display:flex;flex:1 0 auto;flex-direction:column}.docusaurus-mt-lg{margin-top:3rem}#__docusaurus{display:flex;flex-direction:column;min-height:100%}.backToTopButton_sjWU{background-color:var(--ifm-color-emphasis-200);border-radius:50%;bottom:1.3rem;box-shadow:var(--ifm-global-shadow-lw);height:3rem;position:fixed;right:1.3rem;transform:scale(0);transition:all var(--ifm-transition-fast) var(--ifm-transition-timing-default);width:3rem;z-index:calc(var(--ifm-z-index-fixed) - 1)}.chevron_How5,.root_TTsz .content_feMR{position:absolute;left:0}.backToTopButton_sjWU:after{background-color:var(--ifm-color-emphasis-1000);content:" ";display:inline-block;height:100%;-webkit-mask:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem no-repeat;mask:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem no-repeat;width:100%}.backToTopButtonShow_xfvO{opacity:1;transform:scale(1);visibility:visible}.docMainContainer_gTbr,.docPage__5DB{display:flex;width:100%}.docPage__5DB{flex:1 0}.docsWrapper_BCFX{display:flex;flex:1 0 auto}.root_jG9K{align-items:center;display:flex;flex-direction:column;padding:1rem!important;width:100%}.root_jG9K>div{max-width:796px!important;width:100%}.root_jG9K>div>div{align-items:flex-start;display:flex;flex-direction:column;gap:1rem;padding:0!important}.root_jG9K a{margin-top:2.5rem}.mdx-accordion-item__header{align-items:center;display:flex;flex-direction:row;gap:0 16px;justify-content:space-between;min-height:88px;padding:16px 0}.mdx-asset-card,.mdx-erc{min-height:144px;text-decoration:none!important}.mdx-accordion-item__header,.mdx-accordion-item__header *,.mdx-cta-section__list-item-link,.mdx-video-section__list-item-link{cursor:pointer}.mdx-accordion-item__content-wrapper{height:auto;overflow:hidden}.mdx-accordion-item__content{height:0}.mdx-accordion-item--open .mdx-accordion-item__content{height:auto;padding-bottom:1.875rem;padding-top:6px}.mdx-app-card{align-items:flex-start;display:flex;flex-direction:column;padding:1.5rem 0}.mdx-app-card__logo{height:40px;width:auto}.mdx-app-card__description{margin-top:1.5rem}.mdx-app-card__link{margin-top:1.5rem;width:100%}.mdx-app-card__link button{padding:10px 16px;width:100%}.mdx-app-card__link button>span{align-items:center;display:block;display:flex;flex-direction:row;justify-content:space-between;width:100%}.root_TTsz{display:inline-block;font-size:0!important;position:relative}.root_TTsz.fullHeight_ls1u{height:100%;width:auto}.root_TTsz .content_feMR{height:100%;top:0;width:100%}.root_TTsz:not(.keep_Y9Ea) .root_TTsz{height:auto;width:auto}.root_TTsz:not(.keep_Y9Ea) .content_feMR{height:100%;left:unset;position:relative;top:unset;width:100%}.mdx-asset-card{align-items:flex-start;border:1px solid rgb(var(--lsd-border-primary));border-bottom:none;display:flex;flex-direction:column;width:216px}.mdx-asset-card__inner{padding:16px 16px 32px;width:100%}.mdx-asset-card__title,.mdx-event-profile-list__container h2,.mdx-event-profile__role,.mdx-ghc__challenge-labels,.mdx-ghc__issue-title{margin-bottom:16px}.mdx-asset-card__image img{height:100%;object-fit:contain;object-position:center center;width:100%}.mdx-asset-card__downloadables{align-items:flex-start;display:flex;flex-direction:row;justify-content:stretch;width:100%}.mdx-asset-card__downloadables>*{flex-grow:1;font-size:12px;text-decoration:none!important;text-underline-offset:unset!important}.mdx-asset-card__downloadables>* button{justify-content:space-between;text-decoration:none;width:100%}.mdx-asset-card__downloadables>:first-child button{border-left:none}.mdx-asset-card__downloadables>:last-child button,.mdx-asset-card__downloadables>:not(:last-child) button{border-right:none}.mdx-box{padding-bottom:var(--mdx-box-bottom);padding-top:var(--mdx-box-top)}.mdx-cta-button,.mdx-section-header--with-description .mdx-section-header__extra{margin-top:2rem}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>.mdx-cta-section__item-description,.mdx-cta-section__description{margin-top:1rem}.mdx-cta-button button,.mdx-cta-button button span{font-size:inherit;font-weight:inherit;line-height:inherit}.mdx-cta-button .lsd-button--large{padding:9px 39px}.mdx-cta-button .lsd-button--small{padding:5px 11px}.mdx-cta-section,.mdx-video-section{display:grid;padding:24px 0 0}.mdx-cta-section .mdx-cta-section__link,.mdx-jpd__job-link,.mdx-video-section .mdx-video-section__link{display:block;width:-moz-fit-content;width:fit-content}.mdx-cta-section .mdx-cta-section__header__desktop-button .mdx-cta-button{margin-top:unset}.mdx-cta-section .mdx-cta-section--align-center{align-items:center;display:flex;flex-direction:column;margin:auto;max-width:936px}.mdx-cta-section .mdx-cta-section--align-center .mdx-cta-section__description{margin-top:1.5rem}.mdx-cta-section .mdx-cta-section--align-left *{text-align:left}.mdx-cta-section .mdx-cta-section--align-right *{text-align:right}.mdx-cta-section__header,.mdx-profile-card__profile{align-items:center;display:flex;gap:24px}.mdx-cta-section__header button{font-size:12px!important;line-height:16px!important;padding:6px 12px!important}.mdx-cta-section--title-only .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);font-weight:300;line-height:var(--lsd-h5-lineHeight);max-width:886px}.mdx-cta-section--title-button .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-display4-fontSize);font-weight:var(--lsd-display4-fontWeight);font-weight:300;line-height:var(--lsd-display4-lineHeight);max-width:886px}.mdx-cta-section--full-width .mdx-cta-section__container{display:grid;gap:0 1rem;grid-template-columns:repeat(2,1fr)}.mdx-cta-section--full-width .mdx-cta-section__title,.mdx-ns__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h3-fontSize);font-weight:var(--lsd-h3-fontWeight);line-height:var(--lsd-h3-lineHeight)}.mdx-cta-section--full-width .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-cta-section--simple .mdx-cta-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-cta-section--simple .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-cta-section--list{display:grid;gap:0 1rem;grid-template-columns:1fr 1fr}.mdx-cta-section--list .mdx-cta-section__container{grid-column:1/2}.mdx-cta-section--list .mdx-cta-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-cta-section--list .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.container_PkUo,.mdx-hero h1,.mdx-roadmap .mdx-cta-section__container h3,.mdx-roadmap .mdx-cta-section__description,.mdx-roadmap .mdx-cta-section__title{font-family:var(--lsd-typography-generic-font-family)!important}.mdx-cta-section--list .mdx-cta-section__list{display:flex;flex-direction:column;gap:1.5rem 0;grid-column:2/3}.mdx-erc--with-preview .mdx-erc__icon,.mdx-ns__inner .mdx-ns__toast .lsd-toast__close-button{display:none}.mdx-cta-section--list .mdx-cta-section__list>a{display:block;text-decoration:none}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>a>.mdx-cta-section__item-title{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:1rem;color:rgb(var(--lsd-text-primary));display:flex;gap:12px;margin-top:1.5rem;padding:3px 11px;width:-moz-fit-content;width:fit-content}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>a{text-decoration:none}.mdx-doc-metadata{display:flex;flex-direction:row;line-height:0;margin-bottom:2.5rem}.mdx-doc-metadata>span:not(:last-child):after{content:"•";display:inline-block;margin-inline:.75rem}.markdown h1+.mdx-doc-metadata,.markdown header+.mdx-doc-metadata{margin-top:calc(var(--ifm-h1-vertical-rhythm-bottom)*-1*var(--ifm-leading) + 1rem)}.mdx-erc{align-items:flex-start;border:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:row}.mdx-erc__icon{padding:16px 16px 16px 8px}.mdx-erc__inner{flex-grow:1;padding:16px}.mdx-erc__logo,.mdx-erc__logo svg{height:32px!important;width:32px!important}.mdx-erc__title{display:-webkit-box;margin-top:32px;max-height:calc(var(--lsd-body1-lineHeight)*2);overflow:hidden}.mdx-erc__description{margin-top:8px}.mdx-erc--with-preview{display:flex;flex-direction:column}.mdx-erc--with-preview .mdx-erc__preview-image{height:100%;object-fit:cover;width:100%}.mdx-erc--with-preview .mdx-erc__inner{display:grid;gap:0 16px;grid-template-columns:32px auto;grid-template-rows:auto auto;padding:16px}.mdx-erc--with-preview .mdx-erc__logo{align-self:center;grid-column:1;grid-row:1/span 2}.mdx-erc--with-preview .mdx-erc__title{-webkit-line-clamp:1;-webkit-box-orient:vertical;margin-top:0;max-height:calc(var(--lsd-body1-lineHeight)*1)}.mdx-erc--with-preview .mdx-erc__description{margin-top:4px}.mdx-feature-list{border-top:1px solid rgb(var(--lsd-surface-secondary));position:relative}.mdx-feature-list__feature,.mdx-ghc__container,.mdx-showcase-card{border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-feature-list__title{padding:24px 0}.mdx-feature-list__feature-index--circle{align-items:center;background:rgb(var(--lsd-surface-secondary));border-radius:100%;color:rgb(var(--lsd-text-secondary))!important;display:inline-flex;height:1.875rem;justify-content:center;padding:.5rem;width:1.875rem}.mdx-feature-list__feature-index--number{background:#0000;color:rgb(var(--lsd-text-primary))!important;display:block;font-size:28px!important;height:unset;line-height:36px!important;opacity:.3;padding:unset;width:unset}.mdx-feature-list__list{display:grid;grid-template-columns:repeat(2,1fr)}.mdx-event-profile__info,.mdx-feature-list__feature-inner,.mdx-numbers-view-header-title,.roadmap__header{display:flex;flex-direction:column}.mdx-feature-list__feature-title{margin-top:.5rem}.mdx-feature-list__feature-description{max-width:460px}.mdx-feature-list__header{align-items:center;display:flex;flex-direction:row;gap:2rem}.mdx-feature-list--cta-bottom .mdx-feature-list__extra{margin-top:3.5rem;padding:0 1rem}.mdx-feature-list__feature-border--none{width:0!important}.mdx-ghc__container{padding-top:24px}.mdx-ghc__issue-title-link{text-decoration:none!important}.mdx-ghc__header,.mdx-jpd__header{margin-bottom:40px;margin-top:16px}.mdx-ghc__challenge-label{border:1px solid rgb(var(--lsd-border-primary));border-radius:20px;margin-right:8px;padding:4px 8px}.mdx-ghc__view-on-github-link{display:block;margin-bottom:56px;margin-top:40px;text-decoration:none!important;width:-moz-fit-content;width:fit-content}.mdx-ghc__participant-photo{border:1px solid rgb(var(--lsd-border-secondary));border-radius:100%;margin-left:-4px;width:24px}.mdx-timeline-item,.mdx-timeline-item__border{border-bottom:1px solid rgb(var(--lsd-border-primary))}.mdx-ghc__participant-photo-container{align-items:center;direction:rtl;display:flex;justify-content:flex-end;padding-left:4px}.mdx-ghc__comment-count{margin-left:4px}.mdx-ghc__issue-content-grid{align-items:center;display:grid;grid-template-columns:82px 135px 82px 1fr;margin-top:24px;row-gap:12px}.mdx-scroll-buttons>div{display:flex;flex-direction:row;justify-content:flex-end;width:100%}.mdx-hero{--default-z-index:1;--hero-max-height:1080px;display:flex;flex-direction:column;height:calc(100vh - var(--ifm-navbar-height) - 8px);max-height:var(--hero-max-height);position:relative;z-index:var(--default-z-index)}.mdx-hero h1{font-size:var(--lsd-display4-fontSize)!important;font-weight:var(--lsd-display4-fontWeight)!important;line-height:var(--lsd-display4-lineHeight)!important}.mdx-hero--top{justify-content:flex-start}.mdx-hero--center{justify-content:center}.mdx-hero--bottom{justify-content:flex-end}.mdx-hero-action--large button{padding:10px 64px}.mdx-hero-actions{align-items:center;display:flex;flex-direction:row;gap:0 1rem;margin-top:2rem;position:relative;z-index:98}.mdx-hero-description{font-size:18px!important;line-height:24px!important;margin-top:18px;max-width:528px;position:relative;text-shadow:1px 1px 6px rgba(var(--lsd-surface-primary),1);z-index:-2}.mdx-hero-info{display:flex;flex-direction:column;justify-content:end;max-width:596px;padding:32px 0;transition:margin-bottom .05s ease-in-out}.mdx-hero-model{--mdx-hero-model-wrapper-scale:1;position:absolute;transform:translateX(calc(-50vw - var(--container-max-width)*-1/2 + var(--content-padding)*4*-1/2));width:100vw;z-index:0}.mdx-hero-model .mdx-hero-model--inner{left:0;position:relative;top:0;width:100%;z-index:0}.mdx-hero-model .mdx-hero-model--inner canvas{pointer-events:unset!important}.mdx-hero-model--top{top:300px}.mdx-hero-model--bottom{top:calc(var(--logos-hero-info-height) - 10px)}.mdx-hero-model--bottom .mdx-hero-model--shade{bottom:10vh;height:40vh}.mdx-hero-model--ascii{filter:grayscale(100%);opacity:.65}.mdx-hero-model--ascii table,.mdx-hero-model--ascii table *{border:none!important;color:rgb(var(--lsd-text-primary));font-family:monospace}.mdx-hero-model--shade{background:linear-gradient(0deg,rgb(var(--lsd-surface-primary)) 37%,rgb(var(--lsd-surface-primary)) 0,rgba(var(--lsd-surface-primary),0) 100%);bottom:0;height:30vh;left:0;position:absolute;width:100%}.mdx-hero-title{font-size:48px!important;line-height:56px!important}.mdx-jpd__department-title,.mdx-video-section__header button{font-size:12px!important;line-height:16px!important}.mdx-hero-video{--hero-video-height:120%;--hero-video-min-height:100vh;--hero-video-offset-y:-150px;--hero-video-scale:1.70951586;--hero-video-height-mobile:120%;--hero-video-min-height-mobile:100vh;--hero-video-offset-y-mobile:-50px;--hero-video-scale-mobile:1.70951586;display:grid;flex-grow:1;grid-template-columns:1fr;grid-template-rows:1fr;pointer-events:none;position:relative}.mdx-hero-video>*{align-items:center;display:flex;flex-direction:column;grid-column:1/1;grid-row:1/1;height:var(--hero-video-height);justify-content:center;left:0;min-height:var(--hero-video-min-height);position:absolute;top:0;transform:translateY(var(--hero-video-offset-y));width:100%}.mdx-hero-video img{object-fit:contain}.mdx-hero-video img,.mdx-hero-video video{height:100%;max-width:calc(var(--container-max-width)*var(--hero-video-scale));object-position:top;transform-origin:top center;width:auto}.copyButtonCopied_YovB .copyButtonIcon_ZL7v,.mdx-hero-video--loading .mdx-hero-video__video{opacity:0}.mdx-hero-video--loading .mdx-hero-video__placeholder{opacity:1;visibility:visible}.mdx-hero-asset{--default-z-index:-1;position:relative;z-index:var(--default-z-index)}.mdx-hero-asset button{display:none!important}.mdx-hero-asset img{position:absolute;right:0;top:0}.mdx-jpd__single-job-department-container{border-top:1px solid rgb(var(--lsd-border-primary));padding-top:8px}.mdx-jpd__job-title-container{align-items:center;display:flex;padding-bottom:8px}.mdx-jpd__job-list{list-style-type:none;margin:0;padding:8px 0 24px}.mdx-jpd__job-list-item{padding:14px 0}.mdx-jpd__job-link,.mdx-jpd__job-link:hover{text-decoration:none!important}.mdx-event-card__title:hover,.mdx-jpd__job-link:hover .mdx-jpd__job-title,.mdx-news-card:hover .mdx-news-card__title,.mdx-social-card:hover{text-decoration:underline!important}.mdx-logo-carousel .mdx-section-header__title{flex-basis:100%}.mdx-logo-carousel .mdx-logo-carousel__inner{margin-top:32px}.mdx-logo-carousel .mdx-logo-carousel__item{flex-shrink:0}.mdx-logo-carousel .mdx-logo-carousel__logo{height:56px;width:auto}.mdx-ns__inner form{margin-top:40px;width:100%}.mdx-ns__inner .mdx-ns__inputs{align-items:center;display:flex;flex-direction:row;gap:16px;width:100%}.mdx-ns__inner .mdx-ns__inputs>*{flex-grow:1}.mdx-event-card-list__container,.mdx-ns__inner .mdx-ns__submit-button{margin-top:40px}.mdx-ns__inner .mdx-ns__toast{margin-bottom:24px;margin-top:-16px;width:100%}.mdx-profile-card{align-items:flex-start;border-top:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;justify-content:space-between;padding:1.5rem 1rem 0 0}.mdx-profile-card__profile-placeholder{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:50%;display:flex;height:56px!important;justify-content:center;width:56px!important}.mdx-profile-card__profile-placeholder svg,.mdx-social-card__logo,.mdx-social-card__logo svg{height:40px!important;width:40px!important}.mdx-profile-card__profile-placeholder svg rect{fill:unset!important}.mdx-profile-card__profile-image{border-radius:50%;height:56px!important;width:56px!important}.mdx-profile-card__buttons{align-items:center;display:flex;flex-direction:row;gap:8px;height:16px;margin-top:8px}.mdx-profile-card__link{height:-moz-fit-content;height:fit-content;position:relative;text-decoration:none}.mdx-profile-card__link svg{height:16px;width:16px}.mdx-timeline-item{display:flex;flex:0 0 auto;flex-direction:column;gap:1rem;justify-content:space-between;padding-bottom:23px;scroll-snap-align:start!important;width:193px}.mdx-timeline-item__header{align-items:flex-start;display:flex;flex-direction:column;gap:1rem}.mdx-timeline-item__period-container{position:relative;width:100%}.mdx-timeline-item__border{height:50%;left:0;position:absolute;top:0;width:calc(100% + 1rem);z-index:-1}.mdx-timeline-item--border-dashed .mdx-timeline-item__border{border-bottom-style:dashed}.mdx-timeline-item__period{border-radius:10rem;display:inline-block;padding:3px 12px}.docItemContainer_hrrU .tocMobile_imaF:not(:first-of-type),.mdx-roadmap__timeline .mdx-grid-item:last-child .mdx-timeline-item__border,.mdx-timeline-item__item:last-child .mdx-timeline-item__border,.toc_pP_5{display:none}.mdx-timeline-item__period--filled{background-color:rgb(var(--lsd-surface-secondary));border:1px solid rgb(var(--lsd-border-secondary));color:rgb(var(--lsd-text-secondary))!important}.mdx-timeline-item__period--transparent{background-color:rgb(var(--lsd-surface-primary));border:1px solid rgb(var(--lsd-border-primary));color:rgb(var(--lsd-text-primary))!important}.mdx-timeline-item__description{margin-bottom:auto;margin-top:24px;white-space:pre-wrap}.mdx-roadmap .mdx-grid__scroll{margin-bottom:32px!important}.mdx-roadmap .mdx-cta-section__title{font-size:var(--lsd-h2-fontSize)!important;font-weight:var(--lsd-h2-fontWeight)!important;line-height:var(--lsd-h2-lineHeight)!important}.mdx-roadmap .mdx-cta-section__description{font-size:var(--lsd-h5-fontSize)!important;font-weight:var(--lsd-h5-fontWeight)!important;line-height:var(--lsd-h5-lineHeight)!important;margin-top:24px;max-width:528px}.mdx-roadmap .mdx-cta-section__container h3{font-size:var(--lsd-subtitle1-fontSize)!important;font-weight:var(--lsd-subtitle1-fontWeight)!important;line-height:var(--lsd-subtitle1-lineHeight)!important}.mdx-roadmap__timeline .mdx-roadmap__timeline-item--no-border{border:none}.mdx-roadmap__timeline-item{height:100%}.scrollToBottom_NE5w{--offset-y:-2rem;background:rgb(var(--lsd-surface-primary))!important;opacity:1;position:absolute;top:min(100*var(--vh) - var(--ifm-navbar-height),var(--maxTop) - var(--ifm-navbar-height));transform:translateY(-2rem);transform:translateY(calc(-100% + var(--offset-y)));transition:.2s ease-in-out;z-index:100}.scrollToBottom_NE5w.hide_q_kn{opacity:0;transform:translateY(calc(-100% + var(--offset-y) - .625rem))}.mdx-section-header{border-top:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:row;gap:1rem;padding-top:24px;width:100%}.mdx-section-header>*{flex-basis:50%}.mdx-section-header__title{display:flex;flex-direction:row;gap:1.5rem}.mdx-section-header--no-border{border-top:none}.mdx-showcase-card__inner{display:flex;flex-direction:column;gap:1rem;padding-top:1.5rem}.mdx-showcase-card__logo{height:auto;width:40px}.mdx-showcase-card__index{align-items:center;background-color:rgb(var(--lsd-surface-secondary));border-radius:50%;color:rgb(var(--lsd-text-secondary))!important;display:flex;flex-direction:row;height:30px;justify-content:center;width:30px}.mdx-showcase-card__description{margin-top:1rem;padding-top:.5rem}.mdx-showcase-card--large .mdx-showcase-card__logo{width:40px}.mdx-showcase-card--small .mdx-showcase-card__logo{width:34px}.mdx-showcase-card--border-solid .mdx-showcase-card__description{border-top:1px solid rgb(var(--lsd-border-primary));padding-top:1rem}.mdx-showcase-card__cta{align-items:center;display:flex;font-size:14px;gap:6px;line-height:20px}.mdx-showcase-card__cta svg{transform:rotate(-90deg)}.mdx-social-card{align-items:flex-start;border:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;justify-content:space-between;min-height:144px;padding:1rem;text-decoration:none!important}.mdx-social-card__row{display:flex;justify-content:space-between;width:100%}.mdx-event-card{display:flex;flex-direction:column;gap:24px;text-decoration:none!important}.mdx-event-card__row,.mdx-event-profile__row,.mdx-news-card__row{display:flex;gap:8px;justify-content:space-between;width:100%}.mdx-event-card__thumbnail{border:1px solid rgba(var(--lsd-border-primary),.2);height:55px}.mdx-event-card__title{display:-webkit-box;height:48px;overflow:hidden;text-overflow:ellipsis;word-break:break-word}.sidebar_re4s,.tableOfContents_bqdL{max-height:calc(100vh - var(--ifm-navbar-height) - 2rem);overflow-y:auto}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>a,.sidebarItemLink_mo7H:hover,.tag_dtsN:hover{text-decoration:none}.mdx-event-card__info{border-top:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;gap:16px;padding:16px 2px 16px 0;width:100%}.mdx-event-card-list__button,.mdx-news-card-list__button{display:block;margin:0 auto}.mdx-event-banner__thumbnail{aspect-ratio:5/1;border:1px solid rgba(var(--lsd-border-primary),.2);margin-bottom:32px}.mdx-event-info__container{align-self:stretch;border-top:1px solid rgba(var(--lsd-border-primary));display:flex;flex:1 0 0;flex-direction:column;gap:16px;height:100%;padding:16px 24px 0 0}.mdx-event-about__container{display:grid;gap:16px;grid-template-columns:repeat(12,1fr)}.mdx-event-about__container>p{grid-column:1/6;margin-top:0!important}.mdx-event-about__container span{grid-column:7/13}.mdx-event-profile,.mdx-news-card{display:flex;flex-direction:column;gap:16px;text-decoration:none!important}.mdx-event-profile__image{aspect-ratio:1/1}.mdx-event-profile__social{align-items:center;display:flex;gap:8px}.mdx-event-cta__container h2{margin-bottom:32px;margin-top:0!important}.mdx-event-cta__button{height:70px!important;width:100%!important}.mdx-event-cta__button>span{align-items:center;display:flex;height:100%;justify-content:space-between}.mdx-input-cta-section__container{border-top:1px solid rgb(var(--lsd-border-primary));padding-top:24px;position:relative}.mdx-input-cta-section__container h2{margin-bottom:16px;margin-top:0!important}.mdx-input-cta-section__title{margin-bottom:24px!important}.mdx-input-cta-section__cta{height:40px!important;width:160px!important}.input-cta-section__description{margin-bottom:32px!important}.mdx-input-cta-section__form{display:flex;gap:16px}.mdx-input-cta-section__input-fields,.mdx-numbers-view{display:flex;gap:16px;width:100%}.mdx-input-cta-section__input-fields>div,.mdx-numbers-view-title,.mdx-table-view-description,.mdx-table-view-title{width:100%}.mdx-input-cta-section__message{bottom:-32px;display:block;position:absolute}form .mdx-input-cta-section__cta{width:146px!important}.mdx-event-header__container{display:flex;flex-direction:column;gap:24px}.event-details .mdx-event-header__container h1{margin-bottom:0!important}.mdx-news-card:hover .mdx-news-card__external-link-icon{display:flex!important}.mdx-news-card__thumbnail{border:1px solid rgba(var(--lsd-border-primary),.2);display:flex!important;height:144px;width:100%}.mdx-news-card__info{display:flex;gap:8px;width:100%}.mdx-news-card__tags{display:flex;gap:8px;margin-top:4px}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>.mdx-video-section__item-description,.mdx-video-section__description{margin-top:1rem}.mdx-news-card-thumbnail__container{height:144px;position:relative;width:100%}.mdx-news-card__external-link-icon{align-items:center;border:1px solid #fff;bottom:8px;display:none;height:28px;justify-content:center;position:absolute;right:8px;width:28px;z-index:9999}.mdx-table-view-row,.mdx-video-section--list .mdx-video-section__list>:not(:first-child){border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-news-card__external-link-icon svg>path{fill:#fff}.mdx-news-card__external-link-icon>div{margin:unset}.mdx-video-section .mdx-video-section__video-container{grid-column:8/17;height:0;overflow:hidden;padding-top:56.25%;position:relative;width:100%}.mdx-video-section iframe{left:0;position:absolute;top:0}.mdx-video-section .mdx-video-section--align-center{align-items:center;display:flex;flex-direction:column;margin:auto;max-width:936px}.mdx-video-section .mdx-video-section--align-center *{text-align:center}.mdx-video-section .mdx-video-section--align-center .mdx-video-section__description{margin-top:1.5rem}.mdx-video-section .mdx-video-section--align-left *{text-align:left}.mdx-video-section .mdx-video-section--align-right *{text-align:right}.mdx-video-section__header{align-items:center;display:flex;gap:16px}.mdx-video-section__header button{padding:6px 12px!important}.mdx-video-section__header svg{height:26px!important;width:26px!important}.mdx-video-section__header svg path{fill:rgb(var(--lsd-text-secondary))}.mdx-video-section--title-only .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);font-weight:300;line-height:var(--lsd-h5-lineHeight);max-width:886px}.mdx-video-section--title-button .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-display4-fontSize);font-weight:var(--lsd-display4-fontWeight);font-weight:300;line-height:var(--lsd-display4-lineHeight);max-width:886px}.mdx-video-section--full-width .mdx-video-section__container{display:grid;gap:0 1rem;grid-template-columns:repeat(2,1fr)}.mdx-video-section--full-width .mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h3-fontSize);font-weight:var(--lsd-h3-fontWeight);line-height:var(--lsd-h3-lineHeight)}.mdx-video-section--full-width .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-video-section--simple .mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-video-section--simple .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-video-section--list{display:grid;gap:0 1rem;grid-template-columns:repeat(16,1fr)}.mdx-video-section--list .mdx-video-section__container{grid-column:1/7}.mdx-video-section--list .mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-video-section--list .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-video-section--list .mdx-video-section__list{display:flex;flex-direction:column;gap:1.5rem 0;grid-column:2/3}.mdx-video-section--list .mdx-video-section__list>a{display:block;text-decoration:none}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>a>.mdx-video-section__item-title{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:1rem;color:rgb(var(--lsd-text-primary));display:flex;gap:12px;margin-top:1.5rem;padding:3px 11px;width:-moz-fit-content;width:fit-content}.mdx-table-view-table{padding:20px;width:100%}.mdx-table-view-table-title{margin-bottom:32px}.mdx-table-view-row{display:flex;gap:20px;padding:24px 0}.mdx-numbers-view-header{display:flex;flex-direction:column;gap:32px;width:100%}.mdx-numbers-view-content{display:flex;flex-direction:column;gap:16px;width:100%}.mdx-numbers-view-row{display:flex;gap:32px}.mdx-numbers-view-number{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:50%;display:flex;font-size:14px;height:32px;justify-content:center;padding:4px 12px;width:32px}.mdx-numbers-view-description{margin-top:16px;width:100%}.sidebar_re4s{position:sticky;top:calc(var(--ifm-navbar-height) + 2rem)}.sidebarItemTitle_pO2u{font-size:var(--ifm-h3-font-size);font-weight:var(--ifm-font-weight-bold)}.sidebarItemList_Yudw{font-size:.9rem}.sidebarItem__DBe{margin-top:.7rem}.sidebarItemLink_mo7H{color:var(--ifm-font-color-base);display:block}.sidebarItemLinkActive_I1ZP{color:var(--ifm-color-primary)!important}.blogHeader_QORd{border-bottom:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;gap:8px!important;left:0;margin-bottom:40px!important;margin-inline:var(--content-padding);padding-bottom:24px;padding-top:16px}.blogDescription_dds8,.blogTitle_KnCm{grid-column:span 24}.authorCol_y4tx{flex-grow:1!important;max-width:inherit!important}.imageOnlyAuthorRow_n8pT{display:flex;flex-flow:row wrap}.imageOnlyAuthorCol_OswQ{margin-left:.3rem;margin-right:.3rem}.container_PkUo{font-size:var(--lsd-body2-fontSize)!important;font-weight:var(--lsd-body2-fontWeight)!important;line-height:var(--lsd-body2-lineHeight)!important;margin-bottom:0!important;margin-top:32px!important}.blogContainer_wIfk{margin-bottom:16px!important;margin-top:8px!important}.container_PkUo>:not(:last-child):after{content:"•";display:inline-block;font-size:10px;margin-inline:.75rem}.authors_dZ4g>div{align-items:flex-end;display:inline-flex;flex-direction:row;max-width:unset;padding:0;width:auto}.authors_dZ4g>div .avatar{margin:0!important}.authors_dZ4g>div:not(:last-child):after{content:",";display:inline-block;margin-right:4px}.authors_dZ4g .avatar__name{line-height:0}.breadcrumbsContainer_RLvU{--ifm-breadcrumb-size-multiplier:0.8;margin-bottom:24px}.blogPostSubtitle_ysb0{margin-top:12px}.codeBlockContainer_Ckt0{background:var(--prism-background-color);border-radius:var(--ifm-code-border-radius);box-shadow:var(--ifm-global-shadow-lw)}.codeBlockContainer_Ckt0,.codeBlockContainer_EB2s{color:var(--prism-color);margin-bottom:var(--ifm-leading)}.codeBlockContainer_EB2s{background:#0000;border-radius:0}.codeBlockContent_biex{border-radius:inherit;direction:ltr;position:relative}.codeBlockTitle_Ktv7{border-bottom:1px solid var(--ifm-color-emphasis-300);border-top-left-radius:inherit;border-top-right-radius:inherit;font-size:var(--ifm-code-font-size);font-weight:500;padding:.75rem var(--ifm-pre-padding)}.codeBlock_TWhw,.codeBlock_bY9V{--ifm-pre-background:var(--prism-background-color);margin:0;padding:0}.codeBlockTitle_Ktv7+.codeBlockContent_biex .codeBlock_bY9V,.codeBlockTitle_sjMo+.codeBlockContent_ugSV .codeBlock_TWhw{border-top-left-radius:0;border-top-right-radius:0}.codeBlockLines_LDrR,.codeBlockLines_e6Vv{float:left;font:inherit;min-width:100%;padding:var(--ifm-pre-padding)}.buttonGroup_Qu4e button,.buttonGroup__atx button{color:var(--prism-color);line-height:0;transition:opacity var(--ifm-transition-fast) ease-in-out}.codeBlockLinesWithNumbering_bsRF,.codeBlockLinesWithNumbering_o6Pm{display:table;padding:var(--ifm-pre-padding) 0}.buttonGroup_Qu4e,.buttonGroup__atx{column-gap:.2rem;display:flex;position:absolute;right:calc(var(--ifm-pre-padding)/2);top:calc(var(--ifm-pre-padding)/2)}.buttonGroup__atx button{align-items:center;background:var(--prism-background-color);border:1px solid var(--ifm-color-emphasis-300);border-radius:var(--ifm-global-radius);display:flex;opacity:0;padding:.4rem}.buttonGroup__atx button:focus-visible,.buttonGroup__atx button:hover{opacity:1!important}.theme-code-block:hover .buttonGroup__atx button{opacity:.4}:where(:root){--docusaurus-highlighted-code-line-bg:#484d5b}:where([data-theme=dark]){--docusaurus-highlighted-code-line-bg:#646464}.theme-code-block-highlighted-line{background-color:var(--docusaurus-highlighted-code-line-bg);display:block;margin:0 calc(var(--ifm-pre-padding)*-1);padding:0 var(--ifm-pre-padding)}.codeLine_lJS_{counter-increment:a;display:table-row}.codeLineNumber_Tfdd{background:var(--ifm-pre-background);display:table-cell;left:0;overflow-wrap:normal;padding:0 var(--ifm-pre-padding);position:sticky;text-align:right;width:1%}.codeLineNumber_Tfdd:before{content:counter(a);opacity:.4}.codeLineContent_feaV{padding-right:var(--ifm-pre-padding)}.theme-code-block:hover .copyButtonCopied_YovB{opacity:1!important}.copyButtonIcons_an20{align-items:center;display:flex;justify-content:center;position:relative}.copyButtonIcon_ZL7v,.copyButtonSuccessIcon_P2h8{fill:currentColor;height:14px;opacity:inherit;transition:all var(--ifm-transition-fast) ease;width:14px}.copyButtonSuccessIcon_P2h8{color:#00d600;opacity:0}.copyButtonCopied_YovB .copyButtonSuccessIcon_P2h8{opacity:1;transition-delay:75ms}.wordWrapButtonIcon_Bwma{height:1.2rem;width:1.2rem}.codeBlockTitle_sjMo{background:var(--prism-background-color);border-bottom:1px solid var(--ifm-color-primary);border-top-left-radius:inherit;border-top-right-radius:inherit;font-size:var(--ifm-code-font-size);font-weight:500;padding:.75rem var(--ifm-pre-padding)}.buttonGroup_Qu4e button{align-items:center;background:var(--prism-background-color);border:1px solid rgb(var(--lsd-border-primary));display:flex;height:28px;justify-content:center;opacity:0;width:28px}.buttonGroup_Qu4e button:focus-visible,.buttonGroup_Qu4e button:hover{opacity:1!important}.anchorWithStickyNavbar_LWe7{scroll-margin-top:calc(var(--ifm-navbar-height) + .5rem)}.anchorWithHideOnScrollNavbar_WYt5{scroll-margin-top:.5rem}.hash-link{opacity:0;padding-left:.5rem;transition:opacity var(--ifm-transition-fast);-webkit-user-select:none;user-select:none}.hash-link:before{content:"#"}.img_ev3q{height:auto}.admonition_ntHH{border:1px solid var(--ifm-alert-border-color);display:flex;gap:18px;margin-bottom:24px;padding:18px}.admonitionHeading_JPfy{font-size:1.25rem!important;font-weight:400;line-height:1.5rem!important;margin-bottom:.3rem;text-transform:capitalize}.admonitionHeading_JPfy code{text-transform:none}.admonitionIcon_BGV6{display:inline-block}.admonitionIcon_BGV6 svg{display:inline-block;height:16px;width:16px}.admonitionIcon_BGV6 svg *{stroke:none!important;fill:var(--ifm-alert-border-color)}.admonitionContent_iozl{display:block;margin-top:12px}.tag_dtsN{border:1px solid var(--docusaurus-tag-list-border);transition:border var(--ifm-transition-fast)}.tag_dtsN:hover{--docusaurus-tag-list-border:var(--ifm-link-color)}.tagRegular_SGIC{border-radius:var(--ifm-global-radius);font-size:90%;padding:.2rem .5rem .3rem}.tagWithCount_gIvh{align-items:center;border-left:0;display:flex;padding:0 .5rem 0 1rem;position:relative}.tagWithCount_gIvh:after,.tagWithCount_gIvh:before{border:1px solid var(--docusaurus-tag-list-border);content:"";position:absolute;top:50%;transition:inherit}.tagWithCount_gIvh:before{border-bottom:0;border-right:0;height:1.18rem;right:100%;transform:translate(50%,-50%) rotate(-45deg);width:1.18rem}.tagWithCount_gIvh:after{border-radius:50%;height:.5rem;left:0;transform:translateY(-50%);width:.5rem}.tagWithCount_gIvh span{background:var(--ifm-color-secondary);border-radius:var(--ifm-global-radius);color:var(--ifm-color-black);font-size:.7rem;line-height:1.2;margin-left:.3rem;padding:.1rem .4rem}.root_UH70{align-items:center;display:flex;flex-direction:row}.tags_LT8x{display:inline;margin:0 0 0 1rem!important;padding:0!important}.tag_NdBD:not(:first-child){margin-left:.5rem}.tag_NdBD a{margin:0!important;padding:0!important}.lastUpdated_vwxv{font-size:smaller;font-style:italic;margin-top:.2rem}.blogPostFooterDetailsFull_FF8l{flex-direction:column}.readMoreLink_cNfS span{color:rgb(var(--lsd-text-secondary))!important}.tableOfContents_bqdL{position:sticky;top:calc(var(--ifm-navbar-height) + 1rem)}.tocCollapsibleButton_dxRj{align-items:center;border:1px solid rgb(var(--lsd-border-primary));display:flex;font-size:inherit;justify-content:space-between;padding:.4rem .8rem;width:100%}.tocCollapsibleButtonExpanded_TSyC:after,.tocCollapsibleExpanded_zTjk{transform:none}.tocCollapsible_ROek{background-color:var(--ifm-menu-color-background-active);border-radius:var(--ifm-global-radius);margin:1rem 0}.tocCollapsibleContent_Qsjj>ul{border-left:none;border:1px solid rgb(var(--lsd-border-primary))!important;border-top:none!important;font-size:15px;padding:.2rem 0}.tocCollapsibleContent_Qsjj ul li{margin:.4rem .8rem}.tocCollapsibleContent_Qsjj a{display:block;-webkit-text-decoration:var(--ifm-link-text-decoration);text-decoration:var(--ifm-link-text-decoration)}.badge_AsjZ{background:#0000}.docItemGrid_SzoZ{display:grid;gap:16px;grid-template-columns:repeat(14,1fr)}.gap1_XuuQ{grid-column:span 1}.tocMobile_imaF{margin-bottom:2rem;margin-top:-.5rem}.tocMobile_imaF>div{display:block!important;margin:0}.docItemContainer_hrrU h1{margin-bottom:40px!important}.docItemContainer_hrrU h2,.docItemContainer_hrrU h3{margin-top:32px!important}@media (min-width:0px){.mdx-box{--mdx-box-top:var(--mdx-box-top-xs);--mdx-box-bottom:var(--mdx-box-bottom-xs)}}@media (min-width:576px){.mdx-box{--mdx-box-top:var(--mdx-box-top-sm);--mdx-box-bottom:var(--mdx-box-bottom-sm)}}@media (min-width:768px){.blog-archive-page .main-wrapper main>section .container>.row{display:grid;gap:64px 24px;grid-template-columns:repeat(24,1fr)}.blog-archive-page .main-wrapper main>section .container>.row>.col{grid-column:span 12;margin:0!important;max-width:unset;padding:0 8.3333333333% 0 0}.mdx-box{--mdx-box-top:var(--mdx-box-top-md);--mdx-box-bottom:var(--mdx-box-bottom-md)}.mdx-scroll-buttons__button--with-label{gap:12px;min-width:83px;padding:5px 11px 5px 9px!important;width:auto!important}.mdx-scroll-buttons__button--with-label:first-of-type,.mdx-scroll-buttons__button--with-label:last-of-type{justify-content:flex-start}.mdx-scroll-buttons--spaced>div{gap:0 1rem;justify-content:space-between}.mdx-scroll-buttons--spaced>div>button:not(:last-child){border-right:1px solid rgb(var(--lsd-border-primary))!important}}@media (min-width:997px){.full-width>div .main-wrapper main>div>div{grid-template-columns:unset!important}.main-wrapper .container{padding-bottom:96px}.main-wrapper>div{display:grid;gap:1rem;grid-template-columns:repeat(16,45px);margin:auto;width:-moz-fit-content;width:fit-content}.main-wrapper>div aside{border:none;grid-column:1/5;padding-left:var(--content-padding);width:auto}.main-wrapper>div main{grid-column:6/16;margin-top:48px;max-width:none}.navbar__left{flex-basis:8.3333333333%;flex:0 1 auto}.navbar__left-items{flex:1 0;margin:auto}nav.menu{margin-top:48px;padding-top:16px}.container{padding-inline:0!important}.navbar__brand,aside a[tabindex="-1"] img{margin-left:14px}.theme-doc-sidebar-menu .menu__link{padding-left:8px}.navbar .lsd-icon-button--medium{height:28px!important;width:28px!important}.blog-wrapper>.container>.row{display:grid;gap:16px;grid-template-columns:repeat(24,1fr)}.blog-wrapper>.container>.row>aside{grid-column:1/5}.blog-wrapper>.container>.row>main{grid-column:6/20}.blog-wrapper:not(.blog-wrapper--index)>.container>.row>div:last-child{grid-column:21/25;max-width:unset}.wrapper_SWrM:not(.active_qZD5) .fullscreenButton_Bocn{background:rgb(var(--lsd-surface-primary));bottom:8px;display:flex;opacity:0;position:absolute;right:8px;transition:.3s;visibility:hidden}.wrapper_SWrM:not(.active_qZD5):hover .fullscreenButton_Bocn{opacity:1;visibility:visible}:root{--docusaurus-announcement-bar-height:30px}.announcementBarClose_gvF7,.announcementBarPlaceholder_vyr4{flex-basis:50px}.searchBox_ZlJk{padding:var(--ifm-navbar-item-padding-vertical) var(--ifm-navbar-item-padding-horizontal)}.menuHtmlItem_M9Kj{padding:var(--ifm-menu-link-padding-vertical) var(--ifm-menu-link-padding-horizontal)}.menu_NjKY{flex-grow:1;padding-top:16px}@supports (scrollbar-gutter:stable){.menu_NjKY{scrollbar-gutter:stable}}.menuWithAnnouncementBar_U5pk{margin-bottom:var(--docusaurus-announcement-bar-height)}.sidebar_UEyd{display:flex;flex-direction:column;height:100%;padding-top:var(--ifm-navbar-height)}.sidebarWithHideableNavbar__00c{padding-top:0!important}.sidebarHidden_F1ZE{opacity:0;visibility:hidden}.sidebarLogo_Ydl9{align-items:center;color:inherit!important;display:flex!important;margin:0 var(--ifm-navbar-padding-horizontal);max-height:var(--ifm-navbar-height);min-height:var(--ifm-navbar-height);text-decoration:none!important}.sidebarLogo_Ydl9 img{height:2rem;margin-right:.5rem}.expandButton_m80_{align-items:center;background-color:var(--docusaurus-collapse-button-bg);display:flex;height:100%;justify-content:center;position:absolute;right:0;top:0;transition:background-color var(--ifm-transition-fast) ease;width:100%}.expandButton_m80_:focus,.expandButton_m80_:hover{background-color:var(--docusaurus-collapse-button-bg-hover)}.expandButtonIcon_BlDH{transform:rotate(0)}[dir=rtl] .expandButtonIcon_BlDH{transform:rotate(180deg)}.docSidebarContainer_b6E3{border-right:1px solid var(--ifm-toc-border-color);-webkit-clip-path:inset(0);clip-path:inset(0);display:block;margin-top:calc(var(--ifm-navbar-height)*-1);transition:width var(--ifm-transition-fast) ease;width:var(--doc-sidebar-width);will-change:width}.docSidebarContainerHidden_b3ry{cursor:pointer;width:var(--doc-sidebar-hidden-width)}.sidebarViewport_Xe31{height:100%;max-height:100vh;position:sticky;top:0}.docMainContainer_gTbr{flex-grow:1;max-width:calc(100% - var(--doc-sidebar-width))}.docMainContainerEnhanced_Uz_u{max-width:calc(100% - var(--doc-sidebar-hidden-width))}.docItemWrapperEnhanced_czyv{max-width:calc(var(--ifm-container-width) + var(--doc-sidebar-width))!important}.mdx-box{--mdx-box-top:var(--mdx-box-top-lg);--mdx-box-bottom:var(--mdx-box-bottom-lg)}.mdx-cta-section__header__mobile-button{display:none}.mdx-feature-list--bottom-aligned .mdx-feature-list__feature{min-height:332px}.mdx-feature-list--bottom-aligned .mdx-feature-list__feature-description{display:flex;flex:1 0;flex-direction:column;justify-content:flex-end}.mdx-feature-list--top-aligned .mdx-feature-list__feature-description{margin-top:40px}.mdx-feature-list__feature{padding:1rem;position:relative}.mdx-feature-list__feature-inner{height:100%}.mdx-feature-list__feature-border{background:rgb(var(--lsd-border-primary));height:calc(100% - 2rem);position:absolute;right:0;top:1rem;width:1px}.mdx-feature-list__feature:nth-child(odd){padding-left:0}.mdx-feature-list__feature:nth-child(2n){padding-right:0}.mdx-feature-list__feature:nth-child(2n) .mdx-feature-list__feature-border,.mdx-video-section__header__mobile-button,.tocMobile_ITEo{display:none}.lastUpdated_vwxv{text-align:right}.docItemGrid_SzoZ{display:grid;gap:var(--doc-grid-gap);grid-template-columns:repeat(11,1fr)}.docItemCol_F52z{grid-column:span 11}}@media (min-width:997px)and (max-width:1199px){.main-wrapper .container>div>div,.navbar{padding-inline:0!important}.main-wrapper,.navbar{width:960px!important}.events .main-wrapper>div aside{grid-column:1/1}.events .main-wrapper>div main,.navbar__inner{grid-column:1/17}.plugin-id-root-pages main{grid-column:4/14!important}.main-wrapper .container{padding-bottom:96px}.main-wrapper>div{width:960px}.navbar{display:grid;gap:1rem;grid-template-columns:repeat(16,45px);margin:auto}.navbar__left-items{display:flex;justify-content:center}.main-wrapper{margin:0 auto!important}}@media (min-width:1200px){.main-wrapper>div main>.container,.main-wrapper>main>.row>.col,.navbar{padding-inline:0!important}.plugin-id-root-pages main{grid-column:4/16!important}.footer{padding:0!important}.main-wrapper>div{display:grid;gap:1rem;grid-template-columns:repeat(16,1fr);margin-inline:unset;width:unset}.main-wrapper>div aside{border:none;grid-column:1/4;width:auto}.main-wrapper>div main{grid-column:5/17;max-width:none}.theme-doc-toc-desktop{top:calc(var(--ifm-navbar-height))!important}.navbar__inner{display:grid;gap:16px;grid-template-columns:repeat(24,1fr);padding:0}.navbar__left{align-items:center;display:flex;flex-direction:row;grid-column:1/6}.navbar__left-items{grid-column:6/20}.navbar__right-items{flex-direction:row;gap:16px;grid-column:20/25;justify-content:flex-end}.blog-archive-page .main-wrapper main>section .container>.row{display:grid;gap:64px 24px;grid-template-columns:repeat(24,1fr)}.blog-archive-page .main-wrapper main>section .container>.row>.col{grid-column:span 8;margin:0!important;max-width:unset;padding:0 12.5% 0 0}.mdx-box{--mdx-box-top:var(--mdx-box-top-xl);--mdx-box-bottom:var(--mdx-box-bottom-xl)}.docItemGrid_SzoZ{display:grid;gap:var(--doc-grid-gap);grid-template-columns:repeat(12,1fr)}.docItemCol_F52z{grid-column:span 10}.tocMobile_imaF{display:none}}@media (min-width:1440px){.container{max-width:var(--ifm-container-width-xl)}}@media (max-width:1440px)and (min-width:992px){.mdx-hero-model{transform:translateX(calc((var(--content-padding) + 0)*-3));width:calc(100vw + var(--content-padding))}}@media (max-width:1199px){.main-wrapper main>.container{padding-bottom:calc(var(--desktop-footer-gap) - var(--ifm-footer-padding-vertical) - var(--doc-grid-gap))!important}.header-github-link:before{margin-right:16px}.toc_pP_5{display:none}}@media (max-width:997px);@media (max-width:996px){.navbar-sidebar__close,.navbar__toggle{border:1px solid rgb(var(--lsd-border-primary))}.blog-wrapper h1[itemprop=headline],.mdx-cta-button .lsd-button--large,.mdx-cta-section__description,.mdx-cta-section__title,.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__link>div{font-family:var(--lsd-typography-generic-font-family)!important}.col{--ifm-col-width:100%;flex-basis:var(--ifm-col-width);margin-left:0}.footer{--ifm-footer-padding-horizontal:0;padding-bottom:0!important;--ifm-footer-padding-horizontal:var(--content-padding)}.desktop,.footer__link-separator,.navbar__item,.navbar__left-items{display:none}.footer__col{margin-bottom:calc(var(--ifm-spacing-vertical)*3)}.footer__link-item,.main-wrapper>div,.mdx-cta-section--full-width .mdx-cta-section__container,.mdx-cta-section--list,.mdx-video-section--full-width .mdx-video-section__container,.mdx-video-section--list,.tocMobile_imaF{display:block}.hero{padding-left:0;padding-right:0}.navbar>.container,.navbar>.container-fluid,.root_xZfc{padding:0}.navbar__toggle{display:inherit}.navbar__search-input{width:9rem}.mdx-numbers-view,.mdx-section-header,.pills--block,.tabs--block{flex-direction:column}:root{--ifm-background-color:rgb(var(--lsd-surface-primary))}.navbar__color-mode-toggle{display:none!important}.main-wrapper main>.container{padding-bottom:calc(var(--mobile-footer-gap) - var(--ifm-footer-padding-vertical) - var(--doc-grid-gap))!important}.grid{grid-template-columns:repeat(2,1fr)}.footer-grid,.l-modal__container,.mdx-feature-list__list,.mdx-profile-card{grid-template-columns:1fr}.navbar__toggle{align-items:center;box-sizing:border-box;display:flex;height:32px;justify-content:center;margin:0;width:32px}.navbar__toggle svg{height:14px;width:14px}.theme-admonition{display:block!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu{padding-inline:0}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__list{display:flex;flex-direction:column;gap:8px;padding:var(--content-padding)}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__link{border-left:none;padding-left:0}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__link>div{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important;text-decoration:none}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer{margin-top:3rem;padding:0!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer>div{display:flex!important;flex-direction:column!important;gap:32px!important}.colorModeToggle_GSaI,.dropdownNavbarItem_o23I,.l-modal__backdrop,.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer>button{display:none}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer .container.container-fluid{margin-bottom:16px!important;padding-inline:var(--content-padding)}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer .container.container-fluid>.footer__links{gap:0!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer>div[class*=secondRow]{margin:var(--content-padding)}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer .footer__links{gap:16px!important}.navbar-sidebar__brand{height:60px;padding-block:0}.navbar-sidebar__brand>button{height:32px;width:32px}.navbar-sidebar__brand>button:first-of-type{margin-left:auto;margin-right:0!important}.navbar-sidebar__brand>button:not(:last-of-type){border-right:none}.navbar-sidebar__close{align-items:center;display:flex;justify-content:center;margin-left:unset}.blog-divider{margin-block:32px!important}.blog-wrapper main,.blogHeader_QORd{padding-inline:var(--content-padding)}.blog-wrapper article footer{align-items:flex-start;flex-direction:column}.blog-wrapper article footer .read-more-link{margin-top:24px}.blog-wrapper article footer .read-more-link,.blog-wrapper article footer .read-more-link button{flex-basis:100%;width:100%}.blog-wrapper h1[itemprop=headline]{font-size:var(--lsd-h2-fontSize)!important;font-weight:var(--lsd-h2-fontWeight)!important;line-height:var(--lsd-h2-lineHeight)!important}.wrapper_SWrM>div{overflow:visible!important}.nav_cMpg{height:56px}.backToTop_wDfN{bottom:16px;left:16px;margin-top:76px!important;position:relative}.l-modal{overflow:hidden;width:100vw}.l-modal__container{display:grid;height:100vh;max-width:unset;min-height:-webkit-fill-available;width:100%}.docItemGrid_SzoZ,.mdx-showcase{grid-template-columns:repeat(2,1fr)}.root_ONDG,.root_ltHz{height:100%;overflow:auto}.l-modal__content{border:none;grid-column:1/2;grid-row:1/2;height:100%;margin:0;max-height:100vh;overflow:hidden}.modal_kLVz .header_QwCa{padding:12px 16px}.modal_kLVz .closeButton_Rr0e{display:block;height:2rem;width:2rem}.root_EFVO{padding:40px 24px}.root_ONDG{max-height:100%;padding:24px 24px 64px}.groups_p1lF{max-height:unset;overflow:auto}.root_ltHz{padding:24px 0 64px}.searchBox_ZlJk{position:absolute;right:var(--ifm-navbar-padding-horizontal)}.mdx-cta-button .lsd-button--large{font-size:var(--lsd-label2-fontSize)!important;font-weight:var(--lsd-label2-fontWeight)!important;line-height:var(--lsd-label2-lineHeight)!important;padding:5px 11px}.mdx-cta-section--align-center .lsd-button--large{padding:10px 40px}.mdx-cta-section,.mdx-video-section{padding:16px 0 0}.mdx-cta-section--title-button .mdx-cta-section__description,.mdx-video-section--title-button .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight)}.mdx-cta-section--full-width .mdx-cta-section__title,.mdx-cta-section--simple .mdx-cta-section__title,.mdx-video-section--full-width .mdx-video-section__title,.mdx-video-section--simple .mdx-video-section__title{margin-top:1.5rem}.mdx-cta-section--list .mdx-cta-section__list,.mdx-roadmap__timeline{margin-top:4rem}.mdx-cta-section--list .mdx-cta-section__list>*{border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>.mdx-cta-section__item-title{font-size:.875rem!important;line-height:1.25rem!important}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>.mdx-cta-section__item-description{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.mdx-cta-section__header__desktop-button,.sidebar_re4s,.tableOfContents_bqdL{display:none}.mdx-cta-section__header__mobile-button{margin-top:24px}.mdx-cta-section__title{font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.mdx-cta-section__description{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important}.mdx-feature-list .mdx-feature-list__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-subtitle2-fontSize);font-weight:var(--lsd-subtitle2-fontWeight);line-height:var(--lsd-subtitle2-lineHeight);padding:1rem 0}.mdx-feature-list--cta-bottom .mdx-feature-list__extra{margin-top:2rem;padding:0}.mdx-feature-list__header h2,.mdx-roadmap .mdx-cta-section__title,.mdx-section-header h2{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.lsd-button span,.mdx-hero-description{line-height:var(--lsd-body2-lineHeight)!important}.mdx-feature-list__feature{border:none;border-top:1px solid rgb(var(--lsd-border-primary));padding:1.5rem 0}.mdx-feature-list__feature .mdx-feature-list__feature-index--circle{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-subtitle2-fontSize);font-weight:var(--lsd-subtitle2-fontWeight);height:1.625rem;line-height:var(--lsd-subtitle2-lineHeight);width:1.625rem}.mdx-feature-list__feature .mdx-feature-list__feature-title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight);margin-top:1rem}.mdx-feature-list__feature .mdx-feature-list__feature-description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-body1-fontSize);font-weight:var(--lsd-body1-fontWeight);line-height:var(--lsd-body1-lineHeight);margin-top:3rem}.blogPostSubtitle_ysb0,.mdx-hero-description,.mdx-roadmap .mdx-cta-section__description,.mdx-section-header__description,.mdx-section-header__title,.mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family)!important}.mdx-feature-list__feature:nth-child(odd) .mdx-feature-list__feature-inner{border:none}.lsd-button,.mdx-hero-action button{padding:6px 12px!important}.mdx-hero-actions{align-items:center;display:flex;flex-direction:row;justify-content:stretch;width:100%;z-index:0}.mdx-hero-actions>*{flex-basis:50%}.mdx-hero-actions>a,.mdx-hero-actions>a>button{display:block;width:100%}.mdx-hero-description{font-size:var(--lsd-body2-fontSize)!important;font-weight:var(--lsd-body2-fontWeight)!important;margin-top:16px}.mdx-hero-info{padding:28px 0}.mdx-hero-model--top{top:calc(var(--logos-hero-info-height)/ 1 - 10px)}.mdx-hero-model{transform:translate(-12px);width:calc(100vw - 20px)}.mdx-hero-model--shade{bottom:0;height:90vh!important}.mdx-hero-title,.mdx-numbers-view-section-title{font-size:28px!important}.mdx-hero-video>*{height:var(--hero-video-height-mobile);min-height:var(--hero-video-min-height-mobile);transform:translateY(var(--hero-video-offset-y-mobile))}.mdx-hero-video img,.mdx-hero-video video{max-width:calc((100vw - 32px)*var(--hero-video-scale-mobile))}.mdx-logo-carousel .mdx-logo-carousel__inner{margin-top:64px}.mdx-timeline-item{width:204px}.mdx-roadmap .mdx-cta-section__description{font-size:var(--lsd-body1-fontSize)!important;font-weight:var(--lsd-body1-fontWeight)!important;line-height:var(--lsd-body1-lineHeight)!important;margin-top:16px}.scrollToBottom_NE5w{--offset-y:-1rem}.mdx-section-header__title{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important;display:flex;flex-wrap:wrap}.mdx-section-header__description{font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.mdx-showcase{gap:1.5rem 1rem}.mdx-showcase-card__name{font-size:1.375rem!important;line-height:1.75rem!important}.mdx-showcase-card__description{margin-top:1rem}.mdx-showcase-card__logo{height:auto;width:34px}.mdx-news-card-thumbnail__container,.mdx-news-card__thumbnail{height:128px}.mdx-video-section--list .mdx-cta-button{margin-bottom:40px;margin-top:24px}.mdx-video-section--list .mdx-video-section__list{margin-top:4rem}.mdx-video-section--list .mdx-video-section__list>*{border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>.mdx-video-section__item-title{font-size:.875rem!important;line-height:1.25rem!important}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>.mdx-video-section__item-description{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.mdx-video-section__header__mobile-button{margin-top:24px}.mdx-video-section__title{font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.mdx-table-view-table-title{font-size:28px!important;margin-bottom:24px}.mdx-table-view-row{flex-direction:column;gap:32px}.mdx-numbers-view{gap:32px}.mdx-numbers-view-row{gap:16px}.mdx-numbers-view-number{height:32px;padding:4px 12px;width:32px}.mdx-numbers-view-header{align-items:center;display:flex;flex-direction:row;flex-wrap:wrap;gap:16px}.lsd-button span{font-size:var(--lsd-body3-fontSize)!important}.mdx-numbers-view-title{font-size:20px!important}.container_PkUo{margin-top:24px!important}.blogContainer_wIfk{margin-top:8px!important}.blogPostSubtitle_ysb0{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important;margin-top:4px}.docItemContainer_F8PC{padding:0 .3rem}.docItemGrid_SzoZ{display:grid;padding-left:0}.docItemCol_F52z{grid-column:span 2}}@media (max-width:767px){.blog-archive-page .main-wrapper main>section .container>.row{margin-top:0}.blog-archive-page .main-wrapper main>section .container>.row .col{margin-top:40px!important}.mdx-scroll-buttons>div{justify-content:flex-end}.mdx-scroll-buttons>div>button:not(:last-child){border-right:none!important}.mdx-scroll-buttons .mdx-scroll-buttons__label{display:none}.mdx-hero{justify-content:unset}.mdx-hero h1{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h2-fontSize)!important;font-weight:var(--lsd-h2-fontWeight)!important;line-height:var(--lsd-h2-lineHeight)!important}.mdx-ns__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight)}.mdx-ns__inner form{margin-top:40px}.mdx-ns__inner .mdx-ns__inputs{flex-direction:column;gap:24px}.mdx-ns__inner .mdx-ns__inputs>*{width:100%}.mdx-ns__inner .mdx-ns__submit-button{margin-top:24px}.mdx-profile-card{display:flex;scroll-snap-align:start!important;width:var(--mobile-width)}.mdx-profile-card__profile{display:flex;flex-direction:column;gap:16px}.mdx-profile-card__name{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h5-fontSize)!important;font-weight:var(--lsd-h5-fontWeight)!important;line-height:var(--lsd-h5-lineHeight)!important}.mdx-event-card__thumbnail{height:unset}.mdx-news-card-thumbnail__container,.mdx-news-card__thumbnail{height:100%}}@media (max-width:576px){.markdown h1:first-child{--ifm-h1-font-size:2rem}.markdown>h2{--ifm-h2-font-size:1.5rem}.markdown>h3{--ifm-h3-font-size:1.25rem}.title_f1Hy{font-size:2rem}}@media (max-width:575px){:root{--ifm-spacing-vertical:var(--ifm-global-spacing);--ifm-spacing-horizontal:var(--ifm-global-spacing);--ifm-button-size-multiplier:1.25;--ifm-button-padding-horizontal:calc(var(--ifm-button-size-multiplier)*var(--ifm-spacing-horizontal));--ifm-button-padding-vertical:calc(var(--ifm-button-size-multiplier)*var(--ifm-spacing-vertical));--ifm-alert-padding-horizontal:calc(var(--ifm-spacing-horizontal)*2);--ifm-alert-padding-vertical:calc(var(--ifm-spacing-vertical)*2)}.markdown>h2{--ifm-h2-font-size:1.75rem}.markdown h4{--ifm-h4-font-size:1.25rem!important}.footer>div{display:flex!important;flex-direction:column;gap:40px!important;margin-inline:16px;padding-inline:0!important;width:auto}.footer__links{display:flex!important;flex-direction:column!important}.footer__links>div:first-of-type,.footer__links>div:nth-of-type(3){margin-bottom:4px!important}.footer__links>div:nth-of-type(2){margin-bottom:34px!important}.col.footer__col{margin-bottom:0!important}.navbar__left{display:flex}.bottom-cta-sections .mdx-grid__content{gap:98px!important}.bottom-cta-sections .mdx-grid__content:last-of-type>div:last-of-type>.mdx-input-cta-section__container{margin-bottom:120px}.events .main-wrapper main>.container{padding-bottom:0!important}.footer{--ifm-footer-padding-horizontal:0}.firstRow_ar1q{margin-bottom:64px!important;padding-inline:var(--content-padding)!important}.secondRow__ww3{margin-inline:var(--content-padding)!important}.mdx-event-card-list__container,.mdx-news-card-list__container{margin-bottom:48px;margin-top:32px}.mdx-event-card-list__tabs .lsd-tab-item,.mdx-news-card-list__tabs .lsd-tab-item{padding:6px 14px!important}.mdx-event-about__container,.mdx-input-cta-section__input-fields{display:flex;flex-direction:column;gap:32px}.mdx-event-about__container span,.mdx-event-header__container p{font-size:14px!important;line-height:20px!important}.mdx-event-profile-list__container .mdx-grid>.mdx-grid__content{gap:24px!important}.mdx-input-cta-section__input-fields>div{width:100%!important}.mdx-input-cta-section__container{position:relative}.mdx-input-cta-section__container .mdx-input-cta-section__form{flex-direction:column;gap:40px;margin-bottom:-24px}.mdx-input-cta-section__container .input-cta-section__description--no-form{margin-bottom:32px!important}.input-cta-section__description{margin-bottom:24px!important}.event-details .mdx-event-header__container h1{font-size:24px!important;line-height:32px!important;margin-bottom:0!important}.mdx-news-card h3{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-news-card__thumbnail{max-height:198px}}@media (max-width:574px){.mdx-ghc__issue-content-grid{grid-template-columns:82px 1fr}}@media (hover:hover){.backToTopButton_sjWU:hover{background-color:var(--ifm-color-emphasis-300)}}@media (pointer:fine){.thin-scrollbar{scrollbar-width:thin}.thin-scrollbar::-webkit-scrollbar{height:var(--ifm-scrollbar-size);width:var(--ifm-scrollbar-size)}.thin-scrollbar::-webkit-scrollbar-track{background:var(--ifm-scrollbar-track-background-color);border-radius:10px}.thin-scrollbar::-webkit-scrollbar-thumb{background:var(--ifm-scrollbar-thumb-background-color);border-radius:10px}.thin-scrollbar::-webkit-scrollbar-thumb:hover{background:var(--ifm-scrollbar-thumb-hover-background-color)}}@media (prefers-reduced-motion:reduce){:root{--ifm-transition-fast:0ms;--ifm-transition-slow:0ms}}@media print{.announcementBar_mb4j,.footer,.menu,.navbar,.pagination-nav,.table-of-contents,.tocMobile_ITEo{display:none}.tabs{page-break-inside:avoid}.codeBlockLines_LDrR,.codeBlockLines_e6Vv{white-space:pre-wrap}} \ No newline at end of file diff --git a/assets/css/styles.3c000f26.css b/assets/css/styles.3c000f26.css new file mode 100644 index 00000000..8eb54b8f --- /dev/null +++ b/assets/css/styles.3c000f26.css @@ -0,0 +1 @@ +.col,.container{padding:0 var(--ifm-spacing-horizontal);width:100%}.markdown>h2,.markdown>h3,.markdown>h4,.markdown>h5,.markdown>h6{margin-bottom:calc(var(--ifm-heading-vertical-rhythm-bottom)*var(--ifm-leading))}.markdown li,body{word-wrap:break-word}body,ol ol,ol ul,ul ol,ul ul{margin:0}blockquote,pre{margin:0 0 var(--ifm-spacing-vertical)}.breadcrumbs__link,.button{transition-timing-function:var(--ifm-transition-timing-default)}.button,code{vertical-align:middle}.button--outline.button--active,.button--outline:active,.button--outline:hover,:root{--ifm-button-color:var(--ifm-font-color-base-inverse)}.menu__link:hover,a{transition:color var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.navbar--dark,:root{--ifm-navbar-link-hover-color:var(--ifm-color-primary)}:root,html[data-theme=dark]{--ifm-color-emphasis-500:var(--ifm-color-gray-500)}[data-theme=dark],html[data-theme=dark]{--ifm-color-scheme:dark}:root,[data-theme=light]{--ifm-color-scheme:light}.menu,.navbar-sidebar,body,html{overflow-x:hidden}code,pre code{--lsd-typography-generic-font-family:monospace}.events .theme-doc-breadcrumbs .breadcrumbs__link,.text--capitalize{text-transform:capitalize}.button,.dropdown__link,.footer__item,.text--truncate{white-space:nowrap}.mdx-erc__title,.mdx-event-card__title{-webkit-line-clamp:2;-webkit-box-orient:vertical}.mdx-cta-section .mdx-cta-section--align-center .mdx-cta-button,.mdx-video-section .mdx-video-section--align-center .mdx-cta-button{margin-inline:auto}.mdx-cta-section--full-width .mdx-cta-section__link,.mdx-video-section--full-width .mdx-video-section__link{grid-area:2/2/3/3}.toggleButton_dl49,html{-webkit-tap-highlight-color:transparent}*,.buttonGroup_Qu4e button,.dropdownNavbarItem_o23I,.events .theme-doc-breadcrumbs .breadcrumbs__item--active,.mdx-event-info__container,.mdx-news-card__external-link-icon,.mdx-numbers-view-number,.root_xZfc,.toggle_K23S{box-sizing:border-box}.buttonGroup_Qu4e button,.buttonGroup__atx button{transition:opacity var(--ifm-transition-fast) ease-in-out}.clean-list,.containsTaskList_mC6p,.dropdown__menu,.menu__list,.root_N57j ul li{list-style:none}:root{--ifm-dark-value:10%;--ifm-darker-value:15%;--ifm-darkest-value:30%;--ifm-light-value:15%;--ifm-lighter-value:30%;--ifm-lightest-value:50%;--ifm-contrast-background-value:90%;--ifm-contrast-foreground-value:70%;--ifm-contrast-background-dark-value:70%;--ifm-contrast-foreground-dark-value:90%;--ifm-color-primary:#3578e5;--ifm-color-secondary:#ebedf0;--ifm-color-success:#00a400;--ifm-color-info:#54c7ec;--ifm-color-warning:#ffba00;--ifm-color-danger:#fa383e;--ifm-color-primary-dark:#306cce;--ifm-color-primary-darker:#2d66c3;--ifm-color-primary-darkest:#2554a0;--ifm-color-primary-light:#538ce9;--ifm-color-primary-lighter:#72a1ed;--ifm-color-primary-lightest:#9abcf2;--ifm-color-primary-contrast-background:#ebf2fc;--ifm-color-primary-contrast-foreground:#102445;--ifm-color-secondary-dark:#d4d5d8;--ifm-color-secondary-darker:#c8c9cc;--ifm-color-secondary-darkest:#a4a6a8;--ifm-color-secondary-light:#eef0f2;--ifm-color-secondary-lighter:#f1f2f5;--ifm-color-secondary-lightest:#f5f6f8;--ifm-color-secondary-contrast-background:#fdfdfe;--ifm-color-secondary-contrast-foreground:#474748;--ifm-color-success-dark:#009400;--ifm-color-success-darker:#008b00;--ifm-color-success-darkest:#007300;--ifm-color-success-light:#26b226;--ifm-color-success-lighter:#4dbf4d;--ifm-color-success-lightest:#80d280;--ifm-color-success-contrast-background:#e6f6e6;--ifm-color-success-contrast-foreground:#003100;--ifm-color-info-dark:#4cb3d4;--ifm-color-info-darker:#47a9c9;--ifm-color-info-darkest:#3b8ba5;--ifm-color-info-light:#6ecfef;--ifm-color-info-lighter:#87d8f2;--ifm-color-info-lightest:#aae3f6;--ifm-color-info-contrast-background:#eef9fd;--ifm-color-info-contrast-foreground:#193c47;--ifm-color-warning-dark:#e6a700;--ifm-color-warning-darker:#d99e00;--ifm-color-warning-darkest:#b38200;--ifm-color-warning-light:#ffc426;--ifm-color-warning-lighter:#ffcf4d;--ifm-color-warning-lightest:#ffdd80;--ifm-color-warning-contrast-background:#fff8e6;--ifm-color-warning-contrast-foreground:#4d3800;--ifm-color-danger-dark:#e13238;--ifm-color-danger-darker:#d53035;--ifm-color-danger-darkest:#af272b;--ifm-color-danger-light:#fb565b;--ifm-color-danger-lighter:#fb7478;--ifm-color-danger-lightest:#fd9c9f;--ifm-color-danger-contrast-background:#ffebec;--ifm-color-danger-contrast-foreground:#4b1113;--ifm-color-gray-0:var(--ifm-color-white);--ifm-color-gray-100:#f5f6f7;--ifm-color-gray-200:#ebedf0;--ifm-color-gray-300:#dadde1;--ifm-color-gray-400:#ccd0d5;--ifm-color-gray-500:#bec3c9;--ifm-color-gray-600:#8d949e;--ifm-color-gray-700:#606770;--ifm-color-gray-800:#444950;--ifm-color-gray-900:#1c1e21;--ifm-color-emphasis-0:var(--ifm-color-gray-0);--ifm-color-emphasis-100:var(--ifm-color-gray-100);--ifm-color-emphasis-200:var(--ifm-color-gray-200);--ifm-color-emphasis-300:var(--ifm-color-gray-300);--ifm-color-emphasis-400:var(--ifm-color-gray-400);--ifm-color-emphasis-600:var(--ifm-color-gray-600);--ifm-color-emphasis-700:var(--ifm-color-gray-700);--ifm-color-emphasis-800:var(--ifm-color-gray-800);--ifm-color-emphasis-900:var(--ifm-color-gray-900);--ifm-color-emphasis-1000:var(--ifm-color-gray-1000);--ifm-color-content:var(--ifm-color-emphasis-900);--ifm-color-content-inverse:var(--ifm-color-emphasis-0);--ifm-color-content-secondary:#525860;--ifm-background-color:#0000;--ifm-background-surface-color:var(--ifm-color-content-inverse);--ifm-global-radius:0.4rem;--ifm-font-color-base:var(--ifm-color-content);--ifm-font-color-base-inverse:var(--ifm-color-content-inverse);--ifm-font-color-secondary:var(--ifm-color-content-secondary);--ifm-font-family-base:system-ui,-apple-system,Segoe UI,Roboto,Ubuntu,Cantarell,Noto Sans,sans-serif,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";--ifm-font-weight-light:300;--ifm-font-weight-semibold:500;--ifm-font-weight-bold:700;--ifm-line-height-base:1.65;--ifm-global-spacing:1rem;--ifm-spacing-horizontal:var(--ifm-global-spacing);--ifm-transition-fast:200ms;--ifm-transition-slow:400ms;--ifm-transition-timing-default:cubic-bezier(0.08,0.52,0.52,1);--ifm-global-shadow-lw:0 1px 2px 0 #0000001a;--ifm-global-shadow-md:0 5px 40px #0003;--ifm-global-shadow-tl:0 12px 28px 0 #0003,0 2px 4px 0 #0000001a;--ifm-z-index-dropdown:100;--ifm-z-index-fixed:200;--ifm-z-index-overlay:400;--ifm-container-width:1140px;--ifm-container-width-xl:1320px;--ifm-code-background:#f6f7f8;--ifm-code-border-radius:var(--ifm-global-radius);--ifm-code-font-size:90%;--ifm-code-padding-horizontal:0.1rem;--ifm-code-padding-vertical:0.1rem;--ifm-pre-background:var(--ifm-code-background);--ifm-pre-border-radius:var(--ifm-code-border-radius);--ifm-pre-color:inherit;--ifm-pre-line-height:1.45;--ifm-pre-padding:1rem;--ifm-heading-color:inherit;--ifm-heading-margin-top:0;--ifm-heading-margin-bottom:var(--ifm-spacing-vertical);--ifm-heading-font-family:var(--ifm-font-family-base);--ifm-heading-font-weight:var(--ifm-font-weight-bold);--ifm-heading-line-height:1.25;--ifm-h1-font-size:2rem;--ifm-h2-font-size:1.5rem;--ifm-h3-font-size:1.25rem;--ifm-h4-font-size:1rem;--ifm-h5-font-size:0.875rem;--ifm-h6-font-size:0.85rem;--ifm-leading:calc(var(--ifm-leading-desktop)*1rem);--ifm-table-cell-padding:0.75rem;--ifm-table-background:#0000;--ifm-table-stripe-background:#00000008;--ifm-table-border-width:1px;--ifm-table-border-color:var(--ifm-color-emphasis-300);--ifm-table-head-background:inherit;--ifm-table-head-color:inherit;--ifm-table-head-font-weight:var(--ifm-font-weight-bold);--ifm-table-cell-color:inherit;--ifm-link-color:var(--ifm-color-primary);--ifm-link-hover-color:var(--ifm-link-color);--ifm-link-hover-decoration:underline;--ifm-blockquote-font-size:var(--ifm-font-size-base);--ifm-blockquote-border-left-width:2px;--ifm-blockquote-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-blockquote-padding-vertical:0;--ifm-blockquote-color:var(--ifm-color-emphasis-800);--ifm-blockquote-border-color:var(--ifm-color-emphasis-300);--ifm-hr-background-color:var(--ifm-color-emphasis-500);--ifm-scrollbar-track-background-color:#f1f1f1;--ifm-scrollbar-thumb-background-color:silver;--ifm-scrollbar-thumb-hover-background-color:#a7a7a7;--ifm-alert-background-color:inherit;--ifm-alert-border-color:inherit;--ifm-alert-border-radius:var(--ifm-global-radius);--ifm-alert-color:var(--ifm-font-color-base);--ifm-badge-background-color:inherit;--ifm-badge-border-color:inherit;--ifm-badge-color:var(--ifm-color-white);--ifm-breadcrumb-color-active:var(--ifm-color-primary);--ifm-breadcrumb-item-background-active:var(--ifm-hover-overlay);--ifm-button-background-color:inherit;--ifm-button-border-color:var(--ifm-button-background-color);--ifm-button-border-width:var(--ifm-global-border-width);--ifm-button-font-weight:var(--ifm-font-weight-bold);--ifm-button-padding-horizontal:1.5rem;--ifm-button-padding-vertical:0.375rem;--ifm-card-background-color:var(--ifm-background-surface-color);--ifm-card-border-radius:calc(var(--ifm-global-radius)*2);--ifm-card-horizontal-spacing:var(--ifm-global-spacing);--ifm-card-vertical-spacing:var(--ifm-global-spacing);--ifm-toc-border-color:var(--ifm-color-emphasis-300);--ifm-toc-link-color:var(--ifm-color-content-secondary);--ifm-toc-padding-vertical:0.5rem;--ifm-toc-padding-horizontal:0.5rem;--ifm-dropdown-background-color:var(--ifm-background-surface-color);--ifm-dropdown-font-weight:var(--ifm-font-weight-semibold);--ifm-dropdown-link-color:var(--ifm-font-color-base);--ifm-dropdown-hover-background-color:var(--ifm-hover-overlay);--ifm-footer-background-color:var(--ifm-color-emphasis-100);--ifm-footer-color:inherit;--ifm-footer-link-color:var(--ifm-color-emphasis-700);--ifm-footer-link-hover-color:var(--ifm-color-primary);--ifm-footer-link-horizontal-spacing:0.5rem;--ifm-footer-padding-horizontal:calc(var(--ifm-spacing-horizontal)*2);--ifm-footer-padding-vertical:calc(var(--ifm-spacing-vertical)*2);--ifm-footer-title-color:inherit;--ifm-footer-logo-max-width:min(30rem,90vw);--ifm-hero-background-color:var(--ifm-background-surface-color);--ifm-hero-text-color:var(--ifm-color-emphasis-800);--ifm-menu-color:var(--ifm-color-emphasis-700);--ifm-menu-color-active:var(--ifm-color-primary);--ifm-menu-color-background-active:var(--ifm-hover-overlay);--ifm-menu-color-background-hover:var(--ifm-hover-overlay);--ifm-menu-link-padding-horizontal:0.75rem;--ifm-menu-link-padding-vertical:0.375rem;--ifm-navbar-background-color:var(--ifm-background-surface-color);--ifm-navbar-height:3.75rem;--ifm-navbar-link-color:var(--ifm-font-color-base);--ifm-navbar-link-active-color:var(--ifm-link-color);--ifm-navbar-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-navbar-padding-vertical:calc(var(--ifm-spacing-vertical)*0.5);--ifm-navbar-search-input-background-color:var(--ifm-color-emphasis-200);--ifm-navbar-search-input-color:var(--ifm-color-emphasis-800);--ifm-navbar-search-input-placeholder-color:var(--ifm-color-emphasis-500);--ifm-navbar-sidebar-width:83vw;--ifm-pagination-color-active:var(--ifm-color-primary);--ifm-pagination-nav-color-hover:var(--ifm-color-primary);--ifm-pills-color-active:var(--ifm-color-primary);--ifm-pills-color-background-active:var(--ifm-hover-overlay);--ifm-tabs-color:var(--ifm-font-color-secondary);--ifm-tabs-color-active:var(--ifm-color-primary);--ifm-tabs-color-active-border:var(--ifm-tabs-color-active)}.badge--danger,.badge--info,.badge--primary,.badge--secondary,.badge--success,.badge--warning{--ifm-badge-border-color:var(--ifm-badge-background-color)}.button--link,.button--outline{--ifm-button-background-color:#0000}html{background-color:var(--ifm-background-color);color:var(--ifm-font-color-base);color-scheme:var(--ifm-color-scheme);font:var(--ifm-font-size-base)/var(--ifm-line-height-base) var(--ifm-font-family-base);-webkit-font-smoothing:antialiased;text-rendering:optimizelegibility;-webkit-text-size-adjust:100%;text-size-adjust:100%}body{background-color:rgb(var(--lsd-surface-primary));height:auto!important}iframe{border:0;color-scheme:auto}.container{margin:0 auto;max-width:var(--ifm-container-width)}.container--fluid{max-width:inherit}.row{display:flex;flex-wrap:wrap;margin:0 calc(var(--ifm-spacing-horizontal)*-1)}.row--no-gutters{margin-left:0;margin-right:0}.margin-horiz--none,.margin-right--none{margin-right:0!important}.row--no-gutters>.col{padding-left:0;padding-right:0}.row--align-top{align-items:flex-start}.row--align-bottom{align-items:flex-end}.row--align-center{align-items:center}.row--align-stretch{align-items:stretch}.row--align-baseline{align-items:baseline}.col{--ifm-col-width:100%;flex:1 0;margin-left:0;max-width:var(--ifm-col-width)}.padding-bottom--none,.padding-vert--none{padding-bottom:0!important}.padding-top--none,.padding-vert--none{padding-top:0!important}.padding-horiz--none,.padding-left--none{padding-left:0!important}.padding-horiz--none,.padding-right--none{padding-right:0!important}.col[class*=col--]{flex:0 0 var(--ifm-col-width)}.col--1{--ifm-col-width:8.33333%}.col--offset-1{margin-left:8.33333%}.col--2{--ifm-col-width:16.66667%}.col--offset-2{margin-left:16.66667%}.col--3{--ifm-col-width:25%}.col--offset-3{margin-left:25%}.col--4{--ifm-col-width:33.33333%}.col--offset-4{margin-left:33.33333%}.col--5{--ifm-col-width:41.66667%}.col--offset-5{margin-left:41.66667%}.col--6{--ifm-col-width:50%}.col--offset-6{margin-left:50%}.col--7{--ifm-col-width:58.33333%}.col--offset-7{margin-left:58.33333%}.col--8{--ifm-col-width:66.66667%}.col--offset-8{margin-left:66.66667%}.col--9{--ifm-col-width:75%}.col--offset-9{margin-left:75%}.col--10{--ifm-col-width:83.33333%}.col--offset-10{margin-left:83.33333%}.col--11{--ifm-col-width:91.66667%}.col--offset-11{margin-left:91.66667%}.col--12{--ifm-col-width:100%}.col--offset-12{margin-left:100%}.blog-wrapper main,.margin-horiz--none,.margin-left--none{margin-left:0!important}.margin--none{margin:0!important}.blog-archive-page .main-wrapper main>section,.margin-top--none,.mdx-grid>.mdx-grid__content .mdx-news-card .mdx-news-card__title{margin-top:0!important}.margin-bottom--none,.markdown>:last-child{margin-bottom:0!important}.main-wrapper>.container,.margin-vert--none{margin-bottom:0!important;margin-top:0!important}.margin-bottom--xs,.margin-vert--xs{margin-bottom:.25rem!important}.margin-top--xs,.margin-vert--xs{margin-top:.25rem!important}.margin-horiz--xs,.margin-left--xs{margin-left:.25rem!important}.margin-horiz--xs,.margin-right--xs{margin-right:.25rem!important}.margin--xs{margin:.25rem!important}.margin-bottom--sm,.margin-vert--sm{margin-bottom:.5rem!important}.margin-top--sm,.margin-vert--sm{margin-top:.5rem!important}.margin-horiz--sm,.margin-left--sm{margin-left:.5rem!important}.margin-horiz--sm,.margin-right--sm{margin-right:.5rem!important}.margin--sm{margin:.5rem!important}.margin-bottom--md,.margin-vert--md{margin-bottom:1rem!important}.margin-top--md,.margin-vert--md{margin-top:1rem!important}.margin-horiz--md,.margin-left--md{margin-left:1rem!important}.margin-horiz--md,.margin-right--md{margin-right:1rem!important}.margin--md{margin:1rem!important}.margin-bottom--lg,.margin-vert--lg{margin-bottom:2rem!important}.margin-top--lg,.margin-vert--lg{margin-top:2rem!important}.margin-horiz--lg,.margin-left--lg{margin-left:2rem!important}.margin-horiz--lg,.margin-right--lg{margin-right:2rem!important}.margin--lg{margin:2rem!important}.margin-bottom--xl,.margin-vert--xl{margin-bottom:5rem!important}.margin-top--xl,.margin-vert--xl{margin-top:5rem!important}.margin-horiz--xl,.margin-left--xl{margin-left:5rem!important}.margin-horiz--xl,.margin-right--xl{margin-right:5rem!important}.margin--xl{margin:5rem!important}.padding--none{padding:0!important}.padding-bottom--xs,.padding-vert--xs{padding-bottom:.25rem!important}.padding-top--xs,.padding-vert--xs{padding-top:.25rem!important}.padding-horiz--xs,.padding-left--xs{padding-left:.25rem!important}.padding-horiz--xs,.padding-right--xs{padding-right:.25rem!important}.padding--xs{padding:.25rem!important}.padding-bottom--sm,.padding-vert--sm{padding-bottom:.5rem!important}.padding-top--sm,.padding-vert--sm{padding-top:.5rem!important}.padding-horiz--sm,.padding-left--sm{padding-left:.5rem!important}.padding-horiz--sm,.padding-right--sm{padding-right:.5rem!important}.padding--sm{padding:.5rem!important}.padding-bottom--md,.padding-vert--md{padding-bottom:1rem!important}.padding-top--md,.padding-vert--md{padding-top:1rem!important}.padding-horiz--md,.padding-left--md{padding-left:1rem!important}.padding-horiz--md,.padding-right--md{padding-right:1rem!important}.padding--md{padding:1rem!important}.padding-bottom--lg,.padding-vert--lg{padding-bottom:2rem!important}.padding-top--lg,.padding-vert--lg{padding-top:2rem!important}.padding-horiz--lg,.padding-left--lg{padding-left:2rem!important}.padding-horiz--lg,.padding-right--lg{padding-right:2rem!important}.padding--lg{padding:2rem!important}.padding-bottom--xl,.padding-vert--xl{padding-bottom:5rem!important}.padding-top--xl,.padding-vert--xl{padding-top:5rem!important}.padding-horiz--xl,.padding-left--xl{padding-left:5rem!important}.padding-horiz--xl,.padding-right--xl{padding-right:5rem!important}.padding--xl{padding:5rem!important}code{background-color:var(--ifm-code-background);border:.1rem solid #0000001a;border-radius:var(--ifm-code-border-radius);font-family:var(--ifm-font-family-monospace);font-size:var(--ifm-code-font-size);padding:var(--ifm-code-padding-vertical) var(--ifm-code-padding-horizontal)}a code{color:inherit;color:rgb(var(--lsd-text-primary))}pre{background-color:var(--ifm-pre-background);border-radius:var(--ifm-pre-border-radius);color:var(--ifm-pre-color);font:var(--ifm-code-font-size)/var(--ifm-pre-line-height) var(--ifm-font-family-monospace);overflow:auto;padding:var(--ifm-pre-padding)}pre code{background-color:initial;border:none;font-size:100%;line-height:inherit;padding:0}kbd{background-color:var(--ifm-color-emphasis-0);border:1px solid var(--ifm-color-emphasis-400);border-radius:.2rem;box-shadow:inset 0 -1px 0 var(--ifm-color-emphasis-400);color:var(--ifm-color-emphasis-800);font:80% var(--ifm-font-family-monospace);padding:.15rem .3rem}h1,h2,h3,h4,h5,h6{color:var(--ifm-heading-color);font-family:var(--ifm-heading-font-family);font-weight:var(--ifm-heading-font-weight);line-height:var(--ifm-heading-line-height);margin:var(--ifm-heading-margin-top) 0 var(--ifm-heading-margin-bottom) 0}.sub1,h1{font-size:var(--ifm-h1-font-size)}.sub2,h2{font-size:var(--ifm-h2-font-size)}.sub3,h3{font-size:var(--ifm-h3-font-size)}.sub4,h4{font-size:var(--ifm-h4-font-size)}.sub5,h5{font-size:var(--ifm-h5-font-size)}.sub6,h6,small{font-size:var(--ifm-h6-font-size)}.container_lyt7,.container_lyt7>svg,img{max-width:100%}img[align=right]{padding-left:var(--image-alignment-padding)}img[align=left]{padding-right:var(--image-alignment-padding)}.markdown{--ifm-h1-vertical-rhythm-top:3;--ifm-h2-vertical-rhythm-top:2;--ifm-h3-vertical-rhythm-top:1.5;--ifm-heading-vertical-rhythm-top:1.25;--ifm-h1-vertical-rhythm-bottom:1.25;--ifm-heading-vertical-rhythm-bottom:1}.markdown:after,.markdown:before{content:"";display:table}.markdown:after{clear:both}.markdown h1:first-child{--ifm-h1-font-size:3rem;margin-bottom:calc(var(--ifm-h1-vertical-rhythm-bottom)*var(--ifm-leading))}.markdown>h2{--ifm-h2-font-size:2rem;margin-top:calc(var(--ifm-h2-vertical-rhythm-top)*var(--ifm-leading));letter-spacing:-1.5px}.markdown>h3{--ifm-h3-font-size:1.5rem;margin-top:calc(var(--ifm-h3-vertical-rhythm-top)*var(--ifm-leading))}.markdown>h4,.markdown>h5,.markdown>h6{margin-top:calc(var(--ifm-heading-vertical-rhythm-top)*var(--ifm-leading))}.markdown>p,.markdown>pre,.markdown>ul{margin-bottom:var(--ifm-leading)}.markdown li>p{margin-top:var(--ifm-list-paragraph-margin)}.markdown li+li{margin-top:var(--ifm-list-item-margin)}ol,ul{margin:0 0 var(--ifm-list-margin);padding-left:var(--ifm-list-left-padding)}ol ol,ul ol{list-style-type:lower-roman}ol ol ol,ol ul ol,ul ol ol,ul ul ol{list-style-type:lower-alpha}table thead tr{border-bottom:2px solid var(--ifm-table-border-color)}table thead,table tr:nth-child(2n){background-color:var(--ifm-table-stripe-background)}table tr{background-color:var(--ifm-table-background);border-top:var(--ifm-table-border-width) solid var(--ifm-table-border-color)}table td,table th{border:var(--ifm-table-border-width) solid var(--ifm-table-border-color);padding:var(--ifm-table-cell-padding)}table th{background-color:var(--ifm-table-head-background);color:var(--ifm-table-head-color);font-weight:var(--ifm-table-head-font-weight)}strong{font-weight:var(--ifm-font-weight-bold)}a{color:var(--ifm-link-color);text-decoration:var(--ifm-link-decoration)}a:hover{color:var(--ifm-link-hover-color);text-decoration:var(--ifm-link-hover-decoration)}.button:hover,.text--no-decoration,.text--no-decoration:hover,a:not([href]){-webkit-text-decoration:none;text-decoration:none}p{margin:0 0 var(--ifm-paragraph-margin-bottom)}blockquote{border-left:var(--ifm-blockquote-border-left-width) solid var(--ifm-blockquote-border-color);box-shadow:var(--ifm-blockquote-shadow);color:var(--ifm-blockquote-color);font-size:var(--ifm-blockquote-font-size);padding:var(--ifm-blockquote-padding-vertical) var(--ifm-blockquote-padding-horizontal)}blockquote>:first-child{margin-top:0}blockquote>:last-child{margin-bottom:0}hr{background-color:var(--ifm-hr-background-color);border:0;height:var(--ifm-hr-height);margin:var(--ifm-hr-margin-vertical) 0}.shadow--lw{box-shadow:var(--ifm-global-shadow-lw)!important}.shadow--md{box-shadow:var(--ifm-global-shadow-md)!important}.shadow--tl{box-shadow:var(--ifm-global-shadow-tl)!important}.text--primary,.wordWrapButtonEnabled_EoeP .wordWrapButtonIcon_Bwma{color:var(--ifm-color-primary)}.text--secondary,[data-theme=dark] .root_JWD1>div>div{color:var(--ifm-color-secondary)}.text--success{color:var(--ifm-color-success)}.text--info{color:var(--ifm-color-info)}.text--warning{color:var(--ifm-color-warning)}.text--danger{color:var(--ifm-color-danger)}.text--center{text-align:center}.text--left{text-align:left}.text--justify{text-align:justify}.text--right{text-align:right}.text--lowercase{text-transform:lowercase}.alert__heading,.mdx-hero-title--uppercase,.text--uppercase,table td strong,table th{text-transform:uppercase}.text--light{font-weight:var(--ifm-font-weight-light)}.text--normal{font-weight:var(--ifm-font-weight-normal)}.text--semibold{font-weight:var(--ifm-font-weight-semibold)}.text--bold{font-weight:var(--ifm-font-weight-bold)}.text--italic{font-style:italic}.text--truncate{overflow:hidden;text-overflow:ellipsis}.text--break{word-wrap:break-word!important;word-break:break-word!important}.clean-btn{background:none;border:none;color:inherit;cursor:pointer;font-family:inherit;padding:0}.alert,.alert .close{color:var(--ifm-alert-foreground-color)}.clean-list{padding-left:0}.alert--primary{--ifm-alert-background-color:var(--ifm-color-primary-contrast-background);--ifm-alert-background-color-highlight:#3578e526;--ifm-alert-foreground-color:var(--ifm-color-primary-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-primary-dark)}.alert--secondary{--ifm-alert-background-color:var(--ifm-color-secondary-contrast-background);--ifm-alert-background-color-highlight:#ebedf026;--ifm-alert-foreground-color:var(--ifm-color-secondary-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-secondary-dark)}.alert--success{--ifm-alert-background-color:var(--ifm-color-success-contrast-background);--ifm-alert-background-color-highlight:#00a40026;--ifm-alert-foreground-color:var(--ifm-color-success-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-success-dark)}.alert--info{--ifm-alert-background-color:var(--ifm-color-info-contrast-background);--ifm-alert-background-color-highlight:#54c7ec26;--ifm-alert-foreground-color:var(--ifm-color-info-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-info-dark)}.alert--warning{--ifm-alert-background-color:var(--ifm-color-warning-contrast-background);--ifm-alert-background-color-highlight:#ffba0026;--ifm-alert-foreground-color:var(--ifm-color-warning-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-warning-dark)}.alert--danger{--ifm-alert-background-color:var(--ifm-color-danger-contrast-background);--ifm-alert-background-color-highlight:#fa383e26;--ifm-alert-foreground-color:var(--ifm-color-danger-contrast-foreground);--ifm-alert-border-color:var(--ifm-color-danger-dark)}.alert{--ifm-code-background:var(--ifm-alert-background-color-highlight);--ifm-link-color:var(--ifm-alert-foreground-color);--ifm-link-hover-color:var(--ifm-alert-foreground-color);--ifm-link-decoration:underline;--ifm-tabs-color:var(--ifm-alert-foreground-color);--ifm-tabs-color-active:var(--ifm-alert-foreground-color);--ifm-tabs-color-active-border:var(--ifm-alert-border-color);background-color:var(--ifm-alert-background-color);border:var(--ifm-alert-border-width) solid var(--ifm-alert-border-color);border-left-width:var(--ifm-alert-border-left-width);border-radius:var(--ifm-alert-border-radius);box-shadow:var(--ifm-alert-shadow);padding:var(--ifm-alert-padding-vertical) var(--ifm-alert-padding-horizontal)}.alert__heading{align-items:center;display:flex;font:700 var(--ifm-h5-font-size)/var(--ifm-heading-line-height) var(--ifm-heading-font-family);margin-bottom:.5rem}.alert__icon{display:inline-flex;margin-right:.4em}.alert__icon svg{fill:var(--ifm-alert-foreground-color);stroke:var(--ifm-alert-foreground-color);stroke-width:0}.alert .close{margin:calc(var(--ifm-alert-padding-vertical)*-1) calc(var(--ifm-alert-padding-horizontal)*-1) 0 0;opacity:.75}.alert .close:focus,.alert .close:hover{opacity:1}.alert a{text-decoration-color:var(--ifm-alert-border-color)}.alert a:hover{text-decoration-thickness:2px}.avatar{column-gap:var(--ifm-avatar-intro-margin);display:flex}.avatar__photo{border-radius:50%;display:block;height:var(--ifm-avatar-photo-size);overflow:hidden;width:var(--ifm-avatar-photo-size)}.card--full-height,.navbar__logo img{height:100%}.avatar__photo--sm{--ifm-avatar-photo-size:2rem}.avatar__photo--lg{--ifm-avatar-photo-size:4rem}.avatar__photo--xl{--ifm-avatar-photo-size:6rem}.avatar__intro{display:flex;flex:1 1;flex-direction:column;justify-content:center;text-align:var(--ifm-avatar-intro-alignment)}.badge,.breadcrumbs__item,.breadcrumbs__link,.button{display:inline-block}.button,.mdx-cta-section .mdx-cta-section--align-center *{text-align:center}.avatar__name{font:700 var(--ifm-h4-font-size)/var(--ifm-heading-line-height) var(--ifm-font-family-base)}.avatar__subtitle{margin-top:.25rem}.avatar--vertical{--ifm-avatar-intro-alignment:center;--ifm-avatar-intro-margin:0.5rem;align-items:center;flex-direction:column}.badge{background-color:var(--ifm-badge-background-color);border:var(--ifm-badge-border-width) solid var(--ifm-badge-border-color);border-radius:var(--ifm-badge-border-radius);color:var(--ifm-badge-color);font-size:75%;font-weight:var(--ifm-font-weight-bold);line-height:1;padding:var(--ifm-badge-padding-vertical) var(--ifm-badge-padding-horizontal)}.badge--primary{--ifm-badge-background-color:var(--ifm-color-primary)}.badge--secondary{--ifm-badge-background-color:var(--ifm-color-secondary);color:var(--ifm-color-black)}.breadcrumbs__link,.button.button--secondary.button--outline:not(.button--active):not(:hover){color:var(--ifm-font-color-base)}.badge--success{--ifm-badge-background-color:var(--ifm-color-success)}.badge--info{--ifm-badge-background-color:var(--ifm-color-info)}.badge--warning{--ifm-badge-background-color:var(--ifm-color-warning)}.badge--danger{--ifm-badge-background-color:var(--ifm-color-danger)}.breadcrumbs{margin-bottom:0;padding-left:0}.breadcrumbs__item:not(:last-child):after{background:var(--ifm-breadcrumb-separator) center;content:" ";display:inline-block;filter:var(--ifm-breadcrumb-separator-filter);height:calc(var(--ifm-breadcrumb-separator-size)*var(--ifm-breadcrumb-size-multiplier)*var(--ifm-breadcrumb-separator-size-multiplier));margin:0 var(--ifm-breadcrumb-spacing);opacity:.5;width:calc(var(--ifm-breadcrumb-separator-size)*var(--ifm-breadcrumb-size-multiplier)*var(--ifm-breadcrumb-separator-size-multiplier))}.breadcrumbs__item--active .breadcrumbs__link{background:var(--ifm-breadcrumb-item-background-active);color:var(--ifm-breadcrumb-color-active)}.breadcrumbs__link{border-radius:var(--ifm-breadcrumb-border-radius);font-size:calc(1rem*var(--ifm-breadcrumb-size-multiplier));padding:calc(var(--ifm-breadcrumb-padding-vertical)*var(--ifm-breadcrumb-size-multiplier)) calc(var(--ifm-breadcrumb-padding-horizontal)*var(--ifm-breadcrumb-size-multiplier));transition-duration:var(--ifm-transition-fast);transition-property:background,color}.breadcrumbs__link:any-link:hover,.breadcrumbs__link:link:hover,.breadcrumbs__link:visited:hover,area[href].breadcrumbs__link:hover{background:var(--ifm-breadcrumb-item-background-active);-webkit-text-decoration:none;text-decoration:none}.breadcrumbs--sm{--ifm-breadcrumb-size-multiplier:0.8}.breadcrumbs--lg{--ifm-breadcrumb-size-multiplier:1.2}.button{background-color:var(--ifm-button-background-color);border:var(--ifm-button-border-width) solid var(--ifm-button-border-color);border-radius:var(--ifm-button-border-radius);cursor:pointer;font-size:calc(.875rem*var(--ifm-button-size-multiplier));font-weight:var(--ifm-button-font-weight);line-height:1.5;padding:calc(var(--ifm-button-padding-vertical)*var(--ifm-button-size-multiplier)) calc(var(--ifm-button-padding-horizontal)*var(--ifm-button-size-multiplier));transition-duration:var(--ifm-button-transition-duration);transition-property:color,background,border-color;-webkit-user-select:none;user-select:none}.button,.button:hover{color:var(--ifm-button-color)}.button--outline{--ifm-button-color:var(--ifm-button-border-color)}.button--outline:hover{--ifm-button-background-color:var(--ifm-button-border-color)}.button--link{--ifm-button-border-color:#0000;color:var(--ifm-link-color);text-decoration:var(--ifm-link-decoration)}.button--link.button--active,.button--link:active,.button--link:hover{color:var(--ifm-link-hover-color);text-decoration:var(--ifm-link-hover-decoration)}.dropdown__link,.dropdown__link--active,.dropdown__link:hover,.menu__link:hover,.navbar__brand:hover,.navbar__link--active,.navbar__link:hover,.pagination-nav__link:hover,.pagination__link:hover{-webkit-text-decoration:none;text-decoration:none}.button.disabled,.button:disabled,.button[disabled]{opacity:.65;pointer-events:none}.button--sm{--ifm-button-size-multiplier:0.8}.button--lg{--ifm-button-size-multiplier:1.35}.button--block{display:block;width:100%}.button.button--secondary{color:var(--ifm-color-gray-900)}:where(.button--primary){--ifm-button-background-color:var(--ifm-color-primary);--ifm-button-border-color:var(--ifm-color-primary)}:where(.button--primary):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-primary-dark);--ifm-button-border-color:var(--ifm-color-primary-dark)}.button--primary.button--active,.button--primary:active{--ifm-button-background-color:var(--ifm-color-primary-darker);--ifm-button-border-color:var(--ifm-color-primary-darker)}:where(.button--secondary){--ifm-button-background-color:var(--ifm-color-secondary);--ifm-button-border-color:var(--ifm-color-secondary)}:where(.button--secondary):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-secondary-dark);--ifm-button-border-color:var(--ifm-color-secondary-dark)}.button--secondary.button--active,.button--secondary:active{--ifm-button-background-color:var(--ifm-color-secondary-darker);--ifm-button-border-color:var(--ifm-color-secondary-darker)}:where(.button--success){--ifm-button-background-color:var(--ifm-color-success);--ifm-button-border-color:var(--ifm-color-success)}:where(.button--success):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-success-dark);--ifm-button-border-color:var(--ifm-color-success-dark)}.button--success.button--active,.button--success:active{--ifm-button-background-color:var(--ifm-color-success-darker);--ifm-button-border-color:var(--ifm-color-success-darker)}:where(.button--info){--ifm-button-background-color:var(--ifm-color-info);--ifm-button-border-color:var(--ifm-color-info)}:where(.button--info):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-info-dark);--ifm-button-border-color:var(--ifm-color-info-dark)}.button--info.button--active,.button--info:active{--ifm-button-background-color:var(--ifm-color-info-darker);--ifm-button-border-color:var(--ifm-color-info-darker)}:where(.button--warning){--ifm-button-background-color:var(--ifm-color-warning);--ifm-button-border-color:var(--ifm-color-warning)}:where(.button--warning):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-warning-dark);--ifm-button-border-color:var(--ifm-color-warning-dark)}.button--warning.button--active,.button--warning:active{--ifm-button-background-color:var(--ifm-color-warning-darker);--ifm-button-border-color:var(--ifm-color-warning-darker)}:where(.button--danger){--ifm-button-background-color:var(--ifm-color-danger);--ifm-button-border-color:var(--ifm-color-danger)}:where(.button--danger):not(.button--outline):hover{--ifm-button-background-color:var(--ifm-color-danger-dark);--ifm-button-border-color:var(--ifm-color-danger-dark)}.button--danger.button--active,.button--danger:active{--ifm-button-background-color:var(--ifm-color-danger-darker);--ifm-button-border-color:var(--ifm-color-danger-darker)}.button-group{display:inline-flex;gap:var(--ifm-button-group-spacing)}.button-group>.button:not(:first-child){border-bottom-left-radius:0;border-top-left-radius:0}.button-group>.button:not(:last-child){border-bottom-right-radius:0;border-top-right-radius:0}.button-group--block{display:flex;justify-content:stretch}.button-group--block>.button{flex-grow:1}.card{background-color:var(--ifm-card-background-color);border-radius:var(--ifm-card-border-radius);box-shadow:var(--ifm-global-shadow-lw);display:flex;flex-direction:column;overflow:hidden}.card__image{padding-top:var(--ifm-card-vertical-spacing)}.card__image:first-child{padding-top:0}.card__body,.card__footer,.card__header{padding:var(--ifm-card-vertical-spacing) var(--ifm-card-horizontal-spacing)}.card__body:not(:last-child),.card__footer:not(:last-child),.card__header:not(:last-child){padding-bottom:0}.card__body>:last-child,.card__footer>:last-child,.card__header>:last-child{margin-bottom:0}.card__footer{margin-top:auto}.table-of-contents{font-size:.8rem;margin-bottom:0;padding:var(--ifm-toc-padding-vertical) 0}.table-of-contents,.table-of-contents ul{list-style:none;padding-left:var(--ifm-toc-padding-horizontal)}.table-of-contents li{margin:var(--ifm-toc-padding-vertical) var(--ifm-toc-padding-horizontal)}.table-of-contents__left-border{border-left:1px solid var(--ifm-toc-border-color)}.table-of-contents__link{color:var(--ifm-toc-link-color);display:block}.table-of-contents__link--active,.table-of-contents__link--active code,.table-of-contents__link:hover,.table-of-contents__link:hover code{color:var(--ifm-color-primary);-webkit-text-decoration:none;text-decoration:none}.close{color:var(--ifm-color-black);float:right;font-size:1.5rem;font-weight:var(--ifm-font-weight-bold);line-height:1;opacity:.5;padding:1rem;transition:opacity var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.breadcrumbs__item,.close:hover,.table-of-contents__link--active code{opacity:.7}.close:focus,.theme-code-block-highlighted-line .codeLineNumber_Tfdd:before{opacity:.8}.dropdown{display:inline-flex;font-weight:var(--ifm-dropdown-font-weight);position:relative;vertical-align:top}.admonitionIcon_BGV6,.root_JWD1>span svg{vertical-align:middle}.dropdown--hoverable:hover .dropdown__menu,.dropdown--show .dropdown__menu{opacity:1;pointer-events:all;transform:translateY(-1px);visibility:visible}#nprogress,.dropdown__menu,.mdx-hero-video,.navWrapper_ybYI,.navbar__item.dropdown .navbar__link:not([href]){pointer-events:none}.dropdown--right .dropdown__menu{left:inherit;right:0}.dropdown--nocaret .navbar__link:after{content:none!important}.dropdown__menu{background-color:var(--ifm-dropdown-background-color);border-radius:var(--ifm-global-radius);box-shadow:var(--ifm-global-shadow-md);left:0;max-height:80vh;min-width:10rem;opacity:0;overflow-y:auto;position:absolute;top:calc(100% - var(--ifm-navbar-item-padding-vertical) + .3rem);transform:translateY(-.625rem);transition-duration:var(--ifm-transition-fast);transition-property:opacity,transform,visibility;transition-timing-function:var(--ifm-transition-timing-default);visibility:hidden;z-index:var(--ifm-z-index-dropdown)}.menu__caret,.menu__link,.menu__list-item-collapsible{border-radius:.25rem;transition:background var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.dropdown__link{color:var(--ifm-dropdown-link-color);display:block;font-size:.875rem}.dropdown__link--active,.dropdown__link:hover{background-color:var(--ifm-dropdown-hover-background-color);color:var(--ifm-dropdown-link-color)}.dropdown__link--active,.dropdown__link--active:hover{--ifm-dropdown-link-color:var(--ifm-link-color)}.dropdown>.navbar__link:after{border-color:currentcolor #0000;border-style:solid;border-width:.4em .4em 0;content:"";margin-left:.3em;position:relative;top:2px;transform:translateY(-50%);display:none}.menu__list-item--collapsed .menu__caret:before,.menu__list-item--collapsed .menu__link--sublist:after,.menu__list-item-collapsible>a[aria-expanded=false]>svg{transform:rotate(90deg)}.footer{background-color:var(--ifm-footer-background-color);color:var(--ifm-footer-color);padding:var(--ifm-footer-padding-vertical) var(--ifm-footer-padding-horizontal)}.footer--dark{--ifm-footer-background-color:#303846;--ifm-footer-color:var(--ifm-footer-link-color);--ifm-footer-link-color:var(--ifm-color-secondary);--ifm-footer-title-color:var(--ifm-color-white)}.footer__links{margin-bottom:1rem}.footer__link-item{color:var(--ifm-footer-link-color);line-height:2}.footer__link-item:hover{color:var(--ifm-footer-link-hover-color)}.footer__link-separator{margin:0 var(--ifm-footer-link-horizontal-spacing)}.footer__title{color:var(--ifm-footer-title-color);font:700 var(--ifm-h4-font-size)/var(--ifm-heading-line-height) var(--ifm-font-family-base);margin-bottom:var(--ifm-heading-margin-bottom)}.menu,.navbar__link{font-weight:var(--ifm-font-weight-semibold)}.docItemContainer_hrrU article>:first-child,.docItemContainer_hrrU header+*,.footer__item,.mdx-roadmap .mdx-cta-button{margin-top:0}.admonitionContent_iozl>:last-child,.footer__items{margin-bottom:0}.main-wrapper>main,.table-of-contents,[type=checkbox]{padding:0}.hero{align-items:center;background-color:var(--ifm-hero-background-color);color:var(--ifm-hero-text-color);display:flex;padding:4rem 2rem}.hero--primary{--ifm-hero-background-color:var(--ifm-color-primary);--ifm-hero-text-color:var(--ifm-font-color-base-inverse)}.hero--dark{--ifm-hero-background-color:#303846;--ifm-hero-text-color:var(--ifm-color-white)}.hero__title,.title_f1Hy{font-size:3rem}.hero__subtitle{font-size:1.5rem}.menu__list{margin:0;padding-left:0}.menu__caret,.menu__link{padding:var(--ifm-menu-link-padding-vertical) var(--ifm-menu-link-padding-horizontal)}.menu__list .menu__list{flex:0 0 100%;margin-top:.25rem;padding-left:var(--ifm-menu-link-padding-horizontal)}.menu__list-item:not(:first-child){margin-top:.25rem}.menu__list-item--collapsed .menu__list{height:0;overflow:hidden}.menu__list-item-collapsible{display:flex;flex-wrap:wrap;position:relative}.menu__caret:hover,.menu__link:hover,.menu__list-item-collapsible--active,.menu__list-item-collapsible:hover{background:var(--ifm-menu-color-background-hover)}.menu__list-item-collapsible .menu__link--active,.menu__list-item-collapsible .menu__link:hover{background:none!important}.menu__caret,.menu__link{align-items:center;display:flex}.navbar-sidebar,.navbar-sidebar__backdrop{bottom:0;opacity:0;top:0;transition-duration:var(--ifm-transition-fast);transition-timing-function:ease-in-out;left:0;visibility:hidden}.menu__link{color:var(--ifm-menu-color);flex:1;line-height:1.25}.menu__link:hover{color:var(--ifm-menu-color)}.menu__caret:before,.menu__link--sublist-caret:after{content:"";filter:var(--ifm-menu-link-sublist-icon-filter);height:1.25rem;transform:rotate(180deg);transition:transform var(--ifm-transition-fast) linear;width:1.25rem}.menu__link--sublist-caret:after{background:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem;margin-left:auto;min-width:1.25rem}.menu__link--active,.menu__link--active:hover{color:var(--ifm-menu-color-active)}.navbar__brand,.navbar__link{color:var(--ifm-navbar-link-color)}.menu__link--active:not(.menu__link--sublist){background-color:var(--ifm-menu-color-background-active)}.menu__caret:before{background:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem}.navbar--dark,html[data-theme=dark]{--ifm-menu-link-sublist-icon-filter:invert(100%) sepia(94%) saturate(17%) hue-rotate(223deg) brightness(104%) contrast(98%)}.navbar{background-color:var(--ifm-navbar-background-color);box-shadow:var(--ifm-navbar-shadow);padding:var(--ifm-navbar-padding-vertical) var(--ifm-navbar-padding-horizontal)}.icon_S7Kx,.navbar,.navbar>.container,.navbar>.container-fluid{display:flex}.navbar--fixed-top{position:sticky;top:0;z-index:var(--ifm-z-index-fixed)}.navbar__inner{display:flex;flex-wrap:wrap;justify-content:space-between;width:100%}.navbar__brand{align-items:center;display:flex;margin-right:1rem;min-width:0}.navbar__brand:hover{color:var(--ifm-navbar-link-hover-color)}.announcementBarContent_xLdY,.navbar__title{flex:1 1 auto}.navbar__toggle{display:none;margin-right:.5rem}.navbar__logo{flex:0 0 auto;margin-right:.5rem}.navbar__items{align-items:center;display:flex;flex:1;min-width:0}.navbar__items--center{flex:0 0 auto}.blog-wrapper>.container>.row>aside>nav>ul li,.navbar__items--center .navbar__brand{margin:0}.navbar__items--center+.navbar__items--right{flex:1}.navbar__items--right{flex:0 0 auto;justify-content:flex-end}.navbar__items--right>:last-child{padding-right:0}.navbar__item{display:inline-block;padding:var(--ifm-navbar-item-padding-vertical) var(--ifm-navbar-item-padding-horizontal)}.navbar__link--active,.navbar__link:hover{color:var(--ifm-navbar-link-hover-color)}.navbar--dark,.navbar--primary{--ifm-menu-color:var(--ifm-color-gray-300);--ifm-navbar-link-color:var(--ifm-color-gray-100);--ifm-navbar-search-input-background-color:#ffffff1a;--ifm-navbar-search-input-placeholder-color:#ffffff80;color:var(--ifm-color-white)}.navbar--dark{--ifm-navbar-background-color:#242526;--ifm-menu-color-background-active:#ffffff0d;--ifm-navbar-search-input-color:var(--ifm-color-white)}.navbar--primary{--ifm-navbar-background-color:var(--ifm-color-primary);--ifm-navbar-link-hover-color:var(--ifm-color-white);--ifm-menu-color-active:var(--ifm-color-white);--ifm-navbar-search-input-color:var(--ifm-color-emphasis-500)}.navbar__search-input{appearance:none;background:var(--ifm-navbar-search-input-background-color) var(--ifm-navbar-search-input-icon) no-repeat .75rem center/1rem 1rem;border:none;border-radius:2rem;color:var(--ifm-navbar-search-input-color);cursor:text;display:inline-block;font-size:1rem;height:2rem;padding:0 .5rem 0 2.25rem;width:12.5rem}.cursor-pointer,.mdx-accordion-item__header,.mdx-accordion-item__header *,.mdx-cta-section__list-item-link,.mdx-video-section__list-item-link,.pills__item,.tabs__item{cursor:pointer}.navbar__search-input::placeholder{color:var(--ifm-navbar-search-input-placeholder-color)}.navbar-sidebar{background-color:var(--ifm-navbar-background-color);box-shadow:var(--ifm-global-shadow-md);position:fixed;transform:translate3d(-100%,0,0);transition-property:opacity,visibility,transform;width:var(--ifm-navbar-sidebar-width)}.navbar-sidebar--show .navbar-sidebar,.navbar-sidebar__items{transform:translateZ(0)}.navbar-sidebar--show .navbar-sidebar,.navbar-sidebar--show .navbar-sidebar__backdrop{opacity:1;visibility:visible}.navbar-sidebar__backdrop{background-color:#0009;position:fixed;right:0;transition-property:opacity,visibility}.footer,.row{position:relative}.navbar-sidebar__brand{align-items:center;box-shadow:var(--ifm-navbar-shadow);display:flex;flex:1;height:var(--ifm-navbar-height);padding:var(--ifm-navbar-padding-vertical) var(--ifm-navbar-padding-horizontal)}.navbar-sidebar__items{display:flex;height:calc(100% - var(--ifm-navbar-height));transition:transform var(--ifm-transition-fast) ease-in-out}.navbar-sidebar__items--show-secondary{transform:translate3d(calc((var(--ifm-navbar-sidebar-width))*-1),0,0)}.navbar-sidebar__item{flex-shrink:0;padding:.5rem;width:calc(var(--ifm-navbar-sidebar-width))}.navbar-sidebar__back{background:var(--ifm-menu-color-background-active);font-size:15px;font-weight:var(--ifm-button-font-weight);position:relative;text-align:left;top:-.5rem}.navbar-sidebar__close{display:flex;margin-left:auto}.pagination{column-gap:var(--ifm-pagination-page-spacing);display:flex;font-size:var(--ifm-pagination-font-size);padding-left:0}.pagination--sm{--ifm-pagination-font-size:0.8rem;--ifm-pagination-padding-horizontal:0.8rem;--ifm-pagination-padding-vertical:0.2rem}.pagination--lg{--ifm-pagination-font-size:1.2rem;--ifm-pagination-padding-horizontal:1.2rem;--ifm-pagination-padding-vertical:0.3rem}.pagination__item{display:inline-flex}.pagination__item>span{padding:var(--ifm-pagination-padding-vertical)}.pagination__item--active .pagination__link{color:var(--ifm-pagination-color-active)}.pagination__item--active .pagination__link,.pagination__item:not(.pagination__item--active):hover .pagination__link{background:var(--ifm-pagination-item-active-background)}.pagination__item--disabled,.pagination__item[disabled]{opacity:.25;pointer-events:none}.pagination__link{border-radius:var(--ifm-pagination-border-radius);color:var(--ifm-font-color-base);display:inline-block;padding:var(--ifm-pagination-padding-vertical) var(--ifm-pagination-padding-horizontal);transition:background var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.pagination-nav{display:grid;grid-gap:var(--ifm-spacing-horizontal);gap:var(--ifm-spacing-horizontal);grid-template-columns:repeat(2,1fr)}.footer>div,.grid{grid-template-columns:repeat(24,1fr)}.pagination-nav__link{border:1px solid var(--ifm-color-emphasis-300);border-radius:var(--ifm-pagination-nav-border-radius);display:block;height:100%;line-height:var(--ifm-heading-line-height);padding:var(--ifm-global-spacing);transition:border-color var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.pagination-nav__link:hover{border-color:var(--ifm-pagination-nav-color-hover)}.blog-wrapper>.container>.row>aside>nav>div:first-child,.blog-wrapper>.container>.row>aside>nav>ul li a{border-left:1px solid #0000;font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important}.blog-wrapper main a:hover,.dropdown__link:hover,.footer__item a,.markdown a,.markdown a:hover,.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>a:hover,article a,article a:hover{-webkit-text-decoration:underline;text-decoration:underline}.pagination-nav__label{font-size:var(--ifm-h4-font-size);font-weight:var(--ifm-heading-font-weight);word-break:break-word}.pagination-nav__link--prev .pagination-nav__label:before{content:"« "}.pagination-nav__link--next .pagination-nav__label:after{content:" »"}.pagination-nav__sublabel{color:var(--ifm-color-content-secondary);font-size:var(--ifm-h5-font-size);font-weight:var(--ifm-font-weight-semibold);margin-bottom:.25rem}.pills__item,.tabs{font-weight:var(--ifm-font-weight-bold)}.pills{display:flex;gap:var(--ifm-pills-spacing);padding-left:0}.pills__item{border-radius:.5rem;display:inline-block;padding:.25rem 1rem;transition:background var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.navbar__left-items>.navbar__item:first-of-type,.tabs,:not(.containsTaskList_mC6p>li)>.containsTaskList_mC6p{padding-left:0}.pills__item--active{color:var(--ifm-pills-color-active)}.pills__item--active,.pills__item:not(.pills__item--active):hover{background:var(--ifm-pills-color-background-active)}.pills--block{justify-content:stretch}.pills--block .pills__item{flex-grow:1;text-align:center}.tabs{color:var(--ifm-tabs-color);display:flex;margin-bottom:0;overflow-x:auto}.tabs__item{border-bottom:3px solid #0000;border-radius:var(--ifm-global-radius);display:inline-flex;padding:var(--ifm-tabs-padding-vertical) var(--ifm-tabs-padding-horizontal);transition:background-color var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.tabs__item--active{border-bottom-color:var(--ifm-tabs-color-active-border);border-bottom-left-radius:0;border-bottom-right-radius:0;color:var(--ifm-tabs-color-active)}.tabs__item:hover{background-color:var(--ifm-hover-overlay)}.tabs--block{justify-content:stretch}.tabs--block .tabs__item{flex-grow:1;justify-content:center}html[data-theme=dark]{--ifm-color-emphasis-0:var(--ifm-color-gray-1000);--ifm-color-emphasis-100:var(--ifm-color-gray-900);--ifm-color-emphasis-200:var(--ifm-color-gray-800);--ifm-color-emphasis-300:var(--ifm-color-gray-700);--ifm-color-emphasis-400:var(--ifm-color-gray-600);--ifm-color-emphasis-600:var(--ifm-color-gray-400);--ifm-color-emphasis-700:var(--ifm-color-gray-300);--ifm-color-emphasis-800:var(--ifm-color-gray-200);--ifm-color-emphasis-900:var(--ifm-color-gray-100);--ifm-color-emphasis-1000:var(--ifm-color-gray-0);--ifm-background-color:#1b1b1d;--ifm-background-surface-color:#242526;--ifm-hover-overlay:#ffffff0d;--ifm-color-content:#e3e3e3;--ifm-color-content-secondary:#fff;--ifm-breadcrumb-separator-filter:invert(64%) sepia(11%) saturate(0%) hue-rotate(149deg) brightness(99%) contrast(95%);--ifm-code-background:#ffffff1a;--ifm-scrollbar-track-background-color:#444;--ifm-scrollbar-thumb-background-color:#686868;--ifm-scrollbar-thumb-hover-background-color:#7a7a7a;--ifm-table-stripe-background:#ffffff12;--ifm-toc-border-color:var(--ifm-color-emphasis-200);--ifm-color-primary-contrast-background:#102445;--ifm-color-primary-contrast-foreground:#ebf2fc;--ifm-color-secondary-contrast-background:#474748;--ifm-color-secondary-contrast-foreground:#fdfdfe;--ifm-color-success-contrast-background:#003100;--ifm-color-success-contrast-foreground:#e6f6e6;--ifm-color-info-contrast-background:#193c47;--ifm-color-info-contrast-foreground:#eef9fd;--ifm-color-warning-contrast-background:#4d3800;--ifm-color-warning-contrast-foreground:#fff8e6;--ifm-color-danger-contrast-background:#4b1113;--ifm-color-danger-contrast-foreground:#ffebec}:root{--docusaurus-progress-bar-color:var(--ifm-color-primary);background-color:#0000!important;--content-padding:16px;--container-max-width:1072px;--pagination-nav-margin-top:96px;--ifm-navbar-link-color:rgb(var(--lsd-text-primary));--ifm-navbar-background-color:rgb(var(--lsd-surface-primary));--ifm-dropdown-link-color:rgb(var(--lsd-text-primary));--ifm-dropdown-hover-background-color:rgb(var(--lsd-surface-secondary));--ifm-background-color:rgb(var(--lsd-surface-primary));--ifm-color-white:#fff;--ifm-color-black:#000;--ifm-color-primary:rgb(var(--lsd-theme-primary));--ifm-color-secondary:rgb(var(--lsd-theme-secondary));--prism-background-color:rgb(var(--lsd-theme-primary));--ifm-color-gray-0:rgb(var(--lsd-surface-primary));--ifm-color-gray-100:#f8f8fa;--ifm-color-gray-200:#f7f7f8;--ifm-color-gray-300:#eeeef0;--ifm-color-gray-400:#c0c0c2;--ifm-color-gray-500:#828285;--ifm-color-gray-600:#474747;--ifm-color-gray-700:#373738;--ifm-color-gray-800:#1a1a1a;--ifm-color-gray-900:#151515;--ifm-color-gray-1000:var(--ifm-color-black);--ifm-font-color-base-inverse:var(--ifm-color-black);--docusaurus-highlighted-code-line-bg:#ffffff26;--ifm-card-background-color:var(--ifm-color-gray-100)!important;--ifm-alert-foreground-color:var(--ifm-color-gray-700);--ifm-button-background-color:rgb(var(--lsd-surface-primary));--ifm-global-border-width:1px;--ifm-global-radius:0.6rem;--ifm-hover-overlay:#0000000d;--ifm-font-family-base:var(--lsd-typography-generic-font-family);--ifm-font-family-monospace:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;--lgs-font-family-secondary:Georgia,"Times New Roman",serif;--ifm-font-size-base:100%;--ifm-line-height-base:1.625;--ifm-font-size-secondary:100%;--ifm-line-height-secondary:1.5;--ifm-font-weight-light:400;--ifm-font-weight-normal:400;--ifm-font-weight-semibold:400;--ifm-font-weight-bold:600;--ifm-font-weight-base:var(--ifm-font-weight-normal);--ifm-h1-font-size:var(--lsd-h1-fontSize);--ifm-h2-font-size:var(--lsd-h2-fontSize);--ifm-h3-font-size:var(--lsd-h3-fontSize);--ifm-h4-font-size:var(--lsd-h4-fontSize);--ifm-h5-font-size:var(--lsd-h5-fontSize);--ifm-h6-font-size:var(--lsd-h6-fontSize);--ifm-global-spacing:24px;--ifm-spacing-vertical:var(--ifm-global-spacing);--ifm-spacing-horizontal:0.75rem;--ifm-image-alignment-padding:1.25rem;--ifm-leading-desktop:1.25;--ifm-leading:24px;--ifm-list-left-padding:2rem;--ifm-list-margin:1rem;--ifm-list-item-margin:0.25rem;--ifm-list-paragraph-margin:1rem;--ifm-code-background:rgb(var(--lsd-text-primary))!important;--ifm-pre-padding:1.125rem;--ifm-link-decoration:none;--ifm-paragraph-margin-bottom:var(--ifm-leading);--ifm-blockquote-border-left-width:3px;--ifm-blockquote-shadow:none;--ifm-hr-height:1px;--ifm-hr-margin-vertical:1.5rem;--ifm-scrollbar-size:7px;--ifm-alert-border-width:0px;--ifm-alert-border-left-width:5px;--ifm-alert-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-alert-padding-vertical:var(--ifm-spacing-vertical);--ifm-alert-shadow:var(--ifm-global-shadow-lw);--ifm-avatar-intro-margin:1rem;--ifm-avatar-intro-alignment:inherit;--ifm-avatar-photo-size:3rem;--ifm-badge-border-radius:var(--ifm-global-radius);--ifm-badge-border-width:var(--ifm-global-border-width);--ifm-badge-padding-horizontal:calc(var(--ifm-spacing-horizontal)*0.5);--ifm-badge-padding-vertical:calc(var(--ifm-spacing-vertical)*0.25);--ifm-breadcrumb-border-radius:1.5rem;--ifm-breadcrumb-spacing:0.5rem;--ifm-breadcrumb-padding-horizontal:0.8rem;--ifm-breadcrumb-padding-vertical:0.4rem;--ifm-breadcrumb-size-multiplier:1;--ifm-breadcrumb-separator:url('data:image/svg+xml;utf8,');--ifm-breadcrumb-separator-filter:none;--ifm-breadcrumb-separator-size:0.5rem;--ifm-breadcrumb-separator-size-multiplier:1.25;--ifm-button-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-button-padding-vertical:0.4rem;--ifm-button-size-multiplier:1;--ifm-button-transition-duration:var(--ifm-transition-fast);--ifm-button-border-radius:calc(var(--ifm-global-radius)*var(--ifm-button-size-multiplier));--ifm-button-group-spacing:2px;--ifm-card-border-radius:var(--ifm-global-radius);--ifm-card-horizontal-spacing:var(--ifm-spacing-horizontal);--ifm-card-vertical-spacing:var(--ifm-spacing-vertical);--ifm-footer-link-horizontal-spacing:0;--ifm-footer-padding-horizontal:var(--content-padding);--ifm-footer-padding-vertical:24px;--ifm-footer-background-color:#0000;--ifm-menu-link-padding-horizontal:var(--ifm-spacing-horizontal);--ifm-menu-link-padding-vertical:0.25rem;--ifm-menu-color:rgb(var(--lsd-text-primary),0.6);--ifm-menu-color-background-active:#0000;--ifm-menu-link-sublist-icon:url('data:image/svg+xml;utf8,');--ifm-menu-link-sublist-icon-filter:none;--ifm-navbar-height:64px;--ifm-navbar-item-padding-horizontal:0.75rem;--ifm-navbar-item-padding-vertical:0.25rem;--ifm-navbar-padding-horizontal:var(--content-padding);--ifm-navbar-padding-vertical:calc(var(--ifm-spacing-vertical)/3);--ifm-navbar-shadow:var(--ifm-global-shadow-lw);--ifm-navbar-search-input-icon:url('data:image/svg+xml;utf8,');--ifm-navbar-sidebar-width:100vw;--ifm-pagination-border-radius:var(--ifm-global-radius);--ifm-pagination-font-size:1rem;--ifm-pagination-item-active-background:var(--ifm-hover-overlay);--ifm-pagination-page-spacing:0.2em;--ifm-pagination-padding-horizontal:calc(var(--ifm-spacing-horizontal)*1);--ifm-pagination-padding-vertical:calc(var(--ifm-spacing-vertical)*0.25);--ifm-pagination-nav-border-radius:var(--ifm-global-radius);--ifm-pills-spacing:0.125rem;--ifm-tabs-padding-horizontal:1rem;--ifm-tabs-padding-vertical:1rem;--ifm-alert-background-color:#0000;--ifm-color-secondary-contrast-background:#0000;--ifm-color-success-contrast-background:#0000;--ifm-color-info-contrast-background:#0000;--ifm-color-warning-contrast-background:#0000;--ifm-color-danger-contrast-background:#0000;--ifm-alert-border-radius:none;--note:#d4d5d8;--tip:#6ace4b;--caution:#ddaa39;--danger:#e46967;--info:#68b1d0;--ifm-color-secondary-dark:var(--note);--ifm-color-secondary-light:var(--note);--ifm-color-success-dark:var(--tip);--ifm-color-success-light:var(--tip);--ifm-color-info-dark:var(--info);--ifm-color-info-light:var(--info);--ifm-color-warning-dark:var(--caution);--ifm-color-warning-light:var(--caution);--ifm-color-danger-dark:var(--danger);--ifm-color-danger-light:var(--danger);--ifm-footer-link-color:rgb(var(--lsd-text-primary));--ifm-color-content-secondary:rgb(var(--lsd-text-primary));--ifm-hero-background-color:#f8f8fa;--ifm-hero-text-color:var(--ifm-color-black);--desktop-footer-gap:200px;--mobile-footer-gap:144px;--doc-grid-gap:16px;--ifm-table-background:#0000!important;--ifm-table-stripe-background:#0000!important;--ifm-pre-border-radius:0!important;--ifm-code-font-size:100%!important;--doc-sidebar-width:16.66vw!important;--doc-sidebar-max-width:320px;--ifm-navbar-item-padding-vertical:8px;--ifm-navbar-item-padding-horizontal:12px;--mobile-width:100%;--mobile-height:176px;--docusaurus-announcement-bar-height:auto;--doc-sidebar-width:300px;--doc-sidebar-hidden-width:30px;--docusaurus-blog-social-icon-size:1rem;--docusaurus-tag-list-border:var(--ifm-color-emphasis-300)}#nprogress .bar{background:var(--docusaurus-progress-bar-color);height:2px;left:0;position:fixed;top:0;width:100%;z-index:1031}#nprogress .peg{box-shadow:0 0 10px var(--docusaurus-progress-bar-color),0 0 5px var(--docusaurus-progress-bar-color);height:100%;opacity:1;position:absolute;right:0;transform:rotate(3deg) translateY(-4px);width:100px}body,html{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;height:100%}main{min-height:calc(100vh - var(--ifm-navbar-height) - var(--content-padding))}svg *{fill:rgb(var(--lsd-text-primary))}.grid{display:grid;gap:16px}.table-of-contents__link--active,.table-of-contents__link:hover{color:rgb(var(--lsd-text-primary))!important}h1{font-size:var(--lsd-h1-fontSize);font-weight:var(--lsd-h1-fontWeight);line-height:var(--lsd-h1-lineHeight)}h1,h2,h3,h4{font-family:var(--lsd-typography-generic-font-family)}h2{font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight)}h3{font-size:var(--lsd-h3-fontSize);font-weight:var(--lsd-h3-fontWeight);line-height:var(--lsd-h3-lineHeight)}h4{font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight)}code{background:#ffffff26;border-radius:0;color:rgb(var(--lsd-text-primary));background:rgba(var(--lsd-theme-primary),.1)}pre code{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-body1-fontSize)!important;font-weight:var(--lsd-body1-fontWeight)!important;line-height:var(--lsd-body1-lineHeight)!important}.theme-doc-footer-edit-meta-row div[class*=lastUpdated_] b,table td strong,table th{font-weight:400!important}.mdx-page body{overflow-x:hidden!important}.mdx-page h2{margin-top:unset!important}.alert,.card,.theme-code-block{box-shadow:none!important}.menu__list-item-collapsible>a,.navbar{box-shadow:none}.sans{font-family:var(--lgs-font-family-secondary);line-height:var(--ifm-line-height-secondary)}.blog-wrapper--index [itemprop=blogPost] .markdown,.external-link,.footer__bottom,.footer__copyright,.menu__list-item .menu__link,.theme-doc-footer-edit-meta-row div[class*=lastUpdated_],table{font-size:var(--lsd-body2-fontSize)!important;font-weight:var(--lsd-body2-fontWeight)!important;line-height:var(--lsd-body2-lineHeight)!important;font-family:var(--lsd-typography-generic-font-family)!important}.markdown a,.markdown a:hover,article a,article a:hover{text-decoration-color:inherit;text-decoration-color:rgba(var(--lsd-text-primary),.6);text-underline-offset:.3rem}.card{padding:var(--ifm-spacing-vertical) var(--ifm-spacing-horizontal);padding:var(--ifm-alert-padding-vertical) var(--ifm-alert-padding-horizontal)}.mdx-cta-section--no-border,.mdx-cta-section--no-border>*,.mdx-feature-list--border-none,.mdx-video-section--no-border,.mdx-video-section--no-border>*,.table-of-contents__left-border{border:none!important}.menu__link{align-items:center;display:flex;justify-content:space-between}.menu__link span{color:rgba(var(--lsd-text-primary),.6)}.theme-edit-this-page{align-items:center;display:flex;gap:12px}.menuExternalLink_NmtK,.navbar__inner,.navbar__item{align-items:center}.dropdown__link:hover{color:rgb(var(--lsd-text-secondary))}.blog-wrapper>.container>.row>aside>nav>ul li a:hover,.breadcrumbs__item--active,.footerLogoLink_BH7S:hover,.hash-link:focus,.theme-code-block:hover .buttonGroup_Qu4e button,:hover>.hash-link{opacity:1}.breadcrumbs__item:not(:last-child):after{background:none;content:"/";opacity:1}.breadcrumbs__link{background:#0000!important;padding:0}.navbar{height:var(--ifm-navbar-height)}.navbar__logo,a[class^=sidebarLogo_] img{height:40px}a[class^=sidebarLogo_]{margin-left:-16px!important;margin:0}.header-github-link:hover,.mdx-event-card__label,.mdx-event-info__label{opacity:.6}.header-github-link:before{background:url("data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E") no-repeat;content:"";display:flex;height:16px;width:16px}html[data-theme=dark] .header-github-link:before{background:url("data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath fill='%23fff' d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E") no-repeat}.dropdown__link--active{color:rgb(var(--lsd-text-secondary))!important;background-color:#0000}.row{margin:0}.theme-doc-version-banner *,.theme-doc-version-banner:hover *{color:#fff;text-decoration-color:#fff!important}.navbar__left-items>a:not(:last-child){margin-right:1.25rem}.navbar__left-items>a:not(:last-child)[href^="/"]{margin-right:1.75rem}.menu__link--sublist-caret:after{background:none;content:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxNiIgaGVpZ2h0PSIxNiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZmZiIgZD0ibTQgOS41My45NC45NEw4IDcuNDE3bDMuMDYgMy4wNTMuOTQtLjk0LTQtNHoiLz48L3N2Zz4=);filter:unset;height:unset;min-width:unset;width:unset}.menu__list-item--collapsed>.menu__link--sublist-caret:after{transform:rotate(0)!important}.menu__link--active{border-radius:0}.dropdown__menu,.pagination-nav__link{border:1px solid rgb(var(--lsd-border-primary))}.menu__link--active span{color:rgb(var(--lsd-text-primary))}.table-of-contents a{padding-left:16px}.table-of-contents__link--active{border-left:1px solid rgb(var(--lsd-border-primary));padding-left:16px}.navbar__item{display:inline-flex;padding:0}.mdx-accordion-item,.navbar__link--active{border-bottom:1px solid rgb(var(--lsd-border-primary))}.dropdown__menu{background:rgb(var(--lsd-surface-primary));border-radius:0;margin-right:-1px;margin-top:6px;padding:0;transition-duration:0ms}.dropdown__link{background-color:#0000;border-radius:0;margin:0;padding:5px 11px}.dropdown__link *{text-decoration:inherit!important}.dropdown__link:hover{background-color:#0000}.margin-left-8,.mdx-jpd__external-link-icon{margin-left:8px}.pagination-nav{margin-top:var(--pagination-nav-margin-top)!important}.overflow-hidden,.pagination-nav a{overflow:hidden}.theme-admonition,table{overflow:auto}.pagination-nav span{display:-webkit-box;text-overflow:ellipsis;-webkit-line-clamp:1;-webkit-box-orient:vertical;overflow:hidden;width:70%;word-break:break-word}.pagination-nav__link{align-items:center;border-radius:0;display:flex;gap:18px;padding:10px 14px}.theme-doc-version-badge,table td,table th,table thead>tr{border:1px solid rgb(var(--lsd-border-primary))}.pagination-nav__link--next{grid-column:2/3;text-align:right;justify-content:flex-end}.footer__item a svg,.menu__caret,.pagination-nav__label:after,.pagination-nav__label:before,.theme-doc-footer-edit-meta-row div[class*=lastUpdated_] small{display:none}.no-breadcrumbs .theme-doc-breadcrumbs,.theme-back-to-top-button{display:none!important}.authors_dZ4g,.container_PkUo>div,.footer__link-item,.tag_NdBD{display:inline-block}.lsd-collapse__content{padding:24px}.theme-doc-version-badge{background:#0000;color:rgb(var(--lsd-text-primary));margin-bottom:16px;padding:4px 8px}.navbar__right-items{align-items:center;display:flex}table{border-collapse:collapse;margin-bottom:var(--ifm-spacing-vertical);display:table;table-layout:fixed;width:100%}table th{text-align:left}table td{color:var(--ifm-table-cell-color)}.theme-doc-footer{margin-bottom:96px}.theme-doc-footer-edit-meta-row div[class*=lastUpdated_]{font-style:normal;opacity:.7}.navbar-sidebar__item{padding-block:0;padding-inline:var(--content-padding)}.navbar-sidebar__back{align-items:center;display:flex;gap:16px;margin:24px 0!important;padding:0;width:100%!important}.theme-doc-sidebar-menu .menu__link{padding-left:var(--content-padding);padding-right:0}.menu__list-item-collapsible>a[aria-expanded=true]>svg{transform:rotate(180deg)}.events .pagination-nav,.events aside,.menu__list-item-collapsible>a>svg,.news .theme-doc-sidebar-container,.news header h1{display:none}main[itemtype*=Blog]{margin-left:0}.mdx-ghc-subheader-text,main[itemtype*=Blog] .theme-doc-breadcrumbs{margin-bottom:40px}main[itemtype*=Blog] .blog-divider{background:rgb(var(--lsd-theme-primary));margin-block:40px}.events .theme-doc-markdown>h1{margin-bottom:16px!important}.events .breadcrumbs__item,.events .breadcrumbs__link{-webkit-text-decoration:none;text-decoration:none}.events .theme-doc-breadcrumbs{display:block}.full-width>div .main-wrapper main{grid-column:1/17!important}.news .news-title{display:flex;font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h1-fontSize);font-weight:var(--lsd-h1-fontWeight);gap:16px;line-height:var(--lsd-h1-lineHeight);margin-bottom:0!important}.news .news-title span{opacity:.4}.footer__links>div:first-of-type,.footer__links>div:nth-of-type(3),.mdx-event-profile__title{margin-bottom:8px}.footer__links>div:nth-of-type(2){margin-bottom:34px}.container{padding-inline:var(--content-padding)}.main-wrapper>main,.main-wrapper>main>.row,.main-wrapper>main>.row>.col{--ifm-spacing-horizontal:var(--content-padding);flex-basis:100%!important;flex:unset;margin:0;max-width:unset;width:100%!important}.footer{margin-bottom:18px;width:100%!important}.footer>div{display:grid;gap:1rem;max-width:unset;padding:24px 0 0}.footer__bottom{grid-column:1/12;grid-row:1/1;text-align:unset}.blog-archive-page .main-wrapper header .container p,.blog-wrapper main>section article ul li a:after,.blog-wrapper main>section article ul li a:before,.footer__bottom>div:first-of-type,.mdx-accordion-item>input,.themedComponent_mlkZ,.themedImage_kfRS,[data-hidden-doc-sidebar=true] .theme-doc-sidebar-container,div[class^=blogHeader] span{display:none}.footer__title{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-label1-fontSize)!important;font-weight:var(--lsd-label1-fontWeight)!important;font-weight:var(--ifm-font-weight-normal);line-height:var(--lsd-label1-lineHeight)!important;margin-bottom:4px}.blog-wrapper main>section article ul li .lsd-tag>span,.blog-wrapper main>section article ul li .lsd-tag>span>span,.footer__item,.footer__item a{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-label1-fontSize)!important;font-weight:var(--lsd-label1-fontWeight)!important;line-height:var(--lsd-label1-lineHeight)!important}.footer>div:first-child>.footer__links,.footer>div:nth-child(2)>.footer__links{display:grid;grid-column:13/25;grid-row:1/1;margin-bottom:0}.footer>div:nth-child(2)>.footer__links{gap:2rem 1rem;grid-template-columns:repeat(2,1fr)}.footer__col{grid-template-columns:span c}.blog-wrapper main>section article ul li a,.codeBlockStandalone_BRZX,.codeBlockStandalone_MEMb,.col.footer__col,.modal_kLVz>div>div:first-child,.noResults_mD2O,.root_N57j ul,.root_ONDG.empty_a7qb{padding:0}.footer__items{display:flex;flex-direction:row;flex-wrap:wrap;gap:.5rem}.footer__item{display:inline-flex}.errorBoundaryError_a6uf,.input-cta-section__description,.mdx-event-card__info *,.mdx-event-info__content,.mdx-news-card__info *{white-space:pre-wrap}.blog-wrapper .breadcrumbs__item,.blog-wrapper main a,.blog-wrapper--index h2[itemprop=headline]>a,.footer__item:not(:last-child):after,.mdx-asset-card__downloadables>* button,.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>a,.mdx-cta-section--list .mdx-cta-section__list>a{-webkit-text-decoration:none;text-decoration:none}.footer__item:not(:last-child):after{content:"•";display:inline-block;margin-left:8px}.footer__logo{max-width:var(--ifm-footer-logo-max-width);margin:0!important}.navbar__inner{flex-direction:row}.hero--primary{--ifm-hero-background-color:rgb(var(--lsd-surface-primary));--ifm-hero-text-color:rgb(var(--lsd-text-primary))}.docusaurus-mermaid-container{margin-block:40px}.hidden-scrollbar{-ms-overflow-style:none;scrollbar-width:none}.hidden-scrollbar::-webkit-scrollbar{display:none;width:0}[class^=mdx-]{--lsd-h1-fontWeight:100;--lsd-h2-fontWeight:300;--lsd-h3-fontWeight:300;--lsd-h4-fontWeight:300;--lsd-h5-fontWeight:300;--lsd-h6-fontWeight:300;--lsd-subtitle1-fontWeight:300}.hidden{opacity:0;visibility:hidden}.blog-wrapper h1[itemprop=headline]{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h1-fontSize)!important;font-weight:var(--lsd-h1-fontWeight)!important;line-height:var(--lsd-h1-lineHeight)!important}.blog-wrapper>.container{display:flex;margin:0;max-width:unset;padding-inline:0;width:100%}.mdx-cta-section .mdx-cta-section--align-right .mdx-cta-button,.mdx-logo-carousel .mdx-section-header__extra,.mdx-video-section .mdx-video-section--align-right .mdx-cta-button{margin-left:auto}.blog-wrapper>.container>.row,.mdx-roadmap,.mdx-scroll-buttons{width:100%}.blog-wrapper main>header h1{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important;margin-bottom:.5rem}.blog-wrapper>.container>.row>aside{max-width:unset}.blog-wrapper>.container>.row>aside>nav{padding-top:16px;top:var(--ifm-navbar-height)}.blog-wrapper>.container>.row>aside>nav>div:first-child{font-family:var(--lsd-typography-generic-font-family)!important;margin-bottom:0!important;padding:4px 0 4px 15px}.blog-wrapper>.container>.row>aside>nav>ul li a{box-sizing:border-box;display:block;font-family:var(--lsd-typography-generic-font-family)!important;opacity:.6;padding:4px 0 4px 2rem}.blog-wrapper--index>.container>.row>aside>nav>div:first-child,.blog-wrapper>.container>.row>aside>nav>ul li a[class*=sidebarItemLinkActive_]{border-color:rgb(var(--lsd-border-primary))}.mdx-cta-section,.mdx-cta-section--list .mdx-cta-section__list>:not(:first-child),.mdx-video-section{border-top:1px solid rgb(var(--lsd-border-primary))}.blog-wrapper main article,.mdx-asset-card:not(.mdx-asset-card--downloadable) .mdx-asset-card__inner,.mdx-erc--with-preview .mdx-erc__preview-image{border-bottom:1px solid rgb(var(--lsd-border-primary))}.blog-wrapper--index h2[itemprop=headline]{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.mdx-erc:hover .mdx-erc__title,.mdx-ghc__issue-title-link:hover,.mdx-ghc__view-on-github-link:hover{-webkit-text-decoration:underline!important;text-decoration:underline!important}.blog-wrapper>.container>.row>main{margin-top:16px;max-width:unset}.blog-archive-page .main-wrapper header .container h1,.blog-wrapper article header h2,.blog-wrapper main>section article h2{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.blog-wrapper article footer{align-items:center;display:flex;flex-direction:row;justify-content:space-between;margin-top:24px!important}.blog-wrapper main article{margin-bottom:34px!important;padding-bottom:34px}.blog-wrapper main article:last-of-type{border-bottom:none!important}.blog-archive-page .main-wrapper{padding:2rem var(--content-padding)}.blog-archive-page .main-wrapper header{border-bottom:1px solid var(--ifm-hero-text-color);margin:0;padding:0 0 1rem}.blog-archive-page .main-wrapper header .container{margin:0;padding:0!important}.blog-archive-page .main-wrapper main>section .container{margin:0;max-width:unset;padding:0}.blog-archive-page .main-wrapper main>section .container>.row,.mdx-logo-carousel-buttons,.mdx-roadmap__timeline{margin-top:64px}.blog-archive-page .main-wrapper main>section .container>.row>.col{margin:0!important;max-width:unset;padding:0}.blog-archive-page .main-wrapper main>section .container>.row>.col h3{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h5-fontSize)!important;font-weight:var(--lsd-h5-fontWeight)!important;line-height:var(--lsd-h5-lineHeight)!important}.blog-archive-page .main-wrapper main>section .container>.row>.col a{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-body1-fontSize)!important;font-weight:var(--lsd-body1-fontWeight)!important;line-height:var(--lsd-body1-lineHeight)!important}.blog-archive-page .main-wrapper main>section .container>.row>.col ul{margin:1rem 0 0;padding:0 0 0 1.5rem}.blog-wrapper main>section article{border-bottom:none;padding:0}.blog-wrapper main>section article ul,.mdx-app-card__name{margin-top:1rem}.blog-wrapper main>section article ul li{margin:0 .5rem 0 0!important;padding:0!important}.blog-wrapper main>section article ul li .lsd-tag{height:28px;padding:3px 11px}.blog-wrapper main>section article ul li .lsd-tag>span,.blog-wrapper main>section article ul li .lsd-tag>span>span{color:rgb(var(--lsd-text-primary))}.mdx-accordion-item__header{align-items:center;display:flex;flex-direction:row;gap:0 16px;justify-content:space-between;min-height:88px;padding:16px 0}.mdx-asset-card,.mdx-erc{min-height:144px;-webkit-text-decoration:none!important;text-decoration:none!important}.mdx-accordion-item__content-wrapper{height:auto;overflow:hidden}.mdx-accordion-item__content{height:0}.mdx-accordion-item--open .mdx-accordion-item__content{height:auto;padding-bottom:1.875rem;padding-top:6px}.backToTopButton_sjWU{background-color:var(--ifm-color-emphasis-200);border-radius:50%;bottom:1.3rem;box-shadow:var(--ifm-global-shadow-lw);height:3rem;opacity:0;position:fixed;right:1.3rem;transform:scale(0);transition:all var(--ifm-transition-fast) var(--ifm-transition-timing-default);visibility:hidden;width:3rem;z-index:calc(var(--ifm-z-index-fixed) - 1)}.chevron_How5,.root_TTsz .content_feMR{position:absolute;left:0}.backToTopButton_sjWU:after{background-color:var(--ifm-color-emphasis-1000);content:" ";display:inline-block;height:100%;-webkit-mask:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem no-repeat;mask:var(--ifm-menu-link-sublist-icon) 50%/2rem 2rem no-repeat;width:100%}.backToTopButtonShow_xfvO{opacity:1;transform:scale(1);visibility:visible}[data-theme=dark] .themedComponent--dark_xIcU,[data-theme=dark] .themedImage--dark_OvIx,[data-theme=light] .themedComponent--light_NVdE,[data-theme=light] .themedImage--light_BL8e,html:not([data-theme]) .themedComponent--light_NVdE,html:not([data-theme]) .themedImage--light_BL8e{display:initial}.errorBoundaryError_a6uf{color:red}.errorBoundaryFallback_VBag{color:red;padding:.55rem}body:not(.navigation-with-keyboard) :not(input):focus{outline:0}.icon_S7Kx.s_AZDZ svg{width:10px}.icon_S7Kx.m_thRi svg{width:20px}.icon_S7Kx.l_WHPt svg{width:40px}[data-theme=dark] .icon_S7Kx.fill_hNhN,[data-theme=dark] .icon_S7Kx.fill_hNhN *{fill:var(--ifm-color-white)}[data-theme=dark] .icon_S7Kx.stroke_N8dm svg,[data-theme=dark] .icon_S7Kx.stroke_N8dm svg *{stroke:var(--ifm-color-white)}[data-theme=light] .icon_S7Kx.fill_hNhN,[data-theme=light] .icon_S7Kx.fill_hNhN *{fill:var(--ifm-color-black)}[data-theme=light] .icon_S7Kx.stroke_N8dm svg,[data-theme=light] .icon_S7Kx.stroke_N8dm svg *{stroke:var(--ifm-color-black)}.mdx-app-card{align-items:flex-start;display:flex;flex-direction:column;padding:1.5rem 0}.mdx-app-card__logo{height:40px;width:auto}.mdx-app-card__description{margin-top:1.5rem}.mdx-app-card__link{margin-top:1.5rem;width:100%}.mdx-app-card__link button{padding:10px 16px;width:100%}.mdx-app-card__link button>span{align-items:center;display:block;display:flex;flex-direction:row;justify-content:space-between;width:100%}.root_TTsz{display:inline-block;font-size:0!important;position:relative}.externalLinkIcon_Qfue svg,.root_TTsz.fullWidth_HBtH{height:auto;width:100%}.root_TTsz.fullHeight_ls1u{height:100%;width:auto}.root_TTsz .content_feMR{height:100%;top:0;width:100%}.root_TTsz:not(.keep_Y9Ea) .root_TTsz{height:auto;width:auto}.root_TTsz:not(.keep_Y9Ea) .content_feMR{height:100%;left:unset;position:relative;top:unset;width:100%}.mdx-asset-card{align-items:flex-start;border:1px solid rgb(var(--lsd-border-primary));border-bottom:none;display:flex;flex-direction:column;width:216px}.mdx-asset-card__inner{padding:16px 16px 32px;width:100%}.mdx-asset-card__title,.mdx-event-profile-list__container h2,.mdx-event-profile__role,.mdx-ghc__challenge-labels,.mdx-ghc__issue-title{margin-bottom:16px}.mdx-asset-card__image img{height:100%;object-fit:contain;object-position:center center;width:100%}.mdx-asset-card__downloadables{align-items:flex-start;display:flex;flex-direction:row;justify-content:stretch;width:100%}.mdx-asset-card__downloadables>*{flex-grow:1;font-size:12px;-webkit-text-decoration:none!important;text-decoration:none!important;text-underline-offset:unset!important}.mdx-asset-card__downloadables>* button{justify-content:space-between;width:100%}.mdx-asset-card__downloadables>:first-child button{border-left:none}.mdx-asset-card__downloadables>:last-child button,.mdx-asset-card__downloadables>:not(:last-child) button{border-right:none}.mdx-box{padding-bottom:var(--mdx-box-bottom);padding-top:var(--mdx-box-top)}.mdx-cta-button,.mdx-section-header--with-description .mdx-section-header__extra{margin-top:2rem}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>.mdx-cta-section__item-description,.mdx-cta-section__description{margin-top:1rem}.mdx-cta-button button,.mdx-cta-button button span{font-size:inherit;font-weight:inherit;line-height:inherit}.mdx-cta-button .lsd-button--large{padding:9px 39px}.mdx-cta-button .lsd-button--small{padding:5px 11px}.mdx-cta-section,.mdx-video-section{display:grid;padding:24px 0 0}.mdx-cta-section .mdx-cta-section__link,.mdx-jpd__job-link,.mdx-video-section .mdx-video-section__link{display:block;width:fit-content}.mdx-cta-section .mdx-cta-section__header__desktop-button .mdx-cta-button{margin-top:unset}.mdx-cta-section .mdx-cta-section--align-center{align-items:center;display:flex;flex-direction:column;margin:auto;max-width:936px}.mdx-cta-section .mdx-cta-section--align-center .mdx-cta-section__description{margin-top:1.5rem}.mdx-cta-section .mdx-cta-section--align-left *{text-align:left}.mdx-cta-section .mdx-cta-section--align-right *{text-align:right}.content_knG7,.mdx-video-section .mdx-video-section--align-center *,.root_EFVO{text-align:center}.mdx-cta-section__header,.mdx-profile-card__profile{align-items:center;display:flex;gap:24px}.mdx-cta-section__header button{font-size:12px!important;line-height:16px!important;padding:6px 12px!important}.mdx-cta-section--title-only .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);font-weight:300;line-height:var(--lsd-h5-lineHeight);max-width:886px}.mdx-cta-section--title-button .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-display4-fontSize);font-weight:var(--lsd-display4-fontWeight);font-weight:300;line-height:var(--lsd-display4-lineHeight);max-width:886px}.mdx-cta-section--full-width .mdx-cta-section__container{display:grid;gap:0 1rem;grid-template-columns:repeat(2,1fr)}.mdx-cta-section--full-width .mdx-cta-section__title,.mdx-ns__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h3-fontSize);font-weight:var(--lsd-h3-fontWeight);line-height:var(--lsd-h3-lineHeight)}.mdx-cta-section--full-width .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-cta-section--simple .mdx-cta-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-cta-section--simple .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-cta-section--list{display:grid;gap:0 1rem;grid-template-columns:1fr 1fr}.mdx-cta-section--list .mdx-cta-section__container{grid-column:1/2}.mdx-cta-section--list .mdx-cta-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-cta-section--list .mdx-cta-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-hero h1,.mdx-roadmap .mdx-cta-section__container h3,.mdx-roadmap .mdx-cta-section__description,.mdx-roadmap .mdx-cta-section__title{font-family:var(--lsd-typography-generic-font-family)!important}.mdx-cta-section--list .mdx-cta-section__list{display:flex;flex-direction:column;gap:1.5rem 0;grid-column:2/3}.mdx-erc--with-preview .mdx-erc__icon,.mdx-ns__inner .mdx-ns__toast .lsd-toast__close-button{display:none}.mdx-cta-section--list .mdx-cta-section__list>a{display:block}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>a>.mdx-cta-section__item-title{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:1rem;color:rgb(var(--lsd-text-primary));display:flex;gap:12px;margin-top:1.5rem;padding:3px 11px;width:fit-content}.mdx-doc-metadata{display:flex;flex-direction:row;line-height:0;margin-bottom:2.5rem}.mdx-doc-metadata>span:not(:last-child):after{content:"•";display:inline-block;margin-inline:.75rem}.markdown h1+.mdx-doc-metadata,.markdown header+.mdx-doc-metadata{margin-top:calc(var(--ifm-h1-vertical-rhythm-bottom)*-1*var(--ifm-leading) + 1rem)}.mdx-erc{align-items:flex-start;border:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:row}.mdx-erc__icon{padding:16px 16px 16px 8px}.mdx-erc__inner{flex-grow:1;padding:16px}.mdx-erc__logo,.mdx-erc__logo svg{height:32px!important;width:32px!important}.mdx-erc__title{display:-webkit-box;margin-top:32px;overflow:hidden;max-height:calc(var(--lsd-body1-lineHeight)*2)}.mdx-erc__description{margin-top:8px}.mdx-erc--with-preview{display:flex;flex-direction:column}.mdx-erc--with-preview .mdx-erc__preview-image{height:100%;object-fit:cover;width:100%}.mdx-erc--with-preview .mdx-erc__inner{display:grid;gap:0 16px;grid-template-columns:32px auto;grid-template-rows:auto auto;padding:16px}.mdx-erc--with-preview .mdx-erc__logo{align-self:center;grid-column:1;grid-row:1/span 2}.mdx-erc--with-preview .mdx-erc__title{margin-top:0;-webkit-line-clamp:1;-webkit-box-orient:vertical;max-height:calc(var(--lsd-body1-lineHeight)*1)}.mdx-erc--with-preview .mdx-erc__description{margin-top:4px}.mdx-feature-list{border-top:1px solid rgb(var(--lsd-surface-secondary));position:relative}.mdx-feature-list__feature,.mdx-ghc__container,.mdx-showcase-card{border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-feature-list__title,.root_ltHz{padding:24px 0}.mdx-feature-list__feature-index--circle{align-items:center;background:rgb(var(--lsd-surface-secondary));border-radius:100%;color:rgb(var(--lsd-text-secondary))!important;display:inline-flex;height:1.875rem;justify-content:center;padding:.5rem;width:1.875rem}.mdx-feature-list__feature-index--number{background:#0000;color:rgb(var(--lsd-text-primary))!important;display:block;font-size:28px!important;height:unset;line-height:36px!important;opacity:.3;padding:unset;width:unset}.mdx-feature-list__list{display:grid;grid-template-columns:repeat(2,1fr)}.mdx-event-profile__info,.mdx-feature-list__feature-inner,.mdx-numbers-view-header-title,.roadmap__header{display:flex;flex-direction:column}.mdx-feature-list__feature-title{margin-top:.5rem}.mdx-feature-list__feature-description{max-width:460px}.mdx-feature-list__header{align-items:center;display:flex;flex-direction:row;gap:2rem}.mdx-feature-list--cta-bottom .mdx-feature-list__extra{margin-top:3.5rem;padding:0 1rem}.mdx-feature-list__feature-border--none{width:0!important}.mdx-ghc__container{padding-top:24px}.mdx-ghc__issue-title-link{-webkit-text-decoration:none!important;text-decoration:none!important}.mdx-ghc__header,.mdx-jpd__header{margin-bottom:40px;margin-top:16px}.mdx-ghc__challenge-label{border:1px solid rgb(var(--lsd-border-primary));border-radius:20px;margin-right:8px;padding:4px 8px}.mdx-ghc__view-on-github-link{display:block;margin-bottom:56px;margin-top:40px;-webkit-text-decoration:none!important;text-decoration:none!important;width:fit-content}.mdx-ghc__participant-photo{border:1px solid rgb(var(--lsd-border-secondary));border-radius:100%;margin-left:-4px;width:24px}.mdx-timeline-item,.mdx-timeline-item__border{border-bottom:1px solid rgb(var(--lsd-border-primary))}.mdx-ghc__participant-photo-container{align-items:center;direction:rtl;display:flex;justify-content:flex-end;padding-left:4px}.mdx-ghc__comment-count{margin-left:4px}.mdx-ghc__issue-content-grid{align-items:center;display:grid;grid-template-columns:82px 135px 82px 1fr;margin-top:24px;row-gap:12px}.mdx-scroll-buttons>div{display:flex;flex-direction:row;justify-content:flex-end;width:100%}.mdx-hero{--default-z-index:1;--hero-max-height:1080px;display:flex;flex-direction:column;height:calc(100vh - var(--ifm-navbar-height) - 8px);max-height:var(--hero-max-height);position:relative;z-index:var(--default-z-index)}.mdx-hero h1{font-size:var(--lsd-display4-fontSize)!important;font-weight:var(--lsd-display4-fontWeight)!important;line-height:var(--lsd-display4-lineHeight)!important}.mdx-hero--top{justify-content:flex-start}.mdx-hero--center{justify-content:center}.mdx-hero--bottom{justify-content:flex-end}.mdx-hero-action--large button{padding:10px 64px}.mdx-hero-actions{align-items:center;display:flex;flex-direction:row;gap:0 1rem;margin-top:2rem;position:relative;z-index:98}.mdx-hero-description{font-size:18px!important;line-height:24px!important;margin-top:18px;max-width:528px;position:relative;text-shadow:1px 1px 6px rgba(var(--lsd-surface-primary),1);z-index:-2}.mdx-hero-info{display:flex;flex-direction:column;justify-content:end;max-width:596px;padding:32px 0;transition:margin-bottom .05s ease-in-out}.mdx-hero-title{font-size:48px!important;line-height:56px!important}.mdx-jpd__department-title,.mdx-video-section__header button{font-size:12px!important;line-height:16px!important}.mdx-hero-video{display:grid;flex-grow:1;grid-template-columns:1fr;grid-template-rows:1fr;position:relative;--hero-video-height:120%;--hero-video-min-height:100vh;--hero-video-offset-y:-150px;--hero-video-scale:1.70951586;--hero-video-height-mobile:120%;--hero-video-min-height-mobile:100vh;--hero-video-offset-y-mobile:-50px;--hero-video-scale-mobile:1.70951586}.mdx-hero-video>*{align-items:center;display:flex;flex-direction:column;grid-column:1/1;grid-row:1/1;height:var(--hero-video-height);justify-content:center;left:0;min-height:var(--hero-video-min-height);position:absolute;top:0;transform:translateY(var(--hero-video-offset-y));width:100%}.mdx-hero-video .mdx-hero-video__placeholder{opacity:0;visibility:hidden}.mdx-hero-video img{object-fit:contain}.mdx-hero-video img,.mdx-hero-video video{height:100%;max-width:calc(var(--container-max-width)*var(--hero-video-scale));object-position:top;transform-origin:top center;width:auto}.copyButtonCopied_YovB .copyButtonIcon_ZL7v,.mdx-hero-video--loading .mdx-hero-video__video{opacity:0}.mdx-hero-video--loading .mdx-hero-video__placeholder{opacity:1;visibility:visible}.mdx-hero-asset{--default-z-index:-1;position:relative;z-index:var(--default-z-index)}.mdx-hero-asset button{display:none!important}.mdx-hero-asset img{position:absolute;right:0;top:0}.mdx-jpd__single-job-department-container{border-top:1px solid rgb(var(--lsd-border-primary));padding-top:8px}.mdx-jpd__job-title-container{align-items:center;display:flex;padding-bottom:8px}.mdx-jpd__job-list{list-style-type:none;margin:0;padding:8px 0 24px}.mdx-jpd__job-list-item{padding:14px 0}.mdx-jpd__job-link,.mdx-jpd__job-link:hover,.root_Ooa0{-webkit-text-decoration:none!important;text-decoration:none!important}.mdx-event-card__title:hover,.mdx-jpd__job-link:hover .mdx-jpd__job-title,.mdx-news-card:hover .mdx-news-card__title,.mdx-social-card:hover{-webkit-text-decoration:underline!important;text-decoration:underline!important}.mdx-logo-carousel .mdx-section-header__title{flex-basis:100%}.mdx-logo-carousel .mdx-logo-carousel__inner{margin-top:32px}.mdx-logo-carousel .mdx-logo-carousel__item{flex-shrink:0}.mdx-logo-carousel .mdx-logo-carousel__logo{height:56px;width:auto}.mdx-ns__inner form{margin-top:40px;width:100%}.mdx-ns__inner .mdx-ns__inputs{align-items:center;display:flex;flex-direction:row;gap:16px;width:100%}.mdx-ns__inner .mdx-ns__inputs>*{flex-grow:1}.mdx-event-card-list__container,.mdx-ns__inner .mdx-ns__submit-button{margin-top:40px}.mdx-ns__inner .mdx-ns__toast{margin-bottom:24px;margin-top:-16px;width:100%}.mdx-profile-card{align-items:flex-start;border-top:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;justify-content:space-between;padding:1.5rem 1rem 0 0}.mdx-profile-card__profile-placeholder{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:50%;display:flex;height:56px!important;justify-content:center;width:56px!important}.mdx-profile-card__profile-placeholder svg,.mdx-social-card__logo,.mdx-social-card__logo svg{height:40px!important;width:40px!important}.mdx-profile-card__profile-placeholder svg rect{fill:unset!important}.mdx-profile-card__profile-image{border-radius:50%;height:56px!important;width:56px!important}.mdx-profile-card__buttons{align-items:center;display:flex;flex-direction:row;gap:8px;height:16px;margin-top:8px}.mdx-profile-card__link{height:fit-content;position:relative;-webkit-text-decoration:none;text-decoration:none}.mdx-event-card,.mdx-event-profile,.mdx-news-card,.mdx-social-card{-webkit-text-decoration:none!important;text-decoration:none!important}.mdx-profile-card__link svg{height:16px;width:16px}.mdx-timeline-item{display:flex;flex:0 0 auto;flex-direction:column;gap:1rem;justify-content:space-between;padding-bottom:23px;scroll-snap-align:start!important;width:193px}.mdx-timeline-item__header{align-items:flex-start;display:flex;flex-direction:column;gap:1rem}.mdx-timeline-item__period-container{position:relative;width:100%}.mdx-timeline-item__border{height:50%;left:0;position:absolute;top:0;width:calc(100% + 1rem);z-index:-1}.mdx-timeline-item--border-dashed .mdx-timeline-item__border{border-bottom-style:dashed}.mdx-timeline-item__period{border-radius:10rem;display:inline-block;padding:3px 12px}.docSidebarContainer_YfHR,.groups_p1lF .divider_eGUz:last-child,.mdx-roadmap__timeline .mdx-grid-item:last-child .mdx-timeline-item__border,.mdx-timeline-item__item:last-child .mdx-timeline-item__border,.modal_kLVz .closeButton_Rr0e,.navbarSearchContainer_Bca1:empty,.sidebarLogo_Ydl9,[data-theme=dark] .lightToggleIcon_K4TL,[data-theme=light] .darkToggleIcon_lKkA,html[data-announcement-bar-initially-dismissed=true] .announcementBar_mb4j{display:none}.mdx-timeline-item__period--filled{background-color:rgb(var(--lsd-surface-secondary));border:1px solid rgb(var(--lsd-border-secondary));color:rgb(var(--lsd-text-secondary))!important}.mdx-timeline-item__period--transparent{background-color:rgb(var(--lsd-surface-primary));border:1px solid rgb(var(--lsd-border-primary));color:rgb(var(--lsd-text-primary))!important}.mdx-timeline-item__description{margin-bottom:auto;margin-top:24px;white-space:pre-wrap}.mdx-roadmap .mdx-grid__scroll{margin-bottom:32px!important}.mdx-roadmap .mdx-cta-section__title{font-size:var(--lsd-h2-fontSize)!important;font-weight:var(--lsd-h2-fontWeight)!important;line-height:var(--lsd-h2-lineHeight)!important}.mdx-roadmap .mdx-cta-section__description{font-size:var(--lsd-h5-fontSize)!important;font-weight:var(--lsd-h5-fontWeight)!important;line-height:var(--lsd-h5-lineHeight)!important;margin-top:24px;max-width:528px}.mdx-roadmap .mdx-cta-section__container h3{font-size:var(--lsd-subtitle1-fontSize)!important;font-weight:var(--lsd-subtitle1-fontWeight)!important;line-height:var(--lsd-subtitle1-lineHeight)!important}.mdx-roadmap__timeline .mdx-roadmap__timeline-item--no-border{border:none}.mdx-roadmap__timeline-item{height:100%}.scrollToBottom_NE5w{--offset-y:-2rem;background:rgb(var(--lsd-surface-primary))!important;opacity:1;position:absolute;top:min(100*var(--vh) - var(--ifm-navbar-height),var(--maxTop) - var(--ifm-navbar-height));transform:translateY(-2rem);transform:translateY(calc(-100% + var(--offset-y)));transition:.2s ease-in-out;z-index:100}.scrollToBottom_NE5w.hide_q_kn{opacity:0;transform:translateY(calc(-100% + var(--offset-y) - .625rem))}.mdx-section-header{border-top:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:row;gap:1rem;padding-top:24px;width:100%}.mdx-section-header>*{flex-basis:50%}.mdx-section-header__title{display:flex;flex-direction:row;gap:1.5rem}.mdx-section-header--no-border{border-top:none}.mdx-showcase-card__inner{display:flex;flex-direction:column;gap:1rem;padding-top:1.5rem}.mdx-showcase-card__logo{height:auto;width:40px}.mdx-showcase-card__index{align-items:center;background-color:rgb(var(--lsd-surface-secondary));border-radius:50%;color:rgb(var(--lsd-text-secondary))!important;display:flex;flex-direction:row;height:30px;justify-content:center;width:30px}.mdx-showcase-card__title{align-items:center;display:flex;flex-wrap:wrap;gap:8px}.mdx-showcase-card__description{margin-top:1rem;padding-top:.5rem}.mdx-showcase-card--large .mdx-showcase-card__logo{width:40px}.mdx-showcase-card--small .mdx-showcase-card__logo{width:34px}.mdx-showcase-card--border-solid .mdx-showcase-card__description{border-top:1px solid rgb(var(--lsd-border-primary));padding-top:1rem}.mdx-showcase-card__cta{align-items:center;display:flex;font-size:14px;gap:6px;line-height:20px}.mdx-showcase-card__cta svg{transform:rotate(-90deg)}.mdx-social-card{align-items:flex-start;border:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;justify-content:space-between;min-height:144px;padding:1rem}.mdx-social-card__row{display:flex;justify-content:space-between;width:100%}.mdx-event-card{display:flex;flex-direction:column;gap:24px}.mdx-event-card__row,.mdx-event-profile__row,.mdx-news-card__row{display:flex;gap:8px;justify-content:space-between;width:100%}.mdx-event-card__thumbnail{border:1px solid rgba(var(--lsd-border-primary),.2);height:55px}.mdx-event-card__title{display:-webkit-box;text-overflow:ellipsis;height:48px;overflow:hidden;word-break:break-word}.mdx-event-card__info{border-top:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;gap:16px;padding:16px 2px 16px 0;width:100%}.mdx-event-card-list__button,.mdx-news-card-list__button{display:block;margin:0 auto}.mdx-event-banner__thumbnail{aspect-ratio:5/1;border:1px solid rgba(var(--lsd-border-primary),.2);margin-bottom:32px}.mdx-event-info__container{align-self:stretch;border-top:1px solid rgba(var(--lsd-border-primary));display:flex;flex:1 0 0;flex-direction:column;gap:16px;height:100%;padding:16px 24px 0 0}.mdx-event-about__container{display:grid;gap:16px;grid-template-columns:repeat(12,1fr)}.mdx-event-about__container>p{grid-column:1/6;margin-top:0!important}.mdx-event-about__container span{grid-column:7/13}.mdx-event-profile,.mdx-news-card{display:flex;flex-direction:column;gap:16px}.mdx-event-profile__image{aspect-ratio:1/1}.mdx-event-profile__social{align-items:center;display:flex;gap:8px}.mdx-event-cta__container h2{margin-bottom:32px;margin-top:0!important}.mdx-event-cta__button{height:70px!important;width:100%!important}.mdx-event-cta__button>span{align-items:center;display:flex;height:100%;justify-content:space-between}.mdx-input-cta-section__container{border-top:1px solid rgb(var(--lsd-border-primary));padding-top:24px;position:relative}.mdx-input-cta-section__container h2{margin-bottom:16px;margin-top:0!important}.mdx-input-cta-section__title{margin-bottom:24px!important}.mdx-input-cta-section__cta{height:40px!important;width:160px!important}.input-cta-section__description{margin-bottom:32px!important}.mdx-input-cta-section__form{display:flex;gap:16px}.mdx-input-cta-section__input-fields,.mdx-numbers-view{display:flex;gap:16px;width:100%}.mdx-input-cta-section__input-fields>div,.mdx-numbers-view-title,.mdx-table-view-description,.mdx-table-view-title{width:100%}.mdx-input-cta-section__message{bottom:-32px;display:block;position:absolute}form .mdx-input-cta-section__cta{width:146px!important}.mdx-event-header__container{display:flex;flex-direction:column;gap:24px}.event-details .mdx-event-header__container h1{margin-bottom:0!important}.mdx-news-card:hover .mdx-news-card__external-link-icon{display:flex!important}.mdx-news-card__thumbnail{border:1px solid rgba(var(--lsd-border-primary),.2);display:flex!important;height:144px;width:100%}.mdx-news-card__info{display:flex;gap:8px;width:100%}.mdx-news-card__tags{display:flex;gap:8px;margin-top:4px}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>.mdx-video-section__item-description,.mdx-video-section__description{margin-top:1rem}.mdx-news-card-thumbnail__container{height:144px;position:relative;width:100%}.mdx-news-card__external-link-icon{align-items:center;border:1px solid #fff;bottom:8px;display:none;height:28px;justify-content:center;position:absolute;right:8px;width:28px;z-index:9999}.mdx-spec-card,.mdx-table-view-row,.mdx-video-section--list .mdx-video-section__list>:not(:first-child){border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-news-card__external-link-icon svg>path{fill:#fff}.mdx-news-card__external-link-icon>div{margin:unset}.mdx-video-section .mdx-video-section__video-container{grid-column:8/17;height:0;overflow:hidden;padding-top:56.25%;position:relative;width:100%}.mdx-video-section iframe{left:0;position:absolute;top:0}.mdx-video-section .mdx-video-section--align-center{align-items:center;display:flex;flex-direction:column;margin:auto;max-width:936px}.mdx-video-section .mdx-video-section--align-center .mdx-video-section__description{margin-top:1.5rem}.mdx-video-section .mdx-video-section--align-left *{text-align:left}.mdx-video-section .mdx-video-section--align-right *{text-align:right}.mdx-video-section__header{align-items:center;display:flex;gap:16px}.mdx-video-section__header button{padding:6px 12px!important}.mdx-video-section__header svg{height:26px!important;width:26px!important}.mdx-video-section__header svg path{fill:rgb(var(--lsd-text-secondary))}.mdx-video-section--title-only .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);font-weight:300;line-height:var(--lsd-h5-lineHeight);max-width:886px}.mdx-video-section--title-button .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-display4-fontSize);font-weight:var(--lsd-display4-fontWeight);font-weight:300;line-height:var(--lsd-display4-lineHeight);max-width:886px}.mdx-video-section--full-width .mdx-video-section__container{display:grid;gap:0 1rem;grid-template-columns:repeat(2,1fr)}.mdx-video-section--full-width .mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h3-fontSize);font-weight:var(--lsd-h3-fontWeight);line-height:var(--lsd-h3-lineHeight)}.mdx-video-section--full-width .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-video-section--simple .mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-video-section--simple .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-video-section--list{display:grid;gap:0 1rem;grid-template-columns:repeat(16,1fr)}.mdx-video-section--list .mdx-video-section__container{grid-column:1/7}.mdx-video-section--list .mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h2-fontSize);font-weight:var(--lsd-h2-fontWeight);line-height:var(--lsd-h2-lineHeight);margin-top:1.5rem}.mdx-video-section--list .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-video-section--list .mdx-video-section__list{display:flex;flex-direction:column;gap:1.5rem 0;grid-column:2/3}.mdx-video-section--list .mdx-video-section__list>a{display:block;-webkit-text-decoration:none;text-decoration:none}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>a>.mdx-video-section__item-title{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:1rem;color:rgb(var(--lsd-text-primary));display:flex;gap:12px;margin-top:1.5rem;padding:3px 11px;width:fit-content}.mdx-spec-card .mdx-spec-card__link,.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>a,.sidebarItemLink_mo7H:hover{-webkit-text-decoration:none;text-decoration:none}.content_knG7 a,.footerLink_sh7M,.linkContent__x3v:hover,.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>a:hover{-webkit-text-decoration:underline;text-decoration:underline}.mdx-table-view-table{padding:20px;width:100%}.mdx-table-view-table-title{margin-bottom:32px}.mdx-table-view-row{display:flex;gap:20px;padding:24px 0}.mdx-numbers-view-header{display:flex;flex-direction:column;gap:32px;width:100%}.mdx-numbers-view-content{display:flex;flex-direction:column;gap:16px;width:100%}.mdx-numbers-view-row{display:flex;gap:32px}.mdx-numbers-view-number{align-items:center;border:1px solid rgb(var(--lsd-border-primary));border-radius:50%;display:flex;font-size:14px;height:32px;justify-content:center;padding:4px 12px;width:32px}.mdx-numbers-view-description{margin-top:16px;width:100%}.mdx-spec-card{padding-block:16px}.mdx-spec-card .mdx-spec-card__container{display:flex;flex-direction:column;gap:16px}.mdx-spec-card .mdx-spec-card__button{height:28px;margin:0;padding-left:12px;width:fit-content}.mdx-spec-card .mdx-spec-card__button-icon{padding:7px 10px}.skipToContent_fXgn{background-color:var(--ifm-background-surface-color);color:var(--ifm-color-emphasis-900);left:100%;padding:calc(var(--ifm-global-spacing)/2) var(--ifm-global-spacing);position:fixed;top:1rem;z-index:calc(var(--ifm-z-index-fixed) + 1)}.skipToContent_fXgn:focus{box-shadow:var(--ifm-global-shadow-md);left:1rem}.closeButton_CVFx{line-height:0;padding:0}.content_knG7{font-size:85%;padding:5px 0}.content_knG7 a{color:inherit}.announcementBar_mb4j{align-items:center;background-color:var(--ifm-color-white);border-bottom:1px solid var(--ifm-color-emphasis-100);color:var(--ifm-color-black);display:flex;height:var(--docusaurus-announcement-bar-height)}.announcementBarPlaceholder_vyr4{flex:0 0 10px}.announcementBarClose_gvF7{align-self:stretch;flex:0 0 30px}.iconExternalLink_nPIU{margin-left:.3rem}.docMainContainer_TBSr,.docRoot_UBD9{display:flex;width:100%}.docsWrapper_hBAB{display:flex;flex:1 0 auto}.footerLogoLink_BH7S{opacity:.5;transition:opacity var(--ifm-transition-fast) var(--ifm-transition-timing-default)}.firstRow_ar1q{border-top:1px solid rgb(var(--lsd-text-primary));margin-bottom:96px!important}.firstRow_ar1q,.secondRow__ww3{display:flex;flex-direction:row}.secondRow__ww3{border-top:1px solid rgb(var(--lsd-border-primary));padding-bottom:16px!important}.backToTop_wDfN{bottom:16px;position:absolute;width:fit-content}.l-modal,.l-modal__backdrop{position:fixed;top:0;left:0}.footerLink_sh7M{height:fit-content;width:fit-content}.toggle_K23S{border:1px solid rgb(var(--lsd-border-primary));height:32px;width:32px}.toggle_K23S svg{height:14px;width:14px}.toggleButton_dl49{align-items:center;border-radius:50%;display:flex;height:100%;justify-content:center;transition:background var(--ifm-transition-fast);width:100%}.toggleButton_dl49:hover{background:var(--ifm-color-emphasis-200)}.toggleButtonDisabled_AAS_{cursor:not-allowed}.darkNavbarColorModeToggle_X3D1:hover{background:var(--ifm-color-gray-800)}.l-modal{height:100vh;opacity:0;transition:.3s;visibility:hidden;width:100%;z-index:9999}.l-modal__container{display:grid;gap:1rem;grid-template-columns:repeat(24,1fr);margin:0 auto;max-width:1376px;overflow:auto}.l-modal--open{opacity:1;visibility:visible}.l-modal__content{background:rgb(var(--lsd-surface-primary));border:1px solid rgb(var(--lsd-border-primary));grid-column:8/19;height:fit-content;margin:64px 0;max-height:80vh;padding:7px;z-index:9998}.groups_p1lF,.root_ONDG{max-height:60vh;overflow-y:auto}.l-modal__backdrop{background-color:rgba(var(--lsd-surface-primary),.6);height:100%;width:100%;z-index:9997}.modal_kLVz .header_QwCa{align-items:center;display:flex;flex-direction:row;gap:.5rem}.root_EFVO{padding:32px}.root_ONDG{padding:24px}.title_uwS_{font-size:.75rem!important}.item_RHYF{align-items:center;display:flex;flex-direction:row;margin-top:1rem}.item_RHYF .itemTitle_jtAv{flex-grow:1;overflow:hidden;padding-left:1rem;text-overflow:ellipsis;white-space:nowrap}.linkContent__x3v,.root_JWD1{align-items:center;display:flex}.item_RHYF .itemTitle_jtAv mark,.root_JWD1 mark{background:none;color:inherit}.root_xZfc{padding:8px 8px 0;position:relative;width:100%}.collapse_LWBd button,.textField_af43{width:100%!important}.textField_af43 button svg{height:auto;max-height:100%!important;width:14px}.root_N57j>div:first-child{color:#828285;padding:0 24px}.root_N57j>div:first-child,.root_N57j>div:first-child *{font-size:.75rem}.root_N57j ul li{margin-top:1rem}.root_JWD1{padding:0 24px}.root_JWD1>div{margin-left:1rem}.breadcrumbsContainer_RLvU{--ifm-breadcrumb-size-multiplier:0.8;margin-bottom:24px}.codeBlockContainer_Ckt0,.codeBlockContainer_EB2s{margin-bottom:var(--ifm-leading);color:var(--prism-color)}.root_Ooa0.level1_fpUf{display:block;padding-left:32px}.icon_sZn2{height:auto;width:16px}.icon_sZn2.fill_fkan,.icon_sZn2.fill_fkan *{fill:currentColor}.icon_sZn2.stroke_j3wH,.icon_sZn2.stroke_j3wH *{stroke:currentColor}[data-theme=light] .icon_sZn2{color:#000}[data-theme=dark] .icon_sZn2{color:#fff}.groups_p1lF{display:flex;flex-direction:column;gap:16px}.sidebar_re4s,.tableOfContents_bqdL{max-height:calc(100vh - var(--ifm-navbar-height) - 2rem)}[data-theme=dark] .divider_eGUz>div{opacity:.1}[data-theme=dark] .topBar_Dtew button{background-color:var(--ifm-button-color)}.codeBlockContainer_Ckt0{background:var(--prism-background-color);border-radius:var(--ifm-code-border-radius);box-shadow:var(--ifm-global-shadow-lw)}.linkContent__x3v{gap:6px;-webkit-text-decoration:none;text-decoration:none}.linkContent__x3v:hover{text-decoration-color:rgba(var(--lsd-text-primary),.6);text-underline-offset:.3rem}.externalLinkIcon_Qfue{width:8px}.codeBlockContainer_EB2s{background:#0000;border-radius:0}.dropdownNavbarItem_o23I{border:1px solid rgb(var(--lsd-border-primary));height:28px;padding:6px 12px}.codeBlockContent_biex{border-radius:inherit;direction:ltr;position:relative}.codeBlockTitle_Ktv7{border-bottom:1px solid var(--ifm-color-emphasis-300);border-top-left-radius:inherit;border-top-right-radius:inherit;font-size:var(--ifm-code-font-size);font-weight:500;padding:.75rem var(--ifm-pre-padding)}.codeBlock_TWhw,.codeBlock_bY9V{--ifm-pre-background:var(--prism-background-color);margin:0;padding:0}.codeBlockTitle_Ktv7+.codeBlockContent_biex .codeBlock_bY9V,.codeBlockTitle_sjMo+.codeBlockContent_ugSV .codeBlock_TWhw{border-top-left-radius:0;border-top-right-radius:0}.codeBlockLines_LDrR,.codeBlockLines_e6Vv{float:left;font:inherit;min-width:100%;padding:var(--ifm-pre-padding)}.codeBlockLinesWithNumbering_bsRF,.codeBlockLinesWithNumbering_o6Pm{display:table;padding:var(--ifm-pre-padding) 0}.buttonGroup_Qu4e,.buttonGroup__atx{column-gap:.2rem;display:flex;position:absolute;right:calc(var(--ifm-pre-padding)/2);top:calc(var(--ifm-pre-padding)/2)}.buttonGroup__atx button{align-items:center;background:var(--prism-background-color);border:1px solid var(--ifm-color-emphasis-300);border-radius:var(--ifm-global-radius);color:var(--prism-color);display:flex;line-height:0;opacity:0;padding:.4rem}.buttonGroup__atx button:focus-visible,.buttonGroup__atx button:hover{opacity:1!important}.theme-code-block:hover .buttonGroup__atx button{opacity:.4}.anchorWithStickyNavbar_LWe7{scroll-margin-top:calc(var(--ifm-navbar-height) + .5rem)}.anchorWithHideOnScrollNavbar_WYt5{scroll-margin-top:.5rem}.hash-link{opacity:0;padding-left:.5rem;transition:opacity var(--ifm-transition-fast);-webkit-user-select:none;user-select:none}.hash-link:before{content:"#"}:where(:root){--docusaurus-highlighted-code-line-bg:#484d5b}:where([data-theme=dark]){--docusaurus-highlighted-code-line-bg:#646464}.theme-code-block-highlighted-line{background-color:var(--docusaurus-highlighted-code-line-bg);display:block;margin:0 calc(var(--ifm-pre-padding)*-1);padding:0 var(--ifm-pre-padding)}.codeLine_lJS_{counter-increment:a;display:table-row}.codeLineNumber_Tfdd{background:var(--ifm-pre-background);display:table-cell;left:0;overflow-wrap:normal;padding:0 var(--ifm-pre-padding);position:sticky;text-align:right;width:1%}.codeLineNumber_Tfdd:before{content:counter(a);opacity:.4}.codeLineContent_feaV{padding-right:var(--ifm-pre-padding)}.navbarHideable_OoEf{transition:transform var(--ifm-transition-fast) ease}.navbarHidden_zoxl{transform:translate3d(0,calc(-100% - 2px),0)}.theme-code-block:hover .copyButtonCopied_YovB{opacity:1!important}.copyButtonIcons_an20{align-items:center;display:flex;justify-content:center;position:relative}.copyButtonIcon_ZL7v,.copyButtonSuccessIcon_P2h8{fill:currentColor;height:14px;opacity:inherit;transition:all var(--ifm-transition-fast) ease;width:14px}.copyButtonSuccessIcon_P2h8{color:#00d600;opacity:0}.copyButtonCopied_YovB .copyButtonSuccessIcon_P2h8{opacity:1;transition-delay:75ms}.rightSection_CLeF{align-items:center;display:flex;justify-content:flex-end;white-space:nowrap}.iconButtonGroup_ktNv button:not(:last-child){border-right:1px solid!important;margin-right:-1px!important}.wordWrapButtonIcon_Bwma{height:1.2rem;width:1.2rem}.authorSocialIcon_XYv3,.authorSocialLink_owbf,.authorSocials_rSDt{height:var(--docusaurus-blog-social-icon-size)}.mainWrapper_z2l0{display:flex;flex:1 0 auto;flex-direction:column}.docusaurus-mt-lg{margin-top:3rem}#__docusaurus{min-height:100%}#__docusaurus,.root_jG9K{display:flex;flex-direction:column}.root_jG9K{align-items:center;padding:1rem!important;width:100%}.authorSocialIcon_XYv3,.authorSocialLink_owbf{width:var(--docusaurus-blog-social-icon-size)}.root_jG9K>div{max-width:796px!important;width:100%}.root_jG9K>div>div{align-items:flex-start;display:flex;flex-direction:column;gap:1rem;padding:0!important}.root_jG9K a{margin-top:2.5rem}.sidebar_re4s{overflow-y:auto;position:sticky;top:calc(var(--ifm-navbar-height) + 2rem)}.authorSocials_rSDt,.authorTitle_nd0D{overflow:hidden;-webkit-box-orient:vertical}.sidebarItemTitle_pO2u{font-size:var(--ifm-h3-font-size);font-weight:var(--ifm-font-weight-bold)}.sidebarItemList_Yudw{font-size:.9rem}.sidebarItem__DBe{margin-top:.7rem}.sidebarItemLink_mo7H{color:var(--ifm-font-color-base);display:block}.sidebarItemLinkActive_I1ZP{color:var(--ifm-color-primary)!important}.yearGroupHeading_rMGB{margin-bottom:.4rem;margin-top:1.6rem}.yearGroupHeading_QT03{margin:1rem .75rem .5rem}.blogHeader_QORd{border-bottom:1px solid rgb(var(--lsd-border-primary));display:flex;flex-direction:column;gap:8px!important;left:0;margin-bottom:40px!important;margin-inline:var(--content-padding);padding-bottom:24px;padding-top:16px}.blogDescription_dds8,.blogTitle_KnCm{grid-column:span 24}[data-theme=dark] .githubSvg_Uu4N,[data-theme=dark] .instagramSvg_YC40,[data-theme=dark] .threadsSvg_PTXY,[data-theme=dark] .xSvg_y3PF{fill:var(--light)}[data-theme=light] .githubSvg_Uu4N,[data-theme=light] .instagramSvg_YC40,[data-theme=light] .threadsSvg_PTXY,[data-theme=light] .xSvg_y3PF{fill:var(--dark)}.authorSocials_rSDt{align-items:center;display:flex;flex-wrap:wrap;line-clamp:1;-webkit-line-clamp:1}.authorSocialLink_owbf,.authorSocials_rSDt{line-height:0}.authorSocialLink_owbf{margin-right:.4rem}.authorImage_XqGP{--ifm-avatar-photo-size:3.6rem}.author-as-h1_n9oJ .authorImage_XqGP{--ifm-avatar-photo-size:7rem}.author-as-h2_gXvM .authorImage_XqGP{--ifm-avatar-photo-size:5.4rem}.authorDetails_lV9A{align-items:flex-start;display:flex;flex-direction:column;justify-content:space-around}.authorName_yefp{display:flex;flex-direction:row;font-size:1.1rem;line-height:1.1rem}.author-as-h1_n9oJ .authorName_yefp{display:inline;font-size:2.4rem;line-height:2.4rem}.author-as-h2_gXvM .authorName_yefp{display:inline;font-size:1.4rem;line-height:1.4rem}.authorTitle_nd0D{display:-webkit-box;font-size:.8rem;line-height:1rem;line-clamp:1;-webkit-line-clamp:1}.author-as-h1_n9oJ .authorTitle_nd0D{font-size:1.2rem;line-height:1.6rem}.author-as-h2_gXvM .authorTitle_nd0D{font-size:1rem;line-height:1.3rem}.authorBlogPostCount_iiJ5{background:var(--ifm-color-secondary);border-radius:var(--ifm-global-radius);color:var(--ifm-color-black);font-size:.8rem;line-height:1.2;margin-left:.3rem;padding:.1rem .4rem}.authorListItem_n3yI{list-style-type:none;margin-bottom:2rem}.authorCol_y4tx{flex-grow:1!important;max-width:inherit!important}.imageOnlyAuthorRow_n8pT{display:flex;flex-flow:row wrap}.imageOnlyAuthorCol_OswQ{margin-left:.3rem;margin-right:.3rem}.container_PkUo{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-body2-fontSize)!important;font-weight:var(--lsd-body2-fontWeight)!important;line-height:var(--lsd-body2-lineHeight)!important;margin-bottom:0!important;margin-top:32px!important}.blogContainer_wIfk{margin-bottom:16px!important;margin-top:8px!important}.container_PkUo>:not(:last-child):after{content:"•";display:inline-block;font-size:10px;margin-inline:.75rem}.authors_dZ4g>div{align-items:flex-end;display:inline-flex;flex-direction:row;max-width:unset;padding:0;width:auto}.authors_dZ4g>div .avatar{margin:0!important}.authors_dZ4g>div:not(:last-child):after{content:",";display:inline-block;margin-right:4px}.authors_dZ4g .avatar__name{line-height:0}.blogPostSubtitle_ysb0{margin-top:12px}.nav_cMpg,.root_QACb{margin:0 auto;max-width:var(--container-max-width)}.codeBlockContent_ugSV,.wrapper_SWrM{position:relative}.codeBlockTitle_sjMo{background:var(--prism-background-color);border-bottom:1px solid var(--ifm-color-primary);border-top-left-radius:inherit;border-top-right-radius:inherit;font-size:var(--ifm-code-font-size);font-weight:500;padding:.75rem var(--ifm-pre-padding)}.buttonGroup_Qu4e button{align-items:center;background:var(--prism-background-color);border:1px solid rgb(var(--lsd-border-primary));color:var(--prism-color);display:flex;height:28px;justify-content:center;line-height:0;opacity:0;width:28px}.buttonGroup_Qu4e button:focus-visible,.buttonGroup_Qu4e button:hover{opacity:1!important}.img_ev3q{height:auto}.backdrop_Ifvc{background:rgb(var(--lsd-surface-primary));z-index:201}.backdrop_Ifvc,.navWrapper_ybYI{height:100%;left:0;opacity:0;position:fixed;top:0;visibility:hidden;width:100%}.navWrapper_ybYI{background:#0000;z-index:203}.navWrapper_ybYI>*{pointer-events:auto}.nav_cMpg{align-items:center;background:rgb(var(--lsd-surface-primary));display:flex;flex-direction:row;height:var(--ifm-navbar-height);justify-content:space-between;padding:var(--ifm-navbar-padding-vertical) var(--ifm-navbar-padding-horizontal)}.visible_ynAX{opacity:1!important;visibility:visible!important}#__docusaurus-base-url-issue-banner-container,.docItemContainer_hrrU .tocMobile_imaF:not(:first-of-type),.toc_pP_5,.wrapper_SWrM .fullscreenButton_Bocn{display:none}.wrapper_SWrM:not(.active_qZD5) .fullscreenButton_Bocn{background:rgb(var(--lsd-surface-primary));bottom:8px;position:absolute;right:8px;transition:.3s}.root_QACb{padding:0;width:100%}.admonition_ntHH{border:1px solid var(--ifm-alert-border-color);display:flex;gap:18px;margin-bottom:24px;padding:18px}.admonitionHeading_JPfy{font-size:1.25rem!important;font-weight:400;line-height:1.5rem!important;margin-bottom:.3rem;text-transform:capitalize}.admonitionHeading_JPfy code{text-transform:none}.admonitionIcon_BGV6{display:inline-block}.admonitionIcon_BGV6 svg{display:inline-block;height:16px;width:16px}.admonitionIcon_BGV6 svg *{stroke:none!important;fill:var(--ifm-alert-border-color)}.admonitionContent_iozl{display:block;margin-top:12px}.tag_dtsN{border:1px solid var(--docusaurus-tag-list-border);transition:border var(--ifm-transition-fast)}.tag_dtsN:hover{--docusaurus-tag-list-border:var(--ifm-link-color);-webkit-text-decoration:none;text-decoration:none}.tagRegular_SGIC{border-radius:var(--ifm-global-radius);font-size:90%;padding:.2rem .5rem .3rem}.tagWithCount_gIvh{align-items:center;border-left:0;display:flex;padding:0 .5rem 0 1rem;position:relative}.tagWithCount_gIvh:after,.tagWithCount_gIvh:before{border:1px solid var(--docusaurus-tag-list-border);content:"";position:absolute;top:50%;transition:inherit}.tagWithCount_gIvh:before{border-bottom:0;border-right:0;height:1.18rem;right:100%;transform:translate(50%,-50%) rotate(-45deg);width:1.18rem}.tagWithCount_gIvh:after{border-radius:50%;height:.5rem;left:0;transform:translateY(-50%);width:.5rem}.tagWithCount_gIvh span{background:var(--ifm-color-secondary);border-radius:var(--ifm-global-radius);color:var(--ifm-color-black);font-size:.7rem;line-height:1.2;margin-left:.3rem;padding:.1rem .4rem}.root_UH70{align-items:center;display:flex;flex-direction:row}.tags_LT8x{display:inline;margin:0 0 0 1rem!important;padding:0!important}.tag_NdBD:not(:first-child){margin-left:.5rem}.tag_NdBD a{margin:0!important;padding:0!important}.lastUpdated_JAkA{font-size:smaller;font-style:italic;margin-top:.2rem}.blogPostFooterDetailsFull_FF8l{flex-direction:column}.readMoreLink_cNfS span{color:rgb(var(--lsd-text-secondary))!important}.tableOfContents_bqdL{overflow-y:auto;position:sticky;top:calc(var(--ifm-navbar-height) + 1rem)}.tocCollapsibleButton_dxRj{align-items:center;border:1px solid rgb(var(--lsd-border-primary));display:flex;font-size:inherit;justify-content:space-between;padding:.4rem .8rem;width:100%}.tocCollapsibleButtonExpanded_TSyC:after,.tocCollapsibleExpanded_zTjk{transform:none}.tocCollapsible_ROek{background-color:var(--ifm-menu-color-background-active);border-radius:var(--ifm-global-radius);margin:1rem 0}.tocCollapsibleContent_Qsjj>ul{border-left:none;border:1px solid rgb(var(--lsd-border-primary))!important;border-top:none!important;font-size:15px;padding:.2rem 0}.tocCollapsibleContent_Qsjj ul li{margin:.4rem .8rem}.tocCollapsibleContent_Qsjj a{display:block;-webkit-text-decoration:var(--ifm-link-text-decoration);text-decoration:var(--ifm-link-text-decoration)}.badge_AsjZ{background:#0000}.docItemGrid_SzoZ{display:grid;gap:16px;grid-template-columns:repeat(14,1fr)}.gap1_XuuQ{grid-column:span 1}.tocMobile_imaF{margin-bottom:2rem;margin-top:-.5rem}.tocMobile_imaF>div{display:block!important;margin:0}.docItemContainer_hrrU h1{margin-bottom:40px!important}.docItemContainer_hrrU h2,.docItemContainer_hrrU h3{margin-top:32px!important}@media (min-width:0px){.mdx-box{--mdx-box-top:var(--mdx-box-top-xs);--mdx-box-bottom:var(--mdx-box-bottom-xs)}}@media (min-width:576px){.mdx-box{--mdx-box-top:var(--mdx-box-top-sm);--mdx-box-bottom:var(--mdx-box-bottom-sm)}}@media (min-width:768px){.blog-archive-page .main-wrapper main>section .container>.row{display:grid;gap:64px 24px;grid-template-columns:repeat(24,1fr)}.blog-archive-page .main-wrapper main>section .container>.row>.col{grid-column:span 12;margin:0!important;max-width:unset;padding:0 8.3333333333% 0 0}.mdx-box{--mdx-box-top:var(--mdx-box-top-md);--mdx-box-bottom:var(--mdx-box-bottom-md)}.mdx-scroll-buttons__button--with-label{gap:12px;min-width:83px;padding:5px 11px 5px 9px!important;width:auto!important}.mdx-scroll-buttons__button--with-label:first-of-type,.mdx-scroll-buttons__button--with-label:last-of-type{justify-content:flex-start}.mdx-scroll-buttons--spaced>div{gap:0 1rem;justify-content:space-between}.mdx-scroll-buttons--spaced>div>button:not(:last-child){border-right:1px solid rgb(var(--lsd-border-primary))!important}}@media (min-width:997px){.full-width>div .main-wrapper main>div>div{grid-template-columns:unset!important}.main-wrapper .container{padding-bottom:124px!important}.main-wrapper>div>div{display:grid;gap:1rem;grid-template-columns:repeat(16,45px);margin:auto;width:fit-content}.main-wrapper>div>div aside{border:none;grid-column:1/5;width:auto}.main-wrapper>div>div main{grid-column:6/16;margin-top:48px;max-width:none}.navbar__left{flex-basis:8.3333333333%;flex:0 1 auto}.navbar__left-items{flex:1 0;margin:auto}nav.menu{margin-top:48px;padding-top:16px}.container{padding-inline:0!important}aside a[tabindex="-1"] img{margin-left:14px}.theme-doc-sidebar-menu .menu__link{padding-left:8px}.navbar .lsd-icon-button--medium{height:28px!important;width:28px!important}.blog-wrapper>.container>.row{display:grid;gap:16px;grid-template-columns:repeat(24,1fr)}.blog-wrapper>.container>.row>aside{grid-column:1/5}.blog-wrapper>.container>.row>main{grid-column:6/20}.blog-wrapper:not(.blog-wrapper--index)>.container>.row>div:last-child{grid-column:21/25;max-width:unset}.mdx-box{--mdx-box-top:var(--mdx-box-top-lg);--mdx-box-bottom:var(--mdx-box-bottom-lg)}.mdx-cta-section__header__mobile-button{display:none}.mdx-feature-list--bottom-aligned .mdx-feature-list__feature{min-height:332px}.mdx-feature-list--bottom-aligned .mdx-feature-list__feature-description{display:flex;flex:1 0;flex-direction:column;justify-content:flex-end}.mdx-feature-list--top-aligned .mdx-feature-list__feature-description{margin-top:40px}.mdx-feature-list__feature{padding:1rem;position:relative}.mdx-feature-list__feature-inner{height:100%}.mdx-feature-list__feature-border{background:rgb(var(--lsd-border-primary));height:calc(100% - 2rem);position:absolute;right:0;top:1rem;width:1px}.mdx-feature-list__feature:nth-child(odd){padding-left:0}.mdx-feature-list__feature:nth-child(2n){padding-right:0}.mdx-feature-list__feature:nth-child(2n) .mdx-feature-list__feature-border,.mdx-video-section__header__mobile-button,.tocMobile_ITEo{display:none}:root{--docusaurus-announcement-bar-height:30px}.announcementBarClose_gvF7,.announcementBarPlaceholder_vyr4{flex-basis:50px}.menuHtmlItem_M9Kj{padding:var(--ifm-menu-link-padding-vertical) var(--ifm-menu-link-padding-horizontal)}.menu_NjKY{flex-grow:1;padding-top:16px}@supports (scrollbar-gutter:stable){.menu_NjKY{scrollbar-gutter:stable}}.menuWithAnnouncementBar_U5pk{margin-bottom:var(--docusaurus-announcement-bar-height)}.sidebar_UEyd{display:flex;flex-direction:column;height:100%;padding-top:var(--ifm-navbar-height)}.sidebarWithHideableNavbar__00c{padding-top:0!important}.sidebarHidden_F1ZE{opacity:0;visibility:hidden}.sidebarLogo_Ydl9{align-items:center;color:inherit!important;display:flex!important;margin:0 var(--ifm-navbar-padding-horizontal);max-height:var(--ifm-navbar-height);min-height:var(--ifm-navbar-height);-webkit-text-decoration:none!important;text-decoration:none!important}.sidebarLogo_Ydl9 img{height:2rem;margin-right:.5rem}.expandButton_TmdG{align-items:center;background-color:var(--docusaurus-collapse-button-bg);display:flex;height:100%;justify-content:center;position:absolute;right:0;top:0;transition:background-color var(--ifm-transition-fast) ease;width:100%}.expandButton_TmdG:focus,.expandButton_TmdG:hover{background-color:var(--docusaurus-collapse-button-bg-hover)}.expandButtonIcon_i1dp{transform:rotate(0)}[dir=rtl] .expandButtonIcon_i1dp{transform:rotate(180deg)}.docSidebarContainer_YfHR{border-right:1px solid var(--ifm-toc-border-color);clip-path:inset(0);display:block;margin-top:calc(var(--ifm-navbar-height)*-1);transition:width var(--ifm-transition-fast) ease;width:var(--doc-sidebar-width);will-change:width}.docSidebarContainerHidden_DPk8{cursor:pointer;width:var(--doc-sidebar-hidden-width)}.sidebarViewport_aRkj{height:100%;max-height:100vh;position:sticky;top:0}.docMainContainer_TBSr{flex-grow:1;max-width:calc(100% - var(--doc-sidebar-width))}.docMainContainerEnhanced_lQrH{max-width:calc(100% - var(--doc-sidebar-hidden-width))}.docItemWrapperEnhanced_JWYK{max-width:calc(var(--ifm-container-width) + var(--doc-sidebar-width))!important}.navbarSearchContainer_Bca1{padding:var(--ifm-navbar-item-padding-vertical) var(--ifm-navbar-item-padding-horizontal)}.blogHeader_QORd{margin-inline:0}.wrapper_SWrM:not(.active_qZD5) .fullscreenButton_Bocn{background:rgb(var(--lsd-surface-primary));bottom:8px;display:flex;opacity:0;position:absolute;right:8px;transition:.3s;visibility:hidden}.wrapper_SWrM:not(.active_qZD5):hover .fullscreenButton_Bocn{opacity:1;visibility:visible}.lastUpdated_JAkA{text-align:right}.docItemGrid_SzoZ{display:grid;gap:var(--doc-grid-gap);grid-template-columns:repeat(11,1fr)}.docItemCol_F52z{grid-column:span 11}}@media (min-width:997px)and (max-width:1199px){.main-wrapper .container>div>div,.navbar{padding-inline:0!important}.main-wrapper,.navbar{width:960px!important}.events .main-wrapper>div aside{grid-column:1/1}.events .main-wrapper>div main,.navbar__inner{grid-column:1/17}.plugin-id-root-pages main{grid-column:4/14!important}.main-wrapper .container{padding-bottom:96px}.main-wrapper>div{width:960px}.navbar{display:grid;gap:1rem;grid-template-columns:repeat(16,45px);margin:auto}.navbar__left-items{display:flex;justify-content:center}.main-wrapper{margin:0 auto!important}}@media (min-width:1200px){.main-wrapper>div>div main>.container,.main-wrapper>main>.row>.col,.navbar{padding-inline:0!important}.plugin-id-root-pages main{grid-column:4/16!important}.footer{padding:0!important}.main-wrapper>div>div{display:grid;gap:1rem;grid-template-columns:repeat(16,1fr);margin-inline:unset;width:unset}.main-wrapper>div>div aside{border:none;grid-column:1/4;width:auto}.main-wrapper>div>div main{grid-column:5/17;max-width:none}.theme-doc-toc-desktop{top:calc(var(--ifm-navbar-height))!important}.navbar__inner{display:grid;gap:16px;grid-template-columns:repeat(24,1fr);padding:0}.navbar__left{align-items:center;display:flex;flex-direction:row;grid-column:1/6}.navbar__left-items{grid-column:6/20}.navbar__right-items{flex-direction:row;gap:16px;grid-column:20/25;justify-content:flex-end}.blog-archive-page .main-wrapper main>section .container>.row{display:grid;gap:64px 24px;grid-template-columns:repeat(24,1fr)}.blog-archive-page .main-wrapper main>section .container>.row>.col{grid-column:span 8;margin:0!important;max-width:unset;padding:0 12.5% 0 0}.mdx-box{--mdx-box-top:var(--mdx-box-top-xl);--mdx-box-bottom:var(--mdx-box-bottom-xl)}.docItemGrid_SzoZ{display:grid;gap:var(--doc-grid-gap);grid-template-columns:repeat(12,1fr)}.docItemCol_F52z{grid-column:span 10}.tocMobile_imaF{display:none}}@media (min-width:1440px){.container{max-width:var(--ifm-container-width-xl)}}@media (max-width:1199px){.main-wrapper main>.container{padding-bottom:calc(var(--desktop-footer-gap) - var(--ifm-footer-padding-vertical) - var(--doc-grid-gap))!important}.header-github-link:before{margin-right:16px}.toc_pP_5{display:none}}@media (max-width:997px);@media (max-width:996px){.navbar-sidebar__close,.navbar__toggle{border:1px solid rgb(var(--lsd-border-primary))}.blog-wrapper h1[itemprop=headline],.mdx-cta-button .lsd-button--large,.mdx-cta-section__description,.mdx-cta-section__title,.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__link>div{font-family:var(--lsd-typography-generic-font-family)!important}.col{--ifm-col-width:100%;flex-basis:var(--ifm-col-width);margin-left:0}.footer{--ifm-footer-padding-horizontal:0;padding-bottom:0!important;--ifm-footer-padding-horizontal:var(--content-padding)}.desktop,.footer__link-separator,.navbar__item,.navbar__left-items{display:none}.footer__col{margin-bottom:calc(var(--ifm-spacing-vertical)*3)}.footer__link-item{display:block;width:max-content}.hero{padding-left:0;padding-right:0}.navbar>.container,.navbar>.container-fluid,.root_xZfc{padding:0}.navbar__toggle{display:inherit}.navbar__search-input{width:9rem}.mdx-numbers-view,.mdx-section-header,.pills--block,.tabs--block{flex-direction:column}:root{--ifm-background-color:rgb(var(--lsd-surface-primary))}.navbar__color-mode-toggle{display:none!important}.main-wrapper main>.container{padding-bottom:calc(var(--mobile-footer-gap) - var(--ifm-footer-padding-vertical) - var(--doc-grid-gap))!important}.grid{grid-template-columns:repeat(2,1fr)}.footer-grid,.mdx-feature-list__list,.mdx-profile-card{grid-template-columns:1fr}.navbar__toggle{align-items:center;box-sizing:border-box;display:flex;height:32px;justify-content:center;margin:0;width:32px}.navbar__toggle svg{height:14px;width:14px}.main-wrapper>div,.mdx-cta-section--full-width .mdx-cta-section__container,.mdx-cta-section--list,.mdx-video-section--full-width .mdx-video-section__container,.mdx-video-section--list,.tocMobile_imaF{display:block}.theme-admonition{display:block!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu{padding-inline:0}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__list{display:flex;flex-direction:column;gap:8px;padding:var(--content-padding)}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__link{border-left:none;padding-left:0}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .menu__link>div{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important;-webkit-text-decoration:none;text-decoration:none}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer{margin-top:3rem;padding:0!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer>div{display:flex!important;flex-direction:column!important;gap:32px!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer>button{display:none}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer .container.container-fluid{margin-bottom:16px!important;padding-inline:var(--content-padding)}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer .container.container-fluid>.footer__links{gap:0!important}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer>div[class*=secondRow]{margin:var(--content-padding)}.navbar-sidebar:not(.navbar-sidebar--show-secondary) .footer .footer__links{gap:16px!important}.navbar-sidebar__brand{height:60px;padding-block:0}.navbar-sidebar__brand>button{height:32px;width:32px}.navbar-sidebar__brand>button:first-of-type{margin-left:auto;margin-right:0!important}.navbar-sidebar__brand>button:not(:last-of-type){border-right:none}.navbar-sidebar__close{align-items:center;display:flex;justify-content:center;margin-left:unset}.blog-divider{margin-block:32px!important}.blog-wrapper main{padding-inline:var(--content-padding)}.blog-wrapper article footer{align-items:flex-start;flex-direction:column}.blog-wrapper article footer .read-more-link{margin-top:24px}.blog-wrapper article footer .read-more-link,.blog-wrapper article footer .read-more-link button{flex-basis:100%;width:100%}.blog-wrapper h1[itemprop=headline]{font-size:var(--lsd-h2-fontSize)!important;font-weight:var(--lsd-h2-fontWeight)!important;line-height:var(--lsd-h2-lineHeight)!important}.mdx-cta-button .lsd-button--large{font-size:var(--lsd-label2-fontSize)!important;font-weight:var(--lsd-label2-fontWeight)!important;line-height:var(--lsd-label2-lineHeight)!important;padding:5px 11px}.mdx-cta-section--align-center .lsd-button--large{padding:10px 40px}.mdx-cta-section,.mdx-video-section{padding:16px 0 0}.mdx-cta-section--title-button .mdx-cta-section__description,.mdx-video-section--title-button .mdx-video-section__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight)}.mdx-cta-section--full-width .mdx-cta-section__title,.mdx-cta-section--simple .mdx-cta-section__title,.mdx-video-section--full-width .mdx-video-section__title,.mdx-video-section--simple .mdx-video-section__title{margin-top:1.5rem}.mdx-cta-section--list .mdx-cta-section__list,.mdx-roadmap__timeline{margin-top:4rem}.mdx-cta-section--list .mdx-cta-section__list>*{border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>.mdx-cta-section__item-title{font-size:.875rem!important;line-height:1.25rem!important}.mdx-cta-section--list .mdx-cta-section__list .mdx-cta-section__list-item>.mdx-cta-section__item-description{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.colorModeToggle_GSaI,.dropdownNavbarItem_o23I,.l-modal__backdrop,.mdx-cta-section__header__desktop-button,.sidebar_re4s,.tableOfContents_bqdL{display:none}.mdx-cta-section__header__mobile-button{margin-top:24px}.mdx-cta-section__title{font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.mdx-cta-section__description{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important}.mdx-feature-list .mdx-feature-list__title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-subtitle2-fontSize);font-weight:var(--lsd-subtitle2-fontWeight);line-height:var(--lsd-subtitle2-lineHeight);padding:1rem 0}.mdx-feature-list--cta-bottom .mdx-feature-list__extra{margin-top:2rem;padding:0}.mdx-feature-list__header h2,.mdx-roadmap .mdx-cta-section__title,.mdx-section-header h2{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.lsd-button span,.mdx-hero-description{line-height:var(--lsd-body2-lineHeight)!important}.mdx-feature-list__feature{border:none;border-top:1px solid rgb(var(--lsd-border-primary));padding:1.5rem 0}.mdx-feature-list__feature .mdx-feature-list__feature-index--circle{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-subtitle2-fontSize);font-weight:var(--lsd-subtitle2-fontWeight);height:1.625rem;line-height:var(--lsd-subtitle2-lineHeight);width:1.625rem}.mdx-feature-list__feature .mdx-feature-list__feature-title{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight);margin-top:1rem}.mdx-feature-list__feature .mdx-feature-list__feature-description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-body1-fontSize);font-weight:var(--lsd-body1-fontWeight);line-height:var(--lsd-body1-lineHeight);margin-top:3rem}.blogPostSubtitle_ysb0,.mdx-hero-description,.mdx-roadmap .mdx-cta-section__description,.mdx-section-header__description,.mdx-section-header__title,.mdx-video-section__title{font-family:var(--lsd-typography-generic-font-family)!important}.mdx-feature-list__feature:nth-child(odd) .mdx-feature-list__feature-inner{border:none}.lsd-button,.mdx-hero-action button{padding:6px 12px!important}.mdx-hero-actions{align-items:center;display:flex;flex-direction:row;justify-content:stretch;width:100%;z-index:0}.mdx-hero-actions>*{flex-basis:50%}.mdx-hero-actions>a,.mdx-hero-actions>a>button{display:block;width:100%}.mdx-hero-description{font-size:var(--lsd-body2-fontSize)!important;font-weight:var(--lsd-body2-fontWeight)!important;margin-top:16px}.mdx-hero-info{padding:28px 0}.mdx-hero-title,.mdx-numbers-view-section-title{font-size:28px!important}.mdx-hero-video>*{height:var(--hero-video-height-mobile);min-height:var(--hero-video-min-height-mobile);transform:translateY(var(--hero-video-offset-y-mobile))}.mdx-hero-video img,.mdx-hero-video video{max-width:calc((100vw - 32px)*var(--hero-video-scale-mobile))}.mdx-logo-carousel .mdx-logo-carousel__inner{margin-top:64px}.mdx-timeline-item{width:204px}.mdx-roadmap .mdx-cta-section__description{font-size:var(--lsd-body1-fontSize)!important;font-weight:var(--lsd-body1-fontWeight)!important;line-height:var(--lsd-body1-lineHeight)!important;margin-top:16px}.scrollToBottom_NE5w{--offset-y:-1rem}.mdx-section-header__title{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important;display:flex;flex-wrap:wrap}.mdx-section-header__description{font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.mdx-showcase{gap:1.5rem 1rem;grid-template-columns:repeat(2,1fr)}.mdx-showcase-card__name{font-size:1.375rem!important;line-height:1.75rem!important}.mdx-showcase-card__description{margin-top:1rem}.mdx-showcase-card__logo{height:auto;width:34px}.mdx-news-card-thumbnail__container,.mdx-news-card__thumbnail{height:128px}.mdx-video-section--list .mdx-cta-button{margin-bottom:40px;margin-top:24px}.mdx-video-section--list .mdx-video-section__list{margin-top:4rem}.mdx-video-section--list .mdx-video-section__list>*{border-top:1px solid rgb(var(--lsd-border-primary))}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>.mdx-video-section__item-title{font-size:.875rem!important;line-height:1.25rem!important}.mdx-video-section--list .mdx-video-section__list .mdx-video-section__list-item>.mdx-video-section__item-description{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h4-fontSize)!important;font-weight:var(--lsd-h4-fontWeight)!important;line-height:var(--lsd-h4-lineHeight)!important}.mdx-video-section__header__mobile-button{margin-top:24px}.mdx-video-section__title{font-size:var(--lsd-h3-fontSize)!important;font-weight:var(--lsd-h3-fontWeight)!important;line-height:var(--lsd-h3-lineHeight)!important}.mdx-table-view-table-title{font-size:28px!important;margin-bottom:24px}.mdx-table-view-row{flex-direction:column;gap:32px}.mdx-numbers-view{gap:32px}.mdx-numbers-view-row{gap:16px}.mdx-numbers-view-number{height:32px;padding:4px 12px;width:32px}.mdx-numbers-view-header{align-items:center;display:flex;flex-direction:row;flex-wrap:wrap;gap:16px}.lsd-button span{font-size:var(--lsd-body3-fontSize)!important}.mdx-numbers-view-title{font-size:20px!important}.backToTop_wDfN{bottom:16px;left:16px;margin-top:76px!important;position:relative}.l-modal{overflow:hidden;width:100vw}.l-modal__container{display:grid;grid-template-columns:1fr;height:100vh;max-width:unset;min-height:-webkit-fill-available;width:100%}.root_ONDG,.root_ltHz{height:100%;overflow:auto}.l-modal__content{border:none;grid-column:1/2;grid-row:1/2;height:100%;margin:0;max-height:100vh;overflow:hidden}.modal_kLVz .header_QwCa{padding:12px 16px}.modal_kLVz .closeButton_Rr0e{display:block;height:2rem;width:2rem}.root_EFVO{padding:40px 24px}.root_ONDG{max-height:100%;padding:24px 24px 64px}.groups_p1lF{max-height:unset;overflow:auto}.root_ltHz{padding:24px 0 64px}.navbarSearchContainer_Bca1{position:absolute;right:var(--ifm-navbar-padding-horizontal)}.container_PkUo{margin-top:24px!important}.blogContainer_wIfk{margin-top:8px!important}.blogPostSubtitle_ysb0{font-size:var(--lsd-subtitle2-fontSize)!important;font-weight:var(--lsd-subtitle2-fontWeight)!important;line-height:var(--lsd-subtitle2-lineHeight)!important;margin-top:4px}.wrapper_SWrM>div{overflow:visible!important}.nav_cMpg{height:56px}.docItemContainer_F8PC{padding:0 .3rem}.docItemGrid_SzoZ{display:grid;grid-template-columns:repeat(2,1fr);padding-left:0}.docItemCol_F52z{grid-column:span 2}}@media (max-width:767px){.blog-archive-page .main-wrapper main>section .container>.row{margin-top:0}.blog-archive-page .main-wrapper main>section .container>.row .col{margin-top:40px!important}.mdx-scroll-buttons>div{justify-content:flex-end}.mdx-scroll-buttons>div>button:not(:last-child){border-right:none!important}.mdx-scroll-buttons .mdx-scroll-buttons__label{display:none}.mdx-hero{justify-content:unset}.mdx-hero h1{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h2-fontSize)!important;font-weight:var(--lsd-h2-fontWeight)!important;line-height:var(--lsd-h2-lineHeight)!important}.mdx-ns__description{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h4-fontSize);font-weight:var(--lsd-h4-fontWeight);line-height:var(--lsd-h4-lineHeight)}.mdx-ns__inner form{margin-top:40px}.mdx-ns__inner .mdx-ns__inputs{flex-direction:column;gap:24px}.mdx-ns__inner .mdx-ns__inputs>*{width:100%}.mdx-ns__inner .mdx-ns__submit-button{margin-top:24px}.mdx-profile-card{display:flex;scroll-snap-align:start!important;width:var(--mobile-width)}.mdx-profile-card__profile{display:flex;flex-direction:column;gap:16px}.mdx-profile-card__name{font-family:var(--lsd-typography-generic-font-family)!important;font-size:var(--lsd-h5-fontSize)!important;font-weight:var(--lsd-h5-fontWeight)!important;line-height:var(--lsd-h5-lineHeight)!important}.mdx-event-card__thumbnail{height:unset}.mdx-news-card-thumbnail__container,.mdx-news-card__thumbnail{height:100%}}@media (max-width:576px){.markdown h1:first-child{--ifm-h1-font-size:2rem}.markdown>h2{--ifm-h2-font-size:1.5rem}.markdown>h3{--ifm-h3-font-size:1.25rem}.title_f1Hy{font-size:2rem}}@media (max-width:575px){:root{--ifm-spacing-vertical:var(--ifm-global-spacing);--ifm-spacing-horizontal:var(--ifm-global-spacing);--ifm-button-size-multiplier:1.25;--ifm-button-padding-horizontal:calc(var(--ifm-button-size-multiplier)*var(--ifm-spacing-horizontal));--ifm-button-padding-vertical:calc(var(--ifm-button-size-multiplier)*var(--ifm-spacing-vertical));--ifm-alert-padding-horizontal:calc(var(--ifm-spacing-horizontal)*2);--ifm-alert-padding-vertical:calc(var(--ifm-spacing-vertical)*2)}.markdown>h2{--ifm-h2-font-size:1.75rem}.markdown h4{--ifm-h4-font-size:1.25rem!important}.footer>div{display:flex!important;flex-direction:column;gap:40px!important;margin-inline:16px;padding-inline:0!important;width:auto}.footer__links{display:flex!important;flex-direction:column!important}.footer__links>div:first-of-type,.footer__links>div:nth-of-type(3){margin-bottom:4px!important}.footer__links>div:nth-of-type(2){margin-bottom:34px!important}.col.footer__col{margin-bottom:0!important}.navbar__left{display:flex}.bottom-cta-sections .mdx-grid__content{gap:98px!important}.bottom-cta-sections .mdx-grid__content:last-of-type>div:last-of-type>.mdx-input-cta-section__container{margin-bottom:120px}.events .main-wrapper main>.container{padding-bottom:0!important}.mdx-event-card-list__container,.mdx-news-card-list__container{margin-bottom:48px;margin-top:32px}.mdx-event-card-list__tabs .lsd-tab-item,.mdx-news-card-list__tabs .lsd-tab-item{padding:6px 14px!important}.mdx-event-about__container,.mdx-input-cta-section__input-fields{display:flex;flex-direction:column;gap:32px}.mdx-event-about__container span,.mdx-event-header__container p{font-size:14px!important;line-height:20px!important}.mdx-event-profile-list__container .mdx-grid>.mdx-grid__content{gap:24px!important}.mdx-input-cta-section__input-fields>div{width:100%!important}.mdx-input-cta-section__container{position:relative}.mdx-input-cta-section__container .mdx-input-cta-section__form{flex-direction:column;gap:40px;margin-bottom:-24px}.mdx-input-cta-section__container .input-cta-section__description--no-form{margin-bottom:32px!important}.input-cta-section__description{margin-bottom:24px!important}.event-details .mdx-event-header__container h1{font-size:24px!important;line-height:32px!important;margin-bottom:0!important}.mdx-news-card h3{font-family:var(--lsd-typography-generic-font-family);font-size:var(--lsd-h5-fontSize);font-weight:var(--lsd-h5-fontWeight);line-height:var(--lsd-h5-lineHeight)}.mdx-news-card__thumbnail{max-height:198px}.footer{--ifm-footer-padding-horizontal:0}.firstRow_ar1q{margin-bottom:64px!important;padding-inline:var(--content-padding)!important}.secondRow__ww3{margin-inline:var(--content-padding)!important}}@media (max-width:574px){.mdx-ghc__issue-content-grid{grid-template-columns:82px 1fr}}@media (hover:hover){.backToTopButton_sjWU:hover{background-color:var(--ifm-color-emphasis-300)}}@media (pointer:fine){.thin-scrollbar{scrollbar-width:thin}.thin-scrollbar::-webkit-scrollbar{height:var(--ifm-scrollbar-size);width:var(--ifm-scrollbar-size)}.thin-scrollbar::-webkit-scrollbar-track{background:var(--ifm-scrollbar-track-background-color);border-radius:10px}.thin-scrollbar::-webkit-scrollbar-thumb{background:var(--ifm-scrollbar-thumb-background-color);border-radius:10px}.thin-scrollbar::-webkit-scrollbar-thumb:hover{background:var(--ifm-scrollbar-thumb-hover-background-color)}}@media (prefers-reduced-motion:reduce){:root{--ifm-transition-fast:0ms;--ifm-transition-slow:0ms}}@media print{.announcementBar_mb4j,.footer,.menu,.navbar,.pagination-nav,.table-of-contents,.tocMobile_ITEo{display:none}.tabs{page-break-inside:avoid}.codeBlockLines_LDrR,.codeBlockLines_e6Vv{white-space:pre-wrap}} \ No newline at end of file diff --git a/assets/js/04b14403.6af3a6c6.js b/assets/js/04b14403.6af3a6c6.js new file mode 100644 index 00000000..be644a3f --- /dev/null +++ b/assets/js/04b14403.6af3a6c6.js @@ -0,0 +1 @@ +"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[4395],{68216:e=>{e.exports=JSON.parse('{"metadata":{"permalink":"/rlog/page/5","page":5,"postsPerPage":10,"totalPages":5,"totalCount":41,"previousPage":"/rlog/page/4","blogDescription":"Blog","blogTitle":"Research Blog"}}')}}]); \ No newline at end of file diff --git a/assets/js/04ed9dad.6ab3c4da.js b/assets/js/04ed9dad.6ab3c4da.js deleted file mode 100644 index 9c151bf0..00000000 --- a/assets/js/04ed9dad.6ab3c4da.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[1119],{16944:(a,e,t)=>{t.r(e),t.d(e,{assets:()=>i,contentTitle:()=>p,default:()=>h,frontMatter:()=>m,metadata:()=>r,toc:()=>l});var s=t(87462),n=(t(67294),t(3905));const m={title:"Vac 101: Climbing Merkle Trees",date:new Date("2024-12-30T12:00:00.000Z"),authors:"marvin",published:!0,slug:"climbing-merkle-trees",categories:"research",toc_min_heading_level:2,toc_max_heading_level:4},p=void 0,r={permalink:"/rlog/climbing-merkle-trees",source:"@site/rlog/2024-12-30-merkle-tree.mdx",title:"Vac 101: Climbing Merkle Trees",description:"In this post, we introduce a crucial data structure used throughout web3.",date:"2024-12-30T12:00:00.000Z",formattedDate:"December 30, 2024",tags:[],readingTime:9.67,hasTruncateMarker:!0,authors:[{name:"Marvin",github:"jonesmarvin8",key:"marvin"}],frontMatter:{title:"Vac 101: Climbing Merkle Trees",date:"2024-12-30T12:00:00.000Z",authors:"marvin",published:!0,slug:"climbing-merkle-trees",categories:"research",toc_min_heading_level:2,toc_max_heading_level:4},prevItem:{title:"Vac 2024 Year in Review",permalink:"/rlog/2024-recap"},nextItem:{title:"Large Message Handling in GossipSub: Potential Improvements",permalink:"/rlog/gsub-largemsg-improvements"}},i={authorsImageUrls:[void 0]},l=[{value:"Introduction",id:"introduction",level:2},{value:"Tree structure",id:"tree-structure",level:2},{value:"Merkle trees",id:"merkle-trees",level:2},{value:"Construction",id:"construction",level:3},{value:"Merkle tree intregrity",id:"merkle-tree-intregrity",level:3},{value:"Proof of membership",id:"proof-of-membership",level:3},{value:"Capped proof of membership",id:"capped-proof-of-membership",level:4},{value:"Extensions of Merkle trees",id:"extensions-of-merkle-trees",level:2},{value:"Sparse Merkle trees",id:"sparse-merkle-trees",level:3},{value:"Proof of nonmembership",id:"proof-of-nonmembership",level:3},{value:"Verkle Trees",id:"verkle-trees",level:3},{value:"References",id:"references",level:2}],N={toc:l};function h(a){let{components:e,...m}=a;return(0,n.kt)("wrapper",(0,s.Z)({},N,m,{components:e,mdxType:"MDXLayout"}),(0,n.kt)("p",null,"In this post, we introduce a crucial data structure used throughout web3."),(0,n.kt)("h2",{id:"introduction"},"Introduction"),(0,n.kt)("p",null,"A large amount of data is swapped between users on a blockchain in the form of transactions.\nOver the entire life of a blockchain,\nthe storage space required to maintain a copy of every transaction becomes untenable for most users.\nHowever, the integrity of a blockchain relies on a large pool of users that can validate the blockchain's history from its inception to its present state.\nThe data representing the blockchain's state is compressed.\nThis compression addresses the issue of scalability that would otherwise greatly restrict the pool of users."),(0,n.kt)("p",null,"Data compression alone is not the end goal.\nAs mentioned, it is essential for users to be able to validate the blockchain's history.\nThe property of compression and validation was solved in Bitcoin by the use of Merkle trees.\nMerkle trees were introduced first by Ralph Merkle in his dissertation [",(0,n.kt)("a",{parentName:"p",href:"https://www.ralphmerkle.com/papers/Thesis1979.pdf"},"1"),"].\nA Merkle tree is a data structure that compresses a digest of data to a constant size while still providing a method for proving membership of elements of the digest.\nA previous rlog[",(0,n.kt)("a",{parentName:"p",href:"https://vac.dev/rlog/rln-light-verifiers/"},"2"),"] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN. "),(0,n.kt)("h2",{id:"tree-structure"},"Tree structure"),(0,n.kt)("p",null,"A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes.\nThe trees that we consider can be arranged in layers with multiple nodes (children) merged into a single node (parent) in the preceding layer.\nA single node exists in the base layer;\nthis special node is called the root node.\nThe highest level of the tree consists of childless nodes called leaves."),(0,n.kt)("p",null,(0,n.kt)("img",{alt:"Image 1",src:t(65529).Z,width:"1017",height:"456"})),(0,n.kt)("p",null,"A binary tree has one additional property:\neach nonleaf node has exactly two children nodes.\nThat is, we assume that nodes in a binary tree are either a parent node with two children or a leaf.\nAs strange as it sounds, each child node has exactly one parental node."),(0,n.kt)("p",null,(0,n.kt)("img",{alt:"Image 2",src:t(31701).Z,width:"940",height:"467"})),(0,n.kt)("p",null,"A binary tree with ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mi",{parentName:"msup"},"n"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^n")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6644em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.6644em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"))))))))))))," leaves consists of ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"+"),(0,n.kt)("mn",{parentName:"mrow"},"1")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"n+1")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6667em",verticalAlign:"-0.0833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"n"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"+"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6444em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1")))))," layers.\nAdditionally, such a tree has ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mrow",{parentName:"msup"},(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"+"),(0,n.kt)("mn",{parentName:"mrow"},"1"))),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow"},"1")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^{n+1}-1")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8974em",verticalAlign:"-0.0833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.8141em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"),(0,n.kt)("span",{parentName:"span",className:"mbin mtight"},"+"),(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1"))))))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6444em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1")))))," nodes."),(0,n.kt)("h2",{id:"merkle-trees"},"Merkle trees"),(0,n.kt)("p",null,"A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function.\nMerkle trees are usually taken to have a binary tree structure.\nAs such, the presentation we provide in this section will be for binary trees."),(0,n.kt)("h3",{id:"construction"},"Construction"),(0,n.kt)("p",null,"In this section, we show how Merkle trees are constructed to compress a digest ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"D")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"D")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.02778em"}},"D"))))),".\nSuppose that the digest ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"D")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"D")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.02778em"}},"D")))))," consists of ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mi",{parentName:"msup"},"n"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^n")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6644em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.6644em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"))))))))))))," entries;\nwe assume that the digest ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"D")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"D")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.02778em"}},"D")))))," has this many entries since a Merkle tree is a binary tree.\nAdditionally, each digest can be padded to ensure that ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"D")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"D")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.02778em"}},"D")))))," has the desired number of entries."),(0,n.kt)("p",null,"Each leaf of the Merkle tree contains the hash of a digest entry.\nEach parent node contains the hash of the concatenation of their child nodes.\nThrough this iterative construction, we reach the root of the tree.\nThe value contained in the root node is called the root hash.\nThe root hash is a compressed representation of the digest ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"D")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"D")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.02778em"}},"D"))))),"."),(0,n.kt)("p",null,(0,n.kt)("img",{alt:"Image 3",src:t(39701).Z,width:"1035",height:"629"})),(0,n.kt)("p",null,"Each node in the Merkle tree is computed by taking a hash.\nSince a binary tree with ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mi",{parentName:"msup"},"n"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^n")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6644em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.6644em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"))))))))))))," leaves has ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mrow",{parentName:"msup"},(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"+"),(0,n.kt)("mn",{parentName:"mrow"},"1"))),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow"},"1")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^{n+1}-1")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8974em",verticalAlign:"-0.0833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.8141em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"),(0,n.kt)("span",{parentName:"span",className:"mbin mtight"},"+"),(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1"))))))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6444em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1")))))," nodes,\nthen we need to evaluate ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mrow",{parentName:"msup"},(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"+"),(0,n.kt)("mn",{parentName:"mrow"},"1"))),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow"},"1")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^{n+1}-1")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8974em",verticalAlign:"-0.0833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.8141em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"),(0,n.kt)("span",{parentName:"span",className:"mbin mtight"},"+"),(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1"))))))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6444em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1")))))," hashes to construct the Merkle tree."),(0,n.kt)("h3",{id:"merkle-tree-intregrity"},"Merkle tree intregrity"),(0,n.kt)("p",null,"A large quantity of data can be compressed to a single hash value.\nA natural question to ask is: could a clever party find another digest that yields a Merkle tree with the same root hash?\nIf possible, this would compromise the ledger since the blockchain's history could be altered.\nFortunately, Merkle trees are quite secure.\nIn fact, Merkle trees can be used to both bind and hide a digest."),(0,n.kt)("p",null,"The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [",(0,n.kt)("a",{parentName:"p",href:"https://vac.dev/rlog/vac101-fiat-shamir#hash-functions"},"3"),"] for information on hash functions).\nA hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value."),(0,n.kt)("p",null,"This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on."),(0,n.kt)("p",null,"In certain applications,\nit may be desirable for the digest of a Merkle tree to be kept confidential.\nThis is achieved with the preimage resistant property of hash functions.\nA hash function is preimage resistant provided that it is difficult to reverse the hashing operation.\nIt would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest."),(0,n.kt)("p",null,"Now, we see that Merkle trees are secured structures that are tamper resistant."),(0,n.kt)("h3",{id:"proof-of-membership"},"Proof of membership"),(0,n.kt)("p",null,"An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest.\nThis can be done with logarithmic storage and logarithmic computation time."),(0,n.kt)("p",null,"Suppose that we want to show that data ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," is part of the Merkle tree's digest.\nAdditionally, suppose that ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{hash}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"hash"))))))," is the hash function used to construct the tree.\nWe assume that the hash function ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{hash}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"hash"))))))," can be computed in constant-time for any input."),(0,n.kt)("p",null,"Suppose that a prover provides data ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," to a verifier,\nand tells the verifier that ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," corresponds to the ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"i")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"i")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6595em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"i"))))),"th leaf of the Merkle tree.\nFor the verifier to be convinced that ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," is part of the digest, he needs to be able to construct the tree's root hash using ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{hash}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"hash")))))),", ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"i")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"i")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6595em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"i"))))),", ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," and some additional information from the prover.\nSpecifically, the prover must provide the sibling hashes for each value that the verifier can compute.\nThis enables the verifier to compute the parents of the siblings that the prover provides and the values that he was able to produce himself.\nThe last of the computed parents is the root."),(0,n.kt)("p",null,"The leaf index ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"i")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"i")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6595em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"i")))))," indicates whether a hash value provided by the prover is a left or right sibling.\nThis is done by looking at the binary expansion of ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"i")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"i")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6595em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"i"))))),"."),(0,n.kt)("p",null,"The verifier can compute the leaf ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"h"),(0,n.kt)("mn",{parentName:"msub"},"0")),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"h_0 = \\mathsf{hash}(\\ell)")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},"="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"hash")),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113"),(0,n.kt)("span",{parentName:"span",className:"mclose"},")"))))),".\nNext, using ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"h"),(0,n.kt)("mn",{parentName:"msub"},"0"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"h_0")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))))))),"'s sibling, ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msubsup",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msubsup"},"h"),(0,n.kt)("mn",{parentName:"msubsup"},"0"),(0,n.kt)("mo",{parentName:"msubsup",mathvariant:"normal",lspace:"0em",rspace:"0em"},"\u2032"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"h'_0")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.2481em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.7519em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0"))),(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"\u2032"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2481em"}},(0,n.kt)("span",{parentName:"span"})))))))))),", provided by the prover,\nthe verifier can compute ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"h"),(0,n.kt)("mn",{parentName:"msub"},"1")),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"h"),(0,n.kt)("mn",{parentName:"msub"},"0")),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2225"),(0,n.kt)("msubsup",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msubsup"},"h"),(0,n.kt)("mn",{parentName:"msubsup"},"0"),(0,n.kt)("mo",{parentName:"msubsup",mathvariant:"normal",lspace:"0em",rspace:"0em"},"\u2032")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"h_1 = \\mathsf{hash}(h_0 \\|h'_0)")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},"="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1.0019em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"hash")),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2225"),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.7519em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0"))),(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"\u2032"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2481em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mclose"},")")))))," or ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"h"),(0,n.kt)("mn",{parentName:"msub"},"1")),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("msubsup",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msubsup"},"h"),(0,n.kt)("mn",{parentName:"msubsup"},"0"),(0,n.kt)("mo",{parentName:"msubsup",mathvariant:"normal",lspace:"0em",rspace:"0em"},"\u2032")),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2225"),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"h"),(0,n.kt)("mn",{parentName:"msub"},"0")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"h_1 = \\mathsf{hash}(h'_0 \\| h_0)")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},"="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1.0019em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"hash")),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.7519em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0"))),(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"\u2032"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2481em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2225"),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mclose"},")"))))),"\ndepending on whether ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msubsup",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msubsup"},"h"),(0,n.kt)("mn",{parentName:"msubsup"},"0"),(0,n.kt)("mo",{parentName:"msubsup",mathvariant:"normal",lspace:"0em",rspace:"0em"},"\u2032"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"h'_0")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.2481em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"h"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.7519em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0"))),(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"\u2032"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2481em"}},(0,n.kt)("span",{parentName:"span"}))))))))))," is a left or right sibling.\nThis pathing continues until the verifier either successfully computes the root hash (in ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"+"),(0,n.kt)("mn",{parentName:"mrow"},"1")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"n+1")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6667em",verticalAlign:"-0.0833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"n"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"+"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6444em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1")))))," hashes) or fails to do so."),(0,n.kt)("p",null,"The prover has to provide ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"n")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"n")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.4306em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"n")))))," sibling nodes for the proof of membership."),(0,n.kt)("p",null,"There is a key detail that is essential for the proof of membership to be secure.\nThe root hash has to be provided to the verifier prior to the selection of data ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113"))))),".\nOtherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership."),(0,n.kt)("h4",{id:"capped-proof-of-membership"},"Capped proof of membership"),(0,n.kt)("p",null,"Polygon provides an implementation [",(0,n.kt)("a",{parentName:"p",href:"https://github.com/0xPolygonZero/plonky2/blob/main/plonky2/src/hash/merkle_tree.rs"},"4"),"] of a shortened proof of membership with a slight modification.\nA specific layer of the Merkle tree is published instead of just the root hash.\nBy doing this, a capped proof of membership is just the path from leaf to the published layer."),(0,n.kt)("h2",{id:"extensions-of-merkle-trees"},"Extensions of Merkle trees"),(0,n.kt)("p",null,"Merkle trees can be extended in multiple ways.\nIn this section, we explore a select few of these extensions."),(0,n.kt)("h3",{id:"sparse-merkle-trees"},"Sparse Merkle trees"),(0,n.kt)("p",null,"A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries.\nSpecifically, each digest entry has a particular leaf index.\nFor simplicity, we assume that the index value is computed by taking the hash of the entry.\nWe note that this is a sorted SMT."),(0,n.kt)("p",null,"Let ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"n")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"n")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.4306em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"n")))))," denote the number of bits that a hash value can possess. This means that our SMT can have at most ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mi",{parentName:"msup"},"n"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^n")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6644em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.6644em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"))))))))))))," leaves."),(0,n.kt)("p",null,"An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"u"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{null}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"null"))))))," marker inserted in.\nThis means that we can prove membership in the way described.\nHowever, we can also prove nonmembership of an element by showing that ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"u"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{null}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"null"))))))," is located in the element's hash location.\nThe crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership."),(0,n.kt)("p",null,"We can take advantage of the sparse nature of SMTs to provide shortened proofs.\nSpecifically, it is unlikely for entries to cluster together.\nThus, it is efficient to maintain a list of values:"),(0,n.kt)("table",null,(0,n.kt)("thead",{parentName:"table"},(0,n.kt)("tr",{parentName:"thead"},(0,n.kt)("th",{parentName:"tr",align:null},"Null values"))),(0,n.kt)("tbody",{parentName:"table"},(0,n.kt)("tr",{parentName:"tbody"},(0,n.kt)("td",{parentName:"tr",align:null},(0,n.kt)("span",{parentName:"td",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"d"),(0,n.kt)("mn",{parentName:"msub"},"0")),(0,n.kt)("mo",{parentName:"mrow"},":"),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"H"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"u"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("mo",{parentName:"mrow",separator:"true"},",")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"d_0 := \\mathsf{Hash(null)},")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},":="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"Hash"),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"null"),(0,n.kt)("span",{parentName:"span",className:"mclose"},")")),(0,n.kt)("span",{parentName:"span",className:"mpunct"},","))))))),(0,n.kt)("tr",{parentName:"tbody"},(0,n.kt)("td",{parentName:"tr",align:null},(0,n.kt)("span",{parentName:"td",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"d"),(0,n.kt)("mn",{parentName:"msub"},"1")),(0,n.kt)("mo",{parentName:"mrow"},":"),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"H"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub",mathvariant:"sans-serif"},"d"),(0,n.kt)("mn",{parentName:"msub",mathvariant:"sans-serif"},"0")),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"\u2225"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"\u2225"),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub",mathvariant:"sans-serif"},"d"),(0,n.kt)("mn",{parentName:"msub",mathvariant:"sans-serif"},"0")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("mo",{parentName:"mrow",separator:"true"},",")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"d_1 := \\mathsf{Hash(d_0 \\|\\| d_0)},")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},":="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"Hash"),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3089em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2225\u2225"),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3089em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"0")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mclose"},")")),(0,n.kt)("span",{parentName:"span",className:"mpunct"},","))))))),(0,n.kt)("tr",{parentName:"tbody"},(0,n.kt)("td",{parentName:"tr",align:null},(0,n.kt)("span",{parentName:"td",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"d"),(0,n.kt)("mn",{parentName:"msub"},"2")),(0,n.kt)("mo",{parentName:"mrow"},":"),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"H"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub",mathvariant:"sans-serif"},"d"),(0,n.kt)("mn",{parentName:"msub",mathvariant:"sans-serif"},"1")),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"\u2225"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"\u2225"),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub",mathvariant:"sans-serif"},"d"),(0,n.kt)("mn",{parentName:"msub",mathvariant:"sans-serif"},"1")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("mo",{parentName:"mrow",separator:"true"},",")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"d_2 := \\mathsf{Hash(d_1 \\|\\| d_1)},")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"2")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},":="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"Hash"),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3089em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"1")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2225\u2225"),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3089em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"1")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mclose"},")")),(0,n.kt)("span",{parentName:"span",className:"mpunct"},","))))))),(0,n.kt)("tr",{parentName:"tbody"},(0,n.kt)("td",{parentName:"tr",align:null},(0,n.kt)("span",{parentName:"td",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mi",mathvariant:"normal"},"\u22ee"),(0,n.kt)("mpadded",{parentName:"mi",height:"0em",voffset:"0em"},(0,n.kt)("mspace",{parentName:"mpadded",mathbackground:"black",width:"0em",height:"1.5em"})))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\vdots")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1.53em",verticalAlign:"-0.03em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"\u22ee"),(0,n.kt)("span",{parentName:"span",className:"mord rule",style:{borderRightWidth:"0em",borderTopWidth:"1.5em",bottom:"0em"}})))))))),(0,n.kt)("tr",{parentName:"tbody"},(0,n.kt)("td",{parentName:"tr",align:null},(0,n.kt)("span",{parentName:"td",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"d"),(0,n.kt)("mrow",{parentName:"msub"},(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow"},"1"))),(0,n.kt)("mo",{parentName:"mrow"},":"),(0,n.kt)("mo",{parentName:"mrow"},"="),(0,n.kt)("mrow",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"H"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"a"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"s"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"h"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub",mathvariant:"sans-serif"},"d"),(0,n.kt)("mrow",{parentName:"msub"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow",mathvariant:"sans-serif"},"2"))),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"\u2225"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"\u2225"),(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub",mathvariant:"sans-serif"},"d"),(0,n.kt)("mrow",{parentName:"msub"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow",mathvariant:"sans-serif"},"2"))),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},".")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"d_{n-1} := \\mathsf{Hash(d_{n-2} \\|\\| d_{n-2})}.")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.9028em",verticalAlign:"-0.2083em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3011em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"n"),(0,n.kt)("span",{parentName:"span",className:"mbin mtight"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"1"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2083em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}}),(0,n.kt)("span",{parentName:"span",className:"mrel"},":="),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2778em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"Hash"),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3089em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"n"),(0,n.kt)("span",{parentName:"span",className:"mbin mtight"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"2"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2083em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2225\u2225"),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3089em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"n"),(0,n.kt)("span",{parentName:"span",className:"mbin mtight"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mord mathsf mtight"},"2"))))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2083em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mclose"},")")),(0,n.kt)("span",{parentName:"span",className:"mord"},"."))))))))),(0,n.kt)("p",null,"Each of the ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mi",{parentName:"msub"},"d"),(0,n.kt)("mi",{parentName:"msub"},"i"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"d_i")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"d"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.3117em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"i")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.15em"}},(0,n.kt)("span",{parentName:"span"})))))))))),"'s represents the root hash of a Merkle tree with ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mi",{parentName:"msup"},"i"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^i")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8247em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.8247em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight"},"i"))))))))))))," leaves containing ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"u"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{null}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"null")))))),".\nThese values can be used to shorten the time needed to construct an SMT and the length of proofs."),(0,n.kt)("h3",{id:"proof-of-nonmembership"},"Proof of nonmembership"),(0,n.kt)("p",null,"In the first Vac 101 [",(0,n.kt)("a",{parentName:"p",href:"https://vac.dev/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters"},"5"),"], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership.\nHowever, the proof of membership may result in false positives due to collisions.\nThis would affect nonmembership proofs as well.\nSparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest."),(0,n.kt)("p",null,"Why is sorting essential?\nThe sorting mechanism of data can be arbitrarily chosen.\nHowever, it is essential that there are no gaps in the ordering.\nThe maximum number of elements that could ever exist in the digest must be known.\nA simple method for this is to use a hash function to provide fingerprints to the data.\nEach hash using either SHA-256 or Keccak has 256-bits.\nOur entire digest could consist of a maximum of ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msup",{parentName:"mrow"},(0,n.kt)("mn",{parentName:"msup"},"2"),(0,n.kt)("mn",{parentName:"msup"},"256"))),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"2^{256}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.8141em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord"},"2"),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.8141em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-3.063em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mtight"},"256")))))))))))))," entries.\nThis assumes that our digest does not contain collisions."),(0,n.kt)("p",null,"The fingerprint of a piece of data ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," indicates which leaf of the SMT it is contained in.\nThis means that a nonmembership of ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," in the SMT becomes a matter of proving that ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"n"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"u"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l"),(0,n.kt)("mi",{parentName:"mrow",mathvariant:"sans-serif"},"l")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\mathsf{null}")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},(0,n.kt)("span",{parentName:"span",className:"mord mathsf"},"null"))))))," is contained in ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113"))))),"'s location. "),(0,n.kt)("p",null,"It is crucial for the SMT to be sorted.\nOtherwise, a malicious party can append the entry ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow",mathvariant:"normal"},"\u2113")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\ell")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"\u2113")))))," to a random location.\nThis allows for the malicious party to provide contradictory proofs that prove both membership and nonmembership.\nWe note that the requirement that an SMT is sorted may be too strong of an assumption in centralized cases.\nHowever, sortedness is a necessary property of SMTs for decentralized systems."),(0,n.kt)("h3",{id:"verkle-trees"},"Verkle Trees"),(0,n.kt)("p",null,"A proof of membership grows in length as the Merkle tree grows.\nThe most obvious approach to remedy this scalability issue is to use Merkle trees in which each node has more than two children.\nHowever, this does not fix the issue.\nA proof of membership in a ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"k")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"k")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.03148em"}},"k"))))),"-nary Merkle tree [",(0,n.kt)("a",{parentName:"p",href:"https://math.mit.edu/research/highschool/primes/materials/2018/Kuszmaul.pdf"},"6"),"] (each node has ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"k")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"k")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6944em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.03148em"}},"k")))))," children) has a proof size ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("msub",{parentName:"mrow"},(0,n.kt)("mrow",{parentName:"msub"},(0,n.kt)("mi",{parentName:"mrow"},"log"),(0,n.kt)("mo",{parentName:"mrow"},"\u2061")),(0,n.kt)("mi",{parentName:"msub"},"k")),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("mi",{parentName:"mrow"},"n"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},"("),(0,n.kt)("mi",{parentName:"mrow"},"k"),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow"},"1"),(0,n.kt)("mo",{parentName:"mrow",stretchy:"false"},")")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"\\log_k(n)(k-1)")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mop"},(0,n.kt)("span",{parentName:"span",className:"mop"},"lo",(0,n.kt)("span",{parentName:"span",style:{marginRight:"0.01389em"}},"g")),(0,n.kt)("span",{parentName:"span",className:"msupsub"},(0,n.kt)("span",{parentName:"span",className:"vlist-t vlist-t2"},(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.242em"}},(0,n.kt)("span",{parentName:"span",style:{top:"-2.4559em",marginRight:"0.05em"}},(0,n.kt)("span",{parentName:"span",className:"pstrut",style:{height:"2.7em"}}),(0,n.kt)("span",{parentName:"span",className:"sizing reset-size6 size3 mtight"},(0,n.kt)("span",{parentName:"span",className:"mord mathnormal mtight",style:{marginRight:"0.03148em"}},"k")))),(0,n.kt)("span",{parentName:"span",className:"vlist-s"},"\u200b")),(0,n.kt)("span",{parentName:"span",className:"vlist-r"},(0,n.kt)("span",{parentName:"span",className:"vlist",style:{height:"0.2441em"}},(0,n.kt)("span",{parentName:"span"})))))),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal"},"n"),(0,n.kt)("span",{parentName:"span",className:"mclose"},")"),(0,n.kt)("span",{parentName:"span",className:"mopen"},"("),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.03148em"}},"k"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1"),(0,n.kt)("span",{parentName:"span",className:"mclose"},")"))))),".\nThe multiple ",(0,n.kt)("span",{parentName:"p",className:"math math-inline"},(0,n.kt)("span",{parentName:"span",className:"katex"},(0,n.kt)("span",{parentName:"span",className:"katex-mathml"},(0,n.kt)("math",{parentName:"span",xmlns:"http://www.w3.org/1998/Math/MathML"},(0,n.kt)("semantics",{parentName:"math"},(0,n.kt)("mrow",{parentName:"semantics"},(0,n.kt)("mi",{parentName:"mrow"},"k"),(0,n.kt)("mo",{parentName:"mrow"},"\u2212"),(0,n.kt)("mn",{parentName:"mrow"},"1")),(0,n.kt)("annotation",{parentName:"semantics",encoding:"application/x-tex"},"k-1")))),(0,n.kt)("span",{parentName:"span",className:"katex-html","aria-hidden":"true"},(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.7778em",verticalAlign:"-0.0833em"}}),(0,n.kt)("span",{parentName:"span",className:"mord mathnormal",style:{marginRight:"0.03148em"}},"k"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}}),(0,n.kt)("span",{parentName:"span",className:"mbin"},"\u2212"),(0,n.kt)("span",{parentName:"span",className:"mspace",style:{marginRight:"0.2222em"}})),(0,n.kt)("span",{parentName:"span",className:"base"},(0,n.kt)("span",{parentName:"span",className:"strut",style:{height:"0.6444em"}}),(0,n.kt)("span",{parentName:"span",className:"mord"},"1")))))," is the number of silbings that a node has on each layer.\nHence, the proof size grows faster than a logarithmic function of the digest size."),(0,n.kt)("p",null,"An alternate approach is to use a different data structure: Verkle trees [",(0,n.kt)("a",{parentName:"p",href:"https://math.mit.edu/research/highschool/primes/materials/2018/Kuszmaul.pdf"},"6"),"].\nA Verkle tree replaces hash functions with polynomial commitments [",(0,n.kt)("a",{parentName:"p",href:"https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095"},"7"),", ",(0,n.kt)("a",{parentName:"p",href:"https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html"},"8"),"].\nWe will explore Verkle trees in a future Vac 101 edition."),(0,n.kt)("h2",{id:"references"},"References"),(0,n.kt)("ul",null,(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li"},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://www.ralphmerkle.com/papers/Thesis1979.pdf"},"Secrecy, Authentication, and Public Key Systems")," "))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:2},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://vac.dev/rlog/rln-light-verifiers/"},"Verifying RLN Proofs in Light Clients with Subtrees")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:3},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://vac.dev/rlog/vac101-fiat-shamir#hash-functions"},"Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:4},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://github.com/0xPolygonZero/plonky2/blob/main/plonky2/src/hash/merkle_tree.rs"},"Capped merkle tree in Plonky2")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:5},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://vac.dev/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters"},"Vac 101: Membership with Bloom Filters and Cuckoo Filters")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:6},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://math.mit.edu/research/highschool/primes/materials/2018/Kuszmaul.pdf"},"Verkle Trees")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:7},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095"},"Using polynomial commitments to replace state roots")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:8},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html"},"KZG polynomial commitments")))),(0,n.kt)("li",{parentName:"ul"},(0,n.kt)("ol",{parentName:"li",start:9},(0,n.kt)("li",{parentName:"ol"},(0,n.kt)("a",{parentName:"li",href:"https://github.com/o1-labs/verkle-tree"},"O1 labs' Verkle Tree repo"))))))}h.isMDXComponent=!0},3905:(a,e,t)=>{t.d(e,{Zo:()=>N,kt:()=>o});var s=t(67294);function n(a,e,t){return e in a?Object.defineProperty(a,e,{value:t,enumerable:!0,configurable:!0,writable:!0}):a[e]=t,a}function m(a,e){var t=Object.keys(a);if(Object.getOwnPropertySymbols){var s=Object.getOwnPropertySymbols(a);e&&(s=s.filter((function(e){return Object.getOwnPropertyDescriptor(a,e).enumerable}))),t.push.apply(t,s)}return t}function p(a){for(var e=1;e=0||(n[t]=a[t]);return n}(a,e);if(Object.getOwnPropertySymbols){var m=Object.getOwnPropertySymbols(a);for(s=0;s=0||Object.prototype.propertyIsEnumerable.call(a,t)&&(n[t]=a[t])}return n}var i=s.createContext({}),l=function(a){var e=s.useContext(i),t=e;return a&&(t="function"==typeof a?a(e):p(p({},e),a)),t},N=function(a){var e=l(a.components);return s.createElement(i.Provider,{value:e},a.children)},h={inlineCode:"code",wrapper:function(a){var e=a.children;return s.createElement(s.Fragment,{},e)}},k=s.forwardRef((function(a,e){var t=a.components,n=a.mdxType,m=a.originalType,i=a.parentName,N=r(a,["components","mdxType","originalType","parentName"]),k=l(t),o=n,c=k["".concat(i,".").concat(o)]||k[o]||h[o]||m;return t?s.createElement(c,p(p({ref:e},N),{},{components:t})):s.createElement(c,p({ref:e},N))}));function o(a,e){var t=arguments,n=e&&e.mdxType;if("string"==typeof a||n){var m=t.length,p=new Array(m);p[0]=k;var r={};for(var i in e)hasOwnProperty.call(e,i)&&(r[i]=e[i]);r.originalType=a,r.mdxType="string"==typeof a?a:n,p[1]=r;for(var l=2;l{t.d(e,{Z:()=>s});const s=t.p+"assets/images/vac101_binary_tree-f2600381c1537895a063761d315201ce.png"},39701:(a,e,t)=>{t.d(e,{Z:()=>s});const s=t.p+"assets/images/vac101_merkle_tree-a7c86f78d5aa8016921924220d6005fc.png"},65529:(a,e,t)=>{t.d(e,{Z:()=>s});const s=t.p+"assets/images/vac101_tree-c39839d4050c3723ccde9d3622de2870.png"}}]); \ No newline at end of file diff --git a/assets/js/04ed9dad.b6cab548.js b/assets/js/04ed9dad.b6cab548.js new file mode 100644 index 00000000..ab4ef087 --- /dev/null +++ b/assets/js/04ed9dad.b6cab548.js @@ -0,0 +1 @@ +"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[2104],{16455:(s,e,a)=>{a.d(e,{A:()=>n});const n=a.p+"assets/images/vac101_tree-c39839d4050c3723ccde9d3622de2870.png"},28453:(s,e,a)=>{a.d(e,{R:()=>l,x:()=>r});var n=a(96540);const i={},t=n.createContext(i);function l(s){const e=n.useContext(t);return n.useMemo((function(){return"function"==typeof s?s(e):{...e,...s}}),[e,s])}function r(s){let e;return e=s.disableParentContext?"function"==typeof s.components?s.components(i):s.components||i:l(s.components),n.createElement(t.Provider,{value:e},s.children)}},49849:(s,e,a)=>{a.d(e,{A:()=>n});const n=a.p+"assets/images/vac101_binary_tree-f2600381c1537895a063761d315201ce.png"},51840:(s,e,a)=>{a.r(e),a.d(e,{assets:()=>h,contentTitle:()=>r,default:()=>d,frontMatter:()=>l,metadata:()=>n,toc:()=>m});var n=a(74721),i=a(74848),t=a(28453);const l={title:"Vac 101: Climbing Merkle Trees",date:new Date("2024-12-30T12:00:00.000Z"),authors:"marvin",published:!0,slug:"climbing-merkle-trees",categories:"research",toc_min_heading_level:2,toc_max_heading_level:4},r=void 0,h={authorsImageUrls:[void 0]},m=[{value:"Introduction",id:"introduction",level:2},{value:"Tree structure",id:"tree-structure",level:2},{value:"Merkle trees",id:"merkle-trees",level:2},{value:"Construction",id:"construction",level:3},{value:"Merkle tree intregrity",id:"merkle-tree-intregrity",level:3},{value:"Proof of membership",id:"proof-of-membership",level:3},{value:"Capped proof of membership",id:"capped-proof-of-membership",level:4},{value:"Extensions of Merkle trees",id:"extensions-of-merkle-trees",level:2},{value:"Sparse Merkle trees",id:"sparse-merkle-trees",level:3},{value:"Proof of nonmembership",id:"proof-of-nonmembership",level:3},{value:"Verkle Trees",id:"verkle-trees",level:3},{value:"References",id:"references",level:2}];function c(s){const e={a:"a",annotation:"annotation",h2:"h2",h3:"h3",h4:"h4",img:"img",li:"li",math:"math",mi:"mi",mn:"mn",mo:"mo",mpadded:"mpadded",mrow:"mrow",mspace:"mspace",msub:"msub",msubsup:"msubsup",msup:"msup",ol:"ol",p:"p",semantics:"semantics",span:"span",table:"table",tbody:"tbody",td:"td",th:"th",thead:"thead",tr:"tr",ul:"ul",...(0,t.R)(),...s.components};return(0,i.jsxs)(i.Fragment,{children:[(0,i.jsx)(e.p,{children:"In this post, we introduce a crucial data structure used throughout web3."}),"\n","\n",(0,i.jsx)(e.h2,{id:"introduction",children:"Introduction"}),"\n",(0,i.jsx)(e.p,{children:"A large amount of data is swapped between users on a blockchain in the form of transactions.\nOver the entire life of a blockchain,\nthe storage space required to maintain a copy of every transaction becomes untenable for most users.\nHowever, the integrity of a blockchain relies on a large pool of users that can validate the blockchain's history from its inception to its present state.\nThe data representing the blockchain's state is compressed.\nThis compression addresses the issue of scalability that would otherwise greatly restrict the pool of users."}),"\n",(0,i.jsxs)(e.p,{children:["Data compression alone is not the end goal.\nAs mentioned, it is essential for users to be able to validate the blockchain's history.\nThe property of compression and validation was solved in Bitcoin by the use of Merkle trees.\nMerkle trees were introduced first by Ralph Merkle in his dissertation [",(0,i.jsx)(e.a,{href:"https://www.ralphmerkle.com/papers/Thesis1979.pdf",children:"1"}),"].\nA Merkle tree is a data structure that compresses a digest of data to a constant size while still providing a method for proving membership of elements of the digest.\nA previous rlog[",(0,i.jsx)(e.a,{href:"https://vac.dev/rlog/rln-light-verifiers/",children:"2"}),"] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN."]}),"\n",(0,i.jsx)(e.h2,{id:"tree-structure",children:"Tree structure"}),"\n",(0,i.jsx)(e.p,{children:"A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes.\nThe trees that we consider can be arranged in layers with multiple nodes (children) merged into a single node (parent) in the preceding layer.\nA single node exists in the base layer;\nthis special node is called the root node.\nThe highest level of the tree consists of childless nodes called leaves."}),"\n",(0,i.jsx)(e.p,{children:(0,i.jsx)(e.img,{alt:"Image 1",src:a(16455).A+"",width:"1017",height:"456"})}),"\n",(0,i.jsx)(e.p,{children:"A binary tree has one additional property:\neach nonleaf node has exactly two children nodes.\nThat is, we assume that nodes in a binary tree are either a parent node with two children or a leaf.\nAs strange as it sounds, each child node has exactly one parental node."}),"\n",(0,i.jsx)(e.p,{children:(0,i.jsx)(e.img,{alt:"Image 2",src:a(49849).A+"",width:"940",height:"467"})}),"\n",(0,i.jsxs)(e.p,{children:["A binary tree with ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsx)(e.mi,{children:"n"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^n"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6644em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.6644em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"})})]})})})})})]})]})})]})," leaves consists of ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{children:"+"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"n+1"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6667em",verticalAlign:"-0.0833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"n"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"+"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6444em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"})]})]})]})," layers.\nAdditionally, such a tree has ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{children:"+"}),(0,i.jsx)(e.mn,{children:"1"})]})]}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^{n+1}-1"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8974em",verticalAlign:"-0.0833em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.8141em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsxs)(e.span,{className:"mord mtight",children:[(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"}),(0,i.jsx)(e.span,{className:"mbin mtight",children:"+"}),(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})]})})]})})})})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6444em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"})]})]})]})," nodes."]}),"\n",(0,i.jsx)(e.h2,{id:"merkle-trees",children:"Merkle trees"}),"\n",(0,i.jsx)(e.p,{children:"A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function.\nMerkle trees are usually taken to have a binary tree structure.\nAs such, the presentation we provide in this section will be for binary trees."}),"\n",(0,i.jsx)(e.h3,{id:"construction",children:"Construction"}),"\n",(0,i.jsxs)(e.p,{children:["In this section, we show how Merkle trees are constructed to compress a digest ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"D"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"D"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.02778em"},children:"D"})]})})]}),".\nSuppose that the digest ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"D"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"D"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.02778em"},children:"D"})]})})]})," consists of ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsx)(e.mi,{children:"n"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^n"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6644em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.6644em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"})})]})})})})})]})]})})]})," entries;\nwe assume that the digest ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"D"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"D"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.02778em"},children:"D"})]})})]})," has this many entries since a Merkle tree is a binary tree.\nAdditionally, each digest can be padded to ensure that ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"D"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"D"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.02778em"},children:"D"})]})})]})," has the desired number of entries."]}),"\n",(0,i.jsxs)(e.p,{children:["Each leaf of the Merkle tree contains the hash of a digest entry.\nEach parent node contains the hash of the concatenation of their child nodes.\nThrough this iterative construction, we reach the root of the tree.\nThe value contained in the root node is called the root hash.\nThe root hash is a compressed representation of the digest ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"D"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"D"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.02778em"},children:"D"})]})})]}),"."]}),"\n",(0,i.jsx)(e.p,{children:(0,i.jsx)(e.img,{alt:"Image 3",src:a(54950).A+"",width:"1035",height:"629"})}),"\n",(0,i.jsxs)(e.p,{children:["Each node in the Merkle tree is computed by taking a hash.\nSince a binary tree with ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsx)(e.mi,{children:"n"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^n"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6644em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.6644em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"})})]})})})})})]})]})})]})," leaves has ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{children:"+"}),(0,i.jsx)(e.mn,{children:"1"})]})]}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^{n+1}-1"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8974em",verticalAlign:"-0.0833em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.8141em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsxs)(e.span,{className:"mord mtight",children:[(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"}),(0,i.jsx)(e.span,{className:"mbin mtight",children:"+"}),(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})]})})]})})})})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6444em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"})]})]})]})," nodes,\nthen we need to evaluate ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{children:"+"}),(0,i.jsx)(e.mn,{children:"1"})]})]}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^{n+1}-1"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8974em",verticalAlign:"-0.0833em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.8141em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsxs)(e.span,{className:"mord mtight",children:[(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"}),(0,i.jsx)(e.span,{className:"mbin mtight",children:"+"}),(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})]})})]})})})})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6444em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"})]})]})]})," hashes to construct the Merkle tree."]}),"\n",(0,i.jsx)(e.h3,{id:"merkle-tree-intregrity",children:"Merkle tree intregrity"}),"\n",(0,i.jsx)(e.p,{children:"A large quantity of data can be compressed to a single hash value.\nA natural question to ask is: could a clever party find another digest that yields a Merkle tree with the same root hash?\nIf possible, this would compromise the ledger since the blockchain's history could be altered.\nFortunately, Merkle trees are quite secure.\nIn fact, Merkle trees can be used to both bind and hide a digest."}),"\n",(0,i.jsxs)(e.p,{children:["The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [",(0,i.jsx)(e.a,{href:"https://vac.dev/rlog/vac101-fiat-shamir#hash-functions",children:"3"}),"] for information on hash functions).\nA hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value."]}),"\n",(0,i.jsx)(e.p,{children:"This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on."}),"\n",(0,i.jsx)(e.p,{children:"In certain applications,\nit may be desirable for the digest of a Merkle tree to be kept confidential.\nThis is achieved with the preimage resistant property of hash functions.\nA hash function is preimage resistant provided that it is difficult to reverse the hashing operation.\nIt would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest."}),"\n",(0,i.jsx)(e.p,{children:"Now, we see that Merkle trees are secured structures that are tamper resistant."}),"\n",(0,i.jsx)(e.h3,{id:"proof-of-membership",children:"Proof of membership"}),"\n",(0,i.jsx)(e.p,{children:"An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest.\nThis can be done with logarithmic storage and logarithmic computation time."}),"\n",(0,i.jsxs)(e.p,{children:["Suppose that we want to show that data ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," is part of the Merkle tree's digest.\nAdditionally, suppose that ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{hash}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"hash"})})]})})]})," is the hash function used to construct the tree.\nWe assume that the hash function ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{hash}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"hash"})})]})})]})," can be computed in constant-time for any input."]}),"\n",(0,i.jsxs)(e.p,{children:["Suppose that a prover provides data ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," to a verifier,\nand tells the verifier that ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," corresponds to the ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"i"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"i"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6595em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"i"})]})})]}),"th leaf of the Merkle tree.\nFor the verifier to be convinced that ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," is part of the digest, he needs to be able to construct the tree's root hash using ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{hash}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"hash"})})]})})]}),", ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"i"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"i"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6595em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"i"})]})})]}),", ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," and some additional information from the prover.\nSpecifically, the prover must provide the sibling hashes for each value that the verifier can compute.\nThis enables the verifier to compute the parents of the siblings that the prover provides and the values that he was able to produce himself.\nThe last of the computed parents is the root."]}),"\n",(0,i.jsxs)(e.p,{children:["The leaf index ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"i"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"i"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6595em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"i"})]})})]})," indicates whether a hash value provided by the prover is a left or right sibling.\nThis is done by looking at the binary expansion of ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"i"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"i"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6595em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"i"})]})})]}),"."]}),"\n",(0,i.jsxs)(e.p,{children:["The verifier can compute the leaf ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"})]}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"h_0 = \\mathsf{hash}(\\ell)"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:"="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"hash"})}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]})]})]}),".\nNext, using ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"h_0"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]})]})})]}),"'s sibling, ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msubsup,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"}),(0,i.jsx)(e.mo,{mathvariant:"normal",lspace:"0em",rspace:"0em",children:"\u2032"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"h'_0"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.2481em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsxs)(e.span,{className:"vlist",style:{height:"0.7519em"},children:[(0,i.jsxs)(e.span,{style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]}),(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"\u2032"})})})]})]}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2481em"},children:(0,i.jsx)(e.span,{})})})]})})]})]})})]}),", provided by the prover,\nthe verifier can compute ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"})]}),(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2225"}),(0,i.jsxs)(e.msubsup,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"}),(0,i.jsx)(e.mo,{mathvariant:"normal",lspace:"0em",rspace:"0em",children:"\u2032"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"h_1 = \\mathsf{hash}(h_0 \\|h'_0)"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:"="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1.0019em",verticalAlign:"-0.25em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"hash"})}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mord",children:"\u2225"}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsxs)(e.span,{className:"vlist",style:{height:"0.7519em"},children:[(0,i.jsxs)(e.span,{style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]}),(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"\u2032"})})})]})]}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2481em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]})]})]})," or ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsxs)(e.msubsup,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"}),(0,i.jsx)(e.mo,{mathvariant:"normal",lspace:"0em",rspace:"0em",children:"\u2032"})]}),(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2225"}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"h_1 = \\mathsf{hash}(h'_0 \\| h_0)"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:"="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1.0019em",verticalAlign:"-0.25em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"hash"})}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsxs)(e.span,{className:"vlist",style:{height:"0.7519em"},children:[(0,i.jsxs)(e.span,{style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]}),(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"\u2032"})})})]})]}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2481em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mord",children:"\u2225"}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]})]})]}),"\ndepending on whether ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msubsup,{children:[(0,i.jsx)(e.mi,{children:"h"}),(0,i.jsx)(e.mn,{children:"0"}),(0,i.jsx)(e.mo,{mathvariant:"normal",lspace:"0em",rspace:"0em",children:"\u2032"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"h'_0"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.2481em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"h"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsxs)(e.span,{className:"vlist",style:{height:"0.7519em"},children:[(0,i.jsxs)(e.span,{style:{top:"-2.4519em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]}),(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"\u2032"})})})]})]}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2481em"},children:(0,i.jsx)(e.span,{})})})]})})]})]})})]})," is a left or right sibling.\nThis pathing continues until the verifier either successfully computes the root hash (in ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{children:"+"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"n+1"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6667em",verticalAlign:"-0.0833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"n"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"+"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6444em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"})]})]})]})," hashes) or fails to do so."]}),"\n",(0,i.jsxs)(e.p,{children:["The prover has to provide ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"n"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"n"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.4306em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"n"})]})})]})," sibling nodes for the proof of membership."]}),"\n",(0,i.jsxs)(e.p,{children:["There is a key detail that is essential for the proof of membership to be secure.\nThe root hash has to be provided to the verifier prior to the selection of data ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]}),".\nOtherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership."]}),"\n",(0,i.jsx)(e.h4,{id:"capped-proof-of-membership",children:"Capped proof of membership"}),"\n",(0,i.jsxs)(e.p,{children:["Polygon provides an implementation [",(0,i.jsx)(e.a,{href:"https://github.com/0xPolygonZero/plonky2/blob/main/plonky2/src/hash/merkle_tree.rs",children:"4"}),"] of a shortened proof of membership with a slight modification.\nA specific layer of the Merkle tree is published instead of just the root hash.\nBy doing this, a capped proof of membership is just the path from leaf to the published layer."]}),"\n",(0,i.jsx)(e.h2,{id:"extensions-of-merkle-trees",children:"Extensions of Merkle trees"}),"\n",(0,i.jsx)(e.p,{children:"Merkle trees can be extended in multiple ways.\nIn this section, we explore a select few of these extensions."}),"\n",(0,i.jsx)(e.h3,{id:"sparse-merkle-trees",children:"Sparse Merkle trees"}),"\n",(0,i.jsx)(e.p,{children:"A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries.\nSpecifically, each digest entry has a particular leaf index.\nFor simplicity, we assume that the index value is computed by taking the hash of the entry.\nWe note that this is a sorted SMT."}),"\n",(0,i.jsxs)(e.p,{children:["Let ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"n"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"n"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.4306em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"n"})]})})]})," denote the number of bits that a hash value can possess. This means that our SMT can have at most ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsx)(e.mi,{children:"n"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^n"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6644em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.6644em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"})})]})})})})})]})]})})]})," leaves."]}),"\n",(0,i.jsxs)(e.p,{children:["An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"u"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{null}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"null"})})]})})]})," marker inserted in.\nThis means that we can prove membership in the way described.\nHowever, we can also prove nonmembership of an element by showing that ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"u"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{null}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"null"})})]})})]})," is located in the element's hash location.\nThe crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership."]}),"\n",(0,i.jsx)(e.p,{children:"We can take advantage of the sparse nature of SMTs to provide shortened proofs.\nSpecifically, it is unlikely for entries to cluster together.\nThus, it is efficient to maintain a list of values:"}),"\n",(0,i.jsxs)(e.table,{children:[(0,i.jsx)(e.thead,{children:(0,i.jsx)(e.tr,{children:(0,i.jsx)(e.th,{children:"Null values"})})}),(0,i.jsxs)(e.tbody,{children:[(0,i.jsx)(e.tr,{children:(0,i.jsx)(e.td,{children:(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"d"}),(0,i.jsx)(e.mn,{children:"0"})]}),(0,i.jsx)(e.mo,{children:":"}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"H"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"u"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.mo,{separator:"true",children:","})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"d_0 := \\mathsf{Hash(null)},"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:":="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"Hash"}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsx)(e.span,{className:"mord mathsf",children:"null"}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]}),(0,i.jsx)(e.span,{className:"mpunct",children:","})]})]})]})})}),(0,i.jsx)(e.tr,{children:(0,i.jsx)(e.td,{children:(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"d"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.mo,{children:":"}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"H"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"d"}),(0,i.jsx)(e.mn,{mathvariant:"sans-serif",children:"0"})]}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"\u2225"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"\u2225"}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"d"}),(0,i.jsx)(e.mn,{mathvariant:"sans-serif",children:"0"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.mo,{separator:"true",children:","})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"d_1 := \\mathsf{Hash(d_0 \\|\\| d_0)},"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:":="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"Hash"}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3089em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mord",children:"\u2225\u2225"}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3089em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"0"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]}),(0,i.jsx)(e.span,{className:"mpunct",children:","})]})]})]})})}),(0,i.jsx)(e.tr,{children:(0,i.jsx)(e.td,{children:(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"d"}),(0,i.jsx)(e.mn,{children:"2"})]}),(0,i.jsx)(e.mo,{children:":"}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"H"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"d"}),(0,i.jsx)(e.mn,{mathvariant:"sans-serif",children:"1"})]}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"\u2225"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"\u2225"}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"d"}),(0,i.jsx)(e.mn,{mathvariant:"sans-serif",children:"1"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.mo,{separator:"true",children:","})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"d_2 := \\mathsf{Hash(d_1 \\|\\| d_1)},"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"2"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:":="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"Hash"}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3089em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"1"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mord",children:"\u2225\u2225"}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3089em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"1"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]}),(0,i.jsx)(e.span,{className:"mpunct",children:","})]})]})]})})}),(0,i.jsx)(e.tr,{children:(0,i.jsx)(e.td,{children:(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u22ee"}),(0,i.jsx)(e.mpadded,{height:"0em",voffset:"0em",children:(0,i.jsx)(e.mspace,{mathbackground:"black",width:"0em",height:"1.5em"})})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\vdots"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1.53em",verticalAlign:"-0.03em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"\u22ee"}),(0,i.jsx)(e.span,{className:"mord rule",style:{borderRightWidth:"0em",borderTopWidth:"1.5em",bottom:"0em"}})]})]})})]})})}),(0,i.jsx)(e.tr,{children:(0,i.jsx)(e.td,{children:(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"d"}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{children:"1"})]})]}),(0,i.jsx)(e.mo,{children:":"}),(0,i.jsx)(e.mo,{children:"="}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"H"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"a"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"s"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"h"}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"d"}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{mathvariant:"sans-serif",children:"2"})]})]}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"\u2225"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"\u2225"}),(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"d"}),(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{mathvariant:"sans-serif",children:"2"})]})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.mi,{mathvariant:"normal",children:"."})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"d_{n-1} := \\mathsf{Hash(d_{n-2} \\|\\| d_{n-2})}."})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.9028em",verticalAlign:"-0.2083em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3011em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsxs)(e.span,{className:"mord mtight",children:[(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"n"}),(0,i.jsx)(e.span,{className:"mbin mtight",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mord mtight",children:"1"})]})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2083em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}}),(0,i.jsx)(e.span,{className:"mrel",children:":="}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2778em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"Hash"}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3089em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsxs)(e.span,{className:"mord mtight",children:[(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"n"}),(0,i.jsx)(e.span,{className:"mbin mtight",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"2"})]})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2083em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mord",children:"\u2225\u2225"}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathsf",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3089em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsxs)(e.span,{className:"mord mtight",children:[(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"n"}),(0,i.jsx)(e.span,{className:"mbin mtight",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mord mathsf mtight",children:"2"})]})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2083em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]}),(0,i.jsx)(e.span,{className:"mord",children:"."})]})]})]})})})]})]}),"\n",(0,i.jsxs)(e.p,{children:["Each of the ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msub,{children:[(0,i.jsx)(e.mi,{children:"d"}),(0,i.jsx)(e.mi,{children:"i"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"d_i"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8444em",verticalAlign:"-0.15em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord mathnormal",children:"d"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.3117em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.55em",marginLeft:"0em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"i"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.15em"},children:(0,i.jsx)(e.span,{})})})]})})]})]})})]}),"'s represents the root hash of a Merkle tree with ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsx)(e.mi,{children:"i"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^i"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8247em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.8247em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",children:"i"})})]})})})})})]})]})})]})," leaves containing ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"u"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{null}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"null"})})]})})]}),".\nThese values can be used to shorten the time needed to construct an SMT and the length of proofs."]}),"\n",(0,i.jsx)(e.h3,{id:"proof-of-nonmembership",children:"Proof of nonmembership"}),"\n",(0,i.jsxs)(e.p,{children:["In the first Vac 101 [",(0,i.jsx)(e.a,{href:"https://vac.dev/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters",children:"5"}),"], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership.\nHowever, the proof of membership may result in false positives due to collisions.\nThis would affect nonmembership proofs as well.\nSparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest."]}),"\n",(0,i.jsxs)(e.p,{children:["Why is sorting essential?\nThe sorting mechanism of data can be arbitrarily chosen.\nHowever, it is essential that there are no gaps in the ordering.\nThe maximum number of elements that could ever exist in the digest must be known.\nA simple method for this is to use a hash function to provide fingerprints to the data.\nEach hash using either SHA-256 or Keccak has 256-bits.\nOur entire digest could consist of a maximum of ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsxs)(e.msup,{children:[(0,i.jsx)(e.mn,{children:"2"}),(0,i.jsx)(e.mn,{children:"256"})]})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"2^{256}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.8141em"}}),(0,i.jsxs)(e.span,{className:"mord",children:[(0,i.jsx)(e.span,{className:"mord",children:"2"}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsx)(e.span,{className:"vlist-t",children:(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.8141em"},children:(0,i.jsxs)(e.span,{style:{top:"-3.063em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:(0,i.jsx)(e.span,{className:"mord mtight",children:"256"})})})]})})})})})]})]})})]})," entries.\nThis assumes that our digest does not contain collisions."]}),"\n",(0,i.jsxs)(e.p,{children:["The fingerprint of a piece of data ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," indicates which leaf of the SMT it is contained in.\nThis means that a nonmembership of ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," in the SMT becomes a matter of proving that ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"n"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"u"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"}),(0,i.jsx)(e.mi,{mathvariant:"sans-serif",children:"l"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\mathsf{null}"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:(0,i.jsx)(e.span,{className:"mord mathsf",children:"null"})})]})})]})," is contained in ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]}),"'s location."]}),"\n",(0,i.jsxs)(e.p,{children:["It is crucial for the SMT to be sorted.\nOtherwise, a malicious party can append the entry ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{mathvariant:"normal",children:"\u2113"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\ell"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord",children:"\u2113"})]})})]})," to a random location.\nThis allows for the malicious party to provide contradictory proofs that prove both membership and nonmembership.\nWe note that the requirement that an SMT is sorted may be too strong of an assumption in centralized cases.\nHowever, sortedness is a necessary property of SMTs for decentralized systems."]}),"\n",(0,i.jsx)(e.h3,{id:"verkle-trees",children:"Verkle Trees"}),"\n",(0,i.jsxs)(e.p,{children:["A proof of membership grows in length as the Merkle tree grows.\nThe most obvious approach to remedy this scalability issue is to use Merkle trees in which each node has more than two children.\nHowever, this does not fix the issue.\nA proof of membership in a ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"k"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"k"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.03148em"},children:"k"})]})})]}),"-nary Merkle tree [",(0,i.jsx)(e.a,{href:"https://math.mit.edu/research/highschool/primes/materials/2018/Kuszmaul.pdf",children:"6"}),"] (each node has ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsx)(e.mrow,{children:(0,i.jsx)(e.mi,{children:"k"})}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"k"})]})})}),(0,i.jsx)(e.span,{className:"katex-html","aria-hidden":"true",children:(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6944em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.03148em"},children:"k"})]})})]})," children) has a proof size ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsxs)(e.msub,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"log"}),(0,i.jsx)(e.mo,{children:"\u2061"})]}),(0,i.jsx)(e.mi,{children:"k"})]}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsx)(e.mi,{children:"n"}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"}),(0,i.jsx)(e.mo,{stretchy:"false",children:"("}),(0,i.jsx)(e.mi,{children:"k"}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{children:"1"}),(0,i.jsx)(e.mo,{stretchy:"false",children:")"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"\\log_k(n)(k-1)"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsxs)(e.span,{className:"mop",children:[(0,i.jsxs)(e.span,{className:"mop",children:["lo",(0,i.jsx)(e.span,{style:{marginRight:"0.01389em"},children:"g"})]}),(0,i.jsx)(e.span,{className:"msupsub",children:(0,i.jsxs)(e.span,{className:"vlist-t vlist-t2",children:[(0,i.jsxs)(e.span,{className:"vlist-r",children:[(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.242em"},children:(0,i.jsxs)(e.span,{style:{top:"-2.4559em",marginRight:"0.05em"},children:[(0,i.jsx)(e.span,{className:"pstrut",style:{height:"2.7em"}}),(0,i.jsx)(e.span,{className:"sizing reset-size6 size3 mtight",children:(0,i.jsx)(e.span,{className:"mord mathnormal mtight",style:{marginRight:"0.03148em"},children:"k"})})]})}),(0,i.jsx)(e.span,{className:"vlist-s",children:"\u200b"})]}),(0,i.jsx)(e.span,{className:"vlist-r",children:(0,i.jsx)(e.span,{className:"vlist",style:{height:"0.2441em"},children:(0,i.jsx)(e.span,{})})})]})})]}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsx)(e.span,{className:"mord mathnormal",children:"n"}),(0,i.jsx)(e.span,{className:"mclose",children:")"}),(0,i.jsx)(e.span,{className:"mopen",children:"("}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.03148em"},children:"k"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"1em",verticalAlign:"-0.25em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"}),(0,i.jsx)(e.span,{className:"mclose",children:")"})]})]})]}),".\nThe multiple ",(0,i.jsxs)(e.span,{className:"katex",children:[(0,i.jsx)(e.span,{className:"katex-mathml",children:(0,i.jsx)(e.math,{xmlns:"http://www.w3.org/1998/Math/MathML",children:(0,i.jsxs)(e.semantics,{children:[(0,i.jsxs)(e.mrow,{children:[(0,i.jsx)(e.mi,{children:"k"}),(0,i.jsx)(e.mo,{children:"\u2212"}),(0,i.jsx)(e.mn,{children:"1"})]}),(0,i.jsx)(e.annotation,{encoding:"application/x-tex",children:"k-1"})]})})}),(0,i.jsxs)(e.span,{className:"katex-html","aria-hidden":"true",children:[(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.7778em",verticalAlign:"-0.0833em"}}),(0,i.jsx)(e.span,{className:"mord mathnormal",style:{marginRight:"0.03148em"},children:"k"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}}),(0,i.jsx)(e.span,{className:"mbin",children:"\u2212"}),(0,i.jsx)(e.span,{className:"mspace",style:{marginRight:"0.2222em"}})]}),(0,i.jsxs)(e.span,{className:"base",children:[(0,i.jsx)(e.span,{className:"strut",style:{height:"0.6444em"}}),(0,i.jsx)(e.span,{className:"mord",children:"1"})]})]})]})," is the number of silbings that a node has on each layer.\nHence, the proof size grows faster than a logarithmic function of the digest size."]}),"\n",(0,i.jsxs)(e.p,{children:["An alternate approach is to use a different data structure: Verkle trees [",(0,i.jsx)(e.a,{href:"https://math.mit.edu/research/highschool/primes/materials/2018/Kuszmaul.pdf",children:"6"}),"].\nA Verkle tree replaces hash functions with polynomial commitments [",(0,i.jsx)(e.a,{href:"https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095",children:"7"}),", ",(0,i.jsx)(e.a,{href:"https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html",children:"8"}),"].\nWe will explore Verkle trees in a future Vac 101 edition."]}),"\n",(0,i.jsx)(e.h2,{id:"references",children:"References"}),"\n",(0,i.jsxs)(e.ul,{children:["\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://www.ralphmerkle.com/papers/Thesis1979.pdf",children:"Secrecy, Authentication, and Public Key Systems"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"2",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://vac.dev/rlog/rln-light-verifiers/",children:"Verifying RLN Proofs in Light Clients with Subtrees"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"3",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://vac.dev/rlog/vac101-fiat-shamir#hash-functions",children:"Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"4",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://github.com/0xPolygonZero/plonky2/blob/main/plonky2/src/hash/merkle_tree.rs",children:"Capped merkle tree in Plonky2"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"5",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://vac.dev/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters",children:"Vac 101: Membership with Bloom Filters and Cuckoo Filters"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"6",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://math.mit.edu/research/highschool/primes/materials/2018/Kuszmaul.pdf",children:"Verkle Trees"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"7",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://ethresear.ch/t/using-polynomial-commitments-to-replace-state-roots/7095",children:"Using polynomial commitments to replace state roots"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"8",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html",children:"KZG polynomial commitments"})}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(e.li,{children:["\n",(0,i.jsxs)(e.ol,{start:"9",children:["\n",(0,i.jsx)(e.li,{children:(0,i.jsx)(e.a,{href:"https://github.com/o1-labs/verkle-tree",children:"O1 labs' Verkle Tree repo"})}),"\n"]}),"\n"]}),"\n"]})]})}function d(s={}){const{wrapper:e}={...(0,t.R)(),...s.components};return e?(0,i.jsx)(e,{...s,children:(0,i.jsx)(c,{...s})}):c(s)}},54950:(s,e,a)=>{a.d(e,{A:()=>n});const n=a.p+"assets/images/vac101_merkle_tree-a7c86f78d5aa8016921924220d6005fc.png"},74721:s=>{s.exports=JSON.parse('{"permalink":"/rlog/climbing-merkle-trees","source":"@site/rlog/2024-12-30-merkle-tree.mdx","title":"Vac 101: Climbing Merkle Trees","description":"In this post, we introduce a crucial data structure used throughout web3.","date":"2024-12-30T12:00:00.000Z","tags":[],"readingTime":9.68,"hasTruncateMarker":true,"authors":[{"name":"Marvin","github":"jonesmarvin8","key":"marvin","page":null}],"frontMatter":{"title":"Vac 101: Climbing Merkle Trees","date":"2024-12-30T12:00:00.000Z","authors":"marvin","published":true,"slug":"climbing-merkle-trees","categories":"research","toc_min_heading_level":2,"toc_max_heading_level":4},"unlisted":false,"prevItem":{"title":"Vac 2024 Year in Review","permalink":"/rlog/2024-recap"},"nextItem":{"title":"Large Message Handling in GossipSub: Potential Improvements","permalink":"/rlog/gsub-largemsg-improvements"}}')}}]); \ No newline at end of file diff --git a/assets/js/0b46e8e7.64cdc00f.js b/assets/js/0b46e8e7.64cdc00f.js deleted file mode 100644 index a09a1d0e..00000000 --- a/assets/js/0b46e8e7.64cdc00f.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[5387],{94644:(e,t,r)=>{r.r(t),r.d(t,{assets:()=>c,contentTitle:()=>i,default:()=>u,frontMatter:()=>o,metadata:()=>p,toc:()=>l});var n=r(87462),a=(r(67294),r(3905));const o={hide_table_of_contents:!0},i="Publications",p={unversionedId:"publications",id:"publications",title:"Publications",description:"Papers",source:"@site/docs/publications.md",sourceDirName:".",slug:"/publications",permalink:"/publications",draft:!1,tags:[],version:"current",frontMatter:{hide_table_of_contents:!0}},c={},l=[{value:"Papers",id:"papers",level:2},{value:"Write-ups",id:"write-ups",level:2}],s={toc:l};function u(e){let{components:t,...r}=e;return(0,a.kt)("wrapper",(0,n.Z)({},s,r,{components:t,mdxType:"MDXLayout"}),(0,a.kt)("h1",{id:"publications"},"Publications"),(0,a.kt)("h2",{id:"papers"},"Papers"),(0,a.kt)("ul",null,(0,a.kt)("li",{parentName:"ul"},(0,a.kt)("p",{parentName:"li"},(0,a.kt)("a",{parentName:"p",href:"https://arxiv.org/abs/2207.00038"},"Waku: A Family of Modular P2P Protocols For Secure & Censorship-Resistant Communication")," (demo)")),(0,a.kt)("li",{parentName:"ul"},(0,a.kt)("p",{parentName:"li"},(0,a.kt)("a",{parentName:"p",href:"https://arxiv.org/pdf/2207.00117.pdf"},"WAKU-RLN-RELAY: Privacy-Preserving Peer-to-Peer Economic Spam Protection")," (full)")),(0,a.kt)("li",{parentName:"ul"},(0,a.kt)("p",{parentName:"li"},(0,a.kt)("a",{parentName:"p",href:"https://arxiv.org/abs/2207.00116"},"Privacy-Preserving Spam-Protected Gossip-Based Routing")," (poster)"))),(0,a.kt)("h2",{id:"write-ups"},"Write-ups"),(0,a.kt)("p",null,"See ",(0,a.kt)("a",{parentName:"p",href:"/rlog"},"write-ups"),"."))}u.isMDXComponent=!0},3905:(e,t,r)=>{r.d(t,{Zo:()=>s,kt:()=>m});var n=r(67294);function a(e,t,r){return t in e?Object.defineProperty(e,t,{value:r,enumerable:!0,configurable:!0,writable:!0}):e[t]=r,e}function o(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),r.push.apply(r,n)}return r}function i(e){for(var t=1;t=0||(a[r]=e[r]);return a}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,r)&&(a[r]=e[r])}return a}var c=n.createContext({}),l=function(e){var t=n.useContext(c),r=t;return e&&(r="function"==typeof e?e(t):i(i({},t),e)),r},s=function(e){var t=l(e.components);return n.createElement(c.Provider,{value:t},e.children)},u={inlineCode:"code",wrapper:function(e){var t=e.children;return n.createElement(n.Fragment,{},t)}},f=n.forwardRef((function(e,t){var r=e.components,a=e.mdxType,o=e.originalType,c=e.parentName,s=p(e,["components","mdxType","originalType","parentName"]),f=l(r),m=a,d=f["".concat(c,".").concat(m)]||f[m]||u[m]||o;return r?n.createElement(d,i(i({ref:t},s),{},{components:r})):n.createElement(d,i({ref:t},s))}));function m(e,t){var r=arguments,a=t&&t.mdxType;if("string"==typeof e||a){var o=r.length,i=new Array(o);i[0]=f;var p={};for(var c in t)hasOwnProperty.call(t,c)&&(p[c]=t[c]);p.originalType=e,p.mdxType="string"==typeof e?e:a,i[1]=p;for(var l=2;l{s.d(n,{R:()=>o,x:()=>c});var t=s(96540);const r={},i=t.createContext(r);function o(e){const n=t.useContext(i);return t.useMemo((function(){return"function"==typeof e?e(n):{...n,...e}}),[n,e])}function c(e){let n;return n=e.disableParentContext?"function"==typeof e.components?e.components(r):e.components||r:o(e.components),t.createElement(i.Provider,{value:n},e.children)}},85581:(e,n,s)=>{s.r(n),s.d(n,{assets:()=>a,contentTitle:()=>c,default:()=>d,frontMatter:()=>o,metadata:()=>t,toc:()=>l});const t=JSON.parse('{"id":"publications","title":"Publications","description":"Papers","source":"@site/docs/publications.md","sourceDirName":".","slug":"/publications","permalink":"/publications","draft":false,"unlisted":false,"tags":[],"version":"current","frontMatter":{"hide_table_of_contents":true}}');var r=s(74848),i=s(28453);const o={hide_table_of_contents:!0},c="Publications",a={},l=[{value:"Papers",id:"papers",level:2},{value:"Write-ups",id:"write-ups",level:2}];function u(e){const n={a:"a",h1:"h1",h2:"h2",header:"header",li:"li",p:"p",ul:"ul",...(0,i.R)(),...e.components};return(0,r.jsxs)(r.Fragment,{children:[(0,r.jsx)(n.header,{children:(0,r.jsx)(n.h1,{id:"publications",children:"Publications"})}),"\n",(0,r.jsx)(n.h2,{id:"papers",children:"Papers"}),"\n",(0,r.jsxs)(n.ul,{children:["\n",(0,r.jsxs)(n.li,{children:["\n",(0,r.jsxs)(n.p,{children:[(0,r.jsx)(n.a,{href:"https://arxiv.org/abs/2207.00038",children:"Waku: A Family of Modular P2P Protocols For Secure & Censorship-Resistant Communication"})," (demo)"]}),"\n"]}),"\n",(0,r.jsxs)(n.li,{children:["\n",(0,r.jsxs)(n.p,{children:[(0,r.jsx)(n.a,{href:"https://arxiv.org/pdf/2207.00117.pdf",children:"WAKU-RLN-RELAY: Privacy-Preserving Peer-to-Peer Economic Spam Protection"})," (full)"]}),"\n"]}),"\n",(0,r.jsxs)(n.li,{children:["\n",(0,r.jsxs)(n.p,{children:[(0,r.jsx)(n.a,{href:"https://arxiv.org/abs/2207.00116",children:"Privacy-Preserving Spam-Protected Gossip-Based Routing"})," (poster)"]}),"\n"]}),"\n"]}),"\n",(0,r.jsx)(n.h2,{id:"write-ups",children:"Write-ups"}),"\n",(0,r.jsxs)(n.p,{children:["See ",(0,r.jsx)(n.a,{href:"/rlog",children:"write-ups"}),"."]})]})}function d(e={}){const{wrapper:n}={...(0,i.R)(),...e.components};return n?(0,r.jsx)(n,{...e,children:(0,r.jsx)(u,{...e})}):u(e)}}}]); \ No newline at end of file diff --git a/assets/js/0d54ff0d.b508d506.js b/assets/js/0d54ff0d.b508d506.js deleted file mode 100644 index c4f0d470..00000000 --- a/assets/js/0d54ff0d.b508d506.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[4248],{36733:(e,t,n)=>{n.r(t),n.d(t,{assets:()=>u,contentTitle:()=>i,default:()=>p,frontMatter:()=>o,metadata:()=>c,toc:()=>l});var r=n(87462),a=(n(67294),n(3905));const o={layout:"post",name:"Introducing nwaku",title:"Introducing nwaku",date:new Date("2022-04-12T10:00:00.000Z"),authors:"hanno",published:!0,slug:"introducing-nwaku",categories:"research",discuss:"https://forum.vac.dev/",toc_min_heading_level:2,toc_max_heading_level:5},i=void 0,c={permalink:"/rlog/introducing-nwaku",source:"@site/rlog/2022-04-12-introducing-nwaku.mdx",title:"Introducing nwaku",description:"Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.",date:"2022-04-12T10:00:00.000Z",formattedDate:"April 12, 2022",tags:[],readingTime:10.765,hasTruncateMarker:!0,authors:[{name:"Hanno Cornelius",twitter:"4aelius",github:"jm-clius",key:"hanno"}],frontMatter:{layout:"post",name:"Introducing nwaku",title:"Introducing nwaku",date:"2022-04-12T10:00:00.000Z",authors:"hanno",published:!0,slug:"introducing-nwaku",categories:"research",discuss:"https://forum.vac.dev/",toc_min_heading_level:2,toc_max_heading_level:5},prevItem:{title:"Waku v2 Ambient Peer Discovery",permalink:"/rlog/wakuv2-apd"},nextItem:{title:"Opinion: Pseudo-ethics in the Surveillance Tech Industry",permalink:"/rlog/ethics-surveillance-tech"}},u={authorsImageUrls:[void 0]},l=[],s={toc:l};function p(e){let{components:t,...n}=e;return(0,a.kt)("wrapper",(0,r.Z)({},s,n,{components:t,mdxType:"MDXLayout"}),(0,a.kt)("p",null,"Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas."))}p.isMDXComponent=!0},3905:(e,t,n)=>{n.d(t,{Zo:()=>s,kt:()=>m});var r=n(67294);function a(e,t,n){return t in e?Object.defineProperty(e,t,{value:n,enumerable:!0,configurable:!0,writable:!0}):e[t]=n,e}function o(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function i(e){for(var t=1;t=0||(a[n]=e[n]);return a}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(a[n]=e[n])}return a}var u=r.createContext({}),l=function(e){var t=r.useContext(u),n=t;return e&&(n="function"==typeof e?e(t):i(i({},t),e)),n},s=function(e){var t=l(e.components);return r.createElement(u.Provider,{value:t},e.children)},p={inlineCode:"code",wrapper:function(e){var t=e.children;return r.createElement(r.Fragment,{},t)}},d=r.forwardRef((function(e,t){var n=e.components,a=e.mdxType,o=e.originalType,u=e.parentName,s=c(e,["components","mdxType","originalType","parentName"]),d=l(n),m=a,f=d["".concat(u,".").concat(m)]||d[m]||p[m]||o;return n?r.createElement(f,i(i({ref:t},s),{},{components:n})):r.createElement(f,i({ref:t},s))}));function m(e,t){var n=arguments,a=t&&t.mdxType;if("string"==typeof e||a){var o=n.length,i=new Array(o);i[0]=d;var c={};for(var u in t)hasOwnProperty.call(t,u)&&(c[u]=t[u]);c.originalType=e,c.mdxType="string"==typeof e?e:a,i[1]=c;for(var l=2;l{e.exports=JSON.parse('{"permalink":"/rlog/introducing-nwaku","source":"@site/rlog/2022-04-12-introducing-nwaku.mdx","title":"Introducing nwaku","description":"Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.","date":"2022-04-12T10:00:00.000Z","tags":[],"readingTime":10.775,"hasTruncateMarker":true,"authors":[{"name":"Hanno Cornelius","twitter":"4aelius","github":"jm-clius","key":"hanno","page":null}],"frontMatter":{"layout":"post","name":"Introducing nwaku","title":"Introducing nwaku","date":"2022-04-12T10:00:00.000Z","authors":"hanno","published":true,"slug":"introducing-nwaku","categories":"research","discuss":"https://forum.vac.dev/","toc_min_heading_level":2,"toc_max_heading_level":5},"unlisted":false,"prevItem":{"title":"Waku v2 Ambient Peer Discovery","permalink":"/rlog/wakuv2-apd"},"nextItem":{"title":"Opinion: Pseudo-ethics in the Surveillance Tech Industry","permalink":"/rlog/ethics-surveillance-tech"}}')},28453:(e,n,t)=>{t.d(n,{R:()=>u,x:()=>i});var a=t(96540);const r={},o=a.createContext(r);function u(e){const n=a.useContext(o);return a.useMemo((function(){return"function"==typeof e?e(n):{...n,...e}}),[n,e])}function i(e){let n;return n=e.disableParentContext?"function"==typeof e.components?e.components(r):e.components||r:u(e.components),a.createElement(o.Provider,{value:n},e.children)}},50155:(e,n,t)=>{t.r(n),t.d(n,{assets:()=>s,contentTitle:()=>i,default:()=>d,frontMatter:()=>u,metadata:()=>a,toc:()=>c});var a=t(7386),r=t(74848),o=t(28453);const u={layout:"post",name:"Introducing nwaku",title:"Introducing nwaku",date:new Date("2022-04-12T10:00:00.000Z"),authors:"hanno",published:!0,slug:"introducing-nwaku",categories:"research",discuss:"https://forum.vac.dev/",toc_min_heading_level:2,toc_max_heading_level:5},i=void 0,s={authorsImageUrls:[void 0]},c=[];function l(e){const n={p:"p",...(0,o.R)(),...e.components};return(0,r.jsx)(n.p,{children:"Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas."})}function d(e={}){const{wrapper:n}={...(0,o.R)(),...e.components};return n?(0,r.jsx)(n,{...e,children:(0,r.jsx)(l,{...e})}):l(e)}}}]); \ No newline at end of file diff --git a/assets/js/0ecccfe0.4fa4329a.js b/assets/js/0ecccfe0.4fa4329a.js deleted file mode 100644 index 0f4b97f2..00000000 --- a/assets/js/0ecccfe0.4fa4329a.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[8562],{65505:(e,t,r)=>{r.r(t),r.d(t,{assets:()=>c,contentTitle:()=>i,default:()=>u,frontMatter:()=>o,metadata:()=>s,toc:()=>p});var n=r(87462),a=(r(67294),r(3905));const o={title:"Libp2p GossipSub IDONTWANT Message Performance Impact",date:new Date("2024-10-28T12:00:00.000Z"),authors:"farooq",published:!0,slug:"gsub-idontwant-perf-eval",categories:"research",discuss:"https://forum.vac.dev/t/libp2p-gossipsub-idontwant-message-performance-impact/374",toc_min_heading_level:2,toc_max_heading_level:5},i=void 0,s={permalink:"/rlog/gsub-idontwant-perf-eval",source:"@site/rlog/2024-10-28-gsub-idontwant-perf-eval.mdx",title:"Libp2p GossipSub IDONTWANT Message Performance Impact",description:"This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.",date:"2024-10-28T12:00:00.000Z",formattedDate:"October 28, 2024",tags:[],readingTime:3.38,hasTruncateMarker:!0,authors:[{name:"Umar Farooq",github:"ufarooqstatus",key:"farooq"}],frontMatter:{title:"Libp2p GossipSub IDONTWANT Message Performance Impact",date:"2024-10-28T12:00:00.000Z",authors:"farooq",published:!0,slug:"gsub-idontwant-perf-eval",categories:"research",discuss:"https://forum.vac.dev/t/libp2p-gossipsub-idontwant-message-performance-impact/374",toc_min_heading_level:2,toc_max_heading_level:5},prevItem:{title:"Large Message Handling in GossipSub: Potential Improvements",permalink:"/rlog/gsub-largemsg-improvements"},nextItem:{title:"Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument",permalink:"/rlog/vac101-fiat-shamir"}},c={authorsImageUrls:[void 0]},p=[],l={toc:p};function u(e){let{components:t,...r}=e;return(0,a.kt)("wrapper",(0,n.Z)({},l,r,{components:t,mdxType:"MDXLayout"}),(0,a.kt)("p",null,"This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains."))}u.isMDXComponent=!0},3905:(e,t,r)=>{r.d(t,{Zo:()=>l,kt:()=>f});var n=r(67294);function a(e,t,r){return t in e?Object.defineProperty(e,t,{value:r,enumerable:!0,configurable:!0,writable:!0}):e[t]=r,e}function o(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),r.push.apply(r,n)}return r}function i(e){for(var t=1;t=0||(a[r]=e[r]);return a}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,r)&&(a[r]=e[r])}return a}var c=n.createContext({}),p=function(e){var t=n.useContext(c),r=t;return e&&(r="function"==typeof e?e(t):i(i({},t),e)),r},l=function(e){var t=p(e.components);return n.createElement(c.Provider,{value:t},e.children)},u={inlineCode:"code",wrapper:function(e){var t=e.children;return n.createElement(n.Fragment,{},t)}},m=n.forwardRef((function(e,t){var r=e.components,a=e.mdxType,o=e.originalType,c=e.parentName,l=s(e,["components","mdxType","originalType","parentName"]),m=p(r),f=a,g=m["".concat(c,".").concat(f)]||m[f]||u[f]||o;return r?n.createElement(g,i(i({ref:t},l),{},{components:r})):n.createElement(g,i({ref:t},l))}));function f(e,t){var r=arguments,a=t&&t.mdxType;if("string"==typeof e||a){var o=r.length,i=new Array(o);i[0]=m;var s={};for(var c in t)hasOwnProperty.call(t,c)&&(s[c]=t[c]);s.originalType=e,s.mdxType="string"==typeof e?e:a,i[1]=s;for(var p=2;p{e.exports=JSON.parse('{"permalink":"/rlog/gsub-idontwant-perf-eval","source":"@site/rlog/2024-10-28-gsub-idontwant-perf-eval.mdx","title":"Libp2p GossipSub IDONTWANT Message Performance Impact","description":"This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.","date":"2024-10-28T12:00:00.000Z","tags":[],"readingTime":3.39,"hasTruncateMarker":true,"authors":[{"name":"Umar Farooq","github":"ufarooqstatus","key":"farooq","page":null}],"frontMatter":{"title":"Libp2p GossipSub IDONTWANT Message Performance Impact","date":"2024-10-28T12:00:00.000Z","authors":"farooq","published":true,"slug":"gsub-idontwant-perf-eval","categories":"research","discuss":"https://forum.vac.dev/t/libp2p-gossipsub-idontwant-message-performance-impact/374","toc_min_heading_level":2,"toc_max_heading_level":5},"unlisted":false,"prevItem":{"title":"Large Message Handling in GossipSub: Potential Improvements","permalink":"/rlog/gsub-largemsg-improvements"},"nextItem":{"title":"Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument","permalink":"/rlog/vac101-fiat-shamir"}}')},22307:(e,t,n)=>{n.r(t),n.d(t,{assets:()=>c,contentTitle:()=>i,default:()=>m,frontMatter:()=>r,metadata:()=>a,toc:()=>p});var a=n(4053),s=n(74848),o=n(28453);const r={title:"Libp2p GossipSub IDONTWANT Message Performance Impact",date:new Date("2024-10-28T12:00:00.000Z"),authors:"farooq",published:!0,slug:"gsub-idontwant-perf-eval",categories:"research",discuss:"https://forum.vac.dev/t/libp2p-gossipsub-idontwant-message-performance-impact/374",toc_min_heading_level:2,toc_max_heading_level:5},i=void 0,c={authorsImageUrls:[void 0]},p=[];function u(e){const t={p:"p",...(0,o.R)(),...e.components};return(0,s.jsx)(t.p,{children:"This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains."})}function m(e={}){const{wrapper:t}={...(0,o.R)(),...e.components};return t?(0,s.jsx)(t,{...e,children:(0,s.jsx)(u,{...e})}):u(e)}},28453:(e,t,n)=>{n.d(t,{R:()=>r,x:()=>i});var a=n(96540);const s={},o=a.createContext(s);function r(e){const t=a.useContext(o);return a.useMemo((function(){return"function"==typeof e?e(t):{...t,...e}}),[t,e])}function i(e){let t;return t=e.disableParentContext?"function"==typeof e.components?e.components(s):e.components||s:r(e.components),a.createElement(o.Provider,{value:t},e.children)}}}]); \ No newline at end of file diff --git a/assets/js/124805df.8ce48749.js b/assets/js/124805df.8ce48749.js deleted file mode 100644 index ad4bd146..00000000 --- a/assets/js/124805df.8ce48749.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[2206],{9935:e=>{e.exports=JSON.parse('{"permalink":"/rlog","page":1,"postsPerPage":10,"totalPages":5,"totalCount":41,"nextPage":"/rlog/page/2","blogDescription":"Blog","blogTitle":"Research Blog"}')}}]); \ No newline at end of file diff --git a/assets/js/1259.4db4ff1c.js b/assets/js/1259.4db4ff1c.js new file mode 100644 index 00000000..f63592f3 --- /dev/null +++ b/assets/js/1259.4db4ff1c.js @@ -0,0 +1 @@ +"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[1259],{73222:(e,t,s)=>{s.d(t,{A:()=>r,P:()=>a});var i=s(42107),n=s(20007),r=(0,i.K2)(((e,t)=>{let s;"sandbox"===t&&(s=(0,n.Ltv)("#i"+e));return("sandbox"===t?(0,n.Ltv)(s.nodes()[0].contentDocument.body):(0,n.Ltv)("body")).select(`[id="${e}"]`)}),"getDiagramElement"),a=(0,i.K2)(((e,t,s,n)=>{e.attr("class",s);const{width:r,height:a,x:o,y:h}=l(e,t);(0,i.a$)(e,a,r,n);const u=c(o,h,r,a,t);e.attr("viewBox",u),i.Rm.debug(`viewBox configured: ${u} with padding: ${t}`)}),"setupViewPortForSVG"),l=(0,i.K2)(((e,t)=>{const s=e.node()?.getBBox()||{width:0,height:0,x:0,y:0};return{width:s.width+2*t,height:s.height+2*t,x:s.x,y:s.y}}),"calculateDimensionsWithPadding"),c=(0,i.K2)(((e,t,s,i,n)=>`${e-n} ${t-n} ${s} ${i}`),"createViewBox")},81259:(e,t,s)=>{s.d(t,{diagram:()=>m});var i=s(73222),n=s(76044),r=(s(36157),s(67231),s(44007),s(72355),s(46503),s(74431),s(55395)),a=s(42107),l=function(){var e=(0,a.K2)((function(e,t,s,i){for(s=s||{},i=e.length;i--;s[e[i]]=t);return s}),"o"),t=[1,3],s=[1,4],i=[1,5],n=[1,6],r=[5,6,8,9,11,13,21,22,23,24,41,42,43,44,45,46,54,72,74,77,89,90],l=[1,22],c=[2,7],o=[1,26],h=[1,27],u=[1,28],y=[1,29],m=[1,33],d=[1,34],E=[1,35],p=[1,36],R=[1,37],f=[1,38],_=[1,24],g=[1,31],S=[1,32],I=[1,30],b=[1,39],T=[1,40],k=[5,8,9,11,13,21,22,23,24,41,42,43,44,45,46,54,72,74,77,89,90],N=[1,61],q=[89,90],A=[5,8,9,11,13,21,22,23,24,27,29,41,42,43,44,45,46,54,61,63,72,74,75,76,77,80,81,82,83,84,85,86,87,88,89,90],C=[27,29],v=[1,70],w=[1,71],x=[1,72],L=[1,73],D=[1,74],O=[1,75],$=[1,76],M=[1,83],F=[1,80],K=[1,84],P=[1,85],V=[1,86],U=[1,87],Y=[1,88],B=[1,89],Q=[1,90],H=[1,91],W=[1,92],j=[5,8,9,11,13,21,22,23,24,27,41,42,43,44,45,46,54,72,74,75,76,77,80,81,82,83,84,85,86,87,88,89,90],G=[63,64],z=[1,101],X=[5,8,9,11,13,21,22,23,24,41,42,43,44,45,46,54,72,74,76,77,89,90],J=[5,8,9,11,13,21,22,23,24,41,42,43,44,45,46,54,72,74,75,76,77,80,81,82,83,84,85,86,87,88,89,90],Z=[1,110],ee=[1,106],te=[1,107],se=[1,108],ie=[1,109],ne=[1,111],re=[1,116],ae=[1,117],le=[1,114],ce=[1,115],oe={trace:(0,a.K2)((function(){}),"trace"),yy:{},symbols_:{error:2,start:3,directive:4,NEWLINE:5,RD:6,diagram:7,EOF:8,acc_title:9,acc_title_value:10,acc_descr:11,acc_descr_value:12,acc_descr_multiline_value:13,requirementDef:14,elementDef:15,relationshipDef:16,direction:17,styleStatement:18,classDefStatement:19,classStatement:20,direction_tb:21,direction_bt:22,direction_rl:23,direction_lr:24,requirementType:25,requirementName:26,STRUCT_START:27,requirementBody:28,STYLE_SEPARATOR:29,idList:30,ID:31,COLONSEP:32,id:33,TEXT:34,text:35,RISK:36,riskLevel:37,VERIFYMTHD:38,verifyType:39,STRUCT_STOP:40,REQUIREMENT:41,FUNCTIONAL_REQUIREMENT:42,INTERFACE_REQUIREMENT:43,PERFORMANCE_REQUIREMENT:44,PHYSICAL_REQUIREMENT:45,DESIGN_CONSTRAINT:46,LOW_RISK:47,MED_RISK:48,HIGH_RISK:49,VERIFY_ANALYSIS:50,VERIFY_DEMONSTRATION:51,VERIFY_INSPECTION:52,VERIFY_TEST:53,ELEMENT:54,elementName:55,elementBody:56,TYPE:57,type:58,DOCREF:59,ref:60,END_ARROW_L:61,relationship:62,LINE:63,END_ARROW_R:64,CONTAINS:65,COPIES:66,DERIVES:67,SATISFIES:68,VERIFIES:69,REFINES:70,TRACES:71,CLASSDEF:72,stylesOpt:73,CLASS:74,ALPHA:75,COMMA:76,STYLE:77,style:78,styleComponent:79,NUM:80,COLON:81,UNIT:82,SPACE:83,BRKT:84,PCT:85,MINUS:86,LABEL:87,SEMICOLON:88,unqString:89,qString:90,$accept:0,$end:1},terminals_:{2:"error",5:"NEWLINE",6:"RD",8:"EOF",9:"acc_title",10:"acc_title_value",11:"acc_descr",12:"acc_descr_value",13:"acc_descr_multiline_value",21:"direction_tb",22:"direction_bt",23:"direction_rl",24:"direction_lr",27:"STRUCT_START",29:"STYLE_SEPARATOR",31:"ID",32:"COLONSEP",34:"TEXT",36:"RISK",38:"VERIFYMTHD",40:"STRUCT_STOP",41:"REQUIREMENT",42:"FUNCTIONAL_REQUIREMENT",43:"INTERFACE_REQUIREMENT",44:"PERFORMANCE_REQUIREMENT",45:"PHYSICAL_REQUIREMENT",46:"DESIGN_CONSTRAINT",47:"LOW_RISK",48:"MED_RISK",49:"HIGH_RISK",50:"VERIFY_ANALYSIS",51:"VERIFY_DEMONSTRATION",52:"VERIFY_INSPECTION",53:"VERIFY_TEST",54:"ELEMENT",57:"TYPE",59:"DOCREF",61:"END_ARROW_L",63:"LINE",64:"END_ARROW_R",65:"CONTAINS",66:"COPIES",67:"DERIVES",68:"SATISFIES",69:"VERIFIES",70:"REFINES",71:"TRACES",72:"CLASSDEF",74:"CLASS",75:"ALPHA",76:"COMMA",77:"STYLE",80:"NUM",81:"COLON",82:"UNIT",83:"SPACE",84:"BRKT",85:"PCT",86:"MINUS",87:"LABEL",88:"SEMICOLON",89:"unqString",90:"qString"},productions_:[0,[3,3],[3,2],[3,4],[4,2],[4,2],[4,1],[7,0],[7,2],[7,2],[7,2],[7,2],[7,2],[7,2],[7,2],[7,2],[7,2],[17,1],[17,1],[17,1],[17,1],[14,5],[14,7],[28,5],[28,5],[28,5],[28,5],[28,2],[28,1],[25,1],[25,1],[25,1],[25,1],[25,1],[25,1],[37,1],[37,1],[37,1],[39,1],[39,1],[39,1],[39,1],[15,5],[15,7],[56,5],[56,5],[56,2],[56,1],[16,5],[16,5],[62,1],[62,1],[62,1],[62,1],[62,1],[62,1],[62,1],[19,3],[20,3],[20,3],[30,1],[30,3],[30,1],[30,3],[18,3],[73,1],[73,3],[78,1],[78,2],[79,1],[79,1],[79,1],[79,1],[79,1],[79,1],[79,1],[79,1],[79,1],[79,1],[26,1],[26,1],[33,1],[33,1],[35,1],[35,1],[55,1],[55,1],[58,1],[58,1],[60,1],[60,1]],performAction:(0,a.K2)((function(e,t,s,i,n,r,a){var l=r.length-1;switch(n){case 4:this.$=r[l].trim(),i.setAccTitle(this.$);break;case 5:case 6:this.$=r[l].trim(),i.setAccDescription(this.$);break;case 7:this.$=[];break;case 17:i.setDirection("TB");break;case 18:i.setDirection("BT");break;case 19:i.setDirection("RL");break;case 20:i.setDirection("LR");break;case 21:i.addRequirement(r[l-3],r[l-4]);break;case 22:i.addRequirement(r[l-5],r[l-6]),i.setClass([r[l-5]],r[l-3]);break;case 23:i.setNewReqId(r[l-2]);break;case 24:i.setNewReqText(r[l-2]);break;case 25:i.setNewReqRisk(r[l-2]);break;case 26:i.setNewReqVerifyMethod(r[l-2]);break;case 29:this.$=i.RequirementType.REQUIREMENT;break;case 30:this.$=i.RequirementType.FUNCTIONAL_REQUIREMENT;break;case 31:this.$=i.RequirementType.INTERFACE_REQUIREMENT;break;case 32:this.$=i.RequirementType.PERFORMANCE_REQUIREMENT;break;case 33:this.$=i.RequirementType.PHYSICAL_REQUIREMENT;break;case 34:this.$=i.RequirementType.DESIGN_CONSTRAINT;break;case 35:this.$=i.RiskLevel.LOW_RISK;break;case 36:this.$=i.RiskLevel.MED_RISK;break;case 37:this.$=i.RiskLevel.HIGH_RISK;break;case 38:this.$=i.VerifyType.VERIFY_ANALYSIS;break;case 39:this.$=i.VerifyType.VERIFY_DEMONSTRATION;break;case 40:this.$=i.VerifyType.VERIFY_INSPECTION;break;case 41:this.$=i.VerifyType.VERIFY_TEST;break;case 42:i.addElement(r[l-3]);break;case 43:i.addElement(r[l-5]),i.setClass([r[l-5]],r[l-3]);break;case 44:i.setNewElementType(r[l-2]);break;case 45:i.setNewElementDocRef(r[l-2]);break;case 48:i.addRelationship(r[l-2],r[l],r[l-4]);break;case 49:i.addRelationship(r[l-2],r[l-4],r[l]);break;case 50:this.$=i.Relationships.CONTAINS;break;case 51:this.$=i.Relationships.COPIES;break;case 52:this.$=i.Relationships.DERIVES;break;case 53:this.$=i.Relationships.SATISFIES;break;case 54:this.$=i.Relationships.VERIFIES;break;case 55:this.$=i.Relationships.REFINES;break;case 56:this.$=i.Relationships.TRACES;break;case 57:this.$=r[l-2],i.defineClass(r[l-1],r[l]);break;case 58:i.setClass(r[l-1],r[l]);break;case 59:i.setClass([r[l-2]],r[l]);break;case 60:case 62:case 65:this.$=[r[l]];break;case 61:case 63:this.$=r[l-2].concat([r[l]]);break;case 64:this.$=r[l-2],i.setCssStyle(r[l-1],r[l]);break;case 66:r[l-2].push(r[l]),this.$=r[l-2];break;case 68:this.$=r[l-1]+r[l]}}),"anonymous"),table:[{3:1,4:2,6:t,9:s,11:i,13:n},{1:[3]},{3:8,4:2,5:[1,7],6:t,9:s,11:i,13:n},{5:[1,9]},{10:[1,10]},{12:[1,11]},e(r,[2,6]),{3:12,4:2,6:t,9:s,11:i,13:n},{1:[2,2]},{4:17,5:l,7:13,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},e(r,[2,4]),e(r,[2,5]),{1:[2,1]},{8:[1,41]},{4:17,5:l,7:42,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:43,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:44,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:45,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:46,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:47,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:48,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:49,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{4:17,5:l,7:50,8:c,9:s,11:i,13:n,14:14,15:15,16:16,17:18,18:19,19:20,20:21,21:o,22:h,23:u,24:y,25:23,33:25,41:m,42:d,43:E,44:p,45:R,46:f,54:_,72:g,74:S,77:I,89:b,90:T},{26:51,89:[1,52],90:[1,53]},{55:54,89:[1,55],90:[1,56]},{29:[1,59],61:[1,57],63:[1,58]},e(k,[2,17]),e(k,[2,18]),e(k,[2,19]),e(k,[2,20]),{30:60,33:62,75:N,89:b,90:T},{30:63,33:62,75:N,89:b,90:T},{30:64,33:62,75:N,89:b,90:T},e(q,[2,29]),e(q,[2,30]),e(q,[2,31]),e(q,[2,32]),e(q,[2,33]),e(q,[2,34]),e(A,[2,81]),e(A,[2,82]),{1:[2,3]},{8:[2,8]},{8:[2,9]},{8:[2,10]},{8:[2,11]},{8:[2,12]},{8:[2,13]},{8:[2,14]},{8:[2,15]},{8:[2,16]},{27:[1,65],29:[1,66]},e(C,[2,79]),e(C,[2,80]),{27:[1,67],29:[1,68]},e(C,[2,85]),e(C,[2,86]),{62:69,65:v,66:w,67:x,68:L,69:D,70:O,71:$},{62:77,65:v,66:w,67:x,68:L,69:D,70:O,71:$},{30:78,33:62,75:N,89:b,90:T},{73:79,75:M,76:F,78:81,79:82,80:K,81:P,82:V,83:U,84:Y,85:B,86:Q,87:H,88:W},e(j,[2,60]),e(j,[2,62]),{73:93,75:M,76:F,78:81,79:82,80:K,81:P,82:V,83:U,84:Y,85:B,86:Q,87:H,88:W},{30:94,33:62,75:N,76:F,89:b,90:T},{5:[1,95]},{30:96,33:62,75:N,89:b,90:T},{5:[1,97]},{30:98,33:62,75:N,89:b,90:T},{63:[1,99]},e(G,[2,50]),e(G,[2,51]),e(G,[2,52]),e(G,[2,53]),e(G,[2,54]),e(G,[2,55]),e(G,[2,56]),{64:[1,100]},e(k,[2,59],{76:F}),e(k,[2,64],{76:z}),{33:103,75:[1,102],89:b,90:T},e(X,[2,65],{79:104,75:M,80:K,81:P,82:V,83:U,84:Y,85:B,86:Q,87:H,88:W}),e(J,[2,67]),e(J,[2,69]),e(J,[2,70]),e(J,[2,71]),e(J,[2,72]),e(J,[2,73]),e(J,[2,74]),e(J,[2,75]),e(J,[2,76]),e(J,[2,77]),e(J,[2,78]),e(k,[2,57],{76:z}),e(k,[2,58],{76:F}),{5:Z,28:105,31:ee,34:te,36:se,38:ie,40:ne},{27:[1,112],76:F},{5:re,40:ae,56:113,57:le,59:ce},{27:[1,118],76:F},{33:119,89:b,90:T},{33:120,89:b,90:T},{75:M,78:121,79:82,80:K,81:P,82:V,83:U,84:Y,85:B,86:Q,87:H,88:W},e(j,[2,61]),e(j,[2,63]),e(J,[2,68]),e(k,[2,21]),{32:[1,122]},{32:[1,123]},{32:[1,124]},{32:[1,125]},{5:Z,28:126,31:ee,34:te,36:se,38:ie,40:ne},e(k,[2,28]),{5:[1,127]},e(k,[2,42]),{32:[1,128]},{32:[1,129]},{5:re,40:ae,56:130,57:le,59:ce},e(k,[2,47]),{5:[1,131]},e(k,[2,48]),e(k,[2,49]),e(X,[2,66],{79:104,75:M,80:K,81:P,82:V,83:U,84:Y,85:B,86:Q,87:H,88:W}),{33:132,89:b,90:T},{35:133,89:[1,134],90:[1,135]},{37:136,47:[1,137],48:[1,138],49:[1,139]},{39:140,50:[1,141],51:[1,142],52:[1,143],53:[1,144]},e(k,[2,27]),{5:Z,28:145,31:ee,34:te,36:se,38:ie,40:ne},{58:146,89:[1,147],90:[1,148]},{60:149,89:[1,150],90:[1,151]},e(k,[2,46]),{5:re,40:ae,56:152,57:le,59:ce},{5:[1,153]},{5:[1,154]},{5:[2,83]},{5:[2,84]},{5:[1,155]},{5:[2,35]},{5:[2,36]},{5:[2,37]},{5:[1,156]},{5:[2,38]},{5:[2,39]},{5:[2,40]},{5:[2,41]},e(k,[2,22]),{5:[1,157]},{5:[2,87]},{5:[2,88]},{5:[1,158]},{5:[2,89]},{5:[2,90]},e(k,[2,43]),{5:Z,28:159,31:ee,34:te,36:se,38:ie,40:ne},{5:Z,28:160,31:ee,34:te,36:se,38:ie,40:ne},{5:Z,28:161,31:ee,34:te,36:se,38:ie,40:ne},{5:Z,28:162,31:ee,34:te,36:se,38:ie,40:ne},{5:re,40:ae,56:163,57:le,59:ce},{5:re,40:ae,56:164,57:le,59:ce},e(k,[2,23]),e(k,[2,24]),e(k,[2,25]),e(k,[2,26]),e(k,[2,44]),e(k,[2,45])],defaultActions:{8:[2,2],12:[2,1],41:[2,3],42:[2,8],43:[2,9],44:[2,10],45:[2,11],46:[2,12],47:[2,13],48:[2,14],49:[2,15],50:[2,16],134:[2,83],135:[2,84],137:[2,35],138:[2,36],139:[2,37],141:[2,38],142:[2,39],143:[2,40],144:[2,41],147:[2,87],148:[2,88],150:[2,89],151:[2,90]},parseError:(0,a.K2)((function(e,t){if(!t.recoverable){var s=new Error(e);throw s.hash=t,s}this.trace(e)}),"parseError"),parse:(0,a.K2)((function(e){var t=this,s=[0],i=[],n=[null],r=[],l=this.table,c="",o=0,h=0,u=0,y=r.slice.call(arguments,1),m=Object.create(this.lexer),d={yy:{}};for(var E in this.yy)Object.prototype.hasOwnProperty.call(this.yy,E)&&(d.yy[E]=this.yy[E]);m.setInput(e,d.yy),d.yy.lexer=m,d.yy.parser=this,void 0===m.yylloc&&(m.yylloc={});var p=m.yylloc;r.push(p);var R=m.options&&m.options.ranges;function f(){var e;return"number"!=typeof(e=i.pop()||m.lex()||1)&&(e instanceof Array&&(e=(i=e).pop()),e=t.symbols_[e]||e),e}"function"==typeof d.yy.parseError?this.parseError=d.yy.parseError:this.parseError=Object.getPrototypeOf(this).parseError,(0,a.K2)((function(e){s.length=s.length-2*e,n.length=n.length-e,r.length=r.length-e}),"popStack"),(0,a.K2)(f,"lex");for(var _,g,S,I,b,T,k,N,q,A={};;){if(S=s[s.length-1],this.defaultActions[S]?I=this.defaultActions[S]:(null==_&&(_=f()),I=l[S]&&l[S][_]),void 0===I||!I.length||!I[0]){var C="";for(T in q=[],l[S])this.terminals_[T]&&T>2&&q.push("'"+this.terminals_[T]+"'");C=m.showPosition?"Parse error on line "+(o+1)+":\n"+m.showPosition()+"\nExpecting "+q.join(", ")+", got '"+(this.terminals_[_]||_)+"'":"Parse error on line "+(o+1)+": Unexpected "+(1==_?"end of input":"'"+(this.terminals_[_]||_)+"'"),this.parseError(C,{text:m.match,token:this.terminals_[_]||_,line:m.yylineno,loc:p,expected:q})}if(I[0]instanceof Array&&I.length>1)throw new Error("Parse Error: multiple actions possible at state: "+S+", token: "+_);switch(I[0]){case 1:s.push(_),n.push(m.yytext),r.push(m.yylloc),s.push(I[1]),_=null,g?(_=g,g=null):(h=m.yyleng,c=m.yytext,o=m.yylineno,p=m.yylloc,u>0&&u--);break;case 2:if(k=this.productions_[I[1]][1],A.$=n[n.length-k],A._$={first_line:r[r.length-(k||1)].first_line,last_line:r[r.length-1].last_line,first_column:r[r.length-(k||1)].first_column,last_column:r[r.length-1].last_column},R&&(A._$.range=[r[r.length-(k||1)].range[0],r[r.length-1].range[1]]),void 0!==(b=this.performAction.apply(A,[c,h,o,d.yy,I[1],n,r].concat(y))))return b;k&&(s=s.slice(0,-1*k*2),n=n.slice(0,-1*k),r=r.slice(0,-1*k)),s.push(this.productions_[I[1]][0]),n.push(A.$),r.push(A._$),N=l[s[s.length-2]][s[s.length-1]],s.push(N);break;case 3:return!0}}return!0}),"parse")},he=function(){return{EOF:1,parseError:(0,a.K2)((function(e,t){if(!this.yy.parser)throw new Error(e);this.yy.parser.parseError(e,t)}),"parseError"),setInput:(0,a.K2)((function(e,t){return this.yy=t||this.yy||{},this._input=e,this._more=this._backtrack=this.done=!1,this.yylineno=this.yyleng=0,this.yytext=this.matched=this.match="",this.conditionStack=["INITIAL"],this.yylloc={first_line:1,first_column:0,last_line:1,last_column:0},this.options.ranges&&(this.yylloc.range=[0,0]),this.offset=0,this}),"setInput"),input:(0,a.K2)((function(){var e=this._input[0];return this.yytext+=e,this.yyleng++,this.offset++,this.match+=e,this.matched+=e,e.match(/(?:\r\n?|\n).*/g)?(this.yylineno++,this.yylloc.last_line++):this.yylloc.last_column++,this.options.ranges&&this.yylloc.range[1]++,this._input=this._input.slice(1),e}),"input"),unput:(0,a.K2)((function(e){var t=e.length,s=e.split(/(?:\r\n?|\n)/g);this._input=e+this._input,this.yytext=this.yytext.substr(0,this.yytext.length-t),this.offset-=t;var i=this.match.split(/(?:\r\n?|\n)/g);this.match=this.match.substr(0,this.match.length-1),this.matched=this.matched.substr(0,this.matched.length-1),s.length-1&&(this.yylineno-=s.length-1);var n=this.yylloc.range;return this.yylloc={first_line:this.yylloc.first_line,last_line:this.yylineno+1,first_column:this.yylloc.first_column,last_column:s?(s.length===i.length?this.yylloc.first_column:0)+i[i.length-s.length].length-s[0].length:this.yylloc.first_column-t},this.options.ranges&&(this.yylloc.range=[n[0],n[0]+this.yyleng-t]),this.yyleng=this.yytext.length,this}),"unput"),more:(0,a.K2)((function(){return this._more=!0,this}),"more"),reject:(0,a.K2)((function(){return this.options.backtrack_lexer?(this._backtrack=!0,this):this.parseError("Lexical error on line "+(this.yylineno+1)+". You can only invoke reject() in the lexer when the lexer is of the backtracking persuasion (options.backtrack_lexer = true).\n"+this.showPosition(),{text:"",token:null,line:this.yylineno})}),"reject"),less:(0,a.K2)((function(e){this.unput(this.match.slice(e))}),"less"),pastInput:(0,a.K2)((function(){var e=this.matched.substr(0,this.matched.length-this.match.length);return(e.length>20?"...":"")+e.substr(-20).replace(/\n/g,"")}),"pastInput"),upcomingInput:(0,a.K2)((function(){var e=this.match;return e.length<20&&(e+=this._input.substr(0,20-e.length)),(e.substr(0,20)+(e.length>20?"...":"")).replace(/\n/g,"")}),"upcomingInput"),showPosition:(0,a.K2)((function(){var e=this.pastInput(),t=new Array(e.length+1).join("-");return e+this.upcomingInput()+"\n"+t+"^"}),"showPosition"),test_match:(0,a.K2)((function(e,t){var s,i,n;if(this.options.backtrack_lexer&&(n={yylineno:this.yylineno,yylloc:{first_line:this.yylloc.first_line,last_line:this.last_line,first_column:this.yylloc.first_column,last_column:this.yylloc.last_column},yytext:this.yytext,match:this.match,matches:this.matches,matched:this.matched,yyleng:this.yyleng,offset:this.offset,_more:this._more,_input:this._input,yy:this.yy,conditionStack:this.conditionStack.slice(0),done:this.done},this.options.ranges&&(n.yylloc.range=this.yylloc.range.slice(0))),(i=e[0].match(/(?:\r\n?|\n).*/g))&&(this.yylineno+=i.length),this.yylloc={first_line:this.yylloc.last_line,last_line:this.yylineno+1,first_column:this.yylloc.last_column,last_column:i?i[i.length-1].length-i[i.length-1].match(/\r?\n?/)[0].length:this.yylloc.last_column+e[0].length},this.yytext+=e[0],this.match+=e[0],this.matches=e,this.yyleng=this.yytext.length,this.options.ranges&&(this.yylloc.range=[this.offset,this.offset+=this.yyleng]),this._more=!1,this._backtrack=!1,this._input=this._input.slice(e[0].length),this.matched+=e[0],s=this.performAction.call(this,this.yy,this,t,this.conditionStack[this.conditionStack.length-1]),this.done&&this._input&&(this.done=!1),s)return s;if(this._backtrack){for(var r in n)this[r]=n[r];return!1}return!1}),"test_match"),next:(0,a.K2)((function(){if(this.done)return this.EOF;var e,t,s,i;this._input||(this.done=!0),this._more||(this.yytext="",this.match="");for(var n=this._currentRules(),r=0;rt[0].length)){if(t=s,i=r,this.options.backtrack_lexer){if(!1!==(e=this.test_match(s,n[r])))return e;if(this._backtrack){t=!1;continue}return!1}if(!this.options.flex)break}return t?!1!==(e=this.test_match(t,n[i]))&&e:""===this._input?this.EOF:this.parseError("Lexical error on line "+(this.yylineno+1)+". Unrecognized text.\n"+this.showPosition(),{text:"",token:null,line:this.yylineno})}),"next"),lex:(0,a.K2)((function(){var e=this.next();return e||this.lex()}),"lex"),begin:(0,a.K2)((function(e){this.conditionStack.push(e)}),"begin"),popState:(0,a.K2)((function(){return this.conditionStack.length-1>0?this.conditionStack.pop():this.conditionStack[0]}),"popState"),_currentRules:(0,a.K2)((function(){return this.conditionStack.length&&this.conditionStack[this.conditionStack.length-1]?this.conditions[this.conditionStack[this.conditionStack.length-1]].rules:this.conditions.INITIAL.rules}),"_currentRules"),topState:(0,a.K2)((function(e){return(e=this.conditionStack.length-1-Math.abs(e||0))>=0?this.conditionStack[e]:"INITIAL"}),"topState"),pushState:(0,a.K2)((function(e){this.begin(e)}),"pushState"),stateStackSize:(0,a.K2)((function(){return this.conditionStack.length}),"stateStackSize"),options:{"case-insensitive":!0},performAction:(0,a.K2)((function(e,t,s,i){switch(s){case 0:return"title";case 1:return this.begin("acc_title"),9;case 2:return this.popState(),"acc_title_value";case 3:return this.begin("acc_descr"),11;case 4:return this.popState(),"acc_descr_value";case 5:this.begin("acc_descr_multiline");break;case 6:case 58:case 65:this.popState();break;case 7:return"acc_descr_multiline_value";case 8:return 21;case 9:return 22;case 10:return 23;case 11:return 24;case 12:return 5;case 13:case 14:case 15:case 56:break;case 16:return 8;case 17:return 6;case 18:return 27;case 19:return 40;case 20:return 29;case 21:return 32;case 22:return 31;case 23:return 34;case 24:return 36;case 25:return 38;case 26:return 41;case 27:return 42;case 28:return 43;case 29:return 44;case 30:return 45;case 31:return 46;case 32:return 47;case 33:return 48;case 34:return 49;case 35:return 50;case 36:return 51;case 37:return 52;case 38:return 53;case 39:return 54;case 40:return 65;case 41:return 66;case 42:return 67;case 43:return 68;case 44:return 69;case 45:return 70;case 46:return 71;case 47:return 57;case 48:return 59;case 49:return this.begin("style"),77;case 50:case 68:return 75;case 51:return 81;case 52:return 88;case 53:return"PERCENT";case 54:return 86;case 55:return 84;case 57:case 64:this.begin("string");break;case 59:return this.begin("style"),72;case 60:return this.begin("style"),74;case 61:return 61;case 62:return 64;case 63:return 63;case 66:return"qString";case 67:return t.yytext=t.yytext.trim(),89;case 69:return 80;case 70:return 76}}),"anonymous"),rules:[/^(?:title\s[^#\n;]+)/i,/^(?:accTitle\s*:\s*)/i,/^(?:(?!\n||)*[^\n]*)/i,/^(?:accDescr\s*:\s*)/i,/^(?:(?!\n||)*[^\n]*)/i,/^(?:accDescr\s*\{\s*)/i,/^(?:[\}])/i,/^(?:[^\}]*)/i,/^(?:.*direction\s+TB[^\n]*)/i,/^(?:.*direction\s+BT[^\n]*)/i,/^(?:.*direction\s+RL[^\n]*)/i,/^(?:.*direction\s+LR[^\n]*)/i,/^(?:(\r?\n)+)/i,/^(?:\s+)/i,/^(?:#[^\n]*)/i,/^(?:%[^\n]*)/i,/^(?:$)/i,/^(?:requirementDiagram\b)/i,/^(?:\{)/i,/^(?:\})/i,/^(?::{3})/i,/^(?::)/i,/^(?:id\b)/i,/^(?:text\b)/i,/^(?:risk\b)/i,/^(?:verifyMethod\b)/i,/^(?:requirement\b)/i,/^(?:functionalRequirement\b)/i,/^(?:interfaceRequirement\b)/i,/^(?:performanceRequirement\b)/i,/^(?:physicalRequirement\b)/i,/^(?:designConstraint\b)/i,/^(?:low\b)/i,/^(?:medium\b)/i,/^(?:high\b)/i,/^(?:analysis\b)/i,/^(?:demonstration\b)/i,/^(?:inspection\b)/i,/^(?:test\b)/i,/^(?:element\b)/i,/^(?:contains\b)/i,/^(?:copies\b)/i,/^(?:derives\b)/i,/^(?:satisfies\b)/i,/^(?:verifies\b)/i,/^(?:refines\b)/i,/^(?:traces\b)/i,/^(?:type\b)/i,/^(?:docref\b)/i,/^(?:style\b)/i,/^(?:\w+)/i,/^(?::)/i,/^(?:;)/i,/^(?:%)/i,/^(?:-)/i,/^(?:#)/i,/^(?: )/i,/^(?:["])/i,/^(?:\n)/i,/^(?:classDef\b)/i,/^(?:class\b)/i,/^(?:<-)/i,/^(?:->)/i,/^(?:-)/i,/^(?:["])/i,/^(?:["])/i,/^(?:[^"]*)/i,/^(?:[\w][^:,\r\n\{\<\>\-\=]*)/i,/^(?:\w+)/i,/^(?:[0-9]+)/i,/^(?:,)/i],conditions:{acc_descr_multiline:{rules:[6,7,68,69,70],inclusive:!1},acc_descr:{rules:[4,68,69,70],inclusive:!1},acc_title:{rules:[2,68,69,70],inclusive:!1},style:{rules:[50,51,52,53,54,55,56,57,58,68,69,70],inclusive:!1},unqString:{rules:[68,69,70],inclusive:!1},token:{rules:[68,69,70],inclusive:!1},string:{rules:[65,66,68,69,70],inclusive:!1},INITIAL:{rules:[0,1,3,5,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,59,60,61,62,63,64,67,68,69,70],inclusive:!0}}}}();function ue(){this.yy={}}return oe.lexer=he,(0,a.K2)(ue,"Parser"),ue.prototype=oe,oe.Parser=ue,new ue}();l.parser=l;var c=l,o=class{constructor(){this.relations=[],this.latestRequirement=this.getInitialRequirement(),this.requirements=new Map,this.latestElement=this.getInitialElement(),this.elements=new Map,this.classes=new Map,this.direction="TB",this.RequirementType={REQUIREMENT:"Requirement",FUNCTIONAL_REQUIREMENT:"Functional Requirement",INTERFACE_REQUIREMENT:"Interface Requirement",PERFORMANCE_REQUIREMENT:"Performance Requirement",PHYSICAL_REQUIREMENT:"Physical Requirement",DESIGN_CONSTRAINT:"Design Constraint"},this.RiskLevel={LOW_RISK:"Low",MED_RISK:"Medium",HIGH_RISK:"High"},this.VerifyType={VERIFY_ANALYSIS:"Analysis",VERIFY_DEMONSTRATION:"Demonstration",VERIFY_INSPECTION:"Inspection",VERIFY_TEST:"Test"},this.Relationships={CONTAINS:"contains",COPIES:"copies",DERIVES:"derives",SATISFIES:"satisfies",VERIFIES:"verifies",REFINES:"refines",TRACES:"traces"},this.setAccTitle=a.SV,this.getAccTitle=a.iN,this.setAccDescription=a.EI,this.getAccDescription=a.m7,this.setDiagramTitle=a.ke,this.getDiagramTitle=a.ab,this.getConfig=(0,a.K2)((()=>(0,a.D7)().requirement),"getConfig"),this.clear(),this.setDirection=this.setDirection.bind(this),this.addRequirement=this.addRequirement.bind(this),this.setNewReqId=this.setNewReqId.bind(this),this.setNewReqRisk=this.setNewReqRisk.bind(this),this.setNewReqText=this.setNewReqText.bind(this),this.setNewReqVerifyMethod=this.setNewReqVerifyMethod.bind(this),this.addElement=this.addElement.bind(this),this.setNewElementType=this.setNewElementType.bind(this),this.setNewElementDocRef=this.setNewElementDocRef.bind(this),this.addRelationship=this.addRelationship.bind(this),this.setCssStyle=this.setCssStyle.bind(this),this.setClass=this.setClass.bind(this),this.defineClass=this.defineClass.bind(this),this.setAccTitle=this.setAccTitle.bind(this),this.setAccDescription=this.setAccDescription.bind(this)}static{(0,a.K2)(this,"RequirementDB")}getDirection(){return this.direction}setDirection(e){this.direction=e}resetLatestRequirement(){this.latestRequirement=this.getInitialRequirement()}resetLatestElement(){this.latestElement=this.getInitialElement()}getInitialRequirement(){return{requirementId:"",text:"",risk:"",verifyMethod:"",name:"",type:"",cssStyles:[],classes:["default"]}}getInitialElement(){return{name:"",type:"",docRef:"",cssStyles:[],classes:["default"]}}addRequirement(e,t){return this.requirements.has(e)||this.requirements.set(e,{name:e,type:t,requirementId:this.latestRequirement.requirementId,text:this.latestRequirement.text,risk:this.latestRequirement.risk,verifyMethod:this.latestRequirement.verifyMethod,cssStyles:[],classes:["default"]}),this.resetLatestRequirement(),this.requirements.get(e)}getRequirements(){return this.requirements}setNewReqId(e){void 0!==this.latestRequirement&&(this.latestRequirement.requirementId=e)}setNewReqText(e){void 0!==this.latestRequirement&&(this.latestRequirement.text=e)}setNewReqRisk(e){void 0!==this.latestRequirement&&(this.latestRequirement.risk=e)}setNewReqVerifyMethod(e){void 0!==this.latestRequirement&&(this.latestRequirement.verifyMethod=e)}addElement(e){return this.elements.has(e)||(this.elements.set(e,{name:e,type:this.latestElement.type,docRef:this.latestElement.docRef,cssStyles:[],classes:["default"]}),a.Rm.info("Added new element: ",e)),this.resetLatestElement(),this.elements.get(e)}getElements(){return this.elements}setNewElementType(e){void 0!==this.latestElement&&(this.latestElement.type=e)}setNewElementDocRef(e){void 0!==this.latestElement&&(this.latestElement.docRef=e)}addRelationship(e,t,s){this.relations.push({type:e,src:t,dst:s})}getRelationships(){return this.relations}clear(){this.relations=[],this.resetLatestRequirement(),this.requirements=new Map,this.resetLatestElement(),this.elements=new Map,this.classes=new Map,(0,a.IU)()}setCssStyle(e,t){for(const s of e){const e=this.requirements.get(s)??this.elements.get(s);if(!t||!e)return;for(const s of t)s.includes(",")?e.cssStyles.push(...s.split(",")):e.cssStyles.push(s)}}setClass(e,t){for(const s of e){const e=this.requirements.get(s)??this.elements.get(s);if(e)for(const s of t){e.classes.push(s);const t=this.classes.get(s)?.styles;t&&e.cssStyles.push(...t)}}}defineClass(e,t){for(const s of e){let e=this.classes.get(s);void 0===e&&(e={id:s,styles:[],textStyles:[]},this.classes.set(s,e)),t&&t.forEach((function(t){if(/color/.exec(t)){const s=t.replace("fill","bgFill");e.textStyles.push(s)}e.styles.push(t)})),this.requirements.forEach((e=>{e.classes.includes(s)&&e.cssStyles.push(...t.flatMap((e=>e.split(","))))})),this.elements.forEach((e=>{e.classes.includes(s)&&e.cssStyles.push(...t.flatMap((e=>e.split(","))))}))}}getClasses(){return this.classes}getData(){const e=(0,a.D7)(),t=[],s=[];for(const i of this.requirements.values()){const s=i;s.id=i.name,s.cssStyles=i.cssStyles,s.cssClasses=i.classes.join(" "),s.shape="requirementBox",s.look=e.look,t.push(s)}for(const i of this.elements.values()){const s=i;s.shape="requirementBox",s.look=e.look,s.id=i.name,s.cssStyles=i.cssStyles,s.cssClasses=i.classes.join(" "),t.push(s)}for(const i of this.relations){let t=0;const n=i.type===this.Relationships.CONTAINS,r={id:`${i.src}-${i.dst}-${t}`,start:this.requirements.get(i.src)?.name??this.elements.get(i.src)?.name,end:this.requirements.get(i.dst)?.name??this.elements.get(i.dst)?.name,label:`<<${i.type}>>`,classes:"relationshipLine",style:["fill:none",n?"":"stroke-dasharray: 10,7"],labelpos:"c",thickness:"normal",type:"normal",pattern:n?"normal":"dashed",arrowTypeStart:n?"requirement_contains":"",arrowTypeEnd:n?"":"requirement_arrow",look:e.look};s.push(r),t++}return{nodes:t,edges:s,other:{},config:e,direction:this.getDirection()}}},h=(0,a.K2)((e=>`\n\n marker {\n fill: ${e.relationColor};\n stroke: ${e.relationColor};\n }\n\n marker.cross {\n stroke: ${e.lineColor};\n }\n\n svg {\n font-family: ${e.fontFamily};\n font-size: ${e.fontSize};\n }\n\n .reqBox {\n fill: ${e.requirementBackground};\n fill-opacity: 1.0;\n stroke: ${e.requirementBorderColor};\n stroke-width: ${e.requirementBorderSize};\n }\n \n .reqTitle, .reqLabel{\n fill: ${e.requirementTextColor};\n }\n .reqLabelBox {\n fill: ${e.relationLabelBackground};\n fill-opacity: 1.0;\n }\n\n .req-title-line {\n stroke: ${e.requirementBorderColor};\n stroke-width: ${e.requirementBorderSize};\n }\n .relationshipLine {\n stroke: ${e.relationColor};\n stroke-width: 1;\n }\n .relationshipLabel {\n fill: ${e.relationLabelColor};\n }\n .divider {\n stroke: ${e.nodeBorder};\n stroke-width: 1;\n }\n .label {\n font-family: ${e.fontFamily};\n color: ${e.nodeTextColor||e.textColor};\n }\n .label text,span {\n fill: ${e.nodeTextColor||e.textColor};\n color: ${e.nodeTextColor||e.textColor};\n }\n .labelBkg {\n background-color: ${e.edgeLabelBackground};\n }\n\n`),"getStyles"),u={};(0,a.VA)(u,{draw:()=>y});var y=(0,a.K2)((async function(e,t,s,l){a.Rm.info("REF0:"),a.Rm.info("Drawing requirement diagram (unified)",t);const{securityLevel:c,state:o,layout:h}=(0,a.D7)(),u=l.db.getData(),y=(0,i.A)(t,c);u.type=l.type,u.layoutAlgorithm=(0,n.q7)(h),u.nodeSpacing=o?.nodeSpacing??50,u.rankSpacing=o?.rankSpacing??50,u.markers=["requirement_contains","requirement_arrow"],u.diagramId=t,await(0,n.XX)(u,y);r._K.insertTitle(y,"requirementDiagramTitleText",o?.titleTopMargin??25,l.db.getDiagramTitle()),(0,i.P)(y,8,"requirementDiagram",o?.useMaxWidth??!0)}),"draw"),m={parser:c,get db(){return new o},renderer:u,styles:h}}}]); \ No newline at end of file diff --git a/assets/js/12da0170.69b2b946.js b/assets/js/12da0170.69b2b946.js deleted file mode 100644 index 01a21424..00000000 --- a/assets/js/12da0170.69b2b946.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[8056],{16486:(e,t,a)=>{a.r(t),a.d(t,{assets:()=>p,contentTitle:()=>s,default:()=>c,frontMatter:()=>o,metadata:()=>i,toc:()=>u});var n=a(87462),r=(a(67294),a(3905));const o={layout:"post",name:"Presenting JS-Waku: Waku v2 in the Browser",title:"Presenting JS-Waku: Waku v2 in the Browser",date:new Date("2021-06-04T12:00:00.000Z"),authors:"franck",published:!0,slug:"presenting-js-waku",categories:"platform",image:"/img/js-waku-gist.png",discuss:"https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81"},s=void 0,i={permalink:"/rlog/presenting-js-waku",source:"@site/rlog/2021-06-04-presenting-js-waku.mdx",title:"Presenting JS-Waku: Waku v2 in the Browser",description:"JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!",date:"2021-06-04T12:00:00.000Z",formattedDate:"June 4, 2021",tags:[],readingTime:6.84,hasTruncateMarker:!0,authors:[{name:"Franck",twitter:"fryorcraken",github:"fryorcraken",key:"franck"}],frontMatter:{layout:"post",name:"Presenting JS-Waku: Waku v2 in the Browser",title:"Presenting JS-Waku: Waku v2 in the Browser",date:"2021-06-04T12:00:00.000Z",authors:"franck",published:!0,slug:"presenting-js-waku",categories:"platform",image:"/img/js-waku-gist.png",discuss:"https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81"},prevItem:{title:"[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging",permalink:"/rlog/waku-v2-ethereum-coscup"},nextItem:{title:"Privacy-preserving p2p economic spam protection in Waku v2",permalink:"/rlog/rln-relay"}},p={authorsImageUrls:[void 0]},u=[{value:"Waku v2",id:"waku-v2",level:2},{value:"Waku v2 in the browser",id:"waku-v2-in-the-browser",level:2},{value:"Achievements so far",id:"achievements-so-far",level:2},{value:"What's next?",id:"whats-next",level:2}],l={toc:u};function c(e){let{components:t,...a}=e;return(0,r.kt)("wrapper",(0,n.Z)({},l,a,{components:t,mdxType:"MDXLayout"}),(0,r.kt)("p",null,"JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!"),(0,r.kt)("p",null,"For the past 3 months, we have been working on bringing Waku v2 to the browser.\nOur aim is to empower dApps with Waku v2, and it led to the creation of a new library.\nWe believe now is good time to introduce it!"),(0,r.kt)("h2",{id:"waku-v2"},"Waku v2"),(0,r.kt)("p",null,"First, let's review what Waku v2 is and what problem it is trying to solve."),(0,r.kt)("p",null,"Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised\ncommunications on resource restricted devices (i.e., mobile phones)."),(0,r.kt)("p",null,"The Status chat feature was initially built over Whisper.\nHowever, Whisper has a number of caveats which makes it inefficient for mobile phones.\nFor example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans."),(0,r.kt)("p",null,"To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced.\nTo further enable web and restricted resource environments, Waku v2 was created based on libp2p.\nThe migration of the Status chat feature to Waku v2 is currently in progress."),(0,r.kt)("p",null,"We see the need of such solution in the broader Ethereum ecosystem, beyond Status.\nThis is why we are building Waku v2 as a decentralised communication platform for all to use and build on.\nIf you want to read more about Waku v2 and what it aims to achieve,\ncheckout ",(0,r.kt)("a",{parentName:"p",href:"/waku-v2-plan"},"What's the Plan for Waku v2?"),"."),(0,r.kt)("p",null,"Since last year, we have been busy defining and implementing Waku v2 protocols in ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku"},"nim-waku"),",\nfrom which you can build ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku#wakunode"},"wakunode2"),".\nWakunode2 is an adaptive and modular Waku v2 node,\nit allows users to run their own node and use the Waku v2 protocols they need.\nThe nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications."),(0,r.kt)("h2",{id:"waku-v2-in-the-browser"},"Waku v2 in the browser"),(0,r.kt)("p",null,"We believe that dApps and wallets can benefit from the Waku network in several ways.\nFor some dApps, it makes sense to enable peer-to-peer communications.\nFor others, machine-to-machine communications would be a great asset.\nFor example, in the case of a DAO,\nWaku could be used for gas-less voting.\nEnabling the DAO to notify their users of a new vote,\nand users to vote without interacting with the blockchain and spending gas."),(0,r.kt)("p",null,(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/murmur"},"Murmur")," was the first attempt to bring Whisper to the browser,\nacting as a bridge between devp2p and libp2p.\nOnce Waku v2 was started and there was a native implementation on top of libp2p,\na ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/vacp2p/waku-web-chat"},"chat POC")," was created to demonstrate the potential of Waku v2\nin web environment.\nIt showed how using js-libp2p with few modifications enabled access to the Waku v2 network.\nThere was still some unresolved challenges.\nFor example, nim-waku only support TCP connections which are not supported by browser applications.\nHence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,\nwhich in turn could connect to wakunode2 via TCP."),(0,r.kt)("p",null,"However, to enable dApp and Wallet developers to easily integrate Waku in their product,\nwe need to give them a library that is easy to use and works out of the box:\nintroducing ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/js-waku"},"JS-Waku"),"."),(0,r.kt)("p",null,"JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network.\nIt is available right now on ",(0,r.kt)("a",{parentName:"p",href:"https://www.npmjs.com/package/js-waku"},"npm"),":"),(0,r.kt)("p",null,(0,r.kt)("inlineCode",{parentName:"p"},"npm install js-waku"),"."),(0,r.kt)("p",null,"As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript."),(0,r.kt)("p",null,"Key Waku v2 protocols are already available:\n",(0,r.kt)("a",{parentName:"p",href:"https://rfc.vac.dev/waku/standards/core/14/message"},"message"),", ",(0,r.kt)("a",{parentName:"p",href:"https://rfc.vac.dev/waku/standards/core/13/store"},"store"),", ",(0,r.kt)("a",{parentName:"p",href:"https://rfc.vac.dev/waku/standards/core/11/relay"},"relay")," and ",(0,r.kt)("a",{parentName:"p",href:"https://rfc.vac.dev/waku/standards/core/19/lightpush"},"light push"),",\nenabling your dApp to:"),(0,r.kt)("ul",null,(0,r.kt)("li",{parentName:"ul"},"Send and receive near-instant messages on the Waku network (relay),"),(0,r.kt)("li",{parentName:"ul"},"Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),"),(0,r.kt)("li",{parentName:"ul"},"Send messages with confirmations (light push).")),(0,r.kt)("p",null,"JS-Waku needs to operate in the same context from which Waku v2 was born:\na restricted environment were connectivity or uptime are not guaranteed;\nJS-Waku brings Waku v2 to the browser."),(0,r.kt)("h2",{id:"achievements-so-far"},"Achievements so far"),(0,r.kt)("p",null,"We focused the past month on developing a ",(0,r.kt)("a",{parentName:"p",href:"https://status-im.github.io/js-waku/"},"ReactJS Chat App"),".\nThe aim was to create enough building blocks in JS-Waku to enable this showcase web app that\nwe now ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku/issues/399"},"use for dogfooding")," purposes."),(0,r.kt)("p",null,"Most of the effort was on getting familiar with the ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/libp2p/js-libp2p"},"js-libp2p")," library\nthat we heavily rely on.\nJS-Waku is the second implementation of Waku v2 protocol,\nso a lot of effort on interoperability was needed.\nFor example, to ensure compatibility with the nim-waku reference implementation,\nwe run our ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/js-waku/blob/90c90dea11dfd1277f530cf5d683fb92992fe141/src/lib/waku_relay/index.spec.ts#L137"},"tests against wakunode2")," as part of the CI."),(0,r.kt)("p",null,"This interoperability effort helped solidify the current Waku v2 specifications:\nBy clarifying the usage of topics\n(",(0,r.kt)("a",{parentName:"p",href:"https://github.com/vacp2p/rfc/issues/327"},"#327"),", ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/vacp2p/rfc/pull/383"},"#383"),"),\nfix discrepancies between specs and nim-waku\n(",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku/issues/418"},"#418"),", ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku/issues/419"},"#419"),")\nand fix small nim-waku & nim-libp2p bugs\n(",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku/issues/411"},"#411"),", ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku/issues/439"},"#439"),")."),(0,r.kt)("p",null,"To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.\nA standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.\nUnfortunately websocket support is not yet available in ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-libp2p/issues/407"},"nim-libp2p")," so\nwe ended up deploying ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/novnc/websockify"},"websockify")," alongside wakunode2 instances."),(0,r.kt)("p",null,"As we built the ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/js-waku/tree/main/examples/web-chat"},"web chat app"),",\nwe were able to fine tune the API to provide a simple and succinct interface.\nYou can start a node, connect to other nodes and send a message in less than ten lines of code:"),(0,r.kt)("pre",null,(0,r.kt)("code",{parentName:"pre",className:"language-javascript"},"import { Waku } from 'js-waku'\n\nconst waku = await Waku.create({})\n\nconst nodes = await getStatusFleetNodes()\nawait Promise.all(nodes.map((addr) => waku.dial(addr)))\n\nconst msg = WakuMessage.fromUtf8String(\n 'Here is a message!',\n '/my-cool-app/1/my-use-case/proto',\n)\nawait waku.relay.send(msg)\n")),(0,r.kt)("p",null,"We have also put a bounty at ",(0,r.kt)("a",{parentName:"p",href:"https://0xhack.dev/"},"0xHack")," for using JS-Waku\nand running a ",(0,r.kt)("a",{parentName:"p",href:"https://www.youtube.com/watch?v=l77j0VX75QE"},"workshop"),".\nWe were thrilled to have a couple of hackers create new software using our libraries.\nOne of the projects aimed to create a decentralised, end-to-end encrypted messenger app,\nsimilar to what the ",(0,r.kt)("a",{parentName:"p",href:"https://rfc.vac.dev/waku/standards/application/20/toy-eth-pm"},"ETH-DM")," protocol aims to achieve.\nAnother project was a decentralised Twitter platform.\nSuch projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved."),(0,r.kt)("p",null,"As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network\nwhile preserving this out of the box experience."),(0,r.kt)("h2",{id:"whats-next"},"What's next?"),(0,r.kt)("p",null,"Next, we are directing our attention towards ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/js-waku/issues/68"},"Developer Experience"),".\nWe already have ",(0,r.kt)("a",{parentName:"p",href:"https://www.npmjs.com/package/js-waku"},"documentation")," available but we want to provide more:\n",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/js-waku/issues/56"},"Tutorials"),", various examples\nand showing how ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/js-waku/issues/72"},"JS-Waku can be used with Web3"),"."),(0,r.kt)("p",null,"By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets.\nWe think JS-Waku builds a strong case for machine-to-machine (M2M) communications.\nThe first use cases we are looking into are dApp notifications:\nEnabling dApp to notify their user directly in their wallets!\nLeveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified\nof events such as DAO voting."),(0,r.kt)("p",null,"We already have some POC in the pipeline to enable voting and polling on the Waku network,\nallowing users to save gas by ",(0,r.kt)("strong",{parentName:"p"},"not")," broadcasting each individual vote on the blockchain."),(0,r.kt)("p",null,"To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples\nof signing, validating, encrypting and decrypting messages using Web3.\nWaku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity."),(0,r.kt)("p",null,"As you can read, we have grand plans for JS-Waku and Waku v2.\nThere is a lot to do, and we would love some help so feel free to\ncheck out the new role in our team:\n",(0,r.kt)("a",{parentName:"p",href:"https://status.im/our_team/jobs.html?gh_jid=3157894"},"js-waku: Wallet & Dapp Integration Developer"),".\nWe also have a number of ",(0,r.kt)("a",{parentName:"p",href:"https://status.im/our_team/jobs.html"},"positions")," open to work on Waku protocol and nim-waku."),(0,r.kt)("p",null,"If you are as excited as us by JS-Waku, why not build a dApp with it?\nYou can find documentation on the ",(0,r.kt)("a",{parentName:"p",href:"https://www.npmjs.com/package/js-waku"},"npmjs page"),"."),(0,r.kt)("p",null,"Whether you are a developer, you can come chat with us using ",(0,r.kt)("a",{parentName:"p",href:"https://status-im.github.io/js-waku/"},"WakuJS Web Chat"),"\nor ",(0,r.kt)("a",{parentName:"p",href:"https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md"},"chat2"),".\nYou can get support in #dappconnect-support on ",(0,r.kt)("a",{parentName:"p",href:"https://discord.gg/j5pGbn7MHZ"},"Vac Discord")," or ",(0,r.kt)("a",{parentName:"p",href:"https://t.me/dappconnectsupport"},"Telegram"),".\nIf you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it."))}c.isMDXComponent=!0},3905:(e,t,a)=>{a.d(t,{Zo:()=>l,kt:()=>m});var n=a(67294);function r(e,t,a){return t in e?Object.defineProperty(e,t,{value:a,enumerable:!0,configurable:!0,writable:!0}):e[t]=a,e}function o(e,t){var a=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),a.push.apply(a,n)}return a}function s(e){for(var t=1;t=0||(r[a]=e[a]);return r}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,a)&&(r[a]=e[a])}return r}var p=n.createContext({}),u=function(e){var t=n.useContext(p),a=t;return e&&(a="function"==typeof e?e(t):s(s({},t),e)),a},l=function(e){var t=u(e.components);return n.createElement(p.Provider,{value:t},e.children)},c={inlineCode:"code",wrapper:function(e){var t=e.children;return n.createElement(n.Fragment,{},t)}},h=n.forwardRef((function(e,t){var a=e.components,r=e.mdxType,o=e.originalType,p=e.parentName,l=i(e,["components","mdxType","originalType","parentName"]),h=u(a),m=r,d=h["".concat(p,".").concat(m)]||h[m]||c[m]||o;return a?n.createElement(d,s(s({ref:t},l),{},{components:a})):n.createElement(d,s({ref:t},l))}));function m(e,t){var a=arguments,r=t&&t.mdxType;if("string"==typeof e||r){var o=a.length,s=new Array(o);s[0]=h;var i={};for(var p in t)hasOwnProperty.call(t,p)&&(i[p]=t[p]);i.originalType=e,i.mdxType="string"==typeof e?e:r,s[1]=i;for(var u=2;u{n.d(t,{R:()=>o,x:()=>r});var a=n(96540);const s={},i=a.createContext(s);function o(e){const t=a.useContext(i);return a.useMemo((function(){return"function"==typeof e?e(t):{...t,...e}}),[t,e])}function r(e){let t;return t=e.disableParentContext?"function"==typeof e.components?e.components(s):e.components||s:o(e.components),a.createElement(i.Provider,{value:t},e.children)}},51521:e=>{e.exports=JSON.parse('{"permalink":"/rlog/presenting-js-waku","source":"@site/rlog/2021-06-04-presenting-js-waku.mdx","title":"Presenting JS-Waku: Waku v2 in the Browser","description":"JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!","date":"2021-06-04T12:00:00.000Z","tags":[],"readingTime":6.85,"hasTruncateMarker":true,"authors":[{"name":"Franck","twitter":"fryorcraken","github":"fryorcraken","key":"franck","page":null}],"frontMatter":{"layout":"post","name":"Presenting JS-Waku: Waku v2 in the Browser","title":"Presenting JS-Waku: Waku v2 in the Browser","date":"2021-06-04T12:00:00.000Z","authors":"franck","published":true,"slug":"presenting-js-waku","categories":"platform","image":"/img/js-waku-gist.png","discuss":"https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81"},"unlisted":false,"prevItem":{"title":"[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging","permalink":"/rlog/waku-v2-ethereum-coscup"},"nextItem":{"title":"Privacy-preserving p2p economic spam protection in Waku v2","permalink":"/rlog/rln-relay"}}')},63022:(e,t,n)=>{n.r(t),n.d(t,{assets:()=>h,contentTitle:()=>r,default:()=>l,frontMatter:()=>o,metadata:()=>a,toc:()=>c});var a=n(51521),s=n(74848),i=n(28453);const o={layout:"post",name:"Presenting JS-Waku: Waku v2 in the Browser",title:"Presenting JS-Waku: Waku v2 in the Browser",date:new Date("2021-06-04T12:00:00.000Z"),authors:"franck",published:!0,slug:"presenting-js-waku",categories:"platform",image:"/img/js-waku-gist.png",discuss:"https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81"},r=void 0,h={authorsImageUrls:[void 0]},c=[{value:"Waku v2",id:"waku-v2",level:2},{value:"Waku v2 in the browser",id:"waku-v2-in-the-browser",level:2},{value:"Achievements so far",id:"achievements-so-far",level:2},{value:"What's next?",id:"whats-next",level:2}];function u(e){const t={a:"a",code:"code",h2:"h2",li:"li",p:"p",pre:"pre",strong:"strong",ul:"ul",...(0,i.R)(),...e.components};return(0,s.jsxs)(s.Fragment,{children:[(0,s.jsx)(t.p,{children:"JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!"}),"\n","\n",(0,s.jsx)(t.p,{children:"For the past 3 months, we have been working on bringing Waku v2 to the browser.\nOur aim is to empower dApps with Waku v2, and it led to the creation of a new library.\nWe believe now is good time to introduce it!"}),"\n",(0,s.jsx)(t.h2,{id:"waku-v2",children:"Waku v2"}),"\n",(0,s.jsx)(t.p,{children:"First, let's review what Waku v2 is and what problem it is trying to solve."}),"\n",(0,s.jsx)(t.p,{children:"Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised\ncommunications on resource restricted devices (i.e., mobile phones)."}),"\n",(0,s.jsx)(t.p,{children:"The Status chat feature was initially built over Whisper.\nHowever, Whisper has a number of caveats which makes it inefficient for mobile phones.\nFor example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans."}),"\n",(0,s.jsx)(t.p,{children:"To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced.\nTo further enable web and restricted resource environments, Waku v2 was created based on libp2p.\nThe migration of the Status chat feature to Waku v2 is currently in progress."}),"\n",(0,s.jsxs)(t.p,{children:["We see the need of such solution in the broader Ethereum ecosystem, beyond Status.\nThis is why we are building Waku v2 as a decentralised communication platform for all to use and build on.\nIf you want to read more about Waku v2 and what it aims to achieve,\ncheckout ",(0,s.jsx)(t.a,{href:"/waku-v2-plan",children:"What's the Plan for Waku v2?"}),"."]}),"\n",(0,s.jsxs)(t.p,{children:["Since last year, we have been busy defining and implementing Waku v2 protocols in ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku",children:"nim-waku"}),",\nfrom which you can build ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku#wakunode",children:"wakunode2"}),".\nWakunode2 is an adaptive and modular Waku v2 node,\nit allows users to run their own node and use the Waku v2 protocols they need.\nThe nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications."]}),"\n",(0,s.jsx)(t.h2,{id:"waku-v2-in-the-browser",children:"Waku v2 in the browser"}),"\n",(0,s.jsx)(t.p,{children:"We believe that dApps and wallets can benefit from the Waku network in several ways.\nFor some dApps, it makes sense to enable peer-to-peer communications.\nFor others, machine-to-machine communications would be a great asset.\nFor example, in the case of a DAO,\nWaku could be used for gas-less voting.\nEnabling the DAO to notify their users of a new vote,\nand users to vote without interacting with the blockchain and spending gas."}),"\n",(0,s.jsxs)(t.p,{children:[(0,s.jsx)(t.a,{href:"https://github.com/status-im/murmur",children:"Murmur"})," was the first attempt to bring Whisper to the browser,\nacting as a bridge between devp2p and libp2p.\nOnce Waku v2 was started and there was a native implementation on top of libp2p,\na ",(0,s.jsx)(t.a,{href:"https://github.com/vacp2p/waku-web-chat",children:"chat POC"})," was created to demonstrate the potential of Waku v2\nin web environment.\nIt showed how using js-libp2p with few modifications enabled access to the Waku v2 network.\nThere was still some unresolved challenges.\nFor example, nim-waku only support TCP connections which are not supported by browser applications.\nHence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,\nwhich in turn could connect to wakunode2 via TCP."]}),"\n",(0,s.jsxs)(t.p,{children:["However, to enable dApp and Wallet developers to easily integrate Waku in their product,\nwe need to give them a library that is easy to use and works out of the box:\nintroducing ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/js-waku",children:"JS-Waku"}),"."]}),"\n",(0,s.jsxs)(t.p,{children:["JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network.\nIt is available right now on ",(0,s.jsx)(t.a,{href:"https://www.npmjs.com/package/js-waku",children:"npm"}),":"]}),"\n",(0,s.jsxs)(t.p,{children:[(0,s.jsx)(t.code,{children:"npm install js-waku"}),"."]}),"\n",(0,s.jsx)(t.p,{children:"As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript."}),"\n",(0,s.jsxs)(t.p,{children:["Key Waku v2 protocols are already available:\n",(0,s.jsx)(t.a,{href:"https://rfc.vac.dev/waku/standards/core/14/message",children:"message"}),", ",(0,s.jsx)(t.a,{href:"https://rfc.vac.dev/waku/standards/core/13/store",children:"store"}),", ",(0,s.jsx)(t.a,{href:"https://rfc.vac.dev/waku/standards/core/11/relay",children:"relay"})," and ",(0,s.jsx)(t.a,{href:"https://rfc.vac.dev/waku/standards/core/19/lightpush",children:"light push"}),",\nenabling your dApp to:"]}),"\n",(0,s.jsxs)(t.ul,{children:["\n",(0,s.jsx)(t.li,{children:"Send and receive near-instant messages on the Waku network (relay),"}),"\n",(0,s.jsx)(t.li,{children:"Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),"}),"\n",(0,s.jsx)(t.li,{children:"Send messages with confirmations (light push)."}),"\n"]}),"\n",(0,s.jsx)(t.p,{children:"JS-Waku needs to operate in the same context from which Waku v2 was born:\na restricted environment were connectivity or uptime are not guaranteed;\nJS-Waku brings Waku v2 to the browser."}),"\n",(0,s.jsx)(t.h2,{id:"achievements-so-far",children:"Achievements so far"}),"\n",(0,s.jsxs)(t.p,{children:["We focused the past month on developing a ",(0,s.jsx)(t.a,{href:"https://status-im.github.io/js-waku/",children:"ReactJS Chat App"}),".\nThe aim was to create enough building blocks in JS-Waku to enable this showcase web app that\nwe now ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku/issues/399",children:"use for dogfooding"})," purposes."]}),"\n",(0,s.jsxs)(t.p,{children:["Most of the effort was on getting familiar with the ",(0,s.jsx)(t.a,{href:"https://github.com/libp2p/js-libp2p",children:"js-libp2p"})," library\nthat we heavily rely on.\nJS-Waku is the second implementation of Waku v2 protocol,\nso a lot of effort on interoperability was needed.\nFor example, to ensure compatibility with the nim-waku reference implementation,\nwe run our ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/js-waku/blob/90c90dea11dfd1277f530cf5d683fb92992fe141/src/lib/waku_relay/index.spec.ts#L137",children:"tests against wakunode2"})," as part of the CI."]}),"\n",(0,s.jsxs)(t.p,{children:["This interoperability effort helped solidify the current Waku v2 specifications:\nBy clarifying the usage of topics\n(",(0,s.jsx)(t.a,{href:"https://github.com/vacp2p/rfc/issues/327",children:"#327"}),", ",(0,s.jsx)(t.a,{href:"https://github.com/vacp2p/rfc/pull/383",children:"#383"}),"),\nfix discrepancies between specs and nim-waku\n(",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku/issues/418",children:"#418"}),", ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku/issues/419",children:"#419"}),")\nand fix small nim-waku & nim-libp2p bugs\n(",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku/issues/411",children:"#411"}),", ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku/issues/439",children:"#439"}),")."]}),"\n",(0,s.jsxs)(t.p,{children:["To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.\nA standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.\nUnfortunately websocket support is not yet available in ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-libp2p/issues/407",children:"nim-libp2p"})," so\nwe ended up deploying ",(0,s.jsx)(t.a,{href:"https://github.com/novnc/websockify",children:"websockify"})," alongside wakunode2 instances."]}),"\n",(0,s.jsxs)(t.p,{children:["As we built the ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/js-waku/tree/main/examples/web-chat",children:"web chat app"}),",\nwe were able to fine tune the API to provide a simple and succinct interface.\nYou can start a node, connect to other nodes and send a message in less than ten lines of code:"]}),"\n",(0,s.jsx)(t.pre,{children:(0,s.jsx)(t.code,{className:"language-javascript",children:"import { Waku } from 'js-waku'\n\nconst waku = await Waku.create({})\n\nconst nodes = await getStatusFleetNodes()\nawait Promise.all(nodes.map((addr) => waku.dial(addr)))\n\nconst msg = WakuMessage.fromUtf8String(\n 'Here is a message!',\n '/my-cool-app/1/my-use-case/proto',\n)\nawait waku.relay.send(msg)\n"})}),"\n",(0,s.jsxs)(t.p,{children:["We have also put a bounty at ",(0,s.jsx)(t.a,{href:"https://0xhack.dev/",children:"0xHack"})," for using JS-Waku\nand running a ",(0,s.jsx)(t.a,{href:"https://www.youtube.com/watch?v=l77j0VX75QE",children:"workshop"}),".\nWe were thrilled to have a couple of hackers create new software using our libraries.\nOne of the projects aimed to create a decentralised, end-to-end encrypted messenger app,\nsimilar to what the ",(0,s.jsx)(t.a,{href:"https://rfc.vac.dev/waku/standards/application/20/toy-eth-pm",children:"ETH-DM"})," protocol aims to achieve.\nAnother project was a decentralised Twitter platform.\nSuch projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved."]}),"\n",(0,s.jsx)(t.p,{children:"As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network\nwhile preserving this out of the box experience."}),"\n",(0,s.jsx)(t.h2,{id:"whats-next",children:"What's next?"}),"\n",(0,s.jsxs)(t.p,{children:["Next, we are directing our attention towards ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/js-waku/issues/68",children:"Developer Experience"}),".\nWe already have ",(0,s.jsx)(t.a,{href:"https://www.npmjs.com/package/js-waku",children:"documentation"})," available but we want to provide more:\n",(0,s.jsx)(t.a,{href:"https://github.com/status-im/js-waku/issues/56",children:"Tutorials"}),", various examples\nand showing how ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/js-waku/issues/72",children:"JS-Waku can be used with Web3"}),"."]}),"\n",(0,s.jsx)(t.p,{children:"By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets.\nWe think JS-Waku builds a strong case for machine-to-machine (M2M) communications.\nThe first use cases we are looking into are dApp notifications:\nEnabling dApp to notify their user directly in their wallets!\nLeveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified\nof events such as DAO voting."}),"\n",(0,s.jsxs)(t.p,{children:["We already have some POC in the pipeline to enable voting and polling on the Waku network,\nallowing users to save gas by ",(0,s.jsx)(t.strong,{children:"not"})," broadcasting each individual vote on the blockchain."]}),"\n",(0,s.jsx)(t.p,{children:"To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples\nof signing, validating, encrypting and decrypting messages using Web3.\nWaku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity."}),"\n",(0,s.jsxs)(t.p,{children:["As you can read, we have grand plans for JS-Waku and Waku v2.\nThere is a lot to do, and we would love some help so feel free to\ncheck out the new role in our team:\n",(0,s.jsx)(t.a,{href:"https://status.im/our_team/jobs.html?gh_jid=3157894",children:"js-waku: Wallet & Dapp Integration Developer"}),".\nWe also have a number of ",(0,s.jsx)(t.a,{href:"https://status.im/our_team/jobs.html",children:"positions"})," open to work on Waku protocol and nim-waku."]}),"\n",(0,s.jsxs)(t.p,{children:["If you are as excited as us by JS-Waku, why not build a dApp with it?\nYou can find documentation on the ",(0,s.jsx)(t.a,{href:"https://www.npmjs.com/package/js-waku",children:"npmjs page"}),"."]}),"\n",(0,s.jsxs)(t.p,{children:["Whether you are a developer, you can come chat with us using ",(0,s.jsx)(t.a,{href:"https://status-im.github.io/js-waku/",children:"WakuJS Web Chat"}),"\nor ",(0,s.jsx)(t.a,{href:"https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md",children:"chat2"}),".\nYou can get support in #dappconnect-support on ",(0,s.jsx)(t.a,{href:"https://discord.gg/j5pGbn7MHZ",children:"Vac Discord"})," or ",(0,s.jsx)(t.a,{href:"https://t.me/dappconnectsupport",children:"Telegram"}),".\nIf you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it."]})]})}function l(e={}){const{wrapper:t}={...(0,i.R)(),...e.components};return t?(0,s.jsx)(t,{...e,children:(0,s.jsx)(u,{...e})}):u(e)}}}]); \ No newline at end of file diff --git a/assets/js/13a255f1.2c3500df.js b/assets/js/13a255f1.2c3500df.js new file mode 100644 index 00000000..4257df78 --- /dev/null +++ b/assets/js/13a255f1.2c3500df.js @@ -0,0 +1 @@ +"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[9739],{20155:e=>{e.exports=JSON.parse('{"permalink":"/rlog/wakuv2-noise","source":"@site/rlog/2022-05-17-noise.mdx","title":"Noise handshakes as key-exchange mechanism for Waku","description":"We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2.","date":"2022-05-17T10:00:00.000Z","tags":[],"readingTime":21.125,"hasTruncateMarker":true,"authors":[{"name":"s1fr0","github":"s1fr0","key":"s1fr0","page":null}],"frontMatter":{"layout":"post","name":"Noise handshakes as key-exchange mechanism for Waku","title":"Noise handshakes as key-exchange mechanism for Waku","date":"2022-05-17T10:00:00.000Z","authors":"s1fr0","published":true,"slug":"wakuv2-noise","categories":"research","summary":null,"image":"/img/noise/NM.png","discuss":"https://forum.vac.dev/t/discussion-noise-handshakes-as-key-exchange-mechanism-for-waku/137","_includes":["math"]},"unlisted":false,"prevItem":{"title":"Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay","permalink":"/rlog/wakuv2-relay-anon"},"nextItem":{"title":"Waku v2 Ambient Peer Discovery","permalink":"/rlog/wakuv2-apd"}}')},27094:(e,a,s)=>{s.r(a),s.d(a,{assets:()=>c,contentTitle:()=>r,default:()=>m,frontMatter:()=>i,metadata:()=>n,toc:()=>u});var n=s(20155),t=s(74848),o=s(28453);const i={layout:"post",name:"Noise handshakes as key-exchange mechanism for Waku",title:"Noise handshakes as key-exchange mechanism for Waku",date:new Date("2022-05-17T10:00:00.000Z"),authors:"s1fr0",published:!0,slug:"wakuv2-noise",categories:"research",summary:null,image:"/img/noise/NM.png",discuss:"https://forum.vac.dev/t/discussion-noise-handshakes-as-key-exchange-mechanism-for-waku/137",_includes:["math"]},r=void 0,c={authorsImageUrls:[void 0]},u=[];function h(e){const a={p:"p",...(0,o.R)(),...e.components};return(0,t.jsx)(a.p,{children:"We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2."})}function m(e={}){const{wrapper:a}={...(0,o.R)(),...e.components};return a?(0,t.jsx)(a,{...e,children:(0,t.jsx)(h,{...e})}):h(e)}},28453:(e,a,s)=>{s.d(a,{R:()=>i,x:()=>r});var n=s(96540);const t={},o=n.createContext(t);function i(e){const a=n.useContext(o);return n.useMemo((function(){return"function"==typeof e?e(a):{...a,...e}}),[a,e])}function r(e){let a;return a=e.disableParentContext?"function"==typeof e.components?e.components(t):e.components||t:i(e.components),n.createElement(o.Provider,{value:a},e.children)}}}]); \ No newline at end of file diff --git a/assets/js/13a255f1.eeb2e4ba.js b/assets/js/13a255f1.eeb2e4ba.js deleted file mode 100644 index c367e8f4..00000000 --- a/assets/js/13a255f1.eeb2e4ba.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[3415],{94866:(e,t,n)=>{n.r(t),n.d(t,{assets:()=>c,contentTitle:()=>s,default:()=>m,frontMatter:()=>o,metadata:()=>i,toc:()=>u});var r=n(87462),a=(n(67294),n(3905));const o={layout:"post",name:"Noise handshakes as key-exchange mechanism for Waku",title:"Noise handshakes as key-exchange mechanism for Waku",date:new Date("2022-05-17T10:00:00.000Z"),authors:"s1fr0",published:!0,slug:"wakuv2-noise",categories:"research",summary:null,image:"/img/noise/NM.png",discuss:"https://forum.vac.dev/t/discussion-noise-handshakes-as-key-exchange-mechanism-for-waku/137",_includes:["math"]},s=void 0,i={permalink:"/rlog/wakuv2-noise",source:"@site/rlog/2022-05-17-noise.mdx",title:"Noise handshakes as key-exchange mechanism for Waku",description:"We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2.",date:"2022-05-17T10:00:00.000Z",formattedDate:"May 17, 2022",tags:[],readingTime:21.115,hasTruncateMarker:!0,authors:[{name:"s1fr0",github:"s1fr0",key:"s1fr0"}],frontMatter:{layout:"post",name:"Noise handshakes as key-exchange mechanism for Waku",title:"Noise handshakes as key-exchange mechanism for Waku",date:"2022-05-17T10:00:00.000Z",authors:"s1fr0",published:!0,slug:"wakuv2-noise",categories:"research",summary:null,image:"/img/noise/NM.png",discuss:"https://forum.vac.dev/t/discussion-noise-handshakes-as-key-exchange-mechanism-for-waku/137",_includes:["math"]},prevItem:{title:"Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay",permalink:"/rlog/wakuv2-relay-anon"},nextItem:{title:"Waku v2 Ambient Peer Discovery",permalink:"/rlog/wakuv2-apd"}},c={authorsImageUrls:[void 0]},u=[],l={toc:u};function m(e){let{components:t,...n}=e;return(0,a.kt)("wrapper",(0,r.Z)({},l,n,{components:t,mdxType:"MDXLayout"}),(0,a.kt)("p",null,"We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2."))}m.isMDXComponent=!0},3905:(e,t,n)=>{n.d(t,{Zo:()=>l,kt:()=>f});var r=n(67294);function a(e,t,n){return t in e?Object.defineProperty(e,t,{value:n,enumerable:!0,configurable:!0,writable:!0}):e[t]=n,e}function o(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function s(e){for(var t=1;t=0||(a[n]=e[n]);return a}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(a[n]=e[n])}return a}var c=r.createContext({}),u=function(e){var t=r.useContext(c),n=t;return e&&(n="function"==typeof e?e(t):s(s({},t),e)),n},l=function(e){var t=u(e.components);return r.createElement(c.Provider,{value:t},e.children)},m={inlineCode:"code",wrapper:function(e){var t=e.children;return r.createElement(r.Fragment,{},t)}},p=r.forwardRef((function(e,t){var n=e.components,a=e.mdxType,o=e.originalType,c=e.parentName,l=i(e,["components","mdxType","originalType","parentName"]),p=u(n),f=a,h=p["".concat(c,".").concat(f)]||p[f]||m[f]||o;return n?r.createElement(h,s(s({ref:t},l),{},{components:n})):r.createElement(h,s({ref:t},l))}));function f(e,t){var n=arguments,a=t&&t.mdxType;if("string"==typeof e||a){var o=n.length,s=new Array(o);s[0]=p;var i={};for(var c in t)hasOwnProperty.call(t,c)&&(i[c]=t[c]);i.originalType=e,i.mdxType="string"==typeof e?e:a,s[1]=i;for(var u=2;u{r.r(t),r.d(t,{assets:()=>c,contentTitle:()=>i,default:()=>u,frontMatter:()=>o,metadata:()=>s,toc:()=>l});var n=r(87462),a=(r(67294),r(3905));const o={layout:"post",name:"DNS Based Discovery",title:"DNS Based Discovery",date:new Date("2020-02-07T12:00:00.000Z"),authors:"dean",published:!0,slug:"dns-based-discovery",categories:"research"},i=void 0,s={permalink:"/rlog/dns-based-discovery",source:"@site/rlog/2020-02-7-dns-based-discovery.mdx",title:"DNS Based Discovery",description:"A look at EIP-1459 and the benefits of DNS based discovery.",date:"2020-02-07T12:00:00.000Z",formattedDate:"February 7, 2020",tags:[],readingTime:5.635,hasTruncateMarker:!0,authors:[{name:"Dean",twitter:"DeanEigenmann",github:"decanus",website:"https://dean.eigenmann.me",key:"dean"}],frontMatter:{layout:"post",name:"DNS Based Discovery",title:"DNS Based Discovery",date:"2020-02-07T12:00:00.000Z",authors:"dean",published:!0,slug:"dns-based-discovery",categories:"research"},prevItem:{title:"Waku Update",permalink:"/rlog/waku-update"},nextItem:{title:"Fixing Whisper with Waku",permalink:"/rlog/fixing-whisper-with-waku"}},c={authorsImageUrls:[void 0]},l=[],p={toc:l};function u(e){let{components:t,...r}=e;return(0,a.kt)("wrapper",(0,n.Z)({},p,r,{components:t,mdxType:"MDXLayout"}),(0,a.kt)("p",null,"A look at EIP-1459 and the benefits of DNS based discovery."))}u.isMDXComponent=!0},3905:(e,t,r)=>{r.d(t,{Zo:()=>p,kt:()=>f});var n=r(67294);function a(e,t,r){return t in e?Object.defineProperty(e,t,{value:r,enumerable:!0,configurable:!0,writable:!0}):e[t]=r,e}function o(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),r.push.apply(r,n)}return r}function i(e){for(var t=1;t=0||(a[r]=e[r]);return a}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,r)&&(a[r]=e[r])}return a}var c=n.createContext({}),l=function(e){var t=n.useContext(c),r=t;return e&&(r="function"==typeof e?e(t):i(i({},t),e)),r},p=function(e){var t=l(e.components);return n.createElement(c.Provider,{value:t},e.children)},u={inlineCode:"code",wrapper:function(e){var t=e.children;return n.createElement(n.Fragment,{},t)}},d=n.forwardRef((function(e,t){var r=e.components,a=e.mdxType,o=e.originalType,c=e.parentName,p=s(e,["components","mdxType","originalType","parentName"]),d=l(r),f=a,y=d["".concat(c,".").concat(f)]||d[f]||u[f]||o;return r?n.createElement(y,i(i({ref:t},p),{},{components:r})):n.createElement(y,i({ref:t},p))}));function f(e,t){var r=arguments,a=t&&t.mdxType;if("string"==typeof e||a){var o=r.length,i=new Array(o);i[0]=d;var s={};for(var c in t)hasOwnProperty.call(t,c)&&(s[c]=t[c]);s.originalType=e,s.mdxType="string"==typeof e?e:a,i[1]=s;for(var l=2;l{n.d(t,{R:()=>r,x:()=>i});var s=n(96540);const a={},o=s.createContext(a);function r(e){const t=s.useContext(o);return s.useMemo((function(){return"function"==typeof e?e(t):{...t,...e}}),[t,e])}function i(e){let t;return t=e.disableParentContext?"function"==typeof e.components?e.components(a):e.components||a:r(e.components),s.createElement(o.Provider,{value:t},e.children)}},87048:(e,t,n)=>{n.r(t),n.d(t,{assets:()=>d,contentTitle:()=>i,default:()=>l,frontMatter:()=>r,metadata:()=>s,toc:()=>c});var s=n(91223),a=n(74848),o=n(28453);const r={layout:"post",name:"DNS Based Discovery",title:"DNS Based Discovery",date:new Date("2020-02-07T12:00:00.000Z"),authors:"dean",published:!0,slug:"dns-based-discovery",categories:"research"},i=void 0,d={authorsImageUrls:[void 0]},c=[];function u(e){const t={p:"p",...(0,o.R)(),...e.components};return(0,a.jsx)(t.p,{children:"A look at EIP-1459 and the benefits of DNS based discovery."})}function l(e={}){const{wrapper:t}={...(0,o.R)(),...e.components};return t?(0,a.jsx)(t,{...e,children:(0,a.jsx)(u,{...e})}):u(e)}},91223:e=>{e.exports=JSON.parse('{"permalink":"/rlog/dns-based-discovery","source":"@site/rlog/2020-02-7-dns-based-discovery.mdx","title":"DNS Based Discovery","description":"A look at EIP-1459 and the benefits of DNS based discovery.","date":"2020-02-07T12:00:00.000Z","tags":[],"readingTime":5.645,"hasTruncateMarker":true,"authors":[{"name":"Dean","twitter":"DeanEigenmann","github":"decanus","website":"https://dean.eigenmann.me","key":"dean","page":null}],"frontMatter":{"layout":"post","name":"DNS Based Discovery","title":"DNS Based Discovery","date":"2020-02-07T12:00:00.000Z","authors":"dean","published":true,"slug":"dns-based-discovery","categories":"research"},"unlisted":false,"prevItem":{"title":"Waku Update","permalink":"/rlog/waku-update"},"nextItem":{"title":"Fixing Whisper with Waku","permalink":"/rlog/fixing-whisper-with-waku"}}')}}]); \ No newline at end of file diff --git a/assets/js/15a93281.a8246e21.js b/assets/js/15a93281.a8246e21.js deleted file mode 100644 index 8a2a7e48..00000000 --- a/assets/js/15a93281.a8246e21.js +++ /dev/null @@ -1 +0,0 @@ -"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[2594],{32788:(e,t,r)=>{r.r(t),r.d(t,{assets:()=>l,contentTitle:()=>i,default:()=>p,frontMatter:()=>o,metadata:()=>s,toc:()=>h});var n=r(87462),a=(r(67294),r(3905));const o={layout:"post",name:"DNS Based Discovery",title:"DNS Based Discovery",date:new Date("2020-02-07T12:00:00.000Z"),authors:"dean",published:!0,slug:"dns-based-discovery",categories:"research"},i=void 0,s={permalink:"/rlog/dns-based-discovery",source:"@site/rlog/2020-02-7-dns-based-discovery.mdx",title:"DNS Based Discovery",description:"A look at EIP-1459 and the benefits of DNS based discovery.",date:"2020-02-07T12:00:00.000Z",formattedDate:"February 7, 2020",tags:[],readingTime:5.635,hasTruncateMarker:!0,authors:[{name:"Dean",twitter:"DeanEigenmann",github:"decanus",website:"https://dean.eigenmann.me",key:"dean"}],frontMatter:{layout:"post",name:"DNS Based Discovery",title:"DNS Based Discovery",date:"2020-02-07T12:00:00.000Z",authors:"dean",published:!0,slug:"dns-based-discovery",categories:"research"},prevItem:{title:"Waku Update",permalink:"/rlog/waku-update"},nextItem:{title:"Fixing Whisper with Waku",permalink:"/rlog/fixing-whisper-with-waku"}},l={authorsImageUrls:[void 0]},h=[],c={toc:h};function p(e){let{components:t,...r}=e;return(0,a.kt)("wrapper",(0,n.Z)({},c,r,{components:t,mdxType:"MDXLayout"}),(0,a.kt)("p",null,"A look at EIP-1459 and the benefits of DNS based discovery."),(0,a.kt)("p",null,"Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as ",(0,a.kt)("a",{parentName:"p",href:"https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf"},"Kademlia")," which utilizes a ",(0,a.kt)("a",{parentName:"p",href:"https://en.wikipedia.org/wiki/Distributed_hash_table"},"distributed hash table")," or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data."),(0,a.kt)("p",null,"Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a ",(0,a.kt)("a",{parentName:"p",href:"https://rfc.vac.dev/status/deprecated/client/#bootstrapping"},"static list of nodes")," that a client can connect to."),(0,a.kt)("p",null,"In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable."),(0,a.kt)("p",null,"How do we do this?"),(0,a.kt)("p",null,(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-1459"},"EIP 1459: Node Discovery via DNS"),", which is one of the strategies we are using for discovering waku nodes. ",(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-1459"},"EIP-1459")," is a DNS-based discovery protocol that stores ",(0,a.kt)("a",{parentName:"p",href:"https://en.wikipedia.org/wiki/Merkle_tree"},"merkle trees")," in DNS records which contain connection information for nodes."),(0,a.kt)("p",null,(0,a.kt)("em",{parentName:"p"},"Waku is our fork of Whisper. Oskar recently wrote an ",(0,a.kt)("a",{parentName:"em",href:"https://vac.dev/fixing-whisper-with-waku"},"entire post")," explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification ",(0,a.kt)("a",{parentName:"em",href:"https://rfc.vac.dev/waku/standards/legacy/6/waku1/"},"here"))),(0,a.kt)("p",null,"DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be."),(0,a.kt)("pre",null,(0,a.kt)("code",{parentName:"pre",className:"language-console"},"> dig dnsseed.bluematt.me +short\n129.226.73.12\n107.180.78.111\n169.255.56.123\n91.216.149.28\n85.209.240.91\n66.232.124.232\n207.55.53.96\n86.149.241.168\n193.219.38.57\n190.198.210.139\n74.213.232.234\n158.181.226.33\n176.99.2.207\n202.55.87.45\n37.205.10.3\n90.133.4.73\n176.191.182.3\n109.207.166.232\n45.5.117.59\n178.211.170.2\n160.16.0.30\n")),(0,a.kt)("p",null,"The above displays the result of querying on of these DNS seeds. All the nodes are stored as ",(0,a.kt)("a",{parentName:"p",href:"https://simpledns.plus/help/a-records"},(0,a.kt)("inlineCode",{parentName:"a"},"A")," records")," for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the ",(0,a.kt)("a",{parentName:"p",href:"https://en.bitcoin.it/wiki/Network#IRC"},"IRC bootstrapping method in v0.8.2"),"."),(0,a.kt)("p",null,"What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices."),(0,a.kt)("p",null,"Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name."),(0,a.kt)("p",null,"As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so."),(0,a.kt)("p",null,"One thing you notice when looking at ",(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-1459"},"EIP-1459")," is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method?"),(0,a.kt)("p",null,"There are multiple reasons, but the main one is ",(0,a.kt)("strong",{parentName:"p"},"security"),". In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in ",(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-1459"},"EIP-1459")," where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records."),(0,a.kt)("p",null,"Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party."),(0,a.kt)("p",null,"So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using ",(0,a.kt)("strong",{parentName:"p"},"merkle trees")," as follows:"),(0,a.kt)("ul",null,(0,a.kt)("li",{parentName:"ul"},(0,a.kt)("p",{parentName:"li"},"The root of the tree is stored in a ",(0,a.kt)("a",{parentName:"p",href:"https://simpledns.plus/help/txt-records"},(0,a.kt)("inlineCode",{parentName:"a"},"TXT")," record"),", this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above."),(0,a.kt)("p",{parentName:"li"},"Additionally, there is also a root hash to a second tree called a ",(0,a.kt)("strong",{parentName:"p"},"link tree"),", it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names."),(0,a.kt)("p",{parentName:"li"},"The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version.")),(0,a.kt)("li",{parentName:"ul"},(0,a.kt)("p",{parentName:"li"},"Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a ",(0,a.kt)("inlineCode",{parentName:"p"},"TXT")," record. The branch record contains all the hashes of the branch's leafs.")),(0,a.kt)("li",{parentName:"ul"},(0,a.kt)("p",{parentName:"li"},"Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at ",(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-778"},"EIP-778"),", or read a short blog post I wrote explaining them ",(0,a.kt)("a",{parentName:"p",href:"https://dean.eigenmann.me/blog/2020/01/21/network-addresses-in-ethereum/#enr"},"here"),"."))),(0,a.kt)("p",null,"Below is the zone file taken from the ",(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-1459"},"EIP-1459"),", displaying how this looks in practice."),(0,a.kt)("pre",null,(0,a.kt)("code",{parentName:"pre"},"; name ttl class type content\n@ 60 IN TXT enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA\nC7HRFPF3BLGF3YR4DY5KX3SMBE 86900 IN TXT enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org\nJWXYDBPXYWG6FX3GMDIBFA6CJ4 86900 IN TXT enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24\n2XS2367YHAXJFGLZHVAWLQD4ZY 86900 IN TXT enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA\nH4FHT4B454P6UXFD7JCYQ5PWDY 86900 IN TXT enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI\nMHTDO6TMUBRIA2XWG5LUDACK24 86900 IN TXT enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o\n")),(0,a.kt)("p",null,"All of this has already been introduced into go-ethereum with the pull request ",(0,a.kt)("a",{parentName:"p",href:"https://github.com/ethereum/go-ethereum/pull/20094"},"#20094"),", created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on ",(0,a.kt)("a",{parentName:"p",href:"https://github.com/decanus/dns-discovery"},"github"),"."),(0,a.kt)("p",null,"I hope this was a helpful explainer into DNS based discovery, and shows ",(0,a.kt)("a",{parentName:"p",href:"https://eips.ethereum.org/EIPS/eip-1459"},"EIP-1459"),"'s benefits over more traditional DNS-based discovery schemes."))}p.isMDXComponent=!0},3905:(e,t,r)=>{r.d(t,{Zo:()=>c,kt:()=>u});var n=r(67294);function a(e,t,r){return t in e?Object.defineProperty(e,t,{value:r,enumerable:!0,configurable:!0,writable:!0}):e[t]=r,e}function o(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),r.push.apply(r,n)}return r}function i(e){for(var t=1;t=0||(a[r]=e[r]);return a}(e,t);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,r)&&(a[r]=e[r])}return a}var l=n.createContext({}),h=function(e){var t=n.useContext(l),r=t;return e&&(r="function"==typeof e?e(t):i(i({},t),e)),r},c=function(e){var t=h(e.components);return n.createElement(l.Provider,{value:t},e.children)},p={inlineCode:"code",wrapper:function(e){var t=e.children;return n.createElement(n.Fragment,{},t)}},d=n.forwardRef((function(e,t){var r=e.components,a=e.mdxType,o=e.originalType,l=e.parentName,c=s(e,["components","mdxType","originalType","parentName"]),d=h(r),u=a,m=d["".concat(l,".").concat(u)]||d[u]||p[u]||o;return r?n.createElement(m,i(i({ref:t},c),{},{components:r})):n.createElement(m,i({ref:t},c))}));function u(e,t){var r=arguments,a=t&&t.mdxType;if("string"==typeof e||a){var o=r.length,i=new Array(o);i[0]=d;var s={};for(var l in t)hasOwnProperty.call(t,l)&&(s[l]=t[l]);s.originalType=e,s.mdxType="string"==typeof e?e:a,i[1]=s;for(var h=2;h{s.r(t),s.d(t,{assets:()=>h,contentTitle:()=>a,default:()=>c,frontMatter:()=>o,metadata:()=>n,toc:()=>l});var n=s(91223),i=s(74848),r=s(28453);const o={layout:"post",name:"DNS Based Discovery",title:"DNS Based Discovery",date:new Date("2020-02-07T12:00:00.000Z"),authors:"dean",published:!0,slug:"dns-based-discovery",categories:"research"},a=void 0,h={authorsImageUrls:[void 0]},l=[];function d(e){const t={a:"a",code:"code",em:"em",li:"li",p:"p",pre:"pre",strong:"strong",ul:"ul",...(0,r.R)(),...e.components};return(0,i.jsxs)(i.Fragment,{children:[(0,i.jsx)(t.p,{children:"A look at EIP-1459 and the benefits of DNS based discovery."}),"\n","\n",(0,i.jsxs)(t.p,{children:["Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as ",(0,i.jsx)(t.a,{href:"https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf",children:"Kademlia"})," which utilizes a ",(0,i.jsx)(t.a,{href:"https://en.wikipedia.org/wiki/Distributed_hash_table",children:"distributed hash table"})," or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data."]}),"\n",(0,i.jsxs)(t.p,{children:["Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a ",(0,i.jsx)(t.a,{href:"https://rfc.vac.dev/status/deprecated/client/#bootstrapping",children:"static list of nodes"})," that a client can connect to."]}),"\n",(0,i.jsx)(t.p,{children:"In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable."}),"\n",(0,i.jsx)(t.p,{children:"How do we do this?"}),"\n",(0,i.jsxs)(t.p,{children:[(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-1459",children:"EIP 1459: Node Discovery via DNS"}),", which is one of the strategies we are using for discovering waku nodes. ",(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-1459",children:"EIP-1459"})," is a DNS-based discovery protocol that stores ",(0,i.jsx)(t.a,{href:"https://en.wikipedia.org/wiki/Merkle_tree",children:"merkle trees"})," in DNS records which contain connection information for nodes."]}),"\n",(0,i.jsx)(t.p,{children:(0,i.jsxs)(t.em,{children:["Waku is our fork of Whisper. Oskar recently wrote an ",(0,i.jsx)(t.a,{href:"https://vac.dev/fixing-whisper-with-waku",children:"entire post"})," explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification ",(0,i.jsx)(t.a,{href:"https://rfc.vac.dev/waku/standards/legacy/6/waku1/",children:"here"})]})}),"\n",(0,i.jsx)(t.p,{children:"DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be."}),"\n",(0,i.jsx)(t.pre,{children:(0,i.jsx)(t.code,{className:"language-console",children:"> dig dnsseed.bluematt.me +short\n129.226.73.12\n107.180.78.111\n169.255.56.123\n91.216.149.28\n85.209.240.91\n66.232.124.232\n207.55.53.96\n86.149.241.168\n193.219.38.57\n190.198.210.139\n74.213.232.234\n158.181.226.33\n176.99.2.207\n202.55.87.45\n37.205.10.3\n90.133.4.73\n176.191.182.3\n109.207.166.232\n45.5.117.59\n178.211.170.2\n160.16.0.30\n"})}),"\n",(0,i.jsxs)(t.p,{children:["The above displays the result of querying on of these DNS seeds. All the nodes are stored as ",(0,i.jsxs)(t.a,{href:"https://simpledns.plus/help/a-records",children:[(0,i.jsx)(t.code,{children:"A"})," records"]})," for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the ",(0,i.jsx)(t.a,{href:"https://en.bitcoin.it/wiki/Network#IRC",children:"IRC bootstrapping method in v0.8.2"}),"."]}),"\n",(0,i.jsx)(t.p,{children:"What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices."}),"\n",(0,i.jsx)(t.p,{children:"Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name."}),"\n",(0,i.jsx)(t.p,{children:"As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so."}),"\n",(0,i.jsxs)(t.p,{children:["One thing you notice when looking at ",(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-1459",children:"EIP-1459"})," is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method?"]}),"\n",(0,i.jsxs)(t.p,{children:["There are multiple reasons, but the main one is ",(0,i.jsx)(t.strong,{children:"security"}),". In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in ",(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-1459",children:"EIP-1459"})," where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records."]}),"\n",(0,i.jsx)(t.p,{children:"Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party."}),"\n",(0,i.jsxs)(t.p,{children:["So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using ",(0,i.jsx)(t.strong,{children:"merkle trees"})," as follows:"]}),"\n",(0,i.jsxs)(t.ul,{children:["\n",(0,i.jsxs)(t.li,{children:["\n",(0,i.jsxs)(t.p,{children:["The root of the tree is stored in a ",(0,i.jsxs)(t.a,{href:"https://simpledns.plus/help/txt-records",children:[(0,i.jsx)(t.code,{children:"TXT"})," record"]}),", this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above."]}),"\n",(0,i.jsxs)(t.p,{children:["Additionally, there is also a root hash to a second tree called a ",(0,i.jsx)(t.strong,{children:"link tree"}),", it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names."]}),"\n",(0,i.jsx)(t.p,{children:"The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version."}),"\n"]}),"\n",(0,i.jsxs)(t.li,{children:["\n",(0,i.jsxs)(t.p,{children:["Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a ",(0,i.jsx)(t.code,{children:"TXT"})," record. The branch record contains all the hashes of the branch's leafs."]}),"\n"]}),"\n",(0,i.jsxs)(t.li,{children:["\n",(0,i.jsxs)(t.p,{children:["Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at ",(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-778",children:"EIP-778"}),", or read a short blog post I wrote explaining them ",(0,i.jsx)(t.a,{href:"https://dean.eigenmann.me/blog/2020/01/21/network-addresses-in-ethereum/#enr",children:"here"}),"."]}),"\n"]}),"\n"]}),"\n",(0,i.jsxs)(t.p,{children:["Below is the zone file taken from the ",(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-1459",children:"EIP-1459"}),", displaying how this looks in practice."]}),"\n",(0,i.jsx)(t.pre,{children:(0,i.jsx)(t.code,{children:"; name ttl class type content\n@ 60 IN TXT enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA\nC7HRFPF3BLGF3YR4DY5KX3SMBE 86900 IN TXT enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org\nJWXYDBPXYWG6FX3GMDIBFA6CJ4 86900 IN TXT enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24\n2XS2367YHAXJFGLZHVAWLQD4ZY 86900 IN TXT enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA\nH4FHT4B454P6UXFD7JCYQ5PWDY 86900 IN TXT enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI\nMHTDO6TMUBRIA2XWG5LUDACK24 86900 IN TXT enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o\n"})}),"\n",(0,i.jsxs)(t.p,{children:["All of this has already been introduced into go-ethereum with the pull request ",(0,i.jsx)(t.a,{href:"https://github.com/ethereum/go-ethereum/pull/20094",children:"#20094"}),", created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on ",(0,i.jsx)(t.a,{href:"https://github.com/decanus/dns-discovery",children:"github"}),"."]}),"\n",(0,i.jsxs)(t.p,{children:["I hope this was a helpful explainer into DNS based discovery, and shows ",(0,i.jsx)(t.a,{href:"https://eips.ethereum.org/EIPS/eip-1459",children:"EIP-1459"}),"'s benefits over more traditional DNS-based discovery schemes."]})]})}function c(e={}){const{wrapper:t}={...(0,r.R)(),...e.components};return t?(0,i.jsx)(t,{...e,children:(0,i.jsx)(d,{...e})}):d(e)}},28453:(e,t,s)=>{s.d(t,{R:()=>o,x:()=>a});var n=s(96540);const i={},r=n.createContext(i);function o(e){const t=n.useContext(r);return n.useMemo((function(){return"function"==typeof e?e(t):{...t,...e}}),[t,e])}function a(e){let t;return t=e.disableParentContext?"function"==typeof e.components?e.components(i):e.components||i:o(e.components),n.createElement(r.Provider,{value:t},e.children)}},91223:e=>{e.exports=JSON.parse('{"permalink":"/rlog/dns-based-discovery","source":"@site/rlog/2020-02-7-dns-based-discovery.mdx","title":"DNS Based Discovery","description":"A look at EIP-1459 and the benefits of DNS based discovery.","date":"2020-02-07T12:00:00.000Z","tags":[],"readingTime":5.645,"hasTruncateMarker":true,"authors":[{"name":"Dean","twitter":"DeanEigenmann","github":"decanus","website":"https://dean.eigenmann.me","key":"dean","page":null}],"frontMatter":{"layout":"post","name":"DNS Based Discovery","title":"DNS Based Discovery","date":"2020-02-07T12:00:00.000Z","authors":"dean","published":true,"slug":"dns-based-discovery","categories":"research"},"unlisted":false,"prevItem":{"title":"Waku Update","permalink":"/rlog/waku-update"},"nextItem":{"title":"Fixing Whisper with Waku","permalink":"/rlog/fixing-whisper-with-waku"}}')}}]); \ No newline at end of file diff --git a/assets/js/165.6ae0ec2c.js b/assets/js/165.6ae0ec2c.js new file mode 100644 index 00000000..1cf16c84 --- /dev/null +++ b/assets/js/165.6ae0ec2c.js @@ -0,0 +1,2 @@ +/*! For license information please see 165.6ae0ec2c.js.LICENSE.txt */ +"use strict";(self.webpackChunkvac_dev=self.webpackChunkvac_dev||[]).push([[165],{90165:(e,t,n)=>{function r(e,t){(null==t||t>e.length)&&(t=e.length);for(var n=0,r=Array(t);n=e.length?{done:!0}:{done:!1,value:e[r++]}},e:function(e){throw e},f:a}}throw new TypeError("Invalid attempt to iterate non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}var i,o=!0,s=!1;return{s:function(){n=n.call(e)},n:function(){var e=n.next();return o=e.done,e},e:function(e){s=!0,i=e},f:function(){try{o||null==n.return||n.return()}finally{if(s)throw i}}}}function s(e,t,n){return(t=c(t))in e?Object.defineProperty(e,t,{value:n,enumerable:!0,configurable:!0,writable:!0}):e[t]=n,e}function l(e,t){return function(e){if(Array.isArray(e))return e}(e)||function(e,t){var n=null==e?null:"undefined"!=typeof Symbol&&e[Symbol.iterator]||e["@@iterator"];if(null!=n){var r,a,i,o,s=[],l=!0,u=!1;try{if(i=(n=n.call(e)).next,0===t){if(Object(n)!==n)return;l=!1}else for(;!(l=(r=i.call(n)).done)&&(s.push(r.value),s.length!==t);l=!0);}catch(e){u=!0,a=e}finally{try{if(!l&&null!=n.return&&(o=n.return(),Object(o)!==o))return}finally{if(u)throw a}}return s}}(e,t)||h(e,t)||function(){throw new TypeError("Invalid attempt to destructure non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function u(e){return function(e){if(Array.isArray(e))return r(e)}(e)||function(e){if("undefined"!=typeof Symbol&&null!=e[Symbol.iterator]||null!=e["@@iterator"])return Array.from(e)}(e)||h(e)||function(){throw new TypeError("Invalid attempt to spread non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function c(e){var t=function(e,t){if("object"!=typeof e||!e)return e;var n=e[Symbol.toPrimitive];if(void 0!==n){var r=n.call(e,t);if("object"!=typeof r)return r;throw new TypeError("@@toPrimitive must return a primitive value.")}return String(e)}(e,"string");return"symbol"==typeof t?t:t+""}function d(e){return d="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},d(e)}function h(e,t){if(e){if("string"==typeof e)return r(e,t);var n={}.toString.call(e).slice(8,-1);return"Object"===n&&e.constructor&&(n=e.constructor.name),"Map"===n||"Set"===n?Array.from(e):"Arguments"===n||/^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n)?r(e,t):void 0}}n.d(t,{A:()=>Ih});var f="undefined"==typeof window?null:window,p=f?f.navigator:null;f&&f.document;var g,v,y,m,b,x,w,E,k,T,C,P,S,B,D,_,A,M,R,I,N,L,z,O,V,F,j,X,Y=d(""),q=d({}),W=d((function(){})),K="undefined"==typeof HTMLElement?"undefined":d(HTMLElement),U=function(e){return e&&e.instanceString&&G(e.instanceString)?e.instanceString():null},H=function(e){return null!=e&&d(e)==Y},G=function(e){return null!=e&&d(e)===W},Z=function(e){return!ee(e)&&(Array.isArray?Array.isArray(e):null!=e&&e instanceof Array)},$=function(e){return null!=e&&d(e)===q&&!Z(e)&&e.constructor===Object},Q=function(e){return null!=e&&d(e)===d(1)&&!isNaN(e)},J=function(e){return"undefined"===K?void 0:null!=e&&e instanceof HTMLElement},ee=function(e){return te(e)||ne(e)},te=function(e){return"collection"===U(e)&&e._private.single},ne=function(e){return"collection"===U(e)&&!e._private.single},re=function(e){return"core"===U(e)},ae=function(e){return"stylesheet"===U(e)},ie=function(e){return null==e||!(""!==e&&!e.match(/^\s+$/))},oe=function(e){return function(e){return null!=e&&d(e)===q}(e)&&G(e.then)},se=function(e,t){t||(t=function(){if(1===arguments.length)return arguments[0];if(0===arguments.length)return"undefined";for(var e=[],t=0;tt?1:0},me=null!=Object.assign?Object.assign.bind(Object):function(e){for(var t=arguments,n=1;n255)return;t.push(Math.floor(i))}var o=r[1]||r[2]||r[3],s=r[1]&&r[2]&&r[3];if(o&&!s)return;var l=n[4];if(void 0!==l){if((l=parseFloat(l))<0||l>1)return;t.push(l)}}return t}(e)||function(e){var t,n,r,a,i,o,s,l;function u(e,t,n){return n<0&&(n+=1),n>1&&(n-=1),n<1/6?e+6*(t-e)*n:n<.5?t:n<2/3?e+(t-e)*(2/3-n)*6:e}var c=new RegExp("^"+ge+"$").exec(e);if(c){if((n=parseInt(c[1]))<0?n=(360- -1*n%360)%360:n>360&&(n%=360),n/=360,(r=parseFloat(c[2]))<0||r>100)return;if(r/=100,(a=parseFloat(c[3]))<0||a>100)return;if(a/=100,void 0!==(i=c[4])&&((i=parseFloat(i))<0||i>1))return;if(0===r)o=s=l=Math.round(255*a);else{var d=a<.5?a*(1+r):a+r-a*r,h=2*a-d;o=Math.round(255*u(h,d,n+1/3)),s=Math.round(255*u(h,d,n)),l=Math.round(255*u(h,d,n-1/3))}t=[o,s,l,i]}return t}(e)},xe={transparent:[0,0,0,0],aliceblue:[240,248,255],antiquewhite:[250,235,215],aqua:[0,255,255],aquamarine:[127,255,212],azure:[240,255,255],beige:[245,245,220],bisque:[255,228,196],black:[0,0,0],blanchedalmond:[255,235,205],blue:[0,0,255],blueviolet:[138,43,226],brown:[165,42,42],burlywood:[222,184,135],cadetblue:[95,158,160],chartreuse:[127,255,0],chocolate:[210,105,30],coral:[255,127,80],cornflowerblue:[100,149,237],cornsilk:[255,248,220],crimson:[220,20,60],cyan:[0,255,255],darkblue:[0,0,139],darkcyan:[0,139,139],darkgoldenrod:[184,134,11],darkgray:[169,169,169],darkgreen:[0,100,0],darkgrey:[169,169,169],darkkhaki:[189,183,107],darkmagenta:[139,0,139],darkolivegreen:[85,107,47],darkorange:[255,140,0],darkorchid:[153,50,204],darkred:[139,0,0],darksalmon:[233,150,122],darkseagreen:[143,188,143],darkslateblue:[72,61,139],darkslategray:[47,79,79],darkslategrey:[47,79,79],darkturquoise:[0,206,209],darkviolet:[148,0,211],deeppink:[255,20,147],deepskyblue:[0,191,255],dimgray:[105,105,105],dimgrey:[105,105,105],dodgerblue:[30,144,255],firebrick:[178,34,34],floralwhite:[255,250,240],forestgreen:[34,139,34],fuchsia:[255,0,255],gainsboro:[220,220,220],ghostwhite:[248,248,255],gold:[255,215,0],goldenrod:[218,165,32],gray:[128,128,128],grey:[128,128,128],green:[0,128,0],greenyellow:[173,255,47],honeydew:[240,255,240],hotpink:[255,105,180],indianred:[205,92,92],indigo:[75,0,130],ivory:[255,255,240],khaki:[240,230,140],lavender:[230,230,250],lavenderblush:[255,240,245],lawngreen:[124,252,0],lemonchiffon:[255,250,205],lightblue:[173,216,230],lightcoral:[240,128,128],lightcyan:[224,255,255],lightgoldenrodyellow:[250,250,210],lightgray:[211,211,211],lightgreen:[144,238,144],lightgrey:[211,211,211],lightpink:[255,182,193],lightsalmon:[255,160,122],lightseagreen:[32,178,170],lightskyblue:[135,206,250],lightslategray:[119,136,153],lightslategrey:[119,136,153],lightsteelblue:[176,196,222],lightyellow:[255,255,224],lime:[0,255,0],limegreen:[50,205,50],linen:[250,240,230],magenta:[255,0,255],maroon:[128,0,0],mediumaquamarine:[102,205,170],mediumblue:[0,0,205],mediumorchid:[186,85,211],mediumpurple:[147,112,219],mediumseagreen:[60,179,113],mediumslateblue:[123,104,238],mediumspringgreen:[0,250,154],mediumturquoise:[72,209,204],mediumvioletred:[199,21,133],midnightblue:[25,25,112],mintcream:[245,255,250],mistyrose:[255,228,225],moccasin:[255,228,181],navajowhite:[255,222,173],navy:[0,0,128],oldlace:[253,245,230],olive:[128,128,0],olivedrab:[107,142,35],orange:[255,165,0],orangered:[255,69,0],orchid:[218,112,214],palegoldenrod:[238,232,170],palegreen:[152,251,152],paleturquoise:[175,238,238],palevioletred:[219,112,147],papayawhip:[255,239,213],peachpuff:[255,218,185],peru:[205,133,63],pink:[255,192,203],plum:[221,160,221],powderblue:[176,224,230],purple:[128,0,128],red:[255,0,0],rosybrown:[188,143,143],royalblue:[65,105,225],saddlebrown:[139,69,19],salmon:[250,128,114],sandybrown:[244,164,96],seagreen:[46,139,87],seashell:[255,245,238],sienna:[160,82,45],silver:[192,192,192],skyblue:[135,206,235],slateblue:[106,90,205],slategray:[112,128,144],slategrey:[112,128,144],snow:[255,250,250],springgreen:[0,255,127],steelblue:[70,130,180],tan:[210,180,140],teal:[0,128,128],thistle:[216,191,216],tomato:[255,99,71],turquoise:[64,224,208],violet:[238,130,238],wheat:[245,222,179],white:[255,255,255],whitesmoke:[245,245,245],yellow:[255,255,0],yellowgreen:[154,205,50]},we=function(e){for(var t=e.map,n=e.keys,r=n.length,a=0;a=o||t<0||v&&e-p>=c}function x(){var e=t();if(b(e))return w(e);h=setTimeout(x,function(e){var t=o-(e-f);return v?a(t,c-(e-p)):t}(e))}function w(e){return h=void 0,y&&l?m(e):(l=u=void 0,d)}function E(){var e=t(),n=b(e);if(l=arguments,u=this,f=e,n){if(void 0===h)return function(e){return p=e,h=setTimeout(x,o),g?m(e):d}(f);if(v)return clearTimeout(h),h=setTimeout(x,o),m(f)}return void 0===h&&(h=setTimeout(x,o)),d}return o=n(o)||0,e(s)&&(g=!!s.leading,c=(v="maxWait"in s)?r(n(s.maxWait)||0,o):c,y="trailing"in s?!!s.trailing:y),E.cancel=function(){void 0!==h&&clearTimeout(h),p=0,l=f=u=h=void 0},E.flush=function(){return void 0===h?d:w(t())},E}}()),Me=f?f.performance:null,Re=Me&&Me.now?function(){return Me.now()}:function(){return Date.now()},Ie=function(){if(f){if(f.requestAnimationFrame)return function(e){f.requestAnimationFrame(e)};if(f.mozRequestAnimationFrame)return function(e){f.mozRequestAnimationFrame(e)};if(f.webkitRequestAnimationFrame)return function(e){f.webkitRequestAnimationFrame(e)};if(f.msRequestAnimationFrame)return function(e){f.msRequestAnimationFrame(e)}}return function(e){e&&setTimeout((function(){e(Re())}),1e3/60)}}(),Ne=function(e){return Ie(e)},Le=Re,ze=9261,Oe=5381,Ve=function(e){for(var t,n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:ze;!(t=e.next()).done;)n=65599*n+t.value|0;return n},Fe=function(e){return 65599*(arguments.length>1&&void 0!==arguments[1]?arguments[1]:ze)+e|0},je=function(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:Oe;return(t<<5)+t+e|0},Xe=function(e){return 2097152*e[0]+e[1]},Ye=function(e,t){return[Fe(e[0],t[0]),je(e[1],t[1])]},qe=function(e,t){var n={value:0,done:!1},r=0,a=e.length;return Ve({next:function(){return r=0;r--)e[r]===t&&e.splice(r,1)},dt=function(e){e.splice(0,e.length)},ht=function(e,t,n){return n&&(t=ce(n,t)),e[t]},ft=function(e,t,n,r){n&&(t=ce(n,t)),e[t]=r},pt="undefined"!=typeof Map?Map:function(){return i((function e(){a(this,e),this._obj={}}),[{key:"set",value:function(e,t){return this._obj[e]=t,this}},{key:"delete",value:function(e){return this._obj[e]=void 0,this}},{key:"clear",value:function(){this._obj={}}},{key:"has",value:function(e){return void 0!==this._obj[e]}},{key:"get",value:function(e){return this._obj[e]}}])}(),gt=function(){return i((function e(t){if(a(this,e),this._obj=Object.create(null),this.size=0,null!=t){var n;n=null!=t.instanceString&&t.instanceString()===this.instanceString()?t.toArray():t;for(var r=0;r2&&void 0!==arguments[2])||arguments[2];if(void 0!==e&&void 0!==t&&re(e)){var r=t.group;if(null==r&&(r=t.data&&null!=t.data.source&&null!=t.data.target?"edges":"nodes"),"nodes"===r||"edges"===r){this.length=1,this[0]=this;var a=this._private={cy:e,single:!0,data:t.data||{},position:t.position||{x:0,y:0},autoWidth:void 0,autoHeight:void 0,autoPadding:void 0,compoundBoundsClean:!1,listeners:[],group:r,style:{},rstyle:{},styleCxts:[],styleKeys:{},removed:!0,selected:!!t.selected,selectable:void 0===t.selectable||!!t.selectable,locked:!!t.locked,grabbed:!1,grabbable:void 0===t.grabbable||!!t.grabbable,pannable:void 0===t.pannable?"edges"===r:!!t.pannable,active:!1,classes:new vt,animation:{current:[],queue:[]},rscratch:{},scratch:t.scratch||{},edges:[],children:[],parent:t.parent&&t.parent.isNode()?t.parent:null,traversalCache:{},backgrounding:!1,bbCache:null,bbCacheShift:{x:0,y:0},bodyBounds:null,overlayBounds:null,labelBounds:{all:null,source:null,target:null,main:null},arrowBounds:{source:null,target:null,"mid-source":null,"mid-target":null}};if(null==a.position.x&&(a.position.x=0),null==a.position.y&&(a.position.y=0),t.renderedPosition){var i=t.renderedPosition,o=e.pan(),s=e.zoom();a.position={x:(i.x-o.x)/s,y:(i.y-o.y)/s}}var l=[];Z(t.classes)?l=t.classes:H(t.classes)&&(l=t.classes.split(/\s+/));for(var u=0,c=l.length;ut?1:0},u=function(e,t,a,i,o){var s;if(null==a&&(a=0),null==o&&(o=n),a<0)throw new Error("lo must be non-negative");for(null==i&&(i=e.length);an;0<=n?t++:t--)u.push(t);return u}.apply(this).reverse()).length;ig;0<=g?++h:--h)v.push(i(e,r));return v},p=function(e,t,r,a){var i,o,s;for(null==a&&(a=n),i=e[r];r>t&&a(i,o=e[s=r-1>>1])<0;)e[r]=o,r=s;return e[r]=i},g=function(e,t,r){var a,i,o,s,l;for(null==r&&(r=n),i=e.length,l=t,o=e[t],a=2*t+1;a0;){var w=y.pop(),E=g(w),k=w.id();if(d[k]=E,E!==1/0)for(var T=w.neighborhood().intersect(f),C=0;C0)for(n.unshift(t);c[a];){var i=c[a];n.unshift(i.edge),n.unshift(i.node),a=(r=i.node).id()}return o.spawn(n)}}}},Dt={kruskal:function(e){e=e||function(e){return 1};for(var t=this.byGroup(),n=t.nodes,r=t.edges,a=n.length,i=new Array(a),o=n,s=function(e){for(var t=0;t0;){if(l=g.pop(),u=l.id(),v.delete(u),w++,u===d){for(var E=[],k=a,T=d,C=m[T];E.unshift(k),null!=C&&E.unshift(C),null!=(k=y[T]);)C=m[T=k.id()];return{found:!0,distance:h[u],path:this.spawn(E),steps:w}}p[u]=!0;for(var P=l._private.edges,S=0;SP&&(f[C]=P,y[C]=T,m[C]=x),!a){var S=T*u+k;!a&&f[S]>P&&(f[S]=P,y[S]=k,m[S]=x)}}}for(var B=0;B1&&void 0!==arguments[1]?arguments[1]:i,r=[],a=m(e);;){if(null==a)return t.spawn();var o=y(a),l=o.edge,u=o.pred;if(r.unshift(a[0]),a.same(n)&&r.length>0)break;null!=l&&r.unshift(l),a=u}return s.spawn(r)},hasNegativeWeightCycle:p,negativeWeightCycles:g}}},Lt=Math.sqrt(2),zt=function(e,t,n){0===n.length&&nt("Karger-Stein must be run on a connected (sub)graph");for(var r=n[e],a=r[1],i=r[2],o=t[a],s=t[i],l=n,u=l.length-1;u>=0;u--){var c=l[u],d=c[1],h=c[2];(t[d]===o&&t[h]===s||t[d]===s&&t[h]===o)&&l.splice(u,1)}for(var f=0;fr;){var a=Math.floor(Math.random()*t.length);t=zt(a,e,t),n--}return t},Vt={kargerStein:function(){var e=this,t=this.byGroup(),n=t.nodes,r=t.edges;r.unmergeBy((function(e){return e.isLoop()}));var a=n.length,i=r.length,o=Math.ceil(Math.pow(Math.log(a)/Math.LN2,2)),s=Math.floor(a/Lt);if(!(a<2)){for(var l=[],u=0;u0?1:e<0?-1:0},Kt=function(e,t){return Math.sqrt(Ut(e,t))},Ut=function(e,t){var n=t.x-e.x,r=t.y-e.y;return n*n+r*r},Ht=function(e){for(var t=e.length,n=0,r=0;r=e.x1&&e.y2>=e.y1)return{x1:e.x1,y1:e.y1,x2:e.x2,y2:e.y2,w:e.x2-e.x1,h:e.y2-e.y1};if(null!=e.w&&null!=e.h&&e.w>=0&&e.h>=0)return{x1:e.x1,y1:e.y1,x2:e.x1+e.w,y2:e.y1+e.h,w:e.w,h:e.h}}},Jt=function(e,t){e.x1=Math.min(e.x1,t.x1),e.x2=Math.max(e.x2,t.x2),e.w=e.x2-e.x1,e.y1=Math.min(e.y1,t.y1),e.y2=Math.max(e.y2,t.y2),e.h=e.y2-e.y1},en=function(e,t,n){e.x1=Math.min(e.x1,t),e.x2=Math.max(e.x2,t),e.w=e.x2-e.x1,e.y1=Math.min(e.y1,n),e.y2=Math.max(e.y2,n),e.h=e.y2-e.y1},tn=function(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:0;return e.x1-=t,e.x2+=t,e.y1-=t,e.y2+=t,e.w=e.x2-e.x1,e.h=e.y2-e.y1,e},nn=function(e){var t,n,r,a,i=arguments.length>1&&void 0!==arguments[1]?arguments[1]:[0];if(1===i.length)t=n=r=a=i[0];else if(2===i.length)t=r=i[0],a=n=i[1];else if(4===i.length){var o=l(i,4);t=o[0],n=o[1],r=o[2],a=o[3]}return e.x1-=a,e.x2+=n,e.y1-=t,e.y2+=r,e.w=e.x2-e.x1,e.h=e.y2-e.y1,e},rn=function(e,t){e.x1=t.x1,e.y1=t.y1,e.x2=t.x2,e.y2=t.y2,e.w=e.x2-e.x1,e.h=e.y2-e.y1},an=function(e,t){return!(e.x1>t.x2)&&(!(t.x1>e.x2)&&(!(e.x2t.y2)&&!(t.y1>e.y2)))))))},on=function(e,t,n){return e.x1<=t&&t<=e.x2&&e.y1<=n&&n<=e.y2},sn=function(e,t){return on(e,t.x1,t.y1)&&on(e,t.x2,t.y2)},ln=function(e,t,n,r,a,i,o){var s,l,u=arguments.length>7&&void 0!==arguments[7]?arguments[7]:"auto",c="auto"===u?Pn(a,i):u,d=a/2,h=i/2,f=(c=Math.min(c,d,h))!==d,p=c!==h;if(f){var g=r-h-o;if((s=xn(e,t,n,r,n-d+c-o,g,n+d-c+o,g,!1)).length>0)return s}if(p){var v=n+d+o;if((s=xn(e,t,n,r,v,r-h+c-o,v,r+h-c+o,!1)).length>0)return s}if(f){var y=r+h+o;if((s=xn(e,t,n,r,n-d+c-o,y,n+d-c+o,y,!1)).length>0)return s}if(p){var m=n-d-o;if((s=xn(e,t,n,r,m,r-h+c-o,m,r+h-c+o,!1)).length>0)return s}var b=n-d+c,x=r-h+c;if((l=mn(e,t,n,r,b,x,c+o)).length>0&&l[0]<=b&&l[1]<=x)return[l[0],l[1]];var w=n+d-c,E=r-h+c;if((l=mn(e,t,n,r,w,E,c+o)).length>0&&l[0]>=w&&l[1]<=E)return[l[0],l[1]];var k=n+d-c,T=r+h-c;if((l=mn(e,t,n,r,k,T,c+o)).length>0&&l[0]>=k&&l[1]>=T)return[l[0],l[1]];var C=n-d+c,P=r+h-c;return(l=mn(e,t,n,r,C,P,c+o)).length>0&&l[0]<=C&&l[1]>=P?[l[0],l[1]]:[]},un=function(e,t,n,r,a,i,o){var s=o,l=Math.min(n,a),u=Math.max(n,a),c=Math.min(r,i),d=Math.max(r,i);return l-s<=e&&e<=u+s&&c-s<=t&&t<=d+s},cn=function(e,t,n,r,a,i,o,s,l){var u=Math.min(n,o,a)-l,c=Math.max(n,o,a)+l,d=Math.min(r,s,i)-l,h=Math.max(r,s,i)+l;return!(ec||th)},dn=function(e,t,n,r,a,i,o,s){var l=[];!function(e,t,n,r,a){var i,o,s,l,u,c,d,h;0===e&&(e=1e-5),s=-27*(r/=e)+(t/=e)*(9*(n/=e)-t*t*2),i=(o=(3*n-t*t)/9)*o*o+(s/=54)*s,a[1]=0,d=t/3,i>0?(u=(u=s+Math.sqrt(i))<0?-Math.pow(-u,1/3):Math.pow(u,1/3),c=(c=s-Math.sqrt(i))<0?-Math.pow(-c,1/3):Math.pow(c,1/3),a[0]=-d+u+c,d+=(u+c)/2,a[4]=a[2]=-d,d=Math.sqrt(3)*(-c+u)/2,a[3]=d,a[5]=-d):(a[5]=a[3]=0,0===i?(h=s<0?-Math.pow(-s,1/3):Math.pow(s,1/3),a[0]=2*h-d,a[4]=a[2]=-(h+d)):(l=(o=-o)*o*o,l=Math.acos(s/Math.sqrt(l)),h=2*Math.sqrt(o),a[0]=-d+h*Math.cos(l/3),a[2]=-d+h*Math.cos((l+2*Math.PI)/3),a[4]=-d+h*Math.cos((l+4*Math.PI)/3)))}(1*n*n-4*n*a+2*n*o+4*a*a-4*a*o+o*o+r*r-4*r*i+2*r*s+4*i*i-4*i*s+s*s,9*n*a-3*n*n-3*n*o-6*a*a+3*a*o+9*r*i-3*r*r-3*r*s-6*i*i+3*i*s,3*n*n-6*n*a+n*o-n*e+2*a*a+2*a*e-o*e+3*r*r-6*r*i+r*s-r*t+2*i*i+2*i*t-s*t,1*n*a-n*n+n*e-a*e+r*i-r*r+r*t-i*t,l);for(var u=[],c=0;c<6;c+=2)Math.abs(l[c+1])<1e-7&&l[c]>=0&&l[c]<=1&&u.push(l[c]);u.push(1),u.push(0);for(var d,h,f,p=-1,g=0;g=0?fl?(e-a)*(e-a)+(t-i)*(t-i):u-d},fn=function(e,t,n){for(var r,a,i,o,s=0,l=0;l=e&&e>=i||r<=e&&e<=i))continue;(e-r)/(i-r)*(o-a)+a>t&&s++}return s%2!=0},pn=function(e,t,n,r,a,i,o,s,l){var u,c=new Array(n.length);null!=s[0]?(u=Math.atan(s[1]/s[0]),s[0]<0?u+=Math.PI/2:u=-u-Math.PI/2):u=s;for(var d,h=Math.cos(-u),f=Math.sin(-u),p=0;p0){var g=vn(c,-l);d=gn(g)}else d=c;return fn(e,t,d)},gn=function(e){for(var t,n,r,a,i,o,s,l,u=new Array(e.length/2),c=0;c=0&&p<=1&&v.push(p),g>=0&&g<=1&&v.push(g),0===v.length)return[];var y=v[0]*s[0]+e,m=v[0]*s[1]+t;return v.length>1?v[0]==v[1]?[y,m]:[y,m,v[1]*s[0]+e,v[1]*s[1]+t]:[y,m]},bn=function(e,t,n){return t<=e&&e<=n||n<=e&&e<=t?e:e<=t&&t<=n||n<=t&&t<=e?t:n},xn=function(e,t,n,r,a,i,o,s,l){var u=e-a,c=n-e,d=o-a,h=t-i,f=r-t,p=s-i,g=d*h-p*u,v=c*h-f*u,y=p*c-d*f;if(0!==y){var m=g/y,b=v/y,x=-.001;return x<=m&&m<=1.001&&x<=b&&b<=1.001||l?[e+m*c,t+m*f]:[]}return 0===g||0===v?bn(e,n,o)===o?[o,s]:bn(e,n,a)===a?[a,i]:bn(a,o,n)===n?[n,r]:[]:[]},wn=function(e,t,n,r,a,i,o,s){var l,u,c,d,h,f,p=[],g=new Array(n.length),v=!0;if(null==i&&(v=!1),v){for(var y=0;y0){var m=vn(g,-s);u=gn(m)}else u=g}else u=n;for(var b=0;bu&&(u=t)},d=function(e){return l[e]},h=0;h0?b.edgesTo(m)[0]:m.edgesTo(b)[0];var w=r(x);m=m.id(),u[m]>u[g]+w&&(u[m]=u[g]+w,h.nodes.indexOf(m)<0?h.push(m):h.updateItem(m),l[m]=0,n[m]=[]),u[m]==u[g]+w&&(l[m]=l[m]+l[g],n[m].push(g))}else for(var E=0;E0;){for(var P=t.pop(),S=0;S0&&o.push(n[s]);0!==o.length&&a.push(r.collection(o))}return a}(c,l,t,r);return b=function(e){for(var t=0;t5&&void 0!==arguments[5]?arguments[5]:Un,o=r,s=0;s=2?Jn(e,t,n,0,Zn,$n):Jn(e,t,n,0,Gn)},squaredEuclidean:function(e,t,n){return Jn(e,t,n,0,Zn)},manhattan:function(e,t,n){return Jn(e,t,n,0,Gn)},max:function(e,t,n){return Jn(e,t,n,-1/0,Qn)}};function tr(e,t,n,r,a,i){var o;return o=G(e)?e:er[e]||er.euclidean,0===t&&G(e)?o(a,i):o(t,n,r,a,i)}er["squared-euclidean"]=er.squaredEuclidean,er.squaredeuclidean=er.squaredEuclidean;var nr=ut({k:2,m:2,sensitivityThreshold:1e-4,distance:"euclidean",maxIterations:10,attributes:[],testMode:!1,testCentroids:null}),rr=function(e){return nr(e)},ar=function(e,t,n,r,a){var i="kMedoids"!==a?function(e){return n[e]}:function(e){return r[e](n)},o=n,s=t;return tr(e,r.length,i,(function(e){return r[e](t)}),o,s)},ir=function(e,t,n){for(var r=n.length,a=new Array(r),i=new Array(r),o=new Array(t),s=null,l=0;ln)return!1}return!0},cr=function(e,t,n){for(var r=0;ra&&(a=t[l][u],i=u);o[i].push(e[l])}for(var c=0;c=a.threshold||"dendrogram"===a.mode&&1===e.length)return!1;var f,p=t[o],g=t[r[o]];f="dendrogram"===a.mode?{left:p,right:g,key:p.key}:{value:p.value.concat(g.value),key:p.key},e[p.index]=f,e.splice(g.index,1),t[p.key]=f;for(var v=0;vn[g.key][y.key]&&(i=n[g.key][y.key])):"max"===a.linkage?(i=n[p.key][y.key],n[p.key][y.key]1&&void 0!==arguments[1]?arguments[1]:0,n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:e.length,r=!(arguments.length>4&&void 0!==arguments[4])||arguments[4],a=!(arguments.length>5&&void 0!==arguments[5])||arguments[5];arguments.length>3&&void 0!==arguments[3]&&!arguments[3]?(n0&&e.splice(0,t)):e=e.slice(t,n);for(var i=0,o=e.length-1;o>=0;o--){var s=e[o];a?isFinite(s)||(e[o]=-1/0,i++):e.splice(o,1)}r&&e.sort((function(e,t){return e-t}));var l=e.length,u=Math.floor(l/2);return l%2!=0?e[u+1+i]:(e[u-1+i]+e[u+i])/2}(e):"mean"===t?function(e){for(var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:0,n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:e.length,r=0,a=0,i=t;i1&&void 0!==arguments[1]?arguments[1]:0,n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:e.length,r=1/0,a=t;a1&&void 0!==arguments[1]?arguments[1]:0,n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:e.length,r=-1/0,a=t;ao&&(i=l,o=t[a*e+l])}i>0&&r.push(i)}for(var u=0;u=P?(S=P,P=D,B=_):D>S&&(S=D);for(var A=0;A0?1:0;k[E%u.minIterations*t+z]=O,L+=O}if(L>0&&(E>=u.minIterations-1||E==u.maxIterations-1)){for(var V=0,F=0;F0&&r.push(a);return r}(t,i,o),Y=function(e,t,n){for(var r=Br(e,t,n),a=0;al&&(s=u,l=c)}n[a]=i[s]}return Br(e,t,n)}(t,r,X),q={},W=0;W1)}}));var l=Object.keys(t).filter((function(e){return t[e].cutVertex})).map((function(t){return e.getElementById(t)}));return{cut:e.spawn(l),components:a}},Rr=function(){var e=this,t={},n=0,r=[],a=[],i=e.spawn(e),o=function(s){if(a.push(s),t[s]={index:n,low:n++,explored:!1},e.getElementById(s).connectedEdges().intersection(e).forEach((function(e){var n=e.target().id();n!==s&&(n in t||o(n),t[n].explored||(t[s].low=Math.min(t[s].low,t[n].low)))})),t[s].index===t[s].low){for(var l=e.spawn();;){var u=a.pop();if(l.merge(e.getElementById(u)),t[u].low=t[s].index,t[u].explored=!0,u===s)break}var c=l.edgesWith(l),d=l.merge(c);r.push(d),i=i.difference(d)}};return e.forEach((function(e){if(e.isNode()){var n=e.id();n in t||o(n)}})),{cut:i,components:r}},Ir={};[bt,Bt,Dt,At,Rt,Nt,Vt,_n,Mn,In,Ln,Kn,vr,Tr,_r,{hierholzer:function(e){if(!$(e)){var t=arguments;e={root:t[0],directed:t[1]}}var n,r,a,i=Ar(e),o=i.root,s=i.directed,l=this,u=!1;o&&(a=H(o)?this.filter(o)[0].id():o[0].id());var c={},d={};s?l.forEach((function(e){var t=e.id();if(e.isNode()){var a=e.indegree(!0),i=e.outdegree(!0),o=a-i,s=i-a;1==o?n?u=!0:n=t:1==s?r?u=!0:r=t:(s>1||o>1)&&(u=!0),c[t]=[],e.outgoers().forEach((function(e){e.isEdge()&&c[t].push(e.id())}))}else d[t]=[void 0,e.target().id()]})):l.forEach((function(e){var t=e.id();e.isNode()?(e.degree(!0)%2&&(n?r?u=!0:r=t:n=t),c[t]=[],e.connectedEdges().forEach((function(e){return c[t].push(e.id())}))):d[t]=[e.source().id(),e.target().id()]}));var h={found:!1,trail:void 0};if(u)return h;if(r&&n)if(s){if(a&&r!=a)return h;a=r}else{if(a&&r!=a&&n!=a)return h;a||(a=r)}else a||(a=l[0].id());var f=function(e){for(var t,n,r,a=e,i=[e];c[a].length;)t=c[a].shift(),n=d[t][0],a!=(r=d[t][1])?(c[r]=c[r].filter((function(e){return e!=t})),a=r):s||a==n||(c[n]=c[n].filter((function(e){return e!=t})),a=n),i.unshift(t),i.unshift(a);return i},p=[],g=[];for(g=f(a);1!=g.length;)0==c[g[0]].length?(p.unshift(l.getElementById(g.shift())),p.unshift(l.getElementById(g.shift()))):g=f(g.shift()).concat(g);for(var v in p.unshift(l.getElementById(g.shift())),c)if(c[v].length)return h;return h.found=!0,h.trail=this.spawn(p,!0),h}},{hopcroftTarjanBiconnected:Mr,htbc:Mr,htb:Mr,hopcroftTarjanBiconnectedComponents:Mr},{tarjanStronglyConnected:Rr,tsc:Rr,tscc:Rr,tarjanStronglyConnectedComponents:Rr}].forEach((function(e){me(Ir,e)}));var Nr=function(e){if(!(this instanceof Nr))return new Nr(e);this.id="Thenable/1.0.7",this.state=0,this.fulfillValue=void 0,this.rejectReason=void 0,this.onFulfilled=[],this.onRejected=[],this.proxy={then:this.then.bind(this)},"function"==typeof e&&e.call(this,this.fulfill.bind(this),this.reject.bind(this))};Nr.prototype={fulfill:function(e){return Lr(this,1,"fulfillValue",e)},reject:function(e){return Lr(this,2,"rejectReason",e)},then:function(e,t){var n=this,r=new Nr;return n.onFulfilled.push(Vr(e,r,"fulfill")),n.onRejected.push(Vr(t,r,"reject")),zr(n),r.proxy}};var Lr=function(e,t,n,r){return 0===e.state&&(e.state=t,e[n]=r,zr(e)),e},zr=function(e){1===e.state?Or(e,"onFulfilled",e.fulfillValue):2===e.state&&Or(e,"onRejected",e.rejectReason)},Or=function(e,t,n){if(0!==e[t].length){var r=e[t];e[t]=[];var a=function(){for(var e=0;e0:void 0}},clearQueue:function(){return function(){var e=this,t=void 0!==e.length?e:[e];if(!(this._private.cy||this).styleEnabled())return this;for(var n=0;n-1}}(),a=function(){if(Na)return Ia;Na=1;var e=_i();return Ia=function(t,n){var r=this.__data__,a=e(r,t);return a<0?(++this.size,r.push([t,n])):r[a][1]=n,this},Ia}();function i(e){var t=-1,n=null==e?0:e.length;for(this.clear();++t-1&&t%1==0&&t0&&this.spawn(r).updateStyle().emit("class"),t},addClass:function(e){return this.toggleClass(e,!0)},hasClass:function(e){var t=this[0];return null!=t&&t._private.classes.has(e)},toggleClass:function(e,t){Z(e)||(e=e.match(/\S+/g)||[]);for(var n=this,r=void 0===t,a=[],i=0,o=n.length;i0&&this.spawn(a).updateStyle().emit("class"),n},removeClass:function(e){return this.toggleClass(e,!1)},flashClass:function(e,t){var n=this;if(null==t)t=250;else if(0===t)return n;return n.addClass(e),setTimeout((function(){n.removeClass(e)}),t),n}};vo.className=vo.classNames=vo.classes;var yo={metaChar:"[\\!\\\"\\#\\$\\%\\&\\'\\(\\)\\*\\+\\,\\.\\/\\:\\;\\<\\=\\>\\?\\@\\[\\]\\^\\`\\{\\|\\}\\~]",comparatorOp:"=|\\!=|>|>=|<|<=|\\$=|\\^=|\\*=",boolOp:"\\?|\\!|\\^",string:"\"(?:\\\\\"|[^\"])*\"|'(?:\\\\'|[^'])*'",number:he,meta:"degree|indegree|outdegree",separator:"\\s*,\\s*",descendant:"\\s+",child:"\\s+>\\s+",subject:"\\$",group:"node|edge|\\*",directedEdge:"\\s+->\\s+",undirectedEdge:"\\s+<->\\s+"};yo.variable="(?:[\\w-.]|(?:\\\\"+yo.metaChar+"))+",yo.className="(?:[\\w-]|(?:\\\\"+yo.metaChar+"))+",yo.value=yo.string+"|"+yo.number,yo.id=yo.variable,function(){var e,t,n;for(e=yo.comparatorOp.split("|"),n=0;n=0||"="!==t&&(yo.comparatorOp+="|\\!"+t)}();var mo=0,bo=1,xo=2,wo=3,Eo=4,ko=5,To=6,Co=7,Po=8,So=9,Bo=10,Do=11,_o=12,Ao=13,Mo=14,Ro=15,Io=16,No=17,Lo=18,zo=19,Oo=20,Vo=[{selector:":selected",matches:function(e){return e.selected()}},{selector:":unselected",matches:function(e){return!e.selected()}},{selector:":selectable",matches:function(e){return e.selectable()}},{selector:":unselectable",matches:function(e){return!e.selectable()}},{selector:":locked",matches:function(e){return e.locked()}},{selector:":unlocked",matches:function(e){return!e.locked()}},{selector:":visible",matches:function(e){return e.visible()}},{selector:":hidden",matches:function(e){return!e.visible()}},{selector:":transparent",matches:function(e){return e.transparent()}},{selector:":grabbed",matches:function(e){return e.grabbed()}},{selector:":free",matches:function(e){return!e.grabbed()}},{selector:":removed",matches:function(e){return e.removed()}},{selector:":inside",matches:function(e){return!e.removed()}},{selector:":grabbable",matches:function(e){return e.grabbable()}},{selector:":ungrabbable",matches:function(e){return!e.grabbable()}},{selector:":animated",matches:function(e){return e.animated()}},{selector:":unanimated",matches:function(e){return!e.animated()}},{selector:":parent",matches:function(e){return e.isParent()}},{selector:":childless",matches:function(e){return e.isChildless()}},{selector:":child",matches:function(e){return e.isChild()}},{selector:":orphan",matches:function(e){return e.isOrphan()}},{selector:":nonorphan",matches:function(e){return e.isChild()}},{selector:":compound",matches:function(e){return e.isNode()?e.isParent():e.source().isParent()||e.target().isParent()}},{selector:":loop",matches:function(e){return e.isLoop()}},{selector:":simple",matches:function(e){return e.isSimple()}},{selector:":active",matches:function(e){return e.active()}},{selector:":inactive",matches:function(e){return!e.active()}},{selector:":backgrounding",matches:function(e){return e.backgrounding()}},{selector:":nonbackgrounding",matches:function(e){return!e.backgrounding()}}].sort((function(e,t){return function(e,t){return-1*ye(e,t)}(e.selector,t.selector)})),Fo=function(){for(var e,t={},n=0;n0&&u.edgeCount>0)return at("The selector `"+e+"` is invalid because it uses both a compound selector and an edge selector"),!1;if(u.edgeCount>1)return at("The selector `"+e+"` is invalid because it uses multiple edge selectors"),!1;1===u.edgeCount&&at("The selector `"+e+"` is deprecated. Edge selectors do not take effect on changes to source and target nodes after an edge is added, for performance reasons. Use a class or data selector on edges instead, updating the class or data of an edge when your app detects a change in source or target nodes.")}return!0},toString:function(){if(null!=this.toStringCache)return this.toStringCache;for(var e=function(e){return null==e?"":e},t=function(t){return H(t)?'"'+t+'"':e(t)},n=function(e){return" "+e+" "},r=function(r,i){var o=r.type,s=r.value;switch(o){case mo:var l=e(s);return l.substring(0,l.length-1);case wo:var u=r.field,c=r.operator;return"["+u+n(e(c))+t(s)+"]";case ko:var d=r.operator,h=r.field;return"["+e(d)+h+"]";case Eo:return"["+r.field+"]";case To:var f=r.operator;return"[["+r.field+n(e(f))+t(s)+"]]";case Co:return s;case Po:return"#"+s;case So:return"."+s;case No:case Ro:return a(r.parent,i)+n(">")+a(r.child,i);case Lo:case Io:return a(r.ancestor,i)+" "+a(r.descendant,i);case zo:var p=a(r.left,i),g=a(r.subject,i),v=a(r.right,i);return p+(p.length>0?" ":"")+g+v;case Oo:return""}},a=function(e,t){return e.checks.reduce((function(n,a,i){return n+(t===e&&0===i?"$":"")+r(a,t)}),"")},i="",o=0;o1&&o=0&&(t=t.replace("!",""),c=!0),t.indexOf("@")>=0&&(t=t.replace("@",""),u=!0),(o||l||u)&&(a=o||s?""+e:"",i=""+n),u&&(e=a=a.toLowerCase(),n=i=i.toLowerCase()),t){case"*=":r=a.indexOf(i)>=0;break;case"$=":r=a.indexOf(i,a.length-i.length)>=0;break;case"^=":r=0===a.indexOf(i);break;case"=":r=e===n;break;case">":d=!0,r=e>n;break;case">=":d=!0,r=e>=n;break;case"<":d=!0,r=e0;){var u=a.shift();t(u),i.add(u.id()),o&&r(a,i,u)}return e}function os(e,t,n){if(n.isParent())for(var r=n._private.children,a=0;a1&&void 0!==arguments[1])||arguments[1],os)},as.forEachUp=function(e){return is(this,e,!(arguments.length>1&&void 0!==arguments[1])||arguments[1],ss)},as.forEachUpAndDown=function(e){return is(this,e,!(arguments.length>1&&void 0!==arguments[1])||arguments[1],ls)},as.ancestors=as.parents,(ts=ns={data:po.data({field:"data",bindingEvent:"data",allowBinding:!0,allowSetting:!0,settingEvent:"data",settingTriggersEvent:!0,triggerFnName:"trigger",allowGetting:!0,immutableKeys:{id:!0,source:!0,target:!0,parent:!0},updateStyle:!0}),removeData:po.removeData({field:"data",event:"data",triggerFnName:"trigger",triggerEvent:!0,immutableKeys:{id:!0,source:!0,target:!0,parent:!0},updateStyle:!0}),scratch:po.data({field:"scratch",bindingEvent:"scratch",allowBinding:!0,allowSetting:!0,settingEvent:"scratch",settingTriggersEvent:!0,triggerFnName:"trigger",allowGetting:!0,updateStyle:!0}),removeScratch:po.removeData({field:"scratch",event:"scratch",triggerFnName:"trigger",triggerEvent:!0,updateStyle:!0}),rscratch:po.data({field:"rscratch",allowBinding:!1,allowSetting:!0,settingTriggersEvent:!1,allowGetting:!0}),removeRscratch:po.removeData({field:"rscratch",triggerEvent:!1}),id:function(){var e=this[0];if(e)return e._private.data.id}}).attr=ts.data,ts.removeAttr=ts.removeData;var us,cs,ds=ns,hs={};function fs(e){return function(t){var n=this;if(void 0===t&&(t=!0),0!==n.length&&n.isNode()&&!n.removed()){for(var r=0,a=n[0],i=a._private.edges,o=0;ot})),minIndegree:ps("indegree",(function(e,t){return et})),minOutdegree:ps("outdegree",(function(e,t){return et}))}),me(hs,{totalDegree:function(e){for(var t=0,n=this.nodes(),r=0;r0,c=u;u&&(l=l[0]);var d=c?l.position():{x:0,y:0};return a={x:s.x-d.x,y:s.y-d.y},void 0===e?a:a[e]}for(var h=0;h0,v=g;g&&(p=p[0]);var y=v?p.position():{x:0,y:0};void 0!==t?f.position(e,t+y[e]):void 0!==a&&f.position({x:a.x+y.x,y:a.y+y.y})}}else if(!i)return;return this}},us.modelPosition=us.point=us.position,us.modelPositions=us.points=us.positions,us.renderedPoint=us.renderedPosition,us.relativePoint=us.relativePosition;var ys,ms,bs=cs;ys=ms={},ms.renderedBoundingBox=function(e){var t=this.boundingBox(e),n=this.cy(),r=n.zoom(),a=n.pan(),i=t.x1*r+a.x,o=t.x2*r+a.x,s=t.y1*r+a.y,l=t.y2*r+a.y;return{x1:i,x2:o,y1:s,y2:l,w:o-i,h:l-s}},ms.dirtyCompoundBoundsCache=function(){var e=arguments.length>0&&void 0!==arguments[0]&&arguments[0],t=this.cy();return t.styleEnabled()&&t.hasCompoundNodes()?(this.forEachUp((function(t){if(t.isParent()){var n=t._private;n.compoundBoundsClean=!1,n.bbCache=null,e||t.emitAndNotify("bounds")}})),this):this},ms.updateCompoundBounds=function(){var e=arguments.length>0&&void 0!==arguments[0]&&arguments[0],t=this.cy();if(!t.styleEnabled()||!t.hasCompoundNodes())return this;if(!e&&t.batching())return this;function n(e){if(e.isParent()){var t=e._private,n=e.children(),r="include"===e.pstyle("compound-sizing-wrt-labels").value,a={width:{val:e.pstyle("min-width").pfValue,left:e.pstyle("min-width-bias-left"),right:e.pstyle("min-width-bias-right")},height:{val:e.pstyle("min-height").pfValue,top:e.pstyle("min-height-bias-top"),bottom:e.pstyle("min-height-bias-bottom")}},i=n.boundingBox({includeLabels:r,includeOverlays:!1,useCache:!1}),o=t.position;0!==i.w&&0!==i.h||((i={w:e.pstyle("width").pfValue,h:e.pstyle("height").pfValue}).x1=o.x-i.w/2,i.x2=o.x+i.w/2,i.y1=o.y-i.h/2,i.y2=o.y+i.h/2);var s=a.width.left.value;"px"===a.width.left.units&&a.width.val>0&&(s=100*s/a.width.val);var l=a.width.right.value;"px"===a.width.right.units&&a.width.val>0&&(l=100*l/a.width.val);var u=a.height.top.value;"px"===a.height.top.units&&a.height.val>0&&(u=100*u/a.height.val);var c=a.height.bottom.value;"px"===a.height.bottom.units&&a.height.val>0&&(c=100*c/a.height.val);var d=y(a.width.val-i.w,s,l),h=d.biasDiff,f=d.biasComplementDiff,p=y(a.height.val-i.h,u,c),g=p.biasDiff,v=p.biasComplementDiff;t.autoPadding=function(e,t,n,r){if("%"!==n.units)return"px"===n.units?n.pfValue:0;switch(r){case"width":return e>0?n.pfValue*e:0;case"height":return t>0?n.pfValue*t:0;case"average":return e>0&&t>0?n.pfValue*(e+t)/2:0;case"min":return e>0&&t>0?e>t?n.pfValue*t:n.pfValue*e:0;case"max":return e>0&&t>0?e>t?n.pfValue*e:n.pfValue*t:0;default:return 0}}(i.w,i.h,e.pstyle("padding"),e.pstyle("padding-relative-to").value),t.autoWidth=Math.max(i.w,a.width.val),o.x=(-h+i.x1+i.x2+f)/2,t.autoHeight=Math.max(i.h,a.height.val),o.y=(-g+i.y1+i.y2+v)/2}function y(e,t,n){var r=0,a=0,i=t+n;return e>0&&i>0&&(r=t/i*e,a=n/i*e),{biasDiff:r,biasComplementDiff:a}}}for(var r=0;re.x2?r:e.x2,e.y1=ne.y2?a:e.y2,e.w=e.x2-e.x1,e.h=e.y2-e.y1)},Es=function(e,t){return null==t?e:ws(e,t.x1,t.y1,t.x2,t.y2)},ks=function(e,t,n){return ht(e,t,n)},Ts=function(e,t,n){if(!t.cy().headless()){var r,a,i=t._private,o=i.rstyle,s=o.arrowWidth/2;if("none"!==t.pstyle(n+"-arrow-shape").value){"source"===n?(r=o.srcX,a=o.srcY):"target"===n?(r=o.tgtX,a=o.tgtY):(r=o.midX,a=o.midY);var l=i.arrowBounds=i.arrowBounds||{},u=l[n]=l[n]||{};u.x1=r-s,u.y1=a-s,u.x2=r+s,u.y2=a+s,u.w=u.x2-u.x1,u.h=u.y2-u.y1,tn(u,1),ws(e,u.x1,u.y1,u.x2,u.y2)}}},Cs=function(e,t,n){if(!t.cy().headless()){var r;r=n?n+"-":"";var a=t._private,i=a.rstyle;if(t.pstyle(r+"label").strValue){var o,s,l,u,c=t.pstyle("text-halign"),d=t.pstyle("text-valign"),h=ks(i,"labelWidth",n),f=ks(i,"labelHeight",n),p=ks(i,"labelX",n),g=ks(i,"labelY",n),v=t.pstyle(r+"text-margin-x").pfValue,y=t.pstyle(r+"text-margin-y").pfValue,m=t.isEdge(),b=t.pstyle(r+"text-rotation"),x=t.pstyle("text-outline-width").pfValue,w=t.pstyle("text-border-width").pfValue/2,E=t.pstyle("text-background-padding").pfValue,k=f,T=h,C=T/2,P=k/2;if(m)o=p-C,s=p+C,l=g-P,u=g+P;else{switch(c.value){case"left":o=p-T,s=p;break;case"center":o=p-C,s=p+C;break;case"right":o=p,s=p+T}switch(d.value){case"top":l=g-k,u=g;break;case"center":l=g-P,u=g+P;break;case"bottom":l=g,u=g+k}}var S=v-Math.max(x,w)-E-2,B=v+Math.max(x,w)+E+2,D=y-Math.max(x,w)-E-2,_=y+Math.max(x,w)+E+2;o+=S,s+=B,l+=D,u+=_;var A=n||"main",M=a.labelBounds,R=M[A]=M[A]||{};R.x1=o,R.y1=l,R.x2=s,R.y2=u,R.w=s-o,R.h=u-l,R.leftPad=S,R.rightPad=B,R.topPad=D,R.botPad=_;var I=m&&"autorotate"===b.strValue,N=null!=b.pfValue&&0!==b.pfValue;if(I||N){var L=I?ks(a.rstyle,"labelAngle",n):b.pfValue,z=Math.cos(L),O=Math.sin(L),V=(o+s)/2,F=(l+u)/2;if(!m){switch(c.value){case"left":V=s;break;case"right":V=o}switch(d.value){case"top":F=u;break;case"bottom":F=l}}var j=function(e,t){return{x:(e-=V)*z-(t-=F)*O+V,y:e*O+t*z+F}},X=j(o,l),Y=j(o,u),q=j(s,l),W=j(s,u);o=Math.min(X.x,Y.x,q.x,W.x),s=Math.max(X.x,Y.x,q.x,W.x),l=Math.min(X.y,Y.y,q.y,W.y),u=Math.max(X.y,Y.y,q.y,W.y)}var K=A+"Rot",U=M[K]=M[K]||{};U.x1=o,U.y1=l,U.x2=s,U.y2=u,U.w=s-o,U.h=u-l,ws(e,o,l,s,u),ws(a.labelBounds.all,o,l,s,u)}return e}},Ps=function(e,t){var n,r,a,i,o,s,l,u=e._private.cy,c=u.styleEnabled(),d=u.headless(),h=Qt(),f=e._private,p=e.isNode(),g=e.isEdge(),v=f.rstyle,y=p&&c?e.pstyle("bounds-expansion").pfValue:[0],m=function(e){return"none"!==e.pstyle("display").value},b=!c||m(e)&&(!g||m(e.source())&&m(e.target()));if(b){var x=0;c&&t.includeOverlays&&0!==e.pstyle("overlay-opacity").value&&(x=e.pstyle("overlay-padding").value);var w=0;c&&t.includeUnderlays&&0!==e.pstyle("underlay-opacity").value&&(w=e.pstyle("underlay-padding").value);var E=Math.max(x,w),k=0;if(c&&(k=e.pstyle("width").pfValue/2),p&&t.includeNodes){var T=e.position();o=T.x,s=T.y;var C=e.outerWidth()/2,P=e.outerHeight()/2;ws(h,n=o-C,a=s-P,r=o+C,i=s+P),c&&t.includeOutlines&&function(e,t){if(!t.cy().headless()){var n,r,a,i=t.pstyle("outline-opacity").value,o=t.pstyle("outline-width").value;if(i>0&&o>0){var s=t.pstyle("outline-offset").value,l=t.pstyle("shape").value,u=o+s,c=(e.w+2*u)/e.w,d=(e.h+2*u)/e.h,h=0;["diamond","pentagon","round-triangle"].includes(l)?(c=(e.w+2.4*u)/e.w,h=-u/3.6):["concave-hexagon","rhomboid","right-rhomboid"].includes(l)?c=(e.w+2.4*u)/e.w:"star"===l?(c=(e.w+2.8*u)/e.w,d=(e.h+2.6*u)/e.h,h=-u/3.8):"triangle"===l?(c=(e.w+2.8*u)/e.w,d=(e.h+2.4*u)/e.h,h=-u/1.4):"vee"===l&&(c=(e.w+4.4*u)/e.w,d=(e.h+3.8*u)/e.h,h=.5*-u);var f=e.h*d-e.h,p=e.w*c-e.w;if(nn(e,[Math.ceil(f/2),Math.ceil(p/2)]),0!==h){var g=(r=0,a=h,{x1:(n=e).x1+r,x2:n.x2+r,y1:n.y1+a,y2:n.y2+a,w:n.w,h:n.h});Jt(e,g)}}}}(h,e)}else if(g&&t.includeEdges)if(c&&!d){var S=e.pstyle("curve-style").strValue;if(n=Math.min(v.srcX,v.midX,v.tgtX),r=Math.max(v.srcX,v.midX,v.tgtX),a=Math.min(v.srcY,v.midY,v.tgtY),i=Math.max(v.srcY,v.midY,v.tgtY),ws(h,n-=k,a-=k,r+=k,i+=k),"haystack"===S){var B=v.haystackPts;if(B&&2===B.length){if(n=B[0].x,a=B[0].y,n>(r=B[1].x)){var D=n;n=r,r=D}if(a>(i=B[1].y)){var _=a;a=i,i=_}ws(h,n-k,a-k,r+k,i+k)}}else if("bezier"===S||"unbundled-bezier"===S||S.endsWith("segments")||S.endsWith("taxi")){var A;switch(S){case"bezier":case"unbundled-bezier":A=v.bezierPts;break;case"segments":case"taxi":case"round-segments":case"round-taxi":A=v.linePts}if(null!=A)for(var M=0;M(r=N.x)){var L=n;n=r,r=L}if((a=I.y)>(i=N.y)){var z=a;a=i,i=z}ws(h,n-=k,a-=k,r+=k,i+=k)}if(c&&t.includeEdges&&g&&(Ts(h,e,"mid-source"),Ts(h,e,"mid-target"),Ts(h,e,"source"),Ts(h,e,"target")),c)if("yes"===e.pstyle("ghost").value){var O=e.pstyle("ghost-offset-x").pfValue,V=e.pstyle("ghost-offset-y").pfValue;ws(h,h.x1+O,h.y1+V,h.x2+O,h.y2+V)}var F=f.bodyBounds=f.bodyBounds||{};rn(F,h),nn(F,y),tn(F,1),c&&(n=h.x1,r=h.x2,a=h.y1,i=h.y2,ws(h,n-E,a-E,r+E,i+E));var j=f.overlayBounds=f.overlayBounds||{};rn(j,h),nn(j,y),tn(j,1);var X=f.labelBounds=f.labelBounds||{};null!=X.all?((l=X.all).x1=1/0,l.y1=1/0,l.x2=-1/0,l.y2=-1/0,l.w=0,l.h=0):X.all=Qt(),c&&t.includeLabels&&(t.includeMainLabels&&Cs(h,e,null),g&&(t.includeSourceLabels&&Cs(h,e,"source"),t.includeTargetLabels&&Cs(h,e,"target")))}return h.x1=xs(h.x1),h.y1=xs(h.y1),h.x2=xs(h.x2),h.y2=xs(h.y2),h.w=xs(h.x2-h.x1),h.h=xs(h.y2-h.y1),h.w>0&&h.h>0&&b&&(nn(h,y),tn(h,1)),h},Ss=function(e){var t=0,n=function(e){return(e?1:0)<0&&void 0!==arguments[0]?arguments[0]:Us,t=arguments.length>1?arguments[1]:void 0,n=0;n=0;s--)o(s);return this},Gs.removeAllListeners=function(){return this.removeListener("*")},Gs.emit=Gs.trigger=function(e,t,n){var r=this.listeners,a=r.length;return this.emitting++,Z(t)||(t=[t]),Qs(this,(function(e,i){null!=n&&(r=[{event:i.event,type:i.type,namespace:i.namespace,callback:n}],a=r.length);for(var o=function(){var n=r[s];if(n.type===i.type&&(!n.namespace||n.namespace===i.namespace||".*"===n.namespace)&&e.eventMatches(e.context,n,i)){var a=[i];null!=t&&function(e,t){for(var n=0;n1&&!r){var a=this.length-1,i=this[a],o=i._private.data.id;this[a]=void 0,this[e]=i,n.set(o,{ele:i,index:e})}return this.length--,this},unmergeOne:function(e){e=e[0];var t=this._private,n=e._private.data.id,r=t.map.get(n);if(!r)return this;var a=r.index;return this.unmergeAt(a),this},unmerge:function(e){var t=this._private.cy;if(!e)return this;if(e&&H(e)){var n=e;e=t.mutableElements().filter(n)}for(var r=0;r=0;t--){e(this[t])&&this.unmergeAt(t)}return this},map:function(e,t){for(var n=[],r=this,a=0;ar&&(r=s,n=o)}return{value:r,ele:n}},min:function(e,t){for(var n,r=1/0,a=this,i=0;i=0&&a1&&void 0!==arguments[1])||arguments[1],n=this[0],r=n.cy();if(r.styleEnabled()&&n){n._private.styleDirty&&(n._private.styleDirty=!1,r.style().apply(n));var a=n._private.style[e];return null!=a?a:t?r.style().getDefaultProperty(e):null}},numericStyle:function(e){var t=this[0];if(t.cy().styleEnabled()&&t){var n=t.pstyle(e);return void 0!==n.pfValue?n.pfValue:n.value}},numericStyleUnits:function(e){var t=this[0];if(t.cy().styleEnabled())return t?t.pstyle(e).units:void 0},renderedStyle:function(e){var t=this.cy();if(!t.styleEnabled())return this;var n=this[0];return n?t.style().getRenderedStyle(n,e):void 0},style:function(e,t){var n=this.cy();if(!n.styleEnabled())return this;var r=!1,a=n.style();if($(e)){var i=e;a.applyBypass(this,i,r),this.emitAndNotify("style")}else if(H(e)){if(void 0===t){var o=this[0];return o?a.getStylePropertyValue(o,e):void 0}a.applyBypass(this,e,t,r),this.emitAndNotify("style")}else if(void 0===e){var s=this[0];return s?a.getRawStyle(s):void 0}return this},removeStyle:function(e){var t=this.cy();if(!t.styleEnabled())return this;var n=!1,r=t.style(),a=this;if(void 0===e)for(var i=0;i0&&t.push(c[0]),t.push(s[0])}return this.spawn(t,!0).filter(e)}),"neighborhood"),closedNeighborhood:function(e){return this.neighborhood().add(this).filter(e)},openNeighborhood:function(e){return this.neighborhood(e)}}),kl.neighbourhood=kl.neighborhood,kl.closedNeighbourhood=kl.closedNeighborhood,kl.openNeighbourhood=kl.openNeighborhood,me(kl,{source:rs((function(e){var t,n=this[0];return n&&(t=n._private.source||n.cy().collection()),t&&e?t.filter(e):t}),"source"),target:rs((function(e){var t,n=this[0];return n&&(t=n._private.target||n.cy().collection()),t&&e?t.filter(e):t}),"target"),sources:Sl({attr:"source"}),targets:Sl({attr:"target"})}),me(kl,{edgesWith:rs(Bl(),"edgesWith"),edgesTo:rs(Bl({thisIsSrc:!0}),"edgesTo")}),me(kl,{connectedEdges:rs((function(e){for(var t=[],n=0;n0);return i},component:function(){var e=this[0];return e.cy().mutableElements().components(e)[0]}}),kl.componentsOf=kl.components;var _l=function(e,t){var n=arguments.length>2&&void 0!==arguments[2]&&arguments[2],r=arguments.length>3&&void 0!==arguments[3]&&arguments[3];if(void 0!==e){var a=new pt,i=!1;if(t){if(t.length>0&&$(t[0])&&!te(t[0])){i=!0;for(var o=[],s=new vt,l=0,u=t.length;l0&&void 0!==arguments[0])||arguments[0],r=!(arguments.length>1&&void 0!==arguments[1])||arguments[1],a=this,i=a.cy(),o=i._private,s=[],l=[],u=0,c=a.length;u0){for(var I=e.length===a.length?a:new _l(i,e),N=0;N0&&void 0!==arguments[0])||arguments[0],t=!(arguments.length>1&&void 0!==arguments[1])||arguments[1],n=this,r=[],a={},i=n._private.cy;function o(e){var n=a[e.id()];t&&e.removed()||n||(a[e.id()]=!0,e.isNode()?(r.push(e),function(e){for(var t=e._private.edges,n=0;n0&&(e?k.emitAndNotify("remove"):t&&k.emit("remove"));for(var T=0;T=.001?function(t,r){for(var a=0;a<4;++a){var i=h(r,e,n);if(0===i)return r;r-=(d(r,e,n)-t)/i}return r}(t,o):0===l?o:function(t,r,a){var i,o,s=0;do{(i=d(o=r+(a-r)/2,e,n)-t)>0?a=o:r=o}while(Math.abs(i)>1e-7&&++s<10);return o}(t,r,r+a)}var p=!1;function g(){p=!0,e===t&&n===r||function(){for(var t=0;t<11;++t)s[t]=d(t*a,e,n)}()}var v=function(a){return p||g(),e===t&&n===r?a:0===a?0:1===a?1:d(f(a),t,r)};v.getControlPoints=function(){return[{x:e,y:t},{x:n,y:r}]};var y="generateBezier("+[e,t,n,r]+")";return v.toString=function(){return y},v}var Il=function(){function e(e){return-e.tension*e.x-e.friction*e.v}function t(t,n,r){var a={x:t.x+r.dx*n,v:t.v+r.dv*n,tension:t.tension,friction:t.friction};return{dx:a.v,dv:e(a)}}function n(n,r){var a={dx:n.v,dv:e(n)},i=t(n,.5*r,a),o=t(n,.5*r,i),s=t(n,r,o),l=1/6*(a.dx+2*(i.dx+o.dx)+s.dx),u=1/6*(a.dv+2*(i.dv+o.dv)+s.dv);return n.x=n.x+l*r,n.v=n.v+u*r,n}return function e(t,r,a){var i,o,s,l={x:-1,v:0,tension:null,friction:null},u=[0],c=0,d=1e-4;for(t=parseFloat(t)||500,r=parseFloat(r)||20,a=a||null,l.tension=t,l.friction=r,o=(i=null!==a)?(c=e(t,r))/a*.016:.016;s=n(s||l,o),u.push(1+s.x),c+=16,Math.abs(s.x)>d&&Math.abs(s.v)>d;);return i?function(e){return u[e*(u.length-1)|0]}:c}}(),Nl=function(e,t,n,r){var a=Rl(e,t,n,r);return function(e,t,n){return e+(t-e)*a(n)}},Ll={linear:function(e,t,n){return e+(t-e)*n},ease:Nl(.25,.1,.25,1),"ease-in":Nl(.42,0,1,1),"ease-out":Nl(0,0,.58,1),"ease-in-out":Nl(.42,0,.58,1),"ease-in-sine":Nl(.47,0,.745,.715),"ease-out-sine":Nl(.39,.575,.565,1),"ease-in-out-sine":Nl(.445,.05,.55,.95),"ease-in-quad":Nl(.55,.085,.68,.53),"ease-out-quad":Nl(.25,.46,.45,.94),"ease-in-out-quad":Nl(.455,.03,.515,.955),"ease-in-cubic":Nl(.55,.055,.675,.19),"ease-out-cubic":Nl(.215,.61,.355,1),"ease-in-out-cubic":Nl(.645,.045,.355,1),"ease-in-quart":Nl(.895,.03,.685,.22),"ease-out-quart":Nl(.165,.84,.44,1),"ease-in-out-quart":Nl(.77,0,.175,1),"ease-in-quint":Nl(.755,.05,.855,.06),"ease-out-quint":Nl(.23,1,.32,1),"ease-in-out-quint":Nl(.86,0,.07,1),"ease-in-expo":Nl(.95,.05,.795,.035),"ease-out-expo":Nl(.19,1,.22,1),"ease-in-out-expo":Nl(1,0,0,1),"ease-in-circ":Nl(.6,.04,.98,.335),"ease-out-circ":Nl(.075,.82,.165,1),"ease-in-out-circ":Nl(.785,.135,.15,.86),spring:function(e,t,n){if(0===n)return Ll.linear;var r=Il(e,t,n);return function(e,t,n){return e+(t-e)*r(n)}},"cubic-bezier":Nl};function zl(e,t,n,r,a){if(1===r)return n;if(t===n)return n;var i=a(t,n,r);return null==e||((e.roundValue||e.color)&&(i=Math.round(i)),void 0!==e.min&&(i=Math.max(i,e.min)),void 0!==e.max&&(i=Math.min(i,e.max))),i}function Ol(e,t){return null!=e.pfValue||null!=e.value?null==e.pfValue||null!=t&&"%"===t.type.units?e.value:e.pfValue:e}function Vl(e,t,n,r,a){var i=null!=a?a.type:null;n<0?n=0:n>1&&(n=1);var o=Ol(e,a),s=Ol(t,a);if(Q(o)&&Q(s))return zl(i,o,s,n,r);if(Z(o)&&Z(s)){for(var l=[],u=0;u0?("spring"===d&&h.push(o.duration),o.easingImpl=Ll[d].apply(null,h)):o.easingImpl=Ll[d]}var f,p=o.easingImpl;if(f=0===o.duration?1:(n-l)/o.duration,o.applying&&(f=o.progress),f<0?f=0:f>1&&(f=1),null==o.delay){var g=o.startPosition,v=o.position;if(v&&a&&!e.locked()){var y={};jl(g.x,v.x)&&(y.x=Vl(g.x,v.x,f,p)),jl(g.y,v.y)&&(y.y=Vl(g.y,v.y,f,p)),e.position(y)}var m=o.startPan,b=o.pan,x=i.pan,w=null!=b&&r;w&&(jl(m.x,b.x)&&(x.x=Vl(m.x,b.x,f,p)),jl(m.y,b.y)&&(x.y=Vl(m.y,b.y,f,p)),e.emit("pan"));var E=o.startZoom,k=o.zoom,T=null!=k&&r;T&&(jl(E,k)&&(i.zoom=$t(i.minZoom,Vl(E,k,f,p),i.maxZoom)),e.emit("zoom")),(w||T)&&e.emit("viewport");var C=o.style;if(C&&C.length>0&&a){for(var P=0;P=0;t--){(0,e[t])()}e.splice(0,e.length)},c=i.length-1;c>=0;c--){var d=i[c],h=d._private;h.stopped?(i.splice(c,1),h.hooked=!1,h.playing=!1,h.started=!1,u(h.frames)):(h.playing||h.applying)&&(h.playing&&h.applying&&(h.applying=!1),h.started||Xl(0,d,e),Fl(t,d,e,n),h.applying&&(h.applying=!1),u(h.frames),null!=h.step&&h.step(e),d.completed()&&(i.splice(c,1),h.hooked=!1,h.playing=!1,h.started=!1,u(h.completes)),s=!0)}return n||0!==i.length||0!==o.length||r.push(t),s}for(var i=!1,o=0;o0?t.notify("draw",n):t.notify("draw")),n.unmerge(r),t.emit("step")}var ql={animate:po.animate(),animation:po.animation(),animated:po.animated(),clearQueue:po.clearQueue(),delay:po.delay(),delayAnimation:po.delayAnimation(),stop:po.stop(),addToAnimationPool:function(e){this.styleEnabled()&&this._private.aniEles.merge(e)},stopAnimationLoop:function(){this._private.animationsRunning=!1},startAnimationLoop:function(){var e=this;if(e._private.animationsRunning=!0,e.styleEnabled()){var t=e.renderer();t&&t.beforeRender?t.beforeRender((function(t,n){Yl(n,e)}),t.beforeRenderPriorities.animations):function t(){e._private.animationsRunning&&Ne((function(n){Yl(n,e),t()}))}()}}},Wl={qualifierCompare:function(e,t){return null==e||null==t?null==e&&null==t:e.sameText(t)},eventMatches:function(e,t,n){var r=t.qualifier;return null==r||e!==n.target&&te(n.target)&&r.matches(n.target)},addEventFields:function(e,t){t.cy=e,t.target=e},callbackContext:function(e,t,n){return null!=t.qualifier?n.target:e}},Kl=function(e){return H(e)?new Qo(e):e},Ul={createEmitter:function(){var e=this._private;return e.emitter||(e.emitter=new Hs(Wl,this)),this},emitter:function(){return this._private.emitter},on:function(e,t,n){return this.emitter().on(e,Kl(t),n),this},removeListener:function(e,t,n){return this.emitter().removeListener(e,Kl(t),n),this},removeAllListeners:function(){return this.emitter().removeAllListeners(),this},one:function(e,t,n){return this.emitter().one(e,Kl(t),n),this},once:function(e,t,n){return this.emitter().one(e,Kl(t),n),this},emit:function(e,t){return this.emitter().emit(e,t),this},emitAndNotify:function(e,t){return this.emit(e),this.notify(e,t),this}};po.eventAliasesOn(Ul);var Hl={png:function(e){return e=e||{},this._private.renderer.png(e)},jpg:function(e){var t=this._private.renderer;return(e=e||{}).bg=e.bg||"#fff",t.jpg(e)}};Hl.jpeg=Hl.jpg;var Gl={layout:function(e){var t=this;if(null!=e)if(null!=e.name){var n=e.name,r=t.extension("layout",n);if(null!=r){var a;a=H(e.eles)?t.$(e.eles):null!=e.eles?e.eles:t.$();var i=new r(me({},e,{cy:t,eles:a}));return i}nt("No such layout `"+n+"` found. Did you forget to import it and `cytoscape.use()` it?")}else nt("A `name` must be specified to make a layout");else nt("Layout options must be specified to make a layout")}};Gl.createLayout=Gl.makeLayout=Gl.layout;var Zl={notify:function(e,t){var n=this._private;if(this.batching()){n.batchNotifications=n.batchNotifications||{};var r=n.batchNotifications[e]=n.batchNotifications[e]||this.collection();null!=t&&r.merge(t)}else if(n.notificationsEnabled){var a=this.renderer();!this.destroyed()&&a&&a.notify(e,t)}},notifications:function(e){var t=this._private;return void 0===e?t.notificationsEnabled:(t.notificationsEnabled=!!e,this)},noNotifications:function(e){this.notifications(!1),e(),this.notifications(!0)},batching:function(){return this._private.batchCount>0},startBatch:function(){var e=this._private;return null==e.batchCount&&(e.batchCount=0),0===e.batchCount&&(e.batchStyleEles=this.collection(),e.batchNotifications={}),e.batchCount++,this},endBatch:function(){var e=this._private;if(0===e.batchCount)return this;if(e.batchCount--,0===e.batchCount){e.batchStyleEles.updateStyle();var t=this.renderer();Object.keys(e.batchNotifications).forEach((function(n){var r=e.batchNotifications[n];r.empty()?t.notify(n):t.notify(n,r)}))}return this},batch:function(e){return this.startBatch(),e(),this.endBatch(),this},batchData:function(e){var t=this;return this.batch((function(){for(var n=Object.keys(e),r=0;r0;)t.removeChild(t.childNodes[0]);e._private.renderer=null,e.mutableElements().forEach((function(e){var t=e._private;t.rscratch={},t.rstyle={},t.animation.current=[],t.animation.queue=[]}))},onRender:function(e){return this.on("render",e)},offRender:function(e){return this.off("render",e)}};Ql.invalidateDimensions=Ql.resize;var Jl={collection:function(e,t){return H(e)?this.$(e):ee(e)?e.collection():Z(e)?(t||(t={}),new _l(this,e,t.unique,t.removed)):new _l(this)},nodes:function(e){var t=this.$((function(e){return e.isNode()}));return e?t.filter(e):t},edges:function(e){var t=this.$((function(e){return e.isEdge()}));return e?t.filter(e):t},$:function(e){var t=this._private.elements;return e?t.filter(e):t.spawnSelf()},mutableElements:function(){return this._private.elements}};Jl.elements=Jl.filter=Jl.$;var eu={},tu="t";eu.apply=function(e){for(var t=this,n=t._private.cy.collection(),r=0;r0;if(h||d&&f){var p=void 0;h&&f||h?p=u.properties:f&&(p=u.mappedProperties);for(var g=0;g1&&(v=1),s.color){var w=a.valueMin[0],E=a.valueMax[0],k=a.valueMin[1],T=a.valueMax[1],C=a.valueMin[2],P=a.valueMax[2],S=null==a.valueMin[3]?1:a.valueMin[3],B=null==a.valueMax[3]?1:a.valueMax[3],D=[Math.round(w+(E-w)*v),Math.round(k+(T-k)*v),Math.round(C+(P-C)*v),Math.round(S+(B-S)*v)];n={bypass:a.bypass,name:a.name,value:D,strValue:"rgb("+D[0]+", "+D[1]+", "+D[2]+")"}}else{if(!s.number)return!1;var _=a.valueMin+(a.valueMax-a.valueMin)*v;n=this.parse(a.name,_,a.bypass,h)}if(!n)return g(),!1;n.mapping=a,a=n;break;case o.data:for(var A=a.field.split("."),M=d.data,R=0;R0&&i>0){for(var s={},l=!1,u=0;u0?e.delayAnimation(o).play().promise().then(t):t()})).then((function(){return e.animation({style:s,duration:i,easing:e.pstyle("transition-timing-function").value,queue:!1}).play().promise()})).then((function(){n.removeBypasses(e,a),e.emitAndNotify("style"),r.transitioning=!1}))}else r.transitioning&&(this.removeBypasses(e,a),e.emitAndNotify("style"),r.transitioning=!1)},eu.checkTrigger=function(e,t,n,r,a,i){var o=this.properties[t],s=a(o);e.removed()||null!=s&&s(n,r,e)&&i(o)},eu.checkZOrderTrigger=function(e,t,n,r){var a=this;this.checkTrigger(e,t,n,r,(function(e){return e.triggersZOrder}),(function(){a._private.cy.notify("zorder",e)}))},eu.checkBoundsTrigger=function(e,t,n,r){this.checkTrigger(e,t,n,r,(function(e){return e.triggersBounds}),(function(t){e.dirtyCompoundBoundsCache(),e.dirtyBoundingBoxCache()}))},eu.checkConnectedEdgesBoundsTrigger=function(e,t,n,r){this.checkTrigger(e,t,n,r,(function(e){return e.triggersBoundsOfConnectedEdges}),(function(t){e.connectedEdges().forEach((function(e){e.dirtyBoundingBoxCache()}))}))},eu.checkParallelEdgesBoundsTrigger=function(e,t,n,r){this.checkTrigger(e,t,n,r,(function(e){return e.triggersBoundsOfParallelEdges}),(function(t){e.parallelEdges().forEach((function(e){e.dirtyBoundingBoxCache()}))}))},eu.checkTriggers=function(e,t,n,r){e.dirtyStyleCache(),this.checkZOrderTrigger(e,t,n,r),this.checkBoundsTrigger(e,t,n,r),this.checkConnectedEdgesBoundsTrigger(e,t,n,r),this.checkParallelEdgesBoundsTrigger(e,t,n,r)};var nu={applyBypass:function(e,t,n,r){var a=[];if("*"===t||"**"===t){if(void 0!==n)for(var i=0;it.length?i.substr(t.length):""}function s(){n=n.length>r.length?n.substr(r.length):""}for(i=i.replace(/[/][*](\s|.)+?[*][/]/g,"");;){if(i.match(/^\s*$/))break;var l=i.match(/^\s*((?:.|\s)+?)\s*\{((?:.|\s)+?)\}/);if(!l){at("Halting stylesheet parsing: String stylesheet contains more to parse but no selector and block found in: "+i);break}t=l[0];var u=l[1];if("core"!==u)if(new Qo(u).invalid){at("Skipping parsing of block: Invalid selector found in string stylesheet: "+u),o();continue}var c=l[2],d=!1;n=c;for(var h=[];;){if(n.match(/^\s*$/))break;var f=n.match(/^\s*(.+?)\s*:\s*(.+?)(?:\s*;|\s*$)/);if(!f){at("Skipping parsing of block: Invalid formatting of style property and value definitions found in:"+c),d=!0;break}r=f[0];var p=f[1],g=f[2];if(this.properties[p])a.parse(p,g)?(h.push({name:p,val:g}),s()):(at("Skipping property: Invalid property definition in: "+r),s());else at("Skipping property: Invalid property name in: "+r),s()}if(d){o();break}a.selector(u);for(var v=0;v=7&&"d"===t[0]&&(u=new RegExp(s.data.regex).exec(t))){if(n)return!1;var h=s.data;return{name:e,value:u,strValue:""+t,mapped:h,field:u[1],bypass:n}}if(t.length>=10&&"m"===t[0]&&(c=new RegExp(s.mapData.regex).exec(t))){if(n)return!1;if(d.multiple)return!1;var f=s.mapData;if(!d.color&&!d.number)return!1;var p=this.parse(e,c[4]);if(!p||p.mapped)return!1;var g=this.parse(e,c[5]);if(!g||g.mapped)return!1;if(p.pfValue===g.pfValue||p.strValue===g.strValue)return at("`"+e+": "+t+"` is not a valid mapper because the output range is zero; converting to `"+e+": "+p.strValue+"`"),this.parse(e,p.strValue);if(d.color){var v=p.value,y=g.value;if(!(v[0]!==y[0]||v[1]!==y[1]||v[2]!==y[2]||v[3]!==y[3]&&(null!=v[3]&&1!==v[3]||null!=y[3]&&1!==y[3])))return!1}return{name:e,value:c,strValue:""+t,mapped:f,field:c[1],fieldMin:parseFloat(c[2]),fieldMax:parseFloat(c[3]),valueMin:p.value,valueMax:g.value,bypass:n}}}if(d.multiple&&"multiple"!==r){var m;if(m=l?t.split(/\s+/):Z(t)?t:[t],d.evenMultiple&&m.length%2!=0)return null;for(var b=[],x=[],w=[],E="",k=!1,T=0;T0?" ":"")+C.strValue}return d.validate&&!d.validate(b,x)?null:d.singleEnum&&k?1===b.length&&H(b[0])?{name:e,value:b[0],strValue:b[0],bypass:n}:null:{name:e,value:b,pfValue:w,strValue:E,bypass:n,units:x}}var P,S,B=function(){for(var r=0;rd.max||d.strictMax&&t===d.max))return null;var R={name:e,value:t,strValue:""+t+(D||""),units:D,bypass:n};return d.unitless||"px"!==D&&"em"!==D?R.pfValue=t:R.pfValue="px"!==D&&D?this.getEmSizeInPixels()*t:t,"ms"!==D&&"s"!==D||(R.pfValue="ms"===D?t:1e3*t),"deg"!==D&&"rad"!==D||(R.pfValue="rad"===D?t:(P=t,Math.PI*P/180)),"%"===D&&(R.pfValue=t/100),R}if(d.propList){var I=[],N=""+t;if("none"===N);else{for(var L=N.split(/\s*,\s*|\s+/),z=0;z0&&l>0&&!isNaN(n.w)&&!isNaN(n.h)&&n.w>0&&n.h>0)return{zoom:o=(o=(o=Math.min((s-2*t)/n.w,(l-2*t)/n.h))>this._private.maxZoom?this._private.maxZoom:o)=n.minZoom&&(n.maxZoom=t),this},minZoom:function(e){return void 0===e?this._private.minZoom:this.zoomRange({min:e})},maxZoom:function(e){return void 0===e?this._private.maxZoom:this.zoomRange({max:e})},getZoomedViewport:function(e){var t,n,r=this._private,a=r.pan,i=r.zoom,o=!1;if(r.zoomingEnabled||(o=!0),Q(e)?n=e:$(e)&&(n=e.level,null!=e.position?t=Ft(e.position,i,a):null!=e.renderedPosition&&(t=e.renderedPosition),null==t||r.panningEnabled||(o=!0)),n=(n=n>r.maxZoom?r.maxZoom:n)t.maxZoom||!t.zoomingEnabled?i=!0:(t.zoom=s,a.push("zoom"))}if(r&&(!i||!e.cancelOnFailedZoom)&&t.panningEnabled){var l=e.pan;Q(l.x)&&(t.pan.x=l.x,o=!1),Q(l.y)&&(t.pan.y=l.y,o=!1),o||a.push("pan")}return a.length>0&&(a.push("viewport"),this.emit(a.join(" ")),this.notify("viewport")),this},center:function(e){var t=this.getCenterPan(e);return t&&(this._private.pan=t,this.emit("pan viewport"),this.notify("viewport")),this},getCenterPan:function(e,t){if(this._private.panningEnabled){if(H(e)){var n=e;e=this.mutableElements().filter(n)}else ee(e)||(e=this.mutableElements());if(0!==e.length){var r=e.boundingBox(),a=this.width(),i=this.height();return{x:(a-(t=void 0===t?this._private.zoom:t)*(r.x1+r.x2))/2,y:(i-t*(r.y1+r.y2))/2}}}},reset:function(){return this._private.panningEnabled&&this._private.zoomingEnabled?(this.viewport({pan:{x:0,y:0},zoom:1}),this):this},invalidateSize:function(){this._private.sizeCache=null},size:function(){var e,t,n=this._private,r=n.container,a=this;return n.sizeCache=n.sizeCache||(r?(e=a.window().getComputedStyle(r),t=function(t){return parseFloat(e.getPropertyValue(t))},{width:r.clientWidth-t("padding-left")-t("padding-right"),height:r.clientHeight-t("padding-top")-t("padding-bottom")}):{width:1,height:1})},width:function(){return this.size().width},height:function(){return this.size().height},extent:function(){var e=this._private.pan,t=this._private.zoom,n=this.renderedExtent(),r={x1:(n.x1-e.x)/t,x2:(n.x2-e.x)/t,y1:(n.y1-e.y)/t,y2:(n.y2-e.y)/t};return r.w=r.x2-r.x1,r.h=r.y2-r.y1,r},renderedExtent:function(){var e=this.width(),t=this.height();return{x1:0,y1:0,x2:e,y2:t,w:e,h:t}},multiClickDebounceTime:function(e){return e?(this._private.multiClickDebounceTime=e,this):this._private.multiClickDebounceTime}};hu.centre=hu.center,hu.autolockNodes=hu.autolock,hu.autoungrabifyNodes=hu.autoungrabify;var fu={data:po.data({field:"data",bindingEvent:"data",allowBinding:!0,allowSetting:!0,settingEvent:"data",settingTriggersEvent:!0,triggerFnName:"trigger",allowGetting:!0,updateStyle:!0}),removeData:po.removeData({field:"data",event:"data",triggerFnName:"trigger",triggerEvent:!0,updateStyle:!0}),scratch:po.data({field:"scratch",bindingEvent:"scratch",allowBinding:!0,allowSetting:!0,settingEvent:"scratch",settingTriggersEvent:!0,triggerFnName:"trigger",allowGetting:!0,updateStyle:!0}),removeScratch:po.removeData({field:"scratch",event:"scratch",triggerFnName:"trigger",triggerEvent:!0,updateStyle:!0})};fu.attr=fu.data,fu.removeAttr=fu.removeData;var pu=function(e){var t=this,n=(e=me({},e)).container;n&&!J(n)&&J(n[0])&&(n=n[0]);var r=n?n._cyreg:null;(r=r||{})&&r.cy&&(r.cy.destroy(),r={});var a=r.readies=r.readies||[];n&&(n._cyreg=r),r.cy=t;var i=void 0!==f&&void 0!==n&&!e.headless,o=e;o.layout=me({name:i?"grid":"null"},o.layout),o.renderer=me({name:i?"canvas":"null"},o.renderer);var s=function(e,t,n){return void 0!==t?t:void 0!==n?n:e},l=this._private={container:n,ready:!1,options:o,elements:new _l(this),listeners:[],aniEles:new _l(this),data:o.data||{},scratch:{},layout:null,renderer:null,destroyed:!1,notificationsEnabled:!0,minZoom:1e-50,maxZoom:1e50,zoomingEnabled:s(!0,o.zoomingEnabled),userZoomingEnabled:s(!0,o.userZoomingEnabled),panningEnabled:s(!0,o.panningEnabled),userPanningEnabled:s(!0,o.userPanningEnabled),boxSelectionEnabled:s(!0,o.boxSelectionEnabled),autolock:s(!1,o.autolock,o.autolockNodes),autoungrabify:s(!1,o.autoungrabify,o.autoungrabifyNodes),autounselectify:s(!1,o.autounselectify),styleEnabled:void 0===o.styleEnabled?i:o.styleEnabled,zoom:Q(o.zoom)?o.zoom:1,pan:{x:$(o.pan)&&Q(o.pan.x)?o.pan.x:0,y:$(o.pan)&&Q(o.pan.y)?o.pan.y:0},animation:{current:[],queue:[]},hasCompoundNodes:!1,multiClickDebounceTime:s(250,o.multiClickDebounceTime)};this.createEmitter(),this.selectionType(o.selectionType),this.zoomRange({min:o.minZoom,max:o.maxZoom});l.styleEnabled&&t.setStyle([]);var u=me({},o,o.renderer);t.initRenderer(u);!function(e,t){if(e.some(oe))return jr.all(e).then(t);t(e)}([o.style,o.elements],(function(e){var n=e[0],i=e[1];l.styleEnabled&&t.style().append(n),function(e,n,r){t.notifications(!1);var a=t.mutableElements();a.length>0&&a.remove(),null!=e&&($(e)||Z(e))&&t.add(e),t.one("layoutready",(function(e){t.notifications(!0),t.emit(e),t.one("load",n),t.emitAndNotify("load")})).one("layoutstop",(function(){t.one("done",r),t.emit("done")}));var i=me({},t._private.options.layout);i.eles=t.elements(),t.layout(i).run()}(i,(function(){t.startAnimationLoop(),l.ready=!0,G(o.ready)&&t.on("ready",o.ready);for(var e=0;e0,l=!!t.boundingBox,u=n.extent(),c=Qt(l?t.boundingBox:{x1:u.x1,y1:u.y1,w:u.w,h:u.h});if(ee(t.roots))e=t.roots;else if(Z(t.roots)){for(var d=[],h=0;h0;){var D=P.shift(),_=C(D,S);if(_)D.outgoers().filter((function(e){return e.isNode()&&r.has(e)})).forEach(B);else if(null===_){at("Detected double maximal shift for node `"+D.id()+"`. Bailing maximal adjustment due to cycle. Use `options.maximal: true` only on DAGs.");break}}}var A=0;if(t.avoidOverlap)for(var M=0;M0&&m[0].length<=3?i/2:0),s=2*Math.PI/m[r].length*a;return 0===r&&1===m[0].length&&(o=1),{x:W+o*Math.cos(s),y:K+o*Math.sin(s)}}var u=m[r].length,d=Math.max(1===u?0:l?(c.w-2*t.padding-U.w)/((t.grid?$:u)-1):(c.w-2*t.padding-U.w)/((t.grid?$:u)+1),A);return{x:W+(a+1-(u+1)/2)*d,y:K+(r+1-(V+1)/2)*G}})),this};var wu={fit:!0,padding:30,boundingBox:void 0,avoidOverlap:!0,nodeDimensionsIncludeLabels:!1,spacingFactor:void 0,radius:void 0,startAngle:1.5*Math.PI,sweep:void 0,clockwise:!0,sort:void 0,animate:!1,animationDuration:500,animationEasing:void 0,animateFilter:function(e,t){return!0},ready:void 0,stop:void 0,transform:function(e,t){return t}};function Eu(e){this.options=me({},wu,e)}Eu.prototype.run=function(){var e=this.options,t=e,n=e.cy,r=t.eles,a=void 0!==t.counterclockwise?!t.counterclockwise:t.clockwise,i=r.nodes().not(":parent");t.sort&&(i=i.sort(t.sort));for(var o,s=Qt(t.boundingBox?t.boundingBox:{x1:0,y1:0,w:n.width(),h:n.height()}),l=s.x1+s.w/2,u=s.y1+s.h/2,c=(void 0===t.sweep?2*Math.PI-2*Math.PI/i.length:t.sweep)/Math.max(1,i.length-1),d=0,h=0;h1&&t.avoidOverlap){d*=1.75;var v=Math.cos(c)-Math.cos(0),y=Math.sin(c)-Math.sin(0),m=Math.sqrt(d*d/(v*v+y*y));o=Math.max(m,o)}return r.nodes().layoutPositions(this,t,(function(e,n){var r=t.startAngle+n*c*(a?1:-1),i=o*Math.cos(r),s=o*Math.sin(r);return{x:l+i,y:u+s}})),this};var ku,Tu={fit:!0,padding:30,startAngle:1.5*Math.PI,sweep:void 0,clockwise:!0,equidistant:!1,minNodeSpacing:10,boundingBox:void 0,avoidOverlap:!0,nodeDimensionsIncludeLabels:!1,height:void 0,width:void 0,spacingFactor:void 0,concentric:function(e){return e.degree()},levelWidth:function(e){return e.maxDegree()/4},animate:!1,animationDuration:500,animationEasing:void 0,animateFilter:function(e,t){return!0},ready:void 0,stop:void 0,transform:function(e,t){return t}};function Cu(e){this.options=me({},Tu,e)}Cu.prototype.run=function(){for(var e=this.options,t=e,n=void 0!==t.counterclockwise?!t.counterclockwise:t.clockwise,r=e.cy,a=t.eles,i=a.nodes().not(":parent"),o=Qt(t.boundingBox?t.boundingBox:{x1:0,y1:0,w:r.width(),h:r.height()}),s=o.x1+o.w/2,l=o.y1+o.h/2,u=[],c=0,d=0;d0)Math.abs(m[0].value-x.value)>=v&&(m=[],y.push(m));m.push(x)}var w=c+t.minNodeSpacing;if(!t.avoidOverlap){var E=y.length>0&&y[0].length>1,k=(Math.min(o.w,o.h)/2-w)/(y.length+E?1:0);w=Math.min(w,k)}for(var T=0,C=0;C1&&t.avoidOverlap){var D=Math.cos(B)-Math.cos(0),_=Math.sin(B)-Math.sin(0),A=Math.sqrt(w*w/(D*D+_*_));T=Math.max(A,T)}P.r=T,T+=w}if(t.equidistant){for(var M=0,R=0,I=0;I=e.numIter)&&(Iu(r,e),r.temperature=r.temperature*e.coolingFactor,!(r.temperature=e.animationThreshold&&i(),Ne(c)):(Ku(r,e),s())};c()}else{for(;u;)u=o(l),l++;Ku(r,e),s()}return this},Su.prototype.stop=function(){return this.stopped=!0,this.thread&&this.thread.stop(),this.emit("layoutstop"),this},Su.prototype.destroy=function(){return this.thread&&this.thread.stop(),this};var Bu=function(e,t,n){for(var r=n.eles.edges(),a=n.eles.nodes(),i=Qt(n.boundingBox?n.boundingBox:{x1:0,y1:0,w:e.width(),h:e.height()}),o={isCompound:e.hasCompoundNodes(),layoutNodes:[],idToIndex:{},nodeSize:a.size(),graphSet:[],indexToGraph:[],layoutEdges:[],edgeSize:r.size(),temperature:n.initialTemp,clientWidth:i.w,clientHeight:i.h,boundingBox:i},s=n.eles.components(),l={},u=0;u0){o.graphSet.push(w);for(u=0;ur.count?0:r.graph},_u=function(e,t,n,r){var a=r.graphSet[n];if(-10)var s=(u=r.nodeOverlap*o)*a/(g=Math.sqrt(a*a+i*i)),l=u*i/g;else{var u,c=Vu(e,a,i),d=Vu(t,-1*a,-1*i),h=d.x-c.x,f=d.y-c.y,p=h*h+f*f,g=Math.sqrt(p);s=(u=(e.nodeRepulsion+t.nodeRepulsion)/p)*h/g,l=u*f/g}e.isLocked||(e.offsetX-=s,e.offsetY-=l),t.isLocked||(t.offsetX+=s,t.offsetY+=l)}},Ou=function(e,t,n,r){if(n>0)var a=e.maxX-t.minX;else a=t.maxX-e.minX;if(r>0)var i=e.maxY-t.minY;else i=t.maxY-e.minY;return a>=0&&i>=0?Math.sqrt(a*a+i*i):0},Vu=function(e,t,n){var r=e.positionX,a=e.positionY,i=e.height||1,o=e.width||1,s=n/t,l=i/o,u={};return 0===t&&0n?(u.x=r,u.y=a+i/2,u):0t&&-1*l<=s&&s<=l?(u.x=r-o/2,u.y=a-o*n/2/t,u):0=l)?(u.x=r+i*t/2/n,u.y=a+i/2,u):0>n&&(s<=-1*l||s>=l)?(u.x=r-i*t/2/n,u.y=a-i/2,u):u},Fu=function(e,t){for(var n=0;n1){var p=t.gravity*d/f,g=t.gravity*h/f;c.offsetX+=p,c.offsetY+=g}}}}},Xu=function(e,t){var n=[],r=0,a=-1;for(n.push.apply(n,e.graphSet[0]),a+=e.graphSet[0].length;r<=a;){var i=n[r++],o=e.idToIndex[i],s=e.layoutNodes[o],l=s.children;if(0n)var a={x:n*e/r,y:n*t/r};else a={x:e,y:t};return a},Wu=function(e,t){var n=e.parentId;if(null!=n){var r=t.layoutNodes[t.idToIndex[n]],a=!1;return(null==r.maxX||e.maxX+r.padRight>r.maxX)&&(r.maxX=e.maxX+r.padRight,a=!0),(null==r.minX||e.minX-r.padLeftr.maxY)&&(r.maxY=e.maxY+r.padBottom,a=!0),(null==r.minY||e.minY-r.padTopp&&(d+=f+t.componentSpacing,c=0,h=0,f=0)}}},Uu={fit:!0,padding:30,boundingBox:void 0,avoidOverlap:!0,avoidOverlapPadding:10,nodeDimensionsIncludeLabels:!1,spacingFactor:void 0,condense:!1,rows:void 0,cols:void 0,position:function(e){},sort:void 0,animate:!1,animationDuration:500,animationEasing:void 0,animateFilter:function(e,t){return!0},ready:void 0,stop:void 0,transform:function(e,t){return t}};function Hu(e){this.options=me({},Uu,e)}Hu.prototype.run=function(){var e=this.options,t=e,n=e.cy,r=t.eles,a=r.nodes().not(":parent");t.sort&&(a=a.sort(t.sort));var i=Qt(t.boundingBox?t.boundingBox:{x1:0,y1:0,w:n.width(),h:n.height()});if(0===i.h||0===i.w)r.nodes().layoutPositions(this,t,(function(e){return{x:i.x1,y:i.y1}}));else{var o=a.size(),s=Math.sqrt(o*i.h/i.w),l=Math.round(s),u=Math.round(i.w/i.h*s),c=function(e){if(null==e)return Math.min(l,u);Math.min(l,u)==l?l=e:u=e},d=function(e){if(null==e)return Math.max(l,u);Math.max(l,u)==l?l=e:u=e},h=t.rows,f=null!=t.cols?t.cols:t.columns;if(null!=h&&null!=f)l=h,u=f;else if(null!=h&&null==f)l=h,u=Math.ceil(o/l);else if(null==h&&null!=f)u=f,l=Math.ceil(o/u);else if(u*l>o){var p=c(),g=d();(p-1)*g>=o?c(p-1):(g-1)*p>=o&&d(g-1)}else for(;u*l=o?d(y+1):c(v+1)}var m=i.w/u,b=i.h/l;if(t.condense&&(m=0,b=0),t.avoidOverlap)for(var x=0;x=u&&(A=0,_++)},R={},I=0;I(r=hn(e,t,x[w],x[w+1],x[w+2],x[w+3])))return v(n,r),!0}else if("bezier"===i.edgeType||"multibezier"===i.edgeType||"self"===i.edgeType||"compound"===i.edgeType)for(x=i.allpts,w=0;w+5(r=dn(e,t,x[w],x[w+1],x[w+2],x[w+3],x[w+4],x[w+5])))return v(n,r),!0;m=m||a.source,b=b||a.target;var E=o.getArrowWidth(l,c),k=[{name:"source",x:i.arrowStartX,y:i.arrowStartY,angle:i.srcArrowAngle},{name:"target",x:i.arrowEndX,y:i.arrowEndY,angle:i.tgtArrowAngle},{name:"mid-source",x:i.midX,y:i.midY,angle:i.midsrcArrowAngle},{name:"mid-target",x:i.midX,y:i.midY,angle:i.midtgtArrowAngle}];for(w=0;w0&&(y(m),y(b))}function b(e,t,n){return ht(e,t,n)}function x(n,r){var a,i=n._private,o=p;a=r?r+"-":"",n.boundingBox();var s=i.labelBounds[r||"main"],l=n.pstyle(a+"label").value;if("yes"===n.pstyle("text-events").strValue&&l){var u=b(i.rscratch,"labelX",r),c=b(i.rscratch,"labelY",r),d=b(i.rscratch,"labelAngle",r),h=n.pstyle(a+"text-margin-x").pfValue,f=n.pstyle(a+"text-margin-y").pfValue,g=s.x1-o-h,y=s.x2+o-h,m=s.y1-o-f,x=s.y2+o-f;if(d){var w=Math.cos(d),E=Math.sin(d),k=function(e,t){return{x:(e-=u)*w-(t-=c)*E+u,y:e*E+t*w+c}},T=k(g,m),C=k(g,x),P=k(y,m),S=k(y,x),B=[T.x+h,T.y+f,P.x+h,P.y+f,S.x+h,S.y+f,C.x+h,C.y+f];if(fn(e,t,B))return v(n),!0}else if(on(s,e,t))return v(n),!0}}n&&(l=l.interactive);for(var w=l.length-1;w>=0;w--){var E=l[w];E.isNode()?y(E)||x(E):m(E)||x(E)||x(E,"source")||x(E,"target")}return u},getAllInBox:function(e,t,n,r){for(var a,i,o=this.getCachedZSortedEles().interactive,s=[],l=Math.min(e,n),u=Math.max(e,n),c=Math.min(t,r),d=Math.max(t,r),h=Qt({x1:e=l,y1:t=c,x2:n=u,y2:r=d}),f=0;f0?-(Math.PI-i.ang):Math.PI+i.ang),Sc(t,n,Pc),cc=Cc.nx*Pc.ny-Cc.ny*Pc.nx,dc=Cc.nx*Pc.nx-Cc.ny*-Pc.ny,pc=Math.asin(Math.max(-1,Math.min(1,cc))),Math.abs(pc)<1e-6)return lc=t.x,uc=t.y,void(vc=mc=0);hc=1,fc=!1,dc<0?pc<0?pc=Math.PI+pc:(pc=Math.PI-pc,hc=-1,fc=!0):pc>0&&(hc=-1,fc=!0),mc=void 0!==t.radius?t.radius:r,gc=pc/2,bc=Math.min(Cc.len/2,Pc.len/2),a?(yc=Math.abs(Math.cos(gc)*mc/Math.sin(gc)))>bc?(yc=bc,vc=Math.abs(yc*Math.sin(gc)/Math.cos(gc))):vc=mc:(yc=Math.min(bc,mc),vc=Math.abs(yc*Math.sin(gc)/Math.cos(gc))),Ec=t.x+Pc.nx*yc,kc=t.y+Pc.ny*yc,lc=Ec-Pc.ny*vc*hc,uc=kc+Pc.nx*vc*hc,xc=t.x+Cc.nx*yc,wc=t.y+Cc.ny*yc,Tc=t};function Dc(e,t){0===t.radius?e.lineTo(t.cx,t.cy):e.arc(t.cx,t.cy,t.radius,t.startAngle,t.endAngle,t.counterClockwise)}function _c(e,t,n,r){var a=!(arguments.length>4&&void 0!==arguments[4])||arguments[4];return 0===r||0===t.radius?{cx:t.x,cy:t.y,radius:0,startX:t.x,startY:t.y,stopX:t.x,stopY:t.y,startAngle:void 0,endAngle:void 0,counterClockwise:void 0}:(Bc(e,t,n,r,a),{cx:lc,cy:uc,radius:vc,startX:xc,startY:wc,stopX:Ec,stopY:kc,startAngle:Cc.ang+Math.PI/2*hc,endAngle:Pc.ang-Math.PI/2*hc,counterClockwise:fc})}var Ac=.01,Mc=Math.sqrt(.02),Rc={};function Ic(e){var t=[];if(null!=e){for(var n=0;n0?Math.max(e-t,0):Math.min(e+t,0)},S=P(T,E),B=P(C,k),D=!1;"auto"===v?g=Math.abs(S)>Math.abs(B)?a:r:v===l||v===s?(g=r,D=!0):v!==i&&v!==o||(g=a,D=!0);var _,A=g===r,M=A?B:S,R=A?C:T,I=Wt(R),N=!1;(D&&(m||x)||!(v===s&&R<0||v===l&&R>0||v===i&&R>0||v===o&&R<0)||(M=(I*=-1)*Math.abs(M),N=!0),m)?_=(b<0?1+b:b)*M:_=(b<0?M:0)+b*I;var L=function(e){return Math.abs(e)=Math.abs(M)},z=L(_),O=L(Math.abs(M)-Math.abs(_));if((z||O)&&!N)if(A){var V=Math.abs(R)<=d/2,F=Math.abs(T)<=h/2;if(V){var j=(u.x1+u.x2)/2,X=u.y1,Y=u.y2;n.segpts=[j,X,j,Y]}else if(F){var q=(u.y1+u.y2)/2,W=u.x1,K=u.x2;n.segpts=[W,q,K,q]}else n.segpts=[u.x1,u.y2]}else{var U=Math.abs(R)<=c/2,H=Math.abs(C)<=f/2;if(U){var G=(u.y1+u.y2)/2,Z=u.x1,$=u.x2;n.segpts=[Z,G,$,G]}else if(H){var Q=(u.x1+u.x2)/2,J=u.y1,ee=u.y2;n.segpts=[Q,J,Q,ee]}else n.segpts=[u.x2,u.y1]}else if(A){var te=u.y1+_+(p?d/2*I:0),ne=u.x1,re=u.x2;n.segpts=[ne,te,re,te]}else{var ae=u.x1+_+(p?c/2*I:0),ie=u.y1,oe=u.y2;n.segpts=[ae,ie,ae,oe]}if(n.isRound){var se=e.pstyle("taxi-radius").value,le="arc-radius"===e.pstyle("radius-type").value[0];n.radii=new Array(n.segpts.length/2).fill(se),n.isArcRadius=new Array(n.segpts.length/2).fill(le)}},Rc.tryToCorrectInvalidPoints=function(e,t){var n=e._private.rscratch;if("bezier"===n.edgeType){var r=t.srcPos,a=t.tgtPos,i=t.srcW,o=t.srcH,s=t.tgtW,l=t.tgtH,u=t.srcShape,c=t.tgtShape,d=t.srcCornerRadius,h=t.tgtCornerRadius,f=t.srcRs,p=t.tgtRs,g=!Q(n.startX)||!Q(n.startY),v=!Q(n.arrowStartX)||!Q(n.arrowStartY),y=!Q(n.endX)||!Q(n.endY),m=!Q(n.arrowEndX)||!Q(n.arrowEndY),b=3*(this.getArrowWidth(e.pstyle("width").pfValue,e.pstyle("arrow-scale").value)*this.arrowShapeWidth),x=Kt({x:n.ctrlpts[0],y:n.ctrlpts[1]},{x:n.startX,y:n.startY}),w=xg.poolIndex()){var v=p;p=g,g=v}var y=d.srcPos=p.position(),m=d.tgtPos=g.position(),b=d.srcW=p.outerWidth(),x=d.srcH=p.outerHeight(),E=d.tgtW=g.outerWidth(),k=d.tgtH=g.outerHeight(),T=d.srcShape=n.nodeShapes[t.getNodeShape(p)],C=d.tgtShape=n.nodeShapes[t.getNodeShape(g)],P=d.srcCornerRadius="auto"===p.pstyle("corner-radius").value?"auto":p.pstyle("corner-radius").pfValue,S=d.tgtCornerRadius="auto"===g.pstyle("corner-radius").value?"auto":g.pstyle("corner-radius").pfValue,B=d.tgtRs=g._private.rscratch,D=d.srcRs=p._private.rscratch;d.dirCounts={north:0,west:0,south:0,east:0,northwest:0,southwest:0,northeast:0,southeast:0};for(var _=0;_=Mc||(q=Math.sqrt(Math.max(Y*Y,Ac)+Math.max(X*X,Ac)));var W=d.vector={x:Y,y:X},K=d.vectorNorm={x:W.x/q,y:W.y/q},U={x:-K.y,y:K.x};d.nodesOverlap=!Q(q)||C.checkPoint(L[0],L[1],0,E,k,m.x,m.y,S,B)||T.checkPoint(O[0],O[1],0,b,x,y.x,y.y,P,D),d.vectorNormInverse=U,e={nodesOverlap:d.nodesOverlap,dirCounts:d.dirCounts,calculatedIntersection:!0,hasBezier:d.hasBezier,hasUnbundled:d.hasUnbundled,eles:d.eles,srcPos:m,srcRs:B,tgtPos:y,tgtRs:D,srcW:E,srcH:k,tgtW:b,tgtH:x,srcIntn:V,tgtIntn:z,srcShape:C,tgtShape:T,posPts:{x1:j.x2,y1:j.y2,x2:j.x1,y2:j.y1},intersectionPts:{x1:F.x2,y1:F.y2,x2:F.x1,y2:F.y1},vector:{x:-W.x,y:-W.y},vectorNorm:{x:-K.x,y:-K.y},vectorNormInverse:{x:-U.x,y:-U.y}}}var H=N?e:d;M.nodesOverlap=H.nodesOverlap,M.srcIntn=H.srcIntn,M.tgtIntn=H.tgtIntn,M.isRound=R.startsWith("round"),r&&(p.isParent()||p.isChild()||g.isParent()||g.isChild())&&(p.parents().anySame(g)||g.parents().anySame(p)||p.same(g)&&p.isParent())?t.findCompoundLoopPoints(A,H,_,I):p===g?t.findLoopPoints(A,H,_,I):R.endsWith("segments")?t.findSegmentsPoints(A,H):R.endsWith("taxi")?t.findTaxiPoints(A,H):"straight"===R||!I&&d.eles.length%2==1&&_===Math.floor(d.eles.length/2)?t.findStraightEdgePoints(A):t.findBezierPoints(A,H,_,I,N),t.findEndpoints(A),t.tryToCorrectInvalidPoints(A,H),t.checkForInvalidEdgeWarning(A),t.storeAllpts(A),t.storeEdgeProjections(A),t.calculateArrowAngles(A),t.recalculateEdgeLabelProjections(A),t.calculateLabelAngles(A)}},w=0;w0){var W=u,K=Ut(W,Xt(t)),U=Ut(W,Xt(q)),H=K;if(U2)Ut(W,{x:q[2],y:q[3]})0){var le=c,ue=Ut(le,Xt(t)),ce=Ut(le,Xt(se)),de=ue;if(ce2)Ut(le,{x:se[2],y:se[3]})=u||m){c={cp:g,segment:y};break}}if(c)break}var b=c.cp,x=c.segment,w=(u-h)/x.length,E=x.t1-x.t0,k=s?x.t0+E*w:x.t1-E*w;k=$t(0,k,1),t=Zt(b.p0,b.p1,b.p2,k),a=function(e,t,n,r){var a=$t(0,r-.001,1),i=$t(0,r+.001,1),o=Zt(e,t,n,a),s=Zt(e,t,n,i);return Fc(o,s)}(b.p0,b.p1,b.p2,k);break;case"straight":case"segments":case"haystack":for(var T,C,P,S,B=0,D=r.allpts.length,_=0;_+3=u));_+=2);var A=(u-C)/T;A=$t(0,A,1),t=function(e,t,n,r){var a=t.x-e.x,i=t.y-e.y,o=Kt(e,t),s=a/o,l=i/o;return n=null==n?0:n,r=null!=r?r:n*o,{x:e.x+s*r,y:e.y+l*r}}(P,S,A),a=Fc(P,S)}o("labelX",n,t.x),o("labelY",n,t.y),o("labelAutoAngle",n,a)}};u("source"),u("target"),this.applyLabelDimensions(e)}},Oc.applyLabelDimensions=function(e){this.applyPrefixedLabelDimensions(e),e.isEdge()&&(this.applyPrefixedLabelDimensions(e,"source"),this.applyPrefixedLabelDimensions(e,"target"))},Oc.applyPrefixedLabelDimensions=function(e,t){var n=e._private,r=this.getLabelText(e,t),a=We(r,e._private.labelDimsKey);if(ht(n.rscratch,"prefixedLabelDimsKey",t)!==a){ft(n.rscratch,"prefixedLabelDimsKey",t,a);var i=this.calculateLabelDimensions(e,r),o=e.pstyle("line-height").pfValue,s=e.pstyle("text-wrap").strValue,l=ht(n.rscratch,"labelWrapCachedLines",t)||[],u="wrap"!==s?1:Math.max(l.length,1),c=i.height/u,d=c*o,h=i.width,f=i.height+(u-1)*(o-1)*c;ft(n.rstyle,"labelWidth",t,h),ft(n.rscratch,"labelWidth",t,h),ft(n.rstyle,"labelHeight",t,f),ft(n.rscratch,"labelHeight",t,f),ft(n.rscratch,"labelLineHeight",t,d)}},Oc.getLabelText=function(e,t){var n=e._private,r=t?t+"-":"",a=e.pstyle(r+"label").strValue,i=e.pstyle("text-transform").value,s=function(e,r){return r?(ft(n.rscratch,e,t,r),r):ht(n.rscratch,e,t)};if(!a)return"";"none"==i||("uppercase"==i?a=a.toUpperCase():"lowercase"==i&&(a=a.toLowerCase()));var l=e.pstyle("text-wrap").value;if("wrap"===l){var u=s("labelKey");if(null!=u&&s("labelWrapKey")===u)return s("labelWrapCachedText");for(var c=a.split("\n"),d=e.pstyle("text-max-width").pfValue,h="anywhere"===e.pstyle("text-overflow-wrap").value,f=[],p=/[\s\u200b]+|$/g,g=0;gd){var b,x="",w=0,E=o(v.matchAll(p));try{for(E.s();!(b=E.n()).done;){var k=b.value,T=k[0],C=v.substring(w,k.index);w=k.index+T.length;var P=0===x.length?C:x+C+T;this.calculateLabelDimensions(e,P).width<=d?x+=C+T:(x&&f.push(x),x=C+T)}}catch(A){E.e(A)}finally{E.f()}x.match(/^[\s\u200b]+$/)||f.push(x)}else f.push(v)}s("labelWrapCachedLines",f),a=s("labelWrapCachedText",f.join("\n")),s("labelWrapKey",u)}else if("ellipsis"===l){var S=e.pstyle("text-max-width").pfValue,B="",D=!1;if(this.calculateLabelDimensions(e,a).widthS)break;B+=a[_],_===a.length-1&&(D=!0)}return D||(B+="\u2026"),B}return a},Oc.getLabelJustification=function(e){var t=e.pstyle("text-justification").strValue,n=e.pstyle("text-halign").strValue;if("auto"!==t)return t;if(!e.isNode())return"center";switch(n){case"left":return"right";case"right":return"left";default:return"center"}},Oc.calculateLabelDimensions=function(e,t){var n=this.cy.window().document,r=e.pstyle("font-style").strValue,a=e.pstyle("font-size").pfValue,i=e.pstyle("font-family").strValue,o=e.pstyle("font-weight").strValue,s=this.labelCalcCanvas,l=this.labelCalcCanvasContext;if(!s){s=this.labelCalcCanvas=n.createElement("canvas"),l=this.labelCalcCanvasContext=s.getContext("2d");var u=s.style;u.position="absolute",u.left="-9999px",u.top="-9999px",u.zIndex="-1",u.visibility="hidden",u.pointerEvents="none"}l.font="".concat(r," ").concat(o," ").concat(a,"px ").concat(i);for(var c=0,d=0,h=t.split("\n"),f=0;f1&&void 0!==arguments[1])||arguments[1];if(t.merge(e),n)for(var r=0;r=e.desktopTapThreshold2}var P=a(t);v&&(e.hoverData.tapholdCancelled=!0);n=!0,r(g,["mousemove","vmousemove","tapdrag"],t,{x:c[0],y:c[1]});var S=function(){e.data.bgActivePosistion=void 0,e.hoverData.selecting||o.emit({originalEvent:t,type:"boxstart",position:{x:c[0],y:c[1]}}),p[4]=1,e.hoverData.selecting=!0,e.redrawHint("select",!0),e.redraw()};if(3===e.hoverData.which){if(v){var B={originalEvent:t,type:"cxtdrag",position:{x:c[0],y:c[1]}};b?b.emit(B):o.emit(B),e.hoverData.cxtDragged=!0,e.hoverData.cxtOver&&g===e.hoverData.cxtOver||(e.hoverData.cxtOver&&e.hoverData.cxtOver.emit({originalEvent:t,type:"cxtdragout",position:{x:c[0],y:c[1]}}),e.hoverData.cxtOver=g,g&&g.emit({originalEvent:t,type:"cxtdragover",position:{x:c[0],y:c[1]}}))}}else if(e.hoverData.dragging){if(n=!0,o.panningEnabled()&&o.userPanningEnabled()){var D;if(e.hoverData.justStartedPan){var _=e.hoverData.mdownPos;D={x:(c[0]-_[0])*s,y:(c[1]-_[1])*s},e.hoverData.justStartedPan=!1}else D={x:x[0]*s,y:x[1]*s};o.panBy(D),o.emit("dragpan"),e.hoverData.dragged=!0}c=e.projectIntoViewport(t.clientX,t.clientY)}else if(1!=p[4]||null!=b&&!b.pannable()){if(b&&b.pannable()&&b.active()&&b.unactivate(),b&&b.grabbed()||g==y||(y&&r(y,["mouseout","tapdragout"],t,{x:c[0],y:c[1]}),g&&r(g,["mouseover","tapdragover"],t,{x:c[0],y:c[1]}),e.hoverData.last=g),b)if(v){if(o.boxSelectionEnabled()&&P)b&&b.grabbed()&&(d(w),b.emit("freeon"),w.emit("free"),e.dragData.didDrag&&(b.emit("dragfreeon"),w.emit("dragfree"))),S();else if(b&&b.grabbed()&&e.nodeIsDraggable(b)){var A=!e.dragData.didDrag;A&&e.redrawHint("eles",!0),e.dragData.didDrag=!0,e.hoverData.draggingEles||u(w,{inDragLayer:!0});var M={x:0,y:0};if(Q(x[0])&&Q(x[1])&&(M.x+=x[0],M.y+=x[1],A)){var R=e.hoverData.dragDelta;R&&Q(R[0])&&Q(R[1])&&(M.x+=R[0],M.y+=R[1])}e.hoverData.draggingEles=!0,w.silentShift(M).emit("position drag"),e.redrawHint("drag",!0),e.redraw()}}else!function(){var t=e.hoverData.dragDelta=e.hoverData.dragDelta||[];0===t.length?(t.push(x[0]),t.push(x[1])):(t[0]+=x[0],t[1]+=x[1])}();n=!0}else if(v){if(e.hoverData.dragging||!o.boxSelectionEnabled()||!P&&o.panningEnabled()&&o.userPanningEnabled()){if(!e.hoverData.selecting&&o.panningEnabled()&&o.userPanningEnabled()){i(b,e.hoverData.downs)&&(e.hoverData.dragging=!0,e.hoverData.justStartedPan=!0,p[4]=0,e.data.bgActivePosistion=Xt(h),e.redrawHint("select",!0),e.redraw())}}else S();b&&b.pannable()&&b.active()&&b.unactivate()}return p[2]=c[0],p[3]=c[1],n?(t.stopPropagation&&t.stopPropagation(),t.preventDefault&&t.preventDefault(),!1):void 0}}),!1),e.registerBinding(t,"mouseup",(function(t){if((1!==e.hoverData.which||1===t.which||!e.hoverData.capture)&&e.hoverData.capture){e.hoverData.capture=!1;var i=e.cy,o=e.projectIntoViewport(t.clientX,t.clientY),s=e.selection,l=e.findNearestElement(o[0],o[1],!0,!1),u=e.dragData.possibleDragElements,c=e.hoverData.down,h=a(t);if(e.data.bgActivePosistion&&(e.redrawHint("select",!0),e.redraw()),e.hoverData.tapholdCancelled=!0,e.data.bgActivePosistion=void 0,c&&c.unactivate(),3===e.hoverData.which){var f={originalEvent:t,type:"cxttapend",position:{x:o[0],y:o[1]}};if(c?c.emit(f):i.emit(f),!e.hoverData.cxtDragged){var p={originalEvent:t,type:"cxttap",position:{x:o[0],y:o[1]}};c?c.emit(p):i.emit(p)}e.hoverData.cxtDragged=!1,e.hoverData.which=null}else if(1===e.hoverData.which){if(r(l,["mouseup","tapend","vmouseup"],t,{x:o[0],y:o[1]}),e.dragData.didDrag||e.hoverData.dragged||e.hoverData.selecting||e.hoverData.isOverThresholdDrag||(r(c,["click","tap","vclick"],t,{x:o[0],y:o[1]}),x=!1,t.timeStamp-w<=i.multiClickDebounceTime()?(b&&clearTimeout(b),x=!0,w=null,r(c,["dblclick","dbltap","vdblclick"],t,{x:o[0],y:o[1]})):(b=setTimeout((function(){x||r(c,["oneclick","onetap","voneclick"],t,{x:o[0],y:o[1]})}),i.multiClickDebounceTime()),w=t.timeStamp)),null!=c||e.dragData.didDrag||e.hoverData.selecting||e.hoverData.dragged||a(t)||(i.$(n).unselect(["tapunselect"]),u.length>0&&e.redrawHint("eles",!0),e.dragData.possibleDragElements=u=i.collection()),l!=c||e.dragData.didDrag||e.hoverData.selecting||null!=l&&l._private.selectable&&(e.hoverData.dragging||("additive"===i.selectionType()||h?l.selected()?l.unselect(["tapunselect"]):l.select(["tapselect"]):h||(i.$(n).unmerge(l).unselect(["tapunselect"]),l.select(["tapselect"]))),e.redrawHint("eles",!0)),e.hoverData.selecting){var g=i.collection(e.getAllInBox(s[0],s[1],s[2],s[3]));e.redrawHint("select",!0),g.length>0&&e.redrawHint("eles",!0),i.emit({type:"boxend",originalEvent:t,position:{x:o[0],y:o[1]}});var v=function(e){return e.selectable()&&!e.selected()};"additive"===i.selectionType()||h||i.$(n).unmerge(g).unselect(),g.emit("box").stdFilter(v).select().emit("boxselect"),e.redraw()}if(e.hoverData.dragging&&(e.hoverData.dragging=!1,e.redrawHint("select",!0),e.redrawHint("eles",!0),e.redraw()),!s[4]){e.redrawHint("drag",!0),e.redrawHint("eles",!0);var y=c&&c.grabbed();d(u),y&&(c.emit("freeon"),u.emit("free"),e.dragData.didDrag&&(c.emit("dragfreeon"),u.emit("dragfree")))}}s[4]=0,e.hoverData.down=null,e.hoverData.cxtStarted=!1,e.hoverData.draggingEles=!1,e.hoverData.selecting=!1,e.hoverData.isOverThresholdDrag=!1,e.dragData.didDrag=!1,e.hoverData.dragged=!1,e.hoverData.dragDelta=[],e.hoverData.mdownPos=null,e.hoverData.mdownGPos=null,e.hoverData.which=null}}),!1);var k,T,C,P,S,B,D,_,A,M,R,I,N,L=function(t){if(!e.scrollingPage){var n=e.cy,r=n.zoom(),a=n.pan(),i=e.projectIntoViewport(t.clientX,t.clientY),o=[i[0]*r+a.x,i[1]*r+a.y];if(e.hoverData.draggingEles||e.hoverData.dragging||e.hoverData.cxtStarted||0!==e.selection[4])t.preventDefault();else if(n.panningEnabled()&&n.userPanningEnabled()&&n.zoomingEnabled()&&n.userZoomingEnabled()){var s;t.preventDefault(),e.data.wheelZooming=!0,clearTimeout(e.data.wheelTimeout),e.data.wheelTimeout=setTimeout((function(){e.data.wheelZooming=!1,e.redrawHint("eles",!0),e.redraw()}),150),s=null!=t.deltaY?t.deltaY/-250:null!=t.wheelDeltaY?t.wheelDeltaY/1e3:t.wheelDelta/1e3,s*=e.wheelSensitivity,1===t.deltaMode&&(s*=33);var l=n.zoom()*Math.pow(10,s);"gesturechange"===t.type&&(l=e.gestureStartZoom*t.scale),n.zoom({level:l,renderedPosition:{x:o[0],y:o[1]}}),n.emit("gesturechange"===t.type?"pinchzoom":"scrollzoom")}}};e.registerBinding(e.container,"wheel",L,!0),e.registerBinding(t,"scroll",(function(t){e.scrollingPage=!0,clearTimeout(e.scrollingPageTimeout),e.scrollingPageTimeout=setTimeout((function(){e.scrollingPage=!1}),250)}),!0),e.registerBinding(e.container,"gesturestart",(function(t){e.gestureStartZoom=e.cy.zoom(),e.hasTouchStarted||t.preventDefault()}),!0),e.registerBinding(e.container,"gesturechange",(function(t){e.hasTouchStarted||L(t)}),!0),e.registerBinding(e.container,"mouseout",(function(t){var n=e.projectIntoViewport(t.clientX,t.clientY);e.cy.emit({originalEvent:t,type:"mouseout",position:{x:n[0],y:n[1]}})}),!1),e.registerBinding(e.container,"mouseover",(function(t){var n=e.projectIntoViewport(t.clientX,t.clientY);e.cy.emit({originalEvent:t,type:"mouseover",position:{x:n[0],y:n[1]}})}),!1);var z,O,V,F,j,X,Y,q=function(e,t,n,r){return Math.sqrt((n-e)*(n-e)+(r-t)*(r-t))},W=function(e,t,n,r){return(n-e)*(n-e)+(r-t)*(r-t)};if(e.registerBinding(e.container,"touchstart",z=function(t){if(e.hasTouchStarted=!0,m(t)){f(),e.touchData.capture=!0,e.data.bgActivePosistion=void 0;var n=e.cy,a=e.touchData.now,i=e.touchData.earlier;if(t.touches[0]){var o=e.projectIntoViewport(t.touches[0].clientX,t.touches[0].clientY);a[0]=o[0],a[1]=o[1]}if(t.touches[1]){o=e.projectIntoViewport(t.touches[1].clientX,t.touches[1].clientY);a[2]=o[0],a[3]=o[1]}if(t.touches[2]){o=e.projectIntoViewport(t.touches[2].clientX,t.touches[2].clientY);a[4]=o[0],a[5]=o[1]}if(t.touches[1]){e.touchData.singleTouchMoved=!0,d(e.dragData.touchDragEles);var l=e.findContainerClientCoords();A=l[0],M=l[1],R=l[2],I=l[3],k=t.touches[0].clientX-A,T=t.touches[0].clientY-M,C=t.touches[1].clientX-A,P=t.touches[1].clientY-M,N=0<=k&&k<=R&&0<=C&&C<=R&&0<=T&&T<=I&&0<=P&&P<=I;var h=n.pan(),p=n.zoom();S=q(k,T,C,P),B=W(k,T,C,P),_=[((D=[(k+C)/2,(T+P)/2])[0]-h.x)/p,(D[1]-h.y)/p];if(B<4e4&&!t.touches[2]){var g=e.findNearestElement(a[0],a[1],!0,!0),v=e.findNearestElement(a[2],a[3],!0,!0);return g&&g.isNode()?(g.activate().emit({originalEvent:t,type:"cxttapstart",position:{x:a[0],y:a[1]}}),e.touchData.start=g):v&&v.isNode()?(v.activate().emit({originalEvent:t,type:"cxttapstart",position:{x:a[0],y:a[1]}}),e.touchData.start=v):n.emit({originalEvent:t,type:"cxttapstart",position:{x:a[0],y:a[1]}}),e.touchData.start&&(e.touchData.start._private.grabbed=!1),e.touchData.cxt=!0,e.touchData.cxtDragged=!1,e.data.bgActivePosistion=void 0,void e.redraw()}}if(t.touches[2])n.boxSelectionEnabled()&&t.preventDefault();else if(t.touches[1]);else if(t.touches[0]){var y=e.findNearestElements(a[0],a[1],!0,!0),b=y[0];if(null!=b&&(b.activate(),e.touchData.start=b,e.touchData.starts=y,e.nodeIsGrabbable(b))){var x=e.dragData.touchDragEles=n.collection(),w=null;e.redrawHint("eles",!0),e.redrawHint("drag",!0),b.selected()?(w=n.$((function(t){return t.selected()&&e.nodeIsGrabbable(t)})),u(w,{addToList:x})):c(b,{addToList:x}),s(b);var E=function(e){return{originalEvent:t,type:e,position:{x:a[0],y:a[1]}}};b.emit(E("grabon")),w?w.forEach((function(e){e.emit(E("grab"))})):b.emit(E("grab"))}r(b,["touchstart","tapstart","vmousedown"],t,{x:a[0],y:a[1]}),null==b&&(e.data.bgActivePosistion={x:o[0],y:o[1]},e.redrawHint("select",!0),e.redraw()),e.touchData.singleTouchMoved=!1,e.touchData.singleTouchStartTime=+new Date,clearTimeout(e.touchData.tapholdTimeout),e.touchData.tapholdTimeout=setTimeout((function(){!1!==e.touchData.singleTouchMoved||e.pinching||e.touchData.selecting||r(e.touchData.start,["taphold"],t,{x:a[0],y:a[1]})}),e.tapholdDuration)}if(t.touches.length>=1){for(var L=e.touchData.startPosition=[null,null,null,null,null,null],z=0;z=e.touchTapThreshold2}if(n&&e.touchData.cxt){t.preventDefault();var w=t.touches[0].clientX-A,E=t.touches[0].clientY-M,D=t.touches[1].clientX-A,R=t.touches[1].clientY-M,I=W(w,E,D,R);if(I/B>=2.25||I>=22500){e.touchData.cxt=!1,e.data.bgActivePosistion=void 0,e.redrawHint("select",!0);var L={originalEvent:t,type:"cxttapend",position:{x:s[0],y:s[1]}};e.touchData.start?(e.touchData.start.unactivate().emit(L),e.touchData.start=null):o.emit(L)}}if(n&&e.touchData.cxt){L={originalEvent:t,type:"cxtdrag",position:{x:s[0],y:s[1]}};e.data.bgActivePosistion=void 0,e.redrawHint("select",!0),e.touchData.start?e.touchData.start.emit(L):o.emit(L),e.touchData.start&&(e.touchData.start._private.grabbed=!1),e.touchData.cxtDragged=!0;var z=e.findNearestElement(s[0],s[1],!0,!0);e.touchData.cxtOver&&z===e.touchData.cxtOver||(e.touchData.cxtOver&&e.touchData.cxtOver.emit({originalEvent:t,type:"cxtdragout",position:{x:s[0],y:s[1]}}),e.touchData.cxtOver=z,z&&z.emit({originalEvent:t,type:"cxtdragover",position:{x:s[0],y:s[1]}}))}else if(n&&t.touches[2]&&o.boxSelectionEnabled())t.preventDefault(),e.data.bgActivePosistion=void 0,this.lastThreeTouch=+new Date,e.touchData.selecting||o.emit({originalEvent:t,type:"boxstart",position:{x:s[0],y:s[1]}}),e.touchData.selecting=!0,e.touchData.didSelect=!0,a[4]=1,a&&0!==a.length&&void 0!==a[0]?(a[2]=(s[0]+s[2]+s[4])/3,a[3]=(s[1]+s[3]+s[5])/3):(a[0]=(s[0]+s[2]+s[4])/3,a[1]=(s[1]+s[3]+s[5])/3,a[2]=(s[0]+s[2]+s[4])/3+1,a[3]=(s[1]+s[3]+s[5])/3+1),e.redrawHint("select",!0),e.redraw();else if(n&&t.touches[1]&&!e.touchData.didSelect&&o.zoomingEnabled()&&o.panningEnabled()&&o.userZoomingEnabled()&&o.userPanningEnabled()){if(t.preventDefault(),e.data.bgActivePosistion=void 0,e.redrawHint("select",!0),ee=e.dragData.touchDragEles){e.redrawHint("drag",!0);for(var O=0;O0&&!e.hoverData.draggingEles&&!e.swipePanning&&null!=e.data.bgActivePosistion&&(e.data.bgActivePosistion=void 0,e.redrawHint("select",!0),e.redraw())}},!1),e.registerBinding(t,"touchcancel",V=function(t){var n=e.touchData.start;e.touchData.capture=!1,n&&n.unactivate()}),e.registerBinding(t,"touchend",F=function(t){var a=e.touchData.start;if(e.touchData.capture){0===t.touches.length&&(e.touchData.capture=!1),t.preventDefault();var i=e.selection;e.swipePanning=!1,e.hoverData.draggingEles=!1;var o,s=e.cy,l=s.zoom(),u=e.touchData.now,c=e.touchData.earlier;if(t.touches[0]){var h=e.projectIntoViewport(t.touches[0].clientX,t.touches[0].clientY);u[0]=h[0],u[1]=h[1]}if(t.touches[1]){h=e.projectIntoViewport(t.touches[1].clientX,t.touches[1].clientY);u[2]=h[0],u[3]=h[1]}if(t.touches[2]){h=e.projectIntoViewport(t.touches[2].clientX,t.touches[2].clientY);u[4]=h[0],u[5]=h[1]}if(a&&a.unactivate(),e.touchData.cxt){if(o={originalEvent:t,type:"cxttapend",position:{x:u[0],y:u[1]}},a?a.emit(o):s.emit(o),!e.touchData.cxtDragged){var f={originalEvent:t,type:"cxttap",position:{x:u[0],y:u[1]}};a?a.emit(f):s.emit(f)}return e.touchData.start&&(e.touchData.start._private.grabbed=!1),e.touchData.cxt=!1,e.touchData.start=null,void e.redraw()}if(!t.touches[2]&&s.boxSelectionEnabled()&&e.touchData.selecting){e.touchData.selecting=!1;var p=s.collection(e.getAllInBox(i[0],i[1],i[2],i[3]));i[0]=void 0,i[1]=void 0,i[2]=void 0,i[3]=void 0,i[4]=0,e.redrawHint("select",!0),s.emit({type:"boxend",originalEvent:t,position:{x:u[0],y:u[1]}});p.emit("box").stdFilter((function(e){return e.selectable()&&!e.selected()})).select().emit("boxselect"),p.nonempty()&&e.redrawHint("eles",!0),e.redraw()}if(null!=a&&a.unactivate(),t.touches[2])e.data.bgActivePosistion=void 0,e.redrawHint("select",!0);else if(t.touches[1]);else if(t.touches[0]);else if(!t.touches[0]){e.data.bgActivePosistion=void 0,e.redrawHint("select",!0);var g=e.dragData.touchDragEles;if(null!=a){var v=a._private.grabbed;d(g),e.redrawHint("drag",!0),e.redrawHint("eles",!0),v&&(a.emit("freeon"),g.emit("free"),e.dragData.didDrag&&(a.emit("dragfreeon"),g.emit("dragfree"))),r(a,["touchend","tapend","vmouseup","tapdragout"],t,{x:u[0],y:u[1]}),a.unactivate(),e.touchData.start=null}else{var y=e.findNearestElement(u[0],u[1],!0,!0);r(y,["touchend","tapend","vmouseup","tapdragout"],t,{x:u[0],y:u[1]})}var m=e.touchData.startPosition[0]-u[0],b=m*m,x=e.touchData.startPosition[1]-u[1],w=(b+x*x)*l*l;e.touchData.singleTouchMoved||(a||s.$(":selected").unselect(["tapunselect"]),r(a,["tap","vclick"],t,{x:u[0],y:u[1]}),j=!1,t.timeStamp-Y<=s.multiClickDebounceTime()?(X&&clearTimeout(X),j=!0,Y=null,r(a,["dbltap","vdblclick"],t,{x:u[0],y:u[1]})):(X=setTimeout((function(){j||r(a,["onetap","voneclick"],t,{x:u[0],y:u[1]})}),s.multiClickDebounceTime()),Y=t.timeStamp)),null!=a&&!e.dragData.didDrag&&a._private.selectable&&w2){for(var f=[c[0],c[1]],p=Math.pow(f[0]-e,2)+Math.pow(f[1]-t,2),g=1;g0)return g[0]}return null},f=Object.keys(d),p=0;p0?u:ln(a,i,e,t,n,r,o,s)},checkPoint:function(e,t,n,r,a,i,o,s){var l=2*(s="auto"===s?Pn(r,a):s);if(pn(e,t,this.points,i,o,r,a-l,[0,-1],n))return!0;if(pn(e,t,this.points,i,o,r-l,a,[0,-1],n))return!0;var u=r/2+2*n,c=a/2+2*n;return!!fn(e,t,[i-u,o-c,i-u,o,i+u,o,i+u,o-c])||(!!yn(e,t,l,l,i+r/2-s,o+a/2-s,n)||!!yn(e,t,l,l,i-r/2+s,o+a/2-s,n))}}},Hc.registerNodeShapes=function(){var e=this.nodeShapes={},t=this;this.generateEllipse(),this.generatePolygon("triangle",kn(3,0)),this.generateRoundPolygon("round-triangle",kn(3,0)),this.generatePolygon("rectangle",kn(4,0)),e.square=e.rectangle,this.generateRoundRectangle(),this.generateCutRectangle(),this.generateBarrel(),this.generateBottomRoundrectangle();var n=[0,1,1,0,0,-1,-1,0];this.generatePolygon("diamond",n),this.generateRoundPolygon("round-diamond",n),this.generatePolygon("pentagon",kn(5,0)),this.generateRoundPolygon("round-pentagon",kn(5,0)),this.generatePolygon("hexagon",kn(6,0)),this.generateRoundPolygon("round-hexagon",kn(6,0)),this.generatePolygon("heptagon",kn(7,0)),this.generateRoundPolygon("round-heptagon",kn(7,0)),this.generatePolygon("octagon",kn(8,0)),this.generateRoundPolygon("round-octagon",kn(8,0));var r=new Array(20),a=Cn(5,0),i=Cn(5,Math.PI/5),o=.5*(3-Math.sqrt(5));o*=1.57;for(var s=0;s=e.deqFastCost*g)break}else if(a){if(f>=e.deqCost*l||f>=e.deqAvgCost*s)break}else if(p>=e.deqNoDrawCost*Jc)break;var v=e.deq(t,d,c);if(!(v.length>0))break;for(var y=0;y0&&(e.onDeqd(t,u),!a&&e.shouldRedraw(t,u,d,c)&&r())}),a(t))}}},td=function(){return i((function e(t){var n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:Je;a(this,e),this.idsByKey=new pt,this.keyForId=new pt,this.cachesByLvl=new pt,this.lvls=[],this.getKey=t,this.doesEleInvalidateKey=n}),[{key:"getIdsFor",value:function(e){null==e&&nt("Can not get id list for null key");var t=this.idsByKey,n=this.idsByKey.get(e);return n||(n=new vt,t.set(e,n)),n}},{key:"addIdForKey",value:function(e,t){null!=e&&this.getIdsFor(e).add(t)}},{key:"deleteIdForKey",value:function(e,t){null!=e&&this.getIdsFor(e).delete(t)}},{key:"getNumberOfIdsForKey",value:function(e){return null==e?0:this.getIdsFor(e).size}},{key:"updateKeyMappingFor",value:function(e){var t=e.id(),n=this.keyForId.get(t),r=this.getKey(e);this.deleteIdForKey(n,t),this.addIdForKey(r,t),this.keyForId.set(t,r)}},{key:"deleteKeyMappingFor",value:function(e){var t=e.id(),n=this.keyForId.get(t);this.deleteIdForKey(n,t),this.keyForId.delete(t)}},{key:"keyHasChangedFor",value:function(e){var t=e.id();return this.keyForId.get(t)!==this.getKey(e)}},{key:"isInvalid",value:function(e){return this.keyHasChangedFor(e)||this.doesEleInvalidateKey(e)}},{key:"getCachesAt",value:function(e){var t=this.cachesByLvl,n=this.lvls,r=t.get(e);return r||(r=new pt,t.set(e,r),n.push(e)),r}},{key:"getCache",value:function(e,t){return this.getCachesAt(t).get(e)}},{key:"get",value:function(e,t){var n=this.getKey(e),r=this.getCache(n,t);return null!=r&&this.updateKeyMappingFor(e),r}},{key:"getForCachedKey",value:function(e,t){var n=this.keyForId.get(e.id());return this.getCache(n,t)}},{key:"hasCache",value:function(e,t){return this.getCachesAt(t).has(e)}},{key:"has",value:function(e,t){var n=this.getKey(e);return this.hasCache(n,t)}},{key:"setCache",value:function(e,t,n){n.key=e,this.getCachesAt(t).set(e,n)}},{key:"set",value:function(e,t,n){var r=this.getKey(e);this.setCache(r,t,n),this.updateKeyMappingFor(e)}},{key:"deleteCache",value:function(e,t){this.getCachesAt(t).delete(e)}},{key:"delete",value:function(e,t){var n=this.getKey(e);this.deleteCache(n,t)}},{key:"invalidateKey",value:function(e){var t=this;this.lvls.forEach((function(n){return t.deleteCache(e,n)}))}},{key:"invalidate",value:function(e){var t=e.id(),n=this.keyForId.get(t);this.deleteKeyMappingFor(e);var r=this.doesEleInvalidateKey(e);return r&&this.invalidateKey(n),r||0===this.getNumberOfIdsForKey(n)}}])}(),nd=7.99,rd={dequeue:"dequeue",downscale:"downscale",highQuality:"highQuality"},ad=ut({getKey:null,doesEleInvalidateKey:Je,drawElement:null,getBoundingBox:null,getRotationPoint:null,getRotationOffset:null,isVisible:Qe,allowEdgeTxrCaching:!0,allowParentTxrCaching:!0}),id=function(e,t){var n=this;n.renderer=e,n.onDequeues=[];var r=ad(t);me(n,r),n.lookup=new td(r.getKey,r.doesEleInvalidateKey),n.setupDequeueing()},od=id.prototype;od.reasons=rd,od.getTextureQueue=function(e){var t=this;return t.eleImgCaches=t.eleImgCaches||{},t.eleImgCaches[e]=t.eleImgCaches[e]||[]},od.getRetiredTextureQueue=function(e){var t=this.eleImgCaches.retired=this.eleImgCaches.retired||{};return t[e]=t[e]||[]},od.getElementQueue=function(){return this.eleCacheQueue=this.eleCacheQueue||new Pt((function(e,t){return t.reqs-e.reqs}))},od.getElementKeyToQueue=function(){return this.eleKeyToCacheQueue=this.eleKeyToCacheQueue||{}},od.getElement=function(e,t,n,r,a){var i=this,o=this.renderer,s=o.cy.zoom(),l=this.lookup;if(!t||0===t.w||0===t.h||isNaN(t.w)||isNaN(t.h)||!e.visible()||e.removed())return null;if(!i.allowEdgeTxrCaching&&e.isEdge()||!i.allowParentTxrCaching&&e.isParent())return null;if(null==r&&(r=Math.ceil(qt(s*n))),r<-4)r=-4;else if(s>=7.99||r>3)return null;var u=Math.pow(2,r),c=t.h*u,d=t.w*u,h=o.eleTextBiggerThanMin(e,u);if(!this.isVisible(e,h))return null;var f,p=l.get(e,r);if(p&&p.invalidated&&(p.invalidated=!1,p.texture.invalidatedWidth-=p.width),p)return p;if(f=c<=25?25:c<=50?50:50*Math.ceil(c/50),c>1024||d>1024)return null;var g=i.getTextureQueue(f),v=g[g.length-2],y=function(){return i.recycleTexture(f,d)||i.addTexture(f,d)};v||(v=g[g.length-1]),v||(v=y()),v.width-v.usedWidthr;S--)C=i.getElement(e,t,n,S,rd.downscale);P()}else{var B;if(!x&&!w&&!E)for(var D=r-1;D>=-4;D--){var _=l.get(e,D);if(_){B=_;break}}if(b(B))return i.queueElement(e,r),B;v.context.translate(v.usedWidth,0),v.context.scale(u,u),this.drawElement(v.context,e,t,h,!1),v.context.scale(1/u,1/u),v.context.translate(-v.usedWidth,0)}return p={x:v.usedWidth,texture:v,level:r,scale:u,width:d,height:c,scaledLabelShown:h},v.usedWidth+=Math.ceil(d+8),v.eleCaches.push(p),l.set(e,r,p),i.checkTextureFullness(v),p},od.invalidateElements=function(e){for(var t=0;t=.2*e.width&&this.retireTexture(e)},od.checkTextureFullness=function(e){var t=this.getTextureQueue(e.height);e.usedWidth/e.width>.8&&e.fullnessChecks>=10?ct(t,e):e.fullnessChecks++},od.retireTexture=function(e){var t=e.height,n=this.getTextureQueue(t),r=this.lookup;ct(n,e),e.retired=!0;for(var a=e.eleCaches,i=0;i=t)return i.retired=!1,i.usedWidth=0,i.invalidatedWidth=0,i.fullnessChecks=0,dt(i.eleCaches),i.context.setTransform(1,0,0,1,0,0),i.context.clearRect(0,0,i.width,i.height),ct(r,i),n.push(i),i}},od.queueElement=function(e,t){var n=this.getElementQueue(),r=this.getElementKeyToQueue(),a=this.getKey(e),i=r[a];if(i)i.level=Math.max(i.level,t),i.eles.merge(e),i.reqs++,n.updateItem(i);else{var o={eles:e.spawn().merge(e),level:t,reqs:1,key:a};n.push(o),r[a]=o}},od.dequeue=function(e){for(var t=this,n=t.getElementQueue(),r=t.getElementKeyToQueue(),a=[],i=t.lookup,o=0;o<1&&n.size()>0;o++){var s=n.pop(),l=s.key,u=s.eles[0],c=i.hasCache(u,s.level);if(r[l]=null,!c){a.push(s);var d=t.getBoundingBox(u);t.getElement(u,d,e,s.level,rd.dequeue)}}return a},od.removeFromQueue=function(e){var t=this.getElementQueue(),n=this.getElementKeyToQueue(),r=this.getKey(e),a=n[r];null!=a&&(1===a.eles.length?(a.reqs=$e,t.updateItem(a),t.pop(),n[r]=null):a.eles.unmerge(e))},od.onDequeue=function(e){this.onDequeues.push(e)},od.offDequeue=function(e){ct(this.onDequeues,e)},od.setupDequeueing=ed({deqRedrawThreshold:100,deqCost:.15,deqAvgCost:.1,deqNoDrawCost:.9,deqFastCost:.9,deq:function(e,t,n){return e.dequeue(t,n)},onDeqd:function(e,t){for(var n=0;n=3.99||n>2)return null;r.validateLayersElesOrdering(n,e);var o,s,l=r.layersByLevel,u=Math.pow(2,n),c=l[n]=l[n]||[];if(r.levelIsComplete(n,e))return c;!function(){var t=function(t){if(r.validateLayersElesOrdering(t,e),r.levelIsComplete(t,e))return s=l[t],!0},a=function(e){if(!s)for(var r=n+e;-4<=r&&r<=2&&!t(r);r+=e);};a(1),a(-1);for(var i=c.length-1;i>=0;i--){var o=c[i];o.invalid&&ct(c,o)}}();var d=function(t){var a=(t=t||{}).after;!function(){if(!o){o=Qt();for(var t=0;t32767||s>32767)return null;if(i*s>16e6)return null;var l=r.makeLayer(o,n);if(null!=a){var d=c.indexOf(a)+1;c.splice(d,0,l)}else(void 0===t.insert||t.insert)&&c.unshift(l);return l};if(r.skipping&&!i)return null;for(var h=null,f=e.length/1,p=!i,g=0;g=f||!sn(h.bb,v.boundingBox()))&&!(h=d({insert:!0,after:h})))return null;s||p?r.queueLayer(h,v):r.drawEleInLayer(h,v,n,t),h.eles.push(v),m[n]=h}}return s||(p?null:c)},ld.getEleLevelForLayerLevel=function(e,t){return e},ld.drawEleInLayer=function(e,t,n,r){var a=this.renderer,i=e.context,o=t.boundingBox();0!==o.w&&0!==o.h&&t.visible()&&(n=this.getEleLevelForLayerLevel(n,r),a.setImgSmoothing(i,!1),a.drawCachedElement(i,t,null,null,n,true),a.setImgSmoothing(i,!0))},ld.levelIsComplete=function(e,t){var n=this.layersByLevel[e];if(!n||0===n.length)return!1;for(var r=0,a=0;a0)return!1;if(i.invalid)return!1;r+=i.eles.length}return r===t.length},ld.validateLayersElesOrdering=function(e,t){var n=this.layersByLevel[e];if(n)for(var r=0;r0){e=!0;break}}return e},ld.invalidateElements=function(e){var t=this;0!==e.length&&(t.lastInvalidationTime=Le(),0!==e.length&&t.haveLayers()&&t.updateElementsInLayers(e,(function(e,n,r){t.invalidateLayer(e)})))},ld.invalidateLayer=function(e){if(this.lastInvalidationTime=Le(),!e.invalid){var t=e.level,n=e.eles,r=this.layersByLevel[t];ct(r,e),e.elesQueue=[],e.invalid=!0,e.replacement&&(e.replacement.invalid=!0);for(var a=0;a3&&void 0!==arguments[3])||arguments[3],a=!(arguments.length>4&&void 0!==arguments[4])||arguments[4],i=!(arguments.length>5&&void 0!==arguments[5])||arguments[5],o=this,s=t._private.rscratch;if((!i||t.visible())&&!s.badLine&&null!=s.allpts&&!isNaN(s.allpts[0])){var l;n&&(l=n,e.translate(-l.x1,-l.y1));var u=i?t.pstyle("opacity").value:1,c=i?t.pstyle("line-opacity").value:1,d=t.pstyle("curve-style").value,h=t.pstyle("line-style").value,f=t.pstyle("width").pfValue,p=t.pstyle("line-cap").value,g=t.pstyle("line-outline-width").value,v=t.pstyle("line-outline-color").value,y=u*c,m=u*c,b=function(){var n=arguments.length>0&&void 0!==arguments[0]?arguments[0]:y;"straight-triangle"===d?(o.eleStrokeStyle(e,t,n),o.drawEdgeTrianglePath(t,e,s.allpts)):(e.lineWidth=f,e.lineCap=p,o.eleStrokeStyle(e,t,n),o.drawEdgePath(t,e,s.allpts,h),e.lineCap="butt")},x=function(){var n=arguments.length>0&&void 0!==arguments[0]?arguments[0]:m;o.drawArrowheads(e,t,n)};if(e.lineJoin="round","yes"===t.pstyle("ghost").value){var w=t.pstyle("ghost-offset-x").pfValue,E=t.pstyle("ghost-offset-y").pfValue,k=t.pstyle("ghost-opacity").value,T=y*k;e.translate(w,E),b(T),x(T),e.translate(-w,-E)}else!function(){var n=arguments.length>0&&void 0!==arguments[0]?arguments[0]:y;e.lineWidth=f+g,e.lineCap=p,g>0?(o.colorStrokeStyle(e,v[0],v[1],v[2],n),"straight-triangle"===d?o.drawEdgeTrianglePath(t,e,s.allpts):(o.drawEdgePath(t,e,s.allpts,h),e.lineCap="butt")):e.lineCap="butt"}();a&&o.drawEdgeUnderlay(e,t),b(),x(),a&&o.drawEdgeOverlay(e,t),o.drawElementText(e,t,null,r),n&&e.translate(l.x1,l.y1)}}},Pd=function(e){if(!["overlay","underlay"].includes(e))throw new Error("Invalid state");return function(t,n){if(n.visible()){var r=n.pstyle("".concat(e,"-opacity")).value;if(0!==r){var a=this,i=a.usePaths(),o=n._private.rscratch,s=2*n.pstyle("".concat(e,"-padding")).pfValue,l=n.pstyle("".concat(e,"-color")).value;t.lineWidth=s,"self"!==o.edgeType||i?t.lineCap="round":t.lineCap="butt",a.colorStrokeStyle(t,l[0],l[1],l[2],r),a.drawEdgePath(n,t,o.allpts,"solid")}}}};Cd.drawEdgeOverlay=Pd("overlay"),Cd.drawEdgeUnderlay=Pd("underlay"),Cd.drawEdgePath=function(e,t,n,r){var a,i=e._private.rscratch,s=t,l=!1,u=this.usePaths(),c=e.pstyle("line-dash-pattern").pfValue,d=e.pstyle("line-dash-offset").pfValue;if(u){var h=n.join("$");i.pathCacheKey&&i.pathCacheKey===h?(a=t=i.pathCache,l=!0):(a=t=new Path2D,i.pathCacheKey=h,i.pathCache=a)}if(s.setLineDash)switch(r){case"dotted":s.setLineDash([1,1]);break;case"dashed":s.setLineDash(c),s.lineDashOffset=d;break;case"solid":s.setLineDash([])}if(!l&&!i.badLine)switch(t.beginPath&&t.beginPath(),t.moveTo(n[0],n[1]),i.edgeType){case"bezier":case"self":case"compound":case"multibezier":for(var f=2;f+35&&void 0!==arguments[5]?arguments[5]:5,o=arguments.length>6?arguments[6]:void 0;e.beginPath(),e.moveTo(t+i,n),e.lineTo(t+r-i,n),e.quadraticCurveTo(t+r,n,t+r,n+i),e.lineTo(t+r,n+a-i),e.quadraticCurveTo(t+r,n+a,t+r-i,n+a),e.lineTo(t+i,n+a),e.quadraticCurveTo(t,n+a,t,n+a-i),e.lineTo(t,n+i),e.quadraticCurveTo(t,n,t+i,n),e.closePath(),o?e.stroke():e.fill()}Bd.eleTextBiggerThanMin=function(e,t){if(!t){var n=e.cy().zoom(),r=this.getPixelRatio(),a=Math.ceil(qt(n*r));t=Math.pow(2,a)}return!(e.pstyle("font-size").pfValue*t5&&void 0!==arguments[5])||arguments[5],o=this;if(null==r){if(i&&!o.eleTextBiggerThanMin(t))return}else if(!1===r)return;if(t.isNode()){var s=t.pstyle("label");if(!s||!s.value)return;var l=o.getLabelJustification(t);e.textAlign=l,e.textBaseline="bottom"}else{var u=t.element()._private.rscratch.badLine,c=t.pstyle("label"),d=t.pstyle("source-label"),h=t.pstyle("target-label");if(u||(!c||!c.value)&&(!d||!d.value)&&(!h||!h.value))return;e.textAlign="center",e.textBaseline="bottom"}var f,p=!n;n&&(f=n,e.translate(-f.x1,-f.y1)),null==a?(o.drawText(e,t,null,p,i),t.isEdge()&&(o.drawText(e,t,"source",p,i),o.drawText(e,t,"target",p,i))):o.drawText(e,t,a,p,i),n&&e.translate(f.x1,f.y1)},Bd.getFontCache=function(e){var t;this.fontCaches=this.fontCaches||[];for(var n=0;n2&&void 0!==arguments[2])||arguments[2],r=t.pstyle("font-style").strValue,a=t.pstyle("font-size").pfValue+"px",i=t.pstyle("font-family").strValue,o=t.pstyle("font-weight").strValue,s=n?t.effectiveOpacity()*t.pstyle("text-opacity").value:1,l=t.pstyle("text-outline-opacity").value*s,u=t.pstyle("color").value,c=t.pstyle("text-outline-color").value;e.font=r+" "+o+" "+a+" "+i,e.lineJoin="round",this.colorFillStyle(e,u[0],u[1],u[2],s),this.colorStrokeStyle(e,c[0],c[1],c[2],l)},Bd.getTextAngle=function(e,t){var n,r=e._private.rscratch,a=t?t+"-":"",i=e.pstyle(a+"text-rotation");if("autorotate"===i.strValue){var o=ht(r,"labelAngle",t);n=e.isEdge()?o:0}else n="none"===i.strValue?0:i.pfValue;return n},Bd.drawText=function(e,t,n){var r=!(arguments.length>3&&void 0!==arguments[3])||arguments[3],a=!(arguments.length>4&&void 0!==arguments[4])||arguments[4],i=t._private.rscratch,o=a?t.effectiveOpacity():1;if(!a||0!==o&&0!==t.pstyle("text-opacity").value){"main"===n&&(n=null);var s,l,u=ht(i,"labelX",n),c=ht(i,"labelY",n),d=this.getLabelText(t,n);if(null!=d&&""!==d&&!isNaN(u)&&!isNaN(c)){this.setupTextStyle(e,t,a);var h,f=n?n+"-":"",p=ht(i,"labelWidth",n),g=ht(i,"labelHeight",n),v=t.pstyle(f+"text-margin-x").pfValue,y=t.pstyle(f+"text-margin-y").pfValue,m=t.isEdge(),b=t.pstyle("text-halign").value,x=t.pstyle("text-valign").value;switch(m&&(b="center",x="center"),u+=v,c+=y,0!==(h=r?this.getTextAngle(t,n):0)&&(s=u,l=c,e.translate(s,l),e.rotate(h),u=0,c=0),x){case"top":break;case"center":c+=g/2;break;case"bottom":c+=g}var w=t.pstyle("text-background-opacity").value,E=t.pstyle("text-border-opacity").value,k=t.pstyle("text-border-width").pfValue,T=t.pstyle("text-background-padding").pfValue,C=0===t.pstyle("text-background-shape").strValue.indexOf("round");if(w>0||k>0&&E>0){var P=u-T;switch(b){case"left":P-=p;break;case"center":P-=p/2}var S=c-g-T,B=p+2*T,D=g+2*T;if(w>0){var _=e.fillStyle,A=t.pstyle("text-background-color").value;e.fillStyle="rgba("+A[0]+","+A[1]+","+A[2]+","+w*o+")",C?Dd(e,P,S,B,D,2):e.fillRect(P,S,B,D),e.fillStyle=_}if(k>0&&E>0){var M=e.strokeStyle,R=e.lineWidth,I=t.pstyle("text-border-color").value,N=t.pstyle("text-border-style").value;if(e.strokeStyle="rgba("+I[0]+","+I[1]+","+I[2]+","+E*o+")",e.lineWidth=k,e.setLineDash)switch(N){case"dotted":e.setLineDash([1,1]);break;case"dashed":e.setLineDash([4,2]);break;case"double":e.lineWidth=k/4,e.setLineDash([]);break;case"solid":e.setLineDash([])}if(C?Dd(e,P,S,B,D,2,"stroke"):e.strokeRect(P,S,B,D),"double"===N){var L=k/2;C?Dd(e,P+L,S+L,B-2*L,D-2*L,2,"stroke"):e.strokeRect(P+L,S+L,B-2*L,D-2*L)}e.setLineDash&&e.setLineDash([]),e.lineWidth=R,e.strokeStyle=M}}var z=2*t.pstyle("text-outline-width").pfValue;if(z>0&&(e.lineWidth=z),"wrap"===t.pstyle("text-wrap").value){var O=ht(i,"labelWrapCachedLines",n),V=ht(i,"labelLineHeight",n),F=p/2,j=this.getLabelJustification(t);switch("auto"===j||("left"===b?"left"===j?u+=-p:"center"===j&&(u+=-F):"center"===b?"left"===j?u+=-F:"right"===j&&(u+=F):"right"===b&&("center"===j?u+=F:"right"===j&&(u+=p))),x){case"top":case"center":case"bottom":c-=(O.length-1)*V}for(var X=0;X0&&e.strokeText(O[X],u,c),e.fillText(O[X],u,c),c+=V}else z>0&&e.strokeText(d,u,c),e.fillText(d,u,c);0!==h&&(e.rotate(-h),e.translate(-s,-l))}}};var _d={drawNode:function(e,t,n){var r,a,i=!(arguments.length>3&&void 0!==arguments[3])||arguments[3],o=!(arguments.length>4&&void 0!==arguments[4])||arguments[4],s=!(arguments.length>5&&void 0!==arguments[5])||arguments[5],l=this,u=t._private,c=u.rscratch,d=t.position();if(Q(d.x)&&Q(d.y)&&(!s||t.visible())){var h,f,p=s?t.effectiveOpacity():1,g=l.usePaths(),v=!1,y=t.padding();r=t.width()+2*y,a=t.height()+2*y,n&&(f=n,e.translate(-f.x1,-f.y1));for(var m=t.pstyle("background-image").value,b=new Array(m.length),x=new Array(m.length),w=0,E=0;E0&&void 0!==arguments[0]?arguments[0]:S;l.eleFillStyle(e,t,n)},Y=function(){var t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:N;l.colorStrokeStyle(e,B[0],B[1],B[2],t)},q=function(){var t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:V;l.colorStrokeStyle(e,z[0],z[1],z[2],t)},W=function(e,t,n,r){var a,i=l.nodePathCache=l.nodePathCache||[],o=Ke("polygon"===n?n+","+r.join(","):n,""+t,""+e,""+j),s=i[o],u=!1;return null!=s?(a=s,u=!0,c.pathCache=a):(a=new Path2D,i[o]=c.pathCache=a),{path:a,cacheHit:u}},K=t.pstyle("shape").strValue,U=t.pstyle("shape-polygon-points").pfValue;if(g){e.translate(d.x,d.y);var H=W(r,a,K,U);h=H.path,v=H.cacheHit}var G=function(){if(!v){var n=d;g&&(n={x:0,y:0}),l.nodeShapes[l.getNodeShape(t)].draw(h||e,n.x,n.y,r,a,j,c)}g?e.fill(h):e.fill()},Z=function(){for(var n=arguments.length>0&&void 0!==arguments[0]?arguments[0]:p,r=!(arguments.length>1&&void 0!==arguments[1])||arguments[1],a=u.backgrounding,i=0,o=0;o0&&void 0!==arguments[0]&&arguments[0],i=arguments.length>1&&void 0!==arguments[1]?arguments[1]:p;l.hasPie(t)&&(l.drawPie(e,t,i),n&&(g||l.nodeShapes[l.getNodeShape(t)].draw(e,d.x,d.y,r,a,j,c)))},J=function(){var t=(C>0?C:-C)*(arguments.length>0&&void 0!==arguments[0]?arguments[0]:p),n=C>0?0:255;0!==C&&(l.colorFillStyle(e,n,n,n,t),g?e.fill(h):e.fill())},ee=function(){if(P>0){if(e.lineWidth=P,e.lineCap=A,e.lineJoin=_,e.setLineDash)switch(D){case"dotted":e.setLineDash([1,1]);break;case"dashed":e.setLineDash(R),e.lineDashOffset=I;break;case"solid":case"double":e.setLineDash([])}if("center"!==M){if(e.save(),e.lineWidth*=2,"inside"===M)g?e.clip(h):e.clip();else{var t=new Path2D;t.rect(-r/2-P,-a/2-P,r+2*P,a+2*P),t.addPath(h),e.clip(t,"evenodd")}g?e.stroke(h):e.stroke(),e.restore()}else g?e.stroke(h):e.stroke();if("double"===D){e.lineWidth=P/3;var n=e.globalCompositeOperation;e.globalCompositeOperation="destination-out",g?e.stroke(h):e.stroke(),e.globalCompositeOperation=n}e.setLineDash&&e.setLineDash([])}},te=function(){if(L>0){if(e.lineWidth=L,e.lineCap="butt",e.setLineDash)switch(O){case"dotted":e.setLineDash([1,1]);break;case"dashed":e.setLineDash([4,2]);break;case"solid":case"double":e.setLineDash([])}var n=d;g&&(n={x:0,y:0});var i=l.getNodeShape(t),o=P;"inside"===M&&(o=0),"outside"===M&&(o*=2);var s,u=(r+o+(L+F))/r,c=(a+o+(L+F))/a,h=r*u,f=a*c,p=l.nodeShapes[i].points;if(g)s=W(h,f,i,p).path;if("ellipse"===i)l.drawEllipsePath(s||e,n.x,n.y,h,f);else if(["round-diamond","round-heptagon","round-hexagon","round-octagon","round-pentagon","round-polygon","round-triangle","round-tag"].includes(i)){var v=0,y=0,m=0;"round-diamond"===i?v=1.4*(o+F+L):"round-heptagon"===i?(v=1.075*(o+F+L),m=-(o/2+F+L)/35):"round-hexagon"===i?v=1.12*(o+F+L):"round-pentagon"===i?(v=1.13*(o+F+L),m=-(o/2+F+L)/15):"round-tag"===i?(v=1.12*(o+F+L),y=.07*(o/2+L+F)):"round-triangle"===i&&(v=(o+F+L)*(Math.PI/2),m=-(o+F/2+L)/Math.PI),0!==v&&(h=r*(u=(r+v)/r),["round-hexagon","round-tag"].includes(i)||(f=a*(c=(a+v)/a)));for(var b=h/2,x=f/2,w=(j="auto"===j?Sn(h,f):j)+(o+L+F)/2,E=new Array(p.length/2),k=new Array(p.length/2),T=0;T0){if(r=r||n.position(),null==a||null==i){var d=n.padding();a=n.width()+2*d,i=n.height()+2*d}this.colorFillStyle(t,l[0],l[1],l[2],s),this.nodeShapes[u].draw(t,r.x,r.y,a+2*o,i+2*o,c),t.fill()}}}};_d.drawNodeOverlay=Ad("overlay"),_d.drawNodeUnderlay=Ad("underlay"),_d.hasPie=function(e){return(e=e[0])._private.hasPie},_d.drawPie=function(e,t,n,r){t=t[0],r=r||t.position();var a=t.cy().style(),i=t.pstyle("pie-size"),o=r.x,s=r.y,l=t.width(),u=t.height(),c=Math.min(l,u)/2,d=0;this.usePaths()&&(o=0,s=0),"%"===i.units?c*=i.pfValue:void 0!==i.pfValue&&(c=i.pfValue/2);for(var h=1;h<=a.pieBackgroundN;h++){var f=t.pstyle("pie-"+h+"-background-size").value,p=t.pstyle("pie-"+h+"-background-color").value,g=t.pstyle("pie-"+h+"-background-opacity").value*n,v=f/100;v+d>1&&(v=1-d);var y=1.5*Math.PI+2*Math.PI*d,m=y+2*Math.PI*v;0===f||d>=1||d+v>1||(e.beginPath(),e.moveTo(o,s),e.arc(o,s,c,y,m),e.closePath(),this.colorFillStyle(e,p[0],p[1],p[2],g),e.fill(),d+=v)}};var Md,Rd={};function Id(e,t,n){var r=e.createShader(t);if(e.shaderSource(r,n),e.compileShader(r),!e.getShaderParameter(r,e.COMPILE_STATUS))throw new Error(e.getShaderInfoLog(r));return r}function Nd(e,t,n){void 0===n&&(n=t);var r=e.makeOffscreenCanvas(t,n),a=r.context=r.getContext("2d");return r.clear=function(){return a.clearRect(0,0,r.width,r.height)},r.clear(),r}function Ld(e){var t=e.pixelRatio,n=e.cy.zoom(),r=e.cy.pan();return{zoom:n*t,pan:{x:r.x*t,y:r.y*t}}}function zd(e,t,n){var r=e[0]/255,a=e[1]/255,i=e[2]/255,o=t,s=n||new Array(4);return s[0]=r*o,s[1]=a*o,s[2]=i*o,s[3]=o,s}function Od(e,t){var n=t||new Array(4);return n[0]=(255&e)/255,n[1]=(e>>8&255)/255,n[2]=(e>>16&255)/255,n[3]=(e>>24&255)/255,n}function Vd(e){return e[0]+(e[1]<<8)+(e[2]<<16)+(e[3]<<24)}function Fd(e,t){switch(t){case"float":return[1,e.FLOAT,4];case"vec2":return[2,e.FLOAT,4];case"vec3":return[3,e.FLOAT,4];case"vec4":return[4,e.FLOAT,4];case"int":return[1,e.INT,4];case"ivec2":return[2,e.INT,4]}}function jd(e,t,n){switch(t){case e.FLOAT:return new Float32Array(n);case e.INT:return new Int32Array(n)}}function Xd(e,t,n,r,a,i){switch(t){case e.FLOAT:return new Float32Array(n.buffer,i*r,a);case e.INT:return new Int32Array(n.buffer,i*r,a)}}function Yd(e,t,n,r){var a=l(Fd(e,n),3),i=a[0],o=a[1],s=a[2],u=jd(e,o,t*i),c=i*s,d=e.createBuffer();e.bindBuffer(e.ARRAY_BUFFER,d),e.bufferData(e.ARRAY_BUFFER,t*c,e.DYNAMIC_DRAW),e.enableVertexAttribArray(r),o===e.FLOAT?e.vertexAttribPointer(r,i,o,!1,c,0):o===e.INT&&e.vertexAttribIPointer(r,i,o,c,0),e.vertexAttribDivisor(r,1),e.bindBuffer(e.ARRAY_BUFFER,null);for(var h=new Array(t),f=0;ft.minMbLowQualFrames&&(t.motionBlurPxRatio=t.mbPxRBlurry)),t.clearingMotionBlur&&(t.motionBlurPxRatio=1),t.textureDrawLastFrame&&!d&&(c[t.NODE]=!0,c[t.SELECT_BOX]=!0);var m=n.style(),b=n.zoom(),x=void 0!==o?o:b,w=n.pan(),E={x:w.x,y:w.y},k={zoom:b,pan:{x:w.x,y:w.y}},T=t.prevViewport;void 0===T||k.zoom!==T.zoom||k.pan.x!==T.pan.x||k.pan.y!==T.pan.y||g&&!p||(t.motionBlurPxRatio=1),s&&(E=s),x*=l,E.x*=l,E.y*=l;var C=t.getCachedZSortedEles();function P(e,n,r,a,i){var o=e.globalCompositeOperation;e.globalCompositeOperation="destination-out",t.colorFillStyle(e,255,255,255,t.motionBlurTransparency),e.fillRect(n,r,a,i),e.globalCompositeOperation=o}function S(e,n){var i,l,c,d;t.clearingMotionBlur||e!==u.bufferContexts[t.MOTIONBLUR_BUFFER_NODE]&&e!==u.bufferContexts[t.MOTIONBLUR_BUFFER_DRAG]?(i=E,l=x,c=t.canvasWidth,d=t.canvasHeight):(i={x:w.x*f,y:w.y*f},l=b*f,c=t.canvasWidth*f,d=t.canvasHeight*f),e.setTransform(1,0,0,1,0,0),"motionBlur"===n?P(e,0,0,c,d):r||void 0!==n&&!n||e.clearRect(0,0,c,d),a||(e.translate(i.x,i.y),e.scale(l,l)),s&&e.translate(s.x,s.y),o&&e.scale(o,o)}if(d||(t.textureDrawLastFrame=!1),d){if(t.textureDrawLastFrame=!0,!t.textureCache){t.textureCache={},t.textureCache.bb=n.mutableElements().boundingBox(),t.textureCache.texture=t.data.bufferCanvases[t.TEXTURE_BUFFER];var B=t.data.bufferContexts[t.TEXTURE_BUFFER];B.setTransform(1,0,0,1,0,0),B.clearRect(0,0,t.canvasWidth*t.textureMult,t.canvasHeight*t.textureMult),t.render({forcedContext:B,drawOnlyNodeLayer:!0,forcedPxRatio:l*t.textureMult}),(k=t.textureCache.viewport={zoom:n.zoom(),pan:n.pan(),width:t.canvasWidth,height:t.canvasHeight}).mpan={x:(0-k.pan.x)/k.zoom,y:(0-k.pan.y)/k.zoom}}c[t.DRAG]=!1,c[t.NODE]=!1;var D=u.contexts[t.NODE],_=t.textureCache.texture;k=t.textureCache.viewport;D.setTransform(1,0,0,1,0,0),h?P(D,0,0,k.width,k.height):D.clearRect(0,0,k.width,k.height);var A=m.core("outside-texture-bg-color").value,M=m.core("outside-texture-bg-opacity").value;t.colorFillStyle(D,A[0],A[1],A[2],M),D.fillRect(0,0,k.width,k.height);b=n.zoom();S(D,!1),D.clearRect(k.mpan.x,k.mpan.y,k.width/k.zoom/l,k.height/k.zoom/l),D.drawImage(_,k.mpan.x,k.mpan.y,k.width/k.zoom/l,k.height/k.zoom/l)}else t.textureOnViewport&&!r&&(t.textureCache=null);var R=n.extent(),I=t.pinching||t.hoverData.dragging||t.swipePanning||t.data.wheelZooming||t.hoverData.draggingEles||t.cy.animated(),N=t.hideEdgesOnViewport&&I,L=[];if(L[t.NODE]=!c[t.NODE]&&h&&!t.clearedForMotionBlur[t.NODE]||t.clearingMotionBlur,L[t.NODE]&&(t.clearedForMotionBlur[t.NODE]=!0),L[t.DRAG]=!c[t.DRAG]&&h&&!t.clearedForMotionBlur[t.DRAG]||t.clearingMotionBlur,L[t.DRAG]&&(t.clearedForMotionBlur[t.DRAG]=!0),c[t.NODE]||a||i||L[t.NODE]){var z=h&&!L[t.NODE]&&1!==f;S(D=r||(z?t.data.bufferContexts[t.MOTIONBLUR_BUFFER_NODE]:u.contexts[t.NODE]),h&&!z?"motionBlur":void 0),N?t.drawCachedNodes(D,C.nondrag,l,R):t.drawLayeredElements(D,C.nondrag,l,R),t.debug&&t.drawDebugPoints(D,C.nondrag),a||h||(c[t.NODE]=!1)}if(!i&&(c[t.DRAG]||a||L[t.DRAG])){z=h&&!L[t.DRAG]&&1!==f;S(D=r||(z?t.data.bufferContexts[t.MOTIONBLUR_BUFFER_DRAG]:u.contexts[t.DRAG]),h&&!z?"motionBlur":void 0),N?t.drawCachedNodes(D,C.drag,l,R):t.drawCachedElements(D,C.drag,l,R),t.debug&&t.drawDebugPoints(D,C.drag),a||h||(c[t.DRAG]=!1)}if(this.drawSelectionRectangle(e,S),h&&1!==f){var O=u.contexts[t.NODE],V=t.data.bufferCanvases[t.MOTIONBLUR_BUFFER_NODE],F=u.contexts[t.DRAG],j=t.data.bufferCanvases[t.MOTIONBLUR_BUFFER_DRAG],X=function(e,n,r){e.setTransform(1,0,0,1,0,0),r||!y?e.clearRect(0,0,t.canvasWidth,t.canvasHeight):P(e,0,0,t.canvasWidth,t.canvasHeight);var a=f;e.drawImage(n,0,0,t.canvasWidth*a,t.canvasHeight*a,0,0,t.canvasWidth,t.canvasHeight)};(c[t.NODE]||L[t.NODE])&&(X(O,V,L[t.NODE]),c[t.NODE]=!1),(c[t.DRAG]||L[t.DRAG])&&(X(F,j,L[t.DRAG]),c[t.DRAG]=!1)}t.prevViewport=k,t.clearingMotionBlur&&(t.clearingMotionBlur=!1,t.motionBlurCleared=!0,t.motionBlur=!0),h&&(t.motionBlurTimeout=setTimeout((function(){t.motionBlurTimeout=null,t.clearedForMotionBlur[t.NODE]=!1,t.clearedForMotionBlur[t.DRAG]=!1,t.motionBlur=!1,t.clearingMotionBlur=!d,t.mbFrames=0,c[t.NODE]=!0,c[t.DRAG]=!0,t.redraw()}),100)),r||n.emit("render")},Rd.drawSelectionRectangle=function(e,t){var n=this,r=n.cy,a=n.data,i=r.style(),o=e.drawOnlyNodeLayer,s=e.drawAllLayers,l=a.canvasNeedsRedraw,u=e.forcedContext;if(n.showFps||!o&&l[n.SELECT_BOX]&&!s){var c=u||a.contexts[n.SELECT_BOX];if(t(c),1==n.selection[4]&&(n.hoverData.selecting||n.touchData.selecting)){var d=n.cy.zoom(),h=i.core("selection-box-border-width").value/d;c.lineWidth=h,c.fillStyle="rgba("+i.core("selection-box-color").value[0]+","+i.core("selection-box-color").value[1]+","+i.core("selection-box-color").value[2]+","+i.core("selection-box-opacity").value+")",c.fillRect(n.selection[0],n.selection[1],n.selection[2]-n.selection[0],n.selection[3]-n.selection[1]),h>0&&(c.strokeStyle="rgba("+i.core("selection-box-border-color").value[0]+","+i.core("selection-box-border-color").value[1]+","+i.core("selection-box-border-color").value[2]+","+i.core("selection-box-opacity").value+")",c.strokeRect(n.selection[0],n.selection[1],n.selection[2]-n.selection[0],n.selection[3]-n.selection[1]))}if(a.bgActivePosistion&&!n.hoverData.selecting){d=n.cy.zoom();var f=a.bgActivePosistion;c.fillStyle="rgba("+i.core("active-bg-color").value[0]+","+i.core("active-bg-color").value[1]+","+i.core("active-bg-color").value[2]+","+i.core("active-bg-opacity").value+")",c.beginPath(),c.arc(f.x,f.y,i.core("active-bg-size").pfValue/d,0,2*Math.PI),c.fill()}var p=n.lastRedrawTime;if(n.showFps&&p){p=Math.round(p);var g=Math.round(1e3/p),v="1 frame = "+p+" ms = "+g+" fps";if(c.setTransform(1,0,0,1,0,0),c.fillStyle="rgba(255, 0, 0, 0.75)",c.strokeStyle="rgba(255, 0, 0, 0.75)",c.font="30px Arial",!Md){var y=c.measureText(v);Md=y.actualBoundingBoxAscent}c.fillText(v,0,Md);c.strokeRect(0,Md+10,250,20),c.fillRect(0,Md+10,250*Math.min(g/60,1),20)}s||(l[n.SELECT_BOX]=!1)}};var qd="undefined"!=typeof Float32Array?Float32Array:Array;function Wd(){var e=new qd(9);return qd!=Float32Array&&(e[1]=0,e[2]=0,e[3]=0,e[5]=0,e[6]=0,e[7]=0),e[0]=1,e[4]=1,e[8]=1,e}function Kd(e){return e[0]=1,e[1]=0,e[2]=0,e[3]=0,e[4]=1,e[5]=0,e[6]=0,e[7]=0,e[8]=1,e}function Ud(e,t,n){var r=t[0],a=t[1],i=t[2],o=t[3],s=t[4],l=t[5],u=t[6],c=t[7],d=t[8],h=n[0],f=n[1];return e[0]=r,e[1]=a,e[2]=i,e[3]=o,e[4]=s,e[5]=l,e[6]=h*r+f*o+u,e[7]=h*a+f*s+c,e[8]=h*i+f*l+d,e}function Hd(e,t,n){var r=t[0],a=t[1],i=t[2],o=t[3],s=t[4],l=t[5],u=t[6],c=t[7],d=t[8],h=Math.sin(n),f=Math.cos(n);return e[0]=f*r+h*o,e[1]=f*a+h*s,e[2]=f*i+h*l,e[3]=f*o-h*r,e[4]=f*s-h*a,e[5]=f*l-h*i,e[6]=u,e[7]=c,e[8]=d,e}function Gd(e,t,n){var r=n[0],a=n[1];return e[0]=r*t[0],e[1]=r*t[1],e[2]=r*t[2],e[3]=a*t[3],e[4]=a*t[4],e[5]=a*t[5],e[6]=t[6],e[7]=t[7],e[8]=t[8],e}Math.hypot||(Math.hypot=function(){for(var e=0,t=arguments.length;t--;)e+=arguments[t]*arguments[t];return Math.sqrt(e)});var Zd={SCREEN:{name:"screen",screen:!0},PICKING:{name:"picking",picking:!0}},$d=ut({texRows:24}),Qd=ut({collection:"default",getKey:null,drawElement:null,getBoundingBox:null,getRotation:null,getRotationPoint:null,getRotationOffset:null,isVisible:function(){return!0},getPadding:0}),Jd=function(){return i((function e(t,n,r,i){a(this,e),this.debugID=Math.floor(1e4*Math.random()),this.r=t,this.texSize=n,this.texRows=r,this.texHeight=Math.floor(n/r),this.enableWrapping=!0,this.locked=!1,this.texture=null,this.needsBuffer=!0,this.freePointer={x:0,row:0},this.keyToLocation=new Map,this.canvas=i(t,n,n),this.scratch=i(t,n,this.texHeight,"scratch")}),[{key:"lock",value:function(){this.locked=!0}},{key:"getKeys",value:function(){return new Set(this.keyToLocation.keys())}},{key:"getScale",value:function(e){var t=e.w,n=e.h,r=this.texHeight,a=this.texSize,i=r/n,o=t*i,s=n*i;return o>a&&(o=t*(i=a/t),s=n*i),{scale:i,texW:o,texH:s}}},{key:"draw",value:function(e,t,n){var r=this;if(this.locked)throw new Error("can't draw, atlas is locked");var a=this.texSize,i=this.texRows,o=this.texHeight,s=this.getScale(t),l=s.scale,u=s.texW,c=s.texH,d=[null,null],h=function(e,r){if(n&&r){var a=r.context,i=e.x,s=e.row,u=i,c=o*s;a.save(),a.translate(u,c),a.scale(l,l),n(a,t),a.restore()}},f=function(){h(r.freePointer,r.canvas),d[0]={x:r.freePointer.x,y:r.freePointer.row*o,w:u,h:c},d[1]={x:r.freePointer.x+u,y:r.freePointer.row*o,w:0,h:c},r.freePointer.x+=u,r.freePointer.x==a&&(r.freePointer.x=0,r.freePointer.row++)},p=function(){r.freePointer.x=0,r.freePointer.row++};if(this.freePointer.x+u<=a)f();else{if(this.freePointer.row>=i-1)return!1;this.freePointer.x===a?(p(),f()):this.enableWrapping?function(){var e=r.scratch,t=r.canvas;e.clear(),h({x:0,row:0},e);var n=a-r.freePointer.x,i=u-n,s=o,l=r.freePointer.x,f=r.freePointer.row*o,p=n;t.context.drawImage(e,0,0,p,s,l,f,p,s),d[0]={x:l,y:f,w:p,h:c};var g=n,v=(r.freePointer.row+1)*o,y=i;t&&t.context.drawImage(e,g,0,y,s,0,v,y,s),d[1]={x:0,y:v,w:y,h:c},r.freePointer.x=i,r.freePointer.row++}():(p(),f())}return this.keyToLocation.set(e,d),this.needsBuffer=!0,d}},{key:"getOffsets",value:function(e){return this.keyToLocation.get(e)}},{key:"isEmpty",value:function(){return 0===this.freePointer.x&&0===this.freePointer.row}},{key:"canFit",value:function(e){if(this.locked)return!1;var t=this.texSize,n=this.texRows,r=this.getScale(e).texW;return!(this.freePointer.x+r>t)||this.freePointer.row1&&void 0!==arguments[1]?arguments[1]:{},r=n.forceRedraw,a=void 0!==r&&r,i=n.filterEle,s=void 0===i?function(){return!0}:i,l=n.filterType,u=void 0===l?function(){return!0}:l,c=!1,d=!1,h=o(e);try{for(h.s();!(t=h.n()).done;){var f=t.value;if(s(f)){var p,g=o(this.renderTypes.values());try{for(g.s();!(p=g.n()).done;){var v=p.value,y=v.type;if(u(y)){var m=v.getKey(f),b=this.collections.get(v.collection);if(a)b.markKeyForGC(m),d=!0;else{var x=v.getID?v.getID(f):f.id(),w=this._key(y,x),E=this.typeAndIdToKey.get(w);void 0!==E&&E!==m&&(this.typeAndIdToKey.delete(w),b.markKeyForGC(E),c=!0)}}}}catch(k){g.e(k)}finally{g.f()}}}}catch(k){h.e(k)}finally{h.f()}return d&&(this.gc(),c=!1),c}},{key:"gc",value:function(){var e,t=o(this.collections.values());try{for(t.s();!(e=t.n()).done;){e.value.gc()}}catch(n){t.e(n)}finally{t.f()}}},{key:"getOrCreateAtlas",value:function(e,t,n){var r=this.renderTypes.get(t),a=r.getKey(e);n||(n=r.getBoundingBox(e));var i=this.collections.get(r.collection),o=!1,s=i.draw(a,n,(function(t){r.drawElement(t,e,n,!0,!0),o=!0}));if(o){var l=r.getID?r.getID(e):e.id(),u=this._key(t,l);this.typeAndIdToKey.set(u,a)}return s}},{key:"startBatch",value:function(){this.batchAtlases=[]}},{key:"getAtlasCount",value:function(){return this.batchAtlases.length}},{key:"getAtlases",value:function(){return this.batchAtlases}},{key:"canAddToCurrentBatch",value:function(e,t){if(this.batchAtlases.length===this.maxAtlasesPerBatch){var n=this.renderTypes.get(t),r=n.getKey(e),a=this.collections.get(n.collection).getAtlas(r);return Boolean(a)&&this.batchAtlases.includes(a)}return!0}},{key:"getAtlasIndexForBatch",value:function(e){var t=this.batchAtlases.indexOf(e);if(t<0){if(this.batchAtlases.length===this.maxAtlasesPerBatch)return;this.batchAtlases.push(e),t=this.batchAtlases.length-1}return t}},{key:"getIndexArray",value:function(){return Array.from({length:this.maxAtlasesPerBatch},(function(e,t){return t}))}},{key:"getAtlasInfo",value:function(e,t){var n=this.renderTypes.get(t),r=n.getBoundingBox(e),a=this.getOrCreateAtlas(e,t,r),i=this.getAtlasIndexForBatch(a);if(void 0!==i){var o=n.getKey(e),s=l(a.getOffsets(o),2);return{index:i,tex1:s[0],tex2:s[1],bb:r}}}},{key:"setTransformMatrix",value:function(e,t,n,r){var a=!(arguments.length>4&&void 0!==arguments[4])||arguments[4],i=this.getRenderTypeOpts(n),o=i.getPadding?i.getPadding(e):0;if(r){var s=r.bb,l=r.tex1,u=r.tex2,c=l.w/(l.w+u.w);a||(c=1-c);var d=this.getAdjustedBB(s,o,a,c);this._applyTransformMatrix(t,d,i,e)}else{var h=i.getBoundingBox(e),f=this.getAdjustedBB(h,o,!0,1);this._applyTransformMatrix(t,f,i,e)}}},{key:"_applyTransformMatrix",value:function(e,t,n,r){var a,i;Kd(e);var o=n.getRotation?n.getRotation(r):0;if(0!==o){var s=n.getRotationPoint(r);Ud(e,e,[s.x,s.y]),Hd(e,e,o);var l=n.getRotationOffset(r);a=l.x+t.xOffset,i=l.y}else a=t.x1,i=t.y1;Ud(e,e,[a,i]),Gd(e,e,[t.w,t.h])}},{key:"getAdjustedBB",value:function(e,t,n,r){var a=e.x1,i=e.y1,o=e.w,s=e.h;t&&(a-=t,i-=t,o+=2*t,s+=2*t);var l=0,u=o*r;return n&&r<1?o=u:!n&&r<1&&(a+=l=o-u,o=u),{x1:a,y1:i,w:o,h:s,xOffset:l}}},{key:"getDebugInfo",value:function(){var e,t=[],n=o(this.collections);try{for(n.s();!(e=n.n()).done;){var r=l(e.value,2),a=r[0],i=r[1].getCounts(),s=i.keyCount,u=i.atlasCount;t.push({type:a,keyCount:s,atlasCount:u})}}catch(c){n.e(c)}finally{n.f()}return t}}])}(),nh=function(){return i((function e(t,n,r){a(this,e),this.r=t,this.gl=n,this.maxInstances=r.webglBatchSize,this.atlasSize=r.webglTexSize,this.bgColor=r.bgColor,this.debug=r.webglDebug,this.batchDebugInfo=[],r.enableWrapping=!0,r.createTextureCanvas=Nd,this.atlasManager=new th(t,r),this.program=this.createShaderProgram(Zd.SCREEN),this.pickingProgram=this.createShaderProgram(Zd.PICKING),this.vao=this.createVAO()}),[{key:"addAtlasCollection",value:function(e,t){this.atlasManager.addAtlasCollection(e,t)}},{key:"addAtlasRenderType",value:function(e,t){this.atlasManager.addRenderType(e,t)}},{key:"invalidate",value:function(e){var t=(arguments.length>1&&void 0!==arguments[1]?arguments[1]:{}).type,n=this.atlasManager;return t?n.invalidate(e,{filterType:function(e){return e===t},forceRedraw:!0}):n.invalidate(e)}},{key:"gc",value:function(){this.atlasManager.gc()}},{key:"createShaderProgram",value:function(e){var t=this.gl,n="#version 300 es\n precision highp float;\n\n uniform mat3 uPanZoomMatrix;\n uniform int uAtlasSize;\n \n // instanced\n in vec2 aPosition; \n\n in mat3 aTransform;\n\n // what are we rendering?\n in int aVertType;\n\n // for picking\n in vec4 aIndex;\n \n // For textures\n in int aAtlasId; // which shader unit/atlas to use\n in vec4 aTex; // x/y/w/h of texture in atlas\n\n // for edges\n in vec4 aPointAPointB;\n in vec4 aPointCPointD;\n in float aLineWidth;\n in vec4 aColor;\n\n out vec2 vTexCoord;\n out vec4 vColor;\n flat out int vAtlasId;\n flat out vec4 vIndex;\n flat out int vVertType;\n\n void main(void) {\n int vid = gl_VertexID;\n vec2 position = aPosition;\n\n if(aVertType == ".concat(0,") {\n float texX = aTex.x;\n float texY = aTex.y;\n float texW = aTex.z;\n float texH = aTex.w;\n\n int vid = gl_VertexID;\n\n if(vid == 1 || vid == 2 || vid == 4) {\n texX += texW;\n }\n if(vid == 2 || vid == 4 || vid == 5) {\n texY += texH;\n }\n\n float d = float(uAtlasSize);\n vTexCoord = vec2(texX / d, texY / d); // tex coords must be between 0 and 1\n\n gl_Position = vec4(uPanZoomMatrix * aTransform * vec3(position, 1.0), 1.0);\n }\n else if(aVertType == ").concat(4,") {\n gl_Position = vec4(uPanZoomMatrix * aTransform * vec3(position, 1.0), 1.0);\n vColor = aColor;\n }\n else if(aVertType == ").concat(1,") {\n vec2 source = aPointAPointB.xy;\n vec2 target = aPointAPointB.zw;\n\n // adjust the geometry so that the line is centered on the edge\n position.y = position.y - 0.5;\n\n vec2 xBasis = target - source;\n vec2 yBasis = normalize(vec2(-xBasis.y, xBasis.x));\n vec2 point = source + xBasis * position.x + yBasis * aLineWidth * position.y;\n\n gl_Position = vec4(uPanZoomMatrix * vec3(point, 1.0), 1.0);\n vColor = aColor;\n } \n else if(aVertType == ").concat(2,") {\n vec2 pointA = aPointAPointB.xy;\n vec2 pointB = aPointAPointB.zw;\n vec2 pointC = aPointCPointD.xy;\n vec2 pointD = aPointCPointD.zw;\n\n // adjust the geometry so that the line is centered on the edge\n position.y = position.y - 0.5;\n\n vec2 p0 = pointA;\n vec2 p1 = pointB;\n vec2 p2 = pointC;\n vec2 pos = position;\n if(position.x == 1.0) {\n p0 = pointD;\n p1 = pointC;\n p2 = pointB;\n pos = vec2(0.0, -position.y);\n }\n\n vec2 p01 = p1 - p0;\n vec2 p12 = p2 - p1;\n vec2 p21 = p1 - p2;\n\n // Find the normal vector.\n vec2 tangent = normalize(normalize(p12) + normalize(p01));\n vec2 normal = vec2(-tangent.y, tangent.x);\n\n // Find the vector perpendicular to p0 -> p1.\n vec2 p01Norm = normalize(vec2(-p01.y, p01.x));\n\n // Determine the bend direction.\n float sigma = sign(dot(p01 + p21, normal));\n float width = aLineWidth;\n\n if(sign(pos.y) == -sigma) {\n // This is an intersecting vertex. Adjust the position so that there's no overlap.\n vec2 point = 0.5 * width * normal * -sigma / dot(normal, p01Norm);\n gl_Position = vec4(uPanZoomMatrix * vec3(p1 + point, 1.0), 1.0);\n } else {\n // This is a non-intersecting vertex. Treat it like a mitre join.\n vec2 point = 0.5 * width * normal * sigma * dot(normal, p01Norm);\n gl_Position = vec4(uPanZoomMatrix * vec3(p1 + point, 1.0), 1.0);\n }\n\n vColor = aColor;\n } \n else if(aVertType == ").concat(3," && vid < 3) {\n // massage the first triangle into an edge arrow\n if(vid == 0)\n position = vec2(-0.15, -0.3);\n if(vid == 1)\n position = vec2( 0.0, 0.0);\n if(vid == 2)\n position = vec2( 0.15, -0.3);\n\n gl_Position = vec4(uPanZoomMatrix * aTransform * vec3(position, 1.0), 1.0);\n vColor = aColor;\n }\n else {\n gl_Position = vec4(2.0, 0.0, 0.0, 1.0); // discard vertex by putting it outside webgl clip space\n }\n\n vAtlasId = aAtlasId;\n vIndex = aIndex;\n vVertType = aVertType;\n }\n "),r=this.atlasManager.getIndexArray(),a="#version 300 es\n precision highp float;\n\n // define texture unit for each node in the batch\n ".concat(r.map((function(e){return"uniform sampler2D uTexture".concat(e,";")})).join("\n\t"),"\n\n uniform vec4 uBGColor;\n\n in vec2 vTexCoord;\n in vec4 vColor;\n flat in int vAtlasId;\n flat in vec4 vIndex;\n flat in int vVertType;\n\n out vec4 outColor;\n\n void main(void) {\n if(vVertType == ").concat(0,") {\n ").concat(r.map((function(e){return"if(vAtlasId == ".concat(e,") outColor = texture(uTexture").concat(e,", vTexCoord);")})).join("\n\telse "),"\n } else if(vVertType == ").concat(3,") {\n // blend arrow color with background (using premultiplied alpha)\n outColor.rgb = vColor.rgb + (uBGColor.rgb * (1.0 - vColor.a)); \n outColor.a = 1.0; // make opaque, masks out line under arrow\n } else {\n outColor = vColor;\n }\n\n ").concat(e.picking?"if(outColor.a == 0.0) discard;\n else outColor = vIndex;":"","\n }\n "),i=function(e,t,n){var r=Id(e,e.VERTEX_SHADER,t),a=Id(e,e.FRAGMENT_SHADER,n),i=e.createProgram();if(e.attachShader(i,r),e.attachShader(i,a),e.linkProgram(i),!e.getProgramParameter(i,e.LINK_STATUS))throw new Error("Could not initialize shaders");return i}(t,n,a);i.aPosition=t.getAttribLocation(i,"aPosition"),i.aIndex=t.getAttribLocation(i,"aIndex"),i.aVertType=t.getAttribLocation(i,"aVertType"),i.aTransform=t.getAttribLocation(i,"aTransform"),i.aAtlasId=t.getAttribLocation(i,"aAtlasId"),i.aTex=t.getAttribLocation(i,"aTex"),i.aPointAPointB=t.getAttribLocation(i,"aPointAPointB"),i.aPointCPointD=t.getAttribLocation(i,"aPointCPointD"),i.aLineWidth=t.getAttribLocation(i,"aLineWidth"),i.aColor=t.getAttribLocation(i,"aColor"),i.uPanZoomMatrix=t.getUniformLocation(i,"uPanZoomMatrix"),i.uAtlasSize=t.getUniformLocation(i,"uAtlasSize"),i.uBGColor=t.getUniformLocation(i,"uBGColor"),i.uTextures=[];for(var o=0;o1&&void 0!==arguments[1]?arguments[1]:Zd.SCREEN;this.panZoomMatrix=e,this.renderTarget=t,this.batchDebugInfo=[],this.wrappedCount=0,this.rectangleCount=0,this.startBatch()}},{key:"startBatch",value:function(){this.instanceCount=0,this.atlasManager.startBatch()}},{key:"endFrame",value:function(){this.endBatch()}},{key:"getTempMatrix",value:function(){return this.tempMatrix=this.tempMatrix||Wd()}},{key:"drawTexture",value:function(e,t,n){var r=this.atlasManager;if(e.visible()&&r.getRenderTypeOpts(n).isVisible(e)){r.canAddToCurrentBatch(e,n)||this.endBatch(),this.instanceCount+1>=this.maxInstances&&this.endBatch();var a=this.instanceCount;this.vertTypeBuffer.getView(a)[0]=0,Od(t,this.indexBuffer.getView(a));var i=r.getAtlasInfo(e,n),o=i.index,s=i.tex1,l=i.tex2;l.w>0&&this.wrappedCount++;for(var u=!0,c=0,d=[s,l];c=this.maxInstances&&this.endBatch()}}},{key:"drawSimpleRectangle",value:function(e,t,n){if(e.visible()){var r=this.atlasManager,a=this.instanceCount;this.vertTypeBuffer.getView(a)[0]=4,Od(t,this.indexBuffer.getView(a)),zd(e.pstyle("background-color").value,e.pstyle("background-opacity").value,this.colorBuffer.getView(a));var i=this.transformBuffer.getMatrixView(a);r.setTransformMatrix(e,i,n),this.rectangleCount++,this.instanceCount++,this.instanceCount>=this.maxInstances&&this.endBatch()}}},{key:"drawEdgeArrow",value:function(e,t,n){if(e.visible()){var r,a,i,o=e._private.rscratch;if("source"===n?(r=o.arrowStartX,a=o.arrowStartY,i=o.srcArrowAngle):(r=o.arrowEndX,a=o.arrowEndY,i=o.tgtArrowAngle),!(isNaN(r)||null==r||isNaN(a)||null==a||isNaN(i)||null==i))if("none"!==e.pstyle(n+"-arrow-shape").value){var s=e.pstyle(n+"-arrow-color").value,l=e.pstyle("opacity").value*e.pstyle("line-opacity").value,u=e.pstyle("width").pfValue,c=e.pstyle("arrow-scale").value,d=this.r.getArrowWidth(u,c),h=this.instanceCount,f=this.transformBuffer.getMatrixView(h);Kd(f),Ud(f,f,[r,a]),Gd(f,f,[d,d]),Hd(f,f,i),this.vertTypeBuffer.getView(h)[0]=3,Od(t,this.indexBuffer.getView(h)),zd(s,l,this.colorBuffer.getView(h)),this.instanceCount++,this.instanceCount>=this.maxInstances&&this.endBatch()}}}},{key:"drawEdgeLine",value:function(e,t){if(e.visible()){var n=this.getEdgePoints(e);if(n){var r=e.pstyle("opacity").value,a=e.pstyle("line-opacity").value,i=e.pstyle("width").pfValue,o=e.pstyle("line-color").value,s=r*a;if(n.length/2+this.instanceCount>this.maxInstances&&this.endBatch(),4==n.length){var l=this.instanceCount;this.vertTypeBuffer.getView(l)[0]=1,Od(t,this.indexBuffer.getView(l)),zd(o,s,this.colorBuffer.getView(l)),this.lineWidthBuffer.getView(l)[0]=i;var u=this.pointAPointBBuffer.getView(l);u[0]=n[0],u[1]=n[1],u[2]=n[2],u[3]=n[3],this.instanceCount++,this.instanceCount>=this.maxInstances&&this.endBatch()}else for(var c=0;c=this.maxInstances&&this.endBatch()}}}}},{key:"getEdgePoints",value:function(e){var t=e._private.rscratch;if(!t.badLine&&null!=t.allpts&&!isNaN(t.allpts[0])){var n=t.allpts;if(4==n.length)return n;var r=this.getNumSegments(e);return this.getCurveSegmentPoints(n,r)}}},{key:"getNumSegments",value:function(e){return Math.min(Math.max(15,5),this.maxInstances)}},{key:"getCurveSegmentPoints",value:function(e,t){if(4==e.length)return e;for(var n=Array(2*(t+1)),r=0;r<=t;r++)if(0==r)n[0]=e[0],n[1]=e[1];else if(r==t)n[2*r]=e[e.length-2],n[2*r+1]=e[e.length-1];else{var a=r/t;this.setCurvePoint(e,a,n,2*r)}return n}},{key:"setCurvePoint",value:function(e,t,n,r){if(!(e.length<=2)){for(var a=Array(e.length-2),i=0;i0}},{key:"getStyle",value:function(e,t){return{opacity:t.pstyle("".concat(e,"-opacity")).value,color:t.pstyle("".concat(e,"-color")).value,shape:t.pstyle("".concat(e,"-shape")).value}}},{key:"getPadding",value:function(e,t){return t.pstyle("".concat(e,"-padding")).pfValue}},{key:"draw",value:function(e,t,n,r){if(this.isVisible(e,n)){var a=this.r,i=r.w,o=r.h,s=i/2,l=o/2,u=this.getStyle(e,n),c=u.shape,d=u.color,h=u.opacity;t.save(),t.fillStyle=rh(d,h),"round-rectangle"===c||"roundrectangle"===c?a.drawRoundRectanglePath(t,s,l,i,o,"auto"):"ellipse"===c&&a.drawEllipsePath(t,s,l,i,o),t.fill(),t.restore()}}}])}(),ih={};function oh(e,t){var n=e.canvasWidth,r=e.canvasHeight,a=Ld(e),i=a.pan,o=a.zoom;t.setTransform(1,0,0,1,0,0),t.clearRect(0,0,n,r),t.translate(i.x,i.y),t.scale(o,o)}function sh(e,t,n){var r,a=e.drawing;t+=1,n.isNode()?(a.drawTexture(n,t,"node-underlay"),"rectangle"===(r=n).pstyle("shape").value&&"solid"===r.pstyle("background-fill").value&&0===r.pstyle("border-width").pfValue&&"none"===r.pstyle("background-image").strValue?a.drawSimpleRectangle(n,t,"node-body"):a.drawTexture(n,t,"node-body"),a.drawTexture(n,t,"label"),a.drawTexture(n,t,"node-overlay")):(a.drawEdgeLine(n,t),a.drawEdgeArrow(n,t,"source"),a.drawEdgeArrow(n,t,"target"),a.drawTexture(n,t,"label"),a.drawTexture(n,t,"edge-source-label"),a.drawTexture(n,t,"edge-target-label"))}function lh(e,t,n){var r;e.webglDebug&&(r=performance.now());var a=e.drawing,i=0;if(n.screen&&e.data.canvasNeedsRedraw[e.SELECT_BOX]&&function(e,t){e.drawSelectionRectangle(t,(function(t){return oh(e,t)}))}(e,t),e.data.canvasNeedsRedraw[e.NODE]||n.picking){var s=e.data.contexts[e.WEBGL];n.screen?(s.clearColor(0,0,0,0),s.enable(s.BLEND),s.blendFunc(s.ONE,s.ONE_MINUS_SRC_ALPHA)):s.disable(s.BLEND),s.clear(s.COLOR_BUFFER_BIT|s.DEPTH_BUFFER_BIT),s.viewport(0,0,s.canvas.width,s.canvas.height);var l=function(e){var t=e.canvasWidth,n=e.canvasHeight,r=Ld(e),a=r.pan,i=r.zoom,o=Wd();Ud(o,o,[a.x,a.y]),Gd(o,o,[i,i]);var s=Wd();!function(e,t,n){e[0]=2/t,e[1]=0,e[2]=0,e[3]=0,e[4]=-2/n,e[5]=0,e[6]=-1,e[7]=1,e[8]=1}(s,t,n);var l,u,c,d,h,f,p,g,v,y,m,b,x,w,E,k,T,C,P,S,B,D=Wd();return l=D,c=o,d=(u=s)[0],h=u[1],f=u[2],p=u[3],g=u[4],v=u[5],y=u[6],m=u[7],b=u[8],x=c[0],w=c[1],E=c[2],k=c[3],T=c[4],C=c[5],P=c[6],S=c[7],B=c[8],l[0]=x*d+w*p+E*y,l[1]=x*h+w*g+E*m,l[2]=x*f+w*v+E*b,l[3]=k*d+T*p+C*y,l[4]=k*h+T*g+C*m,l[5]=k*f+T*v+C*b,l[6]=P*d+S*p+B*y,l[7]=P*h+S*g+B*m,l[8]=P*f+S*v+B*b,D}(e),u=e.getCachedZSortedEles();if(i=u.length,a.startFrame(l,n),n.screen){for(var c=0;c0&&(r|=n.drawing.invalidate(t)),r&&u()})),function(e){var t=e.render;e.render=function(n){n=n||{};var r=e.cy;e.webgl&&(r.zoom()>nd?(!function(e){var t=e.data.contexts[e.WEBGL];t.clear(t.COLOR_BUFFER_BIT|t.DEPTH_BUFFER_BIT)}(e),t.call(e,n)):(!function(e){var t=function(t){t.save(),t.setTransform(1,0,0,1,0,0),t.clearRect(0,0,e.canvasWidth,e.canvasHeight),t.restore()};t(e.data.contexts[e.NODE]),t(e.data.contexts[e.DRAG])}(e),lh(e,n,Zd.SCREEN)))};var n=e.matchCanvasSize;e.matchCanvasSize=function(t){n.call(e,t),e.pickingFrameBuffer.setFramebufferAttachmentSizes(e.canvasWidth,e.canvasHeight),e.pickingFrameBuffer.needsDraw=!0},e.findNearestElements=function(t,n,r,a){return function(e,t,n){var r,a,i,s=function(e,t,n){var r,a,i,o,s=Ld(e),u=s.pan,c=s.zoom,d=function(e,t,n,r,a){var i=r*n+t.x,o=a*n+t.y;return[i,o=Math.round(e.canvasHeight-o)]}(e,u,c,t,n),h=l(d,2),f=h[0],p=h[1],g=6;if(r=f-g/2,a=p-g/2,o=g,0===(i=g)||0===o)return[];var v=e.data.contexts[e.WEBGL];v.bindFramebuffer(v.FRAMEBUFFER,e.pickingFrameBuffer),e.pickingFrameBuffer.needsDraw&&(v.viewport(0,0,v.canvas.width,v.canvas.height),lh(e,null,Zd.PICKING),e.pickingFrameBuffer.needsDraw=!1);var y=i*o,m=new Uint8Array(4*y);v.readPixels(r,a,i,o,v.RGBA,v.UNSIGNED_BYTE,m),v.bindFramebuffer(v.FRAMEBUFFER,null);for(var b=new Set,x=0;x=0&&b.add(w)}return b}(e,t,n),u=e.getCachedZSortedEles(),c=o(s);try{for(c.s();!(i=c.n()).done;){var d=u[i.value];if(!r&&d.isNode()&&(r=d),!a&&d.isEdge()&&(a=d),r&&a)break}}catch(h){c.e(h)}finally{c.f()}return[r,a].filter(Boolean)}(e,t,n)};var r=e.invalidateCachedZSortedEles;e.invalidateCachedZSortedEles=function(){r.call(e),e.pickingFrameBuffer.needsDraw=!0};var a=e.notify;e.notify=function(t,n){a.call(e,t,n),"viewport"===t||"bounds"===t?e.pickingFrameBuffer.needsDraw=!0:"background"===t&&e.drawing.invalidate(n,{type:"node-body"})}}(n)};for(var uh={drawPolygonPath:function(e,t,n,r,a,i){var o=r/2,s=a/2;e.beginPath&&e.beginPath(),e.moveTo(t+o*i[0],n+s*i[1]);for(var l=1;l0&&i>0){h.clearRect(0,0,a,i),h.globalCompositeOperation="source-over";var f=this.getCachedZSortedEles();if(e.full)h.translate(-n.x1*l,-n.y1*l),h.scale(l,l),this.drawElements(h,f),h.scale(1/l,1/l),h.translate(n.x1*l,n.y1*l);else{var p=t.pan(),g={x:p.x*l,y:p.y*l};l*=t.zoom(),h.translate(g.x,g.y),h.scale(l,l),this.drawElements(h,f),h.scale(1/l,1/l),h.translate(-g.x,-g.y)}e.bg&&(h.globalCompositeOperation="destination-over",h.fillStyle=e.bg,h.rect(0,0,a,i),h.fill())}return d},vh.png=function(e){return mh(e,this.bufferCanvasImage(e),"image/png")},vh.jpg=function(e){return mh(e,this.bufferCanvasImage(e),"image/jpeg")};var bh={nodeShapeImpl:function(e,t,n,r,a,i,o,s){switch(e){case"ellipse":return this.drawEllipsePath(t,n,r,a,i);case"polygon":return this.drawPolygonPath(t,n,r,a,i,o);case"round-polygon":return this.drawRoundPolygonPath(t,n,r,a,i,o,s);case"roundrectangle":case"round-rectangle":return this.drawRoundRectanglePath(t,n,r,a,i,s);case"cutrectangle":case"cut-rectangle":return this.drawCutRectanglePath(t,n,r,a,i,o,s);case"bottomroundrectangle":case"bottom-round-rectangle":return this.drawBottomRoundRectanglePath(t,n,r,a,i,s);case"barrel":return this.drawBarrelPath(t,n,r,a,i)}}},xh=Eh,wh=Eh.prototype;function Eh(e){var t=this,n=t.cy.window().document;e.webgl&&(wh.CANVAS_LAYERS=t.CANVAS_LAYERS=4,console.log("webgl rendering enabled")),t.data={canvases:new Array(wh.CANVAS_LAYERS),contexts:new Array(wh.CANVAS_LAYERS),canvasNeedsRedraw:new Array(wh.CANVAS_LAYERS),bufferCanvases:new Array(wh.BUFFER_COUNT),bufferContexts:new Array(wh.CANVAS_LAYERS)};var r="-webkit-tap-highlight-color",a="rgba(0,0,0,0)";t.data.canvasContainer=n.createElement("div");var i=t.data.canvasContainer.style;t.data.canvasContainer.style[r]=a,i.position="relative",i.zIndex="0",i.overflow="hidden";var o=e.cy.container();o.appendChild(t.data.canvasContainer),o.style[r]=a;var s={"-webkit-user-select":"none","-moz-user-select":"-moz-none","user-select":"none","-webkit-tap-highlight-color":"rgba(0,0,0,0)","outline-style":"none"};p&&p.userAgent.match(/msie|trident|edge/i)&&(s["-ms-touch-action"]="none",s["touch-action"]="none");for(var l=0;l extends ICstVisitor {\n ").concat((0,s.default)(n,(function(e){return function(e){var t=d(e.name);return"".concat(e.name,"(children: ").concat(t,", param?: IN): OUT;")}(e)})).join("\n "),"\n}")))),c.join("\n\n")+"\n"}},62222:function(e,t,r){"use strict";var n,i=this&&this.__extends||(n=function(e,t){return n=Object.setPrototypeOf||{__proto__:[]}instanceof Array&&function(e,t){e.__proto__=t}||function(e,t){for(var r in t)Object.prototype.hasOwnProperty.call(t,r)&&(e[r]=t[r])},n(e,t)},function(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Class extends value "+String(t)+" is not a constructor or null");function r(){this.constructor=e}n(e,t),e.prototype=null===t?Object.create(t):(r.prototype=t.prototype,new r)}),a=this&&this.__importDefault||function(e){return e&&e.__esModule?e:{default:e}};Object.defineProperty(t,"__esModule",{value:!0}),t.buildModel=void 0;var s=r(93062),o=a(r(35161)),l=a(r(85564)),c=a(r(52628)),u=a(r(59704)),h=a(r(7739)),d=a(r(28583));t.buildModel=function(e){var t=new p,r=(0,c.default)(e);return(0,o.default)(r,(function(e){return t.visitRule(e)}))};var p=function(e){function t(){return null!==e&&e.apply(this,arguments)||this}return i(t,e),t.prototype.visitRule=function(e){var t=this.visitEach(e.definition),r=(0,h.default)(t,(function(e){return e.propertyName})),n=(0,o.default)(r,(function(e,t){var r=!(0,u.default)(e,(function(e){return!e.canBeNull})),n=e[0].type;return e.length>1&&(n=(0,o.default)(e,(function(e){return e.type}))),{name:t,type:n,optional:r}}));return{name:e.name,properties:n}},t.prototype.visitAlternative=function(e){return this.visitEachAndOverrideWith(e.definition,{canBeNull:!0})},t.prototype.visitOption=function(e){return this.visitEachAndOverrideWith(e.definition,{canBeNull:!0})},t.prototype.visitRepetition=function(e){return this.visitEachAndOverrideWith(e.definition,{canBeNull:!0})},t.prototype.visitRepetitionMandatory=function(e){return this.visitEach(e.definition)},t.prototype.visitRepetitionMandatoryWithSeparator=function(e){return this.visitEach(e.definition).concat({propertyName:e.separator.name,canBeNull:!0,type:f(e.separator)})},t.prototype.visitRepetitionWithSeparator=function(e){return this.visitEachAndOverrideWith(e.definition,{canBeNull:!0}).concat({propertyName:e.separator.name,canBeNull:!0,type:f(e.separator)})},t.prototype.visitAlternation=function(e){return this.visitEachAndOverrideWith(e.definition,{canBeNull:!0})},t.prototype.visitTerminal=function(e){return[{propertyName:e.label||e.terminalType.name,canBeNull:!1,type:f(e)}]},t.prototype.visitNonTerminal=function(e){return[{propertyName:e.label||e.nonTerminalName,canBeNull:!1,type:f(e)}]},t.prototype.visitEachAndOverrideWith=function(e,t){return(0,o.default)(this.visitEach(e),(function(e){return(0,d.default)({},e,t)}))},t.prototype.visitEach=function(e){var t=this;return(0,l.default)((0,o.default)(e,(function(e){return t.visit(e)})))},t}(s.GAstVisitor);function f(e){return e instanceof s.NonTerminal?{kind:"rule",name:e.referencedRule.name}:{kind:"token"}}},93062:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.isSequenceProd=t.isBranchingProd=t.isOptionalProd=t.getProductionDslName=t.GAstVisitor=t.serializeProduction=t.serializeGrammar=t.Alternative=t.Alternation=t.RepetitionWithSeparator=t.RepetitionMandatoryWithSeparator=t.RepetitionMandatory=t.Repetition=t.Option=t.NonTerminal=t.Terminal=t.Rule=void 0;var n=r(94490);Object.defineProperty(t,"Rule",{enumerable:!0,get:function(){return n.Rule}}),Object.defineProperty(t,"Terminal",{enumerable:!0,get:function(){return n.Terminal}}),Object.defineProperty(t,"NonTerminal",{enumerable:!0,get:function(){return n.NonTerminal}}),Object.defineProperty(t,"Option",{enumerable:!0,get:function(){return n.Option}}),Object.defineProperty(t,"Repetition",{enumerable:!0,get:function(){return n.Repetition}}),Object.defineProperty(t,"RepetitionMandatory",{enumerable:!0,get:function(){return n.RepetitionMandatory}}),Object.defineProperty(t,"RepetitionMandatoryWithSeparator",{enumerable:!0,get:function(){return n.RepetitionMandatoryWithSeparator}}),Object.defineProperty(t,"RepetitionWithSeparator",{enumerable:!0,get:function(){return n.RepetitionWithSeparator}}),Object.defineProperty(t,"Alternation",{enumerable:!0,get:function(){return n.Alternation}}),Object.defineProperty(t,"Alternative",{enumerable:!0,get:function(){return n.Alternative}}),Object.defineProperty(t,"serializeGrammar",{enumerable:!0,get:function(){return n.serializeGrammar}}),Object.defineProperty(t,"serializeProduction",{enumerable:!0,get:function(){return n.serializeProduction}});var i=r(62156);Object.defineProperty(t,"GAstVisitor",{enumerable:!0,get:function(){return i.GAstVisitor}});var a=r(53559);Object.defineProperty(t,"getProductionDslName",{enumerable:!0,get:function(){return a.getProductionDslName}}),Object.defineProperty(t,"isOptionalProd",{enumerable:!0,get:function(){return a.isOptionalProd}}),Object.defineProperty(t,"isBranchingProd",{enumerable:!0,get:function(){return a.isBranchingProd}}),Object.defineProperty(t,"isSequenceProd",{enumerable:!0,get:function(){return a.isSequenceProd}})},53559:function(e,t,r){"use strict";var n=this&&this.__importDefault||function(e){return e&&e.__esModule?e:{default:e}};Object.defineProperty(t,"__esModule",{value:!0}),t.getProductionDslName=t.isBranchingProd=t.isOptionalProd=t.isSequenceProd=void 0;var i=n(r(59704)),a=n(r(711)),s=n(r(64721)),o=r(94490);t.isSequenceProd=function(e){return e instanceof o.Alternative||e instanceof o.Option||e instanceof o.Repetition||e instanceof o.RepetitionMandatory||e instanceof o.RepetitionMandatoryWithSeparator||e instanceof o.RepetitionWithSeparator||e instanceof o.Terminal||e instanceof o.Rule},t.isOptionalProd=function e(t,r){return void 0===r&&(r=[]),!!(t instanceof o.Option||t instanceof o.Repetition||t instanceof o.RepetitionWithSeparator)||(t instanceof o.Alternation?(0,i.default)(t.definition,(function(t){return e(t,r)})):!(t instanceof o.NonTerminal&&(0,s.default)(r,t))&&(t instanceof o.AbstractProduction&&(t instanceof o.NonTerminal&&r.push(t),(0,a.default)(t.definition,(function(t){return e(t,r)})))))},t.isBranchingProd=function(e){return e instanceof o.Alternation},t.getProductionDslName=function(e){if(e instanceof o.NonTerminal)return"SUBRULE";if(e instanceof o.Option)return"OPTION";if(e instanceof o.Alternation)return"OR";if(e instanceof o.RepetitionMandatory)return"AT_LEAST_ONE";if(e instanceof o.RepetitionMandatoryWithSeparator)return"AT_LEAST_ONE_SEP";if(e instanceof o.RepetitionWithSeparator)return"MANY_SEP";if(e instanceof o.Repetition)return"MANY";if(e instanceof o.Terminal)return"CONSUME";throw Error("non exhaustive match")}},94490:function(e,t,r){"use strict";var n,i=this&&this.__extends||(n=function(e,t){return n=Object.setPrototypeOf||{__proto__:[]}instanceof Array&&function(e,t){e.__proto__=t}||function(e,t){for(var r in t)Object.prototype.hasOwnProperty.call(t,r)&&(e[r]=t[r])},n(e,t)},function(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Class extends value "+String(t)+" is not a constructor or null");function r(){this.constructor=e}n(e,t),e.prototype=null===t?Object.create(t):(r.prototype=t.prototype,new r)}),a=this&&this.__importDefault||function(e){return e&&e.__esModule?e:{default:e}};Object.defineProperty(t,"__esModule",{value:!0}),t.serializeProduction=t.serializeGrammar=t.Terminal=t.Alternation=t.RepetitionWithSeparator=t.Repetition=t.RepetitionMandatoryWithSeparator=t.RepetitionMandatory=t.Option=t.Alternative=t.Rule=t.NonTerminal=t.AbstractProduction=void 0;var s=a(r(35161)),o=a(r(84486)),l=a(r(47037)),c=a(r(96347)),u=a(r(35937)),h=a(r(28583));function d(e){return t=e,(0,l.default)(t.LABEL)&&""!==t.LABEL?e.LABEL:e.name;var t}var p=function(){function e(e){this._definition=e}return Object.defineProperty(e.prototype,"definition",{get:function(){return this._definition},set:function(e){this._definition=e},enumerable:!1,configurable:!0}),e.prototype.accept=function(e){e.visit(this),(0,o.default)(this.definition,(function(t){t.accept(e)}))},e}();t.AbstractProduction=p;var f=function(e){function t(t){var r=e.call(this,[])||this;return r.idx=1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),Object.defineProperty(t.prototype,"definition",{get:function(){return void 0!==this.referencedRule?this.referencedRule.definition:[]},set:function(e){},enumerable:!1,configurable:!0}),t.prototype.accept=function(e){e.visit(this)},t}(p);t.NonTerminal=f;var m=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.orgText="",(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.Rule=m;var g=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.ignoreAmbiguities=!1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.Alternative=g;var v=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.idx=1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.Option=v;var A=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.idx=1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.RepetitionMandatory=A;var y=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.idx=1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.RepetitionMandatoryWithSeparator=y;var x=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.idx=1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.Repetition=x;var b=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.idx=1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),t}(p);t.RepetitionWithSeparator=b;var w=function(e){function t(t){var r=e.call(this,t.definition)||this;return r.idx=1,r.ignoreAmbiguities=!1,r.hasPredicates=!1,(0,h.default)(r,(0,u.default)(t,(function(e){return void 0!==e}))),r}return i(t,e),Object.defineProperty(t.prototype,"definition",{get:function(){return this._definition},set:function(e){this._definition=e},enumerable:!1,configurable:!0}),t}(p);t.Alternation=w;var _=function(){function e(e){this.idx=1,(0,h.default)(this,(0,u.default)(e,(function(e){return void 0!==e})))}return e.prototype.accept=function(e){e.visit(this)},e}();function E(e){function t(e){return(0,s.default)(e,E)}if(e instanceof f){var r={type:"NonTerminal",name:e.nonTerminalName,idx:e.idx};return(0,l.default)(e.label)&&(r.label=e.label),r}if(e instanceof g)return{type:"Alternative",definition:t(e.definition)};if(e instanceof v)return{type:"Option",idx:e.idx,definition:t(e.definition)};if(e instanceof A)return{type:"RepetitionMandatory",idx:e.idx,definition:t(e.definition)};if(e instanceof y)return{type:"RepetitionMandatoryWithSeparator",idx:e.idx,separator:E(new _({terminalType:e.separator})),definition:t(e.definition)};if(e instanceof b)return{type:"RepetitionWithSeparator",idx:e.idx,separator:E(new _({terminalType:e.separator})),definition:t(e.definition)};if(e instanceof x)return{type:"Repetition",idx:e.idx,definition:t(e.definition)};if(e instanceof w)return{type:"Alternation",idx:e.idx,definition:t(e.definition)};if(e instanceof _){var n={type:"Terminal",name:e.terminalType.name,label:d(e.terminalType),idx:e.idx};(0,l.default)(e.label)&&(n.terminalLabel=e.label);var i=e.terminalType.PATTERN;return e.terminalType.PATTERN&&(n.pattern=(0,c.default)(i)?i.source:i),n}if(e instanceof m)return{type:"Rule",name:e.name,orgText:e.orgText,definition:t(e.definition)};throw Error("non exhaustive match")}t.Terminal=_,t.serializeGrammar=function(e){return(0,s.default)(e,E)},t.serializeProduction=E},62156:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.GAstVisitor=void 0;var n=r(94490),i=function(){function e(){}return e.prototype.visit=function(e){var t=e;switch(t.constructor){case n.NonTerminal:return this.visitNonTerminal(t);case n.Alternative:return this.visitAlternative(t);case n.Option:return this.visitOption(t);case n.RepetitionMandatory:return this.visitRepetitionMandatory(t);case n.RepetitionMandatoryWithSeparator:return this.visitRepetitionMandatoryWithSeparator(t);case n.RepetitionWithSeparator:return this.visitRepetitionWithSeparator(t);case n.Repetition:return this.visitRepetition(t);case n.Alternation:return this.visitAlternation(t);case n.Terminal:return this.visitTerminal(t);case n.Rule:return this.visitRule(t);default:throw Error("non exhaustive match")}},e.prototype.visitNonTerminal=function(e){},e.prototype.visitAlternative=function(e){},e.prototype.visitOption=function(e){},e.prototype.visitRepetition=function(e){},e.prototype.visitRepetitionMandatory=function(e){},e.prototype.visitRepetitionMandatoryWithSeparator=function(e){},e.prototype.visitRepetitionWithSeparator=function(e){},e.prototype.visitAlternation=function(e){},e.prototype.visitTerminal=function(e){},e.prototype.visitRule=function(e){},e}();t.GAstVisitor=i},68877:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.toFastProperties=t.timer=t.PRINT_ERROR=t.PRINT_WARNING=void 0;var n=r(73017);Object.defineProperty(t,"PRINT_WARNING",{enumerable:!0,get:function(){return n.PRINT_WARNING}}),Object.defineProperty(t,"PRINT_ERROR",{enumerable:!0,get:function(){return n.PRINT_ERROR}});var i=r(99870);Object.defineProperty(t,"timer",{enumerable:!0,get:function(){return i.timer}});var a=r(55591);Object.defineProperty(t,"toFastProperties",{enumerable:!0,get:function(){return a.toFastProperties}})},73017:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.PRINT_WARNING=t.PRINT_ERROR=void 0,t.PRINT_ERROR=function(e){console&&console.error&&console.error("Error: ".concat(e))},t.PRINT_WARNING=function(e){console&&console.warn&&console.warn("Warning: ".concat(e))}},99870:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.timer=void 0,t.timer=function(e){var t=(new Date).getTime(),r=e();return{time:(new Date).getTime()-t,value:r}}},55591:(e,t)=>{"use strict";function r(e){function t(){}t.prototype=e;var r=new t;function n(){return typeof r.bar}return n(),n(),e}Object.defineProperty(t,"__esModule",{value:!0}),t.toFastProperties=void 0,t.toFastProperties=r},23558:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.KeepRatio=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=n.__importDefault(r(58147));t.KeepRatio=e=>{let{children:t,width:r,height:n,fullHeight:o=!1,fullWidth:l=!0,rootProps:c,contentProps:u,containerWidth:h,containerHeight:d,keep:p=!0}=e;const f=!o&&l,m=!f,g=100*(m?n/r:r/n);return a.default.createElement("div",{...c??{},className:(0,i.default)(s.default.root,f&&s.default.fullWidth,m&&s.default.fullHeight,p&&s.default.keep,null==c?void 0:c.className)},a.default.createElement("div",{...u??{},className:(0,i.default)(s.default.content,null==u?void 0:u.className)},t),p&&a.default.createElement("svg",{style:{pointerEvents:"none",height:m?"100%":d?`calc(${n} / ${r} * ${h})`:"auto",width:f?"100%":d?`calc(${r} / ${n} * ${d})`:"auto"},viewBox:`0 0 ${f?g:100} ${m?g:100}`}))}},26534:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(23558),t)},22226:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.AccordionItem=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importStar(r(67294));r(11403);const s=r(31665),o=r(6980),l=n.__importDefault(r(73955));t.AccordionItem=e=>{let{open:t,onToggle:r,title:n,className:c,children:u,...h}=e;const d=(0,l.default)("accordion-item-"),[p,f]=(0,a.useState)(t??!1);void 0!==t&&t!==p&&f(t);return a.default.createElement("div",{className:(0,i.default)(c,"mdx-accordion-item",p&&"mdx-accordion-item--open"),...h},a.default.createElement("input",{type:"checkbox",id:d,checked:p}),a.default.createElement("div",{role:"button",className:"mdx-accordion-item__header",onClick:()=>{void 0!==t?r&&r(!p):f((e=>!e))}},a.default.createElement(s.Typography,{className:"mdx-accordion-item__title",variant:"h5",component:"label",htmlFor:d},n),a.default.createElement("div",{className:"mdx-accordion-item__icon"},p?a.default.createElement(o.IconRemove,null):a.default.createElement(o.IconAdd,null))),a.default.createElement("div",{className:"mdx-accordion-item__content-wrapper"},a.default.createElement("div",{className:"mdx-accordion-item__content"},u)))}},66169:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(22226),t)},3309:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.AppCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294));r(93255);t.AppCard=e=>{let{logoSrc:t,logoSrcDark:r,name:n,description:l,link:c,linkLabel:u,...h}=e;return o.default.createElement("div",{...h,className:(0,s.default)(h.className,"mdx-app-card")},(t||r)&&o.default.createElement(a.default,{sources:{dark:r??t??"",light:t??r??""},alt:"string"==typeof n?n:"",className:"mdx-app-card__logo"}),o.default.createElement(i.Typography,{component:"span",variant:"h5",className:"mdx-app-card__name"},n),o.default.createElement(i.Typography,{variant:"subtitle1",className:"mdx-app-card__description"},l),c&&o.default.createElement("a",{href:c,target:"_blank",className:"mdx-app-card__link"},o.default.createElement(i.Button,{size:"large",variant:"outlined"},o.default.createElement(i.Typography,{variant:"label1",component:"span"},u??o.default.createElement(o.default.Fragment,null,"Visit ",n)),o.default.createElement("span",null,o.default.createElement(i.PickIcon,{color:"primary"})))))}},84614:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(3309),t)},12077:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.AssetCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(6980),l=r(26534);r(77877);t.AssetCard=e=>{let{title:t,previewSrc:r,downloadable:n,forceDownload:c=!1,...u}=e;const h=n&&n.length>0;return s.default.createElement("div",{...u,className:(0,a.default)(u.className,"mdx-asset-card",h&&"mdx-asset-card--downloadable")},s.default.createElement("div",{className:"mdx-asset-card__inner"},t&&s.default.createElement(i.Typography,{component:"div",variant:"subtitle2",className:"mdx-asset-card__title"},t),s.default.createElement(l.KeepRatio,{width:16,height:9,fullWidth:!0,rootProps:{className:"mdx-asset-card__image"}},s.default.createElement("img",{src:r,alt:"string"==typeof t&&t||"asset image"}))),h&&s.default.createElement("div",{className:"mdx-asset-card__downloadables"},n.map(((e,r)=>s.default.createElement("a",{href:e.src,target:"_blank",download:!0,onClick:r=>((e,r)=>{if(!c)return;e.preventDefault();const{src:n,filename:i}=r,a=document.createElement("a");a.download=i||"string"==typeof t&&t||"",fetch(n).then((e=>e.blob())).then((e=>{const t=window.URL.createObjectURL(e);a.href=t,window.document.body.appendChild(a),a.click(),window.document.body.removeChild(a)})).catch((e=>{console.error("failed to download asset:"+e),a.href=n,a.target="_blank",window.document.body.appendChild(a),a.click(),window.document.body.removeChild(a)}))})(r,e)},s.default.createElement(i.Button,{key:r,variant:"outlined",size:"small",icon:s.default.createElement(o.IconDownload,null)},e.title))))))}},64314:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(12077),t)},93152:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.Box=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=r(9259);function o(e,t,r,n){const i="string"==typeof t||"number"==typeof t?{xs:t}:t,a=[],s=e=>"number"==typeof e&&n?`${e}${n}`:`${e}`;return["xs","sm","md","lg","xl"].forEach(((t,n)=>{const o=i[t];if(o)a.push([`${e}-${t}`,s(o)]);else{var l;const i=null==(l=a[n-1])?void 0:l[1];a.push([`${e}-${t}`,i||s(r)])}})),Object.fromEntries(a)}r(56216);t.Box=e=>{let{top:t=0,bottom:r=0,className:n,style:l={},children:c,...u}=e;return a.default.createElement("div",{className:(0,i.default)(n,"mdx-box"),style:(0,s.makeStyle)({...l},{...o("mdx-box-top",t,0,"px"),...o("mdx-box-bottom",r,0,"px")}),...u},c)}},32765:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(93152),t)},40299:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.CallToActionButton=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(88746)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294));r(51928);t.CallToActionButton=e=>{let{className:t,variant:r="filled",size:n="large",children:l,...c}=e;return o.default.createElement(a.default,{className:(0,s.default)("mdx-cta-button",t),...c},o.default.createElement(i.Typography,{component:"span",variant:"large"===n?"label1":"label2"},o.default.createElement(i.Button,{size:n,variant:r},l)))}},54442:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(40299),t)},40824:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.CallToActionSection=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(65701);r(8981);const l=r(6980);t.CallToActionSection=e=>{let{label:t,href:r,title:n,columns:c=1,description:u,list:h=[],target:d,border:p=!0,align:f="unset",className:m,children:g,variant:v="outlined",ctaPosition:A="bottom",...y}=e;const x=!!u,b=h.length>0,w=!!r,_=!n||x||w||b?n&&!u&&!b&&w?"title-button":n&&u&&2===c?"full-width":n&&u&&h.length>0?"list":"simple":"title-only",E=r&&"top"===A&&s.default.createElement(o.CallToActionButton,{target:d,href:r,className:"mdx-cta-section__link",variant:v},t);return s.default.createElement("div",{className:(0,a.default)(m,"mdx-cta-section",`mdx-cta-section--${_}`,!p&&"mdx-cta-section--no-border"),...y},s.default.createElement("div",{className:(0,a.default)("mdx-cta-section__container","unset"!==f&&`mdx-cta-section--align-${f}`)},s.default.createElement("div",{className:"mdx-cta-section__header"},s.default.createElement(i.Typography,{component:"h2",className:(0,a.default)("mdx-cta-section__title")},n),s.default.createElement("div",{className:"mdx-cta-section__header__desktop-button"},E)),u&&s.default.createElement(i.Typography,{component:"h3",className:"mdx-cta-section__description"},u),s.default.createElement("div",{className:"mdx-cta-section__header__mobile-button"},E),r&&"bottom"===A&&s.default.createElement(o.CallToActionButton,{target:d,href:r,className:"mdx-cta-section__link",variant:v},t)),h.length>0&&s.default.createElement("div",{className:(0,a.default)("mdx-cta-section__list",!p&&"mdx-cta-section--no-border")},h.map(((e,t)=>{const r=s.default.createElement("div",{className:"mdx-cta-section__list-item",key:t},s.default.createElement("a",{className:"mdx-cta-section__list-item-link",href:null==e?void 0:e.href,target:e.target},s.default.createElement(i.Typography,{component:"div",variant:"body2",className:"mdx-cta-section__item-title"},e.title,s.default.createElement(l.IconExternalLink,{size:"m"}))),s.default.createElement(i.Typography,{variant:"h4",component:"p",className:"mdx-cta-section__item-description"},e.description));return e.href?s.default.createElement("a",{href:e.href,target:"_blank"},r):r}))))}},78256:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(40824),t)},63906:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.DocMetadata=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(88746)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294)),l=r(38840);r(46774);const c=r(98906);t.DocMetadata=e=>{let{className:t,children:r,...n}=e;const{date:u,authors:h}=(0,c.useDocMetadata)(),{content:{authorPage:d}={}}=(0,l.useDocThemeOptions)();return o.default.createElement("div",{className:(0,s.default)(t,"mdx-doc-metadata"),...n},u&&o.default.createElement(i.Typography,{variant:"body2"},u),h&&h.length>0&&o.default.createElement(o.default.Fragment,null,o.default.createElement(i.Typography,{variant:"body2"},"by"," ",h.map(((e,t)=>o.default.createElement(o.default.Fragment,{key:e.key},d?o.default.createElement(a.default,{to:`author/${e.key}`},e.name):e.name,t{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(63906),t)},98906:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.useDocMetadata=void 0;const n=r(70655),i=r(13616),a=r(38840),s=n.__importDefault(r(99486));t.useDocMetadata=()=>{const e=(0,a.useDocThemeOptions)(),{content:{authors:t=[]}={}}=e,{frontMatter:r={}}=(0,i.useDoc)(),{author:n=[],date:o}=r,l=(Array.isArray(n)?n:[n]).map((e=>t.find((t=>e===t.key)))).filter((e=>!!e));return{date:o?(0,s.default)(new Date(o),"MMM d yyyy"):"",authors:l}}},59669:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventAbout=void 0;const n=r(70655).__importDefault(r(67294));r(6420);const i=r(31665);t.EventAbout=e=>{let{children:t}=e;return n.default.createElement("div",{className:"mdx-event-about__container"},n.default.createElement(i.Typography,{variant:"h2",component:"p",className:"mdx-event-about__title"},"About"),n.default.createElement(i.Typography,{variant:"body1"},t))}},92302:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(59669),t)},12610:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventBanner=void 0;const n=r(70655),i=n.__importDefault(r(25812)),a=n.__importDefault(r(67294));r(34822);t.EventBanner=e=>{let{src:t}=e;return a.default.createElement(i.default,{sources:{dark:t,light:t},alt:"event thumbnail",className:"mdx-event-banner__thumbnail"})}},70136:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(12610),t)},69587:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventCTA=void 0;const n=r(70655),i=n.__importDefault(r(67294));r(52399);const a=r(31665),s=n.__importDefault(r(88746)),o=r(6980);t.EventCTA=e=>{let{title:t,label:r,link:n,linkProps:l}=e;return i.default.createElement("div",{className:"mdx-event-cta__container"},t&&i.default.createElement(a.Typography,{variant:"h2",component:"h2",className:"mdx-event-cta__title"},t),i.default.createElement(s.default,{to:n,...l},i.default.createElement(a.Button,{className:"mdx-event-cta__button"},i.default.createElement(a.Typography,{variant:"body1"},r),i.default.createElement(o.IconExternalLink,null))))}},26498:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(69587),t)},7742:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294));r(95497);t.EventCard=e=>{let{thumbnail:t,title:r,date:n,location:l,target:c="_self",...u}=e;return o.default.createElement("a",{target:c,...u,className:(0,s.default)(u.className,"mdx-event-card")},o.default.createElement(a.default,{sources:{dark:t,light:t},alt:r??"event thumbnail",className:"mdx-event-card__thumbnail"}),o.default.createElement(i.Typography,{component:"h5",variant:"subtitle1",className:"mdx-event-card__title"},r),o.default.createElement("div",{className:"mdx-event-card__row"},o.default.createElement("div",{className:"mdx-event-card__info"},o.default.createElement(i.Typography,{className:"mdx-event-card__label",variant:"body3"},"Date"),o.default.createElement(i.Typography,{variant:"body3"},n)),o.default.createElement("div",{className:"mdx-event-card__info"},o.default.createElement(i.Typography,{className:"mdx-event-card__label",variant:"body3"},"Location"),o.default.createElement(i.Typography,{variant:"body3"},l))))}},85535:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(7742),t)},2310:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventCardList=t.EventStatus=void 0;const n=r(70655).__importStar(r(67294));r(90980);const i=r(31665),a=r(65701),s=r(31665);var o;!function(e){e.UPCOMING="Upcoming",e.PAST="Past"}(o=t.EventStatus||(t.EventStatus={}));t.EventCardList=e=>{let{upcoming:t,past:r,...l}=e;const[c,u]=(0,n.useState)(o.UPCOMING),h=c===o.UPCOMING?t:r,[d,p]=(0,n.useState)((null==h?void 0:h.length)>8),[f,m]=(0,n.useState)(8);return n.default.createElement("div",{className:"mdx-event-card-list__container",...l},n.default.createElement(s.Tabs,{activeTab:c,onChange:e=>{u(e),m(8),p((null==h?void 0:h.length)>8)},className:"mdx-event-card-list__tabs"},n.default.createElement(s.TabItem,{key:"upcoming",name:"Upcoming"},"Upcoming"),n.default.createElement(s.TabItem,{key:"past",name:"Past"},"Past")),n.default.createElement("div",null,n.default.createElement(a.Box,{top:{xs:80,sm:96},bottom:80},n.default.createElement(a.Grid,{xs:{cols:1,gap:"80px 16px"},md:{cols:4}},h.slice(0,f).map(((e,t)=>n.default.createElement(a.Grid.Item,{xs:1,key:t},n.default.createElement(a.EventCard,{...e})))),0===h.length?n.default.createElement(i.Typography,null,"No events found."):null)),d&&n.default.createElement(i.Button,{className:"mdx-event-card-list__button",onClick:()=>{m((e=>e+8)),f+6>=(null==h?void 0:h.length)&&p(!1)},size:"large"},"See more")))}},42174:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(2310),t)},30862:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventHeader=void 0;const n=r(70655).__importDefault(r(67294));r(44429);const i=r(31665);t.EventHeader=e=>{let{title:t,description:r}=e;return n.default.createElement("div",{className:"mdx-event-header__container"},n.default.createElement(i.Typography,{variant:"h1",component:"h1",className:"mdx-event-header__label"},t),n.default.createElement(i.Typography,{variant:"subtitle1",className:"mdx-event-header__content"},r))}},45320:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(30862),t)},74350:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventInfo=void 0;const n=r(70655).__importDefault(r(67294));r(94318);const i=r(31665);t.EventInfo=e=>{let{label:t,content:r}=e;return n.default.createElement("div",{className:"mdx-event-info__container"},n.default.createElement(i.Typography,{variant:"body2",className:"mdx-event-info__label"},t),n.default.createElement(i.Typography,{variant:"body1",className:"mdx-event-info__content"},r))}},94030:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(74350),t)},46450:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventProfile=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(67294));r(6465);const o=r(6980);t.EventProfile=e=>{let{image:t,name:r,role:n,email:l,linkedin:c,...u}=e;return s.default.createElement("div",{...u,className:"mdx-event-profile"},s.default.createElement(a.default,{sources:{dark:t,light:t},alt:t??"event profile",className:"mdx-event-profile__image"}),s.default.createElement("div",{className:"mdx-event-profile__info"},s.default.createElement(i.Typography,{component:"p",variant:"subtitle1",className:"mdx-event-profile__title"},r),s.default.createElement(i.Typography,{variant:"body2",className:"mdx-event-profile__role"},n),s.default.createElement("div",{className:"mdx-event-profile__social"},l&&s.default.createElement("a",{href:`mailto:${l}`,target:"_blank",rel:"noopener noreferrer"},s.default.createElement(o.IconEmail,null)),c&&s.default.createElement("a",{href:c,target:"_blank",rel:"noopener noreferrer"},s.default.createElement(o.IconLinkedin,null)))))}},7472:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(46450),t)},49310:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.EventProfileList=void 0;const n=r(70655).__importDefault(r(67294));r(54627);const i=r(31665),a=r(65701);t.EventProfileList=e=>{let{title:t,description:r,data:s}=e;return n.default.createElement("div",{className:"mdx-event-profile-list__container"},t&&n.default.createElement(i.Typography,{variant:"h2",component:"h2",className:"mdx-event-profile-list__title"},t),r&&n.default.createElement(i.Typography,{variant:"subtitle1"},r),n.default.createElement(a.Box,{top:40},n.default.createElement(a.Grid,{xs:{cols:1,gap:"16px"},md:{cols:3}},s.map(((e,t)=>n.default.createElement(a.Grid.Item,{xs:1,key:t},n.default.createElement(a.EventProfile,{...e})))))))}},24812:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(49310),t)},58135:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.ExternalResourceCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294)),l=r(6980);r(84238);const c=r(26534);t.ExternalResourceCard=e=>{let{title:t,logoSrc:r,logoSrcDark:n,description:u,previewSrc:h,previewSrcDark:d,...p}=e;const f=!(!h&&!d);return o.default.createElement("a",{target:"_blank",...p,className:(0,s.default)(p.className,"mdx-erc",f&&"mdx-erc--with-preview")},f&&o.default.createElement(c.KeepRatio,{width:16,height:9,fullWidth:!0},o.default.createElement(a.default,{sources:{dark:d??h??"",light:h??d??""},alt:"string"==typeof t&&t||"preview image",className:"mdx-erc__preview-image"})),o.default.createElement("div",{className:"mdx-erc__inner"},(r||n)&&o.default.createElement(a.default,{sources:{dark:n??r??"",light:r??n??""},alt:"string"==typeof t&&t||"logo",className:"mdx-erc__logo"}),o.default.createElement(i.Typography,{variant:"body1",component:"div",className:"mdx-erc__title"},t),u&&o.default.createElement(i.Typography,{variant:"label2",component:"div",className:"mdx-erc__description"},u)),o.default.createElement("div",{className:"mdx-erc__icon"},o.default.createElement(l.IconExternalLink,{className:"mdx-erc__external-link"})))}},24084:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(58135),t)},24646:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.FeatureList=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294));r(69774);t.FeatureList=e=>{let{title:t="Features",alignment:r="bottom",features:n=[],className:o,ctaPosition:l="bottom",indexStyle:c="circle",borderStyle:u="solid",children:h,...d}=e;return s.default.createElement("div",{className:(0,a.default)(o,"mdx-feature-list",`mdx-feature-list--${r}-aligned`,`mdx-feature-list--cta-${l}`,"none"===u&&"mdx-feature-list--border-none"),...d},s.default.createElement("div",{className:"mdx-feature-list__header"},s.default.createElement(i.Typography,{variant:"h2",component:"h2",className:"mdx-feature-list__title"},t),h&&"top"===l&&s.default.createElement("div",{className:"mdx-feature-list__extra"},h)),s.default.createElement("div",{className:"mdx-feature-list__list"},n.map(((e,t)=>s.default.createElement("div",{key:t,className:(0,a.default)("mdx-feature-list__feature","none"===u&&"mdx-feature-list--border-none")},s.default.createElement("div",{className:"mdx-feature-list__feature-inner"},"circle"===c?s.default.createElement(i.Typography,{variant:"subtitle1",component:"div",className:(0,a.default)("mdx-feature-list__feature-index--circle")},t+1):s.default.createElement(i.Typography,{variant:"subtitle1",component:"div",className:(0,a.default)("mdx-feature-list__feature-index--number")},"[",function(e,t){for(e=e.toString();e.length{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(24646),t)},51499:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.GithubChallenges=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(67294)),s=r(696),o=r(79527),l=e=>{let{message:t}=e;return a.default.createElement(a.default.Fragment,null,a.default.createElement(i.Typography,{variant:"h1",className:"mdx-ghc__header"},"Open challenges"),!!t&&a.default.createElement(i.Typography,{variant:"body1"},t))};t.GithubChallenges=e=>{let{challengesData:t,useDummyData:r,subheaderText:n,filterByName:c,...u}=e,h=r?[s.dummyGithubIssue]:(e=>{let t=[];return Object.keys(e).forEach((r=>{if(!isNaN(Number(r))){const n=Object.values(e[r])[0];Array.isArray(n)&&(t=t.concat(n))}})),t})(t);if(!h||(d=h,!Array.isArray(d)||!d.length))return a.default.createElement(l,{message:"No challenges to show"});var d;if(c)try{var p;const e=new RegExp(c,"i");h=null==(p=h)?void 0:p.filter((t=>e.test(t.title)))}catch(f){console.error("Invalid regex pattern:",f)}return a.default.createElement("div",{...u},a.default.createElement(l,null),!!n&&a.default.createElement(i.Typography,{variant:"body1",className:"mdx-ghc-subheader-text",component:"div"},n),h.map((e=>a.default.createElement(o.SingleGithubChallenge,{key:e.id,issue:e}))))}},79527:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.SingleGithubChallenge=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(67294)),s=r(6980);r(58918);const o=n.__importDefault(r(88746));function l(e,t){void 0===t&&(t=24);const r=new URL(e),n=new URLSearchParams(r.search);return n.set("s",t.toString()),r.search=Array.from(n.entries()).map((e=>{let[t,r]=e;return`${t}=${r}`})).join("&"),r.toString()}t.SingleGithubChallenge=e=>{let{issue:t}=e;const r=(e=>{const t=new Map;return t.set(e.user.login,{name:e.user.login,avatarUrl:e.user.avatarUrl}),e.assignees.forEach((e=>{t.set(e.login,{name:e.login,avatarUrl:e.avatarUrl})})),e.comments.forEach((e=>{t.set(e.author.login,{name:e.author.login,avatarUrl:e.author.avatarUrl})})),Array.from(t.values())})(t),n=(e=>{const t=e.projects.map((e=>e.name));return 0===t.length?"":Array.from(new Set(t)).join(", ")})(t),c=""!==n,u=!!t.milestone,h=[];return c&&h.push(a.default.createElement(a.default.Fragment,null,a.default.createElement(i.Typography,{variant:"body3",className:"mdx-ghc__label"},n.includes(", ")?"Projects":"Project"),a.default.createElement(i.Typography,{variant:"body3",className:"mdx-ghc__project-name"},n))),u&&h.push(a.default.createElement(a.default.Fragment,null,a.default.createElement(i.Typography,{variant:"body3",className:"mdx-ghc__label"},"Milestone"),a.default.createElement(i.Typography,{variant:"body3",className:"mdx-ghc__milestone-text"},t.milestone))),a.default.createElement("div",{className:"mdx-ghc__container"},a.default.createElement(o.default,{href:t.url,className:"mdx-ghc__issue-title-link"},a.default.createElement(i.Typography,{variant:"h5",className:"mdx-ghc__issue-title"},t.title)),a.default.createElement("div",{className:"mdx-ghc__challenge-labels"},t.labels.map(((e,t)=>a.default.createElement(i.Typography,{variant:"body3",key:t,className:"mdx-ghc__challenge-label"},e)))),a.default.createElement("div",{className:"mdx-ghc__issue-content-grid"},a.default.createElement(i.Typography,{variant:"body3",className:"mdx-ghc__label"},"Participants"),a.default.createElement("div",{className:"mdx-ghc__participant-photo-container"},a.default.createElement(i.Typography,{variant:"body3",className:"mdx-ghc__comment-count",component:"div"},r.length),r.map(((e,t)=>a.default.createElement("img",{key:t,className:"mdx-ghc__participant-photo",src:l(e.avatarUrl),alt:e.name})))),h.map(((e,t)=>a.default.createElement(a.default.Fragment,{key:t},e)))),a.default.createElement(o.default,{href:t.url,className:"mdx-ghc__view-on-github-link"},a.default.createElement(i.Button,{className:"mdx-ghc__view-on-github-button",icon:a.default.createElement(s.IconExternalLink,null)},"View on GitHub")))}},696:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.dummyGithubIssue=void 0,t.dummyGithubIssue={id:"12345",title:"Bug in pagination",body:"When navigating to the second page, the first item repeats.",url:"https://github.com/user/repo/issues/12345",user:{login:"user123",avatarUrl:"https://avatars.githubusercontent.com/u/8811422?v=4"},labels:["bug","frontend"],commentCount:3,comments:[{id:"c1",author:{login:"alice123",avatarUrl:"https://avatars.githubusercontent.com/u/8811422?v=4"},body:"I have also noticed this issue. Working on a fix now.",createdAt:"2021-01-01T12:00:00Z"},{id:"c2",author:{login:"bob456",avatarUrl:"https://avatars.githubusercontent.com/u/8811422?v=4"},body:"Any updates on this?",createdAt:"2021-01-02T15:30:00Z"},{id:"c3",author:{login:"jaquim",avatarUrl:"https://avatars.githubusercontent.com/u/8811422?v=4"},body:"I like turtles.",createdAt:"2021-01-03T09:45:00Z"}],assignees:[{login:"alice123",avatarUrl:"https://avatars.githubusercontent.com/u/8811422?v=4"},{login:"bob456",avatarUrl:"https://avatars.githubusercontent.com/u/8811422?v=4"}],milestone:"v1.0.0",created_at:"2020-12-31T11:00:00Z",updated_at:"2021-01-04T13:00:00Z",projects:[{name:"Awesome Project"}]}},47513:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(51499),t)},7859:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.Grid=void 0;const n=r(70655),i=r(31665),a=r(70917),s=n.__importDefault(r(77366)),o=n.__importDefault(r(86010)),l=n.__importStar(r(67294)),c=r(59567),u=r(3796),h=r(18031);t.Grid=e=>{let{actions:t,leftLabel:r="",rightLabel:n="",spacingButtons:i=!1,children:a,...s}=e;const c=(0,l.useRef)(null);return l.default.createElement(d,{...s,className:(0,o.default)(s.className,"mdx-grid")},l.default.createElement("div",{className:"mdx-grid__actions"},t,l.default.createElement(u.ScrollButtons,{containerRef:c,className:"mdx-grid__scroll",leftLabel:r,rightLabel:n,spacing:i?"spaced":"grouped"})),l.default.createElement("div",{ref:c,className:(0,o.default)("mdx-grid__content","hidden-scrollbar")},a))},t.Grid.Item=h.GridItem;const d=s.default.div` - width: 100%; - - .mdx-grid__scroll { - display: flex; - } - - .mdx-grid__content { - display: grid; - gap: var(--grid-gap); - grid-template-columns: repeat(var(--grid-cols), minmax(0, 1fr)); - overflow: hidden; - } - - .mdx-grid__actions { - display: flex; - flex-direction: row; - align-items: center; - gap: 1rem; - - & > * { - margin-bottom: 3rem; - } - } - - ${e=>i.THEME_BREAKPOINTS.map((t=>{if(!e[t])return null;const r=e[t];return c.lsdUtils.responsive(e.theme,t,"up")(a.css` - ${void 0!==r.cols&&`\n --grid-cols: ${r.cols};\n `} - - ${void 0!==r.gap&&`\n --grid-gap: ${r.gap};\n `} - - ${(void 0===r.wrap||!0===r.wrap)&&a.css` - .mdx-grid__scroll { - display: none; - } - .mdx-grid__content { - display: grid; - flex-wrap: unset; - overflow-x: unset; - overflow-y: unset; - scroll-snap-type: unset; - } - `} - - ${void 0!==r.wrap&&!1===r.wrap&&a.css` - .mdx-grid__scroll { - display: flex; - } - .mdx-grid__content { - display: flex; - flex-wrap: nowrap; - overflow-x: scroll; - overflow-y: hidden; - scroll-snap-type: x mandatory; - } - `} - - ${!1===r.scrollButtons&&a.css` - .mdx-grid__scroll { - display: none; - } - `} - `)}))} -`},18031:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.GridItem=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=n.__importDefault(r(77366)),o=r(31665),l=r(59567),c=r(70917);t.GridItem=e=>{let{children:t,...r}=e;return a.default.createElement(u,{...r,className:(0,i.default)(r.className)},t)};const u=s.default.div` - ${e=>o.THEME_BREAKPOINTS.map((t=>{if(!e[t])return null;const r=e[t];return l.lsdUtils.responsive(e.theme,t,"up")(c.css` - grid-column: span ${r}; - flex-basis: calc(100% / var(--grid-cols) * ${r}); - `)}))} -`},78378:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});const n=r(70655);n.__exportStar(r(7859),t),n.__exportStar(r(18031),t)},94787:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.useHero=t.HeroContext=void 0;const n=r(70655).__importStar(r(67294));t.HeroContext=n.default.createContext({size:"medium"});t.useHero=()=>(0,n.useContext)(t.HeroContext)},53255:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.Hero=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=r(94787);r(91786);t.Hero=e=>{let{size:t="medium",className:r,children:n,verticalAlign:o="top",...l}=e;return a.default.createElement(s.HeroContext.Provider,{value:{size:t}},a.default.createElement("div",{className:(0,i.default)(r,"mdx-hero",`mdx-hero--${t}`,`mdx-hero--${o}`),...l},n))}},72295:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(53255),t)},23908:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroAction=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(88746)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294));r(85661);t.HeroAction=e=>{let{size:t="large",variant:r="outlined",className:n,children:l,...c}=e;return o.default.createElement(a.default,{className:(0,s.default)(n,"mdx-hero-action",`mdx-hero-action--${t}`),...c},o.default.createElement(i.Button,{variant:r,size:t},o.default.createElement(i.Typography,{component:"span",variant:"label1",style:{color:"inherit"}},l)))}},48126:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(23908),t)},82897:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroActions=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294));r(57255);t.HeroActions=e=>{let{className:t,children:r,...n}=e;return a.default.createElement("div",{className:(0,i.default)(t,"mdx-hero-actions"),...n},r)}},16693:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(82897),t)},4891:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroAsset=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294));r(67890);t.HeroAsset=e=>{let{className:t,children:r,...n}=e;return a.default.createElement("div",{className:(0,i.default)(t,"mdx-hero-asset"),...n},r)}},38716:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(4891),t)},28795:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroDescription=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(94787);r(30204);t.HeroDescription=e=>{let{size:t,className:r,uppercase:n=!1,children:l,...c}=e;const u=(0,o.useHero)(),h=t??(u?u.size:"medium");return s.default.createElement(i.Typography,{variant:"h4",className:(0,a.default)(r,"mdx-hero-description",`mdx-hero-description--${h}`,n?"mdx-hero-description--uppercase":""),...c},l)}},22447:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(28795),t)},78274:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroInfo=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=r(67713),o=r(94787),l=r(65701);r(6460);t.HeroInfo=e=>{let{size:t,className:r,hideScrollToBottom:n=!1,children:c,...u}=e;const h=(0,o.useHero)(),d=(0,s.useHydrated)(),p=t||(h?h.size:"medium");return a.default.createElement(a.default.Fragment,null,a.default.createElement("div",{className:(0,i.default)(r,"mdx-hero-info",`mdx-hero-info--${p}`),...u},c),d&&!n&&a.default.createElement(l.ScrollToBottom,null))}},67812:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(78274),t)},84773:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.AsciiRenderer=void 0;const n=r(70655),i=r(44169),a=r(64232),s=n.__importStar(r(67294)),o=r(84849),l=r(24052);t.AsciiRenderer=e=>(function(e){void 0===e&&(e={});const{renderIndex:t=i.defaultAsciiConfigs.renderIndex,characters:r=i.defaultAsciiConfigs.characters,invert:n=i.defaultAsciiConfigs.invert,color:c=i.defaultAsciiConfigs.color,resolution:u=i.defaultAsciiConfigs.resolution,bgColor:h=i.defaultAsciiConfigs.bgColor,fgColor:d=i.defaultAsciiConfigs.fgColor,textShadowSize:p=i.defaultAsciiConfigs.textShadowSize,withTextShadow:f=!1}=e,{gl:m,size:g,scene:v,camera:A,viewport:y}=(0,a.useThree)(),x=(0,s.useMemo)((()=>{const e=new o.AsciiEffect(m,r,{invert:n,color:c,resolution:u});return e.domElement.style.position="absolute",e.domElement.style.top="0px",e.domElement.style.left="0px",e.domElement.style.pointerEvents="none",e}),[r,n,c,u]);(0,s.useLayoutEffect)((()=>{x.domElement.style.color=d}),[d,h]),(0,s.useEffect)((()=>{if(m.domElement.style.opacity="0",m.domElement.parentNode.appendChild(x.domElement),f){const e=document.createElement("style");e.innerHTML=`table *{text-shadow: ${(0,l.generateTextShadow)(p)};`,x.domElement.appendChild(e)}return()=>{m.domElement.style.opacity="1",m.domElement.parentNode.removeChild(x.domElement)}}),[x]),(0,s.useEffect)((()=>{x.setSize(g.width,g.height)}),[x,g]),(0,a.useFrame)((e=>{x.render(v,A)}),t)}(e),s.default.createElement(s.default.Fragment,null))},35901:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.Controls=void 0;const n=r(70655),i=r(44169),a=n.__importStar(r(67294)),s=r(40213),o=r(64232),l=r(90278),c=r(24052),u=(e,t,r)=>(1-r)*e+r*t;function h(e,t,r,n){const i=Math.min(e,t)/t,a=[0,0,0];for(let s=0;s<3;s++)a[s]=u(r[s],n[s],i);return a}t.Controls=e=>{let{rotateSpeed:t=i.ROTATE_SPEED,enableZoom:r=!0,enableRotateOnScroll:n=!0,preset:u,targetPreset:d,children:p,...f}=e;const m=(0,a.useRef)(),g=(0,s.useScrollY)(),{camera:v,size:A}=(0,o.useThree)(),y=(0,a.useRef)(),[x,b]=a.default.useState(!1),[w,_]=a.default.useState(1),[E,S]=a.default.useState(0);return(0,o.useFrame)(((e,r)=>{m.current.rotation.y-=r*((0,c.isMobile)()?.6*t:t)})),(0,a.useEffect)((()=>{const e=()=>{console.log(JSON.stringify({cameraPos:v.position.toArray(),cameraRot:v.rotation.toArray().slice(0,3),controlsTarget:y.current.target.toArray()},null,2))};return window.addEventListener("click",e),()=>window.removeEventListener("click",e)}),[v]),(0,a.useEffect)((()=>{(0,c.isTouchDevice)()&&(y.current.minPolarAngle=Math.PI/2,y.current.maxPolarAngle=Math.PI/2,setTimeout((()=>{b(!0)}),1e3))}),[]),(0,a.useEffect)((()=>{(0,c.isMobile)()&&(_((0,c.mapFloat)(g,0,(0,c.calcScrollThreshold)(),1,.65)),S((0,c.mapFloat)(g,0,(0,c.calcScrollThreshold)(),0,.35)))}),[g]),(0,a.useEffect)((()=>{if(!r)return;if(!d)return;const e=h(g,400,u.cameraPos,d.cameraPos),t=h(g,400,u.cameraRot,d.cameraRot),n=h(g,400,u.controlsTarget,d.controlsTarget);v.position.set(...e),v.rotation.set(...t),y.current.target.set(...n),v.updateProjectionMatrix()}),[g,v]),a.default.createElement("group",{ref:m,...f,scale:w,"position-y":E},p,a.default.createElement(l.OrbitControls,{ref:y,enableZoom:!1,target:y.current?y.current.target:u.controlsTarget,enabled:!x}))}},44169:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.OBJECTS_PRESETS=t.defaultPresets=t.defaultAsciiConfigs=t.INITIAL_ZOOM=t.RESIZE_SPEED_FACTOR=t.MIN_ZOOM=t.MAX_ROTATE_SPEED=t.MIN_ROTATE_SPEED=t.ROTATE_SPEED=void 0,t.ROTATE_SPEED=.1,t.MIN_ROTATE_SPEED=0,t.MAX_ROTATE_SPEED=1,t.MIN_ZOOM=.22,t.RESIZE_SPEED_FACTOR=.9,t.INITIAL_ZOOM=3,t.defaultAsciiConfigs={renderIndex:1,bgColor:"rgb(var(--lsd-surface-primary))",fgColor:"rgb(var(--lsd-text-primary))",characters:" l.o.g.o.s ",invert:!1,color:!1,resolution:.21,textShadowSize:15},t.defaultPresets={modelId:"default",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[-2.733342169570335,1.127956558492365,2.7690180385429666],controlsTarget:[-.2185887974027981,3.4320197290105474,-.08409377618890646]},abstract:{cameraPos:[-1.4826176635786852,4.021180061821954,-1.5929058418153597],cameraRot:[-2.9244096935808908,-.8625529112689497,-2.9755407843387185],controlsTarget:[-.3236695017538898,3.8072918272567,-.6236093222013962]}},t.OBJECTS_PRESETS=[t.defaultPresets,{modelId:"architecture01",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"architecture02",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"architecture03",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"architecture04",simple:{cameraPos:[2.263701079468784,.6448855513810133,-1.1446840846564066],cameraRot:[-2.3314571674867457,.984733935216302,2.422537357648925],controlsTarget:[.6593539926170827,-.1266335925222026,-.4104215479626598]},abstract:{cameraPos:[1.8397825927313005,.5006635210357668,-.7112749496103248],cameraRot:[-2.0637769654190388,.941591559117755,2.1572780914058356],controlsTarget:[.7525282128865571,-.19650999135957886,-.33673737792715125]},targetLook:{cameraPos:[1.2775067913826557,1.3342390202957728,-3.123518037652026],cameraRot:[-2.6905895797482686,.32826936260575107,2.9867046984290964],controlsTarget:[.27965301570358253,.05724884619536415,-.48670374558576035]}},{modelId:"architecture05",simple:{cameraPos:[2.783916402572475,.8560027544276005,2.7596737879252995],cameraRot:[-.16075199314229247,.8310730403125255,.11920589473418516],controlsTarget:[-.13283501637059433,.4299574965789378,.13221598116033606]},abstract:{cameraPos:[-.5465903796360435,.8773980794748312,.08538618930043433],cameraRot:[-.26855166005001807,-1.1713993239674987,-.24830772343977955],controlsTarget:[-.0544632211815667,.8222830795164463,-.1148867151943581]},targetLook:{cameraPos:[-.08633866196975185,1.065498410363367,3.299441795447572],cameraRot:[-.30705338387773307,-.04232616594886318,-.01341603621000156],controlsTarget:[.057360484222938594,.03995565554668334,.06512362298065871]}},{modelId:"architecture06",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"architecture07",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"atlas",simple:{cameraPos:[-.05747471409961126,1.1561369169508278,.2934743027588207],cameraRot:[.13523803197626882,-.45004875426349367,.05912213357583956],controlsTarget:[-.03052411570622545,1.1636581998026059,.23819862568800665]},abstract:{cameraPos:[-.18886266143333627,1.0759713173211645,.49472614307040697],cameraRot:[.11827338438455919,-.49355424749746096,.056236459481599846],controlsTarget:[-.03586918676122722,1.1095292429426495,.21231853618809665]},targetLook:{cameraPos:[-.6194495673706852,.6791728914823414,1.1145248759455844],cameraRot:[.08253419059538356,-.589296072042965,.045942607479927136],controlsTarget:[.09058057232790886,.7667301800204325,.05607398084523753]}},{modelId:"bust01",simple:{cameraPos:[-.6865425525854476,.9101267370893742,.6192780523604176],cameraRot:[-.39660492694232563,-.962317303218196,-.3309893133032883],controlsTarget:[.008888669206507317,.7229784900626205,.17240700391361993]},abstract:{cameraPos:[.025758408225725123,.9432728632646389,.5085343068565109],cameraRot:[-.016796160047877214,.23199352927595504,.003862085219634371],controlsTarget:[-.04849025797636223,.9379943498483801,.19429480114059927]},targetLook:{cameraPos:[-.0995637601904456,.9905193985583883,1.2844592429465957],cameraRot:[-.21489715158374015,-.011908392248228742,-.002599147273856083],controlsTarget:[-.08519853311178426,.7332893758986562,.10595091334036527]}},{modelId:"bust02",simple:{cameraPos:[-1.4566842350476759,1.3228318382357354,.3529107224325513],cameraRot:[-.7212161402570414,-1.161345100016008,-.6787189972933454],controlsTarget:[.13759890903780844,.8659810364239205,-.1666973840261623]},abstract:{cameraPos:[-.3488062269042841,1.3111379960412364,.033354968216059155],cameraRot:[-.7212161402570405,-1.1613451000160089,-.6787189972933446],controlsTarget:[.2227218365835886,1.147363414950647,-.15291722311874764]}},{modelId:"bust03",simple:{cameraPos:[6.898858137575106,4.772099506970454,-3.1821660872368627],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"discobolus",simple:{cameraPos:[.9321278495515372,1.1243517299151449,1.0437243089401456],cameraRot:[-.05449607597993083,.5948614803388417,.030560026854436973],controlsTarget:[.17851738112859908,1.0636847191934362,-.06840974825492452]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"hand",simple:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"vase01",simple:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}},{modelId:"venus",simple:{cameraPos:[-.33960257176056113,1.3731114512210183,.5843851218219973],cameraRot:[.19165735697369993,-.4160172409280952,.07825504508386714],controlsTarget:[.1766806722765915,1.595709756901872,-.5627979418735829]},abstract:{cameraPos:[-.2543046264127692,1.591894996466899,.21161310295797725],cameraRot:[.17078534527374745,-.637502107706861,.10229142114685894],controlsTarget:[.2766995177502174,1.7137410300309837,-.49488235116491014]}},{modelId:"flower",simple:{cameraPos:[-.33960257176056113,1.3731114512210183,.5843851218219973],cameraRot:[0,0,0],controlsTarget:[0,0,0]},abstract:{cameraPos:[0,0,0],cameraRot:[0,0,0],controlsTarget:[0,0,0]}}]},9354:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroModel=void 0;const n=r(70655),i=r(90278),a=r(64232),s=n.__importDefault(r(86010)),o=n.__importStar(r(67294)),l=r(44169),c=r(24052),u=r(84773),h=r(35901);r(61588);const d=r(40213);function p(e){let{url:t,onMount:r=(()=>{}),...n}=e;const{scene:a}=(0,i.useGLTF)(t,"/scripts/draco-1.4.3/");return(0,o.useEffect)((()=>{r()}),[]),o.default.createElement("primitive",{object:a,...n})}t.HeroModel=e=>{const{modelId:t,preset:r,mode:n="simple",className:i,children:f,asciiConfig:m,rotateSpeed:g=l.ROTATE_SPEED,enableZoom:v,enableRotateOnScroll:A,withParallelEffect:y=!0,startY:x="bottom",...b}=e,w=((e,t,r)=>(0,o.useMemo)((()=>{if(t)return{...l.defaultPresets,[e]:t};if(!r)return l.defaultPresets;return l.OBJECTS_PRESETS.find((e=>e.modelId===r))||l.defaultPresets}),[t,r]))(n,r,t),_=(0,d.useScrollY)();return o.default.createElement("div",{className:(0,s.default)(i,"mdx-hero-model","mdx-hero-model--ascii",`mdx-hero-model--${n}`,`mdx-hero-model--${x}`),...b},o.default.createElement("div",{className:"mdx-hero-model--inner",style:{...y?{transform:"translateY(0px) scale(var(--mdx-hero-model-wrapper-scale))"}:{}}},o.default.createElement(o.Suspense,{fallback:o.default.createElement("span",null," ")},o.default.createElement(a.Canvas,{dpr:[1,2],style:{height:"100vh"},camera:{fov:50,position:w[n].cameraPos,rotation:w[n].cameraRot}},o.default.createElement("directionalLight",{position:[-10,10,0],intensity:1.5}),o.default.createElement("directionalLight",{position:[-10,10,5],intensity:.3}),o.default.createElement("directionalLight",{position:[-10,20,0],intensity:1.5}),o.default.createElement("directionalLight",{position:[100,-10,0],intensity:.25}),o.default.createElement(h.Controls,{rotateSpeed:g,preset:w[n],targetPreset:w.targetLook,enableZoom:"simple"!==n},o.default.createElement(o.Suspense,{fallback:o.default.createElement(p,{url:`/hero/${w.modelId}/lo.glb`})},o.default.createElement(p,{url:`/hero/${w.modelId}/hi.glb`}),o.default.createElement(u.AsciiRenderer,{...m})))))),o.default.createElement("div",{className:"mdx-hero-model--shade",style:{opacity:(0,c.mapFloat)(_,0,(0,c.calcScrollThreshold)()*l.RESIZE_SPEED_FACTOR,0,1)}}))}},28292:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroModel=void 0;const n=r(70655).__importDefault(r(67294)),i=r(24052),a=r(40213),s=r(9354);t.HeroModel=e=>{if("undefined"==typeof window)return null;const t=(0,a.useScrollY)();(0,i.mapFloat)(t,0,(0,i.calcScrollThreshold)(),0,.1*window.innerHeight);return n.default.createElement("div",{style:{position:"absolute",top:0,left:0}},n.default.createElement(s.HeroModel,{...e}))}},59533:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroTitle=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(94787);r(92751);t.HeroTitle=e=>{let{size:t,uppercase:r,className:n,children:l,...c}=e;const u=(0,o.useHero)(),h=t||(u?u.size:"medium"),d=void 0!==r?r:"large"===(null==u?void 0:u.size);return s.default.createElement(i.Typography,{variant:"h1",component:"h1",className:(0,a.default)(n,"mdx-hero-title",`mdx-hero-title--${h}`,d&&"mdx-hero-title--uppercase"),...c},l)}},85486:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(59533),t)},37207:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.HeroVideo=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importStar(r(67294)),s=r(9259),o=r(32833),l=r(67713),c=r(94787);r(53175);t.HeroVideo=e=>{let{placeholderSrc:t,desktop:r,mobile:n,className:u,style:h={},children:d,...p}=e;(0,c.useHero)();const f=(0,l.useHydrated)(),m=(0,a.useRef)(null),[g,v]=(0,a.useState)(!0),A=async()=>{const e=m.current;if(!e)return;e.muted=!0,e.defaultMuted=!0;const[t,r]=await(0,o.settle)((()=>e.play()));r||v(!1)};return(0,a.useEffect)((()=>{const e=document.querySelector(".col > article");if(e)return e.classList.add("overflow-hidden"),()=>{e.classList.remove("overflow-hidden")}}),[]),a.default.createElement(a.default.Fragment,null,a.default.createElement("div",{className:(0,i.default)(u,"mdx-hero-video",g&&"mdx-hero-video--loading"),style:(0,s.makeStyle)({...h},{"hero-video-scale":(null==r?void 0:r.scale)??"1.70951586","hero-video-offset-y":(null==r?void 0:r.offsetY)??"-150px","hero-video-height":(null==r?void 0:r.height)??"100%","hero-video-min-height":(null==r?void 0:r.minHeight)??"min(100vh, var(--hero-max-height))","hero-video-scale-mobile":(null==n?void 0:n.scale)??"1.70951586","hero-video-offset-y-mobile":(null==n?void 0:n.offsetY)??"-50px","hero-video-height-mobile":(null==n?void 0:n.height)??"120%","hero-video-min-height-mobile":(null==n?void 0:n.minHeight)??"100vh"}),...p},a.default.createElement("div",{className:"mdx-hero-video__placeholder"},a.default.createElement("img",{src:t,alt:""})),a.default.createElement("div",{className:"mdx-hero-video__video"},f&&a.default.createElement("video",{ref:m,loop:!0,muted:!0,autoPlay:!0,playsInline:!0,onCanPlay:A,onLoadedMetadata:A},d))))}},98130:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(37207),t)},66210:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.InputCTASection=void 0;const n=r(70655),i=n.__importDefault(r(67294));r(43871);const a=r(31665),s=n.__importDefault(r(88746)),o=n.__importDefault(r(86010));t.InputCTASection=e=>{let{title:t,description:r,label:n,link:l,linkProps:c,formInput:u,buType:h,newsletterId:d=null,successMessage:p="Thank you for subscribing!"}=e;const[f,m]=i.default.useState({email:"",name:""}),[g,v]=i.default.useState(""),A=e=>{m({...f,[e.target.name]:e.target.value})};return i.default.createElement("div",{className:"mdx-input-cta-section__container"},t&&i.default.createElement(a.Typography,{variant:"subtitle1",component:"p",className:"mdx-input-cta-section__title"},t),i.default.createElement(a.Typography,{component:"h3",variant:"h3",className:(0,o.default)("input-cta-section__description",!u&&"input-cta-section__description--no-form")},r),null!=u&&u.length?i.default.createElement("form",{className:"mdx-input-cta-section__form",onSubmit:async e=>{if(e.preventDefault(),null!=f&&f.email)try{var t,r,n,i,a,s;const e=await fetch("https://odoo.logos.co/website_mass_mailing/subscribe_ghost",{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({jsonrpc:"2.0",method:"call",params:null==d?{email:null==f?void 0:f.email,type:h,subscription_type:"email"}:{email:null==f?void 0:f.email,type:h,subscription_type:"email",newsletter:d}})}),l=await e.json();var o;if(null!=l&&null!=(t=l.result)&&t.errors&&null!=l&&null!=(r=l.result)&&null!=(n=r.errors[0])&&null!=(i=n.context)&&i.length)return void v(null==l||null==(o=l.result)?void 0:o.errors[0].context);if(null!=l&&null!=(a=l.result)&&null!=(s=a.message)&&s.length)return void v(l.result.message);v(p)}catch(l){console.log(l),v("There was an error submitting the form.")}else v("Please enter an email address")}},i.default.createElement("div",{className:"mdx-input-cta-section__input-fields"},u.map(((e,t)=>i.default.createElement(a.TextField,{key:t,className:"mdx-input-cta-section__input",onChange:A,value:f[e.name],inputProps:{...e}})))),i.default.createElement("div",null,i.default.createElement(a.Button,{type:"submit",className:"mdx-input-cta-section__cta"},i.default.createElement(a.Typography,{variant:"body1"},n)))):i.default.createElement(s.default,{to:l,...c},i.default.createElement(a.Button,{className:"mdx-input-cta-section__cta"},i.default.createElement(a.Typography,{variant:"body1"},n))),g&&i.default.createElement(a.Typography,{className:"mdx-input-cta-section__message",variant:"body2"},g))}},74603:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(66210),t)},48949:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.JobsPerDepartment=void 0;const n=r(70655),i=r(31665),a=r(744),s=n.__importDefault(r(67294)),o=r(31174),l=e=>{let{message:t}=e;return s.default.createElement(s.default.Fragment,null,s.default.createElement(i.Typography,{variant:"h1",className:"mdx-jpd__header"},"Current job openings"),!!t&&s.default.createElement(i.Typography,{variant:"body1"},t))};t.JobsPerDepartment=e=>{let{jobData:t,titleFilter:r="",useDummyData:n=!1,...i}=e,c=n?o.jobsPerDepartmentDummyData:null==t?void 0:t.departments;return c&&(u=c)&&u.some((e=>e.jobs&&e.jobs.length>0))?(c&&r&&(c=c.map((e=>({...e,jobs:e.jobs.filter((e=>e.title.includes(r)))})))),s.default.createElement("div",{...i},s.default.createElement(l,null),c.map((e=>s.default.createElement(a.SingleDepartmentJobs,{key:e.name,department:e}))))):s.default.createElement(l,{message:"No job openings to show"});var u}},744:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.SingleDepartmentJobs=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(67294)),s=r(6980);r(37477);const o=n.__importDefault(r(88746));t.SingleDepartmentJobs=e=>{let{department:t}=e;return t.jobs&&0!==t.jobs.length?a.default.createElement("div",{className:"mdx-jpd__single-job-department-container"},a.default.createElement(i.Typography,{variant:"subtitle2",className:"mdx-jpd__department-title"},t.name),a.default.createElement("ul",{className:"mdx-jpd__job-list"},t.jobs.map(((e,t)=>{var r;return a.default.createElement("li",{key:t,className:"mdx-jpd__job-list-item"},a.default.createElement(o.default,{href:e.absolute_url,target:"_blank",className:"mdx-jpd__job-link"},a.default.createElement("div",{className:"mdx-jpd__job-title-container"},a.default.createElement(i.Typography,{variant:"h5",className:"mdx-jpd__job-title"},e.title),a.default.createElement(s.IconExternalLink,{className:"mdx-jpd__external-link-icon"})),!(null==(r=e.location)||!r.name)&&a.default.createElement(i.Typography,{variant:"subtitle2",component:"div"},e.location.name)))})))):null}},39610:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(48949),t)},31174:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.jobsPerDepartmentDummyData=void 0,t.jobsPerDepartmentDummyData=[{id:87842,name:"App",parent_id:43806,child_ids:[87847,87852,87850,87848,45530,87849],jobs:[]},{id:54504,name:"Brand Design Studio",parent_id:null,child_ids:[],jobs:[]},{id:45532,name:"Business Development",parent_id:null,child_ids:[],jobs:[]},{id:87841,name:"Codex",parent_id:43806,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5329400",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2662332,location:{name:"Remote (Worldwide)"},metadata:null,id:5329400,updated_at:"2023-10-13T09:40:03-04:00",requisition_id:"Cod-6",title:"Technical Business Development Lead [Codex]"}]},{id:84549,name:"Communications",parent_id:null,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5276254",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2645076,location:{name:"Remote (Worldwide)"},metadata:null,id:5276254,updated_at:"2023-10-19T03:08:59-04:00",requisition_id:"PROV-Com-16",title:"Motion Designer"}]},{id:45531,name:"Design",parent_id:null,child_ids:[],jobs:[]},{id:87847,name:"Desktop",parent_id:87842,child_ids:[],jobs:[]},{id:87852,name:"Documentation",parent_id:87842,child_ids:[],jobs:[]},{id:45547,name:"Engineering ",parent_id:null,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5419957",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2693761,location:{name:"Remote (Worldwide)"},metadata:null,id:5419957,updated_at:"2023-10-24T07:30:00-04:00",requisition_id:"APP-QA-2",title:"Desktop QA Engineer "},{absolute_url:"https://jobs.status.im/?gh_jid=3694379",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2055210,location:{name:"Remote (Worldwide)"},metadata:null,id:3694379,updated_at:"2023-10-24T16:23:12-04:00",requisition_id:"BACK-1050",title:"Senior C++ Qt/QML developer for blockchain app"},{absolute_url:"https://jobs.status.im/?gh_jid=3702173",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2058858,location:{name:"Remote (Worldwide)"},metadata:null,id:3702173,updated_at:"2023-10-24T16:21:54-04:00",requisition_id:"93",title:"Senior Mobile ClojureScript UI Developer "}]},{id:49925,name:"Finance",parent_id:87845,child_ids:[],jobs:[]},{id:87854,name:"Infrastructure",parent_id:43806,child_ids:[],jobs:[]},{id:87853,name:"Insights",parent_id:87845,child_ids:[],jobs:[]},{id:87850,name:"Keycard",parent_id:87842,child_ids:[],jobs:[]},{id:145838,name:"Leadership",parent_id:null,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5447463",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2704948,location:{name:"Remote (Worldwide)"},metadata:null,id:5447463,updated_at:"2023-10-24T09:07:22-04:00",requisition_id:"LEAD-1",title:"Chief of Staff [whole ecosystem]"}]},{id:74156,name:"Legal",parent_id:87845,child_ids:[],jobs:[]},{id:91698,name:"Logos",parent_id:null,child_ids:[],jobs:[]},{id:43807,name:"Marketing",parent_id:null,child_ids:[],jobs:[]},{id:87848,name:"Mobile",parent_id:87842,child_ids:[],jobs:[]},{id:87843,name:"Nimbus",parent_id:43806,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5370820",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2679527,location:{name:"Remote (Worldwide)"},metadata:null,id:5370820,updated_at:"2023-10-19T03:05:07-04:00",requisition_id:"LIDO-1",title:"Senior DevOps Engineer (Blockchain)"}]},{id:144866,name:"Nomos",parent_id:43806,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5433423",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2315608,location:{name:"Remote (Worldwide)"},metadata:null,id:5433423,updated_at:"2023-10-12T11:20:52-04:00",requisition_id:"PROV-Nom-5",title:"Applied Network Researcher"}]},{id:45548,name:"People Operations",parent_id:87845,child_ids:[],jobs:[]},{id:45530,name:"Product Design",parent_id:87842,child_ids:[],jobs:[]},{id:90941,name:"Program Management",parent_id:null,child_ids:[],jobs:[]},{id:43806,name:"Research & Development",parent_id:null,child_ids:[87842,87841,87854,87843,144866,87846,87981,87847,87852,87850,87848,45530,87849],jobs:[]},{id:87851,name:"Security",parent_id:87845,child_ids:[],jobs:[]},{id:87845,name:"Services",parent_id:null,child_ids:[49925,87853,74156,45548,87851],jobs:[]},{id:91697,name:"Status App ",parent_id:null,child_ids:[],jobs:[]},{id:54783,name:"Technical Writing ",parent_id:null,child_ids:[],jobs:[]},{id:87846,name:"Vac",parent_id:43806,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=4460860",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2331302,location:{name:"Remote (Worldwide)"},metadata:null,id:4460860,updated_at:"2023-10-04T05:13:53-04:00",requisition_id:"PROV-zkV-1",title:"Zero Knowledge Research Engineer "}]},{id:87981,name:"Waku",parent_id:43806,child_ids:[],jobs:[{absolute_url:"https://jobs.status.im/?gh_jid=5456032",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2707470,location:{name:"Remote (Worldwide)"},metadata:null,id:5456032,updated_at:"2023-10-23T11:40:19-04:00",requisition_id:"WAK-GL-1",title:"Growth Lead"},{absolute_url:"https://jobs.status.im/?gh_jid=3693623",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2055187,location:{name:"Remote (Worldwide)"},metadata:null,id:3693623,updated_at:"2023-10-04T05:13:53-04:00",requisition_id:"PROV-Sec-2",title:"Protocol Engineer"},{absolute_url:"https://jobs.status.im/?gh_jid=3157908",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:1830496,location:{name:"Remote, Worldwide"},metadata:null,id:3157908,updated_at:"2023-10-04T05:13:53-04:00",requisition_id:"PROV-Sec-3",title:"Protocol Researcher (Distributed Systems)"},{absolute_url:"https://jobs.status.im/?gh_jid=5175038",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2584916,location:{name:"Remote (Worldwide)"},metadata:null,id:5175038,updated_at:"2023-10-09T05:53:53-04:00",requisition_id:"SDK-2",title:"Software Engineer (Chat SDK)"},{absolute_url:"https://jobs.status.im/?gh_jid=5310503",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2656108,location:{name:"Remote (Worldwide)"},metadata:null,id:5310503,updated_at:"2023-10-19T03:12:53-04:00",requisition_id:"PROV-Wak-13",title:"Software Engineer Distributed Systems Testing"},{absolute_url:"https://jobs.status.im/?gh_jid=5423094",data_compliance:[{type:"gdpr",requires_consent:!1,requires_processing_consent:!1,requires_retention_consent:!1,retention_period:null}],internal_job_id:2694724,location:{name:"Remote (Worldwide)"},metadata:null,id:5423094,updated_at:"2023-10-24T12:39:05-04:00",requisition_id:"WAK-BD-1",title:"Technical Business Development Lead "}]},{id:87849,name:"Web",parent_id:87842,child_ids:[],jobs:[]},{id:0,name:"No Department",parent_id:null,child_ids:[],jobs:[]}]},69935:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.LogoCarousel=void 0;const n=r(70655),i=n.__importDefault(r(25812)),a=n.__importDefault(r(86010)),s=n.__importStar(r(67294));r(11620);const o=r(65701),l=r(3796),c=r(67713);t.LogoCarousel=e=>{let{title:t,gridGap:r="80px",className:n,items:u=[],children:h,...d}=e;const p=(0,c.useHydrated)(),f=(0,s.useRef)(null),m=(0,s.useRef)(null);var g;"undefined"!=typeof window&&p&&!m.current&&(m.current=(null==(g=f.current)?void 0:g.querySelector(".mdx-grid__content"))??null);return s.default.createElement("div",{ref:f,className:(0,a.default)(n,"mdx-logo-carousel"),...d},s.default.createElement(o.SectionHeader,{title:t,noBorder:!0}),s.default.createElement("div",{className:"mdx-logo-carousel-buttons"},s.default.createElement(l.ScrollButtons,{containerRef:m,spacing:"spaced"})),s.default.createElement(o.Grid,{className:"mdx-logo-carousel__inner",xs:{wrap:!1,gap:r,scrollButtons:!1}},u.map((e=>s.default.createElement(o.Grid.Item,{className:"mdx-logo-carousel__item"},s.default.createElement(i.default,{className:"mdx-logo-carousel__logo",title:e.title,sources:{dark:e.logoSrcDark??e.logoSrc??"",light:e.logoSrc??e.logoSrcDark??""}}))))))}},80756:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(69935),t)},41080:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.NewsCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294));r(10822);const l=r(22197);t.NewsCard=e=>{let{thumbnail:t,title:r,date:n,author:c,description:u="",tags:h=[],...d}=e;return o.default.createElement("a",{target:"_blank",...d,className:(0,s.default)(d.className,"mdx-news-card")},t&&o.default.createElement("div",{className:"mdx-news-card-thumbnail__container"},o.default.createElement(a.default,{sources:{dark:t,light:t},alt:r??"news thumbnail",className:"mdx-news-card__thumbnail"}),o.default.createElement("div",{className:"mdx-news-card__external-link-icon"},o.default.createElement(l.IconExternalLink,{className:"mdx-jpd__external-link-icon"}))),o.default.createElement(i.Typography,{component:"h3",variant:"h4",className:"mdx-news-card__title"},r),o.default.createElement("div",{className:"mdx-news-card__row"},o.default.createElement("div",{className:"mdx-news-card__info"},n&&o.default.createElement(o.default.Fragment,null,o.default.createElement(i.Typography,{variant:"body2"},n),o.default.createElement(i.Typography,{variant:"body2"},"\xb7")),o.default.createElement(i.Typography,{variant:"body2"},"by ",c))),u&&o.default.createElement(i.Typography,{variant:"body2",className:"mdx-news-card__description"},u),(null==h?void 0:h.length)>0&&o.default.createElement("div",{className:"mdx-news-card__tags"},h.map(((e,t)=>o.default.createElement(i.Tag,{key:t,size:"small",className:"mdx-news-card__tag"},e)))))}},66625:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(41080),t)},76114:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.NewsCardList=void 0;const n=r(70655).__importStar(r(67294));r(32068);const i=r(31665),a=r(65701);t.NewsCardList=e=>{let{data:t,...r}=e;const[s,o]=(0,n.useState)((null==t?void 0:t.length)>8),[l,c]=(0,n.useState)(8);return n.default.createElement("div",{className:"mdx-news-card-list__container",...r},n.default.createElement("div",null,n.default.createElement(a.Box,{top:{xs:64},bottom:64},n.default.createElement(a.Grid,{xs:{cols:1,gap:"64px 16px"},md:{cols:4}},t.slice(0,l).map(((e,t)=>n.default.createElement(a.Grid.Item,{xs:1,key:t},n.default.createElement(a.NewsCard,{...e})))))),s&&n.default.createElement(i.Button,{className:"mdx-news-card-list__button",onClick:()=>{c((e=>e+8)),l+6>=(null==t?void 0:t.length)&&o(!1)},size:"large"},"See more")))}},12249:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(76114),t)},45706:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.NewsletterSubscription=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(65701),l=r(3538),c=r(38840);r(58287);t.NewsletterSubscription=e=>{var t,r;let{title:n,description:u,mailingListId:h,...d}=e;const p=(null==(t=(0,c.useThemeOptions)())||null==(r=t.newsletterSubscription)?void 0:r.mailingListId)??0,f=h??p,m=(0,l.useNewsletterApi)(),g=!m.message||m.error,v=n??"Newsletter",A=u??s.default.createElement(s.default.Fragment,null,"Subscribe",s.default.createElement("br",null),"to our newsletter");return s.default.createElement(o.CallToActionSection,{className:"mdx-ns",title:s.default.createElement("span",{className:"mdx-ns__title"},v),description:s.default.createElement("div",{className:"mdx-ns__inner"},s.default.createElement("span",{className:"mdx-ns__description"},A),s.default.createElement("form",{onSubmit:e=>{e.preventDefault();const t=e.target,r=t.elements.namedItem("name").value,n=t.elements.namedItem("email").value;m.subscribe(f,n,r)}},m.message&&s.default.createElement(i.Toast,{title:m.message,className:"mdx-ns__toast",icon:m.error?i.ErrorIcon:i.CheckIcon}),s.default.createElement("div",{className:(0,a.default)("mdx-ns__inputs",!g&&"hidden")},s.default.createElement(i.TextField,{inputProps:{type:"text",name:"name"},variant:"underlined",placeholder:"First name or pseudonym"}),s.default.createElement(i.TextField,{inputProps:{type:"email",name:"email",required:!0},variant:"underlined",placeholder:"Email address (required)"})),s.default.createElement(i.Button,{size:"large",color:"primary",variant:"outlined",className:(0,a.default)("mdx-ns__submit-button",!g&&"hidden"),disabled:m.busy},"Subscribe"))),columns:2})}},1765:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(45706),t)},91589:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.NumbersView=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(67294));r(18935);const s=n.__importDefault(r(88746));t.NumbersView=e=>{let{sectionTitle:t,description:r="",buttonText:n="",buttonLink:o="",buttonTarget:l="_self",data:c,...u}=e;return a.default.createElement("div",{className:"mdx-numbers-view",...u},a.default.createElement("div",{className:"mdx-numbers-view-header"},a.default.createElement("div",{className:"mdx-numbers-view-header-title"},a.default.createElement(i.Typography,{className:"mdx-numbers-view-section-title",component:"h2",variant:"h2"},t),r&&a.default.createElement(i.Typography,{className:"mdx-numbers-view-description",variant:"body1"},r)),o&&a.default.createElement(s.default,{href:o,target:l},a.default.createElement(i.Button,{size:"large"},n||"Learn More"))),a.default.createElement("div",{className:"mdx-numbers-view-content"},c.map(((e,t)=>a.default.createElement("div",{key:t,className:"mdx-numbers-view-row"},a.default.createElement(i.Typography,{variant:"body2",className:"mdx-numbers-view-number"},t+1),a.default.createElement(i.Typography,{component:"h3",variant:"h4",className:"mdx-numbers-view-title"},e.title))))))}},56547:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(91589),t)},9737:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.PoweredBy=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=r(65701);r(63428);t.PoweredBy=e=>{let{items:t=[],className:r,...n}=e;return a.default.createElement(s.Grid,{className:(0,i.default)(r,"mdx-powered-by"),xs:{cols:1,wrap:!0,gap:"0 1rem"},lg:{cols:2},...n},t.map(((e,t)=>a.default.createElement(s.Grid.Item,{key:t,xs:1},a.default.createElement(s.AppCard,{...e})))))}},73465:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(9737),t)},44564:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.ProfileCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(88746)),s=n.__importDefault(r(67294)),o=r(6980);r(36489);const l=n.__importDefault(r(86010));t.ProfileCard=e=>{var t;let{imgSrc:r,name:n,githubUsername:c,githubLink:u,discordUsername:h,discordLink:d,xUsername:p,xLink:f,...m}=e;return s.default.createElement("div",{...m,className:(0,l.default)("mdx-profile-card",m.className)},s.default.createElement("div",{className:"mdx-profile-card__profile"},void 0===r&&n?s.default.createElement("span",{className:"mdx-profile-card__profile-placeholder"},null==(t=n[0])?void 0:t.toUpperCase()):s.default.createElement("img",{alt:"string"==typeof n?n:"",className:"mdx-profile-card__profile-image",src:r}),s.default.createElement("div",null,s.default.createElement(i.Typography,{className:"mdx-profile-card__name",variant:"h4",component:"h4"},n),u||d||f?s.default.createElement("div",{className:"mdx-profile-card__buttons"},u&&s.default.createElement(a.default,{href:u,target:"_blank",className:"mdx-profile-card__link"},s.default.createElement(o.IconGithub,null)),d&&s.default.createElement(a.default,{href:d,target:"_blank",className:"mdx-profile-card__link"},s.default.createElement(o.IconDiscord,{size:"m"})),f&&s.default.createElement(a.default,{href:f,target:"_blank",className:"mdx-profile-card__link"},s.default.createElement(o.IconX,{size:"m"}))):null)))}},46877:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(44564),t)},47562:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.Roadmap=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=r(65701),o=r(74219);r(74166);t.Roadmap=e=>{let{title:t,description:r,timeline:n=[],noBorder:l=!1,href:c,linkLabel:u,className:h,children:d,...p}=e;const f=(new Date).getFullYear(),m=(e,t,r)=>{if(t===r.length-1)return"none";e.period[0],r[t+1].period[0];return"solid"};return a.default.createElement("div",{className:(0,i.default)(h,"mdx-roadmap"),...p},a.default.createElement(s.CallToActionSection,{className:"mdx-roadmap__header",title:t,description:r,columns:1,ctaPosition:"top",border:!1,align:"left",href:c,label:u},d),n.length>0&&a.default.createElement(s.Grid,{className:"mdx-roadmap__timeline",xs:{cols:6,wrap:!1,gap:"0 1rem",scrollButtons:!0},spacingButtons:!0,leftLabel:"Past",rightLabel:"Future"},n.map(((e,t)=>a.default.createElement(s.Grid.Item,{key:t,xs:1},a.default.createElement(o.TimelineItem,{...e,index:t,period:e.period??f,description:e.description,borderStyle:m(e,t,n),periodStyle:"filled",className:(0,i.default)("mdx-roadmap__timeline-item",l&&"mdx-roadmap__timeline-item--no-border",e.className)}))))))}},77383:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(47562),t)},35481:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.ScrollButtons=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294));r(79832);t.ScrollButtons=e=>{let{leftLabel:t,rightLabel:r,containerRef:n,containerId:o,spacing:l="grouped",...c}=e;const u=e=>{var t,r,i;const a=n?n.current:document.querySelector(`#${o}`);if(!a)return;const s=(null==(t=a.children[0])||null==t.getBoundingClientRect||null==(r=t.getBoundingClientRect())?void 0:r.width)??236;a.scrollTo({behavior:"smooth",left:a.scrollLeft+((null==(i=a.getBoundingClientRect())?void 0:i.width)-s)*e})};return s.default.createElement("div",{...c,className:(0,a.default)(c.className,"mdx-scroll-buttons","spaced"===l&&"mdx-scroll-buttons--spaced")},s.default.createElement(i.IconButtonGroup,{size:"small",color:"primary"},s.default.createElement(i.IconButton,{className:(0,a.default)("mdx-scroll-buttons__button",t&&t.length>0&&"mdx-scroll-buttons__button--with-label"),size:"small",onClick:u.bind(null,-1)},s.default.createElement(i.ChevronLeftIcon,null),t&&t.length>0&&s.default.createElement("span",{className:"mdx-scroll-buttons__label"},t)),s.default.createElement(i.IconButton,{className:(0,a.default)("mdx-scroll-buttons__button",r&&r.length&&"mdx-scroll-buttons__button--with-label"),size:"small",onClick:u.bind(null,1)},r&&r.length>0&&s.default.createElement("span",{className:"mdx-scroll-buttons__label"},r),s.default.createElement(i.ChevronRightIcon,null))))}},3796:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(35481),t)},66872:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.ScrollToBottom=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importStar(r(67294)),o=r(89164),l=r(9259),c=r(52606),u=r(40213),h=n.__importDefault(r(49933));t.ScrollToBottom=e=>{const{children:t,className:r,style:n,...d}=e,p=(0,u.useScrollY)(),f=(0,c.useIsMobile)(),m=(0,o.useWindowSize)(),g=(0,s.useMemo)((()=>{var e,t;return((null==(e=document.querySelector(".mdx-hero"))||null==(t=e.getBoundingClientRect())?void 0:t.bottom)??0)+window.scrollY}),[m.height]);return s.default.createElement(i.IconButton,{onClick:()=>{var e;const t=document.querySelector(".main-wrapper article"),r=null==t||null==(e=t.children)?void 0:e[1];if(r)return window.scrollTo({left:0,top:r.getBoundingClientRect().top-(f?200:0),behavior:"smooth"});window.scrollTo({top:document.body.scrollHeight,behavior:"smooth"})},size:f?"small":"large",className:(0,a.default)(h.default.scrollToBottom,r,p>20&&h.default.hide),style:(0,l.makeStyle)({...n??{}},{vh:m.height/100+"px",maxTop:g+"px"}),...d},s.default.createElement(i.ChevronDownIcon,{color:"primary"}))}},19538:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(66872),t)},18291:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.SectionHeader=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(65701);r(92476);t.SectionHeader=e=>{let{title:t,description:r,className:n,noBorder:l,children:c,...u}=e;const h=!!r;return s.default.createElement(o.Box,{className:(0,a.default)(n,"mdx-section-header",h&&"mdx-section-header--with-description",l&&"mdx-section-header--no-border"),...u},s.default.createElement(i.Typography,{className:"mdx-section-header__title",component:"h2",variant:"h2"},t,!h&&c&&s.default.createElement("div",{className:"mdx-section-header__extra"},c)),r&&s.default.createElement(i.Typography,{className:"mdx-section-header__description",component:"p",variant:"h3"},r,s.default.createElement("div",{className:"mdx-section-header__extra"},c)))}},83729:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(18291),t)},36835:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.Showcase=void 0;const n=r(70655),i=n.__importDefault(r(86010)),a=n.__importDefault(r(67294)),s=r(65701);r(13344);t.Showcase=e=>{let{items:t=[],className:r,...n}=e;return a.default.createElement(s.Grid,{className:(0,i.default)(r,"mdx-showcase"),xs:{cols:2,wrap:!0,gap:"1.5rem 1em"},lg:{cols:3,gap:"1rem"},...n},t.map(((e,t)=>a.default.createElement(s.Grid.Item,{key:t,className:"mdx-showcase__item",xs:1},a.default.createElement(s.ShowcaseCard,{...e})))))}},78090:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(36835),t)},40245:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.ShowcaseCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294));r(29188);t.ShowcaseCard=e=>{let{index:t,name:r,logoSrc:n,logoSrcDark:l,description:c,className:u,size:h="large",borderStyle:d="solid",cta:p,children:f,...m}=e;return o.default.createElement("div",{className:(0,s.default)(u,"mdx-showcase-card",`mdx-showcase-card--${h}`,`mdx-showcase-card--border-${d}`),...m},o.default.createElement("div",{className:"mdx-showcase-card__inner"},t&&o.default.createElement(i.Typography,{className:"mdx-showcase-card__index",variant:"subtitle2",component:"div"},t),(n||l)&&o.default.createElement(a.default,{sources:{dark:l??n??"",light:n??l??""},alt:"string"==typeof r?r:"",className:"mdx-showcase-card__logo"}),r&&o.default.createElement(i.Typography,{variant:"h4",component:"h4",className:"mdx-showcase-card__name"},r),c&&o.default.createElement(i.Typography,{variant:"body1",component:"p",className:"mdx-showcase-card__description"},c),p&&o.default.createElement("a",{href:null==p?void 0:p.href,target:null==p?void 0:p.target,className:"mdx-showcase-card__cta"},null==p?void 0:p.label,o.default.createElement(i.ChevronDownIcon,{color:"primary"}))))}},65162:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(40245),t)},82732:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.SocialCard=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(25812)),s=n.__importDefault(r(86010)),o=n.__importDefault(r(67294)),l=r(6980);r(63933);t.SocialCard=e=>{let{title:t,logoSrc:r,logoSrcDark:n,description:c,...u}=e;return o.default.createElement("a",{target:"_blank",...u,className:(0,s.default)(u.className,"mdx-social-card")},o.default.createElement("div",{className:"mdx-social-card__row"},(r||n)&&o.default.createElement(a.default,{sources:{dark:n??r??"",light:r??n??""},alt:t??"social card logo",className:"mdx-social-card__logo"}),o.default.createElement(l.IconExternalLink,{className:"mdx-social-card__external-link"})),o.default.createElement(i.Typography,{variant:"body1",component:"span",className:"mdx-social-card__description"},c))}},92498:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(82732),t)},19980:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.TableView=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(67294));r(58011);t.TableView=e=>{let{tableTitle:t="",data:r,...n}=e;return a.default.createElement("div",{className:"mdx-table-view",...n},(null==t?void 0:t.length)>0?a.default.createElement(i.Typography,{className:"mdx-table-view-table-title",component:"h2",variant:"h2"},t):null,a.default.createElement("div",null,r.map(((e,t)=>a.default.createElement("div",{key:t,className:"mdx-table-view-row"},a.default.createElement(i.Typography,{component:"h3",variant:"h4",className:"mdx-table-view-title"},e.title),a.default.createElement(i.Typography,{component:"p",variant:"subtitle1",className:"mdx-table-view-description"},e.description))))))}},49098:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(19980),t)},89885:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.TimelineItem=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294));r(86782);t.TimelineItem=e=>{let{index:t,period:r,description:n,borderStyle:o,periodStyle:l,className:c,children:u,...h}=e;return s.default.createElement("div",{className:(0,a.default)(c,"mdx-timeline-item","dashed"===o&&"mdx-timeline-item--border-dashed"),...h},s.default.createElement("div",{className:"mdx-timeline-item__header"},s.default.createElement("div",{className:"mdx-timeline-item__period-container"},"none"!==o&&s.default.createElement("div",{className:(0,a.default)("mdx-timeline-item__border")}),s.default.createElement(i.Typography,{variant:"subtitle2",component:"span",className:(0,a.default)("mdx-timeline-item__period",`mdx-timeline-item__period--${l}`)},(e=>{if(Array.isArray(e)&&e.length>0){let t=e[0].toString();return e.length>1&&("+"===e[1]?t+="+":t+=" "+e[1]),t}return""})(r))),s.default.createElement(i.Typography,{variant:"h2",component:"span",className:"mdx-timeline-item__index"},"number"==typeof t?`${t<9?"0":""}${t+1}`:t)),s.default.createElement(i.Typography,{variant:"h5",component:"p",className:"mdx-timeline-item__description"},n))}},74219:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(89885),t)},73024:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.VideoSection=void 0;const n=r(70655),i=r(31665),a=n.__importDefault(r(86010)),s=n.__importDefault(r(67294)),o=r(65701);r(47573);const l=r(6980),c=n.__importDefault(r(88746));t.VideoSection=e=>{let{label:t,href:r,title:n,columns:u=1,description:h,target:d,border:p=!0,src:f,align:m="unset",className:g,children:v,variant:A="outlined",ctaPosition:y="bottom",...x}=e;const b=!!r,w=!n||!!h||b?n&&!h&&b?"title-button":n&&h&&2===u?"full-width":n&&h?"list":"simple":"title-only",_=r&&"top"===y&&s.default.createElement(o.CallToActionButton,{target:d,href:r,className:"mdx-video-section__link",variant:A},t);return s.default.createElement("div",{className:(0,a.default)(g,"mdx-video-section",`mdx-video-section--${w}`,!p&&"mdx-video-section--no-border"),...x},s.default.createElement("div",{className:(0,a.default)("mdx-video-section__container","unset"!==m&&`mdx-video-section--align-${m}`)},s.default.createElement("div",{className:"mdx-video-section__header"},s.default.createElement(l.IconPlay,null),s.default.createElement(i.Typography,{component:"h2",className:(0,a.default)("mdx-video-section__title")},n)),h&&s.default.createElement(i.Typography,{component:"h3",className:"mdx-video-section__description"},h),s.default.createElement("div",{className:"mdx-video-section__header__mobile-button"},_),f&&"bottom"===y&&s.default.createElement(c.default,{href:f,target:"_blank"},s.default.createElement(o.CallToActionButton,{target:d,href:r,className:"mdx-video-section__link",variant:A,size:"small"},t))),s.default.createElement("div",{className:"mdx-video-section__video-container"},s.default.createElement("iframe",{width:"100%",height:"100%",src:f,title:"Waku Tutorial 001 : Introduction to Waku",frameBorder:"0",allow:"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share",referrerPolicy:"strict-origin-when-cross-origin",allowFullScreen:!0,className:"mdx-video-section__video"})))}},82374:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});r(70655).__exportStar(r(73024),t)},65701:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0});const n=r(70655);n.__exportStar(r(66169),t),n.__exportStar(r(84614),t),n.__exportStar(r(64314),t),n.__exportStar(r(32765),t),n.__exportStar(r(54442),t),n.__exportStar(r(78256),t),n.__exportStar(r(6796),t),n.__exportStar(r(24084),t),n.__exportStar(r(34155),t),n.__exportStar(r(47513),t),n.__exportStar(r(78378),t),n.__exportStar(r(72295),t),n.__exportStar(r(48126),t),n.__exportStar(r(16693),t),n.__exportStar(r(22447),t),n.__exportStar(r(67812),t),n.__exportStar(r(28292),t),n.__exportStar(r(85486),t),n.__exportStar(r(98130),t),n.__exportStar(r(38716),t),n.__exportStar(r(39610),t),n.__exportStar(r(80756),t),n.__exportStar(r(1765),t),n.__exportStar(r(73465),t),n.__exportStar(r(46877),t),n.__exportStar(r(77383),t),n.__exportStar(r(19538),t),n.__exportStar(r(83729),t),n.__exportStar(r(78090),t),n.__exportStar(r(65162),t),n.__exportStar(r(92498),t),n.__exportStar(r(74219),t),n.__exportStar(r(85535),t),n.__exportStar(r(42174),t),n.__exportStar(r(70136),t),n.__exportStar(r(94030),t),n.__exportStar(r(92302),t),n.__exportStar(r(7472),t),n.__exportStar(r(24812),t),n.__exportStar(r(26498),t),n.__exportStar(r(74603),t),n.__exportStar(r(45320),t),n.__exportStar(r(66625),t),n.__exportStar(r(12249),t),n.__exportStar(r(82374),t),n.__exportStar(r(49098),t),n.__exportStar(r(56547),t)},59567:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.lsdUtils=t.LsdUtils=void 0;const n=r(31665),i=r(70917);class a{constructor(){var e=this;this._breakpoints={},this.getBreakpoints=e=>{if(this._breakpoints[e.name])return this._breakpoints[e.name];const t={};for(let r=0;rthis.getBreakpoints(e)[t],this.breakpoints=function(e){return void 0===e&&(e=[]),n.THEME_BREAKPOINTS.filter((t=>!e.find((e=>e===t))))},this.typography=function(e,t){return void 0===t&&(t=!1),"subtitle3"===e?"\n font-size: 12px !important;\n font-weight: 400 !important;\n line-height: 16px !important;\n ":`\n font-size: var(--lsd-${e}-fontSize)${t?"!important":""};\n font-weight: var(--lsd-${e}-fontWeight)${t?"!important":""};\n line-height: var(--lsd-${e}-lineHeight)${t?"!important":""};\n `},this.breakpoint=function(t,r,n,i){void 0===n&&(n="up");const{min:a,max:s}=e.getBreakpoint(t,r);let o="@media ";if("up"===n)o+=`(min-width: ${a}px)`;else if("down"===n)o+=`(max-width: ${s}px)`;else if("between"===n&&i){o+=`(min-width: ${a}px) and (max-width: ${e.getBreakpoint(t,i).min-1}px)`}else o+=`(min-width: ${a}px) and (max-width: ${s}px)`;return`${o}`},this.responsive=function(e,r,n){void 0===n&&(n="up");const a=t.lsdUtils.breakpoint(e,r,n);return e=>i.css` - ${a} { - ${e} - } - `}}}t.LsdUtils=a,t.lsdUtils=new a},9259:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.makeStyle=void 0;t.makeStyle=function(e,t){return void 0===t&&(t={}),{...e,...Object.entries(t).reduce(((e,t)=>{let[r,n]=t;return{...e,[`--${r}`]:n}}),{})}}},24052:(e,t)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.calcHeroInfoMb=t.isMobile=t.generateTextShadow=t.isTouchDevice=t.calcScrollThreshold=t.mapFloat=t.random=void 0;t.random=(e,t)=>Math.random()*(t-e)+e;t.mapFloat=(e,t,r,n,i)=>(e-t)*(i-n)/(r-t)+n;t.calcScrollThreshold=()=>.4*window.innerHeight;t.isTouchDevice=()=>"undefined"!=typeof window&&("ontouchstart"in window||navigator.maxTouchPoints>0||navigator.msMaxTouchPoints&&navigator.msMaxTouchPoints>0);t.generateTextShadow=e=>{let t="";for(let r=0;r"undefined"!=typeof window&&window.innerWidth<997;t.calcHeroInfoMb=e=>(0,t.mapFloat)(scrollY,0,(0,t.calcScrollThreshold)(),0,100)},3538:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.useNewsletterApi=void 0;const n=r(67294);t.useNewsletterApi=()=>{const[e,t]=(0,n.useState)(!1),[r,i]=(0,n.useState)({error:!1,message:""});return{busy:e,error:r.error,message:r.message,subscribe:async(e,r,n)=>{t(!0);try{const t=await fetch("https://odoo.logos.co/website_mass_mailing/subscribe_ghost",{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({jsonrpc:"2.0",method:"call",params:{name:n,value:r,list_id:e,subscription_type:"email"}})}),a=await t.json();i({error:!1,message:a.result.message})}catch(a){i({error:!0,message:"Something went wrong!"})}t(!1)}}}},40213:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.useScrollY=void 0;const n=r(21457),i=r(67294);t.useScrollY=()=>{const[e,t]=(0,i.useState)(0);return(0,n.useWindowEventListener)("scroll",(()=>{t(window.scrollY)})),e}},77366:(e,t,r)=>{"use strict";function n(){return n=Object.assign?Object.assign.bind():function(e){for(var t=1;tv});var i=r(67294),a=r(45042),s=/^((children|dangerouslySetInnerHTML|key|ref|autoFocus|defaultValue|defaultChecked|innerHTML|suppressContentEditableWarning|suppressHydrationWarning|valueLink|abbr|accept|acceptCharset|accessKey|action|allow|allowUserMedia|allowPaymentRequest|allowFullScreen|allowTransparency|alt|async|autoComplete|autoPlay|capture|cellPadding|cellSpacing|challenge|charSet|checked|cite|classID|className|cols|colSpan|content|contentEditable|contextMenu|controls|controlsList|coords|crossOrigin|data|dateTime|decoding|default|defer|dir|disabled|disablePictureInPicture|download|draggable|encType|enterKeyHint|form|formAction|formEncType|formMethod|formNoValidate|formTarget|frameBorder|headers|height|hidden|high|href|hrefLang|htmlFor|httpEquiv|id|inputMode|integrity|is|keyParams|keyType|kind|label|lang|list|loading|loop|low|marginHeight|marginWidth|max|maxLength|media|mediaGroup|method|min|minLength|multiple|muted|name|nonce|noValidate|open|optimum|pattern|placeholder|playsInline|poster|preload|profile|radioGroup|readOnly|referrerPolicy|rel|required|reversed|role|rows|rowSpan|sandbox|scope|scoped|scrolling|seamless|selected|shape|size|sizes|slot|span|spellCheck|src|srcDoc|srcLang|srcSet|start|step|style|summary|tabIndex|target|title|translate|type|useMap|value|width|wmode|wrap|about|datatype|inlist|prefix|property|resource|typeof|vocab|autoCapitalize|autoCorrect|autoSave|color|incremental|fallback|inert|itemProp|itemScope|itemType|itemID|itemRef|on|option|results|security|unselectable|accentHeight|accumulate|additive|alignmentBaseline|allowReorder|alphabetic|amplitude|arabicForm|ascent|attributeName|attributeType|autoReverse|azimuth|baseFrequency|baselineShift|baseProfile|bbox|begin|bias|by|calcMode|capHeight|clip|clipPathUnits|clipPath|clipRule|colorInterpolation|colorInterpolationFilters|colorProfile|colorRendering|contentScriptType|contentStyleType|cursor|cx|cy|d|decelerate|descent|diffuseConstant|direction|display|divisor|dominantBaseline|dur|dx|dy|edgeMode|elevation|enableBackground|end|exponent|externalResourcesRequired|fill|fillOpacity|fillRule|filter|filterRes|filterUnits|floodColor|floodOpacity|focusable|fontFamily|fontSize|fontSizeAdjust|fontStretch|fontStyle|fontVariant|fontWeight|format|from|fr|fx|fy|g1|g2|glyphName|glyphOrientationHorizontal|glyphOrientationVertical|glyphRef|gradientTransform|gradientUnits|hanging|horizAdvX|horizOriginX|ideographic|imageRendering|in|in2|intercept|k|k1|k2|k3|k4|kernelMatrix|kernelUnitLength|kerning|keyPoints|keySplines|keyTimes|lengthAdjust|letterSpacing|lightingColor|limitingConeAngle|local|markerEnd|markerMid|markerStart|markerHeight|markerUnits|markerWidth|mask|maskContentUnits|maskUnits|mathematical|mode|numOctaves|offset|opacity|operator|order|orient|orientation|origin|overflow|overlinePosition|overlineThickness|panose1|paintOrder|pathLength|patternContentUnits|patternTransform|patternUnits|pointerEvents|points|pointsAtX|pointsAtY|pointsAtZ|preserveAlpha|preserveAspectRatio|primitiveUnits|r|radius|refX|refY|renderingIntent|repeatCount|repeatDur|requiredExtensions|requiredFeatures|restart|result|rotate|rx|ry|scale|seed|shapeRendering|slope|spacing|specularConstant|specularExponent|speed|spreadMethod|startOffset|stdDeviation|stemh|stemv|stitchTiles|stopColor|stopOpacity|strikethroughPosition|strikethroughThickness|string|stroke|strokeDasharray|strokeDashoffset|strokeLinecap|strokeLinejoin|strokeMiterlimit|strokeOpacity|strokeWidth|surfaceScale|systemLanguage|tableValues|targetX|targetY|textAnchor|textDecoration|textRendering|textLength|to|transform|u1|u2|underlinePosition|underlineThickness|unicode|unicodeBidi|unicodeRange|unitsPerEm|vAlphabetic|vHanging|vIdeographic|vMathematical|values|vectorEffect|version|vertAdvY|vertOriginX|vertOriginY|viewBox|viewTarget|visibility|widths|wordSpacing|writingMode|x|xHeight|x1|x2|xChannelSelector|xlinkActuate|xlinkArcrole|xlinkHref|xlinkRole|xlinkShow|xlinkTitle|xlinkType|xmlBase|xmlns|xmlnsXlink|xmlLang|xmlSpace|y|y1|y2|yChannelSelector|z|zoomAndPan|for|class|autofocus)|(([Dd][Aa][Tt][Aa]|[Aa][Rr][Ii][Aa]|x)-.*))$/,o=(0,a.Z)((function(e){return s.test(e)||111===e.charCodeAt(0)&&110===e.charCodeAt(1)&&e.charCodeAt(2)<91})),l=r(80123),c=r(70444),u=r(48137),h=r(27278),d=o,p=function(e){return"theme"!==e},f=function(e){return"string"==typeof e&&e.charCodeAt(0)>96?d:p},m=function(e,t,r){var n;if(t){var i=t.shouldForwardProp;n=e.__emotion_forwardProp&&i?function(t){return e.__emotion_forwardProp(t)&&i(t)}:i}return"function"!=typeof n&&r&&(n=e.__emotion_forwardProp),n},g=function(e){var t=e.cache,r=e.serialized,n=e.isStringTag;return(0,c.hC)(t,r,n),(0,h.L)((function(){return(0,c.My)(t,r,n)})),null},v=function e(t,r){var a,s,o=t.__emotion_real===t,h=o&&t.__emotion_base||t;void 0!==r&&(a=r.label,s=r.target);var d=m(t,r,o),p=d||f(h),v=!p("as");return function(){var A=arguments,y=o&&void 0!==t.__emotion_styles?t.__emotion_styles.slice(0):[];if(void 0!==a&&y.push("label:"+a+";"),null==A[0]||void 0===A[0].raw)y.push.apply(y,A);else{0,y.push(A[0][0]);for(var x=A.length,b=1;b{"use strict";r.d(t,{Zo:()=>u,kt:()=>p});var n=r(67294);function i(e,t,r){return t in e?Object.defineProperty(e,t,{value:r,enumerable:!0,configurable:!0,writable:!0}):e[t]=r,e}function a(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),r.push.apply(r,n)}return r}function s(e){for(var t=1;t=0||(i[r]=e[r]);return i}(e,t);if(Object.getOwnPropertySymbols){var a=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,r)&&(i[r]=e[r])}return i}var l=n.createContext({}),c=function(e){var t=n.useContext(l),r=t;return e&&(r="function"==typeof e?e(t):s(s({},t),e)),r},u=function(e){var t=c(e.components);return n.createElement(l.Provider,{value:t},e.children)},h={inlineCode:"code",wrapper:function(e){var t=e.children;return n.createElement(n.Fragment,{},t)}},d=n.forwardRef((function(e,t){var r=e.components,i=e.mdxType,a=e.originalType,l=e.parentName,u=o(e,["components","mdxType","originalType","parentName"]),d=c(r),p=i,f=d["".concat(l,".").concat(p)]||d[p]||h[p]||a;return r?n.createElement(f,s(s({ref:t},u),{},{components:r})):n.createElement(f,s({ref:t},u))}));function p(e,t){var r=arguments,i=t&&t.mdxType;if("string"==typeof e||i){var a=r.length,s=new Array(a);s[0]=d;var o={};for(var l in t)hasOwnProperty.call(t,l)&&(o[l]=t[l]);o.originalType=e,o.mdxType="string"==typeof e?e:i,s[1]=o;for(var c=2;c{"use strict";function n(){return n=Object.assign?Object.assign.bind():function(e){for(var t=1;tmh,AdaptiveEvents:()=>gh,ArcballControls:()=>vs,Backdrop:()=>cu,BakeShadows:()=>ch,Billboard:()=>yi,Bounds:()=>jc,Box:()=>bc,CameraShake:()=>Yc,Center:()=>Vc,Circle:()=>wc,Clone:()=>ns,Cloud:()=>wu,ComputedAttribute:()=>$a,Cone:()=>_c,ContactShadows:()=>au,CubeCamera:()=>ss,CubicBezierLine:()=>Mi,CurveModifier:()=>oc,CycleRaycast:()=>A,Cylinder:()=>Ec,Detailed:()=>oh,DeviceOrientationControls:()=>ls,Dodecahedron:()=>Dc,Edges:()=>Va,Effects:()=>Da,Environment:()=>$c,EnvironmentCube:()=>ru,EnvironmentMap:()=>eu,EnvironmentPortal:()=>tu,Extrude:()=>Uc,FirstPersonControls:()=>Ss,Float:()=>qc,FlyControls:()=>us,GizmoHelper:()=>Us,GizmoViewcube:()=>Ks,GizmoViewport:()=>$s,GradientTexture:()=>Ua,Html:()=>v,Icosahedron:()=>Fc,Image:()=>za,Instance:()=>Zu,Instances:()=>$u,IsObject:()=>ka,Lathe:()=>Nc,Lightformer:()=>mu,Line:()=>_i,Loader:()=>E,MapControls:()=>ds,Merged:()=>eh,MeshDistortMaterial:()=>cc,MeshReflectorMaterial:()=>mc,MeshWobbleMaterial:()=>hc,Octahedron:()=>Lc,OrbitControls:()=>ps,OrthographicCamera:()=>is,PerspectiveCamera:()=>as,Plane:()=>Mc,Point:()=>Uu,PointMaterial:()=>vc,PointMaterialImpl:()=>gc,PointerLockControls:()=>_s,Points:()=>ku,PointsBuffer:()=>Nu,Polyhedron:()=>Pc,PositionalAudio:()=>Ti,Preload:()=>lh,PresentationControls:()=>pi,QuadraticBezierLine:()=>Si,Reflector:()=>hu,Ring:()=>Rc,RoundedBox:()=>Oc,Sampler:()=>Za,ScreenQuad:()=>zc,Scroll:()=>P,ScrollControls:()=>I,Segment:()=>sh,Segments:()=>ih,Select:()=>vi,Shadow:()=>uu,Sky:()=>Au,Sphere:()=>Sc,SpotLight:()=>fu,Stage:()=>ou,Stars:()=>bu,Stats:()=>go,Tetrahedron:()=>Bc,Text:()=>Ia,Torus:()=>Cc,TorusKnot:()=>Ic,TrackballControls:()=>ms,Trail:()=>Ka,TransformControls:()=>bs,Tube:()=>Tc,calcPosFromAngles:()=>vu,isWebGL2Available:()=>La,meshBounds:()=>fh,shaderMaterial:()=>Na,softShadows:()=>yc,useAnimations:()=>nc,useAspect:()=>yo,useBVH:()=>tc,useBounds:()=>Wc,useBoxProjectedEnv:()=>ac,useCamera:()=>Ms,useContextBridge:()=>rc,useCubeTexture:()=>eo,useCursor:()=>y,useDepthBuffer:()=>Ao,useDetectGPU:()=>Ro,useFBO:()=>vo,useFBX:()=>ro,useGLTF:()=>lo,useGizmoContext:()=>Cs,useHelper:()=>Po,useIntersect:()=>ic,useKTX2:()=>ho,useMatcapTexture:()=>_u,useNormalTexture:()=>Eu,useProgress:()=>w,useScroll:()=>C,useSelect:()=>Ai,useTexture:()=>Oa,useTrail:()=>qa});var i=r(67294),a=r(73935),s=r(99477),o=r(64232);const l=new s.Vector3,c=new s.Vector3,u=new s.Vector3;function h(e,t,r){const n=l.setFromMatrixPosition(e.matrixWorld);n.project(t);const i=r.width/2,a=r.height/2;return[n.x*i+i,-n.y*a+a]}const d=e=>Math.abs(e)<1e-10?0:e;function p(e,t,r=""){let n="matrix3d(";for(let i=0;16!==i;i++)n+=d(t[i]*e.elements[i])+(15!==i?",":")");return r+n}const f=(m=[1,-1,1,1,1,-1,1,1,1,-1,1,1,1,-1,1,1],e=>p(e,m));var m;const g=(e,t)=>p(e,(e=>[1/e,1/e,1/e,1,-1/e,-1/e,-1/e,-1,1/e,1/e,1/e,1,1,1,1,1])(t),"translate(-50%,-50%)"),v=i.forwardRef((({children:e,eps:t=.001,style:r,className:p,prepend:m,center:v,fullscreen:A,portal:y,distanceFactor:x,sprite:b=!1,transform:w=!1,occlude:_,onOcclude:E,zIndexRange:S=[16777271,0],calculatePosition:M=h,as:T="div",wrapperClass:C,pointerEvents:I="auto",...B},R)=>{var P;const F=(0,o.useThree)((({gl:e})=>e)),L=(0,o.useThree)((({camera:e})=>e)),D=(0,o.useThree)((({scene:e})=>e)),U=(0,o.useThree)((({size:e})=>e)),N=(0,o.useThree)((({raycaster:e})=>e)),[k]=i.useState((()=>document.createElement(T))),O=i.useRef(null),G=i.useRef(0),z=i.useRef([0,0]),V=i.useRef(null),H=i.useRef(null),Q=null!==(P=null==y?void 0:y.current)&&void 0!==P?P:F.domElement.parentNode;i.useEffect((()=>{if(O.current){if(D.updateMatrixWorld(),w)k.style.cssText="position:absolute;top:0;left:0;pointer-events:none;overflow:hidden;";else{const e=M(O.current,L,U);k.style.cssText=`position:absolute;top:0;left:0;transform:translate3d(${e[0]}px,${e[1]}px,0);transform-origin:0 0;`}return Q&&(m?Q.prepend(k):Q.appendChild(k)),()=>{Q&&Q.removeChild(k),a.unmountComponentAtNode(k)}}}),[Q,w]),i.useLayoutEffect((()=>{C&&(k.className=C)}),[C]);const j=i.useMemo((()=>w?{position:"absolute",top:0,left:0,width:U.width,height:U.height,transformStyle:"preserve-3d",pointerEvents:"none"}:{position:"absolute",transform:v?"translate3d(-50%,-50%,0)":"none",...A&&{top:-U.height/2,left:-U.width/2,width:U.width,height:U.height},...r}),[r,v,A,U,w]),W=i.useMemo((()=>({position:"absolute",pointerEvents:I})),[I]);i.useLayoutEffect((()=>{w?a.render(i.createElement("div",{ref:V,style:j},i.createElement("div",{ref:H,style:W},i.createElement("div",{ref:R,className:p,style:r,children:e}))),k):a.render(i.createElement("div",{ref:R,style:j,className:p,children:e}),k)}));const X=i.useRef(!0);return(0,o.useFrame)((()=>{if(O.current){L.updateMatrixWorld(),O.current.updateWorldMatrix(!0,!1);const e=w?z.current:M(O.current,L,U);if(w||Math.abs(G.current-L.zoom)>t||Math.abs(z.current[0]-e[0])>t||Math.abs(z.current[1]-e[1])>t){const t=function(e,t){const r=l.setFromMatrixPosition(e.matrixWorld),n=c.setFromMatrixPosition(t.matrixWorld),i=r.sub(n),a=t.getWorldDirection(u);return i.angleTo(a)>Math.PI/2}(O.current,L);let r=!1;"boolean"==typeof _?!0===_&&(r=[D]):Array.isArray(_)&&(r=_.map((e=>e.current)));const n=X.current;if(r){const e=function(e,t,r,n){const i=l.setFromMatrixPosition(e.matrixWorld),a=i.clone();a.project(t),r.setFromCamera(a,t);const s=r.intersectObjects(n,!0);if(s.length){const e=s[0].distance;return i.distanceTo(r.ray.origin)e.raycaster)),c=(0,o.useThree)((e=>e.get)),u=(0,o.useThree)((e=>e.gl));return i.useEffect((()=>{var i;let o,h=[];const d=l.filter,p=null!==(i=null==t?void 0:t.current)&&void 0!==i?i:u.domElement.parentNode,f=()=>p&&e&&e(h,Math.round(s.current)%h.length);l.filter=(e,t)=>{let r=[...e];r.length===h.length&&h.every((e=>r.map((e=>e.object.uuid)).includes(e.object.uuid)))||(s.current=0,h=r,f()),d&&(r=d(r,t));for(let n=0;n{var t,r;s.current=e(s.current),null==(t=c().events.handlers)||t.onPointerCancel(void 0),null==(r=c().events.handlers)||r.onPointerMove(o),f()},g=e=>{(e.keyCode||e.which===a)&&(r&&e.preventDefault(),h.length>1&&m((e=>e+1)))},v=e=>{r&&e.preventDefault();let t=0;e||(e=window.event),e.wheelDelta?t=e.wheelDelta/120:e.detail&&(t=-e.detail/3),h.length>1&&m((e=>Math.abs(e-t)))},A=e=>o=e;return document.addEventListener("pointermove",A,{passive:!0}),n&&document.addEventListener("wheel",v),void 0!==a&&document.addEventListener("keydown",g),()=>{l.filter=d,void 0!==a&&document.removeEventListener("keydown",g),n&&document.removeEventListener("wheel",v),document.removeEventListener("pointermove",A)}}),[u,c,l,r,n,a]),null}function y(e,t="pointer",r="auto"){i.useEffect((()=>{if(e)return document.body.style.cursor=t,()=>{document.body.style.cursor=r}}),[e])}var x=r(14671);let b=0;const w=(0,x.Z)((e=>(s.DefaultLoadingManager.onStart=(t,r,n)=>{e({active:!0,item:t,loaded:r,total:n,progress:(r-b)/(n-b)*100})},s.DefaultLoadingManager.onLoad=()=>{e({active:!1})},s.DefaultLoadingManager.onError=t=>e((e=>({errors:[...e.errors,t]}))),s.DefaultLoadingManager.onProgress=(t,r,n)=>{r===n&&(b=n),e({active:!0,item:t,loaded:r,total:n,progress:(r-b)/(n-b)*100||100})},{errors:[],active:!1,progress:0,item:"",loaded:0,total:0}))),_=e=>`Loading ${e.toFixed(2)}%`;function E({containerStyles:e,innerStyles:t,barStyles:r,dataStyles:n,dataInterpolation:a=_,initialState:s=(e=>e)}){const{active:o,progress:l}=w(),c=i.useRef(0),u=i.useRef(0),h=i.useRef(null),[d,p]=i.useState(s(o));i.useEffect((()=>{let e;return o!==d&&(e=setTimeout((()=>p(o)),300)),()=>clearTimeout(e)}),[d,o]);const f=i.useCallback((()=>{h.current&&(c.current+=(l-c.current)/2,(c.current>.95*l||100===l)&&(c.current=l),h.current.innerText=a(c.current),c.current(f(),()=>cancelAnimationFrame(u.current))),[f]),d?i.createElement("div",{style:{...S.container,opacity:o?1:0,...e}},i.createElement("div",null,i.createElement("div",{style:{...S.inner,...t}},i.createElement("div",{style:{...S.bar,transform:`scaleX(${l/100})`,...r}}),i.createElement("span",{ref:h,style:{...S.data,...n}})))):null}const S={container:{position:"absolute",top:0,left:0,width:"100%",height:"100%",background:"#171717",display:"flex",alignItems:"center",justifyContent:"center",transition:"opacity 300ms ease",zIndex:1e3},inner:{width:100,height:3,background:"#272727",textAlign:"center"},bar:{height:3,width:"100%",background:"white",transition:"transform 200ms",transformOrigin:"left center"},data:{display:"inline-block",position:"relative",fontVariantNumeric:"tabular-nums",marginTop:"0.8em",color:"#f0f0f0",fontSize:"0.6em",fontFamily:'-apple-system, BlinkMacSystemFont, "Inter", "Segoe UI", "Helvetica Neue", Helvetica, Arial, Roboto, Ubuntu, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"',whiteSpace:"nowrap"}};const M=function(e){return function(t){e.forEach((function(e){"function"==typeof e?e(t):null!=e&&(e.current=t)}))}},T=i.createContext(null);function C(){return i.useContext(T)}function I({eps:e=1e-5,enabled:t=!0,infinite:r,horizontal:n,pages:a=1,distance:l=1,damping:c=4,style:u={},children:h}){const{gl:d,size:p,invalidate:f,events:m,raycaster:g}=(0,o.useThree)(),[v]=i.useState((()=>document.createElement("div"))),[A]=i.useState((()=>document.createElement("div"))),[y]=i.useState((()=>document.createElement("div"))),x=d.domElement.parentNode,b=i.useRef(0),w=i.useMemo((()=>{const t={el:v,eps:e,fill:A,fixed:y,horizontal:n,damping:c,offset:0,delta:0,scroll:b,pages:a,range(e,t,r=0){const n=e-r,i=n+t+2*r;return this.offseti?1:(this.offset-n)/(i-n)},curve(e,t,r=0){return Math.sin(this.range(e,t,r)*Math.PI)},visible(e,t,r=0){const n=e-r,i=n+t+2*r;return this.offset>=n&&this.offset<=i}};return t}),[e,c,n,a]);i.useEffect((()=>{v.style.position="absolute",v.style.width="100%",v.style.height="100%",v.style[n?"overflowX":"overflowY"]="auto",v.style[n?"overflowY":"overflowX"]="hidden",v.style.top="0px",v.style.left="0px";for(const r in u)v.style[r]=u[r];y.style.position="sticky",y.style.top="0px",y.style.left="0px",y.style.width="100%",y.style.height="100%",y.style.overflow="hidden",v.appendChild(y),A.style.height=n?"100%":a*l*100+"%",A.style.width=n?a*l*100+"%":"100%",A.style.pointerEvents="none",v.appendChild(A),x.appendChild(v),v[n?"scrollLeft":"scrollTop"]=1;const e=m.connected||d.domElement;requestAnimationFrame((()=>null==m.connect?void 0:m.connect(v)));const t=g.computeOffsets;return g.computeOffsets=({clientX:e,clientY:t})=>({offsetX:e-x.offsetLeft,offsetY:t-x.offsetTop}),()=>{x.removeChild(v),g.computeOffsets=t,null==m.connect||m.connect(e)}}),[a,l,n,v,A,y,x]),i.useEffect((()=>{const e=p[n?"width":"height"],i=v[n?"scrollWidth":"scrollHeight"],a=i-e;let s=0,o=!0,l=!0;const c=()=>{if(t&&!l&&(f(),s=v[n?"scrollLeft":"scrollTop"],b.current=s/a,r)){if(!o)if(s>=a){const e=1-w.offset;v[n?"scrollLeft":"scrollTop"]=1,b.current=w.offset=-e,o=!0}else if(s<=0){const e=1+w.offset;v[n?"scrollLeft":"scrollTop"]=i,b.current=w.offset=e,o=!0}o&&setTimeout((()=>o=!1),40)}};v.addEventListener("scroll",c,{passive:!0}),requestAnimationFrame((()=>l=!1));const u=e=>v.scrollLeft+=e.deltaY/2;return n&&v.addEventListener("wheel",u,{passive:!0}),()=>{v.removeEventListener("scroll",c),n&&v.removeEventListener("wheel",u)}}),[v,p,r,w,f,n,t]);let _=0;return(0,o.useFrame)(((t,r)=>{w.offset=s.MathUtils.damp(_=w.offset,b.current,c,r),w.delta=s.MathUtils.damp(w.delta,Math.abs(_-w.offset),c,r),w.delta>e&&f()})),i.createElement(T.Provider,{value:w},h)}const B=i.forwardRef((({children:e},t)=>{const r=i.useRef(null),n=C(),{width:a,height:s}=(0,o.useThree)((e=>e.viewport));return(0,o.useFrame)((()=>{r.current.position.x=n.horizontal?-a*(n.pages-1)*n.offset:0,r.current.position.y=n.horizontal?0:s*(n.pages-1)*n.offset})),i.createElement("group",{ref:M([t,r])},e)})),R=i.forwardRef((({children:e,style:t,...r},s)=>{const l=C(),c=i.useRef(null),{width:u,height:h}=(0,o.useThree)((e=>e.size)),d=i.useContext(o.context);return(0,o.useFrame)((()=>{l.delta>l.eps&&(c.current.style.transform=`translate3d(${l.horizontal?-u*(l.pages-1)*l.offset:0}px,${l.horizontal?0:h*(l.pages-1)*-l.offset}px,0)`)})),a.render(i.createElement("div",n({ref:M([s,c]),style:{...t,position:"absolute",top:0,left:0,willChange:"transform"}},r),i.createElement(T.Provider,{value:l},i.createElement(o.context.Provider,{value:d},e))),l.fixed),null})),P=i.forwardRef((({html:e,...t},r)=>{const a=e?R:B;return i.createElement(a,n({ref:r},t))}));var F=q(),L=e=>j(e,F),D=q();L.write=e=>j(e,D);var U=q();L.onStart=e=>j(e,U);var N=q();L.onFrame=e=>j(e,N);var k=q();L.onFinish=e=>j(e,k);var O=[];L.setTimeout=(e,t)=>{let r=L.now()+t,n=()=>{let e=O.findIndex((e=>e.cancel==n));~e&&O.splice(e,1),H-=~e?1:0},i={time:r,handler:e,cancel:n};return O.splice(G(r),0,i),H+=1,W(),i};var G=e=>~(~O.findIndex((t=>t.time>e))||~O.length);L.cancel=e=>{U.delete(e),N.delete(e),k.delete(e),F.delete(e),D.delete(e)},L.sync=e=>{Q=!0,L.batchedUpdates(e),Q=!1},L.throttle=e=>{let t;function r(){try{e(...t)}finally{t=null}}function n(...e){t=e,L.onStart(r)}return n.handler=e,n.cancel=()=>{U.delete(r),t=null},n};var z=typeof window<"u"?window.requestAnimationFrame:()=>{};L.use=e=>z=e,L.now=typeof performance<"u"?()=>performance.now():Date.now,L.batchedUpdates=e=>e(),L.catch=console.error,L.frameLoop="always",L.advance=()=>{"demand"!==L.frameLoop?console.warn("Cannot call the manual advancement of rafz whilst frameLoop is not set as demand"):Y()};var V=-1,H=0,Q=!1;function j(e,t){Q?(t.delete(e),e(0)):(t.add(e),W())}function W(){V<0&&(V=0,"demand"!==L.frameLoop&&z(X))}function X(){~V&&(z(X),L.batchedUpdates(Y))}function Y(){let e=V;V=L.now();let t=G(V);t&&(K(O.splice(0,t),(e=>e.handler())),H-=t),H?(U.flush(),F.flush(e?Math.min(64,V-e):16.667),N.flush(),D.flush(),k.flush()):V=-1}function q(){let e=new Set,t=e;return{add(r){H+=t!=e||e.has(r)?0:1,e.add(r)},delete:r=>(H-=t==e&&e.has(r)?1:0,e.delete(r)),flush(r){t.size&&(e=new Set,H-=t.size,K(t,(t=>t(r)&&e.add(t))),H+=e.size,t=e)}}}function K(e,t){e.forEach((e=>{try{t(e)}catch(Jr){L.catch(Jr)}}))}var J=Object.defineProperty,Z={};function $(){}((e,t)=>{for(var r in t)J(e,r,{get:t[r],enumerable:!0})})(Z,{assign:()=>pe,colors:()=>ue,createStringInterpolator:()=>se,skipAnimation:()=>he,to:()=>oe,willAdvance:()=>de});var ee={arr:Array.isArray,obj:e=>!!e&&"Object"===e.constructor.name,fun:e=>"function"==typeof e,str:e=>"string"==typeof e,num:e=>"number"==typeof e,und:e=>void 0===e};function te(e,t){if(ee.arr(e)){if(!ee.arr(t)||e.length!==t.length)return!1;for(let r=0;re.forEach(t);function ne(e,t,r){if(ee.arr(e))for(let n=0;nee.und(e)?[]:ee.arr(e)?e:[e];function ae(e,t){if(e.size){let r=Array.from(e);e.clear(),re(r,t)}}var se,oe,le=(e,...t)=>ae(e,(e=>e(...t))),ce=()=>typeof window>"u"||!window.navigator||/ServerSideRendering|^Deno\//.test(window.navigator.userAgent),ue=null,he=!1,de=$,pe=e=>{e.to&&(oe=e.to),e.now&&(L.now=e.now),void 0!==e.colors&&(ue=e.colors),null!=e.skipAnimation&&(he=e.skipAnimation),e.createStringInterpolator&&(se=e.createStringInterpolator),e.requestAnimationFrame&&L.use(e.requestAnimationFrame),e.batchedUpdates&&(L.batchedUpdates=e.batchedUpdates),e.willAdvance&&(de=e.willAdvance),e.frameLoop&&(L.frameLoop=e.frameLoop)},fe=new Set,me=[],ge=[],ve=0,Ae={get idle(){return!fe.size&&!me.length},start(e){ve>e.priority?(fe.add(e),L.onStart(ye)):(xe(e),L(we))},advance:we,sort(e){if(ve)L.onFrame((()=>Ae.sort(e)));else{let t=me.indexOf(e);~t&&(me.splice(t,1),be(e))}},clear(){me=[],fe.clear()}};function ye(){fe.forEach(xe),fe.clear(),L(we)}function xe(e){me.includes(e)||be(e)}function be(e){me.splice(function(e,t){let r=e.findIndex(t);return r<0?e.length:r}(me,(t=>t.priority>e.priority)),0,e)}function we(e){let t=ge;for(let r=0;r0}var _e="[-+]?\\d*\\.?\\d+",Ee=_e+"%";function Se(...e){return"\\(\\s*("+e.join(")\\s*,\\s*(")+")\\s*\\)"}var Me=new RegExp("rgb"+Se(_e,_e,_e)),Te=new RegExp("rgba"+Se(_e,_e,_e,_e)),Ce=new RegExp("hsl"+Se(_e,Ee,Ee)),Ie=new RegExp("hsla"+Se(_e,Ee,Ee,_e)),Be=/^#([0-9a-fA-F]{1})([0-9a-fA-F]{1})([0-9a-fA-F]{1})$/,Re=/^#([0-9a-fA-F]{1})([0-9a-fA-F]{1})([0-9a-fA-F]{1})([0-9a-fA-F]{1})$/,Pe=/^#([0-9a-fA-F]{6})$/,Fe=/^#([0-9a-fA-F]{8})$/;function Le(e,t,r){return r<0&&(r+=1),r>1&&(r-=1),r<1/6?e+6*(t-e)*r:r<.5?t:r<2/3?e+(t-e)*(2/3-r)*6:e}function De(e,t,r){let n=r<.5?r*(1+t):r+t-r*t,i=2*r-n,a=Le(i,n,e+1/3),s=Le(i,n,e),o=Le(i,n,e-1/3);return Math.round(255*a)<<24|Math.round(255*s)<<16|Math.round(255*o)<<8}function Ue(e){let t=parseInt(e,10);return t<0?0:t>255?255:t}function Ne(e){return(parseFloat(e)%360+360)%360/360}function ke(e){let t=parseFloat(e);return t<0?0:t>1?255:Math.round(255*t)}function Oe(e){let t=parseFloat(e);return t<0?0:t>100?1:t/100}function Ge(e){let t=function(e){let t;return"number"==typeof e?e>>>0===e&&e>=0&&e<=4294967295?e:null:(t=Pe.exec(e))?parseInt(t[1]+"ff",16)>>>0:ue&&void 0!==ue[e]?ue[e]:(t=Me.exec(e))?(Ue(t[1])<<24|Ue(t[2])<<16|Ue(t[3])<<8|255)>>>0:(t=Te.exec(e))?(Ue(t[1])<<24|Ue(t[2])<<16|Ue(t[3])<<8|ke(t[4]))>>>0:(t=Be.exec(e))?parseInt(t[1]+t[1]+t[2]+t[2]+t[3]+t[3]+"ff",16)>>>0:(t=Fe.exec(e))?parseInt(t[1],16)>>>0:(t=Re.exec(e))?parseInt(t[1]+t[1]+t[2]+t[2]+t[3]+t[3]+t[4]+t[4],16)>>>0:(t=Ce.exec(e))?(255|De(Ne(t[1]),Oe(t[2]),Oe(t[3])))>>>0:(t=Ie.exec(e))?(De(Ne(t[1]),Oe(t[2]),Oe(t[3]))|ke(t[4]))>>>0:null}(e);return null===t?e:(t=t||0,`rgba(${(4278190080&t)>>>24}, ${(16711680&t)>>>16}, ${(65280&t)>>>8}, ${(255&t)/255})`)}var ze=(e,t,r)=>{if(ee.fun(e))return e;if(ee.arr(e))return ze({range:e,output:t,extrapolate:r});if(ee.str(e.output[0]))return se(e);let n=e,i=n.output,a=n.range||[0,1],s=n.extrapolateLeft||n.extrapolate||"extend",o=n.extrapolateRight||n.extrapolate||"extend",l=n.easing||(e=>e);return e=>{let t=function(e,t){for(var r=1;r=e);++r);return r-1}(e,a);return function(e,t,r,n,i,a,s,o,l){let c=l?l(e):e;if(cr){if("identity"===o)return c;"clamp"===o&&(c=r)}return n===i?n:t===r?e<=t?n:i:(t===-1/0?c=-c:r===1/0?c-=t:c=(c-t)/(r-t),c=a(c),n===-1/0?c=-c:i===1/0?c+=n:c=c*(i-n)+n,c)}(e,a[t],a[t+1],i[t],i[t+1],l,s,o,n.map)}};var Ve=1.70158,He=1.525*Ve,Qe=Ve+1,je=2*Math.PI/3,We=2*Math.PI/4.5,Xe=e=>e<1/2.75?7.5625*e*e:e<2/2.75?7.5625*(e-=1.5/2.75)*e+.75:e<2.5/2.75?7.5625*(e-=2.25/2.75)*e+.9375:7.5625*(e-=2.625/2.75)*e+.984375,Ye={linear:e=>e,easeInQuad:e=>e*e,easeOutQuad:e=>1-(1-e)*(1-e),easeInOutQuad:e=>e<.5?2*e*e:1-Math.pow(-2*e+2,2)/2,easeInCubic:e=>e*e*e,easeOutCubic:e=>1-Math.pow(1-e,3),easeInOutCubic:e=>e<.5?4*e*e*e:1-Math.pow(-2*e+2,3)/2,easeInQuart:e=>e*e*e*e,easeOutQuart:e=>1-Math.pow(1-e,4),easeInOutQuart:e=>e<.5?8*e*e*e*e:1-Math.pow(-2*e+2,4)/2,easeInQuint:e=>e*e*e*e*e,easeOutQuint:e=>1-Math.pow(1-e,5),easeInOutQuint:e=>e<.5?16*e*e*e*e*e:1-Math.pow(-2*e+2,5)/2,easeInSine:e=>1-Math.cos(e*Math.PI/2),easeOutSine:e=>Math.sin(e*Math.PI/2),easeInOutSine:e=>-(Math.cos(Math.PI*e)-1)/2,easeInExpo:e=>0===e?0:Math.pow(2,10*e-10),easeOutExpo:e=>1===e?1:1-Math.pow(2,-10*e),easeInOutExpo:e=>0===e?0:1===e?1:e<.5?Math.pow(2,20*e-10)/2:(2-Math.pow(2,-20*e+10))/2,easeInCirc:e=>1-Math.sqrt(1-Math.pow(e,2)),easeOutCirc:e=>Math.sqrt(1-Math.pow(e-1,2)),easeInOutCirc:e=>e<.5?(1-Math.sqrt(1-Math.pow(2*e,2)))/2:(Math.sqrt(1-Math.pow(-2*e+2,2))+1)/2,easeInBack:e=>Qe*e*e*e-Ve*e*e,easeOutBack:e=>1+Qe*Math.pow(e-1,3)+Ve*Math.pow(e-1,2),easeInOutBack:e=>e<.5?Math.pow(2*e,2)*(2*(He+1)*e-He)/2:(Math.pow(2*e-2,2)*((He+1)*(2*e-2)+He)+2)/2,easeInElastic:e=>0===e?0:1===e?1:-Math.pow(2,10*e-10)*Math.sin((10*e-10.75)*je),easeOutElastic:e=>0===e?0:1===e?1:Math.pow(2,-10*e)*Math.sin((10*e-.75)*je)+1,easeInOutElastic:e=>0===e?0:1===e?1:e<.5?-Math.pow(2,20*e-10)*Math.sin((20*e-11.125)*We)/2:Math.pow(2,-20*e+10)*Math.sin((20*e-11.125)*We)/2+1,easeInBounce:e=>1-Xe(1-e),easeOutBounce:Xe,easeInOutBounce:e=>e<.5?(1-Xe(1-2*e))/2:(1+Xe(2*e-1))/2,steps:(e,t="end")=>r=>{let n=(r="end"===t?Math.min(r,.999):Math.max(r,.001))*e;return((e,t,r)=>Math.min(Math.max(r,e),t))(0,1,("end"===t?Math.floor(n):Math.ceil(n))/e)}},qe=Symbol.for("FluidValue.get"),Ke=Symbol.for("FluidValue.observers"),Je=e=>!(!e||!e[qe]),Ze=e=>e&&e[qe]?e[qe]():e,$e=e=>e[Ke]||null;function et(e,t){let r=e[Ke];r&&r.forEach((e=>{!function(e,t){e.eventObserved?e.eventObserved(t):e(t)}(e,t)}))}var tt=class{[qe];[Ke];constructor(e){if(!e&&!(e=this.get))throw Error("Unknown getter");rt(this,e)}},rt=(e,t)=>st(e,qe,t);function nt(e,t){if(e[qe]){let r=e[Ke];r||st(e,Ke,r=new Set),r.has(t)||(r.add(t),e.observerAdded&&e.observerAdded(r.size,t))}return t}function it(e,t){let r=e[Ke];if(r&&r.has(t)){let n=r.size-1;n?r.delete(t):e[Ke]=null,e.observerRemoved&&e.observerRemoved(n,t)}}var at,st=(e,t,r)=>Object.defineProperty(e,t,{value:r,writable:!0,configurable:!0}),ot=/[+\-]?(?:0|[1-9]\d*)(?:\.\d*)?(?:[eE][+\-]?\d+)?/g,lt=/(#(?:[0-9a-f]{2}){2,4}|(#[0-9a-f]{3})|(rgb|hsl)a?\((-?\d+%?[,\s]+){2,3}\s*[\d\.]+%?\))/gi,ct=new RegExp(`(${ot.source})(%|[a-z]+)`,"i"),ut=/rgba\(([0-9\.-]+), ([0-9\.-]+), ([0-9\.-]+), ([0-9\.-]+)\)/gi,ht=/var\((--[a-zA-Z0-9-_]+),? ?([a-zA-Z0-9 ()%#.,-]+)?\)/,dt=e=>{let[t,r]=pt(e);if(!t||ce())return e;let n=window.getComputedStyle(document.documentElement).getPropertyValue(t);if(n)return n.trim();if(r&&r.startsWith("--")){return window.getComputedStyle(document.documentElement).getPropertyValue(r)||e}return r&&ht.test(r)?dt(r):r||e},pt=e=>{let t=ht.exec(e);if(!t)return[,];let[,r,n]=t;return[r,n]},ft=(e,t,r,n,i)=>`rgba(${Math.round(t)}, ${Math.round(r)}, ${Math.round(n)}, ${i})`,mt=e=>{at||(at=ue?new RegExp(`(${Object.keys(ue).join("|")})(?!\\w)`,"g"):/^\b$/);let t=e.output.map((e=>Ze(e).replace(ht,dt).replace(lt,Ge).replace(at,Ge))),r=t.map((e=>e.match(ot).map(Number))),n=r[0].map(((e,t)=>r.map((e=>{if(!(t in e))throw Error('The arity of each "output" value must be equal');return e[t]})))).map((t=>ze({...e,output:t})));return e=>{let r=!ct.test(t[0])&&t.find((e=>ct.test(e)))?.replace(ot,""),i=0;return t[0].replace(ot,(()=>`${n[i++](e)}${r||""}`)).replace(ut,ft)}},gt="react-spring: ",vt=e=>{let t=e,r=!1;if("function"!=typeof t)throw new TypeError(`${gt}once requires a function parameter`);return(...e)=>{r||(t(...e),r=!0)}},At=vt(console.warn);var yt=vt(console.warn);function xt(e){return ee.str(e)&&("#"==e[0]||/\d/.test(e)||!ce()&&ht.test(e)||e in(ue||{}))}new WeakMap;new Set,new WeakMap,new WeakMap,new WeakMap;var bt=ce()?i.useEffect:i.useLayoutEffect;function wt(){let e=(0,i.useState)()[1],t=(()=>{let e=(0,i.useRef)(!1);return bt((()=>(e.current=!0,()=>{e.current=!1})),[]),e})();return()=>{t.current&&e(Math.random())}}var _t=e=>(0,i.useEffect)(e,Et),Et=[];function St(e){let t=(0,i.useRef)();return(0,i.useEffect)((()=>{t.current=e})),t.current}var Mt=Symbol.for("Animated:node"),Tt=e=>e&&e[Mt],Ct=(e,t)=>((e,t,r)=>Object.defineProperty(e,t,{value:r,writable:!0,configurable:!0}))(e,Mt,t),It=e=>e&&e[Mt]&&e[Mt].getPayload(),Bt=class{payload;constructor(){Ct(this,this)}getPayload(){return this.payload||[]}},Rt=class extends Bt{constructor(e){super(),this._value=e,ee.num(this._value)&&(this.lastPosition=this._value)}done=!0;elapsedTime;lastPosition;lastVelocity;v0;durationProgress=0;static create(e){return new Rt(e)}getPayload(){return[this]}getValue(){return this._value}setValue(e,t){return ee.num(e)&&(this.lastPosition=e,t&&(e=Math.round(e/t)*t,this.done&&(this.lastPosition=e))),this._value!==e&&(this._value=e,!0)}reset(){let{done:e}=this;this.done=!1,ee.num(this._value)&&(this.elapsedTime=0,this.durationProgress=0,this.lastPosition=this._value,e&&(this.lastVelocity=null),this.v0=null)}},Pt=class extends Rt{_string=null;_toString;constructor(e){super(0),this._toString=ze({output:[e,e]})}static create(e){return new Pt(e)}getValue(){return this._string??(this._string=this._toString(this._value))}setValue(e){if(ee.str(e)){if(e==this._string)return!1;this._string=e,this._value=1}else{if(!super.setValue(e))return!1;this._string=null}return!0}reset(e){e&&(this._toString=ze({output:[this.getValue(),e]})),this._value=0,super.reset()}},Ft={dependencies:null},Lt=class extends Bt{constructor(e){super(),this.source=e,this.setValue(e)}getValue(e){let t={};return ne(this.source,((r,n)=>{(e=>!!e&&e[Mt]===e)(r)?t[n]=r.getValue(e):Je(r)?t[n]=Ze(r):e||(t[n]=r)})),t}setValue(e){this.source=e,this.payload=this._makePayload(e)}reset(){this.payload&&re(this.payload,(e=>e.reset()))}_makePayload(e){if(e){let t=new Set;return ne(e,this._addToPayload,t),Array.from(t)}}_addToPayload(e){Ft.dependencies&&Je(e)&&Ft.dependencies.add(e);let t=It(e);t&&re(t,(e=>this.add(e)))}},Dt=class extends Lt{constructor(e){super(e)}static create(e){return new Dt(e)}getValue(){return this.source.map((e=>e.getValue()))}setValue(e){let t=this.getPayload();return e.length==t.length?t.map(((t,r)=>t.setValue(e[r]))).some(Boolean):(super.setValue(e.map(Ut)),!0)}};function Ut(e){return(xt(e)?Pt:Rt).create(e)}function Nt(e){let t=Tt(e);return t?t.constructor:ee.arr(e)?Dt:xt(e)?Pt:Rt}var kt=(e,t)=>{let r=!ee.fun(e)||e.prototype&&e.prototype.isReactComponent;return(0,i.forwardRef)(((n,a)=>{let s=(0,i.useRef)(null),o=r&&(0,i.useCallback)((e=>{s.current=function(e,t){return e&&(ee.fun(e)?e(t):e.current=t),t}(a,e)}),[a]),[l,c]=function(e,t){let r=new Set;return Ft.dependencies=r,e.style&&(e={...e,style:t.createAnimatedStyle(e.style)}),e=new Lt(e),Ft.dependencies=null,[e,r]}(n,t),u=wt(),h=()=>{let e=s.current;r&&!e||!1===(!!e&&t.applyAnimatedValues(e,l.getValue(!0)))&&u()},d=new Ot(h,c),p=(0,i.useRef)();bt((()=>(p.current=d,re(c,(e=>nt(e,d))),()=>{p.current&&(re(p.current.deps,(e=>it(e,p.current))),L.cancel(p.current.update))}))),(0,i.useEffect)(h,[]),_t((()=>()=>{let e=p.current;re(e.deps,(t=>it(t,e)))}));let f=t.getComponentProps(l.getValue());return i.createElement(e,{...f,ref:o})}))},Ot=class{constructor(e,t){this.update=e,this.deps=t}eventObserved(e){"change"==e.type&&L.write(this.update)}};var Gt=Symbol.for("AnimatedComponent"),zt=e=>ee.str(e)?e:e&&ee.str(e.displayName)?e.displayName:ee.fun(e)&&e.name||null;function Vt(e,...t){return ee.fun(e)?e(...t):e}var Ht=(e,t)=>!0===e||!!(t&&e&&(ee.fun(e)?e(t):ie(e).includes(t))),Qt=(e,t)=>ee.obj(e)?t&&e[t]:e,jt=(e,t)=>!0===e.default?e[t]:e.default?e.default[t]:void 0,Wt=e=>e,Xt=(e,t=Wt)=>{let r=Yt;e.default&&!0!==e.default&&(e=e.default,r=Object.keys(e));let n={};for(let i of r){let r=t(e[i],i);ee.und(r)||(n[i]=r)}return n},Yt=["config","onProps","onStart","onChange","onPause","onResume","onRest"],qt={config:1,from:1,to:1,ref:1,loop:1,reset:1,pause:1,cancel:1,reverse:1,immediate:1,default:1,delay:1,onProps:1,onStart:1,onChange:1,onPause:1,onResume:1,onRest:1,onResolve:1,items:1,trail:1,sort:1,expires:1,initial:1,enter:1,update:1,leave:1,children:1,onDestroyed:1,keys:1,callId:1,parentId:1};function Kt(e){let t=function(e){let t={},r=0;if(ne(e,((e,n)=>{qt[n]||(t[n]=e,r++)})),r)return t}(e);if(t){let r={to:t};return ne(e,((e,n)=>n in t||(r[n]=e))),r}return{...e}}function Jt(e){return e=Ze(e),ee.arr(e)?e.map(Jt):xt(e)?Z.createStringInterpolator({range:[0,1],output:[e,e]})(1):e}function Zt(e){for(let t in e)return!0;return!1}function $t(e){return ee.fun(e)||ee.arr(e)&&ee.obj(e[0])}function er(e,t){e.ref?.delete(e),t?.delete(e)}function tr(e,t){t&&e.ref!==t&&(e.ref?.delete(e),t.add(e),e.ref=t)}var rr={tension:170,friction:26,mass:1,damping:1,easing:Ye.linear,clamp:!1};function nr(e,t){if(ee.und(t.decay)){let r=!ee.und(t.tension)||!ee.und(t.friction);(r||!ee.und(t.frequency)||!ee.und(t.damping)||!ee.und(t.mass))&&(e.duration=void 0,e.decay=void 0),r&&(e.frequency=void 0)}else e.duration=void 0}var ir=[];function ar(e,{key:t,props:r,defaultProps:n,state:i,actions:a}){return new Promise(((s,o)=>{let l,c,u=Ht(r.cancel??n?.cancel,t);if(u)p();else{ee.und(r.pause)||(i.paused=Ht(r.pause,t));let e=n?.pause;!0!==e&&(e=i.paused||Ht(e,t)),l=Vt(r.delay||0,t),e?(i.resumeQueue.add(d),a.pause()):(a.resume(),d())}function h(){i.resumeQueue.add(d),i.timeouts.delete(c),c.cancel(),l=c.time-L.now()}function d(){l>0&&!Z.skipAnimation?(i.delayed=!0,c=L.setTimeout(p,l),i.pauseQueue.add(h),i.timeouts.add(c)):p()}function p(){i.delayed&&(i.delayed=!1),i.pauseQueue.delete(h),i.timeouts.delete(c),e<=(i.cancelId||0)&&(u=!0);try{a.start({...r,callId:e,cancel:u},s)}catch(j){o(j)}}}))}var sr=(e,t)=>1==t.length?t[0]:t.some((e=>e.cancelled))?cr(e.get()):t.every((e=>e.noop))?or(e.get()):lr(e.get(),t.every((e=>e.finished))),or=e=>({value:e,noop:!0,finished:!0,cancelled:!1}),lr=(e,t,r=!1)=>({value:e,finished:t,cancelled:r}),cr=e=>({value:e,cancelled:!0,finished:!1});function ur(e,t,r,n){let{callId:i,parentId:a,onRest:s}=t,{asyncTo:o,promise:l}=r;return a||e!==o||t.reset?r.promise=(async()=>{r.asyncId=i,r.asyncTo=e;let c,u,h,d=Xt(t,((e,t)=>"onRest"===t?void 0:e)),p=new Promise(((e,t)=>(c=e,u=t))),f=e=>{let t=i<=(r.cancelId||0)&&cr(n)||i!==r.asyncId&&lr(n,!1);if(t)throw e.result=t,u(e),e},m=(e,t)=>{let a=new dr,s=new pr;return(async()=>{if(Z.skipAnimation)throw hr(r),s.result=lr(n,!1),u(s),s;f(a);let o=ee.obj(e)?{...e}:{...t,to:e};o.parentId=i,ne(d,((e,t)=>{ee.und(o[t])&&(o[t]=e)}));let l=await n.start(o);return f(a),r.paused&&await new Promise((e=>{r.resumeQueue.add(e)})),l})()};if(Z.skipAnimation)return hr(r),lr(n,!1);try{let t;t=ee.arr(e)?(async e=>{for(let t of e)await m(t)})(e):Promise.resolve(e(m,n.stop.bind(n))),await Promise.all([t.then(c),p]),h=lr(n.get(),!0,!1)}catch(V){if(V instanceof dr)h=V.result;else{if(!(V instanceof pr))throw V;h=V.result}}finally{i==r.asyncId&&(r.asyncId=a,r.asyncTo=a?o:void 0,r.promise=a?l:void 0)}return ee.fun(s)&&L.batchedUpdates((()=>{s(h,n,n.item)})),h})():l}function hr(e,t){ae(e.timeouts,(e=>e.cancel())),e.pauseQueue.clear(),e.resumeQueue.clear(),e.asyncId=e.asyncTo=e.promise=void 0,t&&(e.cancelId=t)}var dr=class extends Error{result;constructor(){super("An async animation has been interrupted. You see this error because you forgot to use `await` or `.catch(...)` on its returned promise.")}},pr=class extends Error{result;constructor(){super("SkipAnimationSignal")}},fr=e=>e instanceof gr,mr=1,gr=class extends tt{id=mr++;_priority=0;get priority(){return this._priority}set priority(e){this._priority!=e&&(this._priority=e,this._onPriorityChange(e))}get(){let e=Tt(this);return e&&e.getValue()}to(...e){return Z.to(this,e)}interpolate(...e){return At(`${gt}The "interpolate" function is deprecated in v9 (use "to" instead)`),Z.to(this,e)}toJSON(){return this.get()}observerAdded(e){1==e&&this._attach()}observerRemoved(e){0==e&&this._detach()}_attach(){}_detach(){}_onChange(e,t=!1){et(this,{type:"change",parent:this,value:e,idle:t})}_onPriorityChange(e){this.idle||Ae.sort(this),et(this,{type:"priority",parent:this,priority:e})}},vr=Symbol.for("SpringPhase"),Ar=e=>(1&e[vr])>0,yr=e=>(2&e[vr])>0,xr=e=>(4&e[vr])>0,br=(e,t)=>t?e[vr]|=3:e[vr]&=-3,wr=(e,t)=>t?e[vr]|=4:e[vr]&=-5,_r=class extends gr{key;animation=new class{changed=!1;values=ir;toValues=null;fromValues=ir;to;from;config=new class{tension;friction;frequency;damping;mass;velocity=0;restVelocity;precision;progress;duration;easing;clamp;bounce;decay;round;constructor(){Object.assign(this,rr)}};immediate=!1};queue;defaultProps={};_state={paused:!1,delayed:!1,pauseQueue:new Set,resumeQueue:new Set,timeouts:new Set};_pendingCalls=new Set;_lastCallId=0;_lastToId=0;_memoizedDuration=0;constructor(e,t){if(super(),!ee.und(e)||!ee.und(t)){let r=ee.obj(e)?{...e}:{...t,from:e};ee.und(r.default)&&(r.default=!0),this.start(r)}}get idle(){return!(yr(this)||this._state.asyncTo)||xr(this)}get goal(){return Ze(this.animation.to)}get velocity(){let e=Tt(this);return e instanceof Rt?e.lastVelocity||0:e.getPayload().map((e=>e.lastVelocity||0))}get hasAnimated(){return Ar(this)}get isAnimating(){return yr(this)}get isPaused(){return xr(this)}get isDelayed(){return this._state.delayed}advance(e){let t=!0,r=!1,n=this.animation,{toValues:i}=n,{config:a}=n,s=It(n.to);!s&&Je(n.to)&&(i=ie(Ze(n.to))),n.values.forEach(((o,l)=>{if(o.done)return;let c=o.constructor==Pt?1:s?s[l].lastPosition:i[l],u=n.immediate,h=c;if(!u){if(h=o.lastPosition,a.tension<=0)return void(o.done=!0);let t,r=o.elapsedTime+=e,i=n.fromValues[l],s=null!=o.v0?o.v0:o.v0=ee.arr(a.velocity)?a.velocity[l]:a.velocity,d=a.precision||(i==c?.005:Math.min(1,.001*Math.abs(c-i)));if(ee.und(a.duration))if(a.decay){let e=!0===a.decay?.998:a.decay,n=Math.exp(-(1-e)*r);h=i+s/(1-e)*(1-n),u=Math.abs(o.lastPosition-h)<=d,t=s*n}else{t=null==o.lastVelocity?s:o.lastVelocity;let r,n=a.restVelocity||d/10,l=a.clamp?0:a.bounce,p=!ee.und(l),f=i==c?o.v0>0:in,r||(u=Math.abs(c-h)<=d,!u));++e){p&&(m=h==c||h>c==f,m&&(t=-t*l,h=c)),t+=(1e-6*-a.tension*(h-c)+.001*-a.friction*t)/a.mass*g,h+=t*g}}else{let n=1;a.duration>0&&(this._memoizedDuration!==a.duration&&(this._memoizedDuration=a.duration,o.durationProgress>0&&(o.elapsedTime=a.duration*o.durationProgress,r=o.elapsedTime+=e)),n=(a.progress||0)+r/this._memoizedDuration,n=n>1?1:n<0?0:n,o.durationProgress=n),h=i+a.easing(n)*(c-i),t=(h-o.lastPosition)/e,u=1==n}o.lastVelocity=t,Number.isNaN(h)&&(console.warn("Got NaN while animating:",this),u=!0)}s&&!s[l].done&&(u=!1),u?o.done=!0:t=!1,o.setValue(h,a.round)&&(r=!0)}));let o=Tt(this),l=o.getValue();if(t){let e=Ze(n.to);l===e&&!r||a.decay?r&&a.decay&&this._onChange(l):(o.setValue(e),this._onChange(e)),this._stop()}else r&&this._onChange(l)}set(e){return L.batchedUpdates((()=>{this._stop(),this._focus(e),this._set(e)})),this}pause(){this._update({pause:!0})}resume(){this._update({pause:!1})}finish(){if(yr(this)){let{to:e,config:t}=this.animation;L.batchedUpdates((()=>{this._onStart(),t.decay||this._set(e,!1),this._stop()}))}return this}update(e){return(this.queue||(this.queue=[])).push(e),this}start(e,t){let r;return ee.und(e)?(r=this.queue||[],this.queue=[]):r=[ee.obj(e)?e:{...t,to:e}],Promise.all(r.map((e=>this._update(e)))).then((e=>sr(this,e)))}stop(e){let{to:t}=this.animation;return this._focus(this.get()),hr(this._state,e&&this._lastCallId),L.batchedUpdates((()=>this._stop(t,e))),this}reset(){this._update({reset:!0})}eventObserved(e){"change"==e.type?this._start():"priority"==e.type&&(this.priority=e.priority+1)}_prepareNode(e){let t=this.key||"",{to:r,from:n}=e;r=ee.obj(r)?r[t]:r,(null==r||$t(r))&&(r=void 0),n=ee.obj(n)?n[t]:n,null==n&&(n=void 0);let i={to:r,from:n};return Ar(this)||(e.reverse&&([r,n]=[n,r]),n=Ze(n),ee.und(n)?Tt(this)||this._set(r):this._set(n)),i}_update({...e},t){let{key:r,defaultProps:n}=this;e.default&&Object.assign(n,Xt(e,((e,t)=>/^on/.test(t)?Qt(e,r):e))),Br(this,e,"onProps"),Rr(this,"onProps",e,this);let i=this._prepareNode(e);if(Object.isFrozen(this))throw Error("Cannot animate a `SpringValue` object that is frozen. Did you forget to pass your component to `animated(...)` before animating its props?");let a=this._state;return ar(++this._lastCallId,{key:r,props:e,defaultProps:n,state:a,actions:{pause:()=>{xr(this)||(wr(this,!0),le(a.pauseQueue),Rr(this,"onPause",lr(this,Er(this,this.animation.to)),this))},resume:()=>{xr(this)&&(wr(this,!1),yr(this)&&this._resume(),le(a.resumeQueue),Rr(this,"onResume",lr(this,Er(this,this.animation.to)),this))},start:this._merge.bind(this,i)}}).then((r=>{if(e.loop&&r.finished&&(!t||!r.noop)){let t=Sr(e);if(t)return this._update(t,!0)}return r}))}_merge(e,t,r){if(t.cancel)return this.stop(!0),r(cr(this));let n=!ee.und(e.to),i=!ee.und(e.from);if(n||i){if(!(t.callId>this._lastToId))return r(cr(this));this._lastToId=t.callId}let{key:a,defaultProps:s,animation:o}=this,{to:l,from:c}=o,{to:u=l,from:h=c}=e;i&&!n&&(!t.default||ee.und(u))&&(u=h),t.reverse&&([u,h]=[h,u]);let d=!te(h,c);d&&(o.from=h),h=Ze(h);let p=!te(u,l);p&&this._focus(u);let f=$t(t.to),{config:m}=o,{decay:g,velocity:v}=m;(n||i)&&(m.velocity=0),t.config&&!f&&function(e,t,r){r&&(nr(r={...r},t),t={...r,...t}),nr(e,t),Object.assign(e,t);for(let s in rr)null==e[s]&&(e[s]=rr[s]);let{frequency:n,damping:i}=e,{mass:a}=e;ee.und(n)||(n<.01&&(n=.01),i<0&&(i=0),e.tension=Math.pow(2*Math.PI/n,2)*a,e.friction=4*Math.PI*i*a/n)}(m,Vt(t.config,a),t.config!==s.config?Vt(s.config,a):void 0);let A=Tt(this);if(!A||ee.und(u))return r(lr(this,!0));let y=ee.und(t.reset)?i&&!t.default:!ee.und(h)&&Ht(t.reset,a),x=y?h:this.get(),b=Jt(u),w=ee.num(b)||ee.arr(b)||xt(b),_=!f&&(!w||Ht(s.immediate||t.immediate,a));if(p){let e=Nt(u);if(e!==A.constructor){if(!_)throw Error(`Cannot animate between ${A.constructor.name} and ${e.name}, as the "to" prop suggests`);A=this._set(b)}}let E=A.constructor,S=Je(u),M=!1;if(!S){let e=y||!Ar(this)&&d;(p||e)&&(M=te(Jt(x),b),S=!M),(!te(o.immediate,_)&&!_||!te(m.decay,g)||!te(m.velocity,v))&&(S=!0)}if(M&&yr(this)&&(o.changed&&!y?S=!0:S||this._stop(l)),!f&&((S||Je(l))&&(o.values=A.getPayload(),o.toValues=Je(u)?null:E==Pt?[1]:ie(b)),o.immediate!=_&&(o.immediate=_,!_&&!y&&this._set(l)),S)){let{onRest:e}=o;re(Ir,(e=>Br(this,t,e)));let n=lr(this,Er(this,l));le(this._pendingCalls,n),this._pendingCalls.add(r),o.changed&&L.batchedUpdates((()=>{o.changed=!y,e?.(n,this),y?Vt(s.onRest,n):o.onStart?.(n,this)}))}y&&this._set(x),f?r(ur(t.to,t,this._state,this)):S?this._start():yr(this)&&!p?this._pendingCalls.add(r):r(or(x))}_focus(e){let t=this.animation;e!==t.to&&($e(this)&&this._detach(),t.to=e,$e(this)&&this._attach())}_attach(){let e=0,{to:t}=this.animation;Je(t)&&(nt(t,this),fr(t)&&(e=t.priority+1)),this.priority=e}_detach(){let{to:e}=this.animation;Je(e)&&it(e,this)}_set(e,t=!0){let r=Ze(e);if(!ee.und(r)){let e=Tt(this);if(!e||!te(r,e.getValue())){let n=Nt(r);e&&e.constructor==n?e.setValue(r):Ct(this,n.create(r)),e&&L.batchedUpdates((()=>{this._onChange(r,t)}))}}return Tt(this)}_onStart(){let e=this.animation;e.changed||(e.changed=!0,Rr(this,"onStart",lr(this,Er(this,e.to)),this))}_onChange(e,t){t||(this._onStart(),Vt(this.animation.onChange,e,this)),Vt(this.defaultProps.onChange,e,this),super._onChange(e,t)}_start(){let e=this.animation;Tt(this).reset(Ze(e.to)),e.immediate||(e.fromValues=e.values.map((e=>e.lastPosition))),yr(this)||(br(this,!0),xr(this)||this._resume())}_resume(){Z.skipAnimation?this.finish():Ae.start(this)}_stop(e,t){if(yr(this)){br(this,!1);let r=this.animation;re(r.values,(e=>{e.done=!0})),r.toValues&&(r.onChange=r.onPause=r.onResume=void 0),et(this,{type:"idle",parent:this});let n=t?cr(this.get()):lr(this.get(),Er(this,e??r.to));le(this._pendingCalls,n),r.changed&&(r.changed=!1,Rr(this,"onRest",n,this))}}};function Er(e,t){let r=Jt(t);return te(Jt(e.get()),r)}function Sr(e,t=e.loop,r=e.to){let n=Vt(t);if(n){let i=!0!==n&&Kt(n),a=(i||e).reverse,s=!i||i.reset;return Mr({...e,loop:t,default:!1,pause:void 0,to:!a||$t(r)?r:void 0,from:s?e.from:void 0,reset:s,...i})}}function Mr(e){let{to:t,from:r}=e=Kt(e),n=new Set;return ee.obj(t)&&Cr(t,n),ee.obj(r)&&Cr(r,n),e.keys=n.size?Array.from(n):null,e}function Tr(e){let t=Mr(e);return ee.und(t.default)&&(t.default=Xt(t)),t}function Cr(e,t){ne(e,((e,r)=>null!=e&&t.add(r)))}var Ir=["onStart","onRest","onChange","onPause","onResume"];function Br(e,t,r){e.animation[r]=t[r]!==jt(t,r)?Qt(t[r],e.key):void 0}function Rr(e,t,...r){e.animation[t]?.(...r),e.defaultProps[t]?.(...r)}var Pr=["onStart","onChange","onRest"],Fr=1,Lr=class{id=Fr++;springs={};queue=[];ref;_flush;_initialProps;_lastAsyncId=0;_active=new Set;_changed=new Set;_started=!1;_item;_state={paused:!1,pauseQueue:new Set,resumeQueue:new Set,timeouts:new Set};_events={onStart:new Map,onChange:new Map,onRest:new Map};constructor(e,t){this._onFrame=this._onFrame.bind(this),t&&(this._flush=t),e&&this.start({default:!0,...e})}get idle(){return!this._state.asyncTo&&Object.values(this.springs).every((e=>e.idle&&!e.isDelayed&&!e.isPaused))}get item(){return this._item}set item(e){this._item=e}get(){let e={};return this.each(((t,r)=>e[r]=t.get())),e}set(e){for(let t in e){let r=e[t];ee.und(r)||this.springs[t].set(r)}}update(e){return e&&this.queue.push(Mr(e)),this}start(e){let{queue:t}=this;return e?t=ie(e).map(Mr):this.queue=[],this._flush?this._flush(this,t):(zr(this,t),Dr(this,t))}stop(e,t){if(e!==!!e&&(t=e),t){let r=this.springs;re(ie(t),(t=>r[t].stop(!!e)))}else hr(this._state,this._lastAsyncId),this.each((t=>t.stop(!!e)));return this}pause(e){if(ee.und(e))this.start({pause:!0});else{let t=this.springs;re(ie(e),(e=>t[e].pause()))}return this}resume(e){if(ee.und(e))this.start({pause:!1});else{let t=this.springs;re(ie(e),(e=>t[e].resume()))}return this}each(e){ne(this.springs,e)}_onFrame(){let{onStart:e,onChange:t,onRest:r}=this._events,n=this._active.size>0,i=this._changed.size>0;(n&&!this._started||i&&!this._started)&&(this._started=!0,ae(e,(([e,t])=>{t.value=this.get(),e(t,this,this._item)})));let a=!n&&this._started,s=i||a&&r.size?this.get():null;i&&t.size&&ae(t,(([e,t])=>{t.value=s,e(t,this,this._item)})),a&&(this._started=!1,ae(r,(([e,t])=>{t.value=s,e(t,this,this._item)})))}eventObserved(e){if("change"==e.type)this._changed.add(e.parent),e.idle||this._active.add(e.parent);else{if("idle"!=e.type)return;this._active.delete(e.parent)}L.onFrame(this._onFrame)}};function Dr(e,t){return Promise.all(t.map((t=>Ur(e,t)))).then((t=>sr(e,t)))}async function Ur(e,t,r){let{keys:n,to:i,from:a,loop:s,onRest:o,onResolve:l}=t,c=ee.obj(t.default)&&t.default;s&&(t.loop=!1),!1===i&&(t.to=null),!1===a&&(t.from=null);let u=ee.arr(i)||ee.fun(i)?i:void 0;u?(t.to=void 0,t.onRest=void 0,c&&(c.onRest=void 0)):re(Pr,(r=>{let n=t[r];if(ee.fun(n)){let i=e._events[r];t[r]=({finished:e,cancelled:t})=>{let r=i.get(n);r?(e||(r.finished=!1),t&&(r.cancelled=!0)):i.set(n,{value:null,finished:e||!1,cancelled:t||!1})},c&&(c[r]=t[r])}}));let h=e._state;t.pause===!h.paused?(h.paused=t.pause,le(t.pause?h.pauseQueue:h.resumeQueue)):h.paused&&(t.pause=!0);let d=(n||Object.keys(e.springs)).map((r=>e.springs[r].start(t))),p=!0===t.cancel||!0===jt(t,"cancel");(u||p&&h.asyncId)&&d.push(ar(++e._lastAsyncId,{props:t,state:h,actions:{pause:$,resume:$,start(t,r){p?(hr(h,e._lastAsyncId),r(cr(e))):(t.onRest=o,r(ur(u,t,h,e)))}}})),h.paused&&await new Promise((e=>{h.resumeQueue.add(e)}));let f=sr(e,await Promise.all(d));if(s&&f.finished&&(!r||!f.noop)){let r=Sr(t,s,i);if(r)return zr(e,[r]),Ur(e,r,!0)}return l&&L.batchedUpdates((()=>l(f,e,e.item))),f}function Nr(e,t){let r={...e.springs};return t&&re(ie(t),(e=>{ee.und(e.keys)&&(e=Mr(e)),ee.obj(e.to)||(e={...e,to:void 0}),Gr(r,e,(e=>Or(e)))})),kr(e,r),r}function kr(e,t){ne(t,((t,r)=>{e.springs[r]||(e.springs[r]=t,nt(t,e))}))}function Or(e,t){let r=new _r;return r.key=e,t&&nt(r,t),r}function Gr(e,t,r){t.keys&&re(t.keys,(n=>{(e[n]||(e[n]=r(n)))._prepareNode(t)}))}function zr(e,t){re(t,(t=>{Gr(e.springs,t,(t=>Or(t,e)))}))}var Vr=({children:e,...t})=>{let r=(0,i.useContext)(Hr),n=t.pause||!!r.pause,a=t.immediate||!!r.immediate;t=function(e,t){let[r]=(0,i.useState)((()=>({inputs:t,result:e()}))),n=(0,i.useRef)(),a=n.current,s=a;return s?t&&s.inputs&&function(e,t){if(e.length!==t.length)return!1;for(let r=0;r{n.current=s,a==r&&(r.inputs=r.result=void 0)}),[s]),s.result}((()=>({pause:n,immediate:a})),[n,a]);let{Provider:s}=Hr;return i.createElement(s,{value:t},e)},Hr=function(e,t){return Object.assign(e,i.createContext(t)),e.Provider._context=e,e.Consumer._context=e,e}(Vr,{});Vr.Provider=Hr.Provider,Vr.Consumer=Hr.Consumer;var Qr=()=>{let e=[],t=function(t){yt(`${gt}Directly calling start instead of using the api object is deprecated in v9 (use ".start" instead), this will be removed in later 0.X.0 versions`);let n=[];return re(e,((e,i)=>{if(ee.und(t))n.push(e.start());else{let a=r(t,e,i);a&&n.push(e.start(a))}})),n};t.current=e,t.add=function(t){e.includes(t)||e.push(t)},t.delete=function(t){let r=e.indexOf(t);~r&&e.splice(r,1)},t.pause=function(){return re(e,(e=>e.pause(...arguments))),this},t.resume=function(){return re(e,(e=>e.resume(...arguments))),this},t.set=function(t){re(e,((e,r)=>{let n=ee.fun(t)?t(r,e):t;n&&e.set(n)}))},t.start=function(t){let r=[];return re(e,((e,n)=>{if(ee.und(t))r.push(e.start());else{let i=this._getProps(t,e,n);i&&r.push(e.start(i))}})),r},t.stop=function(){return re(e,(e=>e.stop(...arguments))),this},t.update=function(t){return re(e,((e,r)=>e.update(this._getProps(t,e,r)))),this};let r=function(e,t,r){return ee.fun(e)?e(r,t):e};return t._getProps=r,t};function jr(e,t,r){let n=ee.fun(t)&&t;n&&!r&&(r=[]);let a=(0,i.useMemo)((()=>n||3==arguments.length?Qr():void 0),[]),s=(0,i.useRef)(0),o=wt(),l=(0,i.useMemo)((()=>({ctrls:[],queue:[],flush(e,t){let r=Nr(e,t);return s.current>0&&!l.queue.length&&!Object.keys(r).some((t=>!e.springs[t]))?Dr(e,t):new Promise((n=>{kr(e,r),l.queue.push((()=>{n(Dr(e,t))})),o()}))}})),[]),c=(0,i.useRef)([...l.ctrls]),u=[],h=St(e)||0;function d(e,r){for(let i=e;i{re(c.current.slice(e,h),(e=>{er(e,a),e.stop(!0)})),c.current.length=e,d(h,e)}),[e]),(0,i.useMemo)((()=>{d(0,Math.min(h,e))}),r);let p=c.current.map(((e,t)=>Nr(e,u[t]))),f=(0,i.useContext)(Vr),m=St(f),g=f!==m&&Zt(f);bt((()=>{s.current++,l.ctrls=c.current;let{queue:e}=l;e.length&&(l.queue=[],re(e,(e=>e()))),re(c.current,((e,t)=>{a?.add(e),g&&e.start({default:f});let r=u[t];r&&(tr(e,r.ref),e.ref?e.queue.push(r):e.start(r))}))})),_t((()=>()=>{re(l.ctrls,(e=>e.stop(!0)))}));let v=p.map((e=>({...e})));return a?[v,a]:v}function Wr(e,t){let r=ee.fun(e),[[n],i]=jr(1,r?e:[e],r?t||[]:t);return r||2==arguments.length?[n,i]:n}var Xr=class extends gr{constructor(e,t){super(),this.source=e,this.calc=ze(...t);let r=this._get(),n=Nt(r);Ct(this,n.create(r))}key;idle=!0;calc;_active=new Set;advance(e){let t=this._get();te(t,this.get())||(Tt(this).setValue(t),this._onChange(t,this.idle)),!this.idle&&qr(this._active)&&Kr(this)}_get(){let e=ee.arr(this.source)?this.source.map(Ze):ie(Ze(this.source));return this.calc(...e)}_start(){this.idle&&!qr(this._active)&&(this.idle=!1,re(It(this),(e=>{e.done=!1})),Z.skipAnimation?(L.batchedUpdates((()=>this.advance())),Kr(this)):Ae.start(this))}_attach(){let e=1;re(ie(this.source),(t=>{Je(t)&&nt(t,this),fr(t)&&(t.idle||this._active.add(t),e=Math.max(e,t.priority+1))})),this.priority=e,this._start()}_detach(){re(ie(this.source),(e=>{Je(e)&&it(e,this)})),this._active.clear(),Kr(this)}eventObserved(e){"change"==e.type?e.idle?this.advance():(this._active.add(e.parent),this._start()):"idle"==e.type?this._active.delete(e.parent):"priority"==e.type&&(this.priority=ie(this.source).reduce(((e,t)=>Math.max(e,(fr(t)?t.priority:0)+1)),0))}};function Yr(e){return!1!==e.idle}function qr(e){return!e.size||Array.from(e).every(Yr)}function Kr(e){e.idle||(e.idle=!0,re(It(e),(e=>{e.done=!0})),et(e,{type:"idle",parent:e}))}Z.assign({createStringInterpolator:mt,to:(e,t)=>new Xr(e,t)});Ae.advance;var Jr=["primitive"].concat(Object.keys(s).filter((e=>/^[A-Z]/.test(e))).map((e=>e[0].toLowerCase()+e.slice(1))));Z.assign({createStringInterpolator:mt,colors:{transparent:0,aliceblue:4042850303,antiquewhite:4209760255,aqua:16777215,aquamarine:2147472639,azure:4043309055,beige:4126530815,bisque:4293182719,black:255,blanchedalmond:4293643775,blue:65535,blueviolet:2318131967,brown:2771004159,burlywood:3736635391,burntsienna:3934150143,cadetblue:1604231423,chartreuse:2147418367,chocolate:3530104575,coral:4286533887,cornflowerblue:1687547391,cornsilk:4294499583,crimson:3692313855,cyan:16777215,darkblue:35839,darkcyan:9145343,darkgoldenrod:3095792639,darkgray:2846468607,darkgreen:6553855,darkgrey:2846468607,darkkhaki:3182914559,darkmagenta:2332068863,darkolivegreen:1433087999,darkorange:4287365375,darkorchid:2570243327,darkred:2332033279,darksalmon:3918953215,darkseagreen:2411499519,darkslateblue:1211993087,darkslategray:793726975,darkslategrey:793726975,darkturquoise:13554175,darkviolet:2483082239,deeppink:4279538687,deepskyblue:12582911,dimgray:1768516095,dimgrey:1768516095,dodgerblue:512819199,firebrick:2988581631,floralwhite:4294635775,forestgreen:579543807,fuchsia:4278255615,gainsboro:3705462015,ghostwhite:4177068031,gold:4292280575,goldenrod:3668254975,gray:2155905279,green:8388863,greenyellow:2919182335,grey:2155905279,honeydew:4043305215,hotpink:4285117695,indianred:3445382399,indigo:1258324735,ivory:4294963455,khaki:4041641215,lavender:3873897215,lavenderblush:4293981695,lawngreen:2096890111,lemonchiffon:4294626815,lightblue:2916673279,lightcoral:4034953471,lightcyan:3774873599,lightgoldenrodyellow:4210742015,lightgray:3553874943,lightgreen:2431553791,lightgrey:3553874943,lightpink:4290167295,lightsalmon:4288707327,lightseagreen:548580095,lightskyblue:2278488831,lightslategray:2005441023,lightslategrey:2005441023,lightsteelblue:2965692159,lightyellow:4294959359,lime:16711935,limegreen:852308735,linen:4210091775,magenta:4278255615,maroon:2147483903,mediumaquamarine:1724754687,mediumblue:52735,mediumorchid:3126187007,mediumpurple:2473647103,mediumseagreen:1018393087,mediumslateblue:2070474495,mediumspringgreen:16423679,mediumturquoise:1221709055,mediumvioletred:3340076543,midnightblue:421097727,mintcream:4127193855,mistyrose:4293190143,moccasin:4293178879,navajowhite:4292783615,navy:33023,oldlace:4260751103,olive:2155872511,olivedrab:1804477439,orange:4289003775,orangered:4282712319,orchid:3664828159,palegoldenrod:4008225535,palegreen:2566625535,paleturquoise:2951671551,palevioletred:3681588223,papayawhip:4293907967,peachpuff:4292524543,peru:3448061951,pink:4290825215,plum:3718307327,powderblue:2967529215,purple:2147516671,rebeccapurple:1714657791,red:4278190335,rosybrown:3163525119,royalblue:1097458175,saddlebrown:2336560127,salmon:4202722047,sandybrown:4104413439,seagreen:780883967,seashell:4294307583,sienna:2689740287,silver:3233857791,skyblue:2278484991,slateblue:1784335871,slategray:1887473919,slategrey:1887473919,snow:4294638335,springgreen:16744447,steelblue:1182971135,tan:3535047935,teal:8421631,thistle:3636451583,tomato:4284696575,turquoise:1088475391,violet:4001558271,wheat:4125012991,white:4294967295,whitesmoke:4126537215,yellow:4294902015,yellowgreen:2597139199},frameLoop:"demand"}),(0,o.addEffect)((()=>{L.advance()}));var Zr=((e,{applyAnimatedValues:t=(()=>!1),createAnimatedStyle:r=(e=>new Lt(e)),getComponentProps:n=(e=>e)}={})=>{let i={applyAnimatedValues:t,createAnimatedStyle:r,getComponentProps:n},a=e=>{let t=zt(e)||"Anonymous";return(e=ee.str(e)?a[e]||(a[e]=kt(e,i)):e[Gt]||(e[Gt]=kt(e,i))).displayName=`Animated(${t})`,e};return ne(e,((t,r)=>{ee.arr(e)&&(r=zt(t)),a[r]=a(t)})),{animated:a}})(Jr,{applyAnimatedValues:o.applyProps}),$r=Zr.animated;const en={toVector:(e,t)=>(void 0===e&&(e=t),Array.isArray(e)?e:[e,e]),add:(e,t)=>[e[0]+t[0],e[1]+t[1]],sub:(e,t)=>[e[0]-t[0],e[1]-t[1]],addTo(e,t){e[0]+=t[0],e[1]+=t[1]},subTo(e,t){e[0]-=t[0],e[1]-=t[1]}};function tn(e,t,r){return 0===t||Math.abs(t)===1/0?Math.pow(e,5*r):e*t*r/(t+r*e)}function rn(e,t,r,n=.15){return 0===n?function(e,t,r){return Math.max(t,Math.min(e,r))}(e,t,r):er?+tn(e-r,r-t,n)+r:e}function nn(e){var t=function(e,t){if("object"!=typeof e||null===e)return e;var r=e[Symbol.toPrimitive];if(void 0!==r){var n=r.call(e,t||"default");if("object"!=typeof n)return n;throw new TypeError("@@toPrimitive must return a primitive value.")}return("string"===t?String:Number)(e)}(e,"string");return"symbol"==typeof t?t:String(t)}function an(e,t,r){return(t=nn(t))in e?Object.defineProperty(e,t,{value:r,enumerable:!0,configurable:!0,writable:!0}):e[t]=r,e}function sn(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),r.push.apply(r,n)}return r}function on(e){for(var t=1;t{var r,n;return t.target===e.currentTarget||(null===(r=e.currentTarget)||void 0===r||null===(n=r.contains)||void 0===n?void 0:n.call(r,t.target))}))}(e).map((e=>e.identifier))}function yn(e,t){const[r,n]=Array.from(e.touches).filter((e=>t.includes(e.identifier)));return vn(r,n)}function xn(e){const t=gn(e);return fn(e)?t.identifier:t.pointerId}function bn(e){const t=gn(e);return[t.clientX,t.clientY]}function wn(e){let{deltaX:t,deltaY:r,deltaMode:n}=e;return 1===n?(t*=40,r*=40):2===n&&(t*=800,r*=800),[t,r]}function _n(e,...t){return"function"==typeof e?e(...t):e}function En(){}function Sn(...e){return 0===e.length?En:1===e.length?e[0]:function(){let t;for(const r of e)t=r.apply(this,arguments)||t;return t}}function Mn(e,t){return Object.assign({},t,e||{})}class Tn{constructor(e,t,r){this.ctrl=e,this.args=t,this.key=r,this.state||(this.state={},this.computeValues([0,0]),this.computeInitial(),this.init&&this.init(),this.reset())}get state(){return this.ctrl.state[this.key]}set state(e){this.ctrl.state[this.key]=e}get shared(){return this.ctrl.state.shared}get eventStore(){return this.ctrl.gestureEventStores[this.key]}get timeoutStore(){return this.ctrl.gestureTimeoutStores[this.key]}get config(){return this.ctrl.config[this.key]}get sharedConfig(){return this.ctrl.config.shared}get handler(){return this.ctrl.handlers[this.key]}reset(){const{state:e,shared:t,ingKey:r,args:n}=this;t[r]=e._active=e.active=e._blocked=e._force=!1,e._step=[!1,!1],e.intentional=!1,e._movement=[0,0],e._distance=[0,0],e._direction=[0,0],e._delta=[0,0],e._bounds=[[-1/0,1/0],[-1/0,1/0]],e.args=n,e.axis=void 0,e.memo=void 0,e.elapsedTime=e.timeDelta=0,e.direction=[0,0],e.distance=[0,0],e.overflow=[0,0],e._movementBound=[!1,!1],e.velocity=[0,0],e.movement=[0,0],e.delta=[0,0],e.timeStamp=0}start(e){const t=this.state,r=this.config;t._active||(this.reset(),this.computeInitial(),t._active=!0,t.target=e.target,t.currentTarget=e.currentTarget,t.lastOffset=r.from?_n(r.from,t):t.offset,t.offset=t.lastOffset,t.startTime=t.timeStamp=e.timeStamp)}computeValues(e){const t=this.state;t._values=e,t.values=this.config.transform(e)}computeInitial(){const e=this.state;e._initial=e._values,e.initial=e.values}compute(e){const{state:t,config:r,shared:n}=this;t.args=this.args;let i=0;if(e&&(t.event=e,r.preventDefault&&e.cancelable&&t.event.preventDefault(),t.type=e.type,n.touches=this.ctrl.pointerIds.size||this.ctrl.touchIds.size,n.locked=!!document.pointerLockElement,Object.assign(n,function(e){const t={};if("buttons"in e&&(t.buttons=e.buttons),"shiftKey"in e){const{shiftKey:r,altKey:n,metaKey:i,ctrlKey:a}=e;Object.assign(t,{shiftKey:r,altKey:n,metaKey:i,ctrlKey:a})}return t}(e)),n.down=n.pressed=n.buttons%2==1||n.touches>0,i=e.timeStamp-t.timeStamp,t.timeStamp=e.timeStamp,t.elapsedTime=t.timeStamp-t.startTime),t._active){const e=t._delta.map(Math.abs);en.addTo(t._distance,e)}this.axisIntent&&this.axisIntent(e);const[a,s]=t._movement,[o,l]=r.threshold,{_step:c,values:u}=t;if(r.hasCustomTransform?(!1===c[0]&&(c[0]=Math.abs(a)>=o&&u[0]),!1===c[1]&&(c[1]=Math.abs(s)>=l&&u[1])):(!1===c[0]&&(c[0]=Math.abs(a)>=o&&Math.sign(a)*o),!1===c[1]&&(c[1]=Math.abs(s)>=l&&Math.sign(s)*l)),t.intentional=!1!==c[0]||!1!==c[1],!t.intentional)return;const h=[0,0];if(r.hasCustomTransform){const[e,t]=u;h[0]=!1!==c[0]?e-c[0]:0,h[1]=!1!==c[1]?t-c[1]:0}else h[0]=!1!==c[0]?a-c[0]:0,h[1]=!1!==c[1]?s-c[1]:0;this.restrictToAxis&&!t._blocked&&this.restrictToAxis(h);const d=t.offset,p=t._active&&!t._blocked||t.active;p&&(t.first=t._active&&!t.active,t.last=!t._active&&t.active,t.active=n[this.ingKey]=t._active,e&&(t.first&&("bounds"in r&&(t._bounds=_n(r.bounds,t)),this.setup&&this.setup()),t.movement=h,this.computeOffset()));const[f,m]=t.offset,[[g,v],[A,y]]=t._bounds;t.overflow=[fv?1:0,my?1:0],t._movementBound[0]=!!t.overflow[0]&&(!1===t._movementBound[0]?t._movement[0]:t._movementBound[0]),t._movementBound[1]=!!t.overflow[1]&&(!1===t._movementBound[1]?t._movement[1]:t._movementBound[1]);const x=t._active&&r.rubberband||[0,0];if(t.offset=function(e,[t,r],[n,i]){const[[a,s],[o,l]]=e;return[rn(t,a,s,n),rn(r,o,l,i)]}(t._bounds,t.offset,x),t.delta=en.sub(t.offset,d),this.computeMovement(),p&&(!t.last||i>32)){t.delta=en.sub(t.offset,d);const e=t.delta.map(Math.abs);en.addTo(t.distance,e),t.direction=t.delta.map(Math.sign),t._direction=t._delta.map(Math.sign),!t.first&&i>0&&(t.velocity=[e[0]/i,e[1]/i],t.timeDelta=i)}}emit(){const e=this.state,t=this.shared,r=this.config;if(e._active||this.clean(),(e._blocked||!e.intentional)&&!e._force&&!r.triggerAllEvents)return;const n=this.handler(on(on(on({},t),e),{},{[this.aliasKey]:e.values}));void 0!==n&&(e.memo=n)}clean(){this.eventStore.clean(),this.timeoutStore.clean()}}class Cn extends Tn{constructor(...e){super(...e),an(this,"aliasKey","xy")}reset(){super.reset(),this.state.axis=void 0}init(){this.state.offset=[0,0],this.state.lastOffset=[0,0]}computeOffset(){this.state.offset=en.add(this.state.lastOffset,this.state.movement)}computeMovement(){this.state.movement=en.sub(this.state.offset,this.state.lastOffset)}axisIntent(e){const t=this.state,r=this.config;if(!t.axis&&e){const n="object"==typeof r.axisThreshold?r.axisThreshold[mn(e)]:r.axisThreshold;t.axis=function([e,t],r){const n=Math.abs(e),i=Math.abs(t);return n>i&&n>r?"x":i>n&&i>r?"y":void 0}(t._movement,n)}t._blocked=(r.lockDirection||!!r.axis)&&!t.axis||!!r.axis&&r.axis!==t.axis}restrictToAxis(e){if(this.config.axis||this.config.lockDirection)switch(this.state.axis){case"x":e[1]=0;break;case"y":e[0]=0}}}const In=e=>e,Bn={enabled:(e=!0)=>e,eventOptions:(e,t,r)=>on(on({},r.shared.eventOptions),e),preventDefault:(e=!1)=>e,triggerAllEvents:(e=!1)=>e,rubberband(e=0){switch(e){case!0:return[.15,.15];case!1:return[0,0];default:return en.toVector(e)}},from:e=>"function"==typeof e?e:null!=e?en.toVector(e):void 0,transform(e,t,r){const n=e||r.shared.transform;return this.hasCustomTransform=!!n,n||In},threshold:e=>en.toVector(e,0)};const Rn=on(on({},Bn),{},{axis(e,t,{axis:r}){if(this.lockDirection="lock"===r,!this.lockDirection)return r},axisThreshold:(e=0)=>e,bounds(e={}){if("function"==typeof e)return t=>Rn.bounds(e(t));if("current"in e)return()=>e.current;if("function"==typeof HTMLElement&&e instanceof HTMLElement)return e;const{left:t=-1/0,right:r=1/0,top:n=-1/0,bottom:i=1/0}=e;return[[t,r],[n,i]]}}),Pn={ArrowRight:(e,t=1)=>[e*t,0],ArrowLeft:(e,t=1)=>[-1*e*t,0],ArrowUp:(e,t=1)=>[0,-1*e*t],ArrowDown:(e,t=1)=>[0,e*t]};const Fn="undefined"!=typeof window&&window.document&&window.document.createElement;function Ln(){return Fn&&"ontouchstart"in window||Fn&&window.navigator.maxTouchPoints>1}const Dn={isBrowser:Fn,gesture:function(){try{return"constructor"in GestureEvent}catch(Ha){return!1}}(),touch:Ln(),touchscreen:Ln(),pointer:Fn&&"onpointerdown"in window,pointerLock:Fn&&"exitPointerLock"in window.document},Un={mouse:0,touch:0,pen:8},Nn=on(on({},Rn),{},{device(e,t,{pointer:{touch:r=!1,lock:n=!1,mouse:i=!1}={}}){return this.pointerLock=n&&Dn.pointerLock,Dn.touch&&r?"touch":this.pointerLock?"mouse":Dn.pointer&&!i?"pointer":Dn.touch?"touch":"mouse"},preventScrollAxis(e,t,{preventScroll:r}){if(this.preventScrollDelay="number"==typeof r?r:r||void 0===r&&e?250:void 0,Dn.touchscreen&&!1!==r)return e||(void 0!==r?"y":void 0)},pointerCapture(e,t,{pointer:{capture:r=!0,buttons:n=1,keys:i=!0}={}}){return this.pointerButtons=n,this.keys=i,!this.pointerLock&&"pointer"===this.device&&r},threshold(e,t,{filterTaps:r=!1,tapsThreshold:n=3,axis:i}){const a=en.toVector(e,r?n:i?1:0);return this.filterTaps=r,this.tapsThreshold=n,a},swipe({velocity:e=.5,distance:t=50,duration:r=250}={}){return{velocity:this.transform(en.toVector(e)),distance:this.transform(en.toVector(t)),duration:r}},delay(e=0){switch(e){case!0:return 180;case!1:return 0;default:return e}},axisThreshold:e=>e?on(on({},Un),e):Un,keyboardDisplacement:(e=10)=>e});function kn(e){const[t,r]=e.overflow,[n,i]=e._delta,[a,s]=e._direction;(t<0&&n>0&&a<0||t>0&&n<0&&a>0)&&(e._movement[0]=e._movementBound[0]),(r<0&&i>0&&s<0||r>0&&i<0&&s>0)&&(e._movement[1]=e._movementBound[1])}const On=on(on({},Bn),{},{device(e,t,{shared:r,pointer:{touch:n=!1}={}}){if(r.target&&!Dn.touch&&Dn.gesture)return"gesture";if(Dn.touch&&n)return"touch";if(Dn.touchscreen){if(Dn.pointer)return"pointer";if(Dn.touch)return"touch"}},bounds(e,t,{scaleBounds:r={},angleBounds:n={}}){const i=e=>{const t=Mn(_n(r,e),{min:-1/0,max:1/0});return[t.min,t.max]},a=e=>{const t=Mn(_n(n,e),{min:-1/0,max:1/0});return[t.min,t.max]};return"function"!=typeof r&&"function"!=typeof n?[i(),a()]:e=>[i(e),a(e)]},threshold(e,t,r){this.lockDirection="lock"===r.axis;return en.toVector(e,this.lockDirection?[.1,3]:0)},modifierKey:e=>void 0===e?"ctrlKey":e,pinchOnWheel:(e=!0)=>e});const Gn=on(on({},Rn),{},{mouseOnly:(e=!0)=>e});const zn=Rn;const Vn=Rn;const Hn=on(on({},Rn),{},{mouseOnly:(e=!0)=>e}),Qn=new Map,jn=new Map;function Wn(e){Qn.set(e.key,e.engine),jn.set(e.key,e.resolver)}const Xn={key:"drag",engine:class extends Cn{constructor(...e){super(...e),an(this,"ingKey","dragging")}reset(){super.reset();const e=this.state;e._pointerId=void 0,e._pointerActive=!1,e._keyboardActive=!1,e._preventScroll=!1,e._delayed=!1,e.swipe=[0,0],e.tap=!1,e.canceled=!1,e.cancel=this.cancel.bind(this)}setup(){const e=this.state;if(e._bounds instanceof HTMLElement){const t=e._bounds.getBoundingClientRect(),r=e.currentTarget.getBoundingClientRect(),n={left:t.left-r.left+e.offset[0],right:t.right-r.right+e.offset[0],top:t.top-r.top+e.offset[1],bottom:t.bottom-r.bottom+e.offset[1]};e._bounds=Rn.bounds(n)}}cancel(){const e=this.state;e.canceled||(e.canceled=!0,e._active=!1,setTimeout((()=>{this.compute(),this.emit()}),0))}setActive(){this.state._active=this.state._pointerActive||this.state._keyboardActive}clean(){this.pointerClean(),this.state._pointerActive=!1,this.state._keyboardActive=!1,super.clean()}pointerDown(e){const t=this.config,r=this.state;if(null!=e.buttons&&(Array.isArray(t.pointerButtons)?!t.pointerButtons.includes(e.buttons):-1!==t.pointerButtons&&t.pointerButtons!==e.buttons))return;const n=this.ctrl.setEventIds(e);t.pointerCapture&&e.target.setPointerCapture(e.pointerId),n&&n.size>1&&r._pointerActive||(this.start(e),this.setupPointer(e),r._pointerId=xn(e),r._pointerActive=!0,this.computeValues(bn(e)),this.computeInitial(),t.preventScrollAxis&&"mouse"!==mn(e)?(r._active=!1,this.setupScrollPrevention(e)):t.delay>0?(this.setupDelayTrigger(e),t.triggerAllEvents&&(this.compute(e),this.emit())):this.startPointerDrag(e))}startPointerDrag(e){const t=this.state;t._active=!0,t._preventScroll=!0,t._delayed=!1,this.compute(e),this.emit()}pointerMove(e){const t=this.state,r=this.config;if(!t._pointerActive)return;const n=xn(e);if(void 0!==t._pointerId&&n!==t._pointerId)return;const i=bn(e);return document.pointerLockElement===e.target?t._delta=[e.movementX,e.movementY]:(t._delta=en.sub(i,t._values),this.computeValues(i)),en.addTo(t._movement,t._delta),this.compute(e),t._delayed&&t.intentional?(this.timeoutStore.remove("dragDelay"),t.active=!1,void this.startPointerDrag(e)):r.preventScrollAxis&&!t._preventScroll?t.axis?t.axis===r.preventScrollAxis||"xy"===r.preventScrollAxis?(t._active=!1,void this.clean()):(this.timeoutStore.remove("startPointerDrag"),void this.startPointerDrag(e)):void 0:void this.emit()}pointerUp(e){this.ctrl.setEventIds(e);try{this.config.pointerCapture&&e.target.hasPointerCapture(e.pointerId)&&e.target.releasePointerCapture(e.pointerId)}catch(s){0}const t=this.state,r=this.config;if(!t._active||!t._pointerActive)return;const n=xn(e);if(void 0!==t._pointerId&&n!==t._pointerId)return;this.state._pointerActive=!1,this.setActive(),this.compute(e);const[i,a]=t._distance;if(t.tap=i<=r.tapsThreshold&&a<=r.tapsThreshold,t.tap&&r.filterTaps)t._force=!0;else{const[e,n]=t._delta,[i,a]=t._movement,[s,o]=r.swipe.velocity,[l,c]=r.swipe.distance,u=r.swipe.duration;if(t.elapsedTimes&&Math.abs(i)>l&&(t.swipe[0]=Math.sign(e)),u>o&&Math.abs(a)>c&&(t.swipe[1]=Math.sign(n))}}this.emit()}pointerClick(e){!this.state.tap&&e.detail>0&&(e.preventDefault(),e.stopPropagation())}setupPointer(e){const t=this.config,r=t.device;t.pointerLock&&e.currentTarget.requestPointerLock(),t.pointerCapture||(this.eventStore.add(this.sharedConfig.window,r,"change",this.pointerMove.bind(this)),this.eventStore.add(this.sharedConfig.window,r,"end",this.pointerUp.bind(this)),this.eventStore.add(this.sharedConfig.window,r,"cancel",this.pointerUp.bind(this)))}pointerClean(){this.config.pointerLock&&document.pointerLockElement===this.state.currentTarget&&document.exitPointerLock()}preventScroll(e){this.state._preventScroll&&e.cancelable&&e.preventDefault()}setupScrollPrevention(e){this.state._preventScroll=!1,function(e){"persist"in e&&"function"==typeof e.persist&&e.persist()}(e);const t=this.eventStore.add(this.sharedConfig.window,"touch","change",this.preventScroll.bind(this),{passive:!1});this.eventStore.add(this.sharedConfig.window,"touch","end",t),this.eventStore.add(this.sharedConfig.window,"touch","cancel",t),this.timeoutStore.add("startPointerDrag",this.startPointerDrag.bind(this),this.config.preventScrollDelay,e)}setupDelayTrigger(e){this.state._delayed=!0,this.timeoutStore.add("dragDelay",(()=>{this.state._step=[0,0],this.startPointerDrag(e)}),this.config.delay)}keyDown(e){const t=Pn[e.key];if(t){const r=this.state,n=e.shiftKey?10:e.altKey?.1:1;this.start(e),r._delta=t(this.config.keyboardDisplacement,n),r._keyboardActive=!0,en.addTo(r._movement,r._delta),this.compute(e),this.emit()}}keyUp(e){e.key in Pn&&(this.state._keyboardActive=!1,this.setActive(),this.compute(e),this.emit())}bind(e){const t=this.config.device;e(t,"start",this.pointerDown.bind(this)),this.config.pointerCapture&&(e(t,"change",this.pointerMove.bind(this)),e(t,"end",this.pointerUp.bind(this)),e(t,"cancel",this.pointerUp.bind(this)),e("lostPointerCapture","",this.pointerUp.bind(this))),this.config.keys&&(e("key","down",this.keyDown.bind(this)),e("key","up",this.keyUp.bind(this))),this.config.filterTaps&&e("click","",this.pointerClick.bind(this),{capture:!0,passive:!1})}},resolver:Nn},Yn={key:"hover",engine:class extends Cn{constructor(...e){super(...e),an(this,"ingKey","hovering")}enter(e){this.config.mouseOnly&&"mouse"!==e.pointerType||(this.start(e),this.computeValues(bn(e)),this.compute(e),this.emit())}leave(e){if(this.config.mouseOnly&&"mouse"!==e.pointerType)return;const t=this.state;if(!t._active)return;t._active=!1;const r=bn(e);t._movement=t._delta=en.sub(r,t._values),this.computeValues(r),this.compute(e),t.delta=t.movement,this.emit()}bind(e){e("pointer","enter",this.enter.bind(this)),e("pointer","leave",this.leave.bind(this))}},resolver:Hn},qn={key:"move",engine:class extends Cn{constructor(...e){super(...e),an(this,"ingKey","moving")}move(e){this.config.mouseOnly&&"mouse"!==e.pointerType||(this.state._active?this.moveChange(e):this.moveStart(e),this.timeoutStore.add("moveEnd",this.moveEnd.bind(this)))}moveStart(e){this.start(e),this.computeValues(bn(e)),this.compute(e),this.computeInitial(),this.emit()}moveChange(e){if(!this.state._active)return;const t=bn(e),r=this.state;r._delta=en.sub(t,r._values),en.addTo(r._movement,r._delta),this.computeValues(t),this.compute(e),this.emit()}moveEnd(e){this.state._active&&(this.state._active=!1,this.compute(e),this.emit())}bind(e){e("pointer","change",this.move.bind(this)),e("pointer","leave",this.moveEnd.bind(this))}},resolver:Gn},Kn={key:"pinch",engine:class extends Tn{constructor(...e){super(...e),an(this,"ingKey","pinching"),an(this,"aliasKey","da")}init(){this.state.offset=[1,0],this.state.lastOffset=[1,0],this.state._pointerEvents=new Map}reset(){super.reset();const e=this.state;e._touchIds=[],e.canceled=!1,e.cancel=this.cancel.bind(this),e.turns=0}computeOffset(){const{type:e,movement:t,lastOffset:r}=this.state;this.state.offset="wheel"===e?en.add(t,r):[(1+t[0])*r[0],t[1]+r[1]]}computeMovement(){const{offset:e,lastOffset:t}=this.state;this.state.movement=[e[0]/t[0],e[1]-t[1]]}axisIntent(){const e=this.state,[t,r]=e._movement;if(!e.axis){const n=30*Math.abs(t)-Math.abs(r);n<0?e.axis="angle":n>0&&(e.axis="scale")}}restrictToAxis(e){this.config.lockDirection&&("scale"===this.state.axis?e[1]=0:"angle"===this.state.axis&&(e[0]=0))}cancel(){const e=this.state;e.canceled||setTimeout((()=>{e.canceled=!0,e._active=!1,this.compute(),this.emit()}),0)}touchStart(e){this.ctrl.setEventIds(e);const t=this.state,r=this.ctrl.touchIds;if(t._active&&t._touchIds.every((e=>r.has(e))))return;if(r.size<2)return;this.start(e),t._touchIds=Array.from(r).slice(0,2);const n=yn(e,t._touchIds);n&&this.pinchStart(e,n)}pointerStart(e){if(null!=e.buttons&&e.buttons%2!=1)return;this.ctrl.setEventIds(e),e.target.setPointerCapture(e.pointerId);const t=this.state,r=t._pointerEvents,n=this.ctrl.pointerIds;if(t._active&&Array.from(r.keys()).every((e=>n.has(e))))return;if(r.size<2&&r.set(e.pointerId,e),t._pointerEvents.size<2)return;this.start(e);const i=vn(...Array.from(r.values()));i&&this.pinchStart(e,i)}pinchStart(e,t){this.state.origin=t.origin,this.computeValues([t.distance,t.angle]),this.computeInitial(),this.compute(e),this.emit()}touchMove(e){if(!this.state._active)return;const t=yn(e,this.state._touchIds);t&&this.pinchMove(e,t)}pointerMove(e){const t=this.state._pointerEvents;if(t.has(e.pointerId)&&t.set(e.pointerId,e),!this.state._active)return;const r=vn(...Array.from(t.values()));r&&this.pinchMove(e,r)}pinchMove(e,t){const r=this.state,n=r._values[1],i=t.angle-n;let a=0;Math.abs(i)>270&&(a+=Math.sign(i)),this.computeValues([t.distance,t.angle-360*a]),r.origin=t.origin,r.turns=a,r._movement=[r._values[0]/r._initial[0]-1,r._values[1]-r._initial[1]],this.compute(e),this.emit()}touchEnd(e){this.ctrl.setEventIds(e),this.state._active&&this.state._touchIds.some((e=>!this.ctrl.touchIds.has(e)))&&(this.state._active=!1,this.compute(e),this.emit())}pointerEnd(e){const t=this.state;this.ctrl.setEventIds(e);try{e.target.releasePointerCapture(e.pointerId)}catch(r){}t._pointerEvents.has(e.pointerId)&&t._pointerEvents.delete(e.pointerId),t._active&&t._pointerEvents.size<2&&(t._active=!1,this.compute(e),this.emit())}gestureStart(e){e.cancelable&&e.preventDefault();const t=this.state;t._active||(this.start(e),this.computeValues([e.scale,e.rotation]),t.origin=[e.clientX,e.clientY],this.compute(e),this.emit())}gestureMove(e){if(e.cancelable&&e.preventDefault(),!this.state._active)return;const t=this.state;this.computeValues([e.scale,e.rotation]),t.origin=[e.clientX,e.clientY];const r=t._movement;t._movement=[e.scale-1,e.rotation],t._delta=en.sub(t._movement,r),this.compute(e),this.emit()}gestureEnd(e){this.state._active&&(this.state._active=!1,this.compute(e),this.emit())}wheel(e){const t=this.config.modifierKey;t&&!e[t]||(this.state._active?this.wheelChange(e):this.wheelStart(e),this.timeoutStore.add("wheelEnd",this.wheelEnd.bind(this)))}wheelStart(e){this.start(e),this.wheelChange(e)}wheelChange(e){"uv"in e||e.cancelable&&e.preventDefault();const t=this.state;t._delta=[-wn(e)[1]/100*t.offset[0],0],en.addTo(t._movement,t._delta),kn(t),this.state.origin=[e.clientX,e.clientY],this.compute(e),this.emit()}wheelEnd(){this.state._active&&(this.state._active=!1,this.compute(),this.emit())}bind(e){const t=this.config.device;t&&(e(t,"start",this[t+"Start"].bind(this)),e(t,"change",this[t+"Move"].bind(this)),e(t,"end",this[t+"End"].bind(this)),e(t,"cancel",this[t+"End"].bind(this)),e("lostPointerCapture","",this[t+"End"].bind(this))),this.config.pinchOnWheel&&e("wheel","",this.wheel.bind(this),{passive:!1})}},resolver:On},Jn={key:"scroll",engine:class extends Cn{constructor(...e){super(...e),an(this,"ingKey","scrolling")}scroll(e){this.state._active||this.start(e),this.scrollChange(e),this.timeoutStore.add("scrollEnd",this.scrollEnd.bind(this))}scrollChange(e){e.cancelable&&e.preventDefault();const t=this.state,r=function(e){var t,r;const{scrollX:n,scrollY:i,scrollLeft:a,scrollTop:s}=e.currentTarget;return[null!==(t=null!=n?n:a)&&void 0!==t?t:0,null!==(r=null!=i?i:s)&&void 0!==r?r:0]}(e);t._delta=en.sub(r,t._values),en.addTo(t._movement,t._delta),this.computeValues(r),this.compute(e),this.emit()}scrollEnd(){this.state._active&&(this.state._active=!1,this.compute(),this.emit())}bind(e){e("scroll","",this.scroll.bind(this))}},resolver:zn},Zn={key:"wheel",engine:class extends Cn{constructor(...e){super(...e),an(this,"ingKey","wheeling")}wheel(e){this.state._active||this.start(e),this.wheelChange(e),this.timeoutStore.add("wheelEnd",this.wheelEnd.bind(this))}wheelChange(e){const t=this.state;t._delta=wn(e),en.addTo(t._movement,t._delta),kn(t),this.compute(e),this.emit()}wheelEnd(){this.state._active&&(this.state._active=!1,this.compute(),this.emit())}bind(e){e("wheel","",this.wheel.bind(this))}},resolver:Vn};function $n(e,t){if(null==e)return{};var r,n,i=function(e,t){if(null==e)return{};var r,n,i={},a=Object.keys(e);for(n=0;n=0||(i[r]=e[r]);return i}(e,t);if(Object.getOwnPropertySymbols){var a=Object.getOwnPropertySymbols(e);for(n=0;n=0||Object.prototype.propertyIsEnumerable.call(e,r)&&(i[r]=e[r])}return i}const ei={target(e){if(e)return()=>"current"in e?e.current:e},enabled:(e=!0)=>e,window:(e=(Dn.isBrowser?window:void 0))=>e,eventOptions:({passive:e=!0,capture:t=!1}={})=>({passive:e,capture:t}),transform:e=>e},ti=["target","eventOptions","window","enabled","transform"];function ri(e={},t){const r={};for(const[n,i]of Object.entries(t))switch(typeof i){case"function":r[n]=i.call(r,e[n],n,e);break;case"object":r[n]=ri(e[n],i);break;case"boolean":i&&(r[n]=e[n])}return r}class ni{constructor(e,t){an(this,"_listeners",new Set),this._ctrl=e,this._gestureKey=t}add(e,t,r,n,i){const a=this._listeners,s=function(e,t=""){const r=ln[e];return e+(r&&r[t]||t)}(t,r),o=on(on({},this._gestureKey?this._ctrl.config[this._gestureKey].eventOptions:{}),i);e.addEventListener(s,n,o);const l=()=>{e.removeEventListener(s,n,o),a.delete(l)};return a.add(l),l}clean(){this._listeners.forEach((e=>e())),this._listeners.clear()}}class ii{constructor(){an(this,"_timeouts",new Map)}add(e,t,r=140,...n){this.remove(e),this._timeouts.set(e,window.setTimeout(t,r,...n))}remove(e){const t=this._timeouts.get(e);t&&window.clearTimeout(t)}clean(){this._timeouts.forEach((e=>{window.clearTimeout(e)})),this._timeouts.clear()}}class ai{constructor(e){an(this,"gestures",new Set),an(this,"_targetEventStore",new ni(this)),an(this,"gestureEventStores",{}),an(this,"gestureTimeoutStores",{}),an(this,"handlers",{}),an(this,"config",{}),an(this,"pointerIds",new Set),an(this,"touchIds",new Set),an(this,"state",{shared:{shiftKey:!1,metaKey:!1,ctrlKey:!1,altKey:!1}}),function(e,t){t.drag&&si(e,"drag");t.wheel&&si(e,"wheel");t.scroll&&si(e,"scroll");t.move&&si(e,"move");t.pinch&&si(e,"pinch");t.hover&&si(e,"hover")}(this,e)}setEventIds(e){return fn(e)?(this.touchIds=new Set(An(e)),this.touchIds):"pointerId"in e?("pointerup"===e.type||"pointercancel"===e.type?this.pointerIds.delete(e.pointerId):"pointerdown"===e.type&&this.pointerIds.add(e.pointerId),this.pointerIds):void 0}applyHandlers(e,t){this.handlers=e,this.nativeHandlers=t}applyConfig(e,t){this.config=function(e,t,r={}){const n=e,{target:i,eventOptions:a,window:s,enabled:o,transform:l}=n,c=$n(n,ti);if(r.shared=ri({target:i,eventOptions:a,window:s,enabled:o,transform:l},ei),t){const e=jn.get(t);r[t]=ri(on({shared:r.shared},c),e)}else for(const u in c){const e=jn.get(u);e&&(r[u]=ri(on({shared:r.shared},c[u]),e))}return r}(e,t,this.config)}clean(){this._targetEventStore.clean();for(const e of this.gestures)this.gestureEventStores[e].clean(),this.gestureTimeoutStores[e].clean()}effect(){return this.config.shared.target&&this.bind(),()=>this._targetEventStore.clean()}bind(...e){const t=this.config.shared,r={};let n;if(!t.target||(n=t.target(),n)){if(t.enabled){for(const t of this.gestures){const i=this.config[t],a=oi(r,i.eventOptions,!!n);if(i.enabled){new(Qn.get(t))(this,e,t).bind(a)}}const i=oi(r,t.eventOptions,!!n);for(const t in this.nativeHandlers)i(t,"",(r=>this.nativeHandlers[t](on(on({},this.state.shared),{},{event:r,args:e}))),void 0,!0)}for(const e in r)r[e]=Sn(...r[e]);if(!n)return r;for(const e in r){const{device:t,capture:i,passive:a}=pn(e);this._targetEventStore.add(n,t,"",r[e],{capture:i,passive:a})}}}}function si(e,t){e.gestures.add(t),e.gestureEventStores[t]=new ni(e,t),e.gestureTimeoutStores[t]=new ii}const oi=(e,t,r)=>(n,i,a,s={},o=!1)=>{var l,c;const u=null!==(l=s.capture)&&void 0!==l?l:t.capture,h=null!==(c=s.passive)&&void 0!==c?c:t.passive;let d=o?n:hn(n,i,u);r&&h&&(d+="Passive"),e[d]=e[d]||[],e[d].push(a)},li=/^on(Drag|Wheel|Scroll|Move|Pinch|Hover)/;function ci(e,t,r,n,i,a){if(!e.has(r))return;if(!Qn.has(n))return void 0;const s=r+"Start",o=r+"End";i[n]=e=>{let n;return e.first&&s in t&&t[s](e),r in t&&(n=t[r](e)),e.last&&o in t&&t[o](e),n},a[n]=a[n]||{}}function ui(e,t){const[r,n,i]=function(e){const t={},r={},n=new Set;for(let i in e)li.test(i)?(n.add(RegExp.lastMatch),r[i]=e[i]):t[i]=e[i];return[r,t,n]}(e),a={};return ci(i,r,"onDrag","drag",a,t),ci(i,r,"onWheel","wheel",a,t),ci(i,r,"onScroll","scroll",a,t),ci(i,r,"onPinch","pinch",a,t),ci(i,r,"onMove","move",a,t),ci(i,r,"onHover","hover",a,t),{handlers:a,config:t,nativeHandlers:n}}function hi(e,t={},r,n){const a=i.useMemo((()=>new ai(e)),[]);if(a.applyHandlers(e,n),a.applyConfig(t,r),i.useEffect(a.effect.bind(a)),i.useEffect((()=>a.clean.bind(a)),[]),void 0===t.target)return a.bind.bind(a)}function di(e,t){const r=([Xn,Kn,Jn,Zn,qn,Yn].forEach(Wn),function(e,t){const{handlers:r,nativeHandlers:n,config:i}=ui(e,t||{});return hi(r,i,void 0,n)});return r(e,t||{})}function pi({snap:e,global:t,cursor:r=!0,children:a,speed:l=1,rotation:c=[0,0,0],zoom:u=1,polar:h=[0,Math.PI/2],azimuth:d=[-1/0,1/0],config:p={mass:1,tension:170,friction:26}}){const{size:f,gl:m}=(0,o.useThree)(),g=i.useMemo((()=>[c[0]+h[0],c[0]+h[1]]),[c[0],h[0],h[1]]),v=i.useMemo((()=>[c[1]+d[0],c[1]+d[1]]),[c[1],d[0],d[1]]),A=i.useMemo((()=>[s.MathUtils.clamp(c[0],...g),s.MathUtils.clamp(c[1],...v),c[2]]),[c[0],c[1],c[2],g,v]),[y,x]=Wr((()=>({scale:1,rotation:A,config:p})));i.useEffect((()=>{x.start({scale:1,rotation:A,config:p})}),[A]),i.useEffect((()=>{t&&r&&(m.domElement.style.cursor="grab")}),[t,r,m.domElement]);const b=di({onHover:({last:e})=>{r&&!t&&(m.domElement.style.cursor=e?"auto":"grab")},onDrag:({down:t,delta:[n,i],memo:[a,o]=y.rotation.animation.to||A})=>{r&&(m.domElement.style.cursor=t?"grabbing":"grab"),n=s.MathUtils.clamp(o+n/f.width*Math.PI*l,...v),i=s.MathUtils.clamp(a+i/f.height*Math.PI*l,...g);const c=e&&!t&&"boolean"!=typeof e?e:p;return x.start({scale:t&&i>g[1]/2?u:1,rotation:e&&!t?A:[i,n,0],config:e=>"scale"===e?{...c,friction:3*c.friction}:c}),[i,n]}},{target:t?m.domElement:void 0});return i.createElement($r.group,n({},null==b?void 0:b(),y),a)}var fi=r(31480),mi=r(60374);const gi=i.createContext([]);function vi({box:e,multiple:t,children:r,onChange:a,border:l="1px solid #55aaff",backgroundColor:c="rgba(75, 160, 255, 0.1)",filter:u=(e=>e),...h}){const{camera:d,raycaster:p,gl:f,controls:m,size:g,get:v}=(0,o.useThree)(),[A,y]=i.useState(!1),[x,b]=i.useReducer(((e,{object:t,shift:r})=>void 0===t?[]:Array.isArray(t)?t:r?e.includes(t)?e.filter((e=>e!==t)):[t,...e]:e[0]===t?[]:[t]),[]);i.useEffect((()=>{null==a||a(x)}),[x]);const w=i.useCallback((e=>{e.stopPropagation(),b({object:u([e.object])[0],shift:t&&e.shiftKey})}),[]),_=i.useCallback((e=>!A&&b({})),[A]),E=i.useRef(null);return i.useEffect((()=>{if(!e||!t)return;const r=new fi.M(d,E.current),n=document.createElement("div");n.style.pointerEvents="none",n.style.border=l,n.style.backgroundColor=c,n.style.position="fixed";const i=new s.Vector2,a=new s.Vector2,o=new s.Vector2,h=p.enabled,A=null==m?void 0:m.enabled;let y=!1;function x(e,t){var r;const{offsetX:n,offsetY:i}=null!==(r=null==p.computeOffsets?void 0:p.computeOffsets(e,v()))&&void 0!==r?r:e,{width:a,height:s}=g;t.set(n/a*2-1,-i/s*2+1)}function w(e){e.shiftKey&&(!function(e){var t;m&&(m.enabled=!1),p.enabled=!1,y=!0,null==(t=f.domElement.parentElement)||t.appendChild(n),n.style.left=`${e.clientX}px`,n.style.top=`${e.clientY}px`,n.style.width="0px",n.style.height="0px",i.x=e.clientX,i.y=e.clientY}(e),x(e,r.startPoint))}let _=[];function S(e){if(y){!function(e){o.x=Math.max(i.x,e.clientX),o.y=Math.max(i.y,e.clientY),a.x=Math.min(i.x,e.clientX),a.y=Math.min(i.y,e.clientY),n.style.left=`${a.x}px`,n.style.top=`${a.y}px`,n.style.width=o.x-a.x+"px",n.style.height=o.y-a.y+"px"}(e),x(e,r.endPoint);const t=r.select().sort((e=>e.uuid)).filter((e=>e.isMesh));(0,mi.Z)(t,_)||(_=t,b({object:u(t)}))}}function M(e){var t;y&&y&&(m&&(m.enabled=A),p.enabled=h,y=!1,null==(t=n.parentElement)||t.removeChild(n))}return document.addEventListener("pointerdown",w,{passive:!0}),document.addEventListener("pointermove",S,{passive:!0,capture:!0}),document.addEventListener("pointerup",M,{passive:!0}),()=>{document.removeEventListener("pointerdown",w),document.removeEventListener("pointermove",S),document.removeEventListener("pointerup",M)}}),[g,p,d,m,f]),i.createElement("group",n({ref:E,onClick:w,onPointerOver:()=>y(!0),onPointerOut:()=>y(!1),onPointerMissed:_},h),i.createElement(gi.Provider,{value:x},r))}function Ai(){return i.useContext(gi)}const yi=i.forwardRef((function({follow:e=!0,lockX:t=!1,lockY:r=!1,lockZ:a=!1,...s},l){const c=i.useRef();return(0,o.useFrame)((({camera:n})=>{if(!e||!c.current)return;const i=c.current.rotation.clone();c.current.quaternion.copy(n.quaternion),t&&(c.current.rotation.x=i.x),r&&(c.current.rotation.y=i.y),a&&(c.current.rotation.z=i.z)})),i.createElement("group",n({ref:M([c,l])},s))}));var xi=r(43075),bi=r(10977),wi=r(51815);const _i=i.forwardRef((function({points:e,color:t="black",vertexColors:r,lineWidth:a,dashed:o,...l},c){const[u]=i.useState((()=>new xi.w)),[h]=i.useState((()=>new bi.Y)),[d]=i.useState((()=>new s.Vector2(512,512))),p=i.useMemo((()=>{const t=new wi.L,n=e.map((e=>e instanceof s.Vector3?e.toArray():e));if(t.setPositions(n.flat()),r){const e=r.map((e=>e instanceof s.Color?e.toArray():e));t.setColors(e.flat())}return t}),[e,r]);return i.useLayoutEffect((()=>{u.computeLineDistances()}),[e,u]),i.useLayoutEffect((()=>{o?h.defines.USE_DASH="":delete h.defines.USE_DASH,h.needsUpdate=!0}),[o,h]),i.useEffect((()=>()=>p.dispose()),[p]),i.createElement("primitive",n({object:u,ref:c},l),i.createElement("primitive",{object:p,attach:"geometry"}),i.createElement("primitive",n({object:h,attach:"material",color:t,vertexColors:Boolean(r),resolution:d,linewidth:a,dashed:o},l)))})),Ei=new s.Vector3,Si=i.forwardRef((function({start:e=[0,0,0],end:t=[0,0,0],mid:r,segments:a=20,...o},l){const c=i.useRef(null),[u]=i.useState((()=>new s.QuadraticBezierCurve3(void 0,void 0,void 0))),h=i.useCallback(((e,t,r,n=20)=>(e instanceof s.Vector3?u.v0.copy(e):u.v0.set(...e),t instanceof s.Vector3?u.v2.copy(t):u.v2.set(...t),r instanceof s.Vector3?u.v1.copy(r):u.v1.copy(u.v0.clone().add(u.v2.clone().sub(u.v0)).add(Ei.set(0,u.v0.y-u.v2.y,0))),u.getPoints(n))),[]);i.useLayoutEffect((()=>{c.current.setPoints=(e,t,r)=>{const n=h(e,t,r);c.current.geometry&&c.current.geometry.setPositions(n.map((e=>e.toArray())).flat())}}),[]);const d=i.useMemo((()=>h(e,t,r,a)),[e,t,r,a]);return i.createElement(_i,n({ref:M([c,l]),points:d},o))})),Mi=i.forwardRef((function({start:e,end:t,midA:r,midB:a,segments:o=20,...l},c){const u=i.useMemo((()=>{const n=e instanceof s.Vector3?e:new s.Vector3(...e),i=t instanceof s.Vector3?t:new s.Vector3(...t),l=r instanceof s.Vector3?r:new s.Vector3(...r),c=a instanceof s.Vector3?a:new s.Vector3(...a);return new s.CubicBezierCurve3(n,l,c,i).getPoints(o)}),[e,t,r,a,o]);return i.createElement(_i,n({ref:c,points:u},l))})),Ti=i.forwardRef((({url:e,distance:t=1,loop:r=!0,autoplay:a,...l},c)=>{const u=i.useRef(),h=(0,o.useThree)((({camera:e})=>e)),[d]=i.useState((()=>new s.AudioListener)),p=(0,o.useLoader)(s.AudioLoader,e);return i.useEffect((()=>{const e=u.current;e&&(e.setBuffer(p),e.setRefDistance(t),e.setLoop(r),a&&!e.isPlaying&&e.play())}),[p,h,t,r]),i.useEffect((()=>{const e=u.current;return h.add(d),()=>{h.remove(d),e&&(e.isPlaying&&e.stop(),e.source&&e.source._connected&&e.disconnect())}}),[]),i.createElement("positionalAudio",n({ref:M([u,c]),args:[d]},l))}));function Ci(){var e,t=0,r=[],n=0,i=0;var a=d((function(e){i||o(1,e)})),s=d((function(e){i||o(-1,e)}));function o(r,n){i++;var a=0;try{n===m&&p();var s=r>0&&h(n);s?s.call(n,d((function(e){a++,o(1,e)})),d((function(e){a++,o(-1,e)}))):(t=r,e=n,l())}catch(Ha){t||a||o(-1,Ha)}}function l(){n||(setTimeout(c,0),n=1)}function c(){var e=r;n=0,r=[],e.forEach(u)}function u(e){e()}function h(e){var t=e&&(f(e)||"object"==typeof e)&&e.then;return f(t)&&t}function d(e){var t=0;return function(){for(var r=[],n=arguments.length;n--;)r[n]=arguments[n];t++||e.apply(this,r)}}function p(){throw new TypeError("Chaining cycle detected")}var f=function(e){return"function"==typeof e},m={then:function(n,i){var a=Ci();return r.push((function(){var r=t>0?n:i;if(f(r))try{var s=r(e);s===a&&p();var o=h(s);o?o.call(s,a.resolve,a.reject):a.resolve(s)}catch(l){a.reject(l)}else a[t>0?"resolve":"reject"](e)})),t&&l(),a},resolve:a,reject:s};return m}function Ii(){var e,t,r=new Promise((function(r,n){e=r,t=n}));return{then:r.then.bind(r),resolve:e,reject:t}}Ci.all=Ii.all=function(e){var t=0,r=[],n=Bi();return 0===e.length?n.resolve([]):e.forEach((function(i,a){var s=Bi();s.resolve(i),s.then((function(i){t++,r[a]=i,t===e.length&&n.resolve(r)}),n.reject)})),n};var Bi="function"==typeof Promise?Ii:Ci;function Ri(){var e=Object.create(null);function t(n,i){var a=n.id,s=n.name,o=n.dependencies;void 0===o&&(o=[]);var l=n.init;void 0===l&&(l=function(){});var c=n.getTransferables;if(void 0===c&&(c=null),!e[a])try{o=o.map((function(r){return r&&r.isWorkerModule&&(t(r,(function(e){if(e instanceof Error)throw e})),r=e[r.id].value),r})),l=r("<"+s+">.init",l),c&&(c=r("<"+s+">.getTransferables",c));var u=null;"function"==typeof l?u=l.apply(void 0,o):console.error("worker module init function failed to rehydrate"),e[a]={id:a,value:u,getTransferables:c},i(u)}catch(h){h&&h.noLog||console.error(h),i(h)}}function r(e,t){var r=void 0;self.troikaDefine=function(e){return r=e};var n=URL.createObjectURL(new Blob(["/** "+e.replace(/\*/g,"")+" **/\n\ntroikaDefine(\n"+t+"\n)"],{type:"application/javascript"}));try{importScripts(n)}catch(i){console.error(i)}return URL.revokeObjectURL(n),delete self.troikaDefine,r}self.addEventListener("message",(function(r){var n=r.data,i=n.messageId,a=n.action,s=n.data;try{"registerModule"===a&&t(s,(function(e){e instanceof Error?postMessage({messageId:i,success:!1,error:e.message}):postMessage({messageId:i,success:!0,result:{isCallable:"function"==typeof e}})})),"callModule"===a&&function(t,r){var n,i=t.id,a=t.args;e[i]&&"function"==typeof e[i].value||r(new Error("Worker module "+i+": not found or its 'init' did not return a function"));try{var s=(n=e[i]).value.apply(n,a);s&&"function"==typeof s.then?s.then(o,(function(e){return r(e instanceof Error?e:new Error(""+e))})):o(s)}catch(l){r(l)}function o(t){try{var n=e[i].getTransferables&&e[i].getTransferables(t);n&&Array.isArray(n)&&n.length||(n=void 0),r(t,n)}catch(l){console.error(l),r(l)}}}(s,(function(e,t){e instanceof Error?postMessage({messageId:i,success:!1,error:e.message}):postMessage({messageId:i,success:!0,result:e},t||void 0)}))}catch(o){postMessage({messageId:i,success:!1,error:o.stack})}}))}var Pi=function(){var e=!1;if("undefined"!=typeof window&&void 0!==window.document)try{new Worker(URL.createObjectURL(new Blob([""],{type:"application/javascript"}))).terminate(),e=!0}catch(t){console.log("Troika createWorkerModule: web workers not allowed; falling back to main thread execution. Cause: ["+t.message+"]")}return Pi=function(){return e},e},Fi=0,Li=0,Di=!1,Ui=Object.create(null),Ni=Object.create(null),ki=Object.create(null);function Oi(e){if(!(e&&"function"==typeof e.init||Di))throw new Error("requires `options.init` function");var t=e.dependencies,r=e.init,n=e.getTransferables,i=e.workerId;if(!Pi())return function(e){var t=function(){for(var e=[],r=arguments.length;r--;)e[r]=arguments[r];return t._getInitResult().then((function(t){if("function"==typeof t)return t.apply(void 0,e);throw new Error("Worker module function was called but `init` did not return a callable function")}))};return t._getInitResult=function(){var r=e.dependencies,n=e.init;r=Array.isArray(r)?r.map((function(e){return e&&e._getInitResult?e._getInitResult():e})):[];var i=Bi.all(r).then((function(e){return n.apply(null,e)}));return t._getInitResult=function(){return i},i},t}(e);null==i&&(i="#default");var a="workerModule"+ ++Fi,s=e.name||a,o=null;function l(){for(var e=[],t=arguments.length;t--;)e[t]=arguments[t];if(!o){o=zi(i,"registerModule",l.workerModuleData);var r=function(){o=null,Ni[i].delete(r)};(Ni[i]||(Ni[i]=new Set)).add(r)}return o.then((function(t){if(t.isCallable)return zi(i,"callModule",{id:a,args:e});throw new Error("Worker module function was called but `init` did not return a callable function")}))}return t=t&&t.map((function(e){return"function"!=typeof e||e.workerModuleData||(Di=!0,e=Oi({workerId:i,name:"<"+s+"> function dependency: "+e.name,init:"function(){return (\n"+Gi(e)+"\n)}"}),Di=!1),e&&e.workerModuleData&&(e=e.workerModuleData),e})),l.workerModuleData={isWorkerModule:!0,id:a,name:s,dependencies:t,init:Gi(r),getTransferables:n&&Gi(n)},l}function Gi(e){var t=e.toString();return!/^function/.test(t)&&/^\w+\s*\(/.test(t)&&(t="function "+t),t}function zi(e,t,r){var n=Bi(),i=++Li;return ki[i]=function(e){e.success?n.resolve(e.result):n.reject(new Error("Error in worker "+t+" call: "+e.error))},function(e){var t=Ui[e];if(!t){var r=Gi(Ri);(t=Ui[e]=new Worker(URL.createObjectURL(new Blob(["/** Worker Module Bootstrap: "+e.replace(/\*/g,"")+" **/\n\n;("+r+")()"],{type:"application/javascript"})))).onmessage=function(e){var t=e.data,r=t.messageId,n=ki[r];if(!n)throw new Error("WorkerModule response with empty or unknown messageId");delete ki[r],n(t)}}return t}(e).postMessage({messageId:i,action:t,data:r}),n}var Vi=Oi({name:"Thenable",dependencies:[Bi],init:function(e){return e}});function Hi(){var e=function(e){function t(e,t,r,n,i,a,s,o){var l=1-s;o.x=l*l*e+2*l*s*r+s*s*i,o.y=l*l*t+2*l*s*n+s*s*a}function r(e,t,r,n,i,a,s,o,l,c){var u=1-l;c.x=u*u*u*e+3*u*u*l*r+3*u*l*l*i+l*l*l*s,c.y=u*u*u*t+3*u*u*l*n+3*u*l*l*a+l*l*l*o}function n(e,t){for(var r,n,i,a,s,o=/([MLQCZ])([^MLQCZ]*)/g;r=o.exec(e);){var l=r[2].replace(/^\s*|\s*$/g,"").split(/[,\s]+/).map((function(e){return parseFloat(e)}));switch(r[1]){case"M":a=n=l[0],s=i=l[1];break;case"L":l[0]===a&&l[1]===s||t("L",a,s,a=l[0],s=l[1]);break;case"Q":t("Q",a,s,a=l[2],s=l[3],l[0],l[1]);break;case"C":t("C",a,s,a=l[4],s=l[5],l[0],l[1],l[2],l[3]);break;case"Z":a===n&&s===i||t("L",a,s,n,i)}}}function i(e,i,a){void 0===a&&(a=16);var s={x:0,y:0};n(e,(function(e,n,o,l,c,u,h,d,p){switch(e){case"L":i(n,o,l,c);break;case"Q":for(var f=n,m=o,g=1;g0;)n[i]=arguments[i+2];var a=o[t]||(o[t]=r.getUniformLocation(c,t));r["uniform"+e].apply(r,[a].concat(n))},setAttribute:function(e,t,n,a,o){var l=s[e];l||(l=s[e]={buf:r.createBuffer(),loc:r.getAttribLocation(c,e),data:null}),r.bindBuffer(r.ARRAY_BUFFER,l.buf),r.vertexAttribPointer(l.loc,t,r.FLOAT,!1,0,0),r.enableVertexAttribArray(l.loc),i?r.vertexAttribDivisor(l.loc,a):d("ANGLE_instanced_arrays").vertexAttribDivisorANGLE(l.loc,a),o!==l.data&&(r.bufferData(r.ARRAY_BUFFER,o,n),l.data=o)}})}}}l[e].transaction(a)}function m(e,t){u++;try{r.activeTexture(r.TEXTURE0+u);var n=c[e];n||(n=c[e]=r.createTexture(),r.bindTexture(r.TEXTURE_2D,n),r.texParameteri(r.TEXTURE_2D,r.TEXTURE_MIN_FILTER,r.NEAREST),r.texParameteri(r.TEXTURE_2D,r.TEXTURE_MAG_FILTER,r.NEAREST)),r.bindTexture(r.TEXTURE_2D,n),t(n,u)}finally{u--}}function g(e,t,n){var i=r.createFramebuffer();h.push(i),r.bindFramebuffer(r.FRAMEBUFFER,i),r.activeTexture(r.TEXTURE0+t),r.bindTexture(r.TEXTURE_2D,e),r.framebufferTexture2D(r.FRAMEBUFFER,r.COLOR_ATTACHMENT0,r.TEXTURE_2D,e,0);try{n(i)}finally{r.deleteFramebuffer(i),r.bindFramebuffer(r.FRAMEBUFFER,h[--h.length-1]||null)}}function v(){a={},l={},c={},u=-1,h.length=0}r.canvas.addEventListener("webglcontextlost",(function(e){v(),e.preventDefault()}),!1),s.set(r,n={gl:r,isWebGL2:i,getExtension:d,withProgram:f,withTexture:m,withTextureFramebuffer:g,handleContextLoss:v})}t(n)}function c(e,t,r,n,i,s,o,c){void 0===o&&(o=15),void 0===c&&(c=null),l(e,(function(e){var l=e.gl,u=e.withProgram;(0,e.withTexture)("copy",(function(e,h){l.texImage2D(l.TEXTURE_2D,0,l.RGBA,i,s,0,l.RGBA,l.UNSIGNED_BYTE,t),u("copy",a,"precision highp float;uniform sampler2D tex;varying vec2 vUV;void main(){gl_FragColor=texture2D(tex,vUV);}",(function(e){var t=e.setUniform;(0,e.setAttribute)("aUV",2,l.STATIC_DRAW,0,new Float32Array([0,0,2,0,0,2])),t("1i","image",h),l.bindFramebuffer(l.FRAMEBUFFER,c||null),l.disable(l.BLEND),l.colorMask(8&o,4&o,2&o,1&o),l.viewport(r,n,i,s),l.scissor(r,n,i,s),l.drawArrays(l.TRIANGLES,0,3)}))}))}))}var u=Object.freeze({__proto__:null,withWebGLContext:l,renderImageData:c,resizeWebGLCanvasWithoutClearing:function(e,t,r){var n=e.width,i=e.height;l(e,(function(a){var s=a.gl,o=new Uint8Array(n*i*4);s.readPixels(0,0,n,i,s.RGBA,s.UNSIGNED_BYTE,o),e.width=t,e.height=r,c(s,o,0,0,n,i)}))}});function h(e,t,r,n,a,s){void 0===s&&(s=1);var o=new Uint8Array(e*t),l=n[2]-n[0],c=n[3]-n[1],u=[];i(r,(function(e,t,r,n){u.push({x1:e,y1:t,x2:r,y2:n,minX:Math.min(e,r),minY:Math.min(t,n),maxX:Math.max(e,r),maxY:Math.max(t,n)})})),u.sort((function(e,t){return e.maxX-t.maxX}));for(var h=0;ha.minX&&t-na.minY){var s=f(e,t,a.x1,a.y1,a.x2,a.y2);st!=i.y2>t&&e<(i.x2-i.x1)*(t-i.y1)/(i.y2-i.y1)+i.x1&&(r+=i.y1p.y!=seg.w>p.y)&&(p.x<(seg.z-seg.x)*(p.y-seg.y)/(seg.w-seg.y)+seg.x);bool crossingUp=crossing&&vLineSegment.y"),c=l[0],u=l[1];c=String.fromCodePoint(n+=parseInt(c,36)),u=String.fromCodePoint(n+=parseInt(u,36)),i.set(c,u),t&&a.set(u,c)}})),{map:i,reverseMap:a}}function v(){if(!h){var e=g(f,!0),t=e.map,r=e.reverseMap;h=t,d=r,p=g(m,!1).map}}function A(e){return v(),h.get(e)||null}function y(e){return v(),d.get(e)||null}function x(e){return v(),p.get(e)||null}var b=r.L,w=r.R,_=r.EN,E=r.ES,S=r.ET,M=r.AN,T=r.CS,C=r.B,I=r.S,B=r.ON,R=r.BN,P=r.NSM,F=r.AL,L=r.LRO,D=r.RLO,U=r.LRE,N=r.RLE,k=r.PDF,O=r.LRI,G=r.RLI,z=r.FSI,V=r.PDI;var H;function Q(e){return function(){if(!H){var e=g("14>1,j>2,t>2,u>2,1a>g,2v3>1,1>1,1ge>1,1wd>1,b>1,1j>1,f>1,ai>3,-2>3,+1,8>1k0,-1jq>1y7,-1y6>1hf,-1he>1h6,-1h5>1ha,-1h8>1qi,-1pu>1,6>3u,-3s>7,6>1,1>1,f>1,1>1,+2,3>1,1>1,+13,4>1,1>1,6>1eo,-1ee>1,3>1mg,-1me>1mk,-1mj>1mi,-1mg>1mi,-1md>1,1>1,+2,1>10k,-103>1,1>1,4>1,5>1,1>1,+10,3>1,1>8,-7>8,+1,-6>7,+1,a>1,1>1,u>1,u6>1,1>1,+5,26>1,1>1,2>1,2>2,8>1,7>1,4>1,1>1,+5,b8>1,1>1,+3,1>3,-2>1,2>1,1>1,+2,c>1,3>1,1>1,+2,h>1,3>1,a>1,1>1,2>1,3>1,1>1,d>1,f>1,3>1,1a>1,1>1,6>1,7>1,13>1,k>1,1>1,+19,4>1,1>1,+2,2>1,1>1,+18,m>1,a>1,1>1,lk>1,1>1,4>1,2>1,f>1,3>1,1>1,+3,db>1,1>1,+3,3>1,1>1,+2,14qm>1,1>1,+1,6>1,4j>1,j>2,t>2,u>2,2>1,+1",!0),t=e.map;e.reverseMap.forEach((function(e,r){t.set(r,e)})),H=t}}(),H.get(e)||null}function j(e,t,r,n){var i=e.length;r=Math.max(0,null==r?0:+r),n=Math.min(i-1,null==n?i-1:+n);var a=[];return t.paragraphs.forEach((function(i){var s=Math.max(r,i.start),o=Math.min(n,i.end);if(s=s&&u(e[h])&l;h--)c[h]=i.level;for(var d=i.level,p=1/0,f=0;fd&&(d=m),m=p;g--)for(var v=0;v=g){for(var A=v;v+1=g;)v++;v>A&&a.push([A+r,v+r])}}})),a}function W(e,t,r,n){for(var i=j(e,t,r,n),a=[],s=0;s0)Y--;else if(K>0){for(q=0;!W[W.length-1]._isolate;)W.pop();var re=W[W.length-1]._isolInitIndex;null!=re&&(p.set(re,J),p.set(J,re)),W.pop(),K--}X=W[W.length-1],d[J]=X._level,X._override&&h(J,X._override)}else Z&k?(0===Y&&(q>0?q--:!X._isolate&&W.length>1&&(W.pop(),X=W[W.length-1])),d[J]=X._level):Z&C&&(d[J]=m.level);else d[J]=X._level,X._override&&Z!==R&&h(J,X._override)}for(var ne=[],ie=null,ae=m.start;ae<=m.end;ae++){var se=r[ae];if(!(se&o)){var oe=d[ae],le=se&i,ce=se===V;ie&&oe===ie._level?(ie._end=ae,ie._endsWithIsolInit=le):ne.push(ie={_start:ae,_end:ae,_level:oe,_startsWithPDI:ce,_endsWithIsolInit:le})}}for(var ue=[],he=0;he=0;we--)if(!(r[we]&o)){be=d[we];break}var _e=ge[ge.length-1],Ee=d[_e],Se=m.level;if(!(r[_e]&i))for(var Me=_e+1;Me<=m.end;Me++)if(!(r[Me]&o)){Se=d[Me];break}ue.push({_seqIndices:ge,_sosType:Math.max(be,xe)%2?w:b,_eosType:Math.max(Se,Ee)%2?w:b})}}for(var Te=0;Te=0;De--)if(!(r[Ie[De]]&o)){Le=r[Ie[De]];break}h(Fe,Le&(i|V)?B:Le)}}if(c.get(_))for(var Ue=0;Ue=-1;ke--){var Oe=-1===ke?Be:r[Ie[ke]];if(Oe&a){Oe===F&&h(Ne,M);break}}}if(c.get(F))for(var Ge=0;Ge=0&&(Qe=r[Ie[We]])&o;We--);for(var Xe=Ve+1;Xe=0&&r[Ie[Ke]]&(S|o);Ke--)h(Ie[Ke],_);for(var Je=Ye+1;Je=0&&r[Ie[et]]&o;et--)h(Ie[et],B);for(var tt=Ze+1;tt=0;pt--){var ft=ct[pt].char;if(ft===dt||ft===y(x(ht))||A(x(ft))===ht){lt.push([ct[pt].seqIndex,ut]),ct.length=pt;break}}}lt.sort((function(e,t){return e[0]-t[0]}));for(var mt=0;mt=0;Et--){var St=Ie[Et];if(r[St]&ot){var Mt=r[St]&st?w:b;xt=Mt!==Ht(St)?Mt:Ht(St);break}}}if(xt){if(r[Ie[vt]]=r[Ie[At]]=xt,xt!==Ht(Ie[vt]))for(var Tt=vt+1;Tt=0;Ft--){if(!(r[Ie[Ft]]&o)){Pt=r[Ie[Ft]]&st?w:b;break}Bt=Ft}for(var Lt=Re,Dt=It+1;Dt=0&&u(e[Gt])&l;Gt--)d[Gt]=m.level}}return{levels:d,paragraphs:f};function zt(t,n){for(var a=t;a/gm,(function(e,t){let r=s.ShaderChunk[t];return r?Wi(r):e}))}const Xi=[];for(let vh=0;vh<256;vh++)Xi[vh]=(vh<16?"0":"")+vh.toString(16);const Yi=Object.assign||function(){let e=arguments[0];for(let t=1,r=arguments.length;t/gm,"\n//!BEGIN_POST_CHUNK $1\n$&\n//!END_POST_CHUNK\n")));if(d){let r=d({vertexShader:e,fragmentShader:t});e=r.vertexShader,t=r.fragmentShader}if(h){let e=[];t=t.replace(/^\/\/!BEGIN_POST_CHUNK[^]+?^\/\/!END_POST_CHUNK/gm,(t=>(e.push(t),""))),u=`${h}\n${e.join("\n")}\n${u}`}if(p){const e=`\nuniform float ${p};\n`;i=e+i,l=e+l}o&&(i=`${i}\nvoid troikaVertexTransform${n}(inout vec3 position, inout vec3 normal, inout vec2 uv) {\n ${o}\n}\n`,a=`\ntroika_position_${n} = vec3(position);\ntroika_normal_${n} = vec3(normal);\ntroika_uv_${n} = vec2(uv);\ntroikaVertexTransform${n}(troika_position_${n}, troika_normal_${n}, troika_uv_${n});\n${a}\n`,e=(e=`vec3 troika_position_${n};\nvec3 troika_normal_${n};\nvec2 troika_uv_${n};\n${e}\n`).replace(/\b(position|normal|uv)\b/g,((e,t,r,i)=>/\battribute\s+vec[23]\s+$/.test(i.substr(0,r))?t:`troika_${t}_${n}`)));return e=ea(e,n,i,a,s),t=ea(t,n,l,c,u),{vertexShader:e,fragmentShader:t}}(n,t,r);s=Ji[a]=e}n.vertexShader=s.vertexShader,n.fragmentShader=s.fragmentShader,Yi(n.uniforms,this.uniforms),t.timeUniform&&(n.uniforms[t.timeUniform]={get value(){return Date.now()-qi}}),this[i]&&this[i](n)},o=function(){return l(t.chained?e:e.clone())},l=function(n){const i=Object.create(n,c);return Object.defineProperty(i,"baseMaterial",{value:e}),Object.defineProperty(i,"id",{value:Zi++}),i.uuid=function(){const e=4294967295*Math.random()|0,t=4294967295*Math.random()|0,r=4294967295*Math.random()|0,n=4294967295*Math.random()|0;return(Xi[255&e]+Xi[e>>8&255]+Xi[e>>16&255]+Xi[e>>24&255]+"-"+Xi[255&t]+Xi[t>>8&255]+"-"+Xi[t>>16&15|64]+Xi[t>>24&255]+"-"+Xi[63&r|128]+Xi[r>>8&255]+"-"+Xi[r>>16&255]+Xi[r>>24&255]+Xi[255&n]+Xi[n>>8&255]+Xi[n>>16&255]+Xi[n>>24&255]).toUpperCase()}(),i.uniforms=Yi({},n.uniforms,t.uniforms),i.defines=Yi({},n.defines,t.defines),i.defines[`TROIKA_DERIVED_MATERIAL_${r}`]="",i.extensions=Yi({},n.extensions,t.extensions),i._listeners=void 0,i},c={constructor:{value:o},isDerivedMaterial:{value:!0},customProgramCacheKey:{writable:!0,configurable:!0,value:function(){return e.customProgramCacheKey()+"|"+r}},onBeforeCompile:{get:()=>a,set(e){this[i]=e}},copy:{writable:!0,configurable:!0,value:function(t){return e.copy.call(this,t),e.isShaderMaterial||e.isDerivedMaterial||(Yi(this.extensions,t.extensions),Yi(this.defines,t.defines),Yi(this.uniforms,s.UniformsUtils.clone(t.uniforms))),this}},clone:{writable:!0,configurable:!0,value:function(){const t=new e.constructor;return l(t).copy(this)}},getDepthMaterial:{writable:!0,configurable:!0,value:function(){let r=this._depthMaterial;return r||(r=this._depthMaterial=$i(e.isDerivedMaterial?e.getDepthMaterial():new s.MeshDepthMaterial({depthPacking:s.RGBADepthPacking}),t),r.defines.IS_DEPTH_MATERIAL="",r.uniforms=this.uniforms),r}},getDistanceMaterial:{writable:!0,configurable:!0,value:function(){let r=this._distanceMaterial;return r||(r=this._distanceMaterial=$i(e.isDerivedMaterial?e.getDistanceMaterial():new s.MeshDistanceMaterial,t),r.defines.IS_DISTANCE_MATERIAL="",r.uniforms=this.uniforms),r}},dispose:{writable:!0,configurable:!0,value(){const{_depthMaterial:t,_distanceMaterial:r}=this;t&&t.dispose(),r&&r.dispose(),e.dispose.call(this)}}};return n[r]=o,new o}function ea(e,t,r,n,i){return(n||i||r)&&(e=e.replace(ji,`\n${r}\nvoid troikaOrigMain${t}() {`),e+=`\nvoid main() {\n ${n}\n troikaOrigMain${t}();\n ${i}\n}`),e}function ta(e,t){return"uniforms"===e?void 0:"function"==typeof t?t.toString():t}let ra=0;const na=new Map;s.DoubleSide;const ia=()=>(self.performance||Date).now(),aa=Hi();let sa;const oa=function(){const e=[];let t=0;function r(){const n=ia();for(;e.length&&ia()-n<5;)e.shift()();t=e.length?setTimeout(r,0):0}return(...n)=>{const i=Bi();return e.push((()=>{const e=ia();try{aa.webgl.generateIntoCanvas(...n),i.resolve({timing:ia()-e})}catch(t){i.reject(t)}})),t||(t=setTimeout(r,0)),i}}(),la=function(){const e={};let t=0;return function(r,n,i,a,s,o,l,c,u,h){const d="TroikaTextSDFGenerator_JS_"+t++%4;let p=e[d];return p||(p=e[d]={workerModule:Oi({name:d,workerId:d,dependencies:[Hi,ia],init(e,t){const r=e().javascript.generate;return function(...e){const n=t();return{textureData:r(...e),timing:t()-n}}},getTransferables:e=>[e.textureData.buffer]}),requests:0,idleTimer:null}),p.requests++,clearTimeout(p.idleTimer),p.workerModule(r,n,i,a,s,o).then((({textureData:e,timing:t})=>{const i=ia(),a=new Uint8Array(4*e.length);for(let r=0;r{!function(e){Ni[e]&&Ni[e].forEach((function(e){e()})),Ui[e]&&(Ui[e].terminate(),delete Ui[e])}(d)}),2e3)),{timing:t}}))}}();const ca=aa.webglUtils.resizeWebGLCanvasWithoutClearing;const ua=Oi({name:"Typr Font Parser",dependencies:[function(){return"undefined"==typeof window&&(self.window=self),function(e){var t={parse:function(e){var r=t._bin,n=new Uint8Array(e);if("ttcf"==r.readASCII(n,0,4)){var i=4;r.readUshort(n,i),i+=2,r.readUshort(n,i),i+=2;var a=r.readUint(n,i);i+=4;for(var s=[],o=0;o>>r&1)&&t++;return t},t._lctf.readClassDef=function(e,r){var n=t._bin,i=[],a=n.readUshort(e,r);if(r+=2,1==a){var s=n.readUshort(e,r);r+=2;var o=n.readUshort(e,r);r+=2;for(var l=0;l0&&(a.featureParams=i+s);var o=n.readUshort(e,r);r+=2,a.tab=[];for(var l=0;l255?-1:t.CFF.glyphByUnicode(e,t.CFF.tableSE[r])},t.CFF.readEncoding=function(e,r,n){t._bin;var i=[".notdef"],a=e[r];if(r++,0!=a)throw"error: unknown encoding format: "+a;var s=e[r];r++;for(var o=0;o>4,m=15&p;if(15!=f&&d.push(f),15!=m&&d.push(m),15==m)break}for(var g="",v=[0,1,2,3,4,5,6,7,8,9,".","e","e-","reserved","-","endOfNumber"],A=0;A=s.xMax||s.yMin>=s.yMax)return null;if(s.noc>0){s.endPts=[];for(var o=0;o=1&&o.fmt<=2){c=a.readUshort(e,n),n+=2;var h=a.readUshort(e,n);n+=2,u=t._lctf.numOfOnes(c);var d=t._lctf.numOfOnes(h);if(1==o.fmt){o.pairsets=[];var p=a.readUshort(e,n);n+=2;for(var f=0;f=1&&o.fmt<=2){if(1==o.fmt)o.delta=a.readShort(e,n),n+=2;else if(2==o.fmt){var c=a.readUshort(e,n);n+=2,o.newg=a.readUshorts(e,n,c),n+=2*o.newg.length}}else if(4==r){o.vals=[],c=a.readUshort(e,n),n+=2;for(var u=0;u>>8;if(0!=(h&=15))throw"unknown kern table format: "+h;r=t.kern.readFormat0(e,r,l)}return l},t.kern.parseV1=function(e,r,n,i){var a=t._bin;a.readFixed(e,r),r+=4;var s=a.readUint(e,r);r+=4;for(var o={glyph1:[],rval:[]},l=0;l>>8;if(0!=(u&=15))throw"unknown kern table format: "+u;r=t.kern.readFormat0(e,r,o)}return o},t.kern.readFormat0=function(e,r,n){var i=t._bin,a=-1,s=i.readUshort(e,r);r+=2,i.readUshort(e,r),r+=2,i.readUshort(e,r),r+=2,i.readUshort(e,r),r+=2;for(var o=0;o=i.map.length?0:i.map[t];if(4==i.format){for(var a=-1,s=0;st?0:65535&(0!=i.idRangeOffset[a]?i.glyphIdArray[t-i.startCount[a]+(i.idRangeOffset[a]>>1)-(i.idRangeOffset.length-a)]:t+i.idDelta[a])}if(12==i.format){if(t>i.groups[i.groups.length-1][1])return 0;for(s=0;s-1?t.U._simpleGlyph(i,n):t.U._compoGlyph(i,r,n))},t.U._simpleGlyph=function(e,r){for(var n=0;na)){for(var p=!0,f=0,m=0;ma)){for(p=!0,m=0;m>1,s.length=0,l=!0;else if("o3"==S||"o23"==S)s.length%2!=0&&!l&&(c=s.shift()+i.nominalWidthX),o+=s.length>>1,s.length=0,l=!0;else if("o4"==S)s.length>1&&!l&&(c=s.shift()+i.nominalWidthX,l=!0),u&&t.U.P.closePath(a),p+=s.pop(),t.U.P.moveTo(a,d,p),u=!0;else if("o5"==S)for(;s.length>0;)d+=s.shift(),p+=s.shift(),t.U.P.lineTo(a,d,p);else if("o6"==S||"o7"==S)for(var M=s.length,T="o6"==S,C=0;CMath.abs(b-p)?d=x+s.shift():p=b+s.shift(),t.U.P.curveTo(a,f,m,g,v,w,_),t.U.P.curveTo(a,A,y,x,b,d,p));else if("o14"==S){if(s.length>0&&!l&&(c=s.shift()+n.nominalWidthX,l=!0),4==s.length){var R=s.shift(),P=s.shift(),F=s.shift(),L=s.shift(),D=t.CFF.glyphBySE(n,F),U=t.CFF.glyphBySE(n,L);t.U._drawCFF(n.CharStrings[D],r,n,i,a),r.x=R,r.y=P,t.U._drawCFF(n.CharStrings[U],r,n,i,a)}u&&(t.U.P.closePath(a),u=!1)}else if("o19"==S||"o20"==S)s.length%2!=0&&!l&&(c=s.shift()+i.nominalWidthX),o+=s.length>>1,s.length=0,l=!0,h+=o+7>>3;else if("o21"==S)s.length>2&&!l&&(c=s.shift()+i.nominalWidthX,l=!0),p+=s.pop(),d+=s.pop(),u&&t.U.P.closePath(a),t.U.P.moveTo(a,d,p),u=!0;else if("o22"==S)s.length>1&&!l&&(c=s.shift()+i.nominalWidthX,l=!0),d+=s.pop(),u&&t.U.P.closePath(a),t.U.P.moveTo(a,d,p),u=!0;else if("o25"==S){for(;s.length>6;)d+=s.shift(),p+=s.shift(),t.U.P.lineTo(a,d,p);f=d+s.shift(),m=p+s.shift(),g=f+s.shift(),v=m+s.shift(),d=g+s.shift(),p=v+s.shift(),t.U.P.curveTo(a,f,m,g,v,d,p)}else if("o26"==S)for(s.length%2&&(d+=s.shift());s.length>0;)f=d,m=p+s.shift(),d=g=f+s.shift(),p=(v=m+s.shift())+s.shift(),t.U.P.curveTo(a,f,m,g,v,d,p);else if("o27"==S)for(s.length%2&&(p+=s.shift());s.length>0;)m=p,g=(f=d+s.shift())+s.shift(),v=m+s.shift(),d=g+s.shift(),p=v,t.U.P.curveTo(a,f,m,g,v,d,p);else if("o10"==S||"o29"==S){var N="o10"==S?i:n;if(0==s.length)console.debug("error: empty stack");else{var k=s.pop(),O=N.Subrs[k+N.Bias];r.x=d,r.y=p,r.nStems=o,r.haveWidth=l,r.width=c,r.open=u,t.U._drawCFF(O,r,n,i,a),d=r.x,p=r.y,o=r.nStems,l=r.haveWidth,c=r.width,u=r.open}}else if("o30"==S||"o31"==S){var G=s.length,z=(B=0,"o31"==S);for(B+=G-(M=-3&G);B>>1|(21845&p)<<1;f=(61680&(f=(52428&f)>>>2|(13107&f)<<2))>>>4|(3855&f)<<4,d[p]=((65280&f)>>>8|(255&f)<<8)>>>1}var m=function(e,t,n){for(var i=e.length,a=0,s=new r(t);a>>c]=u}else for(o=new r(i),a=0;a>>15-e[a]);return o},g=new t(288);for(p=0;p<144;++p)g[p]=8;for(p=144;p<256;++p)g[p]=9;for(p=256;p<280;++p)g[p]=7;for(p=280;p<288;++p)g[p]=8;var v=new t(32);for(p=0;p<32;++p)v[p]=5;var A=m(g,9,1),y=m(v,5,1),x=function(e){for(var t=e[0],r=1;rt&&(t=e[r]);return t},b=function(e,t,r){var n=t/8|0;return(e[n]|e[n+1]<<8)>>(7&t)&r},w=function(e,t){var r=t/8|0;return(e[r]|e[r+1]<<8|e[r+2]<<16)>>(7&t)},_=["unexpected EOF","invalid block type","invalid length/literal","invalid distance","stream finished","no stream handler",,"no callback","invalid UTF-8 data","extra field too long","date not in range 1980-2099","filename too long","stream finishing","invalid zip data"],E=function(e,t,r){var n=new Error(t||_[e]);if(n.code=e,Error.captureStackTrace&&Error.captureStackTrace(n,E),!r)throw n;return n},S=function(e,o,l){var u=e.length;if(!u||l&&!l.l&&u<5)return o||new t(0);var d=!o||l,p=!l||l.i;l||(l={}),o||(o=new t(3*u));var f,g=function(e){var r=o.length;if(e>r){var n=new t(Math.max(2*r,e));n.set(o),o=n}},v=l.f||0,_=l.p||0,S=l.b||0,M=l.l,T=l.d,C=l.m,I=l.n,B=8*u;do{if(!M){l.f=v=b(e,_,1);var R=b(e,_+1,3);if(_+=3,!R){var P=e[(H=((f=_)/8|0)+(7&f&&1)+4)-4]|e[H-3]<<8,F=H+P;if(F>u){p&&E(0);break}d&&g(S+P),o.set(e.subarray(H,F),S),l.b=S+=P,l.p=_=8*F;continue}if(1==R)M=A,T=y,C=9,I=5;else if(2==R){var L=b(e,_,31)+257,D=b(e,_+10,15)+4,U=L+b(e,_+5,31)+1;_+=14;for(var N=new t(U),k=new t(19),O=0;O>>4)<16)N[O++]=H;else{var j=0,W=0;for(16==H?(W=3+b(e,_,3),_+=2,j=N[O-1]):17==H?(W=3+b(e,_,7),_+=3):18==H&&(W=11+b(e,_,127),_+=7);W--;)N[O++]=j}}var X=N.subarray(0,L),Y=N.subarray(L);C=x(X),I=x(Y),M=m(X,C,1),T=m(Y,I,1)}else E(1);if(_>B){p&&E(0);break}}d&&g(S+131072);for(var q=(1<>>4;if((_+=15&j)>B){p&&E(0);break}if(j||E(2),Z<256)o[S++]=Z;else{if(256==Z){J=_,M=null;break}var $=Z-254;if(Z>264){var ee=i[O=Z-257];$=b(e,_,(1<>>4;if(te||E(3),_+=15&te,Y=h[re],re>3&&(ee=a[re],Y+=w(e,_)&(1<B){p&&E(0);break}d&&g(S+131072);for(var ne=S+$;Se.length)&&(a=e.length);var s=new(e instanceof r?r:e instanceof n?n:t)(a-i);return s.set(e.subarray(i,a)),s}(o,0,S)},M=new t(0),T="undefined"!=typeof TextDecoder&&new TextDecoder;try{T.decode(M,{stream:!0})}catch(e){}return e.convert_streams=function(e){var t=new DataView(e),r=0;function n(){var e=t.getUint16(r);return r+=2,e}function i(){var e=t.getUint32(r);return r+=4,e}function a(e){v.setUint16(A,e),A+=2}function s(e){v.setUint32(A,e),A+=4}for(var o={signature:i(),flavor:i(),length:i(),numTables:n(),reserved:n(),totalSfntSize:i(),majorVersion:n(),minorVersion:n(),metaOffset:i(),metaLength:i(),metaOrigLength:i(),privOffset:i(),privLength:i()},l=0;Math.pow(2,l)<=o.numTables;)l++;l--;for(var c=16*Math.pow(2,l),u=16*o.numTables-c,h=12,d=[],p=0;p{let[a,s]=n.split("+");a=parseInt(a,36),s=s?parseInt(s,36):0,i.set(r+=a,e[t]);for(let o=s;o--;)i.set(++r,e[t])}))}}return i.get(e)||32}const s=[null,"isol","init","fina","medi"];function o(e){const t=new Uint8Array(e.length);let r=32,n=1,i=-1;for(let s=0;s65535&&s++)}return t}function l(t){const n=Object.create(null),i={unitsPerEm:t.head.unitsPerEm,ascender:t.hhea.ascender,descender:t.hhea.descender,forEachGlyph(a,l,c,u){let h=0;const d=1/i.unitsPerEm*l,p=function(t,r){const n=[];for(let a=0;a65535&&a++,n.push(e.U.codeToGlyph(t,i))}const i=t.GSUB;if(i){const{lookupList:t,featureList:a}=i;let l;const c=/^(rlig|liga|mset|isol|init|fina|medi|half|pres|blws)$/,u=[];a.forEach((i=>{if(c.test(i.tag))for(let a=0;a{if(-1!==i){let a=n[i];if(!a){const{cmds:s,crds:o}=e.U.glyphToPath(t,i);let l,c,u,h,d="",p=0;for(let e=0,t=s.length;e1?",":"")+o[p++]}if(o.length){l=c=1/0,u=h=-1/0;for(let e=0,t=o.length;eu&&(u=t),r>h&&(h=r)}}else l=u=c=h=0;a=n[i]={index:i,advanceWidth:t.hmtx.aWidth[i],xMin:l,yMin:c,xMax:u,yMax:h,path:d,pathCommandCount:s.length}}-1!==m&&(h+=e.U.getPairAdjustment(t,m,i)*d),u.call(null,a,h,f),a.advanceWidth&&(h+=a.advanceWidth*d),c&&(h+=c*l),m=i}f+=a.codePointAt(f)>65535?2:1})),h}};return i}return function(r){const n=new Uint8Array(r,0,4),i=e._bin.readASCII(n,0,4);if("wOFF"===i)r=t(r);else if("wOF2"===i)throw new Error("woff2 fonts not supported");return l(e.parse(r)[0])}}],init:(e,t,r)=>r(e(),t())}),ha={defaultFontURL:"https://fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxM.woff",sdfGlyphSize:64,sdfMargin:1/16,sdfExponent:9,textureWidth:2048},da=new s.Color;let pa=!1;function fa(){return(self.performance||Date).now()}const ma=Object.create(null);function ga(e,t){pa=!0,e=Aa({},e);const r=fa();if(e.font=function(e){ya||(ya="undefined"==typeof document?{}:document.createElement("a"));return ya.href=e,ya.href}(e.font||ha.defaultFontURL),e.text=""+e.text,e.sdfGlyphSize=e.sdfGlyphSize||ha.sdfGlyphSize,null!=e.colorRanges){let t={};for(let r in e.colorRanges)if(e.colorRanges.hasOwnProperty(r)){let n=e.colorRanges[r];"number"!=typeof n&&(n=da.set(n).getHex()),t[r]=n}e.colorRanges=t}Object.freeze(e);const{textureWidth:n,sdfExponent:i}=ha,{sdfGlyphSize:a}=e,o=n/a*4;let l=ma[a];if(!l){const e=document.createElement("canvas");e.width=n,e.height=256*a/o,l=ma[a]={glyphCount:0,sdfGlyphSize:a,sdfCanvas:e,sdfTexture:new s.Texture(e,void 0,void 0,void 0,s.LinearFilter,s.LinearFilter),contextLost:!1,glyphsByFont:new Map},l.sdfTexture.generateMipmaps=!1,function(e){const t=e.sdfCanvas;t.addEventListener("webglcontextlost",(t=>{console.log("Context Lost",t),t.preventDefault(),e.contextLost=!0})),t.addEventListener("webglcontextrestored",(t=>{console.log("Context Restored",t),e.contextLost=!1;const r=[];e.glyphsByFont.forEach((t=>{t.forEach((t=>{r.push(va(t,e,!0))}))})),Bi.all(r).then((()=>{xa(e),e.sdfTexture.needsUpdate=!0}))}))}(l)}const{sdfTexture:c,sdfCanvas:u}=l;let h=l.glyphsByFont.get(e.font);h||l.glyphsByFont.set(e.font,h=new Map),wa(e).then((s=>{const{glyphIds:d,glyphPositions:p,fontSize:f,unitsPerEm:m,timings:g}=s,v=[],A=new Float32Array(4*d.length),y=f/m;let x=0,b=0;const w=fa();d.forEach(((e,t)=>{let r=h.get(e);if(!r){const{path:t,pathBounds:n}=s.glyphData[e],i=Math.max(n[2]-n[0],n[3]-n[1])/a*(ha.sdfMargin*a+.5),o=l.glyphCount++,c=[n[0]-i,n[1]-i,n[2]+i,n[3]+i];h.set(e,r={path:t,atlasIndex:o,sdfViewBox:c}),v.push(r)}const{sdfViewBox:n}=r,i=p[b++],o=p[b++];A[x++]=i+n[0]*y,A[x++]=o+n[1]*y,A[x++]=i+n[2]*y,A[x++]=o+n[3]*y,d[t]=r.atlasIndex})),g.quads=(g.quads||0)+(fa()-w);const _=fa();g.sdf={};const E=u.height,S=Math.ceil(l.glyphCount/o),M=Math.pow(2,Math.ceil(Math.log2(S*a)));M>E&&(console.info(`Increasing SDF texture size ${E}->${M}`),ca(u,n,M),c.dispose()),Bi.all(v.map((t=>va(t,l,e.gpuAccelerateSDF).then((({timing:e})=>{g.sdf[t.atlasIndex]=e}))))).then((()=>{v.length&&!l.contextLost&&(xa(l),c.needsUpdate=!0),g.sdfTotal=fa()-_,g.total=fa()-r,t(Object.freeze({parameters:e,sdfTexture:c,sdfGlyphSize:a,sdfExponent:i,glyphBounds:A,glyphAtlasIndices:d,glyphColors:s.glyphColors,caretPositions:s.caretPositions,caretHeight:s.caretHeight,chunkedBounds:s.chunkedBounds,ascender:s.ascender,descender:s.descender,lineHeight:s.lineHeight,topBaseline:s.topBaseline,blockBounds:s.blockBounds,visibleBounds:s.visibleBounds,timings:s.timings,get totalBounds(){return console.log("totalBounds deprecated, use blockBounds instead"),s.blockBounds},get totalBlockSize(){console.log("totalBlockSize deprecated, use blockBounds instead");const[e,t,r,n]=s.blockBounds;return[r-e,n-t]}}))}))})),Bi.all([]).then((()=>{var e;l.contextLost||(e=u)._warm||(aa.webgl.isSupported(e),e._warm=!0)}))}function va({path:e,atlasIndex:t,sdfViewBox:r},{sdfGlyphSize:n,sdfCanvas:i,contextLost:a},s){if(a)return Promise.resolve({timing:-1});const{textureWidth:o,sdfExponent:l}=ha,c=Math.max(r[2]-r[0],r[3]-r[1]),u=Math.floor(t/4);return function(e,t,r,n,i,a,s,o,l,c,u=!0){return u?oa(e,t,r,n,i,a,s,o,l,c).then(null,(u=>(sa||(console.warn("WebGL SDF generation failed, falling back to JS",u),sa=!0),la(e,t,r,n,i,a,s,o,l,c)))):la(e,t,r,n,i,a,s,o,l,c)}(n,n,e,r,c,l,i,u%(o/n)*n,Math.floor(u/(o/n))*n,t%4,s)}function Aa(e,t){for(let r in t)t.hasOwnProperty(r)&&(e[r]=t[r]);return e}let ya;function xa(e){if("function"!=typeof createImageBitmap){console.info("Safari<15: applying SDF canvas workaround");const{sdfCanvas:t,sdfTexture:r}=e,{width:n,height:i}=t,a=e.sdfCanvas.getContext("webgl");let s=r.image.data;s&&s.length===n*i*4||(s=new Uint8Array(n*i*4),r.image={width:n,height:i,data:s},r.flipY=!1,r.isDataTexture=!0),a.readPixels(0,0,n,i,a.RGBA,a.UNSIGNED_BYTE,s)}}const ba=Oi({name:"Typesetter",dependencies:[ha,ua,function(e,t,r){const{defaultFontURL:n}=r,i=Object.create(null),a=1/0,s=/[\u00AD\u034F\u061C\u115F-\u1160\u17B4-\u17B5\u180B-\u180E\u200B-\u200F\u202A-\u202E\u2060-\u206F\u3164\uFE00-\uFE0F\uFEFF\uFFA0\uFFF0-\uFFF8]/,o=/[\s\-\u007C\u00AD\u2010\u2012-\u2014\u2027\u2056\u2E17\u2E40]/;function l(t,r){t||(t=n);let a=i[t];a?a.pending?a.pending.push(r):r(a):(i[t]={pending:[r]},function(t,r){!function i(){const a=e=>{console.error(`Failure loading font ${t}${t===n?"":"; trying fallback"}`,e),t!==n&&(t=n,i())};try{const n=new XMLHttpRequest;n.open("get",t,!0),n.responseType="arraybuffer",n.onload=function(){if(n.status>=400)a(new Error(n.statusText));else if(n.status>0)try{const t=e(n.response);r(t)}catch(Ha){a(Ha)}},n.onerror=a,n.send()}catch(s){a(s)}}()}(t,(e=>{let r=i[t].pending;i[t]=e,r.forEach((t=>t(e)))})))}function c({text:e="",font:r=n,sdfGlyphSize:i=64,fontSize:c=1,letterSpacing:f=0,lineHeight:m="normal",maxWidth:g=a,direction:v,textAlign:A="left",textIndent:y=0,whiteSpace:x="normal",overflowWrap:b="normal",anchorX:w=0,anchorY:_=0,includeCaretPositions:E=!1,chunkedBoundsSize:S=8192,colorRanges:M=null},T,C=!1){const I=d(),B={fontLoad:0,typesetting:0};e.indexOf("\r")>-1&&(console.info("Typesetter: got text with \\r chars; normalizing to \\n"),e=e.replace(/\r\n/g,"\n").replace(/\r/g,"\n")),c=+c,f=+f,g=+g,m=m||"normal",y=+y,l(r,(r=>{const n=isFinite(g);let i=null,l=null,R=null,P=null,F=null,L=null,D=null,U=0,N=0,k="nowrap"!==x;const{ascender:O,descender:G,unitsPerEm:z}=r;B.fontLoad=d()-I;const V=d(),H=c/z;"normal"===m&&(m=(O-G)/z);const Q=((m*=c)-(O-G)*H)/2,j=-(O*H+Q),W=Math.min(m,(O-G)*H),X=(O+G)/2*H-W/2;let Y=y,q=new p;const K=[q];r.forEachGlyph(e,c,f,((t,r,i)=>{const a=e.charAt(i),l=t.advanceWidth*H,u=q.count;let h;if("isEmpty"in t||(t.isWhitespace=!!a&&/\s/.test(a),t.canBreakAfter=!!a&&o.test(a),t.isEmpty=t.xMin===t.xMax||t.yMin===t.yMax||s.test(a)),t.isWhitespace||t.isEmpty||N++,k&&n&&!t.isWhitespace&&r+l+Y>g&&u){if(q.glyphAt(u-1).glyphObj.canBreakAfter)h=new p,Y=-r;else for(let e=u;e--;){if(0===e&&"break-word"===b){h=new p,Y=-r;break}if(q.glyphAt(e).glyphObj.canBreakAfter){h=q.splitAt(e+1);const t=h.glyphAt(0).x;Y-=t;for(let e=h.count;e--;)h.glyphAt(e).x-=t;break}}h&&(q.isSoftWrapped=!0,q=h,K.push(q),U=g)}let d=q.glyphAt(q.count);d.glyphObj=t,d.x=r+Y,d.width=l,d.charIndex=i,"\n"===a&&(q=new p,K.push(q),Y=-(r+l+f*c)+y)})),K.forEach((e=>{for(let t=e.count;t--;){let{glyphObj:r,x:n,width:i}=e.glyphAt(t);if(!r.isWhitespace)return e.width=n+i,void(e.width>U&&(U=e.width))}}));let J=0,Z=0;if(w&&("number"==typeof w?J=-w:"string"==typeof w&&(J=-U*("left"===w?0:"center"===w?.5:"right"===w?1:u(w)))),_)if("number"==typeof _)Z=-_;else if("string"==typeof _){let e=K.length*m;Z="top"===_?0:"top-baseline"===_?-j:"middle"===_?e/2:"bottom"===_?e:"bottom-baseline"===_?e-Q+G*H:u(_)*e}if(!C){const n=t.getEmbeddingLevels(e,v);i=new Uint16Array(N),l=new Float32Array(2*N),R={},L=[a,a,-1/0,-1/0],D=[];let s=j;E&&(F=new Float32Array(3*e.length)),M&&(P=new Uint8Array(3*N));let o,c,u=0,d=-1,p=-1;if(K.forEach(((f,g)=>{let{count:v,width:y}=f;if(v>0){let m=0;for(let e=v;e--&&f.glyphAt(e).glyphObj.isWhitespace;)m++;let g=0,x=0;if("center"===A)g=(U-y)/2;else if("right"===A)g=U-y;else if("justify"===A&&f.isSoftWrapped){let e=0;for(let t=v-m;t--;)f.glyphAt(t).glyphObj.isWhitespace&&e++;x=(U-y)/e}if(x||g){let e=0;for(let t=0;t=t){let t=e,a=e;for(;ar)break;aw=e;for(let A=0;A1&&h(F,d,n),d=e}if(M){const{charIndex:e}=m;for(;e>p;)p++,M.hasOwnProperty(p)&&(c=M[p])}if(!w.isWhitespace&&!w.isEmpty){const e=u++;R[g]||(R[g]={path:w.path,pathBounds:[w.xMin,w.yMin,w.xMax,w.yMax]});const t=m.x+J,r=s+Z;l[2*e]=t,l[2*e+1]=r;const n=t+w.xMin*H,h=r+w.yMin*H,d=t+w.xMax*H,p=r+w.yMax*H;nL[2]&&(L[2]=d),p>L[3]&&(L[3]=p),e%S==0&&(o={start:e,end:e,rect:[a,a,-1/0,-1/0]},D.push(o)),o.end++;const f=o.rect;if(nf[2]&&(f[2]=d),p>f[3]&&(f[3]=p),i[e]=g,M){const t=3*e;P[t]=c>>16&255,P[t+1]=c>>8&255,P[t+2]=255&c}}}}s-=m})),F){const t=e.length-d;t>1&&h(F,d,t)}}B.typesetting=d()-V,T({glyphIds:i,glyphPositions:l,glyphData:R,caretPositions:F,caretHeight:W,glyphColors:P,chunkedBounds:D,fontSize:c,unitsPerEm:z,ascender:O*H,descender:G*H,lineHeight:m,topBaseline:j,blockBounds:[J,Z-K.length*m,J+U,Z],visibleBounds:L,timings:B})}))}function u(e){let t=e.match(/^([\d.]+)%$/),r=t?parseFloat(t[1]):NaN;return isNaN(r)?0:r/100}function h(e,t,r){const n=e[3*t],i=e[3*t+1],a=e[3*t+2],s=(i-n)/r;for(let o=0;o(Object.defineProperty(e,t,{get(){return this.data[this.index*f.length+r]},set(e){this.data[this.index*f.length+r]=e}}),e)),{data:null,index:0}),{typeset:c,measure:function(e,t){c(e,(e=>{const[r,n,i,a]=e.blockBounds;t({width:i-r,height:a-n})}),{metricsOnly:!0})},loadFont:l}},Qi],init(e,t,r,n){const{defaultFontURL:i}=e;return r(t,n(),{defaultFontURL:i})}}),wa=Oi({name:"Typesetter",dependencies:[ba,Vi],init:(e,t)=>function(r){const n=new t;return e.typeset(r,n.resolve),n},getTransferables(e){const t=[e.glyphPositions.buffer,e.glyphIds.buffer];return e.caretPositions&&t.push(e.caretPositions.buffer),e.glyphColors&&t.push(e.glyphColors.buffer),t}});const _a=(()=>{const e={};const t="aTroikaGlyphIndex";class r extends s.InstancedBufferGeometry{constructor(){super(),this.detail=1,this.curveRadius=0,this.groups=[{start:0,count:1/0,materialIndex:0},{start:0,count:1/0,materialIndex:1}],this.boundingSphere=new s.Sphere,this.boundingBox=new s.Box3}computeBoundingSphere(){}computeBoundingBox(){}setSide(e){const t=this.getIndex().count;this.setDrawRange(e===s.BackSide?t/2:0,e===s.DoubleSide?t:t/2)}set detail(t){if(t!==this._detail){this._detail=t,("number"!=typeof t||t<1)&&(t=1);let r=function(t){let r=e[t];if(!r){const n=new s.PlaneBufferGeometry(1,1,t,t),i=n.clone(),a=n.attributes,o=i.attributes,l=new s.BufferGeometry,c=a.uv.count;for(let e=0;e{l.setAttribute(e,new s.Float32BufferAttribute([...a[e].array,...o[e].array],a[e].itemSize))})),l.setIndex([...n.index.array,...i.index.array.map((e=>e+c))]),l.translate(.5,.5,0),r=e[t]=l}return r}(t);["position","normal","uv"].forEach((e=>{this.attributes[e]=r.attributes[e].clone()})),this.setIndex(r.getIndex().clone())}}get detail(){return this._detail}set curveRadius(e){e!==this._curveRadius&&(this._curveRadius=e,this._updateBounds())}get curveRadius(){return this._curveRadius}updateGlyphs(e,r,a,s,o){n(this,"aTroikaGlyphBounds",e,4),n(this,t,r,1),n(this,"aTroikaGlyphColor",o,3),this._blockBounds=a,this._chunkedBounds=s,i(this,r.length),this._updateBounds()}_updateBounds(){const e=this._blockBounds;if(e){const{curveRadius:t,boundingBox:r}=this;if(t){const{PI:n,floor:i,min:a,max:s,sin:o,cos:l}=Math,c=n/2,u=2*n,h=Math.abs(t),d=e[0]/h,p=e[2]/h,f=i((d+c)/u)!==i((p+c)/u)?-h:a(o(d)*h,o(p)*h),m=i((d-c)/u)!==i((p-c)/u)?h:s(o(d)*h,o(p)*h),g=i((d+n)/u)!==i((p+n)/u)?2*h:s(h-l(d)*h,h-l(p)*h);r.min.set(f,e[1],t<0?-g:0),r.max.set(m,e[3],t<0?0:g)}else r.min.set(e[0],e[1],0),r.max.set(e[2],e[3],0);r.getBoundingSphere(this.boundingSphere)}}applyClipRect(e){let r=this.getAttribute(t).count,n=this._chunkedBounds;if(n)for(let t=n.length;t--;){r=n[t].end;let i=n[t].rect;if(i[1]e.y&&i[0]e.x)break}i(this,r)}}function n(e,t,r,n){const i=e.getAttribute(t);r?i&&i.array.length===r.length?(i.array.set(r),i.needsUpdate=!0):(e.setAttribute(t,new s.InstancedBufferAttribute(r,n)),delete e._maxInstanceCount,e.dispose()):i&&e.deleteAttribute(t)}function i(e,t){e[e.hasOwnProperty("instanceCount")?"instanceCount":"maxInstancedCount"]=t}return r.prototype.setAttribute||(r.prototype.setAttribute=function(e,t){return this.attributes[e]=t,this}),r})();const Ea=(()=>{const e=new s.MeshBasicMaterial({color:16777215,side:s.DoubleSide,transparent:!0}),t=8421504,r=new s.Matrix4,n=new s.Vector3,i=new s.Vector3,a=[],o=new s.Vector3,l="+x+y";function c(e){return Array.isArray(e)?e[0]:e}let u=()=>{const t=new s.Mesh(new s.PlaneBufferGeometry(1,1),e);return u=()=>t,t},h=()=>{const t=new s.Mesh(new s.PlaneBufferGeometry(1,1,32,1),e);return h=()=>t,t};const d={type:"syncstart"},p={type:"synccomplete"},f=["font","fontSize","letterSpacing","lineHeight","maxWidth","overflowWrap","text","direction","textAlign","textIndent","whiteSpace","anchorX","anchorY","colorRanges","sdfGlyphSize"],m=f.concat("material","color","depthOffset","clipRect","curveRadius","orientation","glyphGeometryDetail");class g extends s.Mesh{constructor(){super(new _a,null),this.text="",this.anchorX=0,this.anchorY=0,this.curveRadius=0,this.direction="auto",this.font=null,this.fontSize=.1,this.letterSpacing=0,this.lineHeight="normal",this.maxWidth=1/0,this.overflowWrap="normal",this.textAlign="left",this.textIndent=0,this.whiteSpace="normal",this.material=null,this.color=null,this.colorRanges=null,this.outlineWidth=0,this.outlineColor=0,this.outlineOpacity=1,this.outlineBlur=0,this.outlineOffsetX=0,this.outlineOffsetY=0,this.strokeWidth=0,this.strokeColor=t,this.strokeOpacity=1,this.fillOpacity=1,this.depthOffset=0,this.clipRect=null,this.orientation=l,this.glyphGeometryDetail=1,this.sdfGlyphSize=null,this.gpuAccelerateSDF=!0,this.debugSDF=!1}sync(e){this._needsSync&&(this._needsSync=!1,this._isSyncing?(this._queuedSyncs||(this._queuedSyncs=[])).push(e):(this._isSyncing=!0,this.dispatchEvent(d),ga({text:this.text,font:this.font,fontSize:this.fontSize||.1,letterSpacing:this.letterSpacing||0,lineHeight:this.lineHeight||"normal",maxWidth:this.maxWidth,direction:this.direction||"auto",textAlign:this.textAlign,textIndent:this.textIndent,whiteSpace:this.whiteSpace,overflowWrap:this.overflowWrap,anchorX:this.anchorX,anchorY:this.anchorY,colorRanges:this.colorRanges,includeCaretPositions:!0,sdfGlyphSize:this.sdfGlyphSize,gpuAccelerateSDF:this.gpuAccelerateSDF},(t=>{this._isSyncing=!1,this._textRenderInfo=t,this.geometry.updateGlyphs(t.glyphBounds,t.glyphAtlasIndices,t.blockBounds,t.chunkedBounds,t.glyphColors);const r=this._queuedSyncs;r&&(this._queuedSyncs=null,this._needsSync=!0,this.sync((()=>{r.forEach((e=>e&&e()))}))),this.dispatchEvent(p),e&&e()}))))}onBeforeRender(e,t,r,n,i,a){this.sync(),i.isTroikaTextMaterial&&this._prepareForRender(i),i._hadOwnSide=i.hasOwnProperty("side"),this.geometry.setSide(i._actualSide=i.side),i.side=s.FrontSide}onAfterRender(e,t,r,n,i,a){i._hadOwnSide?i.side=i._actualSide:delete i.side}dispose(){this.geometry.dispose()}get textRenderInfo(){return this._textRenderInfo||null}get material(){let t=this._derivedMaterial;const r=this._baseMaterial||this._defaultMaterial||(this._defaultMaterial=e.clone());if(t&&t.baseMaterial===r||(t=this._derivedMaterial=function(e){const t=$i(e,{chained:!0,extensions:{derivatives:!0},uniforms:{uTroikaSDFTexture:{value:null},uTroikaSDFTextureSize:{value:new s.Vector2},uTroikaSDFGlyphSize:{value:0},uTroikaSDFExponent:{value:0},uTroikaTotalBounds:{value:new s.Vector4(0,0,0,0)},uTroikaClipRect:{value:new s.Vector4(0,0,0,0)},uTroikaDistanceOffset:{value:0},uTroikaOutlineOpacity:{value:0},uTroikaFillOpacity:{value:1},uTroikaPositionOffset:{value:new s.Vector2},uTroikaCurveRadius:{value:0},uTroikaBlurRadius:{value:0},uTroikaStrokeWidth:{value:0},uTroikaStrokeColor:{value:new s.Color},uTroikaStrokeOpacity:{value:1},uTroikaOrient:{value:new s.Matrix3},uTroikaUseGlyphColors:{value:!0},uTroikaSDFDebug:{value:!1}},vertexDefs:"\nuniform vec2 uTroikaSDFTextureSize;\nuniform float uTroikaSDFGlyphSize;\nuniform vec4 uTroikaTotalBounds;\nuniform vec4 uTroikaClipRect;\nuniform mat3 uTroikaOrient;\nuniform bool uTroikaUseGlyphColors;\nuniform float uTroikaDistanceOffset;\nuniform float uTroikaBlurRadius;\nuniform vec2 uTroikaPositionOffset;\nuniform float uTroikaCurveRadius;\nattribute vec4 aTroikaGlyphBounds;\nattribute float aTroikaGlyphIndex;\nattribute vec3 aTroikaGlyphColor;\nvarying vec2 vTroikaGlyphUV;\nvarying vec4 vTroikaTextureUVBounds;\nvarying float vTroikaTextureChannel;\nvarying vec3 vTroikaGlyphColor;\nvarying vec2 vTroikaGlyphDimensions;\n",vertexTransform:"\nvec4 bounds = aTroikaGlyphBounds;\nbounds.xz += uTroikaPositionOffset.x;\nbounds.yw -= uTroikaPositionOffset.y;\n\nvec4 outlineBounds = vec4(\n bounds.xy - uTroikaDistanceOffset - uTroikaBlurRadius,\n bounds.zw + uTroikaDistanceOffset + uTroikaBlurRadius\n);\nvec4 clippedBounds = vec4(\n clamp(outlineBounds.xy, uTroikaClipRect.xy, uTroikaClipRect.zw),\n clamp(outlineBounds.zw, uTroikaClipRect.xy, uTroikaClipRect.zw)\n);\n\nvec2 clippedXY = (mix(clippedBounds.xy, clippedBounds.zw, position.xy) - bounds.xy) / (bounds.zw - bounds.xy);\n\nposition.xy = mix(bounds.xy, bounds.zw, clippedXY);\n\nuv = (position.xy - uTroikaTotalBounds.xy) / (uTroikaTotalBounds.zw - uTroikaTotalBounds.xy);\n\nfloat rad = uTroikaCurveRadius;\nif (rad != 0.0) {\n float angle = position.x / rad;\n position.xz = vec2(sin(angle) * rad, rad - cos(angle) * rad);\n normal.xz = vec2(sin(angle), cos(angle));\n}\n \nposition = uTroikaOrient * position;\nnormal = uTroikaOrient * normal;\n\nvTroikaGlyphUV = clippedXY.xy;\nvTroikaGlyphDimensions = vec2(bounds[2] - bounds[0], bounds[3] - bounds[1]);\n\n\nfloat txCols = uTroikaSDFTextureSize.x / uTroikaSDFGlyphSize;\nvec2 txUvPerSquare = uTroikaSDFGlyphSize / uTroikaSDFTextureSize;\nvec2 txStartUV = txUvPerSquare * vec2(\n mod(floor(aTroikaGlyphIndex / 4.0), txCols),\n floor(floor(aTroikaGlyphIndex / 4.0) / txCols)\n);\nvTroikaTextureUVBounds = vec4(txStartUV, vec2(txStartUV) + txUvPerSquare);\nvTroikaTextureChannel = mod(aTroikaGlyphIndex, 4.0);\n",fragmentDefs:"\nuniform sampler2D uTroikaSDFTexture;\nuniform vec2 uTroikaSDFTextureSize;\nuniform float uTroikaSDFGlyphSize;\nuniform float uTroikaSDFExponent;\nuniform float uTroikaDistanceOffset;\nuniform float uTroikaFillOpacity;\nuniform float uTroikaOutlineOpacity;\nuniform float uTroikaBlurRadius;\nuniform vec3 uTroikaStrokeColor;\nuniform float uTroikaStrokeWidth;\nuniform float uTroikaStrokeOpacity;\nuniform bool uTroikaSDFDebug;\nvarying vec2 vTroikaGlyphUV;\nvarying vec4 vTroikaTextureUVBounds;\nvarying float vTroikaTextureChannel;\nvarying vec2 vTroikaGlyphDimensions;\n\nfloat troikaSdfValueToSignedDistance(float alpha) {\n // Inverse of exponential encoding in webgl-sdf-generator\n \n float maxDimension = max(vTroikaGlyphDimensions.x, vTroikaGlyphDimensions.y);\n float absDist = (1.0 - pow(2.0 * (alpha > 0.5 ? 1.0 - alpha : alpha), 1.0 / uTroikaSDFExponent)) * maxDimension;\n float signedDist = absDist * (alpha > 0.5 ? -1.0 : 1.0);\n return signedDist;\n}\n\nfloat troikaGlyphUvToSdfValue(vec2 glyphUV) {\n vec2 textureUV = mix(vTroikaTextureUVBounds.xy, vTroikaTextureUVBounds.zw, glyphUV);\n vec4 rgba = texture2D(uTroikaSDFTexture, textureUV);\n float ch = floor(vTroikaTextureChannel + 0.5); //NOTE: can't use round() in WebGL1\n return ch == 0.0 ? rgba.r : ch == 1.0 ? rgba.g : ch == 2.0 ? rgba.b : rgba.a;\n}\n\nfloat troikaGlyphUvToDistance(vec2 uv) {\n return troikaSdfValueToSignedDistance(troikaGlyphUvToSdfValue(uv));\n}\n\nfloat troikaGetAADist() {\n \n #if defined(GL_OES_standard_derivatives) || __VERSION__ >= 300\n return length(fwidth(vTroikaGlyphUV * vTroikaGlyphDimensions)) * 0.5;\n #else\n return vTroikaGlyphDimensions.x / 64.0;\n #endif\n}\n\nfloat troikaGetFragDistValue() {\n vec2 clampedGlyphUV = clamp(vTroikaGlyphUV, 0.5 / uTroikaSDFGlyphSize, 1.0 - 0.5 / uTroikaSDFGlyphSize);\n float distance = troikaGlyphUvToDistance(clampedGlyphUV);\n \n // Extrapolate distance when outside bounds:\n distance += clampedGlyphUV == vTroikaGlyphUV ? 0.0 : \n length((vTroikaGlyphUV - clampedGlyphUV) * vTroikaGlyphDimensions);\n\n \n\n return distance;\n}\n\nfloat troikaGetEdgeAlpha(float distance, float distanceOffset, float aaDist) {\n #if defined(IS_DEPTH_MATERIAL) || defined(IS_DISTANCE_MATERIAL)\n float alpha = step(-distanceOffset, -distance);\n #else\n\n float alpha = smoothstep(\n distanceOffset + aaDist,\n distanceOffset - aaDist,\n distance\n );\n #endif\n\n return alpha;\n}\n",fragmentColorTransform:"\nfloat aaDist = troikaGetAADist();\nfloat fragDistance = troikaGetFragDistValue();\nfloat edgeAlpha = uTroikaSDFDebug ?\n troikaGlyphUvToSdfValue(vTroikaGlyphUV) :\n troikaGetEdgeAlpha(fragDistance, uTroikaDistanceOffset, max(aaDist, uTroikaBlurRadius));\n\n#if !defined(IS_DEPTH_MATERIAL) && !defined(IS_DISTANCE_MATERIAL)\nvec4 fillRGBA = gl_FragColor;\nfillRGBA.a *= uTroikaFillOpacity;\nvec4 strokeRGBA = uTroikaStrokeWidth == 0.0 ? fillRGBA : vec4(uTroikaStrokeColor, uTroikaStrokeOpacity);\nif (fillRGBA.a == 0.0) fillRGBA.rgb = strokeRGBA.rgb;\ngl_FragColor = mix(fillRGBA, strokeRGBA, smoothstep(\n -uTroikaStrokeWidth - aaDist,\n -uTroikaStrokeWidth + aaDist,\n fragDistance\n));\ngl_FragColor.a *= edgeAlpha;\n#endif\n\nif (edgeAlpha == 0.0) {\n discard;\n}\n",customRewriter({vertexShader:e,fragmentShader:t}){let r=/\buniform\s+vec3\s+diffuse\b/;return r.test(t)&&(t=t.replace(r,"varying vec3 vTroikaGlyphColor").replace(/\bdiffuse\b/g,"vTroikaGlyphColor"),r.test(e)||(e=e.replace(ji,"uniform vec3 diffuse;\n$&\nvTroikaGlyphColor = uTroikaUseGlyphColors ? aTroikaGlyphColor / 255.0 : diffuse;\n"))),{vertexShader:e,fragmentShader:t}}});return t.transparent=!0,Object.defineProperties(t,{isTroikaTextMaterial:{value:!0},shadowSide:{get(){return this.side},set(){}}}),t}(r),r.addEventListener("dispose",(function e(){r.removeEventListener("dispose",e),t.dispose()}))),this.outlineWidth||this.outlineBlur||this.outlineOffsetX||this.outlineOffsetY){let e=t._outlineMtl;return e||(e=t._outlineMtl=Object.create(t,{id:{value:t.id+.1}}),e.isTextOutlineMaterial=!0,e.depthWrite=!1,e.map=null,t.addEventListener("dispose",(function r(){t.removeEventListener("dispose",r),e.dispose()}))),[e,t]}return t}set material(e){e&&e.isTroikaTextMaterial?(this._derivedMaterial=e,this._baseMaterial=e.baseMaterial):this._baseMaterial=e}get glyphGeometryDetail(){return this.geometry.detail}set glyphGeometryDetail(e){this.geometry.detail=e}get curveRadius(){return this.geometry.curveRadius}set curveRadius(e){this.geometry.curveRadius=e}get customDepthMaterial(){return c(this.material).getDepthMaterial()}get customDistanceMaterial(){return c(this.material).getDistanceMaterial()}_prepareForRender(e){const a=e.isTextOutlineMaterial,c=e.uniforms,u=this.textRenderInfo;if(u){const{sdfTexture:e,blockBounds:r}=u;c.uTroikaSDFTexture.value=e,c.uTroikaSDFTextureSize.value.set(e.image.width,e.image.height),c.uTroikaSDFGlyphSize.value=u.sdfGlyphSize,c.uTroikaSDFExponent.value=u.sdfExponent,c.uTroikaTotalBounds.value.fromArray(r),c.uTroikaUseGlyphColors.value=!a&&!!u.glyphColors;let n,i,s,o=0,l=0,h=0,d=0,p=0;if(a){let{outlineWidth:e,outlineOffsetX:t,outlineOffsetY:r,outlineBlur:i,outlineOpacity:a}=this;o=this._parsePercent(e)||0,l=Math.max(0,this._parsePercent(i)||0),n=a,d=this._parsePercent(t)||0,p=this._parsePercent(r)||0}else h=Math.max(0,this._parsePercent(this.strokeWidth)||0),h&&(s=this.strokeColor,c.uTroikaStrokeColor.value.set(null==s?t:s),i=this.strokeOpacity,null==i&&(i=1)),n=this.fillOpacity;c.uTroikaDistanceOffset.value=o,c.uTroikaPositionOffset.value.set(d,p),c.uTroikaBlurRadius.value=l,c.uTroikaStrokeWidth.value=h,c.uTroikaStrokeOpacity.value=i,c.uTroikaFillOpacity.value=null==n?1:n,c.uTroikaCurveRadius.value=this.curveRadius||0;let f=this.clipRect;if(f&&Array.isArray(f)&&4===f.length)c.uTroikaClipRect.value.fromArray(f);else{const e=100*(this.fontSize||.1);c.uTroikaClipRect.value.set(r[0]-e,r[1]-e,r[2]+e,r[3]+e)}this.geometry.applyClipRect(c.uTroikaClipRect.value)}c.uTroikaSDFDebug.value=!!this.debugSDF,e.polygonOffset=!!this.depthOffset,e.polygonOffsetFactor=e.polygonOffsetUnits=this.depthOffset||0;const h=a?this.outlineColor||0:this.color;if(null==h)delete e.color;else{const t=e.hasOwnProperty("color")?e.color:e.color=new s.Color;h===t._input&&"object"!=typeof h||t.set(t._input=h)}let d=this.orientation||l;if(d!==e._orientation){let t=c.uTroikaOrient.value;d=d.replace(/[^-+xyz]/g,"");let a=d!==l&&d.match(/^([-+])([xyz])([-+])([xyz])$/);if(a){let[,e,s,l,c]=a;n.set(0,0,0)[s]="-"===e?1:-1,i.set(0,0,0)[c]="-"===l?-1:1,r.lookAt(o,n.cross(i),i),t.setFromMatrix4(r)}else t.identity();e._orientation=d}}_parsePercent(e){if("string"==typeof e){let t=e.match(/^(-?[\d.]+)%$/),r=t?parseFloat(t[1]):NaN;e=(isNaN(r)?0:r/100)*this.fontSize}return e}localPositionToTextCoords(e,t=new s.Vector2){t.copy(e);const r=this.curveRadius;return r&&(t.x=Math.atan2(e.x,Math.abs(r)-Math.abs(e.z))*Math.abs(r)),t}worldPositionToTextCoords(e,t=new s.Vector2){return n.copy(e),this.localPositionToTextCoords(this.worldToLocal(n),t)}raycast(e,t){const{textRenderInfo:r,curveRadius:n}=this;if(r){const i=r.blockBounds,s=n?h():u(),o=s.geometry,{position:l,uv:c}=o.attributes;for(let e=0;e{this[t]=e[t]})),this}clone(){return(new this.constructor).copy(this)}}f.forEach((e=>{const t="_private_"+e;Object.defineProperty(g.prototype,e,{get(){return this[t]},set(e){e!==this[t]&&(this[t]=e,this._needsSync=!0)}})}));let v=!1;return Object.defineProperty(g.prototype,"anchor",{get(){return this._deprecated_anchor},set(e){this._deprecated_anchor=e,v||(console.warn("TextMesh: `anchor` has been deprecated; use `anchorX` and `anchorY` instead."),v=!0),Array.isArray(e)?(this.anchorX=100*(+e[0]||0)+"%",this.anchorY=100*(+e[1]||0)+"%"):this.anchorX=this.anchorY=0}}),g})();new WeakMap;new WeakMap;function Sa(e,t,r=((e,t)=>e===t)){if(e===t)return!0;if(!e||!t)return!1;const n=e.length;if(t.length!==n)return!1;for(let i=0;ii.response=e)).then((()=>{n.lifespan&&n.lifespan>0&&setTimeout((()=>{const e=Ma.indexOf(i);-1!==e&&Ma.splice(e,1)}),n.lifespan)})).catch((e=>i.error=e))};if(Ma.push(i),!r)throw i.promise}const Ca=(e,t,r)=>Ta(e,t,!1,r),Ia=i.forwardRef((({anchorX:e="center",anchorY:t="middle",font:r,children:a,characters:s,onSync:l,...c},u)=>{const h=(0,o.useThree)((({invalidate:e})=>e)),[d]=i.useState((()=>new Ea)),[p,f]=i.useMemo((()=>{const e=[];let t="";return i.Children.forEach(a,(r=>{"string"==typeof r||"number"==typeof r?t+=r:e.push(r)})),[e,t]}),[a]);return Ca((()=>new Promise((e=>function({font:e,characters:t,sdfGlyphSize:r},n){ga({font:e,sdfGlyphSize:r,text:Array.isArray(t)?t.join("\n"):""+t},n)}({font:r,characters:s},e)))),["troika-text",r,s]),i.useLayoutEffect((()=>{d.sync((()=>{h(),l&&l(d)}))})),i.useEffect((()=>()=>d.dispose()),[d]),i.createElement("primitive",n({object:d,ref:u,font:r,text:f,anchorX:e,anchorY:t},c),p)}));var Ba=r(8064),Ra=r(48197),Pa=r(6073),Fa=r(60809);(0,o.extend)({EffectComposer:Ba.x,RenderPass:Ra.C,ShaderPass:Pa.T});const La=()=>{try{var e=document.createElement("canvas");return!(!window.WebGL2RenderingContext||!e.getContext("webgl2"))}catch(Ha){return!1}},Da=i.forwardRef((({children:e,multisamping:t=8,renderIndex:r=1,disableGamma:a=!1,disableRenderPass:l=!1,...c},u)=>{const h=i.useRef(),d=(0,o.useThree)((({scene:e})=>e)),p=(0,o.useThree)((({camera:e})=>e)),f=(0,o.useThree)((({gl:e})=>e)),m=(0,o.useThree)((({size:e})=>e)),[g]=i.useState((()=>{if(La()&&t>0){const e=new s.WebGLRenderTarget(m.width,m.height,{format:s.RGBAFormat,encoding:s.sRGBEncoding});return e.samples=8,e}}));return i.useEffect((()=>{var e,t;null==(e=h.current)||e.setSize(m.width,m.height),null==(t=h.current)||t.setPixelRatio(f.getPixelRatio())}),[f,m]),(0,o.useFrame)((()=>{var e;return null==(e=h.current)?void 0:e.render()}),r),i.createElement("effectComposer",n({ref:M([u,h]),args:[f,g]},c),!l&&i.createElement("renderPass",{attachArray:"passes",args:[d,p]}),!a&&i.createElement("shaderPass",{attachArray:"passes",args:[Fa.Y]}),e)}));function Ua({stops:e,colors:t,size:r=1024,...a}){const l=(0,o.useThree)((e=>e.gl)),c=i.useMemo((()=>{const n=document.createElement("canvas"),i=n.getContext("2d");n.width=16,n.height=r;const a=i.createLinearGradient(0,0,0,r);let o=e.length;for(;o--;)a.addColorStop(e[o],t[o]);i.fillStyle=a,i.fillRect(0,0,16,r);const l=new s.Texture(n);return l.needsUpdate=!0,l}),[e]);return i.useEffect((()=>()=>{c.dispose()}),[c]),i.createElement("primitive",n({object:c,attach:"map",encoding:l.outputEncoding},a))}function Na(e,t,r,n){return class extends s.ShaderMaterial{constructor(){const i=Object.entries(e);super({uniforms:i.reduce(((e,[t,r])=>({...e,...s.UniformsUtils.clone({[t]:{value:r}})})),{}),vertexShader:t,fragmentShader:r}),i.forEach((([e])=>Object.defineProperty(this,e,{get:()=>this.uniforms[e].value,set:t=>this.uniforms[e].value=t}))),n&&n(this)}}}const ka=e=>e===Object(e)&&!Array.isArray(e)&&"function"!=typeof e;function Oa(e){const t=(0,o.useThree)((e=>e.gl)),r=(0,o.useLoader)(s.TextureLoader,ka(e)?Object.values(e):e);if((0,i.useEffect)((()=>{(Array.isArray(r)?r:[r]).forEach(t.initTexture)}),[t,r]),ka(e)){const t=Object.keys(e),n={};return t.forEach((e=>Object.assign(n,{[e]:r[t.indexOf(e)]}))),n}return r}Oa.preload=e=>o.useLoader.preload(s.TextureLoader,e),Oa.clear=e=>o.useLoader.clear(s.TextureLoader,e);const Ga=Na({color:new s.Color("white"),scale:[1,1],imageBounds:[1,1],map:null,zoom:1,grayscale:0},"\n varying vec2 vUv;\n void main() {\n gl_Position = projectionMatrix * viewMatrix * modelMatrix * vec4(position, 1.);\n vUv = uv;\n }\n","\n // mostly from https://gist.github.com/statico/df64c5d167362ecf7b34fca0b1459a44\n varying vec2 vUv;\n uniform vec2 scale;\n uniform vec2 imageBounds;\n uniform vec3 color;\n uniform sampler2D map;\n uniform float zoom;\n uniform float grayscale;\n const vec3 luma = vec3(.299, 0.587, 0.114);\n vec4 toGrayscale(vec4 color, float intensity) {\n return vec4(mix(color.rgb, vec3(dot(color.rgb, luma)), intensity), color.a);\n }\n vec2 aspect(vec2 size) {\n return size / min(size.x, size.y);\n }\n void main() {\n vec2 s = aspect(scale);\n vec2 i = aspect(imageBounds);\n float rs = s.x / s.y;\n float ri = i.x / i.y;\n vec2 new = rs < ri ? vec2(i.x * s.y / i.y, s.y) : vec2(s.x, i.y * s.x / i.x);\n vec2 offset = (rs < ri ? vec2((new.x - s.x) / 2.0, 0.0) : vec2(0.0, (new.y - s.y) / 2.0)) / new;\n vec2 uv = vUv * s / new + offset;\n vec2 zUv = (uv - vec2(0.5, 0.5)) / zoom + vec2(0.5, 0.5);\n gl_FragColor = toGrayscale(texture2D(map, zUv) * vec4(color, 1.0), grayscale);\n \n #include \n #include \n }\n"),za=i.forwardRef((({children:e,color:t,segments:r=1,scale:a=1,zoom:s=1,grayscale:l=0,url:c,toneMapped:u,...h},d)=>{(0,o.extend)({ImageMaterial:Ga});const p=Oa(c),f=Array.isArray(a)?[a[0],a[1]]:[a,a],m=[p.image.width,p.image.height];return i.createElement("mesh",n({ref:d,scale:a},h),i.createElement("planeGeometry",{args:[1,1,r,r]}),i.createElement("imageMaterial",{color:t,map:p,zoom:s,grayscale:l,scale:f,imageBounds:m,toneMapped:u}),e)}));function Va({userData:e,children:t,geometry:r,threshold:a=15,color:o="black",...l}){const c=i.useRef(null);return i.useLayoutEffect((()=>{const e=c.current.parent;if(e){const t=r||e.geometry;t===c.current.userData.currentGeom&&a===c.current.userData.currentThreshold||(c.current.userData.currentGeom=t,c.current.userData.currentThreshold=a,c.current.geometry=new s.EdgesGeometry(t,a))}})),i.createElement("lineSegments",n({ref:c,raycast:()=>null},l),t||i.createElement("lineBasicMaterial",{color:o}))}function Ha(e,t){const r=new s.Matrix4,n=new s.Ray,i=new s.Sphere,a=new s.Vector3,o=this.geometry;if(i.copy(o.boundingSphere),i.applyMatrix4(this.matrixWorld),!1===e.ray.intersectSphere(i,a))return;r.copy(this.matrixWorld).invert(),n.copy(e.ray).applyMatrix4(r);const l=new s.Vector3,c=new s.Vector3,u=new s.Vector3,h=this instanceof s.LineSegments?2:1,d=o.index,p=o.attributes;if(null!==d){const r=d.array,i=p.position.array,s=p.width.array;for(let o=0,d=r.length-1;og)continue;a.applyMatrix4(this.matrixWorld);const v=e.ray.origin.distanceTo(a);ve.far||(t.push({distance:v,point:u.clone().applyMatrix4(this.matrixWorld),index:o,face:null,faceIndex:null,object:this}),o=d)}}}function Qa(e,t,r,n,i){let a;if(e=e.subarray||e.slice?e:e.buffer,r=r.subarray||r.slice?r:r.buffer,e=t?e.subarray?e.subarray(t,i&&t+i):e.slice(t,i&&t+i):e,r.set)r.set(e,n);else for(a=0;a0&&(r=this.copyV3(n),this.next.push(r[0],r[1],r[2]),this.next.push(r[0],r[1],r[2]))}r=this.compareV3(e-1,0)?this.copyV3(1):this.copyV3(e-1),this.next.push(r[0],r[1],r[2]),this.next.push(r[0],r[1],r[2]),this._attributes&&this._attributes.position.count===this.positions.length?(this._attributes.position.copyArray(new Float32Array(this.positions)),this._attributes.position.needsUpdate=!0,this._attributes.previous.copyArray(new Float32Array(this.previous)),this._attributes.previous.needsUpdate=!0,this._attributes.next.copyArray(new Float32Array(this.next)),this._attributes.next.needsUpdate=!0,this._attributes.side.copyArray(new Float32Array(this.side)),this._attributes.side.needsUpdate=!0,this._attributes.width.copyArray(new Float32Array(this.width)),this._attributes.width.needsUpdate=!0,this._attributes.uv.copyArray(new Float32Array(this.uvs)),this._attributes.uv.needsUpdate=!0,this._attributes.index.copyArray(new Uint16Array(this.indices_array)),this._attributes.index.needsUpdate=!0):this._attributes={position:new s.BufferAttribute(new Float32Array(this.positions),3),previous:new s.BufferAttribute(new Float32Array(this.previous),3),next:new s.BufferAttribute(new Float32Array(this.next),3),side:new s.BufferAttribute(new Float32Array(this.side),1),width:new s.BufferAttribute(new Float32Array(this.width),1),uv:new s.BufferAttribute(new Float32Array(this.uvs),2),index:new s.BufferAttribute(new Uint16Array(this.indices_array),1),counters:new s.BufferAttribute(new Float32Array(this.counters),1)},this.setAttribute("position",this._attributes.position),this.setAttribute("previous",this._attributes.previous),this.setAttribute("next",this._attributes.next),this.setAttribute("side",this._attributes.side),this.setAttribute("width",this._attributes.width),this.setAttribute("uv",this._attributes.uv),this.setAttribute("counters",this._attributes.counters),this.setIndex(this._attributes.index),this.computeBoundingSphere(),this.computeBoundingBox()}advance({x:e,y:t,z:r}){const n=this._attributes.position.array,i=this._attributes.previous.array,a=this._attributes.next.array,s=n.length;Qa(n,0,i,0,s),Qa(n,6,n,0,s-6),n[s-6]=e,n[s-5]=t,n[s-4]=r,n[s-3]=e,n[s-2]=t,n[s-1]=r,Qa(n,6,a,0,s-6),a[s-6]=e,a[s-5]=t,a[s-4]=r,a[s-3]=e,a[s-2]=t,a[s-1]=r,this._attributes.position.needsUpdate=!0,this._attributes.previous.needsUpdate=!0,this._attributes.next.needsUpdate=!0}}s.ShaderChunk.meshline_vert=["","#include ","",s.ShaderChunk.logdepthbuf_pars_vertex,s.ShaderChunk.fog_pars_vertex,"","attribute vec3 previous;","attribute vec3 next;","attribute float side;","attribute float width;","attribute float counters;","","uniform vec2 resolution;","uniform float lineWidth;","uniform vec3 color;","uniform float opacity;","uniform float sizeAttenuation;","","varying vec2 vUV;","varying vec4 vColor;","varying float vCounters;","","vec2 fix( vec4 i, float aspect ) {",""," vec2 res = i.xy / i.w;"," res.x *= aspect;","\t vCounters = counters;"," return res;","","}","","void main() {",""," float aspect = resolution.x / resolution.y;",""," vColor = vec4( color, opacity );"," vUV = uv;",""," mat4 m = projectionMatrix * modelViewMatrix;"," vec4 finalPosition = m * vec4( position, 1.0 );"," vec4 prevPos = m * vec4( previous, 1.0 );"," vec4 nextPos = m * vec4( next, 1.0 );",""," vec2 currentP = fix( finalPosition, aspect );"," vec2 prevP = fix( prevPos, aspect );"," vec2 nextP = fix( nextPos, aspect );",""," float w = lineWidth * width;",""," vec2 dir;"," if( nextP == currentP ) dir = normalize( currentP - prevP );"," else if( prevP == currentP ) dir = normalize( nextP - currentP );"," else {"," vec2 dir1 = normalize( currentP - prevP );"," vec2 dir2 = normalize( nextP - currentP );"," dir = normalize( dir1 + dir2 );",""," vec2 perp = vec2( -dir1.y, dir1.x );"," vec2 miter = vec2( -dir.y, dir.x );"," //w = clamp( w / dot( miter, perp ), 0., 4. * lineWidth * width );",""," }",""," //vec2 normal = ( cross( vec3( dir, 0. ), vec3( 0., 0., 1. ) ) ).xy;"," vec4 normal = vec4( -dir.y, dir.x, 0., 1. );"," normal.xy *= .5 * w;"," normal *= projectionMatrix;"," if( sizeAttenuation == 0. ) {"," normal.xy *= finalPosition.w;"," normal.xy /= ( vec4( resolution, 0., 1. ) * projectionMatrix ).xy;"," }",""," finalPosition.xy += normal.xy * side;",""," gl_Position = finalPosition;","",s.ShaderChunk.logdepthbuf_vertex,s.ShaderChunk.fog_vertex&&" vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );",s.ShaderChunk.fog_vertex,"}"].join("\n"),s.ShaderChunk.meshline_frag=["",s.ShaderChunk.fog_pars_fragment,s.ShaderChunk.logdepthbuf_pars_fragment,"","uniform sampler2D map;","uniform sampler2D alphaMap;","uniform float useMap;","uniform float useAlphaMap;","uniform float useDash;","uniform float dashArray;","uniform float dashOffset;","uniform float dashRatio;","uniform float visibility;","uniform float alphaTest;","uniform vec2 repeat;","","varying vec2 vUV;","varying vec4 vColor;","varying float vCounters;","","void main() {","",s.ShaderChunk.logdepthbuf_fragment,""," vec4 c = vColor;"," if( useMap == 1. ) c *= texture2D( map, vUV * repeat );"," if( useAlphaMap == 1. ) c.a *= texture2D( alphaMap, vUV * repeat ).a;"," if( c.a < alphaTest ) discard;"," if( useDash == 1. ){"," c.a *= ceil(mod(vCounters + dashOffset, dashArray) - (dashArray * dashRatio));"," }"," gl_FragColor = c;"," gl_FragColor.a *= step(vCounters, visibility);","",s.ShaderChunk.fog_fragment,"}"].join("\n");class Wa extends s.ShaderMaterial{constructor(e){super({uniforms:Object.assign({},s.UniformsLib.fog,{lineWidth:{value:1},map:{value:null},useMap:{value:0},alphaMap:{value:null},useAlphaMap:{value:0},color:{value:new s.Color(16777215)},opacity:{value:1},resolution:{value:new s.Vector2(1,1)},sizeAttenuation:{value:1},dashArray:{value:0},dashOffset:{value:0},dashRatio:{value:.5},useDash:{value:0},visibility:{value:1},alphaTest:{value:0},repeat:{value:new s.Vector2(1,1)}}),vertexShader:s.ShaderChunk.meshline_vert,fragmentShader:s.ShaderChunk.meshline_frag}),this.type="MeshLineMaterial",Object.defineProperties(this,{lineWidth:{enumerable:!0,get(){return this.uniforms.lineWidth.value},set(e){this.uniforms.lineWidth.value=e}},map:{enumerable:!0,get(){return this.uniforms.map.value},set(e){this.uniforms.map.value=e}},useMap:{enumerable:!0,get(){return this.uniforms.useMap.value},set(e){this.uniforms.useMap.value=e}},alphaMap:{enumerable:!0,get(){return this.uniforms.alphaMap.value},set(e){this.uniforms.alphaMap.value=e}},useAlphaMap:{enumerable:!0,get(){return this.uniforms.useAlphaMap.value},set(e){this.uniforms.useAlphaMap.value=e}},color:{enumerable:!0,get(){return this.uniforms.color.value},set(e){this.uniforms.color.value=e}},opacity:{enumerable:!0,get(){return this.uniforms.opacity.value},set(e){this.uniforms.opacity.value=e}},resolution:{enumerable:!0,get(){return this.uniforms.resolution.value},set(e){this.uniforms.resolution.value.copy(e)}},sizeAttenuation:{enumerable:!0,get(){return this.uniforms.sizeAttenuation.value},set(e){this.uniforms.sizeAttenuation.value=e}},dashArray:{enumerable:!0,get(){return this.uniforms.dashArray.value},set(e){this.uniforms.dashArray.value=e,this.useDash=0!==e?1:0}},dashOffset:{enumerable:!0,get(){return this.uniforms.dashOffset.value},set(e){this.uniforms.dashOffset.value=e}},dashRatio:{enumerable:!0,get(){return this.uniforms.dashRatio.value},set(e){this.uniforms.dashRatio.value=e}},useDash:{enumerable:!0,get(){return this.uniforms.useDash.value},set(e){this.uniforms.useDash.value=e}},visibility:{enumerable:!0,get(){return this.uniforms.visibility.value},set(e){this.uniforms.visibility.value=e}},alphaTest:{enumerable:!0,get(){return this.uniforms.alphaTest.value},set(e){this.uniforms.alphaTest.value=e}},repeat:{enumerable:!0,get(){return this.uniforms.repeat.value},set(e){this.uniforms.repeat.value.copy(e)}}}),this.setValues(e)}copy(e){return super.copy(e),this.lineWidth=e.lineWidth,this.map=e.map,this.useMap=e.useMap,this.alphaMap=e.alphaMap,this.useAlphaMap=e.useAlphaMap,this.color.copy(e.color),this.opacity=e.opacity,this.resolution.copy(e.resolution),this.sizeAttenuation=e.sizeAttenuation,this.dashArray.copy(e.dashArray),this.dashOffset.copy(e.dashOffset),this.dashRatio.copy(e.dashRatio),this.useDash=e.useDash,this.visibility=e.visibility,this.alphaTest=e.alphaTest,this.repeat.copy(e.repeat),this}}const Xa={width:.2,length:1,decay:1,local:!1,stride:0,interval:1},Ya=(e,t=1)=>(e.set(e.subarray(t)),e.fill(-1/0,-t),e);function qa(e,t){const{length:r,local:n,decay:a,interval:l,stride:c}={...Xa,...t},u=i.useRef(),[h]=i.useState((()=>new s.Vector3));i.useLayoutEffect((()=>{e&&(u.current=Float32Array.from({length:10*r*3},((t,r)=>e.position.getComponent(r%3))))}),[r,e]);const d=i.useRef(new s.Vector3),p=i.useRef(0);return(0,o.useFrame)((()=>{if(e&&u.current){if(0===p.current){let t;n?t=e.position:(e.getWorldPosition(h),t=h);const r=1*a;for(let e=0;e{const{children:r}=e,{width:n,length:a,decay:l,local:c,stride:u,interval:h}={...Xa,...e},{color:d="hotpink",attenuation:p,target:f}=e,m=(0,o.useThree)((e=>e.size)),g=i.useRef(null),[v,A]=i.useState(null),y=qa(v,{length:a,decay:l,local:c,stride:u,interval:h});i.useEffect((()=>{const e=(null==f?void 0:f.current)||g.current.children.find((e=>e instanceof s.Object3D));e&&A(e)}),[y,f]);const x=i.useMemo((()=>new ja),[]),b=i.useMemo((()=>{var e;const t=new Wa({lineWidth:.1*n,color:d,sizeAttenuation:1,resolution:new s.Vector2(m.width,m.height)});let i;if(r)if(Array.isArray(r))i=r.find((e=>{const t=e;return"string"==typeof t.type&&"meshLineMaterial"===t.type}));else{const e=r;"string"==typeof e.type&&"meshLineMaterial"===e.type&&(i=e)}return"object"==typeof(null==(e=i)?void 0:e.props)&&t.setValues(i.props),t}),[n,d,m,r]);return i.useEffect((()=>{b.uniforms.resolution.value.set(m.width,m.height)}),[m]),(0,o.useFrame)((()=>{y.current&&x.setPoints(y.current,p)})),i.createElement("group",null,i.createElement("mesh",{ref:t,geometry:x,material:b}),i.createElement("group",{ref:g},r))}));var Ja=r(96272);const Za=({children:e,weight:t,transform:r,instances:a,mesh:o,...l})=>{const c=i.useRef(null),u=i.useRef(null),h=i.useRef(null);return i.useEffect((()=>{var e,t;u.current=null!==(e=null==a?void 0:a.current)&&void 0!==e?e:c.current.children.find((e=>e.hasOwnProperty("instanceMatrix"))),h.current=null!==(t=null==o?void 0:o.current)&&void 0!==t?t:c.current.children.find((e=>"Mesh"===e.type))}),[e,null==o?void 0:o.current,null==a?void 0:a.current]),i.useEffect((()=>{if(void 0===h.current)return;if(void 0===u.current)return;const e=new Ja.a(h.current);t&&e.setWeightAttribute(t),e.build();const n=new s.Vector3,i=new s.Vector3,a=new s.Color,o=new s.Object3D;h.current.updateMatrixWorld(!0);for(let t=0;t{const[a]=i.useState((()=>new s.BufferAttribute(new Float32Array(0),1))),o=i.useRef(null);return i.useLayoutEffect((()=>{if(o.current){var t;const r=null!==(t=o.current.parent)&&void 0!==t?t:o.current.__r3f.parent,n=e(r);o.current.copy(n)}}),[e]),i.createElement("primitive",n({ref:o,object:a,attachObject:["attributes",t]},r))};var es=r(25030),ts=r.n(es);function rs(e,{keys:t=["near","far","color","distance","decay","penumbra","angle","intensity","skeleton","visible","castShadow","receiveShadow","morphTargetDictionary","morphTargetInfluences","name","geometry","material","position","rotation","scale","up","userData"],deep:r,inject:n,castShadow:a,receiveShadow:s}){let o=ts()(e,t);return r&&(o.geometry&&"materialsOnly"!==r&&(o.geometry=o.geometry.clone()),o.material&&"geometriesOnly"!==r&&(o.material=o.material.clone())),n&&(o="function"==typeof n?{...o,children:n(e)}:i.isValidElement(n)?{...o,children:n}:{...o,...n}),"Mesh"===e.type&&(a&&(o.castShadow=!0),s&&(o.receiveShadow=!0)),o}const ns=i.forwardRef((({object:e,children:t,deep:r,castShadow:a,receiveShadow:s,inject:o,keys:l,...c},u)=>{const h={keys:l,deep:r,inject:o,castShadow:a,receiveShadow:s};if(Array.isArray(e))return i.createElement("group",n({},c,{ref:u}),e.map((e=>i.createElement(ns,n({key:e.uuid,object:e},h)))),t);const{children:d,...p}=rs(e,h),f=e.type[0].toLowerCase()+e.type.slice(1);return i.createElement(f,n({},p,c,{ref:u}),(null==e?void 0:e.children).map((e=>{let t={},r=e.type[0].toLowerCase()+e.type.slice(1);return"group"===r||"object3D"===r?(r=ns,t={object:e,...h}):t=rs(e,h),i.createElement(r,n({key:e.uuid},t))})),t,d)})),is=i.forwardRef((({makeDefault:e,...t},r)=>{const a=(0,o.useThree)((({set:e})=>e)),s=(0,o.useThree)((({camera:e})=>e)),l=(0,o.useThree)((({size:e})=>e)),c=i.useRef();return i.useLayoutEffect((()=>{c.current&&!t.manual&&c.current.updateProjectionMatrix()}),[l,t]),i.useLayoutEffect((()=>{if(e&&c.current){const e=s;return a((()=>({camera:c.current}))),()=>a((()=>({camera:e})))}}),[s,c,e,a]),i.createElement("orthographicCamera",n({left:l.width/-2,right:l.width/2,top:l.height/2,bottom:l.height/-2,ref:M([c,r])},t))})),as=i.forwardRef((({makeDefault:e,...t},r)=>{const a=(0,o.useThree)((({set:e})=>e)),s=(0,o.useThree)((({camera:e})=>e)),l=(0,o.useThree)((({size:e})=>e)),c=i.useRef();return i.useLayoutEffect((()=>{const{current:e}=c;e&&!t.manual&&(e.aspect=l.width/l.height,e.updateProjectionMatrix())}),[l,t]),i.useLayoutEffect((()=>{if(e&&c.current){const e=s;return a((()=>({camera:c.current}))),()=>a((()=>({camera:e})))}}),[s,c,e,a]),i.createElement("perspectiveCamera",n({ref:M([c,r])},t))}));function ss({children:e,fog:t,frames:r=1/0,resolution:n=256,near:a=1,far:l=1e3,...c}){const u=i.useRef(),[h,d]=i.useState(),p=(0,o.useThree)((({scene:e})=>e)),f=(0,o.useThree)((({gl:e})=>e)),m=i.useMemo((()=>{const e=new s.WebGLCubeRenderTarget(n);return e.texture.type=s.HalfFloatType,e}),[n]);let g=0;return(0,o.useFrame)((()=>{if(h&&u.current&&(r===1/0||ge.visible=!1));const e=p.fog;p.fog=t||e,h.update(f,p),p.fog=e,u.current.traverse((e=>e.visible=!0)),g++}})),i.createElement("group",c,i.createElement("cubeCamera",{ref:d,args:[a,l,m]}),i.createElement("group",{ref:u},e(m.texture)))}var os=r(41743);const ls=i.forwardRef(((e,t)=>{const{camera:r,onChange:a,...s}=e,l=(0,o.useThree)((e=>e.camera)),c=(0,o.useThree)((e=>e.invalidate)),u=r||l,[h]=i.useState((()=>new os.W(u)));return i.useEffect((()=>{const e=e=>{c(),a&&a(e)};return null==h||null==h.addEventListener||h.addEventListener("change",e),()=>null==h||null==h.removeEventListener?void 0:h.removeEventListener("change",e)}),[a,h,c]),(0,o.useFrame)((()=>null==h?void 0:h.update())),i.useEffect((()=>{const e=h;return null==e||e.connect(),()=>null==e?void 0:e.dispose()}),[h]),h?i.createElement("primitive",n({ref:t,dispose:void 0,object:h},s)):null}));var cs=r(32044);const us=i.forwardRef(((e,t)=>{const{onChange:r,...a}=e,s=(0,o.useThree)((({invalidate:e})=>e)),l=(0,o.useThree)((({camera:e})=>e)),c=(0,o.useThree)((({gl:e})=>e)),[u]=i.useState((()=>new cs.m(l,c.domElement)));return i.useEffect((()=>{const e=e=>{s(),r&&r(e)};return null==u||null==u.addEventListener||u.addEventListener("change",e),()=>null==u||null==u.removeEventListener?void 0:u.removeEventListener("change",e)}),[r,u,s]),(0,o.useFrame)(((e,t)=>null==u?void 0:u.update(t))),u?i.createElement("primitive",n({ref:t,dispose:void 0,object:u},a)):null}));var hs=r(22227);const ds=i.forwardRef(((e={enableDamping:!0},t)=>{const{camera:r,onChange:a,onStart:s,onEnd:l,...c}=e,u=(0,o.useThree)((({invalidate:e})=>e)),h=(0,o.useThree)((({camera:e})=>e)),d=(0,o.useThree)((({gl:e})=>e.domElement)),p=r||h,f=i.useMemo((()=>new hs.o(p)),[p]);return i.useEffect((()=>{f.connect(d);const e=e=>{u(),a&&a(e)};return f.addEventListener("change",e),s&&f.addEventListener("start",s),l&&f.addEventListener("end",l),()=>{f.dispose(),f.removeEventListener("change",e),s&&f.removeEventListener("start",s),l&&f.removeEventListener("end",l)}}),[a,s,l,f,u,d]),(0,o.useFrame)((()=>f.update())),i.createElement("primitive",n({ref:t,dispose:void 0,object:f,enableDamping:!0},c))})),ps=i.forwardRef((({makeDefault:e,camera:t,regress:r,domElement:a,enableDamping:s=!0,onChange:l,onStart:c,onEnd:u,...h},d)=>{const p=(0,o.useThree)((e=>e.invalidate)),f=(0,o.useThree)((e=>e.camera)),m=(0,o.useThree)((e=>e.gl)),g=(0,o.useThree)((e=>e.events)),v=(0,o.useThree)((e=>e.set)),A=(0,o.useThree)((e=>e.get)),y=(0,o.useThree)((e=>e.performance)),x=t||f,b=a||g.connected||m.domElement,w=i.useMemo((()=>new hs.z(x)),[x]);return(0,o.useFrame)((()=>{w.enabled&&w.update()})),i.useEffect((()=>(w.connect(b),()=>{w.dispose()})),[b,r,w,p]),i.useEffect((()=>{const e=e=>{p(),r&&y.regress(),l&&l(e)};return w.addEventListener("change",e),c&&w.addEventListener("start",c),u&&w.addEventListener("end",u),()=>{c&&w.removeEventListener("start",c),u&&w.removeEventListener("end",u),w.removeEventListener("change",e)}}),[l,c,u]),i.useEffect((()=>{if(e){const e=A().controls;return v({controls:w}),()=>v({controls:e})}}),[e,w]),i.createElement("primitive",n({ref:d,object:w,enableDamping:s},h))}));var fs=r(7703);const ms=i.forwardRef((({makeDefault:e,camera:t,domElement:r,regress:a,onChange:s,onStart:l,onEnd:c,...u},h)=>{const{invalidate:d,camera:p,gl:f,events:m,set:g,get:v,performance:A,viewport:y}=(0,o.useThree)(),x=t||p,b=r||m.connected||f.domElement,w=i.useMemo((()=>new fs.$(x)),[x]);return(0,o.useFrame)((()=>{w.enabled&&w.update()})),i.useEffect((()=>(w.connect(b),()=>{w.dispose()})),[b,a,w,d]),i.useEffect((()=>{const e=e=>{d(),a&&A.regress(),s&&s(e)};return w.addEventListener("change",e),l&&w.addEventListener("start",l),c&&w.addEventListener("end",c),()=>{l&&w.removeEventListener("start",l),c&&w.removeEventListener("end",c),w.removeEventListener("change",e)}}),[s,l,c]),i.useEffect((()=>{w.handleResize()}),[y]),i.useEffect((()=>{if(e){const e=v().controls;return g({controls:w}),()=>g({controls:e})}}),[e,w]),i.createElement("primitive",n({ref:h,object:w},u))}));var gs=r(1687);const vs=(0,i.forwardRef)((({camera:e,makeDefault:t,regress:r,domElement:a,onChange:s,onStart:l,onEnd:c,...u},h)=>{const d=(0,o.useThree)((e=>e.invalidate)),p=(0,o.useThree)((e=>e.camera)),f=(0,o.useThree)((e=>e.gl)),m=(0,o.useThree)((e=>e.events)),g=(0,o.useThree)((e=>e.set)),v=(0,o.useThree)((e=>e.get)),A=(0,o.useThree)((e=>e.performance)),y=e||p,x=a||m.connected||f.domElement,b=(0,i.useMemo)((()=>new gs.A(y)),[y]);return(0,o.useFrame)((()=>{b.enabled&&b.update()})),(0,i.useEffect)((()=>(b.connect(x),()=>{b.dispose()})),[x,r,b,d]),(0,i.useEffect)((()=>{const e=e=>{d(),r&&A.regress(),s&&s(e)};return b.addEventListener("change",e),l&&b.addEventListener("start",l),c&&b.addEventListener("end",c),()=>{b.removeEventListener("change",e),l&&b.removeEventListener("start",l),c&&b.removeEventListener("end",c)}}),[s,l,c]),(0,i.useEffect)((()=>{if(t){const e=v().controls;return g({controls:b}),()=>g({controls:e})}}),[t,b]),i.createElement("primitive",n({ref:h,object:b},u))}));var As=r(76427),ys=r.n(As),xs=r(35041);const bs=i.forwardRef((({children:e,domElement:t,onChange:r,onMouseDown:a,onMouseUp:l,onObjectChange:c,object:u,...h},d)=>{const p=["enabled","axis","mode","translationSnap","rotationSnap","scaleSnap","space","size","showX","showY","showZ"],{camera:f,...m}=h,g=ts()(m,p),v=ys()(m,p),A=(0,o.useThree)((e=>e.controls)),y=(0,o.useThree)((e=>e.gl)),x=(0,o.useThree)((e=>e.events)),b=(0,o.useThree)((e=>e.camera)),w=(0,o.useThree)((e=>e.invalidate)),_=f||b,E=t||x.connected||y.domElement,S=i.useMemo((()=>new xs.Ys(_,E)),[_,E]),M=i.useRef();return i.useLayoutEffect((()=>(u?S.attach(u instanceof s.Object3D?u:u.current):M.current instanceof s.Object3D&&S.attach(M.current),()=>{S.detach()})),[u,e,S]),i.useEffect((()=>{if(A){const e=e=>A.enabled=!e.value;return S.addEventListener("dragging-changed",e),()=>S.removeEventListener("dragging-changed",e)}}),[S,A]),i.useEffect((()=>{const e=e=>{w(),r&&r(e)};return null==S||null==S.addEventListener||S.addEventListener("change",e),a&&(null==S||null==S.addEventListener||S.addEventListener("mouseDown",a)),l&&(null==S||null==S.addEventListener||S.addEventListener("mouseUp",l)),c&&(null==S||null==S.addEventListener||S.addEventListener("objectChange",c)),()=>{null==S||null==S.removeEventListener||S.removeEventListener("change",e),a&&(null==S||null==S.removeEventListener||S.removeEventListener("mouseDown",a)),l&&(null==S||null==S.removeEventListener||S.removeEventListener("mouseUp",l)),c&&(null==S||null==S.removeEventListener||S.removeEventListener("objectChange",c))}}),[r,a,l,c,S,w]),S?i.createElement(i.Fragment,null,i.createElement("primitive",n({ref:d,dispose:void 0,object:S},g)),i.createElement("group",n({ref:M},v),e)):null}));var ws=r(2679);const _s=i.forwardRef((({domElement:e,selector:t,onChange:r,onLock:a,onUnlock:s,enabled:l=!0,...c},u)=>{const{camera:h,...d}=c,p=(0,o.useThree)((({gl:e})=>e)),f=(0,o.useThree)((e=>e.camera)),m=(0,o.useThree)((e=>e.invalidate)),g=(0,o.useThree)((e=>e.raycaster)),v=(0,o.useThree)((e=>e.events)),A=h||f,y=e||v.connected||p.domElement,[x]=i.useState((()=>new ws.q(A)));return i.useEffect((()=>{if(l){x.connect(y);const e=g.computeOffsets;return g.computeOffsets=e=>({offsetX:e.target.width/2,offsetY:e.target.height/2}),()=>{x.disconnect(),g.computeOffsets=e}}}),[l,x]),i.useEffect((()=>{const e=e=>{m(),r&&r(e)};x.addEventListener("change",e),a&&x.addEventListener("lock",a),s&&x.addEventListener("unlock",s);const n=()=>x.lock(),i=t?Array.from(document.querySelectorAll(t)):[document];return i.forEach((e=>e&&e.addEventListener("click",n))),()=>{x.removeEventListener("change",e),a&&x.addEventListener("lock",a),s&&x.addEventListener("unlock",s),i.forEach((e=>e?e.removeEventListener("click",n):void 0))}}),[r,a,s,t]),i.createElement("primitive",n({ref:u,object:x},d))}));var Es=r(55259);const Ss=i.forwardRef(((e,t)=>{const r=(0,o.useThree)((e=>e.camera)),a=(0,o.useThree)((e=>e.gl)),[s]=i.useState((()=>new Es.o(r,a.domElement)));return(0,o.useFrame)(((e,t)=>{s.update(t)})),s?i.createElement("primitive",n({ref:t,object:s},e)):null}));function Ms(e,t){const r=(0,o.useThree)((e=>e.mouse)),[n]=i.useState((()=>{const n=new s.Raycaster;return t&&(0,o.applyProps)(n,t,{}),function(t,i){n.setFromCamera(r,e instanceof s.Camera?e:e.current);const a=this.constructor.prototype.raycast.bind(this);a&&a(n,i)}}));return n}const Ts=i.createContext({}),Cs=()=>i.useContext(Ts),Is=2*Math.PI,Bs=new s.Object3D,Rs=new s.Matrix4,[Ps,Fs]=[new s.Quaternion,new s.Quaternion],Ls=new s.Vector3,Ds=new s.Vector3,Us=({alignment:e="bottom-right",margin:t=[80,80],renderPriority:r=0,autoClear:n=!0,onUpdate:a,onTarget:l,children:c})=>{const u=(0,o.useThree)((({size:e})=>e)),h=(0,o.useThree)((({camera:e})=>e)),d=(0,o.useThree)((({controls:e})=>e)),p=(0,o.useThree)((({gl:e})=>e)),f=(0,o.useThree)((({scene:e})=>e)),m=(0,o.useThree)((({invalidate:e})=>e)),g=i.useRef(),v=i.useRef(),A=i.useRef(null),[y]=i.useState((()=>new s.Scene)),x=i.useRef(!1),b=i.useRef(0),w=i.useRef(new s.Vector3(0,0,0)),_=i.useCallback((e=>{x.current=!0,(d||l)&&(w.current=(null==d?void 0:d.target)||(null==l?void 0:l())),b.current=h.position.distanceTo(Ls),Ps.copy(h.quaternion),Ds.copy(e).multiplyScalar(b.current).add(Ls),Bs.lookAt(Ds),Fs.copy(Bs.quaternion),m()}),[d,h,l,m]);i.useEffect((()=>(f.background&&(g.current=f.background,f.background=null,y.background=g.current),()=>{g.current&&(f.background=g.current)})),[]),(0,o.useFrame)(((e,t)=>{if(A.current&&v.current){var r;if(x.current)if(Ps.angleTo(Fs)<.01)x.current=!1;else{const e=t*Is;Ps.rotateTowards(Fs,e),h.position.set(0,0,1).applyQuaternion(Ps).multiplyScalar(b.current).add(w.current),h.up.set(0,1,0).applyQuaternion(Ps).normalize(),h.quaternion.copy(Ps),a?a():d&&d.update(),m()}Rs.copy(h.matrix).invert(),null==(r=v.current)||r.quaternion.setFromRotationMatrix(Rs),n&&(p.autoClear=!1),p.clearDepth(),p.render(y,A.current)}}),r);const E=Ms(A),S=i.useMemo((()=>({tweenCamera:_,raycast:E})),[_]),[M,T]=t,C=e.endsWith("-left")?-u.width/2+M:u.width/2-M,I=e.startsWith("top-")?u.height/2-T:-u.height/2+T;return(0,o.createPortal)(i.createElement(Ts.Provider,{value:S},i.createElement(is,{ref:A,position:[0,0,200]}),i.createElement("group",{ref:v,position:[C,I,0]},c)),y)},Ns="#f0f0f0",ks="#999",Os="black",Gs="black",zs=["Right","Left","Top","Bottom","Front","Back"],Vs=e=>new s.Vector3(...e).multiplyScalar(.38),Hs=[[1,1,1],[1,1,-1],[1,-1,1],[1,-1,-1],[-1,1,1],[-1,1,-1],[-1,-1,1],[-1,-1,-1]].map(Vs),Qs=[.25,.25,.25],js=[[1,1,0],[1,0,1],[1,0,-1],[1,-1,0],[0,1,1],[0,1,-1],[0,-1,1],[0,-1,-1],[-1,1,0],[-1,0,1],[-1,0,-1],[-1,-1,0]].map(Vs),Ws=js.map((e=>e.toArray().map((e=>0==e?.5:.25)))),Xs=({hover:e,index:t,font:r="20px Inter var, Arial, sans-serif",faces:n=zs,color:a=Ns,hoverColor:l=ks,textColor:c=Os,strokeColor:u=Gs,opacity:h=1})=>{const d=(0,o.useThree)((e=>e.gl)),p=i.useMemo((()=>{const e=document.createElement("canvas");e.width=128,e.height=128;const i=e.getContext("2d");return i.fillStyle=a,i.fillRect(0,0,e.width,e.height),i.strokeStyle=u,i.strokeRect(0,0,e.width,e.height),i.font=r,i.textAlign="center",i.fillStyle=c,i.fillText(n[t].toUpperCase(),64,76),new s.CanvasTexture(e)}),[t,n,r,a,c,u]);return i.createElement("meshLambertMaterial",{map:p,"map-anisotropy":d.capabilities.getMaxAnisotropy()||1,attachArray:"material",color:e?l:"white",transparent:!0,opacity:h})},Ys=e=>{const{tweenCamera:t,raycast:r}=Cs(),[a,s]=i.useState(null);return i.createElement("mesh",{raycast:r,onPointerOut:e=>{e.stopPropagation(),s(null)},onPointerMove:e=>{e.stopPropagation(),s(Math.floor(e.faceIndex/2))},onClick:e.onClick||(e=>{e.stopPropagation(),t(e.face.normal)})},[...Array(6)].map(((t,r)=>i.createElement(Xs,n({key:r,index:r,hover:a===r},e)))),i.createElement("boxGeometry",null))},qs=({onClick:e,dimensions:t,position:r,hoverColor:n=ks})=>{const{tweenCamera:a,raycast:s}=Cs(),[o,l]=i.useState(!1);return i.createElement("mesh",{scale:1.01,position:r,raycast:s,onPointerOver:e=>{e.stopPropagation(),l(!0)},onPointerOut:e=>{e.stopPropagation(),l(!1)},onClick:e||(e=>{e.stopPropagation(),a(r)})},i.createElement("meshBasicMaterial",{color:o?n:"white",transparent:!0,opacity:.6,visible:o}),i.createElement("boxGeometry",{args:t}))},Ks=e=>i.createElement("group",{scale:[60,60,60]},i.createElement(Ys,e),js.map(((t,r)=>i.createElement(qs,n({key:r,position:t,dimensions:Ws[r]},e)))),Hs.map(((t,r)=>i.createElement(qs,n({key:r,position:t,dimensions:Qs},e)))),i.createElement("ambientLight",{intensity:.5}),i.createElement("pointLight",{position:[10,10,10],intensity:.5}));function Js({scale:e=[.8,.05,.05],color:t,rotation:r}){return i.createElement("group",{rotation:r},i.createElement("mesh",{position:[.4,0,0]},i.createElement("boxGeometry",{args:e}),i.createElement("meshBasicMaterial",{color:t,toneMapped:!1})))}function Zs({onClick:e,font:t,disabled:r,arcStyle:a,label:l,labelColor:c,axisHeadScale:u=1,...h}){const d=(0,o.useThree)((e=>e.gl)),p=i.useMemo((()=>{const e=document.createElement("canvas");e.width=64,e.height=64;const r=e.getContext("2d");return r.beginPath(),r.arc(32,32,16,0,2*Math.PI),r.closePath(),r.fillStyle=a,r.fill(),l&&(r.font=t,r.textAlign="center",r.fillStyle=c,r.fillText(l,32,41)),new s.CanvasTexture(e)}),[a,l,c,t]),[f,m]=i.useState(!1),g=(l?1:.75)*(f?1.2:1)*u;return i.createElement("sprite",n({scale:g,onPointerOver:r?void 0:e=>{e.stopPropagation(),m(!0)},onPointerOut:r?void 0:e||(e=>{e.stopPropagation(),m(!1)})},h),i.createElement("spriteMaterial",{map:p,"map-anisotropy":d.capabilities.getMaxAnisotropy()||1,alphaTest:.3,opacity:l?1:.75,toneMapped:!1}))}const $s=({hideNegativeAxes:e,hideAxisHeads:t,disabled:r,font:a="18px Inter var, Arial, sans-serif",axisColors:s=["#ff3653","#0adb50","#2c8fdf"],axisHeadScale:o=1,axisScale:l,labels:c=["X","Y","Z"],labelColor:u="#000",onClick:h,...d})=>{const[p,f,m]=s,{tweenCamera:g,raycast:v}=Cs(),A={font:a,disabled:r,labelColor:u,raycast:v,onClick:h,axisHeadScale:o,onPointerDown:r?void 0:e=>{g(e.object.position),e.stopPropagation()}};return i.createElement("group",n({scale:40},d),i.createElement(Js,{color:p,rotation:[0,0,0],scale:l}),i.createElement(Js,{color:f,rotation:[0,0,Math.PI/2],scale:l}),i.createElement(Js,{color:m,rotation:[0,-Math.PI/2,0],scale:l}),!t&&i.createElement(i.Fragment,null,i.createElement(Zs,n({arcStyle:p,position:[1,0,0],label:c[0]},A)),i.createElement(Zs,n({arcStyle:f,position:[0,1,0],label:c[1]},A)),i.createElement(Zs,n({arcStyle:m,position:[0,0,1],label:c[2]},A)),!e&&i.createElement(i.Fragment,null,i.createElement(Zs,n({arcStyle:p,position:[-1,0,0]},A)),i.createElement(Zs,n({arcStyle:f,position:[0,-1,0]},A)),i.createElement(Zs,n({arcStyle:m,position:[0,0,-1]},A)))),i.createElement("ambientLight",{intensity:.5}),i.createElement("pointLight",{position:[10,10,10],intensity:.5}))};function eo(e,{path:t}){const[r]=(0,o.useLoader)(s.CubeTextureLoader,[e],(e=>e.setPath(t)));return r}eo.preload=(e,{path:t})=>o.useLoader.preload(s.CubeTextureLoader,[e],(e=>e.setPath(t)));var to=r(43953);function ro(e){return(0,o.useLoader)(to.y,e)}ro.preload=e=>o.useLoader.preload(to.y,e),ro.clear=e=>o.useLoader.clear(to.y,e);var no=r(44976),io=r(46295),ao=r(17607);let so=null;function oo(e,t,r){return n=>{r&&r(n),e&&(so||(so=new no._),so.setDecoderPath("string"==typeof e?e:"https://www.gstatic.com/draco/versioned/decoders/1.4.3/"),n.setDRACOLoader(so)),t&&n.setMeshoptDecoder("function"==typeof io.z?(0,io.z)():io.z)}}function lo(e,t=!0,r=!0,n){return(0,o.useLoader)(ao.E,e,oo(t,r,n))}lo.preload=(e,t=!0,r=!0,n)=>o.useLoader.preload(ao.E,e,oo(t,r,n)),lo.clear=e=>o.useLoader.clear(ao.E,e);var co=r(28934);const uo="https://cdn.jsdelivr.net/gh/pmndrs/drei-assets@master";function ho(e,t=`${uo}/basis/`){const r=(0,o.useThree)((e=>e.gl)),n=(0,o.useLoader)(co.a,ka(e)?Object.values(e):e,(e=>{e.detectSupport(r),e.setTranscoderPath(t)}));if((0,i.useEffect)((()=>{(Array.isArray(n)?n:[n]).forEach(r.initTexture)}),[r,n]),ka(e)){const t=Object.keys(e),r={};return t.forEach((e=>Object.assign(r,{[e]:n[t.indexOf(e)]}))),r}return n}ho.preload=(e,t=`${uo}/basis/`)=>o.useLoader.preload(co.a,e,(e=>{e.setTranscoderPath(t)})),ho.clear=e=>o.useLoader.clear(co.a,e);var po=r(73466),fo=r.n(po);function mo(e,t){"function"==typeof e?e(t):null!=e&&(e.current=t)}function go({showPanel:e=0,className:t,parent:r}){const n=function(e,t=[],r){const[n,a]=i.useState();return i.useLayoutEffect((()=>{const t=e();return a(t),mo(r,t),()=>mo(r,null)}),t),n}((()=>new(fo())),[]);return i.useEffect((()=>{if(n){const i=r&&r.current||document.body;n.showPanel(e),null==i||i.appendChild(n.dom),t&&n.dom.classList.add(...t.split(" ").filter((e=>e)));const a=(0,o.addEffect)((()=>n.begin())),s=(0,o.addAfterEffect)((()=>n.end()));return()=>{null==i||i.removeChild(n.dom),a(),s()}}}),[r,n,t,e]),null}function vo(e,t,r){const{gl:n,size:a}=(0,o.useThree)(),l=i.useMemo((()=>n.getPixelRatio()),[n]),c="number"==typeof e?e:a.width*l,u="number"==typeof t?t:a.height*l,h=("number"==typeof e?r:e)||{},{samples:d,...p}=h,f=i.useMemo((()=>{let e;return e=new s.WebGLRenderTarget(c,u,{minFilter:s.LinearFilter,magFilter:s.LinearFilter,encoding:n.outputEncoding,type:s.HalfFloatType,...p}),e.samples=d,e}),[]);return i.useLayoutEffect((()=>{f.setSize(c,u),d&&(f.samples=d)}),[d,f,c,u]),i.useEffect((()=>()=>f.dispose()),[]),f}function Ao({size:e=256,frames:t=1/0}={}){const r=(0,o.useThree)((e=>e.viewport.dpr)),{width:n,height:a}=(0,o.useThree)((e=>e.size)),l=e||n*r,c=e||a*r,u=i.useMemo((()=>{const e=new s.DepthTexture(l,c);return e.format=s.DepthFormat,e.type=s.UnsignedShortType,{depthTexture:e}}),[l,c]);let h=0;const d=vo(l,c,u);return(0,o.useFrame)((e=>{(t===1/0||he.viewport)),i=t*(n.aspect>e/t?n.width/e:n.height/t);return[e*(n.aspect>e/t?n.width/e:n.height/t)*r,i*r,1]}function xo(e,t,r,n){return new(r||(r=Promise))((function(i,a){function s(e){try{l(n.next(e))}catch(e){a(e)}}function o(e){try{l(n.throw(e))}catch(e){a(e)}}function l(e){var t;e.done?i(e.value):(t=e.value,t instanceof r?t:new r((function(e){e(t)}))).then(s,o)}l((n=n.apply(e,t||[])).next())}))}const bo=["geforce 320m","geforce 8600","geforce 8600m gt","geforce 8800 gs","geforce 8800 gt","geforce 9400","geforce 9400m g","geforce 9400m","geforce 9600m gt","geforce 9600m","geforce fx go5200","geforce gt 120","geforce gt 130","geforce gt 330m","geforce gtx 285","google swiftshader","intel g41","intel g45","intel gma 4500mhd","intel gma x3100","intel hd 3000","intel q45","legacy","mali-2","mali-3","mali-4","quadro fx 1500","quadro fx 4","quadro fx 5","radeon hd 2400","radeon hd 2600","radeon hd 4670","radeon hd 4850","radeon hd 4870","radeon hd 5670","radeon hd 5750","radeon hd 6290","radeon hd 6300","radeon hd 6310","radeon hd 6320","radeon hd 6490m","radeon hd 6630m","radeon hd 6750m","radeon hd 6770m","radeon hd 6970m","sgx 543","sgx543"];function wo(e){return e.toLowerCase().replace(/^angle ?\((.+)\)*$/,"$1").replace(/\s(\d{1,2}gb|direct3d.+$)|\(r\)| \([^)]+\)$/g,"").replace(/(?:vulkan|opengl) \d+\.\d+(?:\.\d+)?(?: \((.*)\))?/,"$1")}const _o="undefined"==typeof window,Eo=(()=>{if(_o)return;const{userAgent:e,platform:t,maxTouchPoints:r}=window.navigator,n=/(iphone|ipod|ipad)/i.test(e),i="iPad"===t||"MacIntel"===t&&r>0&&!window.MSStream;return{isIpad:i,isMobile:/android/i.test(e)||n||i,isSafari12:/Version\/12.+Safari/.test(e)}})();const So=[],Mo=[];function To(e,t){if(e===t)return 0;const r=e;e.length>t.length&&(e=t,t=r);let n=e.length,i=t.length;for(;n>0&&e.charCodeAt(~-n)===t.charCodeAt(~-i);)n--,i--;let a,s=0;for(;sc?l>c?c+1:l:l>o?o+1:l;return c}function Co(e){return null!=e}class Io extends Error{constructor(e){super(e),Object.setPrototypeOf(this,new.target.prototype)}}const Bo=({mobileTiers:e=[0,15,30,60],desktopTiers:t=[0,15,30,60],override:r={},glContext:n,failIfMajorPerformanceCaveat:i=!1,benchmarksURL:a="https://unpkg.com/detect-gpu@4.0.50/dist/benchmarks"}={})=>xo(void 0,void 0,void 0,(function*(){const s={};if(_o)return{tier:0,type:"SSR"};const{isIpad:o=!!(null==Eo?void 0:Eo.isIpad),isMobile:l=!!(null==Eo?void 0:Eo.isMobile),screenSize:c=window.screen,loadBenchmarks:u=(e=>xo(void 0,void 0,void 0,(function*(){const t=yield fetch(`${a}/${e}`).then((e=>e.json()));if(parseInt(t.shift().split(".")[0],10)<4)throw new Io("Detect GPU benchmark data is out of date. Please update to version 4x");return t})))}=r;let{renderer:h}=r;const d=(e,t,r,n,i)=>({device:i,fps:n,gpu:r,isMobile:l,tier:e,type:t});let p,f="";if(h)h=wo(h),p=[h];else{const e=n||function(e,t=!1){const r={alpha:!1,antialias:!1,depth:!1,failIfMajorPerformanceCaveat:t,powerPreference:"high-performance",stencil:!1};e&&delete r.powerPreference;const n=window.document.createElement("canvas"),i=n.getContext("webgl",r)||n.getContext("experimental-webgl",r);return null!=i?i:void 0}(null==Eo?void 0:Eo.isSafari12,i);if(!e)return d(0,"WEBGL_UNSUPPORTED");const t=e.getExtension("WEBGL_debug_renderer_info");if(t&&(h=e.getParameter(t.UNMASKED_RENDERER_WEBGL)),!h)return d(1,"FALLBACK");f=h,h=wo(h),p=function(e,t,r){return"apple gpu"===t?function(e,t,r){if(!r)return[t];const n=function(e){const t=e.createShader(35633),r=e.createShader(35632),n=e.createProgram();if(!(r&&t&&n))return;e.shaderSource(t,"\n precision highp float;\n attribute vec3 aPosition;\n varying float vvv;\n void main() {\n vvv = 0.31622776601683794;\n gl_Position = vec4(aPosition, 1.0);\n }\n "),e.shaderSource(r,"\n precision highp float;\n varying float vvv;\n void main() {\n vec4 enc = vec4(1.0, 255.0, 65025.0, 16581375.0) * vvv;\n enc = fract(enc);\n enc -= enc.yzww * vec4(1.0 / 255.0, 1.0 / 255.0, 1.0 / 255.0, 0.0);\n gl_FragColor = enc;\n }\n "),e.compileShader(t),e.compileShader(r),e.attachShader(n,t),e.attachShader(n,r),e.linkProgram(n),e.detachShader(n,t),e.detachShader(n,r),e.deleteShader(t),e.deleteShader(r),e.useProgram(n);const i=e.createBuffer();e.bindBuffer(34962,i),e.bufferData(34962,new Float32Array([-1,-1,0,3,-1,0,-1,3,0]),35044);const a=e.getAttribLocation(n,"aPosition");e.vertexAttribPointer(a,3,5126,!1,0,0),e.enableVertexAttribArray(a),e.clearColor(1,1,1,1),e.clear(16384),e.viewport(0,0,1,1),e.drawArrays(4,0,3);const s=new Uint8Array(4);return e.readPixels(0,0,1,1,6408,5121,s),e.deleteProgram(n),e.deleteBuffer(i),s.join("")}(e),i="801621810",a="8016218135",s="80162181161",o=(null==Eo?void 0:Eo.isIpad)?[["a7",s,12],["a8",a,15],["a8x",a,15],["a9",a,15],["a9x",a,15],["a10",a,15],["a10x",a,15],["a12",i,15],["a12x",i,15],["a12z",i,15],["a14",i,15],["m1",i,15]]:[["a7",s,12],["a8",a,12],["a9",a,15],["a10",a,15],["a11",i,15],["a12",i,15],["a13",i,15],["a14",i,15]];let l;return"80162181255"===n?l=o.filter((([,,e])=>e>=14)):(l=o.filter((([,e])=>e===n)),l.length||(l=o)),l.map((([e])=>`apple ${e} gpu`))}(e,t,r):[t]}(e,h,l)}const m=(yield Promise.all(p.map((function(e){var t;return xo(this,void 0,void 0,(function*(){const r=(e=>{const t=l?["adreno","apple","mali-t","mali","nvidia","powervr"]:["intel","apple","amd","radeon","nvidia","geforce"];for(const r of t)if(e.includes(r))return r})(e);if(!r)return;const n=`${l?"m":"d"}-${r}${o?"-ipad":""}.json`,i=s[n]=null!==(t=s[n])&&void 0!==t?t:u(n);let a;try{a=yield i}catch(r){if(r instanceof Io)throw r;return}const h=function(e){var t;const r=(e=e.replace(/\([^)]+\)/,"")).match(/\d+/)||e.match(/(\W|^)([A-Za-z]{1,3})(\W|$)/g);return null!==(t=null==r?void 0:r.join("").replace(/\W|amd/g,""))&&void 0!==t?t:""}(e);let d=a.filter((([,e])=>e===h));d.length||(d=a.filter((([t])=>t.includes(e))));const p=d.length;if(0===p)return;let f,[m,,,g]=p>1?d.map((t=>[t,To(e,t[0])])).sort((([,e],[,t])=>e-t))[0][0]:d[0],v=Number.MAX_VALUE;const{devicePixelRatio:A}=window,y=c.width*A*c.height*A;for(const e of g){const[t,r]=e,n=t*r,i=Math.abs(y-n);ie===r?t-n:e-r));if(!m.length){const e=bo.find((e=>h.includes(e)));return e?d(0,"BLOCKLISTED",e):d(1,"FALLBACK",`${h} (${f})`)}const[,g,v,A]=m[0];if(-1===g)return d(0,"BLOCKLISTED",v,g,A);const y=l?e:t;let x=0;for(let e=0;e=y[e]&&(x=e);return d(x,"BENCHMARK",v,g,A)})),Ro=e=>Ca((()=>Bo(e)),["useDetectGPU"]);function Po(e,t,...r){const n=i.useRef(),a=(0,o.useThree)((e=>e.scene));return i.useEffect((()=>(e&&t&&null!=e&&e.current&&(n.current=new t(e.current,...r),n.current&&a.add(n.current)),!e&&n.current&&a.remove(n.current),()=>{n.current&&a.remove(n.current)})),[a,t,e,r]),(0,o.useFrame)((()=>{var e;null!=(e=n.current)&&e.update&&n.current.update()})),n}const Fo=new s.Vector3,Lo=new s.Vector3,Do=new s.Vector3,Uo=new s.Vector2,No=new s.Vector2,ko=new s.Vector2,Oo=new s.Vector3;function Go(e,t,r,n,i,a,o){Fo.fromBufferAttribute(t,n),Lo.fromBufferAttribute(t,i),Do.fromBufferAttribute(t,a);const l=function(e,t,r,n,i,a){let o;return o=a===s.BackSide?e.intersectTriangle(n,r,t,!0,i):e.intersectTriangle(t,r,n,a!==s.DoubleSide,i),null===o?null:{distance:e.origin.distanceTo(i),point:i.clone()}}(e,Fo,Lo,Do,Oo,o);if(l){r&&(Uo.fromBufferAttribute(r,n),No.fromBufferAttribute(r,i),ko.fromBufferAttribute(r,a),l.uv=s.Triangle.getUV(Oo,Fo,Lo,Do,Uo,No,ko,new s.Vector2));const e={a:n,b:i,c:a,normal:new s.Vector3,materialIndex:0};s.Triangle.getNormal(Fo,Lo,Do,e.normal),l.face=e,l.faceIndex=n}return l}function zo(e,t,r,n,i){const a=3*n,s=e.index.getX(a),o=e.index.getX(a+1),l=e.index.getX(a+2),c=Go(r,e.attributes.position,e.attributes.uv,s,o,l,t);return c?(c.faceIndex=n,i&&i.push(c),c):null}function Vo(e,t,r){return null===e?null:(e.point.applyMatrix4(t.matrixWorld),e.distance=e.point.distanceTo(r.ray.origin),e.object=t,e.distancer.far?null:e)}const Ho=1.25,Qo=65535,jo=Math.pow(2,-24);class Wo{constructor(){}}function Xo(e,t,r){return r.min.x=t[e],r.min.y=t[e+1],r.min.z=t[e+2],r.max.x=t[e+3],r.max.y=t[e+4],r.max.z=t[e+5],r}function Yo(e){let t=-1,r=-1/0;for(let n=0;n<3;n++){const i=e[n+3]-e[n];i>r&&(r=i,t=n)}return t}function qo(e,t){t.set(e)}function Ko(e,t,r){let n,i;for(let a=0;a<3;a++){const s=a+3;n=e[a],i=t[a],r[a]=ni?n:i}}function Jo(e,t,r){for(let n=0;n<3;n++){const i=t[e+2*n],a=t[e+2*n+1],s=i-a,o=i+a;sr[n+3]&&(r[n+3]=o)}}function Zo(e){const t=e[3]-e[0],r=e[4]-e[1],n=e[5]-e[2];return 2*(t*r+r*n+n*t)}function $o(e,t,r,n,i=null){let a=1/0,s=1/0,o=1/0,l=-1/0,c=-1/0,u=-1/0,h=1/0,d=1/0,p=1/0,f=-1/0,m=-1/0,g=-1/0;const v=null!==i;for(let A=6*t,y=6*(t+r);Al&&(l=i),v&&tf&&(f=t);const y=e[A+2],x=e[A+3],b=y-x,w=y+x;bc&&(c=w),v&&ym&&(m=y);const _=e[A+4],E=e[A+5],S=_-E,M=_+E;Su&&(u=M),v&&_g&&(g=_)}n[0]=a,n[1]=s,n[2]=o,n[3]=l,n[4]=c,n[5]=u,v&&(i[0]=h,i[1]=d,i[2]=p,i[3]=f,i[4]=m,i[5]=g)}const el=32,tl=(e,t)=>e.candidate-t.candidate,rl=new Array(el).fill().map((()=>({count:0,bounds:new Float32Array(6),rightCacheBounds:new Float32Array(6),leftCacheBounds:new Float32Array(6),candidate:0}))),nl=new Float32Array(6);function il(e,t){function r(e){p&&p(e/f)}function n(t,i,s,p=null,f=0){if(!m&&f>=c&&(m=!0,u&&(console.warn(`MeshBVH: Max depth of ${c} reached when generating BVH. Consider increasing maxDepth.`),console.warn(e))),s<=h||f>=c)return r(i+s),t.offset=i,t.count=s,t;const g=function(e,t,r,n,i,a){let s=-1,o=0;if(0===a)s=Yo(t),-1!==s&&(o=(t[s]+t[s+3])/2);else if(1===a)s=Yo(e),-1!==s&&(o=function(e,t,r,n){let i=0;for(let a=t,s=t+r;a=a.candidate?Jo(i,r,a.rightCacheBounds):(Jo(i,r,a.leftCacheBounds),a.count++)}}for(let r=0;r=el&&(t=31);const a=rl[t];a.count++,Jo(i,r,a.bounds)}const t=rl[31];qo(t.bounds,t.rightCacheBounds);for(let e=30;e>=0;e--){const t=rl[e],r=rl[e+1];Ko(t.bounds,r.rightCacheBounds,t.rightCacheBounds)}let d=0;for(let r=0;r<31;r++){const t=rl[r],n=t.count,c=t.bounds,u=rl[r+1].rightCacheBounds;0!==n&&(0===d?qo(c,nl):Ko(c,nl,nl)),d+=n;let h=0,p=0;0!==d&&(h=Zo(nl)/a);const f=i-d;0!==f&&(p=Zo(u)/a);const m=1+Ho*(h*d+p*f);m=o;)s--;if(!(a65535?new Uint32Array(new n(4*r)):new Uint16Array(new n(2*r)),e.setIndex(new s.BufferAttribute(i,1));for(let e=0;em&&(m=l),c>m&&(m=c);const g=(m-h)/2,v=2*n;a[i+v+0]=h+g,a[i+v+1]=g+(Math.abs(h)+g)*jo,ht[n+3]&&(t[n+3]=m)}}return a}(e,i),l=e.index.array,c=t.maxDepth,u=t.verbose,h=t.maxLeafTris,d=t.strategy,p=t.onProgress,f=e.index.count/3;let m=!1;const g=[],v=function(e){if(!e.groups||!e.groups.length)return[{offset:0,count:e.index.count/3}];const t=[],r=new Set;for(const i of e.groups)r.add(i.start),r.add(i.start+i.count);const n=Array.from(r.values()).sort(((e,t)=>e-t));for(let i=0;io&&(o=t);const r=e[u+2];rl&&(l=r);const n=e[u+4];nc&&(c=n)}n[0]=i,n[1]=a,n[2]=s,n[3]=o,n[4]=l,n[5]=c}(o,e.offset,e.count,a),n(t,e.offset,e.count,a),g.push(t)}else for(let s of v){const e=new Wo;e.boundingData=new Float32Array(6),$o(o,s.offset,s.count,e.boundingData,a),n(e,s.offset,s.count,a),g.push(e)}return g}class al{constructor(){this.min=1/0,this.max=-1/0}setFromPointsField(e,t){let r=1/0,n=-1/0;for(let i=0,a=e.length;in?a:n}this.min=r,this.max=n}setFromPoints(e,t){let r=1/0,n=-1/0;for(let i=0,a=t.length;in?s:n}this.min=r,this.max=n}isSeparated(e){return this.min>e.max||e.min>this.max}}al.prototype.setFromBox=function(){const e=new s.Vector3;return function(t,r){const n=r.min,i=r.max;let a=1/0,s=-1/0;for(let o=0;o<=1;o++)for(let r=0;r<=1;r++)for(let l=0;l<=1;l++){e.x=n.x*o+i.x*(1-o),e.y=n.y*r+i.y*(1-r),e.z=n.z*l+i.z*(1-l);const c=t.dot(e);a=Math.min(c,a),s=Math.max(c,s)}this.min=a,this.max=s}}();!function(){const e=new al}();const sl=function(){const e=new s.Vector3,t=new s.Vector3,r=new s.Vector3;return function(n,i,a){const s=n.start,o=e,l=i.start,c=t;r.subVectors(s,l),e.subVectors(n.end,n.start),t.subVectors(i.end,i.start);const u=r.dot(c),h=c.dot(o),d=c.dot(c),p=r.dot(o),f=o.dot(o)*d-h*h;let m,g;m=0!==f?(u*h-p*d)/f:0,g=(u+m*h)/d,a.x=m,a.y=g}}(),ol=function(){const e=new s.Vector2,t=new s.Vector3,r=new s.Vector3;return function(n,i,a,s){sl(n,i,e);let o=e.x,l=e.y;if(o>=0&&o<=1&&l>=0&&l<=1)return n.at(o,a),void i.at(l,s);if(o>=0&&o<=1)return l<0?i.at(0,s):i.at(1,s),void n.closestPointToPoint(s,!0,a);if(l>=0&&l<=1)return o<0?n.at(0,a):n.at(1,a),void i.closestPointToPoint(a,!0,s);{let e,c;e=o<0?n.start:n.end,c=l<0?i.start:i.end;const u=t,h=r;return n.closestPointToPoint(c,!0,t),i.closestPointToPoint(e,!0,r),u.distanceToSquared(c)<=h.distanceToSquared(e)?(a.copy(u),void s.copy(c)):(a.copy(e),void s.copy(h))}}}(),ll=function(){const e=new s.Vector3,t=new s.Vector3,r=new s.Plane,n=new s.Line3;return function(i,a){const{radius:s,center:o}=i,{a:l,b:c,c:u}=a;n.start=l,n.end=c;if(n.closestPointToPoint(o,!0,e).distanceTo(o)<=s)return!0;n.start=l,n.end=u;if(n.closestPointToPoint(o,!0,e).distanceTo(o)<=s)return!0;n.start=c,n.end=u;if(n.closestPointToPoint(o,!0,e).distanceTo(o)<=s)return!0;const h=a.getPlane(r);if(Math.abs(h.distanceToPoint(o))<=s){const e=h.projectPoint(o,t);if(a.containsPoint(e))return!0}return!1}}();function cl(e){return Math.abs(e)<1e-15}class ul extends s.Triangle{constructor(...e){super(...e),this.isExtendedTriangle=!0,this.satAxes=new Array(4).fill().map((()=>new s.Vector3)),this.satBounds=new Array(4).fill().map((()=>new al)),this.points=[this.a,this.b,this.c],this.sphere=new s.Sphere,this.plane=new s.Plane,this.needsUpdate=!0}intersectsSphere(e){return ll(e,this)}update(){const e=this.a,t=this.b,r=this.c,n=this.points,i=this.satAxes,a=this.satBounds,s=i[0],o=a[0];this.getNormal(s),o.setFromPoints(s,n);const l=i[1],c=a[1];l.subVectors(e,t),c.setFromPoints(l,n);const u=i[2],h=a[2];u.subVectors(t,r),h.setFromPoints(u,n);const d=i[3],p=a[3];d.subVectors(r,e),p.setFromPoints(d,n),this.sphere.setFromPoints(this.points),this.plane.setFromNormalAndCoplanarPoint(s,e),this.needsUpdate=!1}}ul.prototype.closestPointToSegment=function(){const e=new s.Vector3,t=new s.Vector3,r=new s.Line3;return function(n,i=null,a=null){const{start:s,end:o}=n,l=this.points;let c,u=1/0;for(let h=0;h<3;h++){const s=(h+1)%3;r.start.copy(l[h]),r.end.copy(l[s]),ol(r,n,e,t),c=e.distanceToSquared(t),c1-1e-10){const e=this.satBounds,o=this.satAxes;r[0]=s.a,r[1]=s.b,r[2]=s.c;for(let t=0;t<4;t++){const i=e[t],a=o[t];if(n.setFromPoints(a,r),i.isSeparated(n))return!1}const l=s.satBounds,c=s.satAxes;t[0]=this.a,t[1]=this.b,t[2]=this.c;for(let r=0;r<4;r++){const e=l[r],i=c[r];if(n.setFromPoints(i,t),e.isSeparated(n))return!1}for(let s=0;s<4;s++){const e=o[s];for(let s=0;s<4;s++){const o=c[s];if(a.crossVectors(e,o),n.setFromPoints(a,t),i.setFromPoints(a,r),n.isSeparated(i))return!1}}return p&&(f||console.warn("ExtendedTriangle.intersectsTriangle: Triangles are coplanar which does not support an output edge. Setting edge to 0, 0, 0."),p.start.set(0,0,0),p.end.set(0,0,0)),!0}{const e=this.points;let t=!1,r=0;for(let s=0;s<3;s++){const n=e[s],i=e[(s+1)%3];u.start.copy(n),u.end.copy(i),u.delta(o);const a=t?h.start:h.end,l=cl(g.distanceToPoint(n));if(cl(g.normal.dot(o))&&l){h.copy(u),r=2;break}if((g.intersectLine(u,a)||l)&&!cl(a.distanceTo(i))){if(r++,t)break;t=!0}}if(1===r&&s.containsPoint(h.end))return p&&(p.start.copy(h.end),p.end.copy(h.end)),!0;if(2!==r)return!1;const n=s.points;let i=!1,a=0;for(let s=0;s<3;s++){const e=n[s],t=n[(s+1)%3];u.start.copy(e),u.end.copy(t),u.delta(l);const r=i?d.start:d.end,o=cl(m.distanceToPoint(e));if(cl(m.normal.dot(l))&&o){d.copy(u),a=2;break}if((m.intersectLine(u,r)||o)&&!cl(r.distanceTo(t))){if(a++,i)break;i=!0}}if(1===a&&this.containsPoint(d.end))return p&&(p.start.copy(d.end),p.end.copy(d.end)),!0;if(2!==a)return!1;if(h.delta(o),d.delta(l),o.dot(l)<0){let e=d.start;d.start=d.end,d.end=e}const f=h.start.dot(o),v=h.end.dot(o),A=d.start.dot(o),y=d.end.dot(o),x=v0?p.start.copy(h.start):p.start.copy(d.start),c.subVectors(h.end,d.end),c.dot(o)<0?p.end.copy(h.end):p.end.copy(d.end)),!0)}}}(),ul.prototype.distanceToPoint=function(){const e=new s.Vector3;return function(t){return this.closestPointToPoint(t,e),t.distanceTo(e)}}(),ul.prototype.distanceToTriangle=function(){const e=new s.Vector3,t=new s.Vector3,r=["a","b","c"],n=new s.Line3,i=new s.Line3;return function(a,s=null,o=null){const l=s||o?n:null;if(this.intersectsTriangle(a,l))return(s||o)&&(s&&l.getCenter(s),o&&l.getCenter(o)),0;let c=1/0;for(let t=0;t<3;t++){let n;const i=r[t],l=a[i];this.closestPointToPoint(l,e),n=l.distanceToSquared(e),nnew s.Vector3)),this.satAxes=new Array(3).fill().map((()=>new s.Vector3)),this.satBounds=new Array(3).fill().map((()=>new al)),this.alignedSatBounds=new Array(3).fill().map((()=>new al)),this.needsUpdate=!1,e&&this.min.copy(e),t&&this.max.copy(t),r&&this.matrix.copy(r)}set(e,t,r){this.min.copy(e),this.max.copy(t),this.matrix.copy(r),this.needsUpdate=!0}copy(e){this.min.copy(e.min),this.max.copy(e.max),this.matrix.copy(e.matrix),this.needsUpdate=!0}}function dl(e,t,r,n){const i=e.a,a=e.b,s=e.c;let o=t,l=t+1,c=t+2;r&&(o=r.getX(t),l=r.getX(t+1),c=r.getX(t+2)),i.x=n.getX(o),i.y=n.getY(o),i.z=n.getZ(o),a.x=n.getX(l),a.y=n.getY(l),a.z=n.getZ(l),s.x=n.getX(c),s.y=n.getY(c),s.z=n.getZ(c)}function pl(e,t,r,n,i,a,s){const o=r.index,l=r.attributes.position;for(let c=e,u=t+e;cnew s.Line3)),r=new Array(12).fill().map((()=>new s.Line3)),n=new s.Vector3,i=new s.Vector3;return function(a,s=0,o=null,l=null){if(this.needsUpdate&&this.update(),this.intersectsBox(a))return(o||l)&&(a.getCenter(i),this.closestPointToPoint(i,n),a.closestPointToPoint(n,i),o&&o.copy(n),l&&l.copy(i)),0;const c=s*s,u=a.min,h=a.max,d=this.points;let p=1/0;for(let e=0;e<8;e++){const t=d[e];i.copy(t).clamp(u,h);const r=t.distanceToSquared(i);if(r=0;let c,u;l?(c=Al(e),u=yl(e,o)):(c=yl(e,o),u=Al(e));const h=Tl(c,a,n,bl)?El(c,t,r,n):null;if(h){const e=h.point[s];if(l?e<=a[u+i]:e>=a[u+i+3])return h}const d=Tl(u,a,n,bl)?El(u,t,r,n):null;return h&&d?h.distance<=d.distance?h:d:h||d||null}}const Sl=function(){let e,t;const r=[],n=new fl((()=>new s.Box3));return function(...a){e=n.getPrimitive(),t=n.getPrimitive(),r.push(e,t);const s=i(...a);n.releasePrimitive(e),n.releasePrimitive(t),r.pop(),r.pop();const o=r.length;return o>0&&(t=r[o-1],e=r[o-2]),s};function i(r,n,a,s,o=null,l=0,c=0){function u(e){let t=2*e,r=Rl,n=Pl;for(;!ml(t,r);)t=2*(e=Al(e));return gl(e,n)}function h(e){let t=2*e,r=Rl,n=Pl;for(;!ml(t,r);)t=2*(e=yl(e,n));return gl(e,n)+vl(t,r)}let d=2*r,p=Bl,f=Rl,m=Pl;if(ml(d,f)){const t=gl(r,m),n=vl(d,f);return Xo(r,p,e),s(t,n,!1,c,l+r,e)}{const d=Al(r),g=yl(r,m);let v,A,y,x,b=d,w=g;if(o&&(y=e,x=t,Xo(b,p,y),Xo(w,p,x),v=o(y),A=o(x),Ai.intersectsBox(e),intersectsTriangle:e=>{e.a.applyMatrix4(c),e.b.applyMatrix4(c),e.c.applyMatrix4(c),e.needsUpdate=!0;for(let r=3*v,n=3*(A+v);rnew ul));class Yl{static serialize(e,t={}){if(t.isBufferGeometry)return console.warn("MeshBVH.serialize: The arguments for the function have changed. See documentation for new signature."),Yl.serialize(arguments[0],{cloneBuffers:void 0===arguments[2]||arguments[2]});t={cloneBuffers:!0,...t};const r=e.geometry,n=e._roots,i=r.getIndex();let a;return a=t.cloneBuffers?{roots:n.map((e=>e.slice())),index:i.array.slice()}:{roots:n,index:i.array},a}static deserialize(e,t,r={}){if("boolean"==typeof r)return console.warn("MeshBVH.deserialize: The arguments for the function have changed. See documentation for new signature."),Yl.deserialize(arguments[0],arguments[1],{setIndex:void 0===arguments[2]||arguments[2]});r={setIndex:!0,...r};const{index:n,roots:i}=e,a=new Yl(t,{...r,[Dl]:!0});if(a._roots=i,r.setIndex){const r=t.getIndex();if(null===r){const r=new s.BufferAttribute(e.index,1,!1);t.setIndex(r)}else r.array!==n&&(r.array.set(n),r.needsUpdate=!0)}return a}constructor(e,t={}){if(!e.isBufferGeometry)throw new Error("MeshBVH: Only BufferGeometries are supported.");if(e.index&&e.index.isInterleavedBufferAttribute)throw new Error("MeshBVH: InterleavedBufferAttribute is not supported for the index attribute.");if((t=Object.assign({strategy:0,maxDepth:40,maxLeafTris:10,verbose:!0,useSharedArrayBuffer:!1,setBoundingBox:!0,onProgress:null,[Dl]:!1},t)).useSharedArrayBuffer&&"undefined"==typeof SharedArrayBuffer)throw new Error("MeshBVH: SharedArrayBuffer is not available.");this._roots=null,t[Dl]||(this._roots=function(e,t){const r=il(e,t);let n,i,a;const s=[],o=t.useSharedArrayBuffer?SharedArrayBuffer:ArrayBuffer;for(let u=0;uMath.pow(2,32))throw new Error("MeshBVH: Cannot store child pointer greater than 32 bits.");return i[r+6]=o/4,o=c(o,a),i[r+7]=s,o}}}(e,t),!e.boundingBox&&t.setBoundingBox&&(e.boundingBox=this.getBoundingBox(new s.Box3))),this.geometry=e}refit(e=null){e&&Array.isArray(e)&&(e=new Set(e));const t=this.geometry,r=t.index.array,n=t.attributes.position;let i,a,s,o,l=0;const c=this._roots;for(let h=0,d=c.length;hh&&(h=a),sd&&(d=s),op&&(p=o)}return(o[t+0]!==i||o[t+1]!==l||o[t+2]!==u||o[t+3]!==h||o[t+4]!==d||o[t+5]!==p)&&(o[t+0]=i,o[t+1]=l,o[t+2]=u,o[t+3]=h,o[t+4]=d,o[t+5]=p,!0)}{const r=t+8,n=a[t+6],s=r+i,c=n+i;let h=l,d=!1,p=!1;e?h||(d=e.has(s),p=e.has(c),h=!d&&!p):(d=!0,p=!0);const f=h||p;let m=!1;(h||d)&&(m=u(r,i,h));let g=!1;f&&(g=u(n,i,h));const v=m||g;if(v)for(let e=0;e<3;e++){const i=r+e,a=n+e,s=o[i],l=o[i+3],c=o[a],u=o[a+3];o[t+e]=su?l:u}return v}}}traverse(e,t=0){const r=this._roots[t],n=new Uint32Array(r),i=new Uint16Array(r);!function t(a,s=0){const o=2*a,l=i[o+15]===Qo;if(l){const t=n[a+6],c=i[o+14];e(s,l,new Float32Array(r,4*a,6),t,c)}else{const i=a+8,o=n[a+6],c=n[a+7];e(s,l,new Float32Array(r,4*a,6),c)||(t(i,s+1),t(o,s+1))}}(0)}raycast(e,t=s.FrontSide){const r=this._roots,n=this.geometry,i=[],a=t.isMaterial,o=Array.isArray(t),l=n.groups,c=a?t.side:t;for(let s=0,u=r.length;s{const a=3*r;return e(t,a,a+1,a+2,n,i)}}e={boundsTraverseOrder:r,intersectsBounds:e,intersectsTriangle:t,intersectsRange:null},console.warn("MeshBVH: Shapecast function signature has changed and now takes an object of callbacks as a second argument. See docs for new signature.")}const i=Xl.getPrimitive();let{boundsTraverseOrder:a,intersectsBounds:s,intersectsRange:o,intersectsTriangle:l}=e;if(o&&l){const e=o;o=(t,r,a,s,o)=>!!e(t,r,a,s,o)||pl(t,r,n,l,a,s,i)}else o||(o=l?(e,t,r,a)=>pl(e,t,n,l,r,a,i):(e,t,r)=>r);let c=!1,u=0;for(const h of this._roots){if(Fl(h),c=Sl(0,n,s,o,a,u),Ll(),c)break;u+=h.byteLength}return Xl.releasePrimitive(i),c}bvhcast(e,t,r){let{intersectsRanges:n,intersectsTriangles:i}=r;const a=this.geometry.index,s=this.geometry.attributes.position,o=e.geometry.index,l=e.geometry.attributes.position;kl.copy(t).invert();const c=Xl.getPrimitive(),u=Xl.getPrimitive();if(i){function d(e,r,n,h,d,p,f,m){for(let g=n,v=n+h;gNl.intersectsBox(e),intersectsRange:(t,r,i,a,s,o)=>(Ul.copy(o),Ul.applyMatrix4(kl),e.shapecast({intersectsBounds:e=>Ul.intersectsBox(e),intersectsRange:(e,i,o,l,c)=>n(t,r,e,i,a,s,l,c)}))});return Xl.releasePrimitive(c),Xl.releasePrimitive(u),h}intersectsBox(e,t){return Ol.set(e.min,e.max,t),Ol.needsUpdate=!0,this.shapecast({intersectsBounds:e=>Ol.intersectsBox(e),intersectsTriangle:e=>Ol.intersectsTriangle(e)})}intersectsSphere(e){return this.shapecast({intersectsBounds:t=>e.intersectsBox(t),intersectsTriangle:t=>t.intersectsSphere(e)})}closestPointToGeometry(e,t,r={},n={},i=0,a=1/0){e.boundingBox||e.computeBoundingBox(),Ol.set(e.boundingBox.min,e.boundingBox.max,t),Ol.needsUpdate=!0;const s=this.geometry,o=s.attributes.position,l=s.index,c=e.attributes.position,u=e.index,h=Xl.getPrimitive(),d=Xl.getPrimitive();let p=Vl,f=Hl,m=null,g=null;n&&(m=Ql,g=jl);let v=1/0,A=null,y=null;return kl.copy(t).invert(),Gl.matrix.copy(kl),this.shapecast({boundsTraverseOrder:e=>Ol.distanceToBox(e),intersectsBounds:(e,t,r)=>r{if(e.boundsTree)return e.boundsTree.shapecast({boundsTraverseOrder:e=>Gl.distanceToBox(e),intersectsBounds:(e,t,r)=>r{for(let s=3*e,x=3*(e+a);s(zl.copy(e).clamp(t.min,t.max),zl.distanceToSquared(e)),intersectsBounds:(e,t,r)=>r{t.closestPointToPoint(e,zl);const n=e.distanceToSquared(zl);return n{Xo(0,new Float32Array(t),Wl),e.union(Wl)})),e}}const ql=new s.Ray,Kl=new s.Matrix4,Jl=s.Mesh.prototype.raycast;function Zl(e,t){if(this.geometry.boundsTree){if(void 0===this.material)return;Kl.copy(this.matrixWorld).invert(),ql.copy(e.ray).applyMatrix4(Kl);const r=this.geometry.boundsTree;if(!0===e.firstHitOnly){const n=Vo(r.raycastFirst(ql,this.material),this,e);n&&t.push(n)}else{const n=r.raycast(ql,this.material);for(let r=0,i=n.length;r{if(e.current){e.current.raycast=Zl;const r=e.current.geometry;return r.computeBoundsTree=$l,r.disposeBoundsTree=ec,r.computeBoundsTree(t),()=>{r.boundsTree&&r.disposeBoundsTree()}}}),[e,t])}function rc(...e){const t=i.useRef([]);return t.current=e.map((e=>i.useContext(e))),i.useMemo((()=>({children:r})=>e.reduceRight(((e,r,n)=>i.createElement(r.Provider,{value:t.current[n],children:e})),r)),[])}function nc(e,t){const r=i.useRef(),[n]=i.useState((()=>t?t instanceof s.Object3D?{current:t}:t:r)),[a]=i.useState((()=>new s.AnimationMixer(void 0))),l=i.useRef({}),[c]=i.useState((()=>{const t={};return e.forEach((e=>Object.defineProperty(t,e.name,{enumerable:!0,get(){if(n.current)return l.current[e.name]||(l.current[e.name]=a.clipAction(e,n.current))}}))),{ref:n,clips:e,actions:t,names:e.map((e=>e.name)),mixer:a}}));return(0,o.useFrame)(((e,t)=>a.update(t))),i.useEffect((()=>{const e=n.current;return()=>{l.current={},Object.values(c.actions).forEach((t=>{e&&a.uncacheAction(t,e)}))}}),[e]),c}function ic(e){const t=i.useRef(null),r=i.useRef(!1),n=i.useRef(!1);return i.useEffect((()=>{const i=t.current;if(i){const t=(0,o.addEffect)((()=>(r.current=!1,!0))),a=i.onBeforeRender;i.onBeforeRender=()=>r.current=!0;const s=(0,o.addAfterEffect)((()=>(r.current!==n.current&&e(n.current=r.current),!0)));return()=>{i.onBeforeRender=a,t(),s()}}}),[]),t}function ac(e=new s.Vector3,t=new s.Vector3){const[r]=i.useState((()=>({position:new s.Vector3,size:new s.Vector3})));(0,o.applyProps)(r,{position:e,size:t});const n=i.useRef(null),a=i.useMemo((()=>({ref:n,onBeforeCompile:e=>function(e,t,r){e.defines.BOX_PROJECTED_ENV_MAP=!0,e.uniforms.envMapPosition={value:t},e.uniforms.envMapSize={value:r},e.vertexShader=`\n varying vec3 vWorldPosition;\n ${e.vertexShader.replace("#include ","\n#if defined( USE_ENVMAP ) || defined( DISTANCE ) || defined ( USE_SHADOWMAP )\n vec4 worldPosition = modelMatrix * vec4( transformed, 1.0 );\n #ifdef BOX_PROJECTED_ENV_MAP\n vWorldPosition = worldPosition.xyz;\n #endif\n#endif\n")}`,e.fragmentShader=`\n \n#ifdef BOX_PROJECTED_ENV_MAP\n uniform vec3 envMapSize;\n uniform vec3 envMapPosition;\n varying vec3 vWorldPosition;\n \n vec3 parallaxCorrectNormal( vec3 v, vec3 cubeSize, vec3 cubePos ) {\n vec3 nDir = normalize( v );\n vec3 rbmax = ( .5 * cubeSize + cubePos - vWorldPosition ) / nDir;\n vec3 rbmin = ( -.5 * cubeSize + cubePos - vWorldPosition ) / nDir;\n vec3 rbminmax;\n rbminmax.x = ( nDir.x > 0. ) ? rbmax.x : rbmin.x;\n rbminmax.y = ( nDir.y > 0. ) ? rbmax.y : rbmin.y;\n rbminmax.z = ( nDir.z > 0. ) ? rbmax.z : rbmin.z;\n float correction = min( min( rbminmax.x, rbminmax.y ), rbminmax.z );\n vec3 boxIntersection = vWorldPosition + nDir * correction; \n return boxIntersection - cubePos;\n }\n#endif\n\n ${e.fragmentShader.replace("#include ",s.ShaderChunk.envmap_physical_pars_fragment).replace("vec3 worldNormal = inverseTransformDirection( normal, viewMatrix );","vec3 worldNormal = inverseTransformDirection( normal, viewMatrix );\n \n#ifdef BOX_PROJECTED_ENV_MAP\n worldNormal = parallaxCorrectNormal( worldNormal, envMapSize, envMapPosition );\n#endif\n\n ").replace("reflectVec = inverseTransformDirection( reflectVec, viewMatrix );","reflectVec = inverseTransformDirection( reflectVec, viewMatrix );\n \n#ifdef BOX_PROJECTED_ENV_MAP\n reflectVec = parallaxCorrectNormal( reflectVec, envMapSize, envMapPosition );\n#endif\n\n ")}`}(e,r.position,r.size),customProgramCacheKey:()=>JSON.stringify(r.position.toArray())+JSON.stringify(r.size.toArray())})),[...r.position.toArray(),...r.size.toArray()]);return i.useLayoutEffect((()=>{n.current.needsUpdate=!0}),[r]),a}var sc=r(53615);const oc=i.forwardRef((({children:e,curve:t},r)=>{const[n]=i.useState((()=>new s.Scene)),[a,l]=i.useState(),c=i.useRef();return i.useEffect((()=>{c.current=new sc.Zw(n.children[0]),l(c.current.object3D)}),[e]),i.useEffect((()=>{var e;t&&(null==(e=c.current)||e.updateCurve(0,t))}),[t]),i.useImperativeHandle(r,(()=>({moveAlongCurve:e=>{var t;null==(t=c.current)||t.moveAlongCurve(e)}}))),i.createElement(i.Fragment,null,(0,o.createPortal)(e,n),a&&i.createElement("primitive",{object:a}))}));class lc extends s.MeshPhysicalMaterial{constructor(e={}){super(e),this.setValues(e),this._time={value:0},this._distort={value:.4},this._radius={value:1}}onBeforeCompile(e){e.uniforms.time=this._time,e.uniforms.radius=this._radius,e.uniforms.distort=this._distort,e.vertexShader=`\n uniform float time;\n uniform float radius;\n uniform float distort;\n #define GLSLIFY 1\nvec3 mod289(vec3 x){return x-floor(x*(1.0/289.0))*289.0;}vec4 mod289(vec4 x){return x-floor(x*(1.0/289.0))*289.0;}vec4 permute(vec4 x){return mod289(((x*34.0)+1.0)*x);}vec4 taylorInvSqrt(vec4 r){return 1.79284291400159-0.85373472095314*r;}float snoise(vec3 v){const vec2 C=vec2(1.0/6.0,1.0/3.0);const vec4 D=vec4(0.0,0.5,1.0,2.0);vec3 i=floor(v+dot(v,C.yyy));vec3 x0=v-i+dot(i,C.xxx);vec3 g=step(x0.yzx,x0.xyz);vec3 l=1.0-g;vec3 i1=min(g.xyz,l.zxy);vec3 i2=max(g.xyz,l.zxy);vec3 x1=x0-i1+C.xxx;vec3 x2=x0-i2+C.yyy;vec3 x3=x0-D.yyy;i=mod289(i);vec4 p=permute(permute(permute(i.z+vec4(0.0,i1.z,i2.z,1.0))+i.y+vec4(0.0,i1.y,i2.y,1.0))+i.x+vec4(0.0,i1.x,i2.x,1.0));float n_=0.142857142857;vec3 ns=n_*D.wyz-D.xzx;vec4 j=p-49.0*floor(p*ns.z*ns.z);vec4 x_=floor(j*ns.z);vec4 y_=floor(j-7.0*x_);vec4 x=x_*ns.x+ns.yyyy;vec4 y=y_*ns.x+ns.yyyy;vec4 h=1.0-abs(x)-abs(y);vec4 b0=vec4(x.xy,y.xy);vec4 b1=vec4(x.zw,y.zw);vec4 s0=floor(b0)*2.0+1.0;vec4 s1=floor(b1)*2.0+1.0;vec4 sh=-step(h,vec4(0.0));vec4 a0=b0.xzyw+s0.xzyw*sh.xxyy;vec4 a1=b1.xzyw+s1.xzyw*sh.zzww;vec3 p0=vec3(a0.xy,h.x);vec3 p1=vec3(a0.zw,h.y);vec3 p2=vec3(a1.xy,h.z);vec3 p3=vec3(a1.zw,h.w);vec4 norm=taylorInvSqrt(vec4(dot(p0,p0),dot(p1,p1),dot(p2,p2),dot(p3,p3)));p0*=norm.x;p1*=norm.y;p2*=norm.z;p3*=norm.w;vec4 m=max(0.6-vec4(dot(x0,x0),dot(x1,x1),dot(x2,x2),dot(x3,x3)),0.0);m=m*m;return 42.0*dot(m*m,vec4(dot(p0,x0),dot(p1,x1),dot(p2,x2),dot(p3,x3)));}\n ${e.vertexShader}\n `,e.vertexShader=e.vertexShader.replace("#include ","\n float updateTime = time / 50.0;\n float noise = snoise(vec3(position / 2.0 + updateTime * 5.0));\n vec3 transformed = vec3(position * (noise * pow(distort, 2.0) + radius));\n ")}get time(){return this._time.value}set time(e){this._time.value=e}get distort(){return this._distort.value}set distort(e){this._distort.value=e}get radius(){return this._radius.value}set radius(e){this._radius.value=e}}const cc=i.forwardRef((({speed:e=1,...t},r)=>{const[a]=i.useState((()=>new lc));return(0,o.useFrame)((t=>a&&(a.time=t.clock.getElapsedTime()*e))),i.createElement("primitive",n({dispose:void 0,object:a,ref:r,attach:"material"},t))}));class uc extends s.MeshStandardMaterial{constructor(e={}){super(e),this.setValues(e),this._time={value:0},this._factor={value:1}}onBeforeCompile(e){e.uniforms.time=this._time,e.uniforms.factor=this._factor,e.vertexShader=`\n uniform float time;\n uniform float factor;\n ${e.vertexShader}\n `,e.vertexShader=e.vertexShader.replace("#include ","float theta = sin( time + position.y ) / 2.0 * factor;\n float c = cos( theta );\n float s = sin( theta );\n mat3 m = mat3( c, 0, s, 0, 1, 0, -s, 0, c );\n vec3 transformed = vec3( position ) * m;\n vNormal = vNormal * m;")}get time(){return this._time.value}set time(e){this._time.value=e}get factor(){return this._factor.value}set factor(e){this._factor.value=e}}const hc=i.forwardRef((({speed:e=1,...t},r)=>{const[a]=i.useState((()=>new uc));return(0,o.useFrame)((t=>a&&(a.time=t.clock.getElapsedTime()*e))),i.createElement("primitive",n({dispose:void 0,object:a,ref:r,attach:"material"},t))}));class dc extends s.ShaderMaterial{constructor(e=new s.Vector2){super({uniforms:{inputBuffer:new s.Uniform(null),depthBuffer:new s.Uniform(null),resolution:new s.Uniform(new s.Vector2),texelSize:new s.Uniform(new s.Vector2),halfTexelSize:new s.Uniform(new s.Vector2),kernel:new s.Uniform(0),scale:new s.Uniform(1),cameraNear:new s.Uniform(0),cameraFar:new s.Uniform(1),minDepthThreshold:new s.Uniform(0),maxDepthThreshold:new s.Uniform(1),depthScale:new s.Uniform(0),depthToBlurRatioBias:new s.Uniform(.25)},fragmentShader:"#include \n #include \n uniform sampler2D inputBuffer;\n uniform sampler2D depthBuffer;\n uniform float cameraNear;\n uniform float cameraFar;\n uniform float minDepthThreshold;\n uniform float maxDepthThreshold;\n uniform float depthScale;\n uniform float depthToBlurRatioBias;\n varying vec2 vUv;\n varying vec2 vUv0;\n varying vec2 vUv1;\n varying vec2 vUv2;\n varying vec2 vUv3;\n\n void main() {\n float depthFactor = 0.0;\n \n #ifdef USE_DEPTH\n vec4 depth = texture2D(depthBuffer, vUv);\n depthFactor = smoothstep(minDepthThreshold, maxDepthThreshold, 1.0-(depth.r * depth.a));\n depthFactor *= depthScale;\n depthFactor = max(0.0, min(1.0, depthFactor + 0.25));\n #endif\n \n vec4 sum = texture2D(inputBuffer, mix(vUv0, vUv, depthFactor));\n sum += texture2D(inputBuffer, mix(vUv1, vUv, depthFactor));\n sum += texture2D(inputBuffer, mix(vUv2, vUv, depthFactor));\n sum += texture2D(inputBuffer, mix(vUv3, vUv, depthFactor));\n gl_FragColor = sum * 0.25 ;\n\n #include \n #include \n #include \n }",vertexShader:"uniform vec2 texelSize;\n uniform vec2 halfTexelSize;\n uniform float kernel;\n uniform float scale;\n varying vec2 vUv;\n varying vec2 vUv0;\n varying vec2 vUv1;\n varying vec2 vUv2;\n varying vec2 vUv3;\n\n void main() {\n vec2 uv = position.xy * 0.5 + 0.5;\n vUv = uv;\n\n vec2 dUv = (texelSize * vec2(kernel) + halfTexelSize) * scale;\n vUv0 = vec2(uv.x - dUv.x, uv.y + dUv.y);\n vUv1 = vec2(uv.x + dUv.x, uv.y + dUv.y);\n vUv2 = vec2(uv.x + dUv.x, uv.y - dUv.y);\n vUv3 = vec2(uv.x - dUv.x, uv.y - dUv.y);\n\n gl_Position = vec4(position.xy, 1.0, 1.0);\n }",blending:s.NoBlending,depthWrite:!1,depthTest:!1}),this.toneMapped=!1,this.setTexelSize(e.x,e.y),this.kernel=new Float32Array([0,1,2,2,3])}setTexelSize(e,t){this.uniforms.texelSize.value.set(e,t),this.uniforms.halfTexelSize.value.set(e,t).multiplyScalar(.5)}setResolution(e){this.uniforms.resolution.value.copy(e)}}class pc{constructor({gl:e,resolution:t,width:r=500,height:n=500,minDepthThreshold:i=0,maxDepthThreshold:a=1,depthScale:o=0,depthToBlurRatioBias:l=.25}){this.renderToScreen=!1,this.renderTargetA=new s.WebGLRenderTarget(t,t,{minFilter:s.LinearFilter,magFilter:s.LinearFilter,stencilBuffer:!1,depthBuffer:!1,encoding:e.outputEncoding}),this.renderTargetB=this.renderTargetA.clone(),this.convolutionMaterial=new dc,this.convolutionMaterial.setTexelSize(1/r,1/n),this.convolutionMaterial.setResolution(new s.Vector2(r,n)),this.scene=new s.Scene,this.camera=new s.Camera,this.convolutionMaterial.uniforms.minDepthThreshold.value=i,this.convolutionMaterial.uniforms.maxDepthThreshold.value=a,this.convolutionMaterial.uniforms.depthScale.value=o,this.convolutionMaterial.uniforms.depthToBlurRatioBias.value=l,this.convolutionMaterial.defines.USE_DEPTH=o>0;const c=new Float32Array([-1,-1,0,3,-1,0,-1,3,0]),u=new Float32Array([0,0,2,0,0,2]),h=new s.BufferGeometry;h.setAttribute("position",new s.BufferAttribute(c,3)),h.setAttribute("uv",new s.BufferAttribute(u,2)),this.screen=new s.Mesh(h,this.convolutionMaterial),this.screen.frustumCulled=!1,this.scene.add(this.screen)}render(e,t,r){const n=this.scene,i=this.camera,a=this.renderTargetA,s=this.renderTargetB;let o=this.convolutionMaterial,l=o.uniforms;l.depthBuffer.value=t.depthTexture;const c=o.kernel;let u,h,d,p=t;for(h=0,d=c.length-1;h","#include \n my_vUv = textureMatrix * vec4( position, 1.0 );\n gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );"),e.fragmentShader=`\n uniform sampler2D tDiffuse;\n uniform sampler2D tDiffuseBlur;\n uniform sampler2D tDepth;\n uniform sampler2D distortionMap;\n uniform float distortion;\n uniform float cameraNear;\n\t\t\t uniform float cameraFar;\n uniform bool hasBlur;\n uniform float mixBlur;\n uniform float mirror;\n uniform float mixStrength;\n uniform float minDepthThreshold;\n uniform float maxDepthThreshold;\n uniform float mixContrast;\n uniform float depthScale;\n uniform float depthToBlurRatioBias;\n varying vec4 my_vUv; \n ${e.fragmentShader}`,e.fragmentShader=e.fragmentShader.replace("#include ","#include \n \n float distortionFactor = 0.0;\n #ifdef USE_DISTORTION\n distortionFactor = texture2D(distortionMap, vUv).r * distortion;\n #endif\n\n vec4 new_vUv = my_vUv;\n new_vUv.x += distortionFactor;\n new_vUv.y += distortionFactor;\n\n vec4 base = texture2DProj(tDiffuse, new_vUv);\n vec4 blur = texture2DProj(tDiffuseBlur, new_vUv);\n \n vec4 merge = base;\n \n #ifdef USE_NORMALMAP\n vec2 normal_uv = vec2(0.0);\n vec4 normalColor = texture2D(normalMap, vUv * normalScale);\n vec3 my_normal = normalize( vec3( normalColor.r * 2.0 - 1.0, normalColor.b, normalColor.g * 2.0 - 1.0 ) );\n vec3 coord = new_vUv.xyz / new_vUv.w;\n normal_uv = coord.xy + coord.z * my_normal.xz * 0.05;\n vec4 base_normal = texture2D(tDiffuse, normal_uv);\n vec4 blur_normal = texture2D(tDiffuseBlur, normal_uv);\n merge = base_normal;\n blur = blur_normal;\n #endif\n\n float depthFactor = 0.0001;\n float blurFactor = 0.0;\n\n #ifdef USE_DEPTH\n vec4 depth = texture2DProj(tDepth, new_vUv);\n depthFactor = smoothstep(minDepthThreshold, maxDepthThreshold, 1.0-(depth.r * depth.a));\n depthFactor *= depthScale;\n depthFactor = max(0.0001, min(1.0, depthFactor));\n\n #ifdef USE_BLUR\n blur = blur * min(1.0, depthFactor + depthToBlurRatioBias);\n merge = merge * min(1.0, depthFactor + 0.5);\n #else\n merge = merge * depthFactor;\n #endif\n \n #endif\n\n float reflectorRoughnessFactor = roughness;\n #ifdef USE_ROUGHNESSMAP\n vec4 reflectorTexelRoughness = texture2D( roughnessMap, vUv );\n reflectorRoughnessFactor *= reflectorTexelRoughness.g;\n #endif\n \n #ifdef USE_BLUR\n blurFactor = min(1.0, mixBlur * reflectorRoughnessFactor);\n merge = mix(merge, blur, blurFactor);\n #endif\n\n vec4 newMerge = vec4(0.0, 0.0, 0.0, 1.0);\n newMerge.r = (merge.r - 0.5) * mixContrast + 0.5;\n newMerge.g = (merge.g - 0.5) * mixContrast + 0.5;\n newMerge.b = (merge.b - 0.5) * mixContrast + 0.5;\n\n diffuseColor.rgb = diffuseColor.rgb * ((1.0 - min(1.0, mirror)) + newMerge.rgb * mixStrength);\n ")}get tDiffuse(){return this._tDiffuse.value}set tDiffuse(e){this._tDiffuse.value=e}get tDepth(){return this._tDepth.value}set tDepth(e){this._tDepth.value=e}get distortionMap(){return this._distortionMap.value}set distortionMap(e){this._distortionMap.value=e}get tDiffuseBlur(){return this._tDiffuseBlur.value}set tDiffuseBlur(e){this._tDiffuseBlur.value=e}get textureMatrix(){return this._textureMatrix.value}set textureMatrix(e){this._textureMatrix.value=e}get hasBlur(){return this._hasBlur.value}set hasBlur(e){this._hasBlur.value=e}get mirror(){return this._mirror.value}set mirror(e){this._mirror.value=e}get mixBlur(){return this._mixBlur.value}set mixBlur(e){this._mixBlur.value=e}get mixStrength(){return this._blurStrength.value}set mixStrength(e){this._blurStrength.value=e}get minDepthThreshold(){return this._minDepthThreshold.value}set minDepthThreshold(e){this._minDepthThreshold.value=e}get maxDepthThreshold(){return this._maxDepthThreshold.value}set maxDepthThreshold(e){this._maxDepthThreshold.value=e}get depthScale(){return this._depthScale.value}set depthScale(e){this._depthScale.value=e}get depthToBlurRatioBias(){return this._depthToBlurRatioBias.value}set depthToBlurRatioBias(e){this._depthToBlurRatioBias.value=e}get distortion(){return this._distortion.value}set distortion(e){this._distortion.value=e}get mixContrast(){return this._mixContrast.value}set mixContrast(e){this._mixContrast.value=e}}(0,o.extend)({MeshReflectorMaterialImpl:fc});const mc=i.forwardRef((({mixBlur:e=0,mixStrength:t=1,resolution:r=256,blur:a=[0,0],minDepthThreshold:l=.9,maxDepthThreshold:c=1,depthScale:u=0,depthToBlurRatioBias:h=.25,mirror:d=0,distortion:p=1,mixContrast:f=1,distortionMap:m,reflectorOffset:g=0,...v},A)=>{const y=(0,o.useThree)((({gl:e})=>e)),x=(0,o.useThree)((({camera:e})=>e)),b=(0,o.useThree)((({scene:e})=>e)),w=(a=Array.isArray(a)?a:[a,a])[0]+a[1]>0,_=i.useRef(null),[E]=i.useState((()=>new s.Plane)),[S]=i.useState((()=>new s.Vector3)),[T]=i.useState((()=>new s.Vector3)),[C]=i.useState((()=>new s.Vector3)),[I]=i.useState((()=>new s.Matrix4)),[B]=i.useState((()=>new s.Vector3(0,0,-1))),[R]=i.useState((()=>new s.Vector4)),[P]=i.useState((()=>new s.Vector3)),[F]=i.useState((()=>new s.Vector3)),[L]=i.useState((()=>new s.Vector4)),[D]=i.useState((()=>new s.Matrix4)),[U]=i.useState((()=>new s.PerspectiveCamera)),N=i.useCallback((()=>{var e;const t=_.current.parent||(null==(e=_.current)?void 0:e.__r3f.parent);if(!t)return;if(T.setFromMatrixPosition(t.matrixWorld),C.setFromMatrixPosition(x.matrixWorld),I.extractRotation(t.matrixWorld),S.set(0,0,1),S.applyMatrix4(I),T.addScaledVector(S,g),P.subVectors(T,C),P.dot(S)>0)return;P.reflect(S).negate(),P.add(T),I.extractRotation(x.matrixWorld),B.set(0,0,-1),B.applyMatrix4(I),B.add(C),F.subVectors(T,B),F.reflect(S).negate(),F.add(T),U.position.copy(P),U.up.set(0,1,0),U.up.applyMatrix4(I),U.up.reflect(S),U.lookAt(F),U.far=x.far,U.updateMatrixWorld(),U.projectionMatrix.copy(x.projectionMatrix),D.set(.5,0,0,.5,0,.5,0,.5,0,0,.5,.5,0,0,0,1),D.multiply(U.projectionMatrix),D.multiply(U.matrixWorldInverse),D.multiply(t.matrixWorld),E.setFromNormalAndCoplanarPoint(S,T),E.applyMatrix4(U.matrixWorldInverse),R.set(E.normal.x,E.normal.y,E.normal.z,E.constant);const r=U.projectionMatrix;L.x=(Math.sign(R.x)+r.elements[8])/r.elements[0],L.y=(Math.sign(R.y)+r.elements[9])/r.elements[5],L.z=-1,L.w=(1+r.elements[10])/r.elements[14],R.multiplyScalar(2/R.dot(L)),r.elements[2]=R.x,r.elements[6]=R.y,r.elements[10]=R.z+1,r.elements[14]=R.w}),[x,g]),[k,O,G,z]=i.useMemo((()=>{const n={minFilter:s.LinearFilter,magFilter:s.LinearFilter,encoding:y.outputEncoding,type:s.HalfFloatType},i=new s.WebGLRenderTarget(r,r,n);i.depthBuffer=!0,i.depthTexture=new s.DepthTexture(r,r),i.depthTexture.format=s.DepthFormat,i.depthTexture.type=s.UnsignedShortType;const o=new s.WebGLRenderTarget(r,r,n);return[i,o,new pc({gl:y,resolution:r,width:a[0],height:a[1],minDepthThreshold:l,maxDepthThreshold:c,depthScale:u,depthToBlurRatioBias:h}),{mirror:d,textureMatrix:D,mixBlur:e,tDiffuse:i.texture,tDepth:i.depthTexture,tDiffuseBlur:o.texture,hasBlur:w,mixStrength:t,minDepthThreshold:l,maxDepthThreshold:c,depthScale:u,depthToBlurRatioBias:h,transparent:!0,distortion:p,distortionMap:m,mixContrast:f,"defines-USE_BLUR":w?"":void 0,"defines-USE_DEPTH":u>0?"":void 0,"defines-USE_DISTORTION":m?"":void 0}]}),[y,a,D,r,d,w,e,t,l,c,u,h,p,m,f]);return(0,o.useFrame)((()=>{var e;const t=_.current.parent||(null==(e=_.current)?void 0:e.__r3f.parent);if(!t)return;t.visible=!1;const r=y.xr.enabled,n=y.shadowMap.autoUpdate;N(),y.xr.enabled=!1,y.shadowMap.autoUpdate=!1,y.setRenderTarget(k),y.state.buffers.depth.setMask(!0),y.autoClear||y.clear(),y.render(b,U),w&&G.render(y,k,O),y.xr.enabled=r,y.shadowMap.autoUpdate=n,t.visible=!0,y.setRenderTarget(null)})),i.createElement("meshReflectorMaterialImpl",n({attach:"material",key:"key"+z["defines-USE_BLUR"]+z["defines-USE_DEPTH"]+z["defines-USE_DISTORTION"],ref:M([_,A])},z,v))}));class gc extends s.PointsMaterial{constructor(e){super(e),this.onBeforeCompile=e=>{e.fragmentShader=e.fragmentShader.replace("#include ","\n #include \n vec2 cxy = 2.0 * gl_PointCoord - 1.0;\n float r = dot(cxy, cxy);\n float delta = fwidth(r); \n float mask = 1.0 - smoothstep(1.0 - delta, 1.0 + delta, r);\n gl_FragColor = vec4(gl_FragColor.rgb, mask * gl_FragColor.a );\n ")}}}const vc=i.forwardRef(((e,t)=>{const[r]=i.useState((()=>new gc(null)));return i.createElement("primitive",n({},e,{object:r,ref:t,attach:"material"}))}));let Ac=!1;const yc=e=>{if(!Ac){Ac=!0;let t=s.ShaderChunk.shadowmap_pars_fragment;t=t.replace("#ifdef USE_SHADOWMAP","#ifdef USE_SHADOWMAP\n"+(({frustum:e=3.75,size:t=.005,near:r=9.5,samples:n=17,rings:i=11}={})=>`#define LIGHT_WORLD_SIZE ${t}\n#define LIGHT_FRUSTUM_WIDTH ${e}\n#define LIGHT_SIZE_UV (LIGHT_WORLD_SIZE / LIGHT_FRUSTUM_WIDTH)\n#define NEAR_PLANE ${r}\n\n#define NUM_SAMPLES ${n}\n#define NUM_RINGS ${i}\n#define BLOCKER_SEARCH_NUM_SAMPLES NUM_SAMPLES\n#define PCF_NUM_SAMPLES NUM_SAMPLES\n\nvec2 poissonDisk[NUM_SAMPLES];\n\nvoid initPoissonSamples(const in vec2 randomSeed) {\n float ANGLE_STEP = PI2 * float(NUM_RINGS) / float(NUM_SAMPLES);\n float INV_NUM_SAMPLES = 1.0 / float(NUM_SAMPLES);\n float angle = rand(randomSeed) * PI2;\n float radius = INV_NUM_SAMPLES;\n float radiusStep = radius;\n for (int i = 0; i < NUM_SAMPLES; i++) {\n poissonDisk[i] = vec2(cos(angle), sin(angle)) * pow(radius, 0.75);\n radius += radiusStep;\n angle += ANGLE_STEP;\n }\n}\n\nfloat penumbraSize(const in float zReceiver, const in float zBlocker) { // Parallel plane estimation\n return (zReceiver - zBlocker) / zBlocker;\n}\n\nfloat findBlocker(sampler2D shadowMap, const in vec2 uv, const in float zReceiver) {\n float searchRadius = LIGHT_SIZE_UV * (zReceiver - NEAR_PLANE) / zReceiver;\n float blockerDepthSum = 0.0;\n int numBlockers = 0;\n for (int i = 0; i < BLOCKER_SEARCH_NUM_SAMPLES; i++) {\n float shadowMapDepth = unpackRGBAToDepth(texture2D(shadowMap, uv + poissonDisk[i] * searchRadius));\n if (shadowMapDepth < zReceiver) {\n blockerDepthSum += shadowMapDepth;\n numBlockers++;\n }\n }\n if (numBlockers == 0) return -1.0;\n return blockerDepthSum / float(numBlockers);\n}\n\nfloat PCF_Filter(sampler2D shadowMap, vec2 uv, float zReceiver, float filterRadius) {\n float sum = 0.0;\n for (int i = 0; i < PCF_NUM_SAMPLES; i++) {\n float depth = unpackRGBAToDepth(texture2D(shadowMap, uv + poissonDisk[ i ] * filterRadius));\n if (zReceiver <= depth) sum += 1.0;\n }\n for (int i = 0; i < PCF_NUM_SAMPLES; i++) {\n float depth = unpackRGBAToDepth(texture2D(shadowMap, uv + -poissonDisk[ i ].yx * filterRadius));\n if (zReceiver <= depth) sum += 1.0;\n }\n return sum / (2.0 * float(PCF_NUM_SAMPLES));\n}\n\nfloat PCSS(sampler2D shadowMap, vec4 coords) {\n vec2 uv = coords.xy;\n float zReceiver = coords.z; // Assumed to be eye-space z in this code\n initPoissonSamples(uv);\n float avgBlockerDepth = findBlocker(shadowMap, uv, zReceiver);\n if (avgBlockerDepth == -1.0) return 1.0;\n float penumbraRatio = penumbraSize(zReceiver, avgBlockerDepth);\n float filterRadius = penumbraRatio * LIGHT_SIZE_UV * NEAR_PLANE / zReceiver;\n return PCF_Filter(shadowMap, uv, zReceiver, filterRadius);\n}`)({...e})),t=t.replace("#if defined( SHADOWMAP_TYPE_PCF )","\nreturn PCSS(shadowMap, shadowCoord);\n#if defined( SHADOWMAP_TYPE_PCF )"),s.ShaderChunk.shadowmap_pars_fragment=t}};function xc(e){const t=e+"BufferGeometry";return i.forwardRef((({args:e,children:r,...a},s)=>i.createElement("mesh",n({ref:s},a),i.createElement(t,{attach:"geometry",args:e}),r)))}const bc=xc("box"),wc=xc("circle"),_c=xc("cone"),Ec=xc("cylinder"),Sc=xc("sphere"),Mc=xc("plane"),Tc=xc("tube"),Cc=xc("torus"),Ic=xc("torusKnot"),Bc=xc("tetrahedron"),Rc=xc("ring"),Pc=xc("polyhedron"),Fc=xc("icosahedron"),Lc=xc("octahedron"),Dc=xc("dodecahedron"),Uc=xc("extrude"),Nc=xc("lathe"),kc=1e-5;const Oc=i.forwardRef((function({args:[e=1,t=1,r=1]=[],radius:a=.05,steps:o=1,smoothness:l=4,children:c,...u},h){const d=i.useMemo((()=>function(e,t,r){const n=new s.Shape,i=r-kc;return n.absarc(kc,kc,kc,-Math.PI/2,-Math.PI,!0),n.absarc(kc,t-2*i,kc,Math.PI,Math.PI/2,!0),n.absarc(e-2*i,t-2*i,kc,Math.PI/2,0,!0),n.absarc(e-2*i,kc,kc,0,-Math.PI/2,!0),n}(e,t,a)),[e,t,a]),p=i.useMemo((()=>({depth:r-2*a,bevelEnabled:!0,bevelSegments:2*l,steps:o,bevelSize:a-kc,bevelThickness:a,curveSegments:l})),[r,a,l]),f=i.useRef();return i.useLayoutEffect((()=>{f.current&&f.current.center()}),[d,p]),i.createElement("mesh",n({ref:h},u),i.createElement("extrudeBufferGeometry",{attach:"geometry",ref:f,args:[d,p]}),c)}));function Gc(){const e=new s.BufferGeometry,t=new Float32Array([-1,-1,3,-1,-1,3]);return e.setAttribute("position",new s.BufferAttribute(t,2)),e}const zc=i.forwardRef((function({children:e,...t},r){const a=i.useMemo(Gc,[]);return i.createElement("mesh",n({ref:r,geometry:a,frustumCulled:!1},t),e)})),Vc=i.forwardRef((function({children:e,alignTop:t,...r},a){const o=i.useRef(null),l=i.useRef(null);return i.useLayoutEffect((()=>{o.current.position.set(0,0,0),o.current.updateWorldMatrix(!0,!0);const e=(new s.Box3).setFromObject(l.current),r=new s.Vector3,n=new s.Sphere,i=e.max.y-e.min.y;e.getCenter(r),e.getBoundingSphere(n),o.current.position.set(-r.x,-r.y+(t?i/2:0),-r.z)}),[e]),i.createElement("group",n({ref:a},r),i.createElement("group",{ref:o},i.createElement("group",{ref:l},e)))})),Hc=e=>e&&e.isOrthographicCamera,Qc=i.createContext(null);function jc({children:e,damping:t=6,fit:r,clip:n,observe:a,margin:l=1.2,eps:c=.01,onFit:u}){const h=i.useRef(null),{camera:d,invalidate:p,size:f,controls:m}=(0,o.useThree)(),g=i.useRef(u);function v(e,t){return Math.abs(e.x-t.x)({animating:!1,focus:new s.Vector3,camera:new s.Vector3,zoom:1}))),[x]=i.useState((()=>({focus:new s.Vector3,camera:new s.Vector3,zoom:1}))),[b]=i.useState((()=>new s.Box3)),w=i.useMemo((()=>{function e(){const e=b.getSize(new s.Vector3),t=b.getCenter(new s.Vector3),r=Math.max(e.x,e.y,e.z),n=Hc(d)?4*r:r/(2*Math.atan(Math.PI*d.fov/360)),i=Hc(d)?4*r:n/d.aspect,a=l*Math.max(n,i);return{box:b,size:e,center:t,distance:a}}return{getSize:e,refresh(t){if((r=t)&&r.isBox3)b.copy(t);else{const e=t||h.current;e.updateWorldMatrix(!0,!0),b.setFromObject(e)}var r;if(b.isEmpty()){const e=d.position.length()||10;b.setFromCenterAndSize(new s.Vector3,new s.Vector3(e,e,e))}if("OrthographicTrackballControls"===(null==m?void 0:m.constructor.name)){const{distance:t}=e(),r=d.position.clone().sub(m.target).normalize().multiplyScalar(t),n=m.target.clone().add(r);d.position.copy(n)}return this},clip(){const{distance:t}=e();return m&&(m.maxDistance=10*t),d.near=t/100,d.far=100*t,d.updateProjectionMatrix(),m&&m.update(),p(),this},fit(){y.camera.copy(d.position),m&&y.focus.copy(m.target);const{center:r,distance:n}=e(),i=r.clone().sub(d.position).normalize().multiplyScalar(n);if(x.camera.copy(r).sub(i),x.focus.copy(r),Hc(d)){y.zoom=d.zoom;let e=0,n=0;const i=[new s.Vector3(b.min.x,b.min.y,b.min.z),new s.Vector3(b.min.x,b.max.y,b.min.z),new s.Vector3(b.min.x,b.min.y,b.max.z),new s.Vector3(b.min.x,b.max.y,b.max.z),new s.Vector3(b.max.x,b.max.y,b.max.z),new s.Vector3(b.max.x,b.max.y,b.min.z),new s.Vector3(b.max.x,b.min.y,b.max.z),new s.Vector3(b.max.x,b.min.y,b.min.z)];r.applyMatrix4(d.matrixWorldInverse);for(const t of i)t.applyMatrix4(d.matrixWorldInverse),e=Math.max(e,Math.abs(t.y-r.y)),n=Math.max(n,Math.abs(t.x-r.x));e*=2,n*=2;const a=(d.top-d.bottom)/e,o=(d.right-d.left)/n;x.zoom=Math.min(a,o)/l,t||(d.zoom=x.zoom,d.updateProjectionMatrix())}return t?y.animating=!0:(d.position.copy(x.camera),d.lookAt(x.focus),m&&(m.target.copy(x.focus),m.update())),g.current&&g.current(this.getSize()),p(),this}}}),[b,d,m,l,t,p]);i.useLayoutEffect((()=>{if(m){const e=()=>y.animating=!1;return m.addEventListener("start",e),()=>m.removeEventListener("start",e)}}),[m]);const _=i.useRef(0);return i.useLayoutEffect((()=>{(a||0==_.current++)&&(w.refresh(),r&&w.fit(),n&&w.clip())}),[f,n,r,a]),(0,o.useFrame)(((e,r)=>{if(y.animating){if(A(y.focus,x.focus,t,r),A(y.camera,x.camera,t,r),y.zoom=s.MathUtils.damp(y.zoom,x.zoom,t,r),d.position.copy(y.camera),Hc(d)&&(d.zoom=y.zoom,d.updateProjectionMatrix()),m?(m.target.copy(y.focus),m.update()):d.lookAt(y.focus),p(),Hc(d)&&!(Math.abs(y.zoom-x.zoom){const p=(0,o.useThree)((e=>e.camera)),f=(0,o.useThree)((e=>e.controls)),m=i.useRef(e),g=i.useRef(p.rotation.clone()),[v]=i.useState((()=>new Xc.L)),[A]=i.useState((()=>new Xc.L)),[y]=i.useState((()=>new Xc.L)),x=()=>{(m.current<0||m.current>1)&&(m.current=m.current<0?0:1)};return i.useImperativeHandle(d,(()=>({getIntensity:()=>m.current,setIntensity:e=>{m.current=e,x()}})),[]),i.useEffect((()=>{const e=f||(null==h?void 0:h.current),t=()=>{g.current=p.rotation.clone()};return null==e||e.addEventListener("change",t),()=>{null==e||e.removeEventListener("change",t)}}),[h,f]),(0,o.useFrame)((({clock:e},i)=>{const o=Math.pow(m.current,2),h=n*o*v.noise(e.elapsedTime*l,1),d=a*o*A.noise(e.elapsedTime*c,1),f=s*o*y.noise(e.elapsedTime*u,1);p.rotation.set(g.current.x+d,g.current.y+h,g.current.z+f),t&&m.current>0&&(m.current-=r*i,x())})),null})),qc=i.forwardRef((({children:e,speed:t=1,rotationIntensity:r=1,floatIntensity:n=1,...a},s)=>{const l=i.useRef(null),c=i.useRef(1e4*Math.random());return(0,o.useFrame)((e=>{const i=c.current+e.clock.getElapsedTime();l.current.rotation.x=Math.cos(i/4*t)/8*r,l.current.rotation.y=Math.sin(i/4*t)/8*r,l.current.rotation.z=Math.sin(i/4*t)/20*r,l.current.position.y=Math.sin(i/4*t)/10*n})),i.createElement("group",a,i.createElement("group",{ref:M([l,s])},e))}));var Kc=r(8197);const Jc={sunset:"venice/venice_sunset_1k.hdr",dawn:"kiara/kiara_1_dawn_1k.hdr",night:"dikhololo/dikhololo_night_1k.hdr",warehouse:"empty-wharehouse/empty_warehouse_01_1k.hdr",forest:"forrest-slope/forest_slope_1k.hdr",apartment:"lebombo/lebombo_1k.hdr",studio:"studio-small-3/studio_small_03_1k.hdr",city:"potsdamer-platz/potsdamer_platz_1k.hdr",park:"rooitou/rooitou_park_1k.hdr",lobby:"st-fagans/st_fagans_interior_1k.hdr"},Zc=e=>{return(t=e).current&&t.current.isScene?e.current:e;var t};function $c(e){return e.map?i.createElement(eu,e):e.children?i.createElement(tu,e):i.createElement(ru,e)}function eu({scene:e,background:t=!1,map:r}){const n=(0,o.useThree)((e=>e.scene));return i.useLayoutEffect((()=>{if(r){const i=Zc(e||n),a=i.background,s=i.environment;return"only"!==t&&(i.environment=r),t&&(i.background=r),()=>{"only"!==t&&(i.environment=s),t&&(i.background=a)}}}),[e,r]),null}function tu({children:e,near:t=1,far:r=1e3,resolution:n=256,frames:a=1,background:l=!1,scene:c,files:u,path:h,preset:d,extensions:p}){const f=(0,o.useThree)((e=>e.gl)),m=(0,o.useThree)((e=>e.scene)),g=i.useRef(null),[v]=i.useState((()=>new s.Scene)),A=i.useMemo((()=>{const e=new s.WebGLCubeRenderTarget(n);return e.texture.type=s.HalfFloatType,e}),[n]);i.useLayoutEffect((()=>{1===a&&g.current.update(f,v);const e=Zc(c||m),t=e.background,r=e.environment;return"only"!==l&&(e.environment=A.texture),l&&(e.background=A.texture),()=>{"only"!==l&&(e.environment=r),l&&(e.background=t)}}),[e,c]);let y=1;return(0,o.useFrame)((()=>{(a===1/0||ye.scene)),u=Array.isArray(t),h=u?s.CubeTextureLoader:Kc.x,d=(0,o.useLoader)(h,u?[t]:t,(e=>{e.setPath(r),null==e.setDataType||e.setDataType(s.FloatType),l&&l(e)})),p=u?d[0]:d;return p.mapping=u?s.CubeReflectionMapping:s.EquirectangularReflectionMapping,i.useLayoutEffect((()=>{const t=Zc(a||c),r=t.background,n=t.environment;return"only"!==e&&(t.environment=p),e&&(t.background=p),()=>{"only"!==e&&(t.environment=n),e&&(t.background=r)}}),[p,e,a]),null}var nu=r(78466),iu=r(37312);const au=i.forwardRef((({scale:e,frames:t=1/0,opacity:r=1,width:a=1,height:l=1,blur:c=1,far:u=10,resolution:h=256,smooth:d=!0,color:p="#000000",...f},m)=>{const g=(0,o.useThree)((({scene:e})=>e)),v=(0,o.useThree)((({gl:e})=>e)),A=i.useRef(null);a*=Array.isArray(e)?e[0]:e||1,l*=Array.isArray(e)?e[1]:e||1;const[y,x,b,w,_,E,S]=i.useMemo((()=>{const e=new s.WebGLRenderTarget(h,h),t=new s.WebGLRenderTarget(h,h);t.texture.generateMipmaps=e.texture.generateMipmaps=!1;const r=new s.PlaneBufferGeometry(a,l).rotateX(Math.PI/2),n=new s.Mesh(r),i=new s.MeshDepthMaterial;i.depthTest=i.depthWrite=!1,i.onBeforeCompile=e=>{e.uniforms={...e.uniforms,ucolor:{value:new s.Color(p).convertSRGBToLinear()}},e.fragmentShader=e.fragmentShader.replace("void main() {","uniform vec3 ucolor;\n void main() {\n "),e.fragmentShader=e.fragmentShader.replace("vec4( vec3( 1.0 - fragCoordZ ), opacity );","vec4( ucolor, ( 1.0 - fragCoordZ ) * 1.0 );")};const o=new s.ShaderMaterial(nu.P),c=new s.ShaderMaterial(iu.Z);return c.depthTest=o.depthTest=!1,[e,r,i,n,o,c,t]}),[h,a,l,e]),M=i.useCallback((e=>{w.visible=!0,w.material=_,_.uniforms.tDiffuse.value=y.texture,_.uniforms.h.value=1*e/256,v.setRenderTarget(S),v.render(w,A.current),w.material=E,E.uniforms.tDiffuse.value=S.texture,E.uniforms.v.value=1*e/256,v.setRenderTarget(y),v.render(w,A.current),w.visible=!1}),[]);let T=0;return(0,o.useFrame)((()=>{if(A.current&&(t===1/0||Te.camera)),g=(0,o.useThree)((e=>e.controls)),v=i.useRef(null),A=i.useRef(null),[{radius:y,width:x,height:b},w]=i.useState({radius:0,width:0,height:0});return i.useLayoutEffect((()=>{v.current.position.set(0,0,0),v.current.updateWorldMatrix(!0,!0);const e=(new s.Box3).setFromObject(A.current),t=new s.Vector3,r=new s.Sphere,n=e.max.y-e.min.y,i=e.max.x-e.min.x;e.getCenter(t),e.getBoundingSphere(r),w({radius:r.radius,width:i,height:n}),v.current.position.set(-t.x,-t.y+n/2,-t.z)}),[e]),i.useLayoutEffect((()=>{if(a){const e=y/(b>x?1.5:2.5);m.position.set(0,.5*y,2.5*y),m.near=.1,m.far=Math.max(5e3,4*y),m.lookAt(0,e,0);const r=g||(null==t?void 0:t.current);r&&(r.target.set(0,e,0),r.update())}}),[g,y,b,x,a]),i.createElement("group",p,i.createElement("group",{ref:v},i.createElement("group",{ref:A},e)),d&&i.createElement(au,n({scale:2*y,far:y/2},d)),l&&i.createElement($c,{preset:l}),i.createElement("ambientLight",{intensity:c/3}),i.createElement("spotLight",{penumbra:1,position:[f.main[0]*y,f.main[1]*y,f.main[2]*y],intensity:2*c,castShadow:r,"shadow-bias":h}),i.createElement("pointLight",{position:[f.fill[0]*y,f.fill[1]*y,f.fill[2]*y],intensity:c}))}const lu=e=>0===e?0:Math.pow(2,10*e-10);function cu({children:e,floor:t=.25,segments:r=20,receiveShadow:n,...a}){const s=i.useRef(null);return i.useLayoutEffect((()=>{let e=0;const n=r/r/2,i=s.current.attributes.position;for(let a=0;a{const c=i.useMemo((()=>{const e=document.createElement("canvas");e.width=128,e.height=128;const n=e.getContext("2d"),i=n.createRadialGradient(e.width/2,e.height/2,0,e.width/2,e.height/2,e.width/2);return i.addColorStop(t,new s.Color(r).getStyle()),i.addColorStop(1,"rgba(0,0,0,0)"),n.fillStyle=i,n.fillRect(0,0,e.width,e.height),e}),[r,t]);return i.createElement("mesh",n({ref:l},o),i.createElement("planeBufferGeometry",{attach:"geometry",args:[1,1]}),i.createElement("meshBasicMaterial",{attach:"material",transparent:!0,opacity:a,fog:e},i.createElement("canvasTexture",{attach:"map",args:[c]})))}));(0,o.extend)({MeshReflectorMaterial:fc});const hu=i.forwardRef((({mixBlur:e=0,mixStrength:t=.5,resolution:r=256,blur:a=[0,0],args:l=[1,1],minDepthThreshold:c=.9,maxDepthThreshold:u=1,depthScale:h=0,depthToBlurRatioBias:d=.25,mirror:p=0,children:f,debug:m=0,distortion:g=1,mixContrast:v=1,distortionMap:A,...y},x)=>{i.useEffect((()=>{console.warn("Reflector has been deprecated and will be removed next major. Replace it with !")}),[]);const b=(0,o.useThree)((({gl:e})=>e)),w=(0,o.useThree)((({camera:e})=>e)),_=(0,o.useThree)((({scene:e})=>e)),E=(a=Array.isArray(a)?a:[a,a])[0]+a[1]>0,S=i.useRef(null),[T]=i.useState((()=>new s.Plane)),[C]=i.useState((()=>new s.Vector3)),[I]=i.useState((()=>new s.Vector3)),[B]=i.useState((()=>new s.Vector3)),[R]=i.useState((()=>new s.Matrix4)),[P]=i.useState((()=>new s.Vector3(0,0,-1))),[F]=i.useState((()=>new s.Vector4)),[L]=i.useState((()=>new s.Vector3)),[D]=i.useState((()=>new s.Vector3)),[U]=i.useState((()=>new s.Vector4)),[N]=i.useState((()=>new s.Matrix4)),[k]=i.useState((()=>new s.PerspectiveCamera)),O=i.useCallback((()=>{if(I.setFromMatrixPosition(S.current.matrixWorld),B.setFromMatrixPosition(w.matrixWorld),R.extractRotation(S.current.matrixWorld),C.set(0,0,1),C.applyMatrix4(R),L.subVectors(I,B),L.dot(C)>0)return;L.reflect(C).negate(),L.add(I),R.extractRotation(w.matrixWorld),P.set(0,0,-1),P.applyMatrix4(R),P.add(B),D.subVectors(I,P),D.reflect(C).negate(),D.add(I),k.position.copy(L),k.up.set(0,1,0),k.up.applyMatrix4(R),k.up.reflect(C),k.lookAt(D),k.far=w.far,k.updateMatrixWorld(),k.projectionMatrix.copy(w.projectionMatrix),N.set(.5,0,0,.5,0,.5,0,.5,0,0,.5,.5,0,0,0,1),N.multiply(k.projectionMatrix),N.multiply(k.matrixWorldInverse),N.multiply(S.current.matrixWorld),T.setFromNormalAndCoplanarPoint(C,I),T.applyMatrix4(k.matrixWorldInverse),F.set(T.normal.x,T.normal.y,T.normal.z,T.constant);const e=k.projectionMatrix;U.x=(Math.sign(F.x)+e.elements[8])/e.elements[0],U.y=(Math.sign(F.y)+e.elements[9])/e.elements[5],U.z=-1,U.w=(1+e.elements[10])/e.elements[14],F.multiplyScalar(2/F.dot(U)),e.elements[2]=F.x,e.elements[6]=F.y,e.elements[10]=F.z+1,e.elements[14]=F.w}),[]),[G,z,V,H]=i.useMemo((()=>{const n={minFilter:s.LinearFilter,magFilter:s.LinearFilter,encoding:b.outputEncoding},i=new s.WebGLRenderTarget(r,r,n);i.depthBuffer=!0,i.depthTexture=new s.DepthTexture(r,r),i.depthTexture.format=s.DepthFormat,i.depthTexture.type=s.UnsignedShortType;const o=new s.WebGLRenderTarget(r,r,n);return[i,o,new pc({gl:b,resolution:r,width:a[0],height:a[1],minDepthThreshold:c,maxDepthThreshold:u,depthScale:h,depthToBlurRatioBias:d}),{mirror:p,textureMatrix:N,mixBlur:e,tDiffuse:i.texture,tDepth:i.depthTexture,tDiffuseBlur:o.texture,hasBlur:E,mixStrength:t,minDepthThreshold:c,maxDepthThreshold:u,depthScale:h,depthToBlurRatioBias:d,transparent:!0,debug:m,distortion:g,distortionMap:A,mixContrast:v,"defines-USE_BLUR":E?"":void 0,"defines-USE_DEPTH":h>0?"":void 0,"defines-USE_DISTORTION":A?"":void 0}]}),[b,a,N,r,p,E,e,t,c,u,h,d,m,g,A,v]);return(0,o.useFrame)((()=>{if(null==S||!S.current)return;S.current.visible=!1;const e=b.xr.enabled,t=b.shadowMap.autoUpdate;O(),b.xr.enabled=!1,b.shadowMap.autoUpdate=!1,b.setRenderTarget(G),b.state.buffers.depth.setMask(!0),b.autoClear||b.clear(),b.render(_,k),E&&V.render(b,G,z),b.xr.enabled=e,b.shadowMap.autoUpdate=t,S.current.visible=!0,b.setRenderTarget(null)})),i.createElement("mesh",n({ref:M([S,x])},y),i.createElement("planeBufferGeometry",{args:l}),f?f("meshReflectorMaterial",H):i.createElement("meshReflectorMaterial",H))}));class du extends s.ShaderMaterial{constructor(){super({uniforms:{depth:{value:null},opacity:{value:1},attenuation:{value:2.5},anglePower:{value:12},spotPosition:{value:new s.Vector3(0,0,0)},lightColor:{value:new s.Color("white")},cameraNear:{value:0},cameraFar:{value:1},resolution:{value:new s.Vector2(0,0)}},transparent:!0,depthWrite:!1,vertexShader:"\n varying vec3 vNormal;\n varying vec3 vWorldPosition;\n varying float vViewZ;\n varying float vIntensity;\n uniform vec3 spotPosition;\n uniform float attenuation; \n\n void main() {\n // compute intensity\n vNormal = normalize( normalMatrix * normal );\n vec4 worldPosition\t= modelMatrix * vec4( position, 1.0 );\n vWorldPosition = worldPosition.xyz;\n vec4 viewPosition = viewMatrix * worldPosition;\n vViewZ = viewPosition.z;\n float intensity\t= distance(worldPosition.xyz, spotPosition) / attenuation;\n intensity\t= 1.0 - clamp(intensity, 0.0, 1.0);\n vIntensity = intensity; \n // set gl_Position\n gl_Position\t= projectionMatrix * viewPosition;\n\n }",fragmentShader:"\n #include \n\n varying vec3 vNormal;\n varying vec3 vWorldPosition;\n uniform vec3 lightColor;\n uniform vec3 spotPosition;\n uniform float attenuation;\n uniform float anglePower;\n uniform sampler2D depth;\n uniform vec2 resolution;\n uniform float cameraNear;\n uniform float cameraFar;\n varying float vViewZ;\n varying float vIntensity;\n uniform float opacity;\n\n float readDepth( sampler2D depthSampler, vec2 coord ) {\n float fragCoordZ = texture2D( depthSampler, coord ).x;\n float viewZ = perspectiveDepthToViewZ(fragCoordZ, cameraNear, cameraFar);\n return viewZ;\n }\n\n void main() {\n float d = 1.0;\n bool isSoft = resolution[0] > 0.0 && resolution[1] > 0.0;\n if (isSoft) {\n vec2 sUv = gl_FragCoord.xy / resolution;\n d = readDepth(depth, sUv);\n }\n float intensity = vIntensity;\n vec3 normal\t= vec3(vNormal.x, vNormal.y, abs(vNormal.z));\n float angleIntensity\t= pow( dot(normal, vec3(0.0, 0.0, 1.0)), anglePower );\n intensity\t*= angleIntensity;\n // fades when z is close to sampled depth, meaning the cone is intersecting existing geometry\n if (isSoft) {\n intensity\t*= smoothstep(0., 1., vViewZ - d);\n }\n gl_FragColor = vec4(lightColor, intensity * opacity);\n\n #include \n\t #include \n }"})}}const pu=new s.Vector3,fu=i.forwardRef((({opacity:e=1,radiusTop:t,radiusBottom:r,depthBuffer:a,color:l="white",distance:c=5,angle:u=.15,attenuation:h=5,anglePower:d=5,...p},f)=>{const m=i.useRef(null),g=(0,o.useThree)((e=>e.size)),v=(0,o.useThree)((e=>e.camera)),A=(0,o.useThree)((e=>e.viewport.dpr)),[y]=i.useState((()=>new du));t=void 0===t?.1:t,r=void 0===r?7*u:r,(0,o.useFrame)((()=>{y.uniforms.spotPosition.value.copy(m.current.getWorldPosition(pu)),m.current.lookAt(m.current.parent.target.getWorldPosition(pu))}));const x=i.useMemo((()=>{const e=new s.CylinderGeometry(t,r,c,128,64,!0);return e.applyMatrix4((new s.Matrix4).makeTranslation(0,-c/2,0)),e.applyMatrix4((new s.Matrix4).makeRotationX(-Math.PI/2)),e}),[u,c,t,r]);return i.createElement("spotLight",n({ref:f,angle:u,color:l,distance:c},p),i.createElement("mesh",{ref:m,geometry:x,raycast:()=>null},i.createElement("primitive",{object:y,attach:"material","uniforms-opacity-value":e,"uniforms-lightColor-value":l,"uniforms-attenuation-value":h,"uniforms-anglePower-value":d,"uniforms-depth-value":a,"uniforms-cameraNear-value":v.near,"uniforms-cameraFar-value":v.far,"uniforms-resolution-value":a?[g.width*A,g.height*A]:[0,0]})))})),mu=i.forwardRef((({args:e,map:t,toneMapped:r=!1,color:a="white",form:l="rect",intensity:c=1,scale:u=1,target:h,children:d,...p},f)=>{const m=i.useRef(null);return i.useLayoutEffect((()=>{d||p.material||((0,o.applyProps)(m.current.material,{color:a}),m.current.material.color.multiplyScalar(c))}),[a,c,d,p.material]),i.useLayoutEffect((()=>{h&&m.current.lookAt(Array.isArray(h)?new s.Vector3(...h):h)}),[h]),u=Array.isArray(u)&&2===u.length?[u[0],u[1],1]:u,i.createElement("mesh",n({ref:M([m,f]),scale:u},p),"circle"===l?i.createElement("ringGeometry",{args:[0,1,64]}):"ring"===l?i.createElement("ringGeometry",{args:[.5,1,64]}):"rect"===l?i.createElement("planeGeometry",null):i.createElement(l,{args:e}),d||(p.material?null:i.createElement("meshBasicMaterial",{toneMapped:r,map:t,side:s.DoubleSide})))}));var gu=r(48756);function vu(e,t,r=new s.Vector3){const n=Math.PI*(e-.5),i=2*Math.PI*(t-.5);return r.x=Math.cos(i),r.y=Math.sin(n),r.z=Math.sin(i),r}const Au=i.forwardRef((({inclination:e=.6,azimuth:t=.1,distance:r=1e3,mieCoefficient:a=.005,mieDirectionalG:o=.8,rayleigh:l=.5,turbidity:c=10,sunPosition:u=vu(e,t),...h},d)=>{const p=i.useMemo((()=>(new s.Vector3).setScalar(r)),[r]),[f]=i.useState((()=>new gu.q));return i.createElement("primitive",n({dispose:void 0,object:f,ref:d,"material-uniforms-mieCoefficient-value":a,"material-uniforms-mieDirectionalG-value":o,"material-uniforms-rayleigh-value":l,"material-uniforms-sunPosition-value":u,"material-uniforms-turbidity-value":c,scale:p},h))}));class yu extends s.ShaderMaterial{constructor(){super({uniforms:{time:{value:0},fade:{value:1}},vertexShader:"\n uniform float time;\n attribute float size;\n varying vec3 vColor;\n void main() {\n vColor = color;\n vec4 mvPosition = modelViewMatrix * vec4(position, 0.5);\n gl_PointSize = size * (30.0 / -mvPosition.z) * (3.0 + sin(mvPosition.x + 2.0 * time + 100.0));\n gl_Position = projectionMatrix * mvPosition;\n }",fragmentShader:"\n uniform sampler2D pointTexture;\n uniform float fade;\n varying vec3 vColor;\n void main() {\n float opacity = 1.0;\n if (fade == 1.0) {\n float d = distance(gl_PointCoord, vec2(0.5, 0.5));\n opacity = 1.0 / (1.0 + exp(16.0 * (d - 0.25)));\n }\n gl_FragColor = vec4(vColor, opacity);\n\n #include \n\t #include \n }"})}}const xu=e=>(new s.Vector3).setFromSpherical(new s.Spherical(e,Math.acos(1-2*Math.random()),2*Math.random()*Math.PI)),bu=i.forwardRef((({radius:e=100,depth:t=50,count:r=5e3,saturation:n=0,factor:a=4,fade:l=!1},c)=>{const u=i.useRef(),[h,d,p]=i.useMemo((()=>{const i=[],o=[],l=Array.from({length:r},(()=>(.5+.5*Math.random())*a)),c=new s.Color;let u=e+t;const h=t/r;for(let e=0;eu.current&&(u.current.uniforms.time.value=e.clock.getElapsedTime())));const[f]=i.useState((()=>new yu));return i.createElement("points",{ref:c},i.createElement("bufferGeometry",{attach:"geometry"},i.createElement("bufferAttribute",{attachObject:["attributes","position"],args:[h,3]}),i.createElement("bufferAttribute",{attachObject:["attributes","color"],args:[d,3]}),i.createElement("bufferAttribute",{attachObject:["attributes","size"],args:[p,1]})),i.createElement("primitive",{dispose:void 0,ref:u,object:f,attach:"material",blending:s.AdditiveBlending,"uniforms-fade-value":l,transparent:!0,vertexColors:!0}))}));function wu({opacity:e=.5,speed:t=.4,width:r=10,depth:n=1.5,segments:a=20,texture:s="https://rawcdn.githack.com/pmndrs/drei-assets/9225a9f1fbd449d9411125c2f419b843d0308c9f/cloud.png",color:l="#ffffff",depthTest:c=!0,...u}){const h=i.useRef(),d=Oa(s),p=i.useMemo((()=>[...new Array(a)].map(((e,n)=>({x:r/2-Math.random()*r,y:r/2-Math.random()*r,scale:.4+Math.sin((n+1)/a*Math.PI)*(10*(.2+Math.random())),density:Math.max(.2,Math.random()),rotation:Math.max(.002,.005*Math.random())*t})))),[r,a,t]);return(0,o.useFrame)((e=>{var t;return null==(t=h.current)?void 0:t.children.forEach(((t,r)=>{t.children[0].rotation.z+=p[r].rotation,t.children[0].scale.setScalar(p[r].scale+(1+Math.sin(e.clock.getElapsedTime()/10))/2*r/10)}))})),i.createElement("group",u,i.createElement("group",{position:[0,0,a/2*n],ref:h},p.map((({x:t,y:r,scale:a,density:s},o)=>i.createElement(yi,{key:o,position:[t,r,-o*n]},i.createElement(Mc,{scale:a,rotation:[0,0,0]},i.createElement("meshStandardMaterial",{map:d,transparent:!0,opacity:a/6*s*e,depthTest:c,color:l})))))))}function _u(e=0,t=1024){const r=Ca((()=>fetch("https://cdn.jsdelivr.net/gh/pmndrs/drei-assets@master/matcaps.json").then((e=>e.json()))),["matcapList"]),n=r[0],a=i.useMemo((()=>Object.keys(r).length),[]),s=`${i.useMemo((()=>"string"==typeof e?e:"number"==typeof e?r[e]:null),[e])||n}${function(e){switch(e){case 64:return"-64px";case 128:return"-128px";case 256:return"-256px";case 512:return"-512px";default:return""}}(t)}.png`,o=`https://rawcdn.githack.com/emmelleppi/matcaps/9b36ccaaf0a24881a39062d05566c9e92be4aa0d/${t}/${s}`;return[Oa(o),o,a]}function Eu(e=0,t={}){const{repeat:r=[1,1],anisotropy:n=1,offset:a=[0,0]}=t,o=Ca((()=>fetch("https://cdn.jsdelivr.net/gh/pmndrs/drei-assets@master/normals/normals.json").then((e=>e.json()))),["normalsList"]),l=i.useMemo((()=>Object.keys(o).length),[]),c=o[0],u=`https://rawcdn.githack.com/pmndrs/drei-assets/7a3104997e1576f83472829815b00880d88b32fb/normals/${o[e]||c}`,h=Oa(u);return i.useLayoutEffect((()=>{h&&(h.wrapS=h.wrapT=s.RepeatWrapping,h.repeat=new s.Vector2(r[0],r[1]),h.offset=new s.Vector2(a[0],a[1]),h.anisotropy=n)}),[h,n,r,a]),[h,u,l]}const Su=new s.Matrix4,Mu=new s.Matrix4,Tu=[],Cu=new s.Mesh;class Iu extends s.Group{constructor(){super(),this.color=new s.Color("white"),this.instance={current:void 0},this.instanceKey={current:void 0}}get geometry(){var e;return null==(e=this.instance.current)?void 0:e.geometry}raycast(e,t){const r=this.instance.current;if(!r)return;if(!r.geometry||!r.material)return;Cu.geometry=r.geometry;const n=r.matrixWorld;let i=r.userData.instances.indexOf(this.instanceKey);if(!(-1===i||i>r.count)){r.getMatrixAt(i,Su),Mu.multiplyMatrices(n,Su),Cu.matrixWorld=Mu,Cu.raycast(e,Tu);for(let e=0,r=Tu.length;e{const c=i.useRef(null),[u,h]=i.useState([]),[[d,p,f]]=i.useState((()=>[new Float32Array(3*r),Float32Array.from({length:3*r},(()=>1)),Float32Array.from({length:r},(()=>1))]));i.useLayoutEffect((()=>{c.current.geometry.drawRange.count=Math.min(r,void 0!==t?t:r,u.length)}),[u,t]),i.useEffect((()=>{c.current.geometry.attributes.position.needsUpdate=!0})),(0,o.useFrame)((()=>{for(c.current.updateMatrix(),c.current.updateMatrixWorld(),Fu.copy(c.current.matrixWorld).invert(),Bu=0;Bu{const e={};for(Bu=0;Bu({...e,[t]:e=>{var r,n,i;const a=null==(r=u[e.index])?void 0:r.current;return null==a||null==(n=a.__r3f)||null==(i=n.handlers)?void 0:i[t]({...e,object:a})}})),{})}),[e,u]),g=i.useMemo((()=>({subscribe:e=>(h((t=>[...t,e])),()=>h((t=>t.filter((t=>t.current!==e.current)))))})),[]);return i.createElement("points",n({matrixAutoUpdate:!1,ref:M([l,c])},m,a),i.createElement("bufferGeometry",null,i.createElement("bufferAttribute",{attachObject:["attributes","position"],count:d.length/3,array:d,itemSize:3,usage:s.DynamicDrawUsage}),i.createElement("bufferAttribute",{attachObject:["attributes","color"],count:p.length/3,array:p,itemSize:3,usage:s.DynamicDrawUsage}),i.createElement("bufferAttribute",{attachObject:["attributes","size"],count:f.length,array:f,itemSize:1,usage:s.DynamicDrawUsage})),i.createElement(Pu.Provider,{value:g},e))})),Uu=i.forwardRef((({children:e,...t},r)=>{i.useMemo((()=>(0,o.extend)({Position:Iu})),[]);const a=i.useRef(),{subscribe:s}=i.useContext(Pu);return i.useLayoutEffect((()=>s(a)),[]),i.createElement("position",n({ref:M([r,a])},t),e)})),Nu=i.forwardRef((({children:e,positions:t,colors:r,sizes:a,stride:l=3,...c},u)=>{const h=i.useRef(null);return(0,o.useFrame)((()=>{const e=h.current.geometry.attributes;e.position.needsUpdate=!0,r&&(e.color.needsUpdate=!0),a&&(e.size.needsUpdate=!0)})),i.createElement("points",n({ref:M([u,h])},c),i.createElement("bufferGeometry",null,i.createElement("bufferAttribute",{attachObject:["attributes","position"],count:t.length/l,array:t,itemSize:l,usage:s.DynamicDrawUsage}),r&&i.createElement("bufferAttribute",{attachObject:["attributes","color"],count:r.length/l,array:r,itemSize:3,usage:s.DynamicDrawUsage}),a&&i.createElement("bufferAttribute",{attachObject:["attributes","size"],count:a.length/l,array:a,itemSize:1,usage:s.DynamicDrawUsage})),e)})),ku=i.forwardRef(((e,t)=>e.positions instanceof Float32Array?i.createElement(Nu,n({},e,{ref:t})):i.createElement(Du,n({},e,{ref:t}))));var Ou=r(45697),Gu=r.n(Ou);function zu(e){return Vu(e.children,e.components)}function Vu(e,t,r){if(r=r||[],!t[0])return e(r);function n(n){return Vu(e,t.slice(1),r.concat([n]))}return"function"==typeof t[0]?t[0]({results:r,render:n}):(0,i.cloneElement)(t[0],{children:n})}let Hu,Qu;zu.propTypes={children:Gu().func.isRequired,components:Gu().arrayOf(Gu().oneOfType([Gu().element,Gu().func])).isRequired};const ju=i.createContext(null),Wu=new s.Matrix4,Xu=new s.Matrix4,Yu=new s.Matrix4;new s.Color;const qu=new s.Vector3,Ku=new s.Quaternion,Ju=new s.Vector3,Zu=i.forwardRef((({context:e,children:t,...r},a)=>{i.useMemo((()=>(0,o.extend)({Position:Iu})),[]);const s=i.useRef(),{subscribe:l,getParent:c}=i.useContext(e||ju);return i.useLayoutEffect((()=>l(s)),[]),i.createElement("position",n({instance:c(),instanceKey:s,ref:M([a,s])},r),t)})),$u=i.forwardRef((({children:e,range:t,limit:r=1e3,frames:a=1/0,...l},c)=>{const[{context:u,instance:h}]=i.useState((()=>{const e=i.createContext(null);return{context:e,instance:i.forwardRef(((t,r)=>i.createElement(Zu,n({context:e},t,{ref:r}))))}})),d=i.useRef(null),[p,f]=i.useState([]),[[m,g]]=i.useState((()=>{const e=new Float32Array(16*r);for(Hu=0;Hu1)))]}));i.useLayoutEffect((()=>{d.current.count=d.current.instanceMatrix.updateRange.count=d.current.instanceColor.updateRange.count=Math.min(r,void 0!==t?t:r,p.length)}),[p,t]),i.useEffect((()=>{d.current.instanceMatrix.needsUpdate=!0}));let v=0;(0,o.useFrame)((()=>{if(a===1/0||v({getParent:()=>d,subscribe:e=>(f((t=>[...t,e])),()=>f((t=>t.filter((t=>t.current!==e.current)))))})),[]);return i.createElement("instancedMesh",n({userData:{instances:p},matrixAutoUpdate:!1,ref:M([c,d]),args:[null,null,0],raycast:()=>null},l),i.createElement("instancedBufferAttribute",{attach:"instanceMatrix",count:m.length/16,array:m,itemSize:16,usage:s.DynamicDrawUsage}),i.createElement("instancedBufferAttribute",{attach:"instanceColor",count:g.length/3,array:g,itemSize:3,usage:s.DynamicDrawUsage}),"function"==typeof e?i.createElement(u.Provider,{value:A},e(h)):i.createElement(ju.Provider,{value:A},e))}));function eh({meshes:e,children:t,...r}){const a=Array.isArray(e);if(!a)for(const n of Object.keys(e))e[n].isMesh||delete e[n];return i.createElement(zu,{components:(a?e:Object.values(e)).map((({geometry:e,material:t})=>i.createElement($u,n({key:e.uuid,geometry:e,material:t},r))))},(r=>a?t(...r):t(Object.keys(e).filter((t=>e[t].isMesh)).reduce(((e,t,n)=>({...e,[t]:r[n]})),{}))))}var th=r(3415);const rh=i.createContext(null),nh=e=>e instanceof s.Vector3?e.toArray():e,ih=i.forwardRef(((e,t)=>{const{limit:r=1e3,lineWidth:a=1,children:l,...c}=e,[u,h]=i.useState([]),[d]=i.useState((()=>new xi.w)),[p]=i.useState((()=>new bi.Y)),[f]=i.useState((()=>new th.z)),[m]=i.useState((()=>new s.Vector2(512,512))),[g]=i.useState((()=>Array(6*r).fill(0))),[v]=i.useState((()=>Array(6*r).fill(0))),A=i.useMemo((()=>({subscribe:e=>(h((t=>[...t,e])),()=>h((t=>t.filter((t=>t.current!==e.current)))))})),[]);return(0,o.useFrame)((()=>{for(let i=0;i{const r=i.useContext(rh);if(!r)throw"Segment must used inside Segments component.";const a=i.useRef();return i.useMemo((()=>(0,o.extend)({SegmentObject:ah})),[]),i.useLayoutEffect((()=>r.subscribe(a)),[]),i.createElement("segmentObject",n({ref:M([a,t])},e))})),oh=i.forwardRef((({children:e,distances:t,...r},a)=>{const s=i.useRef(null);return i.useLayoutEffect((()=>{const{current:e}=s;e.levels.length=0,e.children.forEach(((r,n)=>e.levels.push({object:r,distance:t[n]})))})),(0,o.useFrame)((e=>{var t;return null==(t=s.current)?void 0:t.update(e.camera)})),i.createElement("lOD",n({ref:M([s,a])},r),e)}));function lh({all:e,scene:t,camera:r}){const n=(0,o.useThree)((({gl:e})=>e)),a=(0,o.useThree)((({camera:e})=>e)),l=(0,o.useThree)((({scene:e})=>e));return i.useLayoutEffect((()=>{const i=[];e&&(t||l).traverse((e=>{!1===e.visible&&(i.push(e),e.visible=!0)})),n.compile(t||l,r||a);const o=new s.WebGLCubeRenderTarget(128);new s.CubeCamera(.01,1e5,o).update(n,t||l),o.dispose(),i.forEach((e=>e.visible=!1))}),[]),null}function ch(){const e=(0,o.useThree)((e=>e.gl));return(0,i.useEffect)((()=>(e.shadowMap.autoUpdate=!1,e.shadowMap.needsUpdate=!0,()=>{e.shadowMap.autoUpdate=e.shadowMap.needsUpdate=!0})),[e.shadowMap]),null}const uh=new s.Matrix4,hh=new s.Ray,dh=new s.Sphere,ph=new s.Vector3;function fh(e,t){const r=this.geometry,n=this.material,i=this.matrixWorld;void 0!==n&&(null===r.boundingSphere&&r.computeBoundingSphere(),dh.copy(r.boundingSphere),dh.applyMatrix4(i),!1!==e.ray.intersectsSphere(dh)&&(uh.copy(i).invert(),hh.copy(e.ray).applyMatrix4(uh),null!==r.boundingBox&&null===hh.intersectBox(r.boundingBox,ph)||t.push({distance:ph.distanceTo(e.ray.origin),point:ph.clone(),object:this})))}function mh({pixelated:e}){const t=(0,o.useThree)((e=>e.gl)),r=(0,o.useThree)((e=>e.internal.active)),n=(0,o.useThree)((e=>e.performance.current)),a=(0,o.useThree)((e=>e.viewport.initialDpr)),s=(0,o.useThree)((e=>e.setDpr));return i.useEffect((()=>{const n=t.domElement;return()=>{r&&s(a),e&&n&&(n.style.imageRendering="auto")}}),[]),i.useEffect((()=>{s(n*a),e&&t.domElement&&(t.domElement.style.imageRendering=1===n?"auto":"pixelated")}),[n]),null}function gh(){const e=(0,o.useThree)((e=>e.get)),t=(0,o.useThree)((e=>e.performance.current));return i.useEffect((()=>{const t=e().raycaster.enabled;return()=>{e().raycaster.enabled=t}}),[]),i.useEffect((()=>{e().raycaster.enabled=1===t}),[t]),null}},64232:(e,t,r)=>{"use strict";r.r(t),r.d(t,{Canvas:()=>K,ReactThreeFiber:()=>w,_roots:()=>ne,act:()=>pe,addAfterEffect:()=>V,addEffect:()=>z,addTail:()=>H,advance:()=>se,applyProps:()=>le,context:()=>D,createPortal:()=>me,dispose:()=>de,events:()=>W,extend:()=>P,invalidate:()=>ae,reconciler:()=>oe,render:()=>ce,unmountComponentAtNode:()=>he,useFrame:()=>Z,useGraph:()=>ee,useLoader:()=>re,useThree:()=>J});var n=r(99477),i=r(67294),a=r(14671),s=r(60374),o=r(76525),l=r.n(o),c=r(63840),u=r(64063),h=r.n(u);const d=[];function p(e,t,r,n=0,i=!1){for(const s of t)if(h()(r,s.args)){if(i)return;if(s.error)throw s.error;if(s.response)return s.response;throw s.promise}const a={args:r,promise:e(...r).then((e=>a.response=null==e||e)).catch((e=>a.error=null!=e?e:"unknown error")).then((()=>{n>0&&setTimeout((()=>{const e=t.indexOf(a);-1!==e&&t.splice(e,1)}),n)}))};if(t.push(a),!i)throw a.promise}function f(e,...t){if(void 0===t||0===t.length)e.splice(0,e.length);else{const r=e.find((e=>h()(t,e.args)));if(r){const t=e.indexOf(r);-1!==t&&e.splice(t,1)}}}function m(e,...t){return p(e,d,t,m.lifespan)}m.lifespan=0,m.clear=(...e)=>f(d,...e),m.preload=(e,...t)=>{p(e,d,t,m.lifespan,!0)},m.peek=(...e)=>{var t;return null==(t=d.find((t=>h()(e,t.args))))?void 0:t.response};var g=r(20296),v=r.n(g);function A(e){let{debounce:t,scroll:r,polyfill:n,offsetSize:a}=void 0===e?{debounce:0,scroll:!1,offsetSize:!1}:e;const s=n||("undefined"==typeof window?class{}:window.ResizeObserver);if(!s)throw new Error("This browser does not support ResizeObserver out of the box. See: https://github.com/react-spring/react-use-measure/#resize-observer-polyfills");const[o,l]=(0,i.useState)({left:0,top:0,width:0,height:0,bottom:0,right:0,x:0,y:0}),c=(0,i.useRef)({element:null,scrollContainers:null,resizeObserver:null,lastBounds:o}),u=t?"number"==typeof t?t:t.scroll:null,h=t?"number"==typeof t?t:t.resize:null,d=(0,i.useRef)(!1);(0,i.useEffect)((()=>(d.current=!0,()=>{d.current=!1})));const[p,f,m]=(0,i.useMemo)((()=>{const e=()=>{if(!c.current.element)return;const{left:e,top:t,width:r,height:n,bottom:i,right:s,x:o,y:u}=c.current.element.getBoundingClientRect(),h={left:e,top:t,width:r,height:n,bottom:i,right:s,x:o,y:u};c.current.element instanceof HTMLElement&&a&&(h.height=c.current.element.offsetHeight,h.width=c.current.element.offsetWidth),Object.freeze(h),d.current&&!b(c.current.lastBounds,h)&&l(c.current.lastBounds=h)};return[e,h?v()(e,h):e,u?v()(e,u):e]}),[l,a,u,h]);function g(){c.current.scrollContainers&&(c.current.scrollContainers.forEach((e=>e.removeEventListener("scroll",m,!0))),c.current.scrollContainers=null),c.current.resizeObserver&&(c.current.resizeObserver.disconnect(),c.current.resizeObserver=null)}function A(){c.current.element&&(c.current.resizeObserver=new s(m),c.current.resizeObserver.observe(c.current.element),r&&c.current.scrollContainers&&c.current.scrollContainers.forEach((e=>e.addEventListener("scroll",m,{capture:!0,passive:!0}))))}var x,w,_;return x=m,w=Boolean(r),(0,i.useEffect)((()=>{if(w){const e=x;return window.addEventListener("scroll",e,{capture:!0,passive:!0}),()=>{window.removeEventListener("scroll",e,!0)}}}),[x,w]),_=f,(0,i.useEffect)((()=>{const e=_;return window.addEventListener("resize",e),()=>{window.removeEventListener("resize",e)}}),[_]),(0,i.useEffect)((()=>{g(),A()}),[r,m,f]),(0,i.useEffect)((()=>g),[]),[e=>{e&&e!==c.current.element&&(g(),c.current.element=e,c.current.scrollContainers=y(e),A())},o,p]}function y(e){const t=[];if(!e||e===document.body)return t;const{overflow:r,overflowX:n,overflowY:i}=window.getComputedStyle(e);return[r,n,i].some((e=>"auto"===e||"scroll"===e))&&t.push(e),[...t,...y(e.parentElement)]}const x=["x","y","top","bottom","left","right","width","height"],b=(e,t)=>x.every((r=>e[r]===t[r]));var w=Object.freeze({__proto__:null});const _={obj:e=>e===Object(e)&&!_.arr(e)&&"function"!=typeof e,fun:e=>"function"==typeof e,str:e=>"string"==typeof e,num:e=>"number"==typeof e,und:e=>void 0===e,arr:e=>Array.isArray(e),equ(e,t){if(typeof e!=typeof t||!!e!=!!t)return!1;if(_.str(e)||_.num(e)||_.obj(e))return e===t;if(_.arr(e)&&e==t)return!0;let r;for(r in e)if(!(r in t))return!1;for(r in t)if(e[r]!==t[r])return!1;return!_.und(r)||e===t}};function E(e){return(e.eventObject||e.object).uuid+"/"+e.index}function S(e){const t=new n.Vector3;function r(t){const{internal:r}=e.getState(),n=t.offsetX-r.initialClick[0],i=t.offsetY-r.initialClick[1];return Math.round(Math.sqrt(n*n+i*i))}function i(e){return e.filter((e=>["Move","Over","Enter","Out","Leave"].some((t=>{var r;return null==(r=e.__r3f.handlers)?void 0:r["onPointer"+t]}))))}function a(t){const{internal:r}=e.getState();Array.from(r.hovered.values()).forEach((e=>{if(!t.length||!t.find((t=>t.object===e.object&&t.index===e.index))){const n=e.eventObject.__r3f.handlers;if(r.hovered.delete(E(e)),n){const r={...e,intersections:t||[]};null==n.onPointerOut||n.onPointerOut(r),null==n.onPointerLeave||n.onPointerLeave(r)}}}))}function s(e,t){t.forEach((t=>{var r;return null==(r=t.__r3f.handlers)||null==r.onPointerMissed?void 0:r.onPointerMissed(e)}))}return{handlePointer:n=>{switch(n){case"onPointerLeave":case"onPointerCancel":return()=>a([]);case"onLostPointerCapture":return t=>{"pointerId"in t&&e.getState().internal.capturedMap.delete(t.pointerId),a([])}}return o=>{const{onPointerMissed:l,internal:c}=e.getState();!function(t){var r;const n=e.getState(),{raycaster:i,mouse:a,camera:s,size:o}=n,{offsetX:l,offsetY:c}=null!=(r=null==i.computeOffsets?void 0:i.computeOffsets(t,n))?r:t,{width:u,height:h}=o;a.set(l/u*2-1,-c/h*2+1),i.setFromCamera(a,s)}(o);const u="onPointerMove"===n,h=function(t,r){const{internal:n}=e.getState();return"pointerId"in r&&n.capturedMap.has(r.pointerId)&&t.push(...n.capturedMap.get(r.pointerId).values()),t}(function(t){const r=e.getState(),{raycaster:n,internal:i}=r;if(!n.enabled)return[];const a=new Set,s=[],o=t?t(i.interaction):i.interaction;let l=n.intersectObjects(o,!0).filter((e=>{const t=E(e);return!a.has(t)&&(a.add(t),!0)}));n.filter&&(l=n.filter(l,r));for(const e of l){let t=e.object;for(;t;){var c;(null==(c=t.__r3f)?void 0:c.handlers)&&s.push({...e,eventObject:t}),t=t.parent}}return s}(u?i:void 0),o);u&&a(h),function(n,i,s){const{raycaster:o,mouse:l,camera:c,internal:u}=e.getState();if(n.length){const e=t.set(l.x,l.y,0).unproject(c),h="click"===i.type?r(i):0,d=e=>i.target.releasePointerCapture(e),p={stopped:!1};for(const t of n){const r=e=>{var r,n;return null!=(r=null==(n=u.capturedMap.get(e))?void 0:n.has(t.eventObject))&&r},f=e=>{u.capturedMap.has(e)?u.capturedMap.get(e).set(t.eventObject,t):u.capturedMap.set(e,new Map([[t.eventObject,t]])),i.target.setPointerCapture(e)};let m={};for(let e in Object.getPrototypeOf(i)){let t=i[e];"function"!=typeof t&&(m[e]=t)}let g={...t,...m,spaceX:l.x,spaceY:l.y,intersections:n,stopped:p.stopped,delta:h,unprojectedPoint:e,ray:o.ray,camera:c,stopPropagation:()=>{const e="pointerId"in i&&u.capturedMap.get(i.pointerId);(!e||e.has(t.eventObject))&&(g.stopped=p.stopped=!0,u.hovered.size&&Array.from(u.hovered.values()).find((e=>e.eventObject===t.eventObject)))&&a([...n.slice(0,n.indexOf(t)),t])},target:{hasPointerCapture:r,setPointerCapture:f,releasePointerCapture:d},currentTarget:{hasPointerCapture:r,setPointerCapture:f,releasePointerCapture:d},sourceEvent:i,nativeEvent:i};if(s(g),!0===p.stopped)break}}}(h,o,(e=>{const t=e.eventObject,r=t.__r3f.handlers;if(r)if(u){if(r.onPointerOver||r.onPointerEnter||r.onPointerOut||r.onPointerLeave){const t=E(e),n=c.hovered.get(t);n?n.stopped&&e.stopPropagation():(c.hovered.set(t,e),null==r.onPointerOver||r.onPointerOver(e),null==r.onPointerEnter||r.onPointerEnter(e))}null==r.onPointerMove||r.onPointerMove(e)}else{const i=null==r?void 0:r[n];i&&("onClick"!==n&&"onContextMenu"!==n&&"onDoubleClick"!==n||c.initialHits.includes(t))&&(i(e),s(o,c.interaction.filter((e=>e!==t))))}})),"onPointerDown"===n&&(c.initialClick=[o.offsetX,o.offsetY],c.initialHits=h.map((e=>e.eventObject))),"onClick"!==n&&"onContextMenu"!==n&&"onDoubleClick"!==n||h.length||r(o)<=2&&(s(o,c.interaction),l&&l(o))}}}}const M=e=>e&&!!e.getState,T=(e,t)=>{var r,n;return{root:M(e)?e:null!=(r=null==(n=e.__r3f)?void 0:n.root)?r:t.__r3f.root,container:M(e)?e.getState().scene:e}},C="__default",I={},B=["children","key","ref"];let R={},P=e=>{R={...R,...e}};function F(e,t){const r=e;return(null!=t&&t.instance||!r.__r3f)&&(r.__r3f={root:null,memoizedProps:{},objects:[],...t}),e}const L=e=>e&&e.isOrthographicCamera,D=i.createContext(null);function U(e,t){const r=t.length;return t.push(e),()=>{t.splice(r,1)}}let N,k=[],O=[],G=[];const z=e=>U(e,k),V=e=>U(e,O),H=e=>U(e,G);function Q(e,t){for(N=0;N({...e,[r]:t(r)})),{}),connect:t=>{var n;const{set:i,events:a}=e.getState();null==a.disconnect||a.disconnect(),i((e=>({events:{...e.events,connected:t}}))),Object.entries(null!=(n=null==a?void 0:a.handlers)?n:[]).forEach((([e,n])=>{const[i,a]=r[e];t.addEventListener(i,n,{passive:a})}))},disconnect:()=>{const{set:t,events:n}=e.getState();var i;n.connected&&(Object.entries(null!=(i=n.handlers)?i:[]).forEach((([e,t])=>{if(n&&n.connected instanceof HTMLElement){const[i]=r[e];n.connected.removeEventListener(i,t)}})),t((e=>({events:{...e.events,connected:!1}}))))}}}const X="undefined"!=typeof window?i.useLayoutEffect:i.useEffect;function Y({set:e}){return X((()=>(e(new Promise((()=>null))),()=>e(!1))),[]),null}class q extends i.Component{constructor(...e){super(...e),this.state={error:!1}}componentDidCatch(e){this.props.set(e)}render(){return this.state.error?null:this.props.children}}function K({children:e,fallback:t,tabIndex:r,resize:n,id:a,style:s,className:o,events:l,...c}){const[u,h]=A({scroll:!0,debounce:{scroll:50,resize:0},...n}),d=i.useRef(null),[p,f]=i.useState(!1),[m,g]=i.useState(!1);if(p)throw p;if(m)throw m;return X((()=>{h.width>0&&h.height>0&&ce(i.createElement(q,{set:g},i.createElement(i.Suspense,{fallback:i.createElement(Y,{set:f})},e)),d.current,{...c,size:h,events:l||W})}),[h,e]),X((()=>{const e=d.current;return()=>he(e)}),[]),i.createElement("div",{ref:u,id:a,className:o,tabIndex:r,style:{position:"relative",width:"100%",height:"100%",overflow:"hidden",...s}},i.createElement("canvas",{ref:d,style:{display:"block"}},t))}function J(e=(e=>e),t){const r=i.useContext(D);if(!r)throw"R3F hooks can only be used within the Canvas component!";return r(e,t)}function Z(e,t=0){const{subscribe:r}=i.useContext(D).getState().internal,n=i.useRef(e);return i.useLayoutEffect((()=>{n.current=e}),[e]),i.useLayoutEffect((()=>{const e=r(n,t);return()=>e()}),[t,r]),null}function $(e){const t={nodes:{},materials:{}};return e&&e.traverse((e=>{e.name&&(t.nodes[e.name]=e),e.material&&!t.materials[e.material.name]&&(t.materials[e.material.name]=e.material)})),t}function ee(e){return i.useMemo((()=>$(e)),[e])}function te(e,t){return function(r,...n){const i=new r;return e&&e(i),Promise.all(n.map((e=>new Promise(((r,n)=>i.load(e,(e=>{e.scene&&Object.assign(e,$(e.scene)),r(e)}),t,(t=>n(`Could not load ${e}: ${t.message}`))))))))}}function re(e,t,r,n){const i=Array.isArray(t)?t:[t],a=m(te(r,n),e,...i);return Array.isArray(t)?a:a[0]}q.getDerivedStateFromError=()=>({error:!0}),re.preload=function(e,t,r){const n=Array.isArray(t)?t:[t];return m.preload(te(r),e,...n)};const ne=new Map,ie=["legacy","blocking","concurrent"],{invalidate:ae,advance:se}=function(e){let t,r=!1;function n(i){if(r=!0,t=0,Q(k,i),e.forEach((e=>{const r=e.store.getState();r.internal.active&&("always"===r.frameloop||r.internal.frames>0)&&(t+=j(i,r))})),Q(O,i),t>0)return requestAnimationFrame(n);Q(G,i),r=!1}return{loop:n,invalidate:function t(i){if(!i)return e.forEach((e=>t(e.store.getState())));!i.vr&&i.internal.active&&"never"!==i.frameloop&&(i.internal.frames=Math.min(60,i.internal.frames+1),r||(r=!0,requestAnimationFrame(n)))},advance:function(t,r=!0,n){r&&Q(k,t),n?j(t,n):e.forEach((e=>j(t,e.store.getState()))),r&&Q(O,t)}}}(ne),{reconciler:oe,applyProps:le}=function(e){function t(e,t,a={},s=!1){var o,l,c;const u=null!=(o=null==e?void 0:e.__r3f)?o:{},h=u.root,d=null!=(l=null==h||null==h.getState?void 0:h.getState())?l:{},p=[],f=[],m={};let g=0;Object.entries(t).forEach((([e,t])=>{-1===B.indexOf(e)&&(m[e]=t)})),u.memoizedProps&&u.memoizedProps.args&&(m.args=u.memoizedProps.args),u.memoizedProps&&u.memoizedProps.attach&&(m.attach=u.memoizedProps.attach),e.__r3f&&(e.__r3f.memoizedProps=m);let v=Object.keys(t);for(g=0;g-1&&delete x[v[g]];const b=Object.entries(x);for(g=0;g0){if(b.forEach((([t,i])=>{if(!f.includes(t)){let a=e,s=a[t];if(t.includes("-")){const r=t.split("-");if(s=r.reduce(((e,t)=>e[t]),e),!s||!s.set){const[n,...i]=r.reverse();a=i.reverse().reduce(((e,t)=>e[t]),e),t=n}}if(i===C+"remove")if(s&&s.constructor)i=new s.constructor(m.args);else if(a.constructor){const e=new a.constructor(a.__r3f.memoizedProps.args);i=e[s],e.dispose&&e.dispose()}else i=0;if(s&&s.set&&(s.copy||s instanceof n.Layers)){if(Array.isArray(i))s.fromArray?s.fromArray(i):s.set(...i);else if(s.copy&&i&&i.constructor&&s.constructor.name===i.constructor.name)s.copy(i);else if(void 0!==i){const e=s instanceof n.Color;!e&&s.setScalar?s.setScalar(i):s instanceof n.Layers&&i instanceof n.Layers?s.mask=i.mask:s.set(i),!d.linear&&e&&s.convertSRGBToLinear()}}else a[t]=i,!d.linear&&a[t]instanceof n.Texture&&(a[t].encoding=n.sRGBEncoding);r(e)}})),s&&h&&e.raycast&&u.handlers){u.handlers=void 0;const t=d.internal.interaction.indexOf(e);t>-1&&d.internal.interaction.splice(t,1)}f.length&&(s&&h&&e.raycast&&d.internal.interaction.push(e),u.handlers=f.reduce(((e,r)=>({...e,[r]:t[r]})),{})),e.parent&&i(e)}}function r(e){var t,r;const n=null==(t=e.__r3f)||null==(r=t.root)||null==r.getState?void 0:r.getState();n&&0===n.internal.frames&&n.invalidate()}function i(e){null==e.onUpdate||e.onUpdate(e)}function a(e,{args:r=[],...i},a,s,o){let l,c=`${e[0].toUpperCase()}${e.slice(1)}`;if(!M(a)&&o){const e=t=>t.return?e(t.return):t.stateNode&&t.stateNode.containerInfo;a=e(o)}if(!a||!M(a))throw`No valid root for ${c}!`;if("primitive"===e){if(void 0===i.object)throw"Primitives without 'object' are invalid!";l=F(i.object,{root:a,instance:!0})}else{const e=R[c]||n[c];if(!e)throw`${c} is not part of the THREE namespace! Did you forget to extend? See: https://github.com/pmndrs/react-three-fiber/blob/master/markdown/api.md#using-3rd-party-objects-declaratively`;const t=_.arr(r);l=F(t?new e(...r):new e(r),{root:a,memoizedProps:{args:t&&0===r.length?null:r}})}return c.endsWith("Geometry")?i={attach:"geometry",...i}:c.endsWith("Material")&&(i={attach:"material",...i}),t(l,i,{}),l}function s(e,t){let n=!1;t&&(t.attachArray?(_.arr(e[t.attachArray])||(e[t.attachArray]=[]),e[t.attachArray].push(t)):t.attachObject?(_.obj(e[t.attachObject[0]])||(e[t.attachObject[0]]={}),e[t.attachObject[0]][t.attachObject[1]]=t):t.attach&&!_.fun(t.attach)?e[t.attach]=t:t.isObject3D&&(e.add(t),n=!0),n||(e.__r3f.objects.push(t),t.parent=e),i(t),r(t))}function o(e,t,n){let a=!1;if(t){if(t.attachArray){const r=e[t.attachArray];_.arr(r)||(e[t.attachArray]=[]),r.splice(r.indexOf(n),0,t)}else{if(t.attachObject||t.attach&&!_.fun(t.attach))return a=!0,s(e,t);if(t.isObject3D){t.parent=e,t.dispatchEvent({type:"added"});const r=e.children.filter((e=>e!==t)),i=r.indexOf(n);e.children=[...r.slice(0,i),t,...r.slice(i)],a=!0}}a||(e.__r3f.objects.push(t),t.parent=e),i(t),r(t)}}function u(e,t,r=!1){e&&[...e].forEach((e=>h(t,e,r)))}function h(e,t,n){if(t){var i;if(e.__r3f.objects){const r=e.__r3f.objects.length;e.__r3f.objects=e.__r3f.objects.filter((e=>e!==t));e.__r3f.objects.lengthe!==t));else if(t.attachObject)delete e[t.attachObject[0]][t.attachObject[1]];else if(t.attach&&!_.fun(t.attach))e[t.attach]=null;else if(t.isObject3D){var a;e.remove(t),null!=(a=t.__r3f)&&a.root&&function(e,t){const{internal:r}=e.getState();r.interaction=r.interaction.filter((e=>e!==t)),r.initialHits=r.initialHits.filter((e=>e!==t)),r.hovered.forEach(((e,n)=>{e.eventObject!==t&&e.object!==t||r.hovered.delete(n)}))}(t.__r3f.root,t)}const o=null==(i=t.__r3f)?void 0:i.instance,l=void 0===n?null!==t.dispose&&!o:n;var s;if(!o)u(null==(s=t.__r3f)?void 0:s.objects,t,l),u(t.children,t,l);t.__r3f&&(delete t.__r3f.root,delete t.__r3f.objects,delete t.__r3f.handlers,delete t.__r3f.memoizedProps,o||delete t.__r3f),l&&t.dispose&&"Scene"!==t.type&&(0,c.unstable_runWithPriority)(c.unstable_IdlePriority,(()=>t.dispose())),r(e)}}function d(e,t,r,n){const i=e.parent;if(!i)return;const o=a(t,r,e.__r3f.root);e.children&&(e.children.forEach((e=>s(o,e))),e.children=[]),e.__r3f.objects.forEach((e=>s(o,e))),e.__r3f.objects=[],h(i,e),s(i,o),[n,n.alternate].forEach((e=>{null!==e&&(e.stateNode=o,e.ref&&("function"==typeof e.ref?e.ref(o):e.ref.current=o))}))}return{reconciler:l()({now:c.unstable_now,createInstance:a,removeChild:h,appendChild:s,appendInitialChild:s,insertBefore:o,warnsIfNotActing:!0,supportsMutation:!0,isPrimaryRenderer:!1,scheduleTimeout:_.fun(setTimeout)?setTimeout:void 0,cancelTimeout:_.fun(clearTimeout)?clearTimeout:void 0,setTimeout:_.fun(setTimeout)?setTimeout:void 0,clearTimeout:_.fun(clearTimeout)?clearTimeout:void 0,noTimeout:-1,appendChildToContainer:(e,t)=>{const{container:r,root:n}=T(e,t);r.__r3f.root=n,s(r,t)},removeChildFromContainer:(e,t)=>{const{container:r}=T(e,t);h(r,t)},insertInContainerBefore:(e,t,r)=>{const{container:n}=T(e,t);o(n,t,r)},commitUpdate(e,r,n,i,a,s){if(e.__r3f.instance&&a.object&&a.object!==e)d(e,n,a,s);else{const{args:r=[],...o}=a,{args:l=[],...c}=i;r.some(((e,t)=>_.obj(e)?Object.entries(e).some((([e,r])=>r!==l[t][e])):e!==l[t]))?d(e,n,a,s):t(e,o,c,!0)}},hideInstance(e){e.isObject3D&&(e.visible=!1,r(e))},unhideInstance(e,t){(e.isObject3D&&null==t.visible||t.visible)&&(e.visible=!0,r(e))},hideTextInstance(){throw new Error("Text is not allowed in the R3F tree.")},getPublicInstance:e=>e,getRootHostContext:e=>I,getChildHostContext:e=>I,createTextInstance(){},finalizeInitialChildren:e=>!!e.__r3f.handlers,commitMount(e){e.raycast&&e.__r3f.handlers&&e.__r3f.root.getState().internal.interaction.push(e)},prepareUpdate:()=>I,shouldDeprioritizeSubtree:()=>!1,prepareForCommit:()=>null,preparePortalMount(...e){},resetAfterCommit(){},shouldSetTextContent:()=>!1,clearContainer:()=>!1}),applyProps:t}}();function ce(e,t,{gl:r,size:o,mode:l=ie[1],events:c,onCreated:u,...h}={}){var d,p,f,m,g;o||(o={width:null!=(p=null==(f=t.parentElement)?void 0:f.clientWidth)?p:0,height:null!=(m=null==(g=t.parentElement)?void 0:g.clientHeight)?m:0});let v=ne.get(t),A=null==v?void 0:v.fiber,y=null==v?void 0:v.store,x=null==(d=y)?void 0:d.getState();if(A&&x){const e=x.internal.lastProps;void 0===h.dpr||_.equ(e.dpr,h.dpr)||x.setDpr(h.dpr),void 0===o||_.equ(e.size,o)||x.setSize(o.width,o.height);h.linear!==e.linear&&(he(t),A=void 0)}if(!A){const e=((e,t)=>{return(r=e)&&r.render?e:new n.WebGLRenderer({powerPreference:"high-performance",canvas:t,antialias:!0,alpha:!0,...e});var r})(r,t);h.vr&&(e.xr.enabled=!0,e.setAnimationLoop((e=>se(e,!0)))),y=((e,t,r,i)=>{const{gl:o,size:l,shadows:c=!1,linear:u=!1,flat:h=!1,vr:d=!1,orthographic:p=!1,frameloop:f="always",dpr:m=1,performance:g,clock:v=new n.Clock,raycaster:A,camera:y,onPointerMissed:x}=i;c&&(o.shadowMap.enabled=!0,"object"==typeof c?Object.assign(o.shadowMap,c):o.shadowMap.type=n.PCFSoftShadowMap),u||(h||(o.toneMapping=n.ACESFilmicToneMapping),o.outputEncoding=n.sRGBEncoding),"never"===f&&(v.stop(),v.elapsedTime=0);const b=(0,a.Z)(((a,s)=>{const l=new n.Raycaster,{params:c,...b}=A||{};e(l,{enabled:!0,...b,params:{...l.params,...c}},{});const w=y instanceof n.Camera,_=w?y:p?new n.OrthographicCamera(0,0,0,0,.1,1e3):new n.PerspectiveCamera(75,0,.1,1e3);function E(e){return Array.isArray(e)?Math.min(Math.max(e[0],window.devicePixelRatio),e[1]):e}w||(_.position.z=5,y&&e(_,y,{}),_.lookAt(0,0,0));const S=E(m),M=new n.Vector3,T=new n.Vector3;function C(e=s().camera,t=T,r=s().size){const{width:n,height:i}=r,a=n/i,o=e.getWorldPosition(M).distanceTo(t);if(L(e))return{width:n/e.zoom,height:i/e.zoom,factor:1,distance:o,aspect:a};{const t=e.fov*Math.PI/180,r=2*Math.tan(t/2)*o,s=r*(n/i);return{width:s,height:r,factor:n/s,distance:o,aspect:a}}}let I;const B=e=>a((t=>({performance:{...t.performance,current:e}})));return{gl:o,set:a,get:s,invalidate:()=>t(s()),advance:(e,t)=>r(e,t,s()),linear:u,flat:h,scene:F(new n.Scene),camera:_,raycaster:l,clock:v,mouse:new n.Vector2,vr:d,frameloop:f,onPointerMissed:x,performance:{current:1,min:.5,max:1,debounce:200,...g,regress:()=>{const e=s();I&&clearTimeout(I),e.performance.current!==e.performance.min&&B(e.performance.min),I=setTimeout((()=>B(s().performance.max)),e.performance.debounce)}},size:{width:0,height:0},viewport:{initialDpr:S,dpr:S,width:0,height:0,aspect:0,distance:0,factor:0,getCurrentViewport:C},setSize:(e,t)=>{const r={width:e,height:t};a((e=>({size:r,viewport:{...e.viewport,...C(_,T,r)}})))},setDpr:e=>a((t=>({viewport:{...t.viewport,dpr:E(e)}}))),events:{connected:!1},internal:{active:!1,priority:0,frames:0,lastProps:i,interaction:[],hovered:new Map,subscribers:[],initialClick:[0,0],initialHits:[],capturedMap:new Map,subscribe:(e,t=0)=>(a((({internal:r})=>({internal:{...r,priority:r.priority+(t?1:0),subscribers:[...r.subscribers,{ref:e,priority:t}].sort(((e,t)=>e.priority-t.priority))}}))),()=>{a((({internal:r})=>({internal:{...r,priority:r.priority-(t?1:0),subscribers:r.subscribers.filter((t=>t.ref!==e))}})))})}}}));b.subscribe((()=>{const{camera:e,size:t,viewport:r,internal:i}=b.getState();i.lastProps.camera instanceof n.Camera||(L(e)?(e.left=t.width/-2,e.right=t.width/2,e.top=t.height/2,e.bottom=t.height/-2):e.aspect=t.width/t.height,e.updateProjectionMatrix(),e.updateMatrixWorld()),o.setPixelRatio(r.dpr),o.setSize(t.width,t.height)}),(e=>[e.viewport.dpr,e.size]),s.Z);const w=b.getState();return l&&w.setSize(l.width,l.height),b.subscribe((e=>t(e))),b})(le,ae,se,{gl:e,size:o,...h});const i=y.getState();i.get,A=oe.createContainer(y,ie.indexOf(l),!1,null),ne.set(t,{fiber:A,store:y}),c&&i.set({events:c(y)})}if(y&&A)return oe.updateContainer(i.createElement(ue,{store:y,element:e,onCreated:u,target:t}),A,null,(()=>{})),y;throw"Error creating root!"}function ue({store:e,element:t,onCreated:r,target:n}){return i.useEffect((()=>{const t=e.getState();t.set((e=>({internal:{...e.internal,active:!0}}))),null==t.events.connect||t.events.connect(n),r&&r(t)}),[]),i.createElement(D.Provider,{value:e},t)}function he(e,t){const r=ne.get(e),n=null==r?void 0:r.fiber;if(n){const i=null==r?void 0:r.store.getState();i&&(i.internal.active=!1),oe.updateContainer(null,n,null,(()=>{i&&setTimeout((()=>{var r,n,a;null==i.events.disconnect||i.events.disconnect(),null==(r=i.gl)||null==(n=r.renderLists)||null==n.dispose||n.dispose(),null==(a=i.gl)||null==a.forceContextLoss||a.forceContextLoss(),de(i),ne.delete(e),t&&t(e)}),500)}))}}function de(e){e.dispose&&"Scene"!==e.type&&e.dispose();for(const n in e){var t,r;null==(t=(r=n).dispose)||t.call(r),delete e[n]}}const pe=oe.act,fe=_.fun(Symbol)&&Symbol.for?Symbol.for("react.portal"):60106;function me(e,t,r,n=null){return{$$typeof:fe,key:null==n?null:""+n,children:e,containerInfo:F(t),implementation:r}}oe.injectIntoDevTools({bundleType:0,rendererPackageName:"@react-three/fiber",version:"17.0.2"})},40417:(e,t,r)=>{"use strict";t._b=t.H7=t.LK=t.Kv=t.Mf=t.SE=t.N$=t.oI=t.$9=t.pT=t.fK=t.ej=t.hI=t.Wx=t.Sj=t.r3=t.ue=t.ZW=t.z5=t.Oq=t.P7=t.bS=t.N7=t.Hs=t.dV=t.oC=t.vA=t.ol=t.l$=t.sd=t.dK=t.V3=t.Bw=t.hW=t._o=t.b4=t.nu=t.wd=t.q4=void 0;var n=r(97657);Object.defineProperty(t,"q4",{enumerable:!0,get:function(){return n.VERSION}});var i=r(81183);Object.defineProperty(t,"wd",{enumerable:!0,get:function(){return i.CstParser}}),Object.defineProperty(t,"nu",{enumerable:!0,get:function(){return i.EmbeddedActionsParser}}),Object.defineProperty(t,"b4",{enumerable:!0,get:function(){return i.ParserDefinitionErrorType}}),Object.defineProperty(t,"_o",{enumerable:!0,get:function(){return i.EMPTY_ALT}});var a=r(88200);Object.defineProperty(t,"hW",{enumerable:!0,get:function(){return a.Lexer}}),Object.defineProperty(t,"Bw",{enumerable:!0,get:function(){return a.LexerDefinitionErrorType}});var s=r(58980);Object.defineProperty(t,"V3",{enumerable:!0,get:function(){return s.createToken}}),Object.defineProperty(t,"dK",{enumerable:!0,get:function(){return s.createTokenInstance}}),Object.defineProperty(t,"sd",{enumerable:!0,get:function(){return s.EOF}}),Object.defineProperty(t,"l$",{enumerable:!0,get:function(){return s.tokenLabel}}),Object.defineProperty(t,"ol",{enumerable:!0,get:function(){return s.tokenMatcher}}),Object.defineProperty(t,"vA",{enumerable:!0,get:function(){return s.tokenName}});var o=r(63670);Object.defineProperty(t,"oC",{enumerable:!0,get:function(){return o.getLookaheadPaths}});var l=r(3449);Object.defineProperty(t,"dV",{enumerable:!0,get:function(){return l.LLkLookaheadStrategy}});var c=r(27621);Object.defineProperty(t,"Hs",{enumerable:!0,get:function(){return c.defaultParserErrorProvider}});var u=r(34400);Object.defineProperty(t,"N7",{enumerable:!0,get:function(){return u.EarlyExitException}}),Object.defineProperty(t,"bS",{enumerable:!0,get:function(){return u.isRecognitionException}}),Object.defineProperty(t,"P7",{enumerable:!0,get:function(){return u.MismatchedTokenException}}),Object.defineProperty(t,"Oq",{enumerable:!0,get:function(){return u.NotAllInputParsedException}}),Object.defineProperty(t,"z5",{enumerable:!0,get:function(){return u.NoViableAltException}});var h=r(58232);Object.defineProperty(t,"ZW",{enumerable:!0,get:function(){return h.defaultLexerErrorProvider}});var d=r(93062);Object.defineProperty(t,"ue",{enumerable:!0,get:function(){return d.Alternation}}),Object.defineProperty(t,"r3",{enumerable:!0,get:function(){return d.Alternative}}),Object.defineProperty(t,"Sj",{enumerable:!0,get:function(){return d.NonTerminal}}),Object.defineProperty(t,"Wx",{enumerable:!0,get:function(){return d.Option}}),Object.defineProperty(t,"hI",{enumerable:!0,get:function(){return d.Repetition}}),Object.defineProperty(t,"ej",{enumerable:!0,get:function(){return d.RepetitionMandatory}}),Object.defineProperty(t,"fK",{enumerable:!0,get:function(){return d.RepetitionMandatoryWithSeparator}}),Object.defineProperty(t,"pT",{enumerable:!0,get:function(){return d.RepetitionWithSeparator}}),Object.defineProperty(t,"$9",{enumerable:!0,get:function(){return d.Rule}}),Object.defineProperty(t,"oI",{enumerable:!0,get:function(){return d.Terminal}});var p=r(93062);Object.defineProperty(t,"N$",{enumerable:!0,get:function(){return p.serializeGrammar}}),Object.defineProperty(t,"SE",{enumerable:!0,get:function(){return p.serializeProduction}}),Object.defineProperty(t,"Mf",{enumerable:!0,get:function(){return p.GAstVisitor}});var f=r(82688);Object.defineProperty(t,"Kv",{enumerable:!0,get:function(){return f.generateCstDts}}),t.LK=function(){console.warn("The clearCache function was 'soft' removed from the Chevrotain API.\n\t It performs no action other than printing this message.\n\t Please avoid using it as it will be completely removed in the future")};var m=r(77141);Object.defineProperty(t,"H7",{enumerable:!0,get:function(){return m.createSyntaxDiagramsCode}});var g=function(){throw new Error("The Parser class has been deprecated, use CstParser or EmbeddedActionsParser instead.\t\nSee: https://chevrotain.io/docs/changes/BREAKING_CHANGES.html#_7-0-0")};t._b=g},77141:(e,t,r)=>{"use strict";Object.defineProperty(t,"__esModule",{value:!0}),t.createSyntaxDiagramsCode=void 0;var n=r(97657);t.createSyntaxDiagramsCode=function(e,t){var r=void 0===t?{}:t,i=r.resourceBase,a=void 0===i?"https://unpkg.com/chevrotain@".concat(n.VERSION,"/diagrams/"):i,s=r.css,o=void 0===s?"https://unpkg.com/chevrotain@".concat(n.VERSION,"/diagrams/diagrams.css"):s;return'\n\x3c!-- This is a generated file --\x3e\n\n\n\n\n'+"\n\n")+"\n + \ No newline at end of file diff --git a/build.json b/build.json index 344289ef..99b0ff0f 100644 --- a/build.json +++ b/build.json @@ -1,15 +1,15 @@ { - "timestamp": "2025-04-24T18:43:43Z", + "timestamp": "2025-04-25T08:12:44Z", "git": { - "commit": "94c2339e04bbe4826a7e8279682f3faa3386da0a", + "commit": "3512fea9dc9c66e7f0edc56776df5645ada72bd4", "branch": "origin/master", "url": "git@github.com:vacp2p/vac.dev.git" }, "build": { - "id": "4445", - "number": "4445", + "id": "4446", + "number": "4446", "name": "website/vac.dev", "slave": "linux-02", - "url": "https://ci.infra.status.im/job/website/job/vac.dev/4445/" + "url": "https://ci.infra.status.im/job/website/job/vac.dev/4446/" } } \ No newline at end of file diff --git a/community/index.html b/community/index.html index 9cbd517f..4acf6ff2 100644 --- a/community/index.html +++ b/community/index.html @@ -1,26 +1,26 @@ - + - -Join the community | Vac Research + +Join the community | Vac Research - - - - + + + + -
-

Join the community

Join the Vac Community!

Keep up to date with our latest research by connecting with us on our communities channels.
Follow us on X
Join the community on Discord
Share your thoughts on the latest research on the Vac research forum
- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Join the community

+

Join the Vac Community!
+Keep up to date with our latest research by connecting with us on our communities channels.

+
Follow us on X
Join the community on Discord
Share your thoughts on the latest research on the Vac research forum
\ No newline at end of file diff --git a/contribute/index.html b/contribute/index.html index 789ed1fa..f3f02884 100644 --- a/contribute/index.html +++ b/contribute/index.html @@ -1,29 +1,37 @@ - + - -Contribute | Vac Research + +Contribute | Vac Research - - - - + + + + -
-

Contribute

How to Contribute​

Get in touch with us by + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

- - +or opening issues / PRs on GitHub.

+

Also, see our current job openings.

+

What to Contribute​

+

We are interested in both research and code contributions.

+

For code contributions, see our "good first issue" lists for various Vac-related code bases:

+
\ No newline at end of file diff --git a/index.html b/index.html index e363b465..6704ab05 100644 --- a/index.html +++ b/index.html @@ -1,32 +1,34 @@ - + - -About Vac | Vac Research + +About Vac | Vac Research - - - - + + + + -
-

About Vac

Vac is a principle-driven R&D team that provides comprehensive technical support to IFT startups. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

About Vac

+

Vac is a principle-driven R&D team that provides comprehensive technical support to IFT startups. Vac consists of specialized R&D service units and innovative incubator projects. -We focus on developing foundational components and unopinionated protocols that both IFT projects and the broader community can utilize.

We offer support in various ways, including direct assistance where projects can outsource the development of specific protocols or components to Vac. +We focus on developing foundational components and unopinionated protocols that both IFT projects and the broader community can utilize.

+

We offer support in various ways, including direct assistance where projects can outsource the development of specific protocols or components to Vac. This approach is particularly beneficial when the developed components have potential applications across multiple projects, -maximizing the value and efficiency of our work.

Part of our work is conducting applied research that drives the development of cutting-edge protocols, libraries, specifications, and publications. -Our dual approach combines hands-on development with in-depth research, allowing us to create robust solutions tailored to the needs of IFT projects and decentralized technologies in general.

Vac maintains an active forum and research platform, fostering collaboration and knowledge exchange within the broader community.

As custodians of the components and protocols we develop, we adhere a set of principles that guide our work, -ensuring alignment with our values and objectives in advancing free and open technologies.

- - +maximizing the value and efficiency of our work.

+

Part of our work is conducting applied research that drives the development of cutting-edge protocols, libraries, specifications, and publications. +Our dual approach combines hands-on development with in-depth research, allowing us to create robust solutions tailored to the needs of IFT projects and decentralized technologies in general.

+

Vac maintains an active forum and research platform, fostering collaboration and knowledge exchange within the broader community.

+

As custodians of the components and protocols we develop, we adhere a set of principles that guide our work, +ensuring alignment with our values and objectives in advancing free and open technologies.

\ No newline at end of file diff --git a/join-us/index.html b/join-us/index.html index eb3a3409..03b6f827 100644 --- a/join-us/index.html +++ b/join-us/index.html @@ -1,26 +1,23 @@ - + - -Join Us | Vac Research + +Join Us | Vac Research - - - - + + + + -
-
- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}
\ No newline at end of file diff --git a/media/index.html b/media/index.html index 62e7e718..614bb491 100644 --- a/media/index.html +++ b/media/index.html @@ -1,26 +1,31 @@ - + - -Media | Vac Research + +Media | Vac Research - - - - + + + + -
-
- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}
\ No newline at end of file diff --git a/principles/index.html b/principles/index.html index 202fc6e0..56c454c4 100644 --- a/principles/index.html +++ b/principles/index.html @@ -1,53 +1,74 @@ - + - -Principles | Vac Research + +Principles | Vac Research - - - - + + + + -
-

These principles have been inherited from https://our.status.im/our-principles/. Only minor stylistic changes have been made to them.

Principles


The goal of Vac is widespread adoption of the decentralised web. -Our challenge is achieving mass adoption while staying true to the principles outlined below.

I. Liberty​

We believe in the sovereignty of individuals. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

+

These principles have been inherited from https://our.status.im/our-principles/. Only minor stylistic changes have been made to them.

+

Principles

+
+

The goal of Vac is widespread adoption of the decentralised web. +Our challenge is achieving mass adoption while staying true to the principles outlined below.

+

I. Liberty​

+

We believe in the sovereignty of individuals. As a research organisation that stands for the cause of personal liberty, we aim to maximise social, political, and economic freedoms. -This includes being coercion resistant.

II. Censorship resistance​

We enable the free flow of information. +This includes being coercion resistant.

+

II. Censorship resistance​

+

We enable the free flow of information. No content is under surveillance. We abide by the cryptoeconomic design principle of censorship resistance. -Even stronger, we design agnostic infrastructures for information.

III. Security​

We don't compromise on security when building features. +Even stronger, we design agnostic infrastructures for information.

+

III. Security​

+

We don't compromise on security when building features. We use state-of-the-art technologies, -and research new security methods and technologies to make strong security guarantees.

IV. Privacy​

Privacy is the power to selectively reveal oneself to the world. +and research new security methods and technologies to make strong security guarantees.

+

IV. Privacy​

+

Privacy is the power to selectively reveal oneself to the world. For us, it's essential to protect privacy in both communications and transactions, as well as pseudo-anonymity. -Additionally, we strive to provide the right of total anonymity.

V. Transparency​

We strive for complete openness and symmetry of information within the organisation, +Additionally, we strive to provide the right of total anonymity.

+

V. Transparency​

+

We strive for complete openness and symmetry of information within the organisation, and have no border between our core contributors and our community. We are frank about our shortcomings, -especially when making short-term tradeoffs in service of our long-term goals.

VI. Openness​

The software we create is a public good. +especially when making short-term tradeoffs in service of our long-term goals.

+

VI. Openness​

+

The software we create is a public good. It is made available via a free and open-source licence, for anyone to share, modify, and benefit from. -We believe in permissionless participation.

VII. Decentralisation​

We minimise centralisation across both the software and the organisation itself. +We believe in permissionless participation.

+

VII. Decentralisation​

+

We minimise centralisation across both the software and the organisation itself. In other words, we maximise the number of physical computers composing the network, -and maximise the number of individuals who have control over the system(s) we are building.

VIII. Inclusivity​

We believe in fair and widespread access to our software, with an emphasis on ease of use. +and maximise the number of individuals who have control over the system(s) we are building.

+

VIII. Inclusivity​

+

We believe in fair and widespread access to our software, with an emphasis on ease of use. This also extends to social inclusivity, permissionless participation, interoperability, -and investing in educational efforts.

IX. Continuance​

We create software incentivised to continue to exist and improve -without the stewardship of a single entity or any of the current team members.

X. Resourcefulness​

We are relentlessly resourceful. +and investing in educational efforts.

+

IX. Continuance​

+

We create software incentivised to continue to exist and improve +without the stewardship of a single entity or any of the current team members.

+

X. Resourcefulness​

+

We are relentlessly resourceful. As we grow and have ready access to capital, it is our obligation to token holders to fight bureaucracy and inefficiencies within the organisation. This means solving problems in the most effective way possible at lower economic costs -(in terms of capital, time, and resources).

- - +(in terms of capital, time, and resources).

\ No newline at end of file diff --git a/privacy-policy/index.html b/privacy-policy/index.html index 8820864b..04ae85ef 100644 --- a/privacy-policy/index.html +++ b/privacy-policy/index.html @@ -1,26 +1,70 @@ - + - -Privacy Policy | Vac Research + +Privacy Policy | Vac Research - - - - + + + + -
-

Privacy Policy

Last updated: 9 February 2024

This Privacy Policy is intended to inform users of our approach to privacy in respect of this website ("Website"). In this regard, if you are visiting our Website, this Privacy Policy applies to you.

1) Who we are​

For the purposes of this Privacy Policy and the collection and processing of personal data as a controller, the relevant entity is the Logos Collective Association, which has its registered office in Zug and its legal domicile address at

Logos Collective Association
c/o PST Consulting GmbH
Baarerstrasse 10
6300 Zug
Switzerland

Whenever we refer to “Logos”, “we” or other similar references, we are referring to the Logos Collective Association.

2) We limit the collection and processing of personal data from your use of the Website​

We aim to limit the collection and collection and processing of personal data from users of the Website. We only collect and process certain personal data for specific purposes and where we have the legal basis to do so under applicable privacy legislation. We will not collect or process any personal data that we don’t need and where we do store any personal data, we will only store it for the least amount of time needed for the indicated purpose.

In this regard, we collect and process the following personal data from your use of the Website:

  • IP address: As part of such use of the Website we briefly process your IP address but we have no way of identifying you. We however have a legitimate interest in processing such IP addresses to ensure the technical functionality and enhance the security measures of the Website. This IP address is not stored by us over time.

3) Third party processing of personal data​

In addition to our limited and collection of personal data, third parties may collect or process personal data as a result of the Website making use of certain features or to provide certain content. To the extent you interact with such third party content or features, their respective privacy policies will apply.

4) Security measures we take in respect of the Website​

As a general approach, we take data security seriously and we have implemented a variety of security measures on the Website to maintain the safety of your personal data when you submit such information to us.

5) Exporting data outside the European Union and Switzerland​

We are obliged to protect the privacy of personal data that you may have submitted in the unlikely event that we export your personal data to places outside the European Union or Switzerland. This means that personal data will only be processed in countries or by parties that provide an adequate level of protection as deemed by Switzerland or the European Commission. Otherwise, we will use other forms of protections, such as specific forms of contractual clauses to ensure such personal data is provided the same protection as required in Switzerland or Europe. In any event, the transmission of personal data outside the European Union and Switzerland will always occur in conformity with applicable privacy legislation.

6) Your choices and rights​

As explained in this Privacy Policy, we limit our collection and processing of your personal data wherever possible. Nonetheless, you still have certain choices and rights in respect of the personal data which we do collect and process. As laid out in relevant privacy legislation, you have the right to:

  • Ask us to correct or update your personal data (where reasonably possible);

  • Ask us to remove your personal data from our systems;

  • Ask us for a copy of your personal data, which may also be transferred to another data controller at your request;

  • Withdraw your consent to process your personal data (only if consent was asked for a processing activity), which only affects processing activities that are based on your consent and doesn’t affect the validity of such processing activities before you have withdrawn your consent;

  • Object to the processing of your personal data; and

  • File a complaint with the Federal Data Protection and Information Commissioner (FDPIC), if you believe that your personal data has been processed unlawfully.

On this Website, you may come across links to third party websites. These third party sites have separate and independent privacy policies. We therefore have no responsibility or liability for the content and activities of these third party websites.

8) This Privacy Policy might change​

We may modify or replace any part of this Privacy Policy at any time and without notice. Please check the Website periodically for any changes. The new Privacy Policy will be effective immediately upon its posting on our Website.

9) Contact information​

To the extent that you have any questions about the Privacy Policy, please contact us at legal@free.technology.

This document is licensed under CC-BY-SA.

- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Privacy Policy

+

Last updated: 9 February 2024

+

This Privacy Policy is intended to inform users of our approach to privacy in respect of this website ("Website"). In this regard, if you are visiting our Website, this Privacy Policy applies to you.

+

1) Who we are​

+

For the purposes of this Privacy Policy and the collection and processing of personal data as a controller, the relevant entity is the Logos Collective Association, which has its registered office in Zug and its legal domicile address at

+
Logos Collective Association
c/o PST Consulting GmbH
Baarerstrasse 10
6300 Zug
Switzerland
+

Whenever we refer to “Logos”, “we” or other similar references, we are referring to the Logos Collective Association.

+

2) We limit the collection and processing of personal data from your use of the Website​

+

We aim to limit the collection and collection and processing of personal data from users of the Website. We only collect and process certain personal data for specific purposes and where we have the legal basis to do so under applicable privacy legislation. We will not collect or process any personal data that we don’t need and where we do store any personal data, we will only store it for the least amount of time needed for the indicated purpose.

+

In this regard, we collect and process the following personal data from your use of the Website:

+
    +
  • IP address: As part of such use of the Website we briefly process your IP address but we have no way of identifying you. We however have a legitimate interest in processing such IP addresses to ensure the technical functionality and enhance the security measures of the Website. This IP address is not stored by us over time.
    • +
+

3) Third party processing of personal data​

+

In addition to our limited and collection of personal data, third parties may collect or process personal data as a result of the Website making use of certain features or to provide certain content. To the extent you interact with such third party content or features, their respective privacy policies will apply.

+

4) Security measures we take in respect of the Website​

+

As a general approach, we take data security seriously and we have implemented a variety of security measures on the Website to maintain the safety of your personal data when you submit such information to us.

+

5) Exporting data outside the European Union and Switzerland​

+

We are obliged to protect the privacy of personal data that you may have submitted in the unlikely event that we export your personal data to places outside the European Union or Switzerland. This means that personal data will only be processed in countries or by parties that provide an adequate level of protection as deemed by Switzerland or the European Commission. Otherwise, we will use other forms of protections, such as specific forms of contractual clauses to ensure such personal data is provided the same protection as required in Switzerland or Europe. In any event, the transmission of personal data outside the European Union and Switzerland will always occur in conformity with applicable privacy legislation.

+

6) Your choices and rights​

+

As explained in this Privacy Policy, we limit our collection and processing of your personal data wherever possible. Nonetheless, you still have certain choices and rights in respect of the personal data which we do collect and process. As laid out in relevant privacy legislation, you have the right to:

+
    +
  • +

    Ask us to correct or update your personal data (where reasonably possible);

    +
    • +
  • +

    Ask us to remove your personal data from our systems;

    +
    • +
  • +

    Ask us for a copy of your personal data, which may also be transferred to another data controller at your request;

    +
    • +
  • +

    Withdraw your consent to process your personal data (only if consent was asked for a processing activity), which only affects processing activities that are based on your consent and doesn’t affect the validity of such processing activities before you have withdrawn your consent;

    +
    • +
  • +

    Object to the processing of your personal data; and

    +
    • +
  • +

    File a complaint with the Federal Data Protection and Information Commissioner (FDPIC), if you believe that your personal data has been processed unlawfully.

    +
    • +
+ +

On this Website, you may come across links to third party websites. These third party sites have separate and independent privacy policies. We therefore have no responsibility or liability for the content and activities of these third party websites.

+

8) This Privacy Policy might change​

+

We may modify or replace any part of this Privacy Policy at any time and without notice. Please check the Website periodically for any changes. The new Privacy Policy will be effective immediately upon its posting on our Website.

+

9) Contact information​

+

To the extent that you have any questions about the Privacy Policy, please contact us at legal@free.technology.

+

This document is licensed under CC-BY-SA.

\ No newline at end of file diff --git a/publications/index.html b/publications/index.html index b42e4a89..4b2a9ec9 100644 --- a/publications/index.html +++ b/publications/index.html @@ -1,26 +1,37 @@ - + - -Publications | Vac Research + +Publications | Vac Research - - - - + + + + -
-
- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}
\ No newline at end of file diff --git a/research/index.html b/research/index.html index fce0304a..6ba718ac 100644 --- a/research/index.html +++ b/research/index.html @@ -1,34 +1,33 @@ - + - -Research | Vac Research + +Research | Vac Research - - - - + + + + -
-

Vac Research

Vac Research conducts innovative exploration within the IFT. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Vac Research

+

Vac Research conducts innovative exploration within the IFT. Embedded within R&D service units, we ensure the work remains practical and relevant to IFT projects. Rather than focusing on academic publications, we aim to actively bridge theory and practice. Vac collaborates with IFT projects to implement our research findings, and identifies and initiates new incubator projects, -translating our research into real-world applications.

Vac research spanns multiple R&D service units, primarily ACZ and P2P. +translating our research into real-world applications.

+

Vac research spanns multiple R&D service units, primarily ACZ and P2P. Key focus areas of Vac Research include zero-knowledge proofs (ZKP) and their applications, libp2p gossipsub improvements, and anonymization networks. By maintaining close ties with various teams, we drive innovation and practical advancements across the IFT ecosystem. -In the realm of ZKP, Vac Deep Research has made contributions that have given rise to the incubator project Nescience.

- - +In the realm of ZKP, Vac Deep Research has made contributions that have given rise to the incubator project Nescience.

\ No newline at end of file diff --git a/rfcprocess/index.html b/rfcprocess/index.html index 1deb72e8..961f1b9b 100644 --- a/rfcprocess/index.html +++ b/rfcprocess/index.html @@ -1,29 +1,27 @@ - + - -RFC Process | Vac Research + +RFC Process | Vac Research - - - - + + + + -
-

Vac RFC Process

The Vac RFC unit serves as a vital cornerstone in the Logos collective, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Vac RFC Process

+

The Vac RFC unit serves as a vital cornerstone in the Logos collective, taking on the responsibility of shepherding and editing specifications for Logos projects and Vac incubator projects. By meticulously crafting and overseeing these specifications, the Vac RFC unit acts as a linchpin for ensuring standardised and interoperable protocols within the Logos ecosystem. -Their expertise and attention to detail contribute to a cohesive and collaborative environment, facilitating seamless integration and advancement of decentralised technologies throughout the Logos collective and beyond.

- - +Their expertise and attention to detail contribute to a cohesive and collaborative environment, facilitating seamless integration and advancement of decentralised technologies throughout the Logos collective and beyond.

\ No newline at end of file diff --git a/rlog/2024-recap/index.html b/rlog/2024-recap/index.html index 6e2b0300..76340cad 100644 --- a/rlog/2024-recap/index.html +++ b/rlog/2024-recap/index.html @@ -1,64 +1,239 @@ - + - -Vac 2024 Year in Review | Vac Research + +Vac 2024 Year in Review | Vac Research - - - - + + + + -
-

Vac 2024 Year in Review

by
9 min read

In this post, we recap Vac's achievements in 2024 and look forward to 2025.

With 2024 now behind us and a new year ahead, -Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem.

Vac comprises various subteams and service units, each with its own focus. -Below, we celebrate each unit's achievements and look forward to its 2025 plans.

Nescience​

Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.

Highlights​

This year, the Nescience state separation architecture moved from exploration to real progress, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Vac 2024 Year in Review

by
9 min read

In this post, we recap Vac's achievements in 2024 and look forward to 2025.

+ +

With 2024 now behind us and a new year ahead, +Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem.

+

Vac comprises various subteams and service units, each with its own focus. +Below, we celebrate each unit's achievements and look forward to its 2025 plans.

+

Nescience​

+

Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.

+

Highlights​

+

This year, the Nescience state separation architecture moved from exploration to real progress, taking significant steps towards building a functional and reliable system. The team focused on turning ideas into something real, testing the proposed architecture, -and understanding its strengths and weaknesses.

We also made progress on the essential parts of NSSA’s system, including:

  • Key protocol for secure key management
  • Execution types and circuits for reliable computation
  • UTXO specification to manage state transitions effectively
  • Cryptography module to ensure privacy and security

Looking forward​

In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use.

  • Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement.
  • Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness).
  • Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem.
  • Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact.
  • Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.

Token Economics (TKE)​

The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.

Highlights​

  • Formalized and implemented Codex economic incentives in the Litepaper and simulations
  • Orchestrated Status Network incentive structure and smart contract implementation
  • Started building Nomos’s economic model
  • Consulted and provided analysis of incentives for the Logos Operators ordinals project
  • Drove discussions on the economic sustainability of Waku; -helped define RLN membership and its payment mechanism

Looking forward​

In 2025, TKE will continue to support IFT portfolio projects, +and understanding its strengths and weaknesses.

+
    +
  • ZkVM exploration and benchmarks + +
    • +
  • Defined the NSSA architecture + +
    • +
  • Built the sandboxed testnet +
      +
    • Designed the first version of the node specification
      • +
    • All core components (execution types, UTXOs, cryptographic primitives) implemented and being tested
      • +
    • Testing the performance of all execution types in various scenarios
      • +
    +
    • +
+

We also made progress on the essential parts of NSSA’s system, including:

+
    +
  • Key protocol for secure key management
    • +
  • Execution types and circuits for reliable computation
    • +
  • UTXO specification to manage state transitions effectively
    • +
  • Cryptography module to ensure privacy and security
    • +
+

Looking forward​

+

In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use.

+
    +
  • Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement.
    • +
  • Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness).
    • +
  • Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem.
    • +
  • Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact.
    • +
  • Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.
    • +
+

Token Economics (TKE)​

+

The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.

+

Highlights​

+
    +
  • Formalized and implemented Codex economic incentives in the Litepaper and simulations
    • +
  • Orchestrated Status Network incentive structure and smart contract implementation
    • +
  • Started building Nomos’s economic model
    • +
  • Consulted and provided analysis of incentives for the Logos Operators ordinals project
    • +
  • Drove discussions on the economic sustainability of Waku; +helped define RLN membership and its payment mechanism
    • +
+

Looking forward​

+

In 2025, TKE will continue to support IFT portfolio projects, working toward economic sustainability while strengthening relationships within the organization. -Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.

Quality Assurance (QA)​

The QA Service Unit focuses on the development and execution of comprehensive test plans, -including implementing unit and interoperability testing.

Highlights​

  • Matured Waku interoperability testing framework with coverage for all major protocols and features
  • Began collaboration with Nomos, contributing to unit and integration testing
  • Partnered with the Status team to test message reliability under unstable network conditions

Looking forward​

  • Extend collaboration with the Waku team on go-waku bindings and message reliability testing
  • Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation
  • Work closely with Status’s QA team to enhance the functional testing framework
  • Continue work on nim-libp2p testing
  • Expand collaboration to additional projects

RFC​

The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. -The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.

Highlights​

  • Working to implement RFC culture across the IFT ecosystem
  • Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex.
  • Reworked our standards with regard to writing RFCs to a consensus-oriented specification system

Looking forward​

  • Continue to implement RFC culture across the IFT ecosystem
  • Broaden the number of RFCs produced -– particularly for IFT portfolio projects nearing public releases
  • Include new projects with the rfc-index
  • Encourage external projects requiring RFCs to establish relationships with the service unit

Applied Cryptography and ZK (ACZ)​

The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, +Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.

+

Quality Assurance (QA)​

+

The QA Service Unit focuses on the development and execution of comprehensive test plans, +including implementing unit and interoperability testing.

+

Highlights​

+
    +
  • Matured Waku interoperability testing framework with coverage for all major protocols and features
    • +
  • Began collaboration with Nomos, contributing to unit and integration testing
    • +
  • Partnered with the Status team to test message reliability under unstable network conditions
    • +
+

Looking forward​

+
    +
  • Extend collaboration with the Waku team on go-waku bindings and message reliability testing
    • +
  • Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation
    • +
  • Work closely with Status’s QA team to enhance the functional testing framework
    • +
  • Continue work on nim-libp2p testing
    • +
  • Expand collaboration to additional projects
    • +
+

RFC​

+

The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. +The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.

+

Highlights​

+
    +
  • Working to implement RFC culture across the IFT ecosystem
    • +
  • Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex.
    • +
  • Reworked our standards with regard to writing RFCs to a consensus-oriented specification system
    • +
+

Looking forward​

+
    +
  • Continue to implement RFC culture across the IFT ecosystem
    • +
  • Broaden the number of RFCs produced +– particularly for IFT portfolio projects nearing public releases
    • +
  • Include new projects with the rfc-index
    • +
  • Encourage external projects requiring RFCs to establish relationships with the service unit
    • +
+

Applied Cryptography and ZK (ACZ)​

+

The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, enhancing the security, privacy, and trustworthiness of IFT portfolio projects -and contributing to the overall integrity and resilience of the decentralized web ecosystem.

Highlights​

  • Researched a libp2p mix protocol and first proof-of-concept implementation (including ping and GossipSub over mix)
  • Researched a decentralized version of MLS (message layer security) with a first proof of concept
  • Released Zerokit v0.6.0 -and v0.5.0
  • Added gnark RLN implementation
  • Released Stealth Address Kit v0.3.1, +and contributing to the overall integrity and resilience of the decentralized web ecosystem.

    +

    Highlights​

    +

    Looking forward​

    • Ensure libp2p mix protocol is production-ready -and support with the publishing of a paper and blog posts
    • Ensure decentralized MLS is production-ready -and support with the publishing of a paper and blog posts
    • Begin explorations of additional research topics
    • Release Zerokit v0.7 and future versions

    P2P​

    The P2P Service Unit specializes in peer-to-peer technologies -and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.

    Highlights​

    Looking forward​

    • Add new features to nim-libp2p: -QUIC transport, web transport
    • Update specifications for libp2p GossipSub, -aiming to significantly improve its performance

    Distributed Systems Testing (DST)​

    The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. -By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.

    Highlights​

    +

    Looking forward​

    +
      +
    • Ensure libp2p mix protocol is production-ready +and support with the publishing of a paper and blog posts
      • +
    • Ensure decentralized MLS is production-ready +and support with the publishing of a paper and blog posts
      • +
    • Begin explorations of additional research topics
      • +
    • Release Zerokit v0.7 and future versions
      • +
    +

    P2P​

    +

    The P2P Service Unit specializes in peer-to-peer technologies +and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.

    +

    Highlights​

    + +

    Looking forward​

    +
      +
    • Add new features to nim-libp2p: +QUIC transport, web transport
      • +
    • Update specifications for libp2p GossipSub, +aiming to significantly improve its performance
      • +
    +

    Distributed Systems Testing (DST)​

    +

    The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. +By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.

    +

    Highlights​

    +
      +
    • DST compute resources transitioned from a hosted environment to a dedicated Vac Lab, enabling better customization of resources and adding significantly more compute power -– enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control.
    • Maintained monthly regression simulations for both Waku and Nim-libp2p, -helping us to detect several issues and ensure that future versions do not introduce new ones.
    • Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.

    Looking forward​

    • More testing and simulations for Codex and Nomos
    • Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard

    Nim​

    Several IFT portfolio projects use the Nim ecosystem for its efficiency. -The Nim Service Unit is responsible for the development and maintenance of Nim tooling.

    Highlights​

    • Released Nim-libp2p (v1.7.1, +– enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control.
      • +
    • Maintained monthly regression simulations for both Waku and Nim-libp2p, +helping us to detect several issues and ensure that future versions do not introduce new ones.
      • +
    • Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.
      • +
    +

    Looking forward​

    +
      +
    • More testing and simulations for Codex and Nomos
      • +
    • Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard
      • +
    +

    Nim​

    +

    Several IFT portfolio projects use the Nim ecosystem for its efficiency. +The Nim Service Unit is responsible for the development and maintenance of Nim tooling.

    +

    Highlights​

    +
      +
    • Released Nim-libp2p (v1.7.1, v1.7.0, v1.6.0, v1.5.0, v1.4.0, v1.3.0, -v1.2.0)
    • Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution
    • Nim VSCode Extension
    • Stabilized Nim Language Server

    Smart Contracts (SC)​

    Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.

    Highlights​

    • Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network
    • Wrote specifications for Codex's architectural components and Status's staking contracts
    • Delivered several learn-up sessions on a variety of topics for IFT contributors, including:
      • Stealth addresses
      • Tokenized vaults
      • Rental NFTs
      • Merkle trees
      • Account abstraction
      • EVM deep dive

    Looking forward​

    • Deploy the SNT staking protocol on the Status Network testnet
    • Encourage community security audits via contests
    • Provide smart contract consultation services for IFT portfolio products
    • Engage in more learn-up sessions to promote org-wide knowledge sharing.

    Heading into 2025​

    This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. -The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit.

    As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. +v1.2.0)

    • +
  • Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution
    • +
  • Nim VSCode Extension
    • +
  • Stabilized Nim Language Server
    • +
+

Smart Contracts (SC)​

+

Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.

+

Highlights​

+
    +
  • Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network
    • +
  • Wrote specifications for Codex's architectural components and Status's staking contracts
    • +
  • Delivered several learn-up sessions on a variety of topics for IFT contributors, including: +
      +
    • Stealth addresses
      • +
    • Tokenized vaults
      • +
    • Rental NFTs
      • +
    • Merkle trees
      • +
    • Account abstraction
      • +
    • EVM deep dive
      • +
    +
    • +
+

Looking forward​

+
    +
  • Deploy the SNT staking protocol on the Status Network testnet
    • +
  • Encourage community security audits via contests
    • +
  • Provide smart contract consultation services for IFT portfolio products
    • +
  • Engage in more learn-up sessions to promote org-wide knowledge sharing.
    • +
+

Heading into 2025​

+

This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. +The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit.

+

As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. Our goal is to serve as an internal conduit of expertise within the organization, supported by a strong RFC culture, maintaining a repository of internal knowledge creation, and identifying and facilitating IFT project synergies. -Such an approach should lead to greater efficiencies across the organization.

We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. +Such an approach should lead to greater efficiencies across the organization.

+

We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. In the final quarter of 2024, Vac stepped up its collaboration with the libp2p community and made a concerted effort to engage the community on the Vac forum. -In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems.

We look forward to continuing our journey with you!

Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.

- - +In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems.

+

We look forward to continuing our journey with you!

+

Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.

\ No newline at end of file diff --git a/rlog/GossipSub Improvements/index.html b/rlog/GossipSub Improvements/index.html index 5f9796aa..f8565f79 100644 --- a/rlog/GossipSub Improvements/index.html +++ b/rlog/GossipSub Improvements/index.html @@ -1,109 +1,210 @@ - + - -GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks | Vac Research + +GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks | Vac Research - - - - + + + + -
-

GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

by
14 min read

GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

Motivitation​

We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, -as in the case of Ethereum Improvement Proposal EIP-4844. -This work led to a comprehensive study of unstructured P2P networks. -The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.

Introduction​

Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, -query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh.

Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. -At the same time, the underlying IP layer is unreliable.

Therefore, tree-like overlays are not best suited for reliable information propagation. -Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches.

Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. -Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links.

An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DD random peers. -Publishing through a D-regular overlay triggers approximately N×DN \times D transmissions. -Reducing DD reduces the redundant transmissions but compromises reachability and latency. -Sharing metadata through a K-regular overlay (where K>DK > D) allows nodes to pull missing messages.

GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. -Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency.

Striking the right balance between parameters like D,KD, K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem.

At the same time, many other aspects can significantly contribute to the network's performance and scalability. -One option is to realize peers' suitability and continuously changing capabilities while forming overlays.

For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. -Reshuffling of peering links according to the changing network conditions can lead to superior performance.

Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. -Grouping multiple super nodes to form virtual node(s) can solve this problem.

Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks.

Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. -Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn.

This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. -We look into different constraints imposed by application-specific needs (performance goals) and investigate various choices that can augment the network's performance. -We explore overlay designs/freshness, peer selection approaches, message-relaying mechanisms, and resilience against adversaries/peer churn. -We consider GossipSub a baseline protocol to explore various possibilities and decisively commit to the ones demonstrating superior performance. -We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.

GOAL1: Low Latency Operation​

Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. -A message delivered after the imposed time bounds is considered as dropped. -An early message delivery in applications like live streaming can further enhance the viewing quality.

The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. -Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance.

Interestingly, shortest-path message delivery trees have many limitations:

1) Changing network dynamics requires a quicker and continuous readjustment of the multicast tree. -2) The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion. -3) Node failure can result in partitions, making many segments unreachable. -4) Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology.

Solutions involve creating multiple random trees to add redundancy [2]. -Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push).

Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. -New nodes are added from the overlay on node failure, but it requires non-faulty node selection.

GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages).

The mesh degree DLow≤D≤DHighD_{Low} \leq D \leq D_{High} is crucial in deciding message dissemination latency. -A smaller value for DD results in higher latency due to increased rounds, whereas a higher DD reduces latency on the cost of increased bandwidth. -At the same time, keeping DD independent of the growing network size (NN) may increase network-wide message dissemination latency. -Adjusting DD with NN maintains similar latency on the cost of increased workload for peers. -Authors in [3] suggest only a logarithmic increase in DD to maintain a manageable workload for peers. -In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \ln(N) + C for an optimal operation, -where CC is a small constant.

Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].

GOAL2: Considering Heterogeneity In Overlay Design​

Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. -Most of the research on P2P networks provides simulation results assuming nodes with similar capabilities. -The aspect of dissimilar capabilities and resource-constrained nodes is less explored.

It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg} does not exceed ln⁡(N)+C\ln(N) + C. -Enforcing all the nodes to have approximately ln⁡(N)+C\ln(N) + C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. -At the same time, connecting high-bandwidth nodes through a low-bandwidth node undermines the network's performance. -Ideally, the workload on any node should not exceed its available resources. -A better solution involves a two-phased operation:

  1. Every node computes its available bandwidth and selects a node degree DD proportional to its available bandwidth [4]. -Different bandwidth estimation approaches are suggested in literature [5,6]. -Simple bandwidth estimation approaches like variable packet size probing [6] yield similar results with less complexity. -It is also worth mentioning that many nodes may want to allocate only a capped share of their bandwidth to the network. -Lowering DD according to the available bandwidth can still prove helpful. -Additionally, bandwidth preservation at the transport layer through approaches like µTP can be useful. -To further conform to the suggested mesh-degree average DavgD_{avg}, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}.

  2. From the available local view, every node tries connecting peers with the lowest latency until DD connections are made. -We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning.

The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \leq D \leq D_{High} connections.

GOAL3: Bandwidth Optimization​

Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. -This not only adds to the network-wide message dissemination latency but a significant share of the network's bandwidth is wasted on (usually) unnecessary transmissions. -It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures.

Many efforts have been made to minimize the impact of redundant transmissions. -These solutions include multicast delivery trees, metadata sharing to enable pull-based operation, in-network information caching, etc. [7,8]. -GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages).

A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. -As a result, redundant message transmissions can serve as a critical advantage of the GossipSub protocol. -Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3. Assuming an XOR coding function, -we know two trivial properties: x1⊕x2⊕x2=x1x1 \oplus x2 \oplus x2 = x1 and ∣x1∣=∣x1⊕x2⊕x2∣\vert x1 \vert = \vert x1 \oplus x2 \oplus x2 \vert.

This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. -The receiver can reconstruct the original message from encoded segments. -As a result, fewer transmissions are sufficient for sending more messages to the network.

However, sharing linear combinations of messages requires organizing messages in intervals, -and devising techniques to identify all messages belonging to each interval. -In addition, combining messages from different publishers requires more complex arrangements, -involving embedding publisher/message IDs, delayed forwarding (to accommodate more messages), and mechanisms to ensure the decoding of messages at all peers. -Careful application-specific need analysis can help decide the benefits against the added complexity.

GOAL4: Handling Large Messages​

Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. -This introduces two challenges:

1) Redundant large message transmissions result in severe network congestion. -2) Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages.

The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. -Most of the work done for handling large messages involves curtailing redundant transmissions using multicast delivery trees, -reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc.

Approaches like message aggregation also prove helpful in minimizing bandwidth wastage.

Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions:

  1. Using IDontWant message proposal [13] and staggered sending.

    IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. -Staggered sending enables relaying the message to a short subset of peers in each round. -We argue that simultaneously relaying a message to all peers hampers the effectiveness of the IDontWant message. -Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages.

  2. Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. -We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.

GOAL5: Scalability​

P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. -In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \times D bandwidth, -where RR is average data arrival rate. -It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\lceil \log_D (N) \rceil hops. -Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size.

Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. -Most efforts for bringing scalability to the P2P systems have focused on curtailing redundant transmissions and flat overlay adjustments. -Hierarchical overlay designs, on the other hand, are less explored.

Placing a logical structure in unstructured P2P systems can help scale P2P networks.

One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. -An abstract operation of such overlay design is provided below:

  1. Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. -For this purpose, every node tries connecting peers with the lowest latency until DD connections are made or the cluster limit is reached.

  2. A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. -These super nodes form a fully connected mesh and jointly act as a virtual node, -mitigating the problem of peer churn among super nodes.

  3. Virtual nodes form a fully connected mesh to construct a hierarchical overlay. -Each virtual node is essentially a collection of super nodes; -a link to any of the constituent super nodes represents a link to the virtual node.

  4. One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.

Summary​

Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. -On the other hand, a hierarchical overlay can bring scalability at the cost of increased complexity. -Regardless of the overlay design, a continuous readjustment to appropriate peering links is essential for superior performance. -At the same time, bandwidth preservation (through message aggregation, caching at strategic locations, metadata sharing, pull-based operation, etc.) can help minimize latency. -However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.

References

  • [1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf
  • [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905
  • [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf
  • [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document
  • [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454
  • [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658
  • [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473
  • [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12
  • [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576
  • [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676
  • [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446
  • [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844
  • [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413
  • [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5
  • [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16
  • [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458
- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

by
14 min read

GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

+ +

Motivitation​

+

We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, +as in the case of Ethereum Improvement Proposal EIP-4844. +This work led to a comprehensive study of unstructured P2P networks. +The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.

+

Introduction​

+

Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, +query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh.

+

Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. +At the same time, the underlying IP layer is unreliable.

+

Therefore, tree-like overlays are not best suited for reliable information propagation. +Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches.

+

Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. +Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links.

+

An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DD random peers. +Publishing through a D-regular overlay triggers approximately N×DN \times D transmissions. +Reducing DD reduces the redundant transmissions but compromises reachability and latency. +Sharing metadata through a K-regular overlay (where K>DK > D) allows nodes to pull missing messages.

+

GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. +Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency.

+

Striking the right balance between parameters like D,KD, K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem.

+

At the same time, many other aspects can significantly contribute to the network's performance and scalability. +One option is to realize peers' suitability and continuously changing capabilities while forming overlays.

+

For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. +Reshuffling of peering links according to the changing network conditions can lead to superior performance.

+

Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. +Grouping multiple super nodes to form virtual node(s) can solve this problem.

+

Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks.

+

Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. +Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn.

+

This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. +We look into different constraints imposed by application-specific needs (performance goals) and investigate various choices that can augment the network's performance. +We explore overlay designs/freshness, peer selection approaches, message-relaying mechanisms, and resilience against adversaries/peer churn. +We consider GossipSub a baseline protocol to explore various possibilities and decisively commit to the ones demonstrating superior performance. +We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.

+

GOAL1: Low Latency Operation​

+

Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. +A message delivered after the imposed time bounds is considered as dropped. +An early message delivery in applications like live streaming can further enhance the viewing quality.

+

The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. +Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance.

+

Interestingly, shortest-path message delivery trees have many limitations:

+
    +
  1. Changing network dynamics requires a quicker and continuous readjustment of the multicast tree.
    2. +
  2. The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion.
    3. +
  3. Node failure can result in partitions, making many segments unreachable.
    4. +
  4. Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology.
    5. +
+

Solutions involve creating multiple random trees to add redundancy [2]. +Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push).

+

Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. +New nodes are added from the overlay on node failure, but it requires non-faulty node selection.

+

GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages).

+

The mesh degree DLow≤D≤DHighD_{Low} \leq D \leq D_{High} is crucial in deciding message dissemination latency. +A smaller value for DD results in higher latency due to increased rounds, whereas a higher DD reduces latency on the cost of increased bandwidth. +At the same time, keeping DD independent of the growing network size (NN) may increase network-wide message dissemination latency. +Adjusting DD with NN maintains similar latency on the cost of increased workload for peers. +Authors in [3] suggest only a logarithmic increase in DD to maintain a manageable workload for peers. +In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \ln(N) + C for an optimal operation, +where CC is a small constant.

+

Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].

+

GOAL2: Considering Heterogeneity In Overlay Design​

+

Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. +Most of the research on P2P networks provides simulation results assuming nodes with similar capabilities. +The aspect of dissimilar capabilities and resource-constrained nodes is less explored.

+

It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg} does not exceed ln⁡(N)+C\ln(N) + C. +Enforcing all the nodes to have approximately ln⁡(N)+C\ln(N) + C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. +At the same time, connecting high-bandwidth nodes through a low-bandwidth node undermines the network's performance. +Ideally, the workload on any node should not exceed its available resources. +A better solution involves a two-phased operation:

+
    +
  1. +

    Every node computes its available bandwidth and selects a node degree DD proportional to its available bandwidth [4]. +Different bandwidth estimation approaches are suggested in literature [5,6]. +Simple bandwidth estimation approaches like variable packet size probing [6] yield similar results with less complexity. +It is also worth mentioning that many nodes may want to allocate only a capped share of their bandwidth to the network. +Lowering DD according to the available bandwidth can still prove helpful. +Additionally, bandwidth preservation at the transport layer through approaches like µTP can be useful. +To further conform to the suggested mesh-degree average DavgD_{avg}, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}.

    +
    2. +
  2. +

    From the available local view, every node tries connecting peers with the lowest latency until DD connections are made. +We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning.

    +
    3. +
+

The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \leq D \leq D_{High} connections.

+

GOAL3: Bandwidth Optimization​

+

Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. +This not only adds to the network-wide message dissemination latency but a significant share of the network's bandwidth is wasted on (usually) unnecessary transmissions. +It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures.

+

Many efforts have been made to minimize the impact of redundant transmissions. +These solutions include multicast delivery trees, metadata sharing to enable pull-based operation, in-network information caching, etc. [7,8]. +GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages).

+

A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. +As a result, redundant message transmissions can serve as a critical advantage of the GossipSub protocol. +Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3. Assuming an XOR coding function, +we know two trivial properties: x1⊕x2⊕x2=x1x1 \oplus x2 \oplus x2 = x1 and ∣x1∣=∣x1⊕x2⊕x2∣\vert x1 \vert = \vert x1 \oplus x2 \oplus x2 \vert.

+

This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. +The receiver can reconstruct the original message from encoded segments. +As a result, fewer transmissions are sufficient for sending more messages to the network.

+

However, sharing linear combinations of messages requires organizing messages in intervals, +and devising techniques to identify all messages belonging to each interval. +In addition, combining messages from different publishers requires more complex arrangements, +involving embedding publisher/message IDs, delayed forwarding (to accommodate more messages), and mechanisms to ensure the decoding of messages at all peers. +Careful application-specific need analysis can help decide the benefits against the added complexity.

+

GOAL4: Handling Large Messages​

+

Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. +This introduces two challenges:

+
    +
  1. Redundant large message transmissions result in severe network congestion.
    2. +
  2. Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages.
    3. +
+

The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. +Most of the work done for handling large messages involves curtailing redundant transmissions using multicast delivery trees, +reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc.

+

Approaches like message aggregation also prove helpful in minimizing bandwidth wastage.

+

Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions:

+
    +
  1. +

    Using IDontWant message proposal [13] and staggered sending.

    +

    IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. +Staggered sending enables relaying the message to a short subset of peers in each round. +We argue that simultaneously relaying a message to all peers hampers the effectiveness of the IDontWant message. +Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages.

    +
    2. +
  2. +

    Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. +We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.

    +
    3. +
+

GOAL5: Scalability​

+

P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. +In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \times D bandwidth, +where RR is average data arrival rate. +It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\lceil \log_D (N) \rceil hops. +Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size.

+

Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. +Most efforts for bringing scalability to the P2P systems have focused on curtailing redundant transmissions and flat overlay adjustments. +Hierarchical overlay designs, on the other hand, are less explored.

+

Placing a logical structure in unstructured P2P systems can help scale P2P networks.

+

One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. +An abstract operation of such overlay design is provided below:

+
    +
  1. +

    Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. +For this purpose, every node tries connecting peers with the lowest latency until DD connections are made or the cluster limit is reached.

    +
    2. +
  2. +

    A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. +These super nodes form a fully connected mesh and jointly act as a virtual node, +mitigating the problem of peer churn among super nodes.

    +
    3. +
  3. +

    Virtual nodes form a fully connected mesh to construct a hierarchical overlay. +Each virtual node is essentially a collection of super nodes; +a link to any of the constituent super nodes represents a link to the virtual node.

    +
    4. +
  4. +

    One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.

    +
    5. +
+

Summary​

+

Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. +On the other hand, a hierarchical overlay can bring scalability at the cost of increased complexity. +Regardless of the overlay design, a continuous readjustment to appropriate peering links is essential for superior performance. +At the same time, bandwidth preservation (through message aggregation, caching at strategic locations, metadata sharing, pull-based operation, etc.) can help minimize latency. +However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.

+

References

+
    +
  • [1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf
    • +
  • [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905
    • +
  • [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf
    • +
  • [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document
    • +
  • [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454
    • +
  • [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658
    • +
  • [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473
    • +
  • [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12
    • +
  • [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576
    • +
  • [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676
    • +
  • [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446
    • +
  • [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844
    • +
  • [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413
    • +
  • [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5
    • +
  • [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16
    • +
  • [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458
    • +
\ No newline at end of file diff --git a/rlog/Nescience-A-zkVM-leveraging-hiding-properties/index.html b/rlog/Nescience-A-zkVM-leveraging-hiding-properties/index.html index 400e8b8f..f5068916 100644 --- a/rlog/Nescience-A-zkVM-leveraging-hiding-properties/index.html +++ b/rlog/Nescience-A-zkVM-leveraging-hiding-properties/index.html @@ -1,166 +1,731 @@ - + - -Nescience - A zkVM leveraging hiding properties | Vac Research + +Nescience - A zkVM leveraging hiding properties | Vac Research - - - - + + + + -
-

Nescience - A zkVM leveraging hiding properties

by
32 min read

Nescience, a privacy-first blockchain zkVM.

Introduction​

Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Nescience - A zkVM leveraging hiding properties

by
32 min read

Nescience, a privacy-first blockchain zkVM.

+ +

Introduction​

+

Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. The goals include creating a state separation architecture for public/private computation, designing a versatile virtual machine based on mainstream instruction sets, creating proofs for private state updates, implementing a kernel-based architecture for correct execution of private functions, -and implementing core DeFi protocols such as AMMs and staking from a privacy perspective.

It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. +and implementing core DeFi protocols such as AMMs and staking from a privacy perspective.

+

It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. To achieve this goal, Nescience will implement a versatile virtual machine that can be used to implement existing blockchain applications, -while also enabling the development of privacy-centric protocols such as private staking and private DEXs.

To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, -while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover.

It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. +while also enabling the development of privacy-centric protocols such as private staking and private DEXs.

+

To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, +while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover.

+

It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. Finally, Nescience intends to implement permissive licensing, which means that the source code will be open-source, -and developers will be able to use and modify the code without any restriction.

Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, +and developers will be able to use and modify the code without any restriction.

+

Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, potential solutions, and alternatives that lay ahead. Each step is a testament to our commitment to thoroughness; we systematically test various possibilities and decisively commit to the one that demonstrates paramount performance and utility. For instance, as we progress towards achieving Goal 2, we are undertaking a rigorous benchmarking of the Nova proof system against its contemporaries. Should Nova showcase superior performance metrics, we stand ready to integrate it as our proof system of choice. Through such meticulous approaches, -we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.

Goal 1: Create a State Separation Architecture​

The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, -employing an account-based framework for the public state and a UTXO-based structure for the private state.

The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, +we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.

+

Goal 1: Create a State Separation Architecture​

+

The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, +employing an account-based framework for the public state and a UTXO-based structure for the private state.

+

The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, while the account-based paradigm assigns balances to accounts that transactions can modify. Although the UTXO model bolsters privacy by concealing comprehensive balances, the pursuit of a dual architecture mandates a meticulous synchronization of these state models, -ensuring that private transactions remain inconspicuous in the wider public network state.

This task is further complicated by the divergent transaction processing methods intrinsic to each model, +ensuring that private transactions remain inconspicuous in the wider public network state.

+

This task is further complicated by the divergent transaction processing methods intrinsic to each model, necessitating a thoughtful and innovative approach to harmonize their functionality. To seamlessly bring together the dual architecture, harmonizing the account-based model for public state with the UTXO-based model for private state, -a comprehensive strategy is essential.

The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. -It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model.

Here's a breakdown and a potential strategy for harmonizing these models:

Rationale Behind the Dual Architecture: ​

  • Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, -and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications.

  • UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. -By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.

Harmonizing the Two Systems: ​

  1. Translation Layer

    • Role: Interface between UTXO and account-based states.

    • UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. +a comprehensive strategy is essential.

      +

      The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. +It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model.

      +

      Here's a breakdown and a potential strategy for harmonizing these models:

      +

      Rationale Behind the Dual Architecture: ​

      +
        +
      • +

        Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, +and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications.

        +
        • +
      • +

        UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. +By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.

        +
        • +
      +

      Harmonizing the Two Systems: ​

      +
        +
      1. +

        Translation Layer

        +
          +
        • +

          Role: Interface between UTXO and account-based states.

          +
          • +
        • +

          UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. This could involve creating a temporary 'pseudo-account' that mirrors the -UTXO's attributes.

        • Account-to-UTXO Adapter: When an account wishes to make a private transaction, -it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction.

      2. Unified Identity Management

        • Role: Maintain a unified identity (or address) system that works across both state models, -allowing users to easily manage their public and private states without requiring separate identities.

        • Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. -This ensures privacy while keeping management centralized for the user.

      1. State Commitments

        • Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions.

        • Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. -Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies.

        • Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy.

      1. Batch Processing & Anonymity Sets

        • Role: Group several UTXO-based private transactions into a single public account-based transaction. -This can provide a level of obfuscation and can make synchronization between the two models more efficient.

        • CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy.

        • Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, -providing zk-SNARKs-based private transactions.

      2. Event Hooks & Smart Contracts

        • Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. -For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model.

        • Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, -a smart contract might release funds (account-based) once a specific UTXO is spent.

        • Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, -allowing for private logic execution.

      Challenges and Solutions ​

      1. Synchronization Overhead

        • Challenge: Combining two distinct transaction models creates an inherent synchronization challenge.

        • State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. -Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required.

        • Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. -This structure helps reduce the immediate load on the primary system.

        • Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. -This can ensure consistency without constant synchronization.

      2. Double Spending

        • Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks.

        • Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements.

        • Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. -If discrepancies arise, the transaction can be halted.

        • Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending.

      3. Complexity in User Experience

        • Challenge: The dual model, while powerful, is inherently complex.

        • Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, -allowing users to make transactions without needing to understand the nuances of the dual model.

        • Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, -especially emphasizing when and why they might choose one transaction type over the other.

        • Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. -This real-time feedback can be invaluable for iterative design improvements.

      4. Security

        • Challenge: Merging two systems can introduce unforeseen vulnerabilities.

        • Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, -especially those that might exploit the interaction between the two systems.

        • Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. -Each layer can act as a fail-safe if a potential threat bypasses another.

        • Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. -If any suspicious activity is detected, they can take corrective measures or raise alerts.

      5. Gas & Fee Management:

        • Challenge: A dual model can lead to convoluted fee structures.

        • Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. -This can ensure fairness and prevent network abuse.

        • Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. -This helps users understand potential costs upfront.

        • Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, -simplifying the gas acquisition process.

      By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, -combining the strengths of both UTXO and account-based models without their standalone limitations.

      AspectDetails
      Harmony- Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures.
      Challenges- Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs.
      Proposed Solutions- Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.

      Goal 2: Virtual Machine Creation​

      The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. -Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications.

      This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. -Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures.

      WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, -and boasts seamless integration across diverse blockchain platforms.

      Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, +UTXO's attributes.

      +
      • +
    • +

      Account-to-UTXO Adapter: When an account wishes to make a private transaction, +it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction.

      +
      • +
    +
    2. +
  2. +

    Unified Identity Management

    +
      +
    • +

      Role: Maintain a unified identity (or address) system that works across both state models, +allowing users to easily manage their public and private states without requiring separate identities.

      +
      • +
    • +

      Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. +This ensures privacy while keeping management centralized for the user.

      +
      • +
    +
    3. +
  3. +

    State Commitments

    +
      +
    • +

      Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions.

      +
      • +
    • +

      Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. +Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies.

      +
      • +
    • +

      Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy.

      +
      • +
    +
    4. +
  4. +

    Batch Processing & Anonymity Sets

    +
      +
    • +

      Role: Group several UTXO-based private transactions into a single public account-based transaction. +This can provide a level of obfuscation and can make synchronization between the two models more efficient.

      +
      • +
    • +

      CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy.

      +
      • +
    • +

      Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, +providing zk-SNARKs-based private transactions.

      +
      • +
    +
    5. +
  5. +

    Event Hooks & Smart Contracts

    +
      +
    • +

      Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. +For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model.

      +
      • +
    • +

      Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, +a smart contract might release funds (account-based) once a specific UTXO is spent.

      +
      • +
    • +

      Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, +allowing for private logic execution.

      +
      • +
    +
    6. +
+

Challenges and Solutions ​

+
    +
  1. +

    Synchronization Overhead

    +
      +
    • +

      Challenge: Combining two distinct transaction models creates an inherent synchronization challenge.

      +
      • +
    • +

      State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. +Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required.

      +
      • +
    • +

      Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. +This structure helps reduce the immediate load on the primary system.

      +
      • +
    • +

      Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. +This can ensure consistency without constant synchronization.

      +
      • +
    +
    2. +
  2. +

    Double Spending

    +
      +
    • +

      Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks.

      +
      • +
    • +

      Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements.

      +
      • +
    • +

      Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. +If discrepancies arise, the transaction can be halted.

      +
      • +
    • +

      Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending.

      +
      • +
    +
    3. +
  3. +

    Complexity in User Experience

    +
      +
    • +

      Challenge: The dual model, while powerful, is inherently complex.

      +
      • +
    • +

      Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, +allowing users to make transactions without needing to understand the nuances of the dual model.

      +
      • +
    • +

      Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, +especially emphasizing when and why they might choose one transaction type over the other.

      +
      • +
    • +

      Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. +This real-time feedback can be invaluable for iterative design improvements.

      +
      • +
    +
    4. +
  4. +

    Security

    +
      +
    • +

      Challenge: Merging two systems can introduce unforeseen vulnerabilities.

      +
      • +
    • +

      Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, +especially those that might exploit the interaction between the two systems.

      +
      • +
    • +

      Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. +Each layer can act as a fail-safe if a potential threat bypasses another.

      +
      • +
    • +

      Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. +If any suspicious activity is detected, they can take corrective measures or raise alerts.

      +
      • +
    +
    5. +
  5. +

    Gas & Fee Management:

    +
      +
    • +

      Challenge: A dual model can lead to convoluted fee structures.

      +
      • +
    • +

      Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. +This can ensure fairness and prevent network abuse.

      +
      • +
    • +

      Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. +This helps users understand potential costs upfront.

      +
      • +
    • +

      Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, +simplifying the gas acquisition process.

      +
      • +
    +
    6. +
+

By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, +combining the strengths of both UTXO and account-based models without their standalone limitations.

+
AspectDetails
Harmony- Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures.
Challenges- Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs.
Proposed Solutions- Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.
+

Goal 2: Virtual Machine Creation​

+

The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. +Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications.

+

This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. +Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures.

+

WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, +and boasts seamless integration across diverse blockchain platforms.

+

Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, elevating the levels of security and privacy. However, it is crucial to acknowledge that employing WASM or RISC-V might present challenges, -given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs).

Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, +given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs).

+

Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, while potentially addressing optimization concerns, necessitate extensive research and testing due to their relatively nascent status within the field. -This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance.

The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. +This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance.

+

The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. The combination of two renowned instruction sets, WASM and RISC-V, in tandem with ZK, is an innovation that could redefine privacy standards in blockchain. -Let's dissect the challenges and possibilities inherent in this goal:

  1. Established Mainstream Instruction Sets - WASM and RISC-V

    • Strengths:

      • WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. -Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape.

      • RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. +Let's dissect the challenges and possibilities inherent in this goal:

        +
          +
        1. +

          Established Mainstream Instruction Sets - WASM and RISC-V

          +
            +
          • +

            Strengths:

            +
              +
            • +

              WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. +Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape.

              +
              • +
            • +

              RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. Its adaptability to both UTXO and account-based structures coupled with its compatibility with trusted execution environments like SGX and TEE augments its appeal, -especially in domains that prioritize security and privacy.

          • Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, -they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts.

        1. Pioneering a New, Specialized Instruction Set

          • Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, -potentially offering unmatched performance and optimizations tailored to the project's requirements.

          • Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. -It would also need to garner community trust and support over time.

        1. Contemporary Proof Systems - (Super)Nova, STARKs, Sangria

          • Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, -they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization.

          • Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. +especially in domains that prioritize security and privacy.

            +
            • +
          +
          2. +
        2. +

          Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, +they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts.

          +
          3. +
      +
      • +
    • +

      Pioneering a New, Specialized Instruction Set

      +
        +
      • +

        Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, +potentially offering unmatched performance and optimizations tailored to the project's requirements.

        +
        • +
      • +

        Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. +It would also need to garner community trust and support over time.

        +
        • +
      +
      • +
    • +

      Contemporary Proof Systems - (Super)Nova, STARKs, Sangria

      +
        +
      • +

        Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, +they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization.

        +
        • +
      • +

        Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. The unknowns associated with these systems could introduce unforeseen vulnerabilities or complexities. -While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing.

Mainstream (WASM, RISC-V)ZK-optimized (New Instruction Set)
Existing ToolingYESNO
Blockchain-focusedNOYES
PerformantDEPENDSYES

Optimization Concerns for WASM and RISC-V: ​

  • Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. -Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed.

  • Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. -Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently.

  • Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. -Fine-tuned memory management mechanisms are essential to prevent bottlenecks.

Emerging ZKP Optimized Systems Considerations: ​

  • Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. -The trade-offs between proof size, computational efficiency, and security need to be balanced.

  • Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. -A universal system can be more versatile for diverse applications on the blockchain.

  • Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. -Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.

Strategies for Integration: ​

  • Iterative Development: Given the complexities, an iterative development approach can be beneficial. -Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities.

  • Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization.

  • External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.

Goal 3: Proofs Creation and Verification​

The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. +While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing.

+ + + + +
Mainstream (WASM, RISC-V)ZK-optimized (New Instruction Set)
Existing ToolingYESNO
Blockchain-focusedNOYES
PerformantDEPENDSYES
+

Optimization Concerns for WASM and RISC-V: ​

+
    +
  • +

    Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. +Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed.

    +
    • +
  • +

    Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. +Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently.

    +
    • +
  • +

    Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. +Fine-tuned memory management mechanisms are essential to prevent bottlenecks.

    +
    • +
+

Emerging ZKP Optimized Systems Considerations: ​

+
    +
  • +

    Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. +The trade-offs between proof size, computational efficiency, and security need to be balanced.

    +
    • +
  • +

    Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. +A universal system can be more versatile for diverse applications on the blockchain.

    +
    • +
  • +

    Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. +Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.

    +
    • +
+

Strategies for Integration: ​

+
    +
  • +

    Iterative Development: Given the complexities, an iterative development approach can be beneficial. +Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities.

    +
    • +
  • +

    Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization.

    +
    • +
  • +

    External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.

    +
    • +
+

Goal 3: Proofs Creation and Verification​

+

The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. Concurrently, the responsibility of verifying these proofs and executing public functions within the virtual machine can be effectively delegated to an external prover, a role that is incentivized to operate with utmost honesty and integrity. This intricate balance seeks to safeguard against information leakage, -preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework.

To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. +preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework.

+

To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. This two-pronged approach serves as a compelling deterrent against dishonest behavior and fosters an environment of accountability. In addition to incentivization, a crucial consideration is the economic aspect of verification and execution. -The verification process has been intentionally designed to be more cost-effective than execution.

This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. +The verification process has been intentionally designed to be more cost-effective than execution.

+

This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. By maintaining a cost balance that favors verification, we bolster the system’s resilience against fraudulent activities while ensuring its efficiency. In sum, our multifaceted approach endeavors to strike an intricate equilibrium between user-initiated proof creation, external verification, and incentivization. -This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability.

As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, +This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability.

+

As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, and the overall integrity of the system is fortified against potential adversarial actions. To achieve the goals of user-initiated proof creation, -external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed:

  1. User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies.

    • Challenges:

      • Maintaining the quality and integrity of the proofs generated by users.

      • Ensuring that users have the tools and knowledge to produce valid proofs.

    • Solutions:

      • Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process.

      • Implement checks at the verifier's end to ensure the quality of proofs.

  1. External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine.

    • Challenges:

      • Ensuring that the external prover acts honestly.

      • Avoiding centralized points of failure.

    • Solutions:

      • Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work.

      • Use reputation systems to rank provers based on their past performances, creating a trust hierarchy.

  2. Incentivization Framework: A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task.

    • Challenges:

      • Determining the right balance of rewards and penalties.

      • Ensuring that the system cannot be gamed for undue advantage.

    • Solutions1:

      • Implement a dynamic reward system that adjusts based on network metrics and provers' performance.

      • Use a staking mechanism where provers need to lock up a certain amount of assets. -Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets.

  3. Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks.

    • Challenges:

      • Setting the right cost metrics for both verification and execution.

      • Ensuring that genuine users aren’t priced out of the system.

    • Solutions:

      • Use a dynamic pricing model, adjusting costs in real-time based on network demand.

      • Implement gas-like mechanisms to differentiate operation costs and ensure fairness.

  4. Maintaining Trustworthiness: Create a system that's transparent, holds all actors accountable, and is economically sound.

    • Challenges:

      • Keeping the balance where users feel their privacy is intact, while provers feel incentivized.

      • Ensuring the system remains resilient against adversarial attacks.

    • Solutions:

      • Implement layered checks and balances.

      • Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO).

Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, +external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed:

+
    +
  1. +

    User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies.

    +
      +
    • +

      Challenges:

      +
        +
      • +

        Maintaining the quality and integrity of the proofs generated by users.

        +
        • +
      • +

        Ensuring that users have the tools and knowledge to produce valid proofs.

        +
        • +
      +
      • +
    • +

      Solutions:

      +
        +
      • +

        Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process.

        +
        • +
      • +

        Implement checks at the verifier's end to ensure the quality of proofs.

        +
        • +
      +
      • +
    +
    2. +
  2. +

    External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine.

    +
      +
    • +

      Challenges:

      +
        +
      • +

        Ensuring that the external prover acts honestly.

        +
        • +
      • +

        Avoiding centralized points of failure.

        +
        • +
      +
      • +
    • +

      Solutions:

      +
        +
      • +

        Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work.

        +
        • +
      • +

        Use reputation systems to rank provers based on their past performances, creating a trust hierarchy.

        +
        • +
      +
      • +
    +
    3. +
  3. +

    ** Incentivization Framework:** A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task.

    +
      +
    • +

      Challenges:

      +
        +
      • +

        Determining the right balance of rewards and penalties.

        +
        • +
      • +

        Ensuring that the system cannot be gamed for undue advantage.

        +
        • +
      +
      • +
    • +

      Solutions1:

      +
        +
      • +

        Implement a dynamic reward system that adjusts based on network metrics and provers' performance.

        +
        • +
      • +

        Use a staking mechanism where provers need to lock up a certain amount of assets. +Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets.

        +
        • +
      +
      • +
    +
    4. +
  4. +

    Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks.

    +
      +
    • +

      Challenges:

      +
        +
      • +

        Setting the right cost metrics for both verification and execution.

        +
        • +
      • +

        Ensuring that genuine users aren’t priced out of the system.

        +
        • +
      +
      • +
    • +

      Solutions:

      +
        +
      • +

        Use a dynamic pricing model, adjusting costs in real-time based on network demand.

        +
        • +
      • +

        Implement gas-like mechanisms to differentiate operation costs and ensure fairness.

        +
        • +
      +
      • +
    +
    5. +
  5. +

    ** Maintaining Trustworthiness:** Create a system that's transparent, holds all actors accountable, and is economically sound.

    +
      +
    • +

      Challenges:

      +
        +
      • +

        Keeping the balance where users feel their privacy is intact, while provers feel incentivized.

        +
        • +
      • +

        Ensuring the system remains resilient against adversarial attacks.

        +
        • +
      +
      • +
    • +

      Solutions:

      +
        +
      • +

        Implement layered checks and balances.

        +
        • +
      • +

        Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO).

        +
        • +
      +
      • +
    +
    6. +
+

Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, cost dynamics, and verification integrity. A thoughtful combination of these mechanisms ensures that the system remains robust, resilient, -and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation.

AspectDetails
Design Principle- User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions.
Trust & Privacy- Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage.
Incentivization Framework- Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior.
Economic Considerations- Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency.
OutcomeAn ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.

Goal 4: Kernel-based Architecture Implementation​

This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. +and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation.

+
AspectDetails
Design Principle- User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions.
Trust & Privacy- Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage.
Incentivization Framework- Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior.
Economic Considerations- Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency.
OutcomeAn ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.
+

Goal 4: Kernel-based Architecture Implementation​

+

This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. This innovative approach employs recursion to construct a call stack, which is then validated through iterative recursive computations. -At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack.

The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. -While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies.

The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. +At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack.

+

The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. +While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies.

+

The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. Moreover, the iterative recursive computations could potentially exhibit computational expansion as the depth of recursion increases. -This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail.

In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. +This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail.

+

In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. By weaving recursion and iterative recursive computations into the fabric of our system, we aim to establish a mechanism that accentuates the trustworthiness of private function executions, -while conscientiously navigating the computational demands that ensue.

To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, -several strategic steps and considerations can be undertaken: recursion handling and depth management.

Recursion Handling

  • Call Stack Management:

    • Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state.

  • Proof Accumulation:

    • Design a mechanism to accumulate proof data for each function call within the call stack. -This includes cryptographic commitments, intermediate results, and cryptographic challenges.

    • Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process.

  • Intermediary SNARK Proofs:

    • Develop an intermediary SNARK proof for each function call’s correctness within the call stack. -This proof should demonstrate that the function executed correctly and produced expected outputs.

    • Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack.

Depth management

  • Depth Limitation:

    • Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations.

    • Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth.

  • Graceful Degradation:

    • Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. -This may involve transitioning to alternative execution modes or optimization techniques.

    • Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed.

  • Resource Monitoring:

    • Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. -Provide real-time feedback to users about the cost and impact of recursive execution.

  • Dynamic Depth Adjustment:

    • Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources.

    • Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints.

  • Fallback Mechanisms:

    • Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. -These mechanisms could involve alternative verification methods or delayed execution.

  • User Notifications:

    • Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage.

Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. +while conscientiously navigating the computational demands that ensue.

+

To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, +several strategic steps and considerations can be undertaken: recursion handling and depth management.

+ Recursion Handling +
    +
  • +

    Call Stack Management:

    +
      +
    • Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state.
      • +
    +
    • +
  • +

    _Proof Accumulation: _

    +
      +
    • +

      Design a mechanism to accumulate proof data for each function call within the call stack. +This includes cryptographic commitments, intermediate results, and cryptographic challenges.

      +
      • +
    • +

      Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process.

      +
      • +
    +
    • +
  • +

    Intermediary SNARK Proofs:

    +
      +
    • +

      Develop an intermediary SNARK proof for each function call’s correctness within the call stack. +This proof should demonstrate that the function executed correctly and produced expected outputs.

      +
      • +
    • +

      Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack.

      +
      • +
    +
    • +
+ Depth management +
    +
  • +

    Depth Limitation:

    +
      +
    • +

      Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations.

      +
      • +
    • +

      Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth.

      +
      • +
    +
    • +
  • +

    Graceful Degradation:

    +
      +
    • +

      Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. +This may involve transitioning to alternative execution modes or optimization techniques.

      +
      • +
    • +

      Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed.

      +
      • +
    +
    • +
  • +

    Resource Monitoring:

    +
      +
    • Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. +Provide real-time feedback to users about the cost and impact of recursive execution.
      • +
    +
    • +
  • +

    Dynamic Depth Adjustment:

    +
      +
    • +

      Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources.

      +
      • +
    • +

      Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints.

      +
      • +
    +
    • +
  • +

    Fallback Mechanisms:

    +
      +
    • Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. +These mechanisms could involve alternative verification methods or delayed execution.
      • +
    +
    • +
  • +

    User Notifications:

    +
      +
    • Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage.
      • +
    +
    • +
+

Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. While the approach promises robust outcomes, it's pivotal to maneuver through its intricacies with astute strategies, ensuring computational efficiency and economic viability. -By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.

Goal 5: Seamless Interaction Design​

Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. -This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions.

A notable challenge in this endeavor lies in the intricate interplay between public and private states, -wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage.

The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. -This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality.

A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. +By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.

+

Goal 5: Seamless Interaction Design​

+

Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. +This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions.

+

A notable challenge in this endeavor lies in the intricate interplay between public and private states, +wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage.

+

The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. +This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality.

+

A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. The solution involves devising strategies to mitigate the risk of revealing private transaction details when connected to corresponding public actions. -By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability.

Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, -without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design.

To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. -In the table below, a comprehensive list of solutions that address these objectives:

Solution CategoryDescription
Layer 2 SolutionsEmploy zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs.
Intermediary Smart ContractsCraft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially.
Decentralized Identity & PseudonymityImplement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs.
Confidential Sidechains & Cross-ChainSet up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains.
Temporal Data StructuresCreate chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy.
Homomorphic Encryption & MPCApply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers.
Commit-Reveal SchemesIntroduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions.
Auditability & VerifiabilityUse on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation.
Data Fragmentation & ShardingFragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography.
Ring Signatures & CoinJoinIncorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.

Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework​

The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. +By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability.

+

Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, +without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design.

+

To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. +In the table below, a comprehensive list of solutions that address these objectives:

+
Solution CategoryDescription
Layer 2 SolutionsEmploy zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs.
Intermediary Smart ContractsCraft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially.
Decentralized Identity & PseudonymityImplement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs.
Confidential Sidechains & Cross-ChainSet up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains.
Temporal Data StructuresCreate chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy.
Homomorphic Encryption & MPCApply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers.
Commit-Reveal SchemesIntroduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions.
Auditability & VerifiabilityUse on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation.
Data Fragmentation & ShardingFragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography.
Ring Signatures & CoinJoinIncorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.
+

Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework​

+

The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. This endeavor comes with inherent challenges, especially considering the heterogeneity of existing DeFi protocols, predominantly built on Ethereum. These variations in programming languages and VMs exacerbate the quest for interoperability. Furthermore, the success and functionality of DeFi protocols is closely tied to liquidity, -which in turn is influenced by user engagement and the amount of funds locked into the system.

Strategic Roadmap for Goal 6 ​

  1. Pioneering Privacy-Centric DeFi Models: Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity.

  2. Specialized Smart Contracts with Privacy: Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape.

  3. Optimized User Interfaces: Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security.

  4. Tackling Interoperability:

    • Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments.

    • Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly.

  1. Enhancing and Sustaining Liquidity:

    • Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space.

    • Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity.

    • Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies.

  1. Amplifying Community Engagement: Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, -thereby nurturing a vibrant, privacy-advocating DeFi community.

Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, -all while effectively overcoming the obstacles related to interoperability and liquidity concerns.

Summary of the Architecture​

In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). +which in turn is influenced by user engagement and the amount of funds locked into the system.

+

Strategic Roadmap for Goal 6 ​

+
    +
  1. +

    ** Pioneering Privacy-Centric DeFi Models: ** Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity.

    +
    2. +
  2. +

    ** Specialized Smart Contracts with Privacy: ** Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape.

    +
    3. +
  3. +

    ** Optimized User Interfaces: ** Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security.

    +
    4. +
  4. +

    ** Tackling Interoperability: **

    +
      +
    • +

      Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments.

      +
      • +
    • +

      Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly.

      +
      • +
    +
    5. +
  5. +

    ** Enhancing and Sustaining Liquidity: **

    +
      +
    • +

      Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space.

      +
      • +
    • +

      Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity.

      +
      • +
    • +

      Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies.

      +
      • +
    +
    6. +
  6. +

    ** Amplifying Community Engagement:** Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, +thereby nurturing a vibrant, privacy-advocating DeFi community.

    +
    7. +
+

Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, +all while effectively overcoming the obstacles related to interoperability and liquidity concerns.

+

Summary of the Architecture​

+

In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). These proofs ensure that while private state data remains undisclosed, public state transitions can still be carried out and subsequently verified by third parties. This blend of public and private state is envisaged to be achieved through a state tree representing the public state, while the encrypted state leaves stand for the private state. Each user's private state indicates validity through the absence of a corresponding nullifier. A nullifier is a unique cryptographic value generated in privacy-preserving blockchain transactions to prevent double-spending, -ensuring that each private transaction is spent only once without revealing its details.

Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. +ensuring that each private transaction is spent only once without revealing its details.

+

Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. For validating a singular private function call, we're leaning into the kernel-based model inspired by the ZEXE protocol. Defined as kernel circuits, these functions validate the correct execution of each private function call. Due to their recursive circuit structure, a succession of private function calls can be executed by calculating proofs in an iterative manner. -Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs.

Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. -The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities.

We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. +Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs.

+

Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. +The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities.

+

We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. Instead of the ZEXE protocol's structure, our approach pivots to a method anchored in the security of a secret combined with the uniqueness from hashing with the contract address. The underlying rationale is straightforward: the sender, equipped with a unique nonce and salt for the transaction, hashes the secret, payload, nonce, and salt. This result is then hashed with the contract address for the final value. The hash function's unidirectional nature ensures that the input cannot be deduced easily from its output. A specific concern, however, is the potential repetition of secret and payload values across transactions, which could jeopardize privacy. -Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation.

Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. +Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation.

+

Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. Fully Homomorphic Encryption (FHE) offers another pathway, enabling function execution on encrypted data, or Multi-Party Computation (MPC) which guarantees non-disclosure of function or inputs. Yet, integrating ZKPs with either FHE or MPC presents a challenge. Combining cryptographic functions like SHA-3 and BLAKE2 can also bolster security and uniqueness. -It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.

Current State​

Our aim is to revolutionize the privacy and security paradigms through Nescience. +It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.

+

Current State​

+

Our aim is to revolutionize the privacy and security paradigms through Nescience. As we strive to set milestones and achieve groundbreaking advancements, -our current focus narrows onto the realization of Goal 2 and Goal 3.

Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. +our current focus narrows onto the realization of Goal 2 and Goal 3.

+

Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. We believe that integrating the right proof system is pivotal to our project's success, which brings us to Nova [8]. In our project journey, we have opted to integrate the Nova proof system, recognizing its potential alignment with our overarching goals. However, as part of our meticulous approach to innovation and optimization, we acknowledge the need to thoroughly examine Nova’s performance capabilities, -particularly due to its status as a pioneering and relatively unexplored proof system.

This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], +particularly due to its status as a pioneering and relatively unexplored proof system.

+

This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], pitting Nova against other prominent proof systems in the field, including Halo2 [10], Plonky2 [11], and Starky [12]. This ongoing and methodical initiative is designed to ensure a fair and impartial assessment, enabling us to draw meaningful conclusions about Nova’s strengths and limitations in relation to its counterparts. @@ -168,18 +733,51 @@ By leveraging the Poseidon recursion technique, we are poised to conduct an exha Through this testing framework, we aim to discern whether Nova possesses the potential to outshine its contemporaries in terms of efficiency, scalability, and overall performance. The outcome of this rigorous evaluation will be pivotal in shaping our strategic decisions moving forward. Armed with a comprehensive understanding of Nova’s performance metrics vis-à-vis other proof systems, -we can confidently chart a course that maximizes the benefits of our project’s optimization efforts.

Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, +we can confidently chart a course that maximizes the benefits of our project’s optimization efforts.

+

Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, incentivizing honest behaviour, and ensuring the cost-effective verification of transactions. At the heart of this endeavor is our drive to empower users by allowing them the autonomy of generating proofs for private state updates, thereby reducing dependencies and enhancing privacy. We would like to actively work on providing comprehensive documentation, user-friendly tools, -and tutorials to aid users in this intricate process.

Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. +and tutorials to aid users in this intricate process.

+

Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. Our commitment is further cemented by our efforts to introduce a dynamic reward system that adjusts based on network metrics and prover performance. -This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.

References

[1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

[2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

[3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

[4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

[5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md

[6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin

[7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui

[8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370

[9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations

[10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/

[11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2

[12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582

  1. Incentive Mechanisms:

    • Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. -This incentivizes participation and encourages integrity.

    • Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. -Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded.

    • Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, -aligning incentives with the blockchain’s broader consensus mechanism.

    ↩

- - +This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.

+

References

+

[1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

+

[2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

+

[3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

+

[4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

+

[5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md

+

[6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin

+

[7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui

+

[8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370

+

[9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations

+

[10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/

+

[11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2

+

[12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582

+ +

Footnotes​

+
    +
  1. +

    Incentive Mechanisms:

    +
      +
    • +

      Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. +This incentivizes participation and encourages integrity.

      +
      • +
    • +

      Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. +Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded.

      +
      • +
    • +

      Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, +aligning incentives with the blockchain’s broader consensus mechanism.

      +
      • +
    +↩ +
    2. +
+
\ No newline at end of file diff --git a/rlog/Nescience-state-separation-architecture/index.html b/rlog/Nescience-state-separation-architecture/index.html index e5839b86..63e10aaa 100644 --- a/rlog/Nescience-state-separation-architecture/index.html +++ b/rlog/Nescience-state-separation-architecture/index.html @@ -1,477 +1,1000 @@ - + - -Nescience: A User-Centric State-Separation Architecture | Vac Research + +Nescience: A User-Centric State-Separation Architecture | Vac Research - - - - + + + + -
-

Nescience: A User-Centric State-Separation Architecture

by
87 min read

Nescience: A user-centric state-separation architecture.

Disclaimer: This content is a work in progress. Some components may be updated, changed, or expanded as new research findings become available.

A. Introduction

In blockchain applications, privacy settings are typically predefined by developers, leaving users with limited control. This traditional, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Nescience: A User-Centric State-Separation Architecture

by
87 min read

Nescience: A user-centric state-separation architecture.

+ +

Disclaimer: This content is a work in progress. Some components may be updated, changed, or expanded as new research findings become available.

+

In blockchain applications, privacy settings are typically predefined by developers, leaving users with limited control. This traditional, one-size-fits-all approach often leads to inefficiencies and potential privacy concerns as it fails to cater to the diverse needs of individual users. The Nescience state-separation architecture (NSSA) aims to address these issues by shifting privacy control from developers to users. NSSA introduces a flexible, user-centric approach that allows for customized privacy settings to better meet individual needs. This blog post will delve into the details of NSSA, -including its different execution types, cryptographic foundations, and unique challenges.

Introducing NSSA: A user-centric approach​

NSSA gives users control over their privacy settings by introducing shielded (which creates a layer of privacy for the outputs, and only the necessary details are shared) +including its different execution types, cryptographic foundations, and unique challenges.

+

Introducing NSSA: A user-centric approach​

+

NSSA gives users control over their privacy settings by introducing shielded (which creates a layer of privacy for the outputs, and only the necessary details are shared) and deshielded (which reveal private details, making them publicly visible) execution types in addition to the traditional public and private modes. This flexibility allows users to customize their privacy settings to match their unique needs, whether they require high levels of confidentiality or more transparency. In NSSA, the system is divided into two states: public and private. The public state uses an account-based model while the private state employs a UTXO-based (unspent transaction output) model. Private executions within NSSA utilize UTXO exchanges, ensuring that transaction details remain confidential. The sequencer verifies these exchanges without accessing specific details, enhancing privacy by unlinking sender and receiver identities. Zero-knowledge proofs (ZKPs) allow users to prove transaction validity without revealing data, maintaining the integrity and confidentiality of private transactions. UTXOs contain assets such as balances, NFTs, or private storage data, and are stored in plaintext within Sparse Merkle trees (SMTs) in the private state and -as hashes in the public state. This dual-storage approach keeps UTXO details confidential while allowing public verification through hashes, achieving a balance between privacy and transparency.

Implementing NSSA introduces unique challenges, particularly in cryptographic implementation and maintaining the integrity of private executions. These challenges are addressed +as hashes in the public state. This dual-storage approach keeps UTXO details confidential while allowing public verification through hashes, achieving a balance between privacy and transparency.

+

Implementing NSSA introduces unique challenges, particularly in cryptographic implementation and maintaining the integrity of private executions. These challenges are addressed through various solutions such as ZKPs, which ensure transaction validity without compromising privacy, and the dual-storage approach, which maintains confidentiality while enabling public verification. By allowing users to customize their privacy settings, NSSA enhances user experience and promotes wider adoption of private execution platforms. As we move towards -a future where user-empowered privacy control is crucial, NSSA provides a flexible and user-centric solution that meets the diverse needs of blockchain users.

Why NSSA differs from other hybrid execution platforms​

In many existing hybrid execution platforms, privacy settings are predefined by developers, often applying a one-size-fits-all approach that does not accommodate the +a future where user-empowered privacy control is crucial, NSSA provides a flexible and user-centric solution that meets the diverse needs of blockchain users.

+

Why NSSA differs from other hybrid execution platforms​

+

In many existing hybrid execution platforms, privacy settings are predefined by developers, often applying a one-size-fits-all approach that does not accommodate the diverse privacy needs of users. These platforms blend public and private states, but control over privacy remains with the application developers. While this approach is straightforward for developers (who bear the responsibility for any potential privacy leaks), it leaves users with no control over their own privacy settings. This rigidity becomes problematic as user needs evolve over time, or as new regulations necessitate changes to privacy configurations. In such cases, -updates to decentralized applications are required to adjust privacy settings, which can disrupt the user experience and create friction.

NSSA addresses these limitations by introducing a groundbreaking concept: selective privacy. Unlike traditional platforms where privacy +updates to decentralized applications are required to adjust privacy settings, which can disrupt the user experience and create friction.

+

NSSA addresses these limitations by introducing a groundbreaking concept: selective privacy. Unlike traditional platforms where privacy is static and determined by developers, selective privacy empowers users to dynamically choose their own privacy levels based on their unique needs and sensitivity. -This flexibility is critical in a decentralized ecosystem where the diversity of users and use cases demands a more adaptable privacy solution.

In the NSSA model, users have the autonomy to select how they interact with decentralized applications (dapps) by choosing from four types of transaction executions: public, +This flexibility is critical in a decentralized ecosystem where the diversity of users and use cases demands a more adaptable privacy solution.

+

In the NSSA model, users have the autonomy to select how they interact with decentralized applications (dapps) by choosing from four types of transaction executions: public, private, shielded, and deshielded. This model allows users to tailor their privacy settings on a per-transaction basis, selecting the most appropriate execution type for each specific interaction. For instance, a user concerned about data confidentiality might opt for a fully private transaction while another user, wary of privacy but seeking transparency, -might choose a public execution.

While selective privacy may appear complex, especially for users who are not technically inclined, Nescience mitigates this by allowing the community or developers to +might choose a public execution.

+

While selective privacy may appear complex, especially for users who are not technically inclined, Nescience mitigates this by allowing the community or developers to establish best practices and recommended approaches. These guidelines provide users with an informed starting point, and over time, users can adjust their privacy settings as their preferences and trust in the platform evolve. Importantly, selective privacy gives users the right to alter their privacy level at any point in the future, -ensuring that their privacy settings remain aligned with their needs as they change.

This approach not only empowers users but also facilitates greater adoption of dapps. Users who are skeptical about privacy concerns can initially engage with transparent +ensuring that their privacy settings remain aligned with their needs as they change.

+

This approach not only empowers users but also facilitates greater adoption of dapps. Users who are skeptical about privacy concerns can initially engage with transparent transactions and gradually shift towards more private executions as they gain confidence in the system and vice versa for users who start with privacy but later find transparency beneficial for certain transactions. In this way, selective privacy bridges the gap between privacy and transparency, allowing for an optimal balance to emerge from the community’s -collective preferences.

To liken this to open-source projects: in traditional systems, developers fix privacy rules much like immutable code—users must comply with these fixed settings. -In contrast, with selective privacy, the rules are malleable and shaped by the users’ preferences, enabling the community to find the ideal balance between privacy and efficiency over time.

NSSA is distinct from traditional zero-knowledge (ZK) rollups in several key ways. One of the unique features of NSSA is its public execution type, which does not +collective preferences.

+

To liken this to open-source projects: in traditional systems, developers fix privacy rules much like immutable code—users must comply with these fixed settings. +In contrast, with selective privacy, the rules are malleable and shaped by the users’ preferences, enabling the community to find the ideal balance between privacy and efficiency over time.

+

NSSA is distinct from traditional zero-knowledge (ZK) rollups in several key ways. One of the unique features of NSSA is its public execution type, which does not require ZKPs or a zero-knowledge virtual machine (zkVM). This provides a significant advantage in terms of scalability and efficiency as users can choose public executions for -transactions that do not require enhanced privacy, avoiding the overhead associated with ZKP generation and verification.

Moreover, NSSA introduces two additional execution types—shielded and deshielded—which further distinguish it from traditional privacy-preserving rollups. +transactions that do not require enhanced privacy, avoiding the overhead associated with ZKP generation and verification.

+

Moreover, NSSA introduces two additional execution types—shielded and deshielded—which further distinguish it from traditional privacy-preserving rollups. These execution types allow for more nuanced control over privacy, giving users the ability to shield certain aspects of a transaction while deshielding others. -This flexibility sets NSSA apart as a more adaptable and user-centric platform, catering to a wide range of privacy needs without imposing a one-size-fits-all solution.

By combining selective privacy with a flexible execution model, NSSA offers a more robust and adaptable framework for decentralized applications, -ensuring that users maintain control over their privacy while benefiting from the security and efficiency of blockchain technology.

How Nescience state-separation architecture can be used​

NSSA offers a flexible, privacy-preserving add-on that can be applied to existing dapps. +This flexibility sets NSSA apart as a more adaptable and user-centric platform, catering to a wide range of privacy needs without imposing a one-size-fits-all solution.

+

By combining selective privacy with a flexible execution model, NSSA offers a more robust and adaptable framework for decentralized applications, +ensuring that users maintain control over their privacy while benefiting from the security and efficiency of blockchain technology.

+

How Nescience state-separation architecture can be used​

+

NSSA offers a flexible, privacy-preserving add-on that can be applied to existing dapps. One of the emerging trends in the blockchain space is that each dapp is expected to have its own rollup for efficiency, and it is estimated that Ethereum could see the deployment of different rollups in the near future. A key question arises: how many of these rollups will incorporate privacy? For dapp developers who want to offer flexible, -user-centric privacy features, NSSA provides a solution through selective privacy.

Use case: Adding privacy to existing dapps​

Consider a dapp running on a transparent network that offers no inherent privacy to its users. Converting this dapp to a privacy-preserving architecture from scratch would +user-centric privacy features, NSSA provides a solution through selective privacy.

+

Use case: Adding privacy to existing dapps​

+

Consider a dapp running on a transparent network that offers no inherent privacy to its users. Converting this dapp to a privacy-preserving architecture from scratch would require significant effort, restructuring, and a deep understanding of cryptographic frameworks. However, with NSSA, the dapp does not need to undergo extensive changes. -Instead, the Nescience state-separation model can be deployed as an add-on, offering selective privacy as an option for the dapp’s users.

This allows the dapp to retain its existing functionality while providing users with a choice between the traditional, transparent version and a new version with selective privacy features. +Instead, the Nescience state-separation model can be deployed as an add-on, offering selective privacy as an option for the dapp’s users.

+

This allows the dapp to retain its existing functionality while providing users with a choice between the traditional, transparent version and a new version with selective privacy features. With NSSA, the privacy settings are flexible, meaning users can tailor their level of privacy according to their individual needs while the dapp operates on its current infrastructure. -This contrasts sharply with the typical approach, where dapps are either entirely transparent or fully private, with no flexibility for users to select their own privacy preferences.

Key advantage: Decoupling from the host chain​

A key feature of NSSA is that it operates independently of the privacy characteristics of the host blockchain. Whether the host chain is fully transparent or fully private, +This contrasts sharply with the typical approach, where dapps are either entirely transparent or fully private, with no flexibility for users to select their own privacy preferences.

+

Key advantage: Decoupling from the host chain​

+

A key feature of NSSA is that it operates independently of the privacy characteristics of the host blockchain. Whether the host chain is fully transparent or fully private, the Nescience state-separation architecture can be deployed on top of it, offering users the ability to choose their own privacy settings. -This decoupling from the host chain’s inherent privacy model is critical as it allows users to benefit from selective privacy even in environments that were not originally designed to offer it.

In fully private chains, NSSA allows users to selectively reveal transaction details when compliance with regulations or other requirements is necessary. -In fully transparent chains, NSSA allows users to maintain privacy for specific transactions, offering flexibility that would not otherwise be possible.

Conclusion​

NSSA provides a powerful tool for dapp developers who want to offer selective privacy to their users without the need for a complete overhaul of their existing systems. +This decoupling from the host chain’s inherent privacy model is critical as it allows users to benefit from selective privacy even in environments that were not originally designed to offer it.

+

In fully private chains, NSSA allows users to selectively reveal transaction details when compliance with regulations or other requirements is necessary. +In fully transparent chains, NSSA allows users to maintain privacy for specific transactions, offering flexibility that would not otherwise be possible.

+

Conclusion​

+

NSSA provides a powerful tool for dapp developers who want to offer selective privacy to their users without the need for a complete overhaul of their existing systems. By deploying NSSA as an add-on, dapps can give users the ability to choose their own privacy settings whether they are operating on -transparent or private blockchains. This flexibility makes NSSA a valuable option for any dapp looking to provide enhanced privacy options while maintaining efficiency and ease of use.

B. Design

In this section, we will delve into the core design components of the Nescience state-separation architecture, covering its key structural elements and the mechanisms -that drive its functionality. We will explore the following topics:

  1. Architecture's components: An in-depth look at the foundational building blocks of NSSA, including the public and private states, UTXO structures, zkVM, and smart contracts. -These components work together to facilitate secure, flexible, and scalable transactions within the architecture.

  2. General execution overview: We will outline the overall flow of transaction execution within NSSA, describing how users interact with the system and how the architecture -supports various types of executions—public, private, shielded, and deshielded—while preserving privacy and efficiency.

  3. Execution processes and UTXO management: This section will focus on the lifecycle of UTXOs within the architecture, from their generation to consumption. -We will also cover the processes involved in managing UTXOs, including proof generation, state transitions, and ensuring transaction validity.

These topics will provide a comprehensive understanding of how NSSA enables flexible and secure interactions within dapps.

1. Architecture's components​

NSSA introduces an advanced prototype execution framework designed to enhance privacy and security in blockchain applications. -This framework integrates several essential components: the public state, private state, zkVM, various execution types, Nescience users, and smart contracts.

a) Public state​

The public state in the NSSA is a fundamental component designed to hold all publicly accessible information within +transparent or private blockchains. This flexibility makes NSSA a valuable option for any dapp looking to provide enhanced privacy options while maintaining efficiency and ease of use.

+

B. Design

+

In this section, we will delve into the core design components of the Nescience state-separation architecture, covering its key structural elements and the mechanisms +that drive its functionality. We will explore the following topics:

+
    +
  1. +

    Architecture's components: An in-depth look at the foundational building blocks of NSSA, including the public and private states, UTXO structures, zkVM, and smart contracts. +These components work together to facilitate secure, flexible, and scalable transactions within the architecture.

    +
    2. +
  2. +

    General execution overview: We will outline the overall flow of transaction execution within NSSA, describing how users interact with the system and how the architecture +supports various types of executions—public, private, shielded, and deshielded—while preserving privacy and efficiency.

    +
    3. +
  3. +

    Execution processes and UTXO management: This section will focus on the lifecycle of UTXOs within the architecture, from their generation to consumption. +We will also cover the processes involved in managing UTXOs, including proof generation, state transitions, and ensuring transaction validity.

    +
    4. +
+

These topics will provide a comprehensive understanding of how NSSA enables flexible and secure interactions within dapps.

+

1. Architecture's components​

+
+

NSSA introduces an advanced prototype execution framework designed to enhance privacy and security in blockchain applications. +This framework integrates several essential components: the public state, private state, zkVM, various execution types, Nescience users, and smart contracts.

+

a) Public state​

+
+

The public state in the NSSA is a fundamental component designed to hold all publicly accessible information within the blockchain network. This state is organized as a single Merkle tree structure, a sophisticated data structure that ensures efficient and secure data verification. -The public state includes critical information such as user balances and the public storage data of smart contracts.

In an account-based model, the public state operates by storing each account or smart contract's public data as individual leaf nodes within the Merkle tree. +The public state includes critical information such as user balances and the public storage data of smart contracts.

+

In an account-based model, the public state operates by storing each account or smart contract's public data as individual leaf nodes within the Merkle tree. When transactions occur, they directly modify the state by updating these leaf nodes. This direct modification ensures that the most current state of the network -is always reflected accurately.

The Merkle tree structure is essential for maintaining data integrity. Each leaf node contains a hash of a data block, and each non-leaf node contains the +is always reflected accurately.

+

The Merkle tree structure is essential for maintaining data integrity. Each leaf node contains a hash of a data block, and each non-leaf node contains the hash of its child nodes. This hierarchical arrangement means that any change in the data will result in a change in the corresponding hash, making it easy to detect any tampering. The root hash, or Merkle root, is stored on the blockchain, providing a cryptographic guarantee of the data's integrity. This root hash serves as a single, -concise representation of the entire state, enabling quick and reliable verification by any network participant.

Transparency is a key feature of the public state. All data stored within this state is openly accessible and verifiable by any participant in the network. +concise representation of the entire state, enabling quick and reliable verification by any network participant.

+

Transparency is a key feature of the public state. All data stored within this state is openly accessible and verifiable by any participant in the network. This openness ensures that all transactions and state changes are visible and auditable, fostering trust and accountability. For example, user balances are publicly viewable, which helps ensure transparency and trust in the system. Similarly, public smart contract storage can be accessed and verified by anyone, -making it suitable for applications that require public scrutiny and auditability, such as public record updates and some financial transactions.

The workflow of managing the public state involves several steps to ensure data integrity and transparency. When a user initiates a transaction involving public data, +making it suitable for applications that require public scrutiny and auditability, such as public record updates and some financial transactions.

+

The workflow of managing the public state involves several steps to ensure data integrity and transparency. When a user initiates a transaction involving public data, the relevant changes are proposed and applied to the public state tree. The transaction details, such as transferring funds between accounts or updating smart contract storage, update the corresponding leaf nodes in the Merkle tree. Following this, the hashes of the affected nodes are recalculated up to the root, ensuring that the entire tree accurately reflects the new state of the network. The updated Merkle root is then recorded on the blockchain, allowing all network participants to verify the integrity -of the public state. Any discrepancy in the data will result in a mismatched root hash, signaling potential tampering or errors.

In summary, the public state in NSSA leverages the robustness of the Merkle tree structure to provide a secure, transparent, and verifiable environment for publicly +of the public state. Any discrepancy in the data will result in a mismatched root hash, signaling potential tampering or errors.

+

In summary, the public state in NSSA leverages the robustness of the Merkle tree structure to provide a secure, transparent, and verifiable environment for publicly accessible information. By operating on an account-based model and maintaining rigorous data integrity checks, the public state ensures that all transactions are -transparent and trustworthy, laying a strong foundation for a reliable blockchain network.

b) Private state​

The private state in the NSSA is a sophisticated system designed to maintain user privacy while ensuring transaction integrity. +transparent and trustworthy, laying a strong foundation for a reliable blockchain network.

+

b) Private state​

+
+

The private state in the NSSA is a sophisticated system designed to maintain user privacy while ensuring transaction integrity. Each user has their own individual Merkle tree, which holds their private information such as balances and storage data. This structure is distinct from the public state, which uses an account-based model. Instead, the private state employs a UTXO-based model. In this model, each transaction output is a discrete -unit that can be independently spent in future transactions. This design provides users with granular control over their transaction outputs.

A key aspect of maintaining privacy within the private state is the use of ZKPs. ZKPs allow transactions to be validated without revealing any +unit that can be independently spent in future transactions. This design provides users with granular control over their transaction outputs.

+

A key aspect of maintaining privacy within the private state is the use of ZKPs. ZKPs allow transactions to be validated without revealing any underlying private data. This means that while the system can verify that a transaction is legitimate, the details of the transaction remain confidential. Only parties -with the appropriate viewing key can access and reconstruct the user’s list of UTXOs, ensuring that sensitive information is protected.

The private state also employs a dual-storage approach to balance privacy and transparency. UTXOs are stored in plaintext within SMTs in the private state, +with the appropriate viewing key can access and reconstruct the user’s list of UTXOs, ensuring that sensitive information is protected.

+

The private state also employs a dual-storage approach to balance privacy and transparency. UTXOs are stored in plaintext within SMTs in the private state, providing detailed and accessible records for the user. In contrast, the public state only holds hashes of these UTXOs. This method ensures that while the public can verify -the existence and integrity of private transactions through these hashes, they cannot access the specific details.

The workflow for a transaction in the private state begins with the user initiating a transaction involving their private data, such as transferring a private balance or +the existence and integrity of private transactions through these hashes, they cannot access the specific details.

+

The workflow for a transaction in the private state begins with the user initiating a transaction involving their private data, such as transferring a private balance or updating private smart contract storage. The transaction involves spending existing UTXOs, represented as leaves in the Merkle tree, and creating new UTXOs, which are then appended to the user’s private list. The zkVM generates a ZKP to validate the transaction without revealing -any private data, ensuring the transaction adheres to the system's rules.

Once the proof is generated, it is submitted to the sequencer, which verifies the transaction’s validity. Upon successful verification, the nullifier is added to the nullifier set, -preventing double spending of the same UTXO. The use of ZKPs and nullifiers ensures that the private state maintains both security and privacy.

In summary, the private state in NSSA is meticulously designed to provide users with control over their private information while ensuring the security and integrity of transactions. +any private data, ensuring the transaction adheres to the system's rules.

+

Once the proof is generated, it is submitted to the sequencer, which verifies the transaction’s validity. Upon successful verification, the nullifier is added to the nullifier set, +preventing double spending of the same UTXO. The use of ZKPs and nullifiers ensures that the private state maintains both security and privacy.

+

In summary, the private state in NSSA is meticulously designed to provide users with control over their private information while ensuring the security and integrity of transactions. By utilizing a UTXO-based model, individual Merkle trees, ZKPs, and a dual-storage system, NSSA achieves a balance between confidentiality and verifiability, -making it a robust solution for managing private blockchain transactions.

c) ZkVM (zero-knowledge virtual machine)​

The zkVM is a pivotal component in NSSA, designed to uphold the highest standards +making it a robust solution for managing private blockchain transactions.

+

c) ZkVM (zero-knowledge virtual machine)​

+
+

The zkVM is a pivotal component in NSSA, designed to uphold the highest standards of privacy and security in blockchain transactions. Its primary function is to generate and aggregate ZKPs, enabling users to validate the correctness of their transactions without disclosing any underlying details. This capability is crucial for maintaining the confidentiality and integrity of sensitive -data within the blockchain network.

ZKPs are sophisticated cryptographic protocols that allow one party, the prover, to convince another party, the verifier, that a certain statement is true, +data within the blockchain network.

+

ZKPs are sophisticated cryptographic protocols that allow one party, the prover, to convince another party, the verifier, that a certain statement is true, without revealing any information beyond the validity of the statement itself. In the context of the zkVM, this means users can prove their transactions are valid without -exposing transaction specifics, such as amounts or parties involved. This process is essential for transactions within the private state, where maintaining confidentiality is paramount.

The generation of ZKPs involves intricate cryptographic computations. When a user initiates a transaction, the zkVM processes the transaction inputs and produces a proof +exposing transaction specifics, such as amounts or parties involved. This process is essential for transactions within the private state, where maintaining confidentiality is paramount.

+

The generation of ZKPs involves intricate cryptographic computations. When a user initiates a transaction, the zkVM processes the transaction inputs and produces a proof that the transaction adheres to the protocol's rules. This proof must be robust enough to convince the verifier of the transaction's validity while preserving the privacy -of the transaction details.

Performance optimization is another critical function of the zkVM. In a typical blockchain scenario, verifying multiple individual proofs can be computationally intensive +of the transaction details.

+

Performance optimization is another critical function of the zkVM. In a typical blockchain scenario, verifying multiple individual proofs can be computationally intensive and time consuming, potentially leading to network congestion and delays. To address this, the zkVM can aggregate multiple ZKPs into a single, consolidated proof. This aggregation significantly reduces the verification overhead as the verifier needs to check only one comprehensive proof rather than multiple individual ones. -This efficiency is vital for maintaining high throughput and low latency in the blockchain network, ensuring that the system can handle a large volume of transactions swiftly and securely.

Furthermore, the zkVM's role extends beyond mere proof generation and aggregation. It also ensures that all transactions meet the required privacy and security standards +This efficiency is vital for maintaining high throughput and low latency in the blockchain network, ensuring that the system can handle a large volume of transactions swiftly and securely.

+

Furthermore, the zkVM's role extends beyond mere proof generation and aggregation. It also ensures that all transactions meet the required privacy and security standards before they are recorded on the blockchain. By interacting seamlessly with other components such as the public and private states, the zkVM ensures that any transaction, -whether it involves public data, private data, or a mix of both, is thoroughly validated and secured.

In summary, the zkVM is essential for the NSSA, providing the cryptographic backbone necessary to support secure and private transactions. Its ability to generate and +whether it involves public data, private data, or a mix of both, is thoroughly validated and secured.

+

In summary, the zkVM is essential for the NSSA, providing the cryptographic backbone necessary to support secure and private transactions. Its ability to generate and aggregate ZKPs not only preserves the confidentiality of user data but also enhances the overall efficiency and scalability of the blockchain network. -By ensuring that all transactions are validated without revealing sensitive information, the zkVM upholds the integrity and trustworthiness of the Nescience blockchain system.

d) Execution types in NSSA​

NSSA incorporates multiple execution types to cater to varying levels of privacy and security requirements. -These execution types—public, private, shielded, and deshielded—are designed to provide users with flexible options for managing their transactions based on their specific privacy needs.

1. Public executions​

Public executions are straightforward transactions that involve reading from and writing to the public state. In this model, data is openly accessible and verifiable +By ensuring that all transactions are validated without revealing sensitive information, the zkVM upholds the integrity and trustworthiness of the Nescience blockchain system.

+

d) Execution types in NSSA​

+
+

NSSA incorporates multiple execution types to cater to varying levels of privacy and security requirements. +These execution types—public, private, shielded, and deshielded—are designed to provide users with flexible options for managing their transactions based on their specific privacy needs.

+

1. Public executions​

+

Public executions are straightforward transactions that involve reading from and writing to the public state. In this model, data is openly accessible and verifiable by all participants in the network. Public executions do not require ZKPs since transparency is the primary goal. This execution type is ideal -for non-sensitive transactions where public visibility is beneficial, such as updating public records, performing open financial transactions, or interacting with public smart contracts.

The workflow for a public execution starts with a user initiating a transaction that modifies public data. The transaction details are then used to update the relevant +for non-sensitive transactions where public visibility is beneficial, such as updating public records, performing open financial transactions, or interacting with public smart contracts.

+

The workflow for a public execution starts with a user initiating a transaction that modifies public data. The transaction details are then used to update the relevant leaf nodes in the Merkle tree. As changes are made, the hashes of affected nodes are recalculated up to the root, ensuring that the entire tree reflects the most recent state. -Finally, the updated Merkle root is recorded on the blockchain, making the new state publicly verifiable.

2. Private executions​

Private executions are designed for confidential transactions, reading from and writing to the private state. These transactions require ZKPs to ensure that while the +Finally, the updated Merkle root is recorded on the blockchain, making the new state publicly verifiable.

+

2. Private executions​

+

Private executions are designed for confidential transactions, reading from and writing to the private state. These transactions require ZKPs to ensure that while the transaction details are validated, the actual data remains private. This execution type is suitable for scenarios where privacy is crucial, such as private financial -transactions or sensitive data management within smart contracts.

In a private execution, the user initiates a transaction involving private data. The transaction spends existing UTXOs and creates new ones, all of which are represented as +transactions or sensitive data management within smart contracts.

+

In a private execution, the user initiates a transaction involving private data. The transaction spends existing UTXOs and creates new ones, all of which are represented as leaves in the Merkle tree. The zkVM generates a ZKP to validate the transaction without revealing private data. This proof is submitted to the sequencer, which verifies the proof to ensure the transaction's validity. Upon successful verification, the nullifier is added to the nullifier set, and the private state is updated -with the new Merkle root.

3. Shielded executions​

Shielded executions create a layer of privacy for the outputs by allowing interactions between the public and private states. When a transaction occurs in a shielded execution, +with the new Merkle root.

+

3. Shielded executions​

+

Shielded executions create a layer of privacy for the outputs by allowing interactions between the public and private states. When a transaction occurs in a shielded execution, details of the transaction are processed within the private state, ensuring that sensitive information remains confidential. Only the necessary details are shared with the public state, -often in a masked or encrypted form. This approach allows for the validation of the transaction without revealing critical data, thus preserving the privacy of the involved parties.

The workflow for shielded executions begins with the user initiating a transaction that reads from the public state and prepares to write to the private state. Public data is accessed, +often in a masked or encrypted form. This approach allows for the validation of the transaction without revealing critical data, thus preserving the privacy of the involved parties.

+

The workflow for shielded executions begins with the user initiating a transaction that reads from the public state and prepares to write to the private state. Public data is accessed, and the private state is prepared to receive new data. The zkVM generates a ZKP to hide the receiver’s identity. This proof is submitted to the sequencer, which verifies the proof to ensure the transaction's validity. If valid, the private state is updated with the new data while the public state reflects the change without revealing private details. -This type of execution is particularly useful for scenarios where the receiver’s identity needs to be hidden, such as in anonymous donation systems or confidential data storage.

4. Deshielded executions​

Deshielded executions operate in the opposite manner of shielded executions, where data is read from the private state and written to the public state. This execution type is useful -in situations where the sender's identity needs to be kept confidential while making the transaction results publicly visible.

In a deshielded execution, the user initiates a transaction that reads from the private state and prepares to write to the public state. Private data is accessed, +This type of execution is particularly useful for scenarios where the receiver’s identity needs to be hidden, such as in anonymous donation systems or confidential data storage.

+

4. Deshielded executions​

+

Deshielded executions operate in the opposite manner of shielded executions, where data is read from the private state and written to the public state. This execution type is useful +in situations where the sender's identity needs to be kept confidential while making the transaction results publicly visible.

+

In a deshielded execution, the user initiates a transaction that reads from the private state and prepares to write to the public state. Private data is accessed, and the transaction details are prepared. The zkVM generates a ZKP to hide the sender’s identity. This proof is then submitted to the sequencer, which verifies the proof to ensure the transaction's validity. Once verified, the public state is updated with the new data, reflecting the change while keeping the sender’s details confidential. This can be useful when transparency is needed, such as when auditing or proving certain aspects of a transaction to a wider audience. By selectively deshielding certain transactions, users can control what information is shared publicly, thus maintaining a balance between privacy and transparency -as required by their specific use case.

Table of execution types​

TypeRead fromWrite toZKP requiredUse caseDescription
PublicPublic statePublic stateNoNon-sensitive transactions requiring transparency.Ideal for transactions that do not require privacy, ensuring full transparency.
PrivatePrivate statePrivate stateYesConfidential transactions needing privacy.Suitable for transactions that require confidentiality. Ensures that transaction details remain private through the use of ZKPs.
ShieldedPublic statePrivate stateYesTransactions where the receiver’s identity needs to be hidden.Hides the identity of the receiver while keeping the transaction details private. Suitable for anonymous donations or confidential data storage.
DeshieldedPrivate statePublic stateYesTransactions where the sender’s identity needs to be hidden.Ensures the sender’s identity remains confidential while making the transaction results public. Suitable for confidential disbursements or anonymized data publication.

By supporting a range of execution types, NSSA provides a flexible and robust framework for managing privacy and security in blockchain transactions. +as required by their specific use case.

+

Table of execution types​

+
TypeRead fromWrite toZKP requiredUse caseDescription
PublicPublic statePublic stateNoNon-sensitive transactions requiring transparency.Ideal for transactions that do not require privacy, ensuring full transparency.
PrivatePrivate statePrivate stateYesConfidential transactions needing privacy.Suitable for transactions that require confidentiality. Ensures that transaction details remain private through the use of ZKPs.
ShieldedPublic statePrivate stateYesTransactions where the receiver’s identity needs to be hidden.Hides the identity of the receiver while keeping the transaction details private. Suitable for anonymous donations or confidential data storage.
DeshieldedPrivate statePublic stateYesTransactions where the sender’s identity needs to be hidden.Ensures the sender’s identity remains confidential while making the transaction results public. Suitable for confidential disbursements or anonymized data publication.
+
+

By supporting a range of execution types, NSSA provides a flexible and robust framework for managing privacy and security in blockchain transactions. Whether the need is for complete transparency, total privacy, or a balanced approach, NSSA's execution types allow users to select the level of confidentiality -that best fits their requirements. This flexibility enhances the overall utility of the blockchain, making it suitable for a wide array of applications and use cases.

e) Nescience users​

Nescience users are integral to the architecture, managing balances and assets within the blockchain network and invoking smart contracts with various privacy options. -They can choose the appropriate execution type—public, private, shielded, or deshielded—based on their specific privacy and security needs.

Users handle both public and private balances. Public balances are visible to all network participants and suitable for non-sensitive transactions, +that best fits their requirements. This flexibility enhances the overall utility of the blockchain, making it suitable for a wide array of applications and use cases.

+

e) Nescience users​

+
+

Nescience users are integral to the architecture, managing balances and assets within the blockchain network and invoking smart contracts with various privacy options. +They can choose the appropriate execution type—public, private, shielded, or deshielded—based on their specific privacy and security needs.

+

Users handle both public and private balances. Public balances are visible to all network participants and suitable for non-sensitive transactions, while private balances are confidential and used for transactions requiring privacy. Digital wallets provide a user-friendly interface for managing -these balances, assets, and transactions, allowing users to select the desired execution type seamlessly.

Security is ensured through the use of cryptographic keys, which authenticate and verify transactions. ZKPs maintain privacy -by validating transaction correctness without revealing underlying data, ensuring sensitive information remains confidential even during verification.

The workflow for users involves initiating a transaction, preparing inputs, interacting with smart contracts, generating proofs if needed, +these balances, assets, and transactions, allowing users to select the desired execution type seamlessly.

+

Security is ensured through the use of cryptographic keys, which authenticate and verify transactions. ZKPs maintain privacy +by validating transaction correctness without revealing underlying data, ensuring sensitive information remains confidential even during verification.

+

The workflow for users involves initiating a transaction, preparing inputs, interacting with smart contracts, generating proofs if needed, and submitting the transaction to the sequencer for verification and state update. This flexible approach supports various use cases, -from financial transactions and decentralized applications to data privacy management, allowing users to maintain control over their privacy settings.

By offering this high degree of flexibility and security, Nescience enables users to tailor their privacy settings to their specific needs, +from financial transactions and decentralized applications to data privacy management, allowing users to maintain control over their privacy settings.

+

By offering this high degree of flexibility and security, Nescience enables users to tailor their privacy settings to their specific needs, ensuring sensitive transactions remain confidential while non-sensitive ones are transparent. This integration of cryptographic keys and ZKPs -provides a robust framework for a wide range of blockchain applications, enhancing both utility and trust within the network.

f) Smart contracts in NSSA​

Smart contracts are a core feature of NSSA, providing a way to automate and execute predefined actions based on coded rules. +provides a robust framework for a wide range of blockchain applications, enhancing both utility and trust within the network.

+

f) Smart contracts in NSSA​

+
+

Smart contracts are a core feature of NSSA, providing a way to automate and execute predefined actions based on coded rules. Once deployed on the blockchain, these contracts become immutable, meaning their behavior cannot be altered. This ensures that they perform exactly as intended without the risk of tampering. Because the state and data of the contract are stored permanently on the blockchain, all interactions are fully -transparent and auditable, creating a reliable and trustworthy environment.

One of the key strengths of smart contracts is their ability to automate processes. They are designed to automatically execute when specific conditions are met, +transparent and auditable, creating a reliable and trustworthy environment.

+

One of the key strengths of smart contracts is their ability to automate processes. They are designed to automatically execute when specific conditions are met, reducing the need for manual oversight or intermediaries. For example, a smart contract might transfer funds when a certain deadline is reached or update a record -once a task is completed. This self-executing nature makes them efficient and minimizes human error.

Smart contracts operate deterministically, meaning they will always produce the same result given the same inputs. This predictability is crucial for ensuring reliability, -especially in complex systems. Additionally, they run in isolated environments on the blockchain, which enhances security by preventing unintended interactions with other processes.

Security is another critical feature of smart contracts. They leverage the underlying cryptographic protections of the blockchain, ensuring that every interaction +once a task is completed. This self-executing nature makes them efficient and minimizes human error.

+

Smart contracts operate deterministically, meaning they will always produce the same result given the same inputs. This predictability is crucial for ensuring reliability, +especially in complex systems. Additionally, they run in isolated environments on the blockchain, which enhances security by preventing unintended interactions with other processes.

+

Security is another critical feature of smart contracts. They leverage the underlying cryptographic protections of the blockchain, ensuring that every interaction is secure and authenticated. Before deployment, the contract code can be audited and verified to ensure it functions correctly. Once on the blockchain, -the immutable nature of the code prevents unauthorized modifications, further ensuring the integrity of the system.

Running smart contracts requires computational resources, which are compensated through gas fees. These fees vary depending on the complexity of the operations within the contract. -More resource-intensive contracts incur higher fees, which helps manage the computational load on the blockchain network.

The workflow of a smart contract begins with its development, where developers code the contract using languages like Rust. Once the code is compiled and deployed to the blockchain, +the immutable nature of the code prevents unauthorized modifications, further ensuring the integrity of the system.

+

Running smart contracts requires computational resources, which are compensated through gas fees. These fees vary depending on the complexity of the operations within the contract. +More resource-intensive contracts incur higher fees, which helps manage the computational load on the blockchain network.

+

The workflow of a smart contract begins with its development, where developers code the contract using languages like Rust. Once the code is compiled and deployed to the blockchain, it becomes a permanent part of the network. Users can then interact with the contract by sending transactions that invoke specific functions. The contract checks whether the -required conditions are met, and if so, it automatically executes the specified actions, such as transferring tokens or updating data on the blockchain.

The benefits of smart contracts are numerous. They eliminate the need for intermediaries by providing a system where trust is built into the code itself. +required conditions are met, and if so, it automatically executes the specified actions, such as transferring tokens or updating data on the blockchain.

+

The benefits of smart contracts are numerous. They eliminate the need for intermediaries by providing a system where trust is built into the code itself. This not only reduces costs but also increases efficiency by automating repetitive processes. The inherent security of smart contracts, combined with their transparency—where every action is recorded and visible on the blockchain—makes them a powerful tool for ensuring accountability and trust in decentralized systems. -They can be ideal for managing decentralized autonomous organizations (DAOs), where governance decisions are automated through coded rules.

By integrating smart contracts, NSSA offers a highly versatile, secure, and transparent framework that can support a wide range of applications -across various industries, from finance to governance, supply chains, and more.

2. General execution overview​

This section explains the execution process within NSSA, providing an overview of how it works from start to finish. -It outlines the steps involved in each execution type, guiding the reader through the entire process from user interaction to completion.

The process begins when a user initiates a transaction by invoking a smart contract. This invocation involves selecting at least one of +They can be ideal for managing decentralized autonomous organizations (DAOs), where governance decisions are automated through coded rules.

+

By integrating smart contracts, NSSA offers a highly versatile, secure, and transparent framework that can support a wide range of applications +across various industries, from finance to governance, supply chains, and more.

+

2. General execution overview​

+
+

This section explains the execution process within NSSA, providing an overview of how it works from start to finish. +It outlines the steps involved in each execution type, guiding the reader through the entire process from user interaction to completion.

+

The process begins when a user initiates a transaction by invoking a smart contract. This invocation involves selecting at least one of the four execution types: public, private, shielded, or deshielded. The choice of execution type determines how data will be read from and written to the blockchain, affecting the transaction's privacy and security levels. Each execution type caters to different privacy needs, allowing the user to tailor the transaction according -to their specific requirements, whether it be full transparency or complete confidentiality.

general

User actions​

Step 1: Smart contract selection and input creation

  • Smart contract selection: The user selects a smart contract they wish to invoke.
  • Input creation: The user creates a set of inputs required for the invocation by reading the necessary data from both the public and private states. This includes:
    • Public data such as current account balances, public keys, and smart contract states.
    • Private data such as private account balances and UTXOs.

Step 2: Choosing execution type

  • Execution type selection: The user selects the type of execution based on their privacy needs. The options include:
    • Public execution: Suitable for transactions where transparency is desired.
    • Private execution: Used when transaction details need to be confidential.
    • Shielded execution: Hides the receiver's identity.
    • Deshielded execution: Hides the sender's identity.
  • ZkVM requirement: If the execution involves private, shielded, or deshielded types, the user must call the zkVM to handle these confidential transactions. -For purely public executions, the zkVM is not needed, and the user can directly transmit the transaction code to the sequencer.

Step 3: Calling zkVM for proof generation

  • ZkVM compilation: The user calls the zkVM to compile the smart contract with both public and private inputs.
    • Kernel circuit proofs: The zkVM generates individual proofs for each execution type through kernel circuits.
    • Proof aggregation: The zkVM aggregates these individual proofs into a single comprehensive proof, combining both private and public inputs.

Step 4: Transmitting public inputs and retaining private inputs

  • Retaining private inputs: The user keeps the private inputs secure and does not transmit them.
  • Revealing public inputs: The user transmits the following public inputs to the sequencer:
    • Public inputs of the recursive proof
    • Hashes of UTXOs
    • Updates to the public state
    • Transaction signature
    • Nullifiers (to prevent double spending)

After completing these steps, the user's part of the execution is done, and the sequencer takes over the process.

Sequencer actions​

Step 5: Proof verification

  • Proof and data reception: The sequencer receives the proof and public inputs from the user.
  • Verification process:
    • For private, shielded, and deshielded executions, the sequencer verifies the proof using the provided public data.
    • For public executions, the sequencer reruns the smart contract code with the provided inputs to check the results.
  • Validation: If both the zkVM proofs and public execution results are verified successfully, the sequencer collects the proof and public data to proceed. -If verification fails, the process is aborted, and the transaction is rejected.

Step 6: Aggregating proofs and finalizing the block

  • Proof aggregation: The sequencer calls the zkVM again to aggregate all received proofs into one comprehensive proof to finalize the block.
  • Finalizing the block:
    • Public state update: The sequencer updates the public state with the new transaction data.
    • Nullifier tree update: Updates the nullifier tree to reflect the new state and prevent double spending.
    • Synchronization mechanism: Runs synchronization mechanisms to ensure fairness and consistency across the network.
    • UTXO validation: Validates the exchanged UTXOs to complete the transaction process.

This comprehensive process ensures that transactions are executed securely, with the appropriate level of privacy and state updates synchronized across the network.

Below, we outline the execution process of the four different execution types within NSSA:

  • Public execution:

public

  • Private execution:

private

  • Shielded execution:

shielded

  • Deshielded execution:

deshielded

3. Execution processes and UTXO management​

In Nescience state-separation architecture, UTXOs are key components for managing private data and assets. They serve as private entities that hold both storage and assets, +to their specific requirements, whether it be full transparency or complete confidentiality.

+

general

+

User actions​

+
+

Step 1: Smart contract selection and input creation

+
    +
  • Smart contract selection: The user selects a smart contract they wish to invoke.
    • +
  • Input creation: The user creates a set of inputs required for the invocation by reading the necessary data from both the public and private states. This includes: +
      +
    • Public data such as current account balances, public keys, and smart contract states.
      • +
    • Private data such as private account balances and UTXOs.
      • +
    +
    • +
+

Step 2: Choosing execution type

+
    +
  • Execution type selection: The user selects the type of execution based on their privacy needs. The options include: +
      +
    • Public execution: Suitable for transactions where transparency is desired.
      • +
    • Private execution: Used when transaction details need to be confidential.
      • +
    • Shielded execution: Hides the receiver's identity.
      • +
    • Deshielded execution: Hides the sender's identity.
      • +
    +
    • +
  • ZkVM requirement: If the execution involves private, shielded, or deshielded types, the user must call the zkVM to handle these confidential transactions. +For purely public executions, the zkVM is not needed, and the user can directly transmit the transaction code to the sequencer.
    • +
+

Step 3: Calling zkVM for proof generation

+
    +
  • ZkVM compilation: The user calls the zkVM to compile the smart contract with both public and private inputs. +
      +
    • Kernel circuit proofs: The zkVM generates individual proofs for each execution type through kernel circuits.
      • +
    • Proof aggregation: The zkVM aggregates these individual proofs into a single comprehensive proof, combining both private and public inputs.
      • +
    +
    • +
+

Step 4: Transmitting public inputs and retaining private inputs

+
    +
  • Retaining private inputs: The user keeps the private inputs secure and does not transmit them.
    • +
  • Revealing public inputs: The user transmits the following public inputs to the sequencer: +
      +
    • Public inputs of the recursive proof
      • +
    • Hashes of UTXOs
      • +
    • Updates to the public state
      • +
    • Transaction signature
      • +
    • Nullifiers (to prevent double spending)
      • +
    +
    • +
+

After completing these steps, the user's part of the execution is done, and the sequencer takes over the process.

+

Sequencer actions​

+
+

Step 5: Proof verification

+
    +
  • Proof and data reception: The sequencer receives the proof and public inputs from the user.
    • +
  • Verification process: +
      +
    • For private, shielded, and deshielded executions, the sequencer verifies the proof using the provided public data.
      • +
    • For public executions, the sequencer reruns the smart contract code with the provided inputs to check the results.
      • +
    +
    • +
  • Validation: If both the zkVM proofs and public execution results are verified successfully, the sequencer collects the proof and public data to proceed. +If verification fails, the process is aborted, and the transaction is rejected.
    • +
+

Step 6: Aggregating proofs and finalizing the block

+
    +
  • Proof aggregation: The sequencer calls the zkVM again to aggregate all received proofs into one comprehensive proof to finalize the block.
    • +
  • Finalizing the block: +
      +
    • Public state update: The sequencer updates the public state with the new transaction data.
      • +
    • Nullifier tree update: Updates the nullifier tree to reflect the new state and prevent double spending.
      • +
    • Synchronization mechanism: Runs synchronization mechanisms to ensure fairness and consistency across the network.
      • +
    • UTXO validation: Validates the exchanged UTXOs to complete the transaction process.
      • +
    +
    • +
+

This comprehensive process ensures that transactions are executed securely, with the appropriate level of privacy and state updates synchronized across the network.

+

Below, we outline the execution process of the four different execution types within NSSA:

+
    +
  • Public execution:
    • +
+

public

+
    +
  • Private execution:
    • +
+

private

+
    +
  • Shielded execution:
    • +
+

shielded

+
    +
  • Deshielded execution:
    • +
+

deshielded

+

3. Execution processes and UTXO management​

+
+

In Nescience state-separation architecture, UTXOs are key components for managing private data and assets. They serve as private entities that hold both storage and assets, facilitating secure and confidential transactions. UTXOs are utilized in three of the four execution types within NSSA: private, shielded, -and deshielded executions. This section explores the lifecycle of UTXOs, detailing their generation, transfer, encryption, and eventual consumption within the private execution framework.

a) Components of a Nescience UTXO​

A Nescience UTXO is a critical and versatile component of the private state in the Nescience state-separation architecture. +and deshielded executions. This section explores the lifecycle of UTXOs, detailing their generation, transfer, encryption, and eventual consumption within the private execution framework.

+

a) Components of a Nescience UTXO​

+
+

A Nescience UTXO is a critical and versatile component of the private state in the Nescience state-separation architecture. It carries essential information that ensures its proper functionality within private execution, such as the owner, value, private storage slot, non-fungibles, -and other cryptographic components. Below is a detailed breakdown of each component and its role in maintaining the integrity, security, and privacy of the system:

  • Owner: +and other cryptographic components. Below is a detailed breakdown of each component and its role in maintaining the integrity, security, and privacy of the system:

    +
      +
    • +

      Owner: The owner component represents the public key of the entity that controls the UTXO. Only the owner can spend this UTXO, ensuring its security and privacy through public key cryptography. This means that the UTXO remains secure as only the rightful owner, using their private key, can generate valid signatures to authorize the transaction. For example, if Alice owns a UTXO linked to her public key, she must sign any transaction to spend it using her private key. This cryptographic protection ensures that only Alice can authorize -spending the UTXO and transfer it to someone else, such as Bob.

    • Value: +spending the UTXO and transfer it to someone else, such as Bob.

      +
      • +
    • +

      Value: The value in a UTXO represents the balance or asset contained within it. This could be cryptocurrency, tokens, or other digital assets. The value ensures accurate accounting, preventing double spending and maintaining the overall integrity of the system. For instance, if Alice's UTXO has a value of 10 tokens, this represents her ownership of that amount -within the network, and when spent, this value will be deducted from her UTXO and transferred accordingly.

    • Private storage slot: +within the network, and when spent, this value will be deducted from her UTXO and transferred accordingly.

      +
      • +
    • +

      Private storage slot: The private storage slot is an arbitrary and flexible storage space within the UTXO for Nescience applications. It allows users and smart contracts to store additional private data that is only accessible by the owner. This could be used to hold metadata, smart contract states, or user-specific information. For example, if a smart contract is holding private user data, -this information is securely stored in the private storage slot and can only be accessed or modified by the owner, ensuring privacy and security.

    • Non-fungibles: +this information is securely stored in the private storage slot and can only be accessed or modified by the owner, ensuring privacy and security.

      +
      • +
    • +

      Non-fungibles: Non-fungibles within the UTXO represent unique assets, such as NFTs (Non-Fungible Tokens). Each non-fungible asset is assigned a unique serial number or identifier within the UTXO, ensuring its distinctiveness and traceability. For example, if Alice owns a digital artwork represented as an NFT, the non-fungible component of the UTXO will store the unique identifier -for this NFT, preventing duplication or forgery of the digital asset.

    • Random commitment key: +for this NFT, preventing duplication or forgery of the digital asset.

      +
      • +
    • +

      Random commitment key: The random commitment key (RCK) is a randomly generated number used to create a cryptographic commitment to the contents of the UTXO. This commitment ensures the integrity of the data without revealing any private information. By generating a random key for the commitment, the system ensures that even if someone observes the commitment, they cannot infer any details -about the underlying UTXO. For example, RCK helps maintain confidentiality in the system while still allowing the verification of transactions.

    • Nullifier key: +about the underlying UTXO. For example, RCK helps maintain confidentiality in the system while still allowing the verification of transactions.

      +
      • +
    • +

      Nullifier key: The Nullifier key is another randomly generated number, used to ensure that a UTXO is only spent once. When a UTXO is spent, its nullifier key is recorded in a nullifier set to prevent double spending. This key guarantees that once a UTXO is spent, it cannot be reused in another transaction, effectively nullifying it from future use. This mechanism is crucial for -maintaining the security and integrity of the system, as it ensures that no UTXO can be spent more than once.

    b) UTXO lifecycle: From generation to consumption​

    UTXOs in NSSA are created when a transaction outputs a specific value, asset, or data intended for future use. Once generated, these UTXOs become private entities -owned by specific users, containing sensitive information such as balances, private data, or unique assets like NFTs.

    To maintain the required level of confidentiality, UTXOs are encrypted and transferred anonymously across the network. This encryption process ensures that the data within each UTXO +maintaining the security and integrity of the system, as it ensures that no UTXO can be spent more than once.

    +
    • +
+

b) UTXO lifecycle: From generation to consumption​

+
+

UTXOs in NSSA are created when a transaction outputs a specific value, asset, or data intended for future use. Once generated, these UTXOs become private entities +owned by specific users, containing sensitive information such as balances, private data, or unique assets like NFTs.

+

To maintain the required level of confidentiality, UTXOs are encrypted and transferred anonymously across the network. This encryption process ensures that the data within each UTXO remains hidden from network participants, including the sequencer, while still allowing for verification and validation through ZKPs. These proofs enable the network -to ensure that UTXOs are valid, prevent double spending, and maintain security, all without revealing any sensitive information.

When a user wishes to spend or transfer a UTXO, the lifecycle progresses towards its consumption. The user must prove ownership and validity of the UTXO through a ZKP, +to ensure that UTXOs are valid, prevent double spending, and maintain security, all without revealing any sensitive information.

+

When a user wishes to spend or transfer a UTXO, the lifecycle progresses towards its consumption. The user must prove ownership and validity of the UTXO through a ZKP, which is then verified by the sequencer. This process occurs in private, shielded, and deshielded executions, where confidentiality is a priority. Once the proof is validated, -the UTXO is consumed, meaning it is marked as spent and cannot be reused, ensuring the integrity of the transaction and preventing double spending.

UTXOs are central to the private, shielded, and deshielded execution types in Nescience. In private executions, UTXOs are transferred securely between parties without revealing any +the UTXO is consumed, meaning it is marked as spent and cannot be reused, ensuring the integrity of the transaction and preventing double spending.

+

UTXOs are central to the private, shielded, and deshielded execution types in Nescience. In private executions, UTXOs are transferred securely between parties without revealing any details to the public state. In shielded executions, UTXOs are used to receive assets from the public state while keeping the recipient's identity confidential. Finally, -in deshielded executions, UTXOs are used to send assets from the private state to the public state, while preserving the sender's anonymity.

Since UTXOs are not exchanged in public executions, this lifecycle analysis is focused solely on private, shielded, and deshielded executions, where privacy and confidentiality are essential. +in deshielded executions, UTXOs are used to send assets from the private state to the public state, while preserving the sender's anonymity.

+

Since UTXOs are not exchanged in public executions, this lifecycle analysis is focused solely on private, shielded, and deshielded executions, where privacy and confidentiality are essential. In these contexts, the careful management and transfer of UTXOs ensure that the users' private data and assets remain secure, while still allowing for seamless and confidential transactions -within the network.

At this point, it's crucial to introduce two key components that will play a significant role in the next section: the ephemeral key and the nillifier.

  • Ephemeral key: The ephemeral key is embedded in the transaction message and plays a crucial role in maintaining privacy. It is used by the sender, alongside the receiver's public key, +within the network.

    +

    At this point, it's crucial to introduce two key components that will play a significant role in the next section: the ephemeral key and the nillifier.

    +
      +
    • +

      Ephemeral key: The ephemeral key is embedded in the transaction message and plays a crucial role in maintaining privacy. It is used by the sender, alongside the receiver's public key, in a key agreement protocol to derive a shared secret. This shared secret is then employed to encrypt the transaction details, ensuring that only those with the receiver's viewing key can decrypt the transaction. By using the ephemeral key, the receiver can regenerate the shared secret, granting access to the transaction's contents. The sender generates the ephemeral key -using their spending key and the UTXO's nullifier, reinforcing the security of the transaction. (more details in key management and addresses section)

    • Nullifier: A nullifier is a unique value tied to a specific UTXO, ensuring that it has not been spent before. Its uniqueness is essential, as a nullifier must never correspond to more +using their spending key and the UTXO's nullifier, reinforcing the security of the transaction. (more details in key management and addresses section)

      +
      • +
    • +

      Nullifier: A nullifier is a unique value tied to a specific UTXO, ensuring that it has not been spent before. Its uniqueness is essential, as a nullifier must never correspond to more than one UTXO—otherwise, even if both UTXOs are valid, only one could be spent. This would undermine the integrity of the system. To spend a UTXO, a proof must be provided showing that the nullifier does not already exist in the Nullifier Tree. Once the transaction is confirmed and included in the blockchain, the nullifier is added to the Nullifier Tree, preventing any future reuse of the same UTXO. A UTXO's nullifier is generated by combining the receiver's nullifier key with the transaction note's commitment, further ensuring its distinctiveness -and security. (More details in nullifier tree section.)

    I) UTXOs in private executions​

    In private executions within NSSA, transactions are handled ensuring maximum privacy by concealing all transaction details from the public state. +and security. (More details in nullifier tree section.)

    +
    • +
+

I) UTXOs in private executions​

+
+

In private executions within NSSA, transactions are handled ensuring maximum privacy by concealing all transaction details from the public state. This approach is particularly useful for confidential payments, where the identities of the sender and receiver, as well as the transaction amounts, must remain hidden. -The process is powered by ZKPs, ensuring that only the involved parties have access to the transaction details while maintaining the integrity of the network.

  • Stages of private execution: Private executions operate in two key stages: UTXO consumption and UTXO creation. In the first stage, UTXOs from the private state are used +The process is powered by ZKPs, ensuring that only the involved parties have access to the transaction details while maintaining the integrity of the network.

    +
      +
    • +

      Stages of private execution: Private executions operate in two key stages: UTXO consumption and UTXO creation. In the first stage, UTXOs from the private state are used as inputs for the transaction. In the second stage, new UTXOs are generated as outputs and stored back in the private state. Throughout this process, the details of the -transaction are kept confidential and only shared between the sender and receiver.

    • Private transaction workflow (transaction initialization): The user initiates a private transaction by selecting the input UTXOs that will be spent and determining the +transaction are kept confidential and only shared between the sender and receiver.

      +
      • +
    • +

      Private transaction workflow (transaction initialization): The user initiates a private transaction by selecting the input UTXOs that will be spent and determining the output UTXOs to be created. This involves specifying the amounts to be transferred and the recipient’s private address (a divestified address that hides the recipient's public address from the network). The nullifier key and random number for commitments (RCK) are also generated at this stage to define how these UTXOs can be spent or nullified in the -future by the receiver.

    • Proof generation and verification: Next, the zkVM generates a ZKP to validate the transaction. This proof includes both a membership proof for the input UTXOs, +future by the receiver.

      +
      • +
    • +

      Proof generation and verification: Next, the zkVM generates a ZKP to validate the transaction. This proof includes both a membership proof for the input UTXOs, confirming their presence in the hashed UTXO tree, and a non-membership proof to ensure that the input UTXOs have not already been spent (i.e., they are not in the nullifier tree). -The proof also confirms that the total input value matches the total output value, ensuring no discrepancies. The user then submits the proof, along with the necessary metadata, to the sequencer.

    • Shared secret and encryption: To maintain confidentiality, the sender uses the receiver’s divestified address to generate an ephemeral public key. +The proof also confirms that the total input value matches the total output value, ensuring no discrepancies. The user then submits the proof, along with the necessary metadata, to the sequencer.

      +
      • +
    • +

      Shared secret and encryption: To maintain confidentiality, the sender uses the receiver’s divestified address to generate an ephemeral public key. This allows the creation of a shared secret between the sender and receiver. Using a key derivation function, a symmetric encryption key is generated from the shared secret. -The input and output UTXOs are then encrypted using this symmetric key, ensuring that only the intended recipient can decrypt the data.

    • Broadcasting the transaction: The user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. -These committed UTXOs are sent to the sequencer, which updates the hashed UTXO tree without knowing the transaction details.

    • Decryption by the receiver: After the broadcast, the receiver attempts to decrypt the broadcast UTXOs using their symmetric key, derived from the ephemeral public key. +The input and output UTXOs are then encrypted using this symmetric key, ensuring that only the intended recipient can decrypt the data.

      +
      • +
    • +

      Broadcasting the transaction: The user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. +These committed UTXOs are sent to the sequencer, which updates the hashed UTXO tree without knowing the transaction details.

      +
      • +
    • +

      Decryption by the receiver: After the broadcast, the receiver attempts to decrypt the broadcast UTXOs using their symmetric key, derived from the ephemeral public key. If the receiver successfully decrypts a UTXO, it confirms ownership of that UTXO. The receiver then computes the nullifier for the UTXO and verifies its presence in the hashed -UTXO tree and its absence from the nullifier tree, ensuring it has not been spent. Finally, the new UTXO is added to the receiver’s locally stored UTXO tree for future transactions.

    Throughout the private execution process, the identities of both the sender and receiver, as well as all transaction details, remain hidden from the public. +UTXO tree and its absence from the nullifier tree, ensuring it has not been spent. Finally, the new UTXO is added to the receiver’s locally stored UTXO tree for future transactions.

    +
    • +
+

Throughout the private execution process, the identities of both the sender and receiver, as well as all transaction details, remain hidden from the public. The use of ZKPs ensures that the integrity of the transaction is verified without revealing any sensitive information. At the end of the process, -the network guarantees that no participant, aside from the sender and receiver, can deduce any details about the transaction or the involved parties.

II) UTXOs in shielded executions​

In shielded executions, the interaction between public and private states provides a hybrid privacy model that balances transparency and confidentiality. +the network guarantees that no participant, aside from the sender and receiver, can deduce any details about the transaction or the involved parties.

+

II) UTXOs in shielded executions​

+
+

In shielded executions, the interaction between public and private states provides a hybrid privacy model that balances transparency and confidentiality. This model is suitable for scenarios where the initial step, such as a public transaction, requires visibility, while subsequent actions, such as private asset management, need to remain confidential. One common use case is asset conversion—where a public token is converted into a private token. The conversion is visible on the public ledger, -but subsequent transactions remain private.

a) How shielded executions work​

Shielded executions operate in two distinct stages: first, there is a modification of the public state, and then new UTXOs are created and stored in the private state. +but subsequent transactions remain private.

+
a) How shielded executions work​
+

Shielded executions operate in two distinct stages: first, there is a modification of the public state, and then new UTXOs are created and stored in the private state. Importantly, shielded executions do not consume UTXOs but instead mint them, as new UTXOs are created to reflect the changes in the private state. This structure demands ZKPs to ensure that the newly minted UTXOs are consistent with the modifications in the public state. Here’s a step-by-step breakdown of how the shielded -execution process unfolds:

  1. Transaction initiation: The user initiates a transaction that modifies the public state, such as converting a public token to a private token. -The transaction alters the public state (e.g., balances or smart contract storage) while simultaneously preparing to mint new UTXOs in the private state.

  2. Generating UTXOs: After modifying the public state, the system mints new UTXOs in the private state. These UTXOs must be securely created, ensuring their integrity -and consistency with the initial public state modification. A ZKP is generated by the user to prove that these new UTXOs align with the changes made in the public state.

  3. Key setup for privacy: The sender retrieves the receiver's address and uses it to create a shared secret through an ephemeral public key. This shared secret is then used -to derive a symmetric key, which encrypts the output UTXOs. This encryption ensures that only the intended receiver can decrypt and access the UTXOs.

  4. Broadcasting and verifying UTXOs: After encrypting the UTXOs, the sender broadcasts them to the network. The new hashed UTXOs are sent to the sequencer, +execution process unfolds:

    +
      +
    1. +

      Transaction initiation: The user initiates a transaction that modifies the public state, such as converting a public token to a private token. +The transaction alters the public state (e.g., balances or smart contract storage) while simultaneously preparing to mint new UTXOs in the private state.

      +
      2. +
    2. +

      Generating UTXOs: After modifying the public state, the system mints new UTXOs in the private state. These UTXOs must be securely created, ensuring their integrity +and consistency with the initial public state modification. A ZKP is generated by the user to prove that these new UTXOs align with the changes made in the public state.

      +
      3. +
    3. +

      Key setup for privacy: The sender retrieves the receiver's address and uses it to create a shared secret through an ephemeral public key. This shared secret is then used +to derive a symmetric key, which encrypts the output UTXOs. This encryption ensures that only the intended receiver can decrypt and access the UTXOs.

      +
      4. +
    4. +

      Broadcasting and verifying UTXOs: After encrypting the UTXOs, the sender broadcasts them to the network. The new hashed UTXOs are sent to the sequencer, which verifies the validity of the UTXOs and attaches them to the hashed UTXO tree within the private state. The public inputs for the ZKP circuits consist of the -Pedersen-hashed UTXOs and the modifications in the public state.

    5. Receiver's role: Once the UTXOs are broadcast, the receiver attempts to decrypt each UTXO using the symmetric key derived from the shared secret. If the decryption is successful, -the UTXO belongs to the receiver. The receiver then verifies the UTXO’s validity by checking its inclusion in the hashed UTXO tree and ensuring that its nullifier has not yet been used.

    6. Nullifier check and integration: To prevent double spending, the receiver computes the nullifier for the received UTXO and verifies that it is not already present in the nullifier tree. -Once verified, the receiver adds the UTXO to their locally stored UTXO tree for future use in private transactions.

    While shielded executions offer privacy, certain information is still exposed to the public state, such as the sender's identity. To further enhance privacy, +Pedersen-hashed UTXOs and the modifications in the public state.

    +
    5. +
  5. +

    Receiver's role: Once the UTXOs are broadcast, the receiver attempts to decrypt each UTXO using the symmetric key derived from the shared secret. If the decryption is successful, +the UTXO belongs to the receiver. The receiver then verifies the UTXO’s validity by checking its inclusion in the hashed UTXO tree and ensuring that its nullifier has not yet been used.

    +
    6. +
  6. +

    Nullifier check and integration: To prevent double spending, the receiver computes the nullifier for the received UTXO and verifies that it is not already present in the nullifier tree. +Once verified, the receiver adds the UTXO to their locally stored UTXO tree for future use in private transactions.

    +
    7. +
+

While shielded executions offer privacy, certain information is still exposed to the public state, such as the sender's identity. To further enhance privacy, the sender can create empty UTXOs—UTXOs that don’t belong to anyone but are included in the transaction to obfuscate the true details of the transaction. -Though this approach increases the size of the data, it adds a layer of privacy by complicating the identification of meaningful transactions.

b) Summary of shielded execution flow​
  • Stage 1 (public modification): The user modifies public state data, such as converting tokens from public to private. This stage is visible to the public.
  • Stage 2 (UTXO minting and privacy): New UTXOs are minted in the private state, encrypted, and broadcast to the network. The transaction remains private from this point forward, -secured by ZKPs and cryptographic keys.
  • Receiver’s role: The receiver decrypts the UTXOs and verifies their validity, ensuring the UTXOs are not double spent and are ready for future transactions.

In summary, shielded executions enable a hybrid privacy model in Nescience, balancing public transparency and private confidentiality. They are well-suited for -transactions requiring initial public visibility, such as asset conversions, while ensuring that subsequent actions remain secure and private within the network.

III) UTXOs in deshielded executions​

In NSSA, deshielded executions offer a unique way to move data and assets from the private state to the public state, revealing previously private +Though this approach increases the size of the data, it adds a layer of privacy by complicating the identification of meaningful transactions.

+
b) Summary of shielded execution flow​
+
    +
  • Stage 1 (public modification): The user modifies public state data, such as converting tokens from public to private. This stage is visible to the public.
    • +
  • Stage 2 (UTXO minting and privacy): New UTXOs are minted in the private state, encrypted, and broadcast to the network. The transaction remains private from this point forward, +secured by ZKPs and cryptographic keys.
    • +
  • Receiver’s role: The receiver decrypts the UTXOs and verifies their validity, ensuring the UTXOs are not double spent and are ready for future transactions.
    • +
+

In summary, shielded executions enable a hybrid privacy model in Nescience, balancing public transparency and private confidentiality. They are well-suited for +transactions requiring initial public visibility, such as asset conversions, while ensuring that subsequent actions remain secure and private within the network.

+

III) UTXOs in deshielded executions​

+
+

In NSSA, deshielded executions offer a unique way to move data and assets from the private state to the public state, revealing previously private information in a controlled and verifiable manner. This type of execution allows for selective disclosure, ensuring transparency when needed while still maintaining the security and privacy of critical details through cryptographic techniques like ZKPs. Deshielded executions are particularly valuable for use cases -such as regulatory compliance reporting, where specific transaction details must be revealed to meet legal requirements, while other sensitive transactions remain private.

a) Stages of deshielded executions​

  • Stage 1 (UTXO consumption): The process begins in the private state, where UTXOs are consumed as inputs for the transaction. This involves gathering all necessary -UTXOs that contain the assets or balances to be made public, as well as any associated private data stored in memory slots.

  • Stage 2 (public state modification): After the UTXOs are consumed, the transaction details are made public by modifying the public state. This update includes changes +such as regulatory compliance reporting, where specific transaction details must be revealed to meet legal requirements, while other sensitive transactions remain private.

    +
    a) Stages of deshielded executions​
    +
      +
    • +

      Stage 1 (UTXO consumption): The process begins in the private state, where UTXOs are consumed as inputs for the transaction. This involves gathering all necessary +UTXOs that contain the assets or balances to be made public, as well as any associated private data stored in memory slots.

      +
      • +
    • +

      Stage 2 (public state modification): After the UTXOs are consumed, the transaction details are made public by modifying the public state. This update includes changes to the public balances, storage data, and any necessary public records. While the public state is updated, the sender’s identity and other sensitive information remain hidden, -thanks to the privacy-preserving properties of ZKPs.

    This model ensures that private data can be selectively revealed when needed, offering both flexibility and transparency. It is particularly useful for scenarios requiring -auditing or compliance reporting, where specific details must be made publicly verifiable without exposing the entire history or contents of private transactions.

    b) How deshielded executions work​

    The deshielded execution process starts when a user initiates a transaction using private UTXOs. The Nescience zkVM is called to generate a ZKP, -which validates the transaction without revealing sensitive details such as the sender's identity or the specifics of the Nescience application being executed.

    During the transaction, the UTXOs from the private state are consumed, meaning they are used up as inputs and will no longer be available for future transactions. +thanks to the privacy-preserving properties of ZKPs.

    +
    • +
+

This model ensures that private data can be selectively revealed when needed, offering both flexibility and transparency. It is particularly useful for scenarios requiring +auditing or compliance reporting, where specific details must be made publicly verifiable without exposing the entire history or contents of private transactions.

+
b) How deshielded executions work​
+

The deshielded execution process starts when a user initiates a transaction using private UTXOs. The Nescience zkVM is called to generate a ZKP, +which validates the transaction without revealing sensitive details such as the sender's identity or the specifics of the Nescience application being executed.

+

During the transaction, the UTXOs from the private state are consumed, meaning they are used up as inputs and will no longer be available for future transactions. Instead of generating new UTXOs, the transaction modifies the public state, updating the necessary balances or memory slots related to the transaction. -Here’s a step-by-step breakdown of how the deshielded execution process unfolds:

  1. Get receiver's public address: The sender first identifies the public address of the receiver, to which the information or assets will be made public.

  2. Determine input UTXOs and public state modifications: The sender gathers all the input UTXOs needed for the transaction and determines the public state modifications -necessary for the Nescience applications and token transfers involved.

  3. Calculate nullifiers: Nullifiers are generated for each input UTXO, ensuring that these UTXOs cannot be reused or double spent. The nullifiers are derived from the -corresponding UTXO commitments.

  4. Call zkVM with deshielded circuits: The sender invokes the zkVM with deshielded kernel circuits, which generates the proof. The proof ensures that all input UTXOs -are valid by verifying their membership in the UTXO tree and their non-membership in the nullifier tree, ensuring they haven’t been spent.

  5. Generate and submit proof: The zkVM generates a ZKP that verifies the correctness of the transaction without revealing private details. -The proof includes the nullifiers and the planned modifications to the public state.

  6. Send proof to sequencer: The sender then sends the proof and any relevant public information to the sequencer. The sequencer is responsible for verifying the proof, -updating the public state accordingly, and adding the nullifiers to the nullifier tree.

Once the proof and public information have been broadcast to the network, the receiver does not need to take any further action. +Here’s a step-by-step breakdown of how the deshielded execution process unfolds:

+
    +
  1. +

    Get receiver's public address: The sender first identifies the public address of the receiver, to which the information or assets will be made public.

    +
    2. +
  2. +

    Determine input UTXOs and public state modifications: The sender gathers all the input UTXOs needed for the transaction and determines the public state modifications +necessary for the Nescience applications and token transfers involved.

    +
    3. +
  3. +

    Calculate nullifiers: Nullifiers are generated for each input UTXO, ensuring that these UTXOs cannot be reused or double spent. The nullifiers are derived from the +corresponding UTXO commitments.

    +
    4. +
  4. +

    Call zkVM with deshielded circuits: The sender invokes the zkVM with deshielded kernel circuits, which generates the proof. The proof ensures that all input UTXOs +are valid by verifying their membership in the UTXO tree and their non-membership in the nullifier tree, ensuring they haven’t been spent.

    +
    5. +
  5. +

    Generate and submit proof: The zkVM generates a ZKP that verifies the correctness of the transaction without revealing private details. +The proof includes the nullifiers and the planned modifications to the public state.

    +
    6. +
  6. +

    Send proof to sequencer: The sender then sends the proof and any relevant public information to the sequencer. The sequencer is responsible for verifying the proof, +updating the public state accordingly, and adding the nullifiers to the nullifier tree.

    +
    7. +
+

Once the proof and public information have been broadcast to the network, the receiver does not need to take any further action. The sequencer manages the public state updates and ensures that the transaction is properly executed. By the end of the deshielded execution, specific transaction details become publicly visible, such as the identity of the receiver and the outcome of the transaction. This allows participants in the public state to extract information about the transaction, including the receiver's identity and some details about the execution. While the receiver's identity is revealed, the sender's identity and sensitive transaction details remain hidden, thanks to the use of ZKPs. -This makes deshielded executions ideal for cases where transparency is needed, but complete privacy is still a priority for certain elements of the transaction.

Summary of UTXO consumption in NSSA​

In NSSA, consuming UTXOs is a critical step in maintaining the security and integrity of the blockchain by preventing double spending. -When a UTXO is consumed, it is used as an input in a transaction, effectively marking it as spent. This ensures that the UTXO cannot be reused, preserving the integrity of the blockchain.

  1. The process of consuming UTXOs: The process of consuming a UTXO begins when a user selects a UTXO from their private state. The user verifies the UTXO’s existence and -ownership using their viewing key, ensuring that they are the legitimate owner of the UTXO. Once verified, the user generates two key cryptographic proofs:
    * **Membership proof:** This proof confirms that the UTXO exists within the hashed UTXO tree, ensuring its validity within the system.
    * **Non-membership proof:** This proof ensures that the UTXO has not been previously consumed by checking its absence in the nullifier tree, which tracks spent UTXOs.

To mark the UTXO as spent, a nullifier is generated. This nullifier is a unique cryptographic hash derived from the UTXO, which is then added to the nullifier tree in the public state. -Adding the nullifier to the tree prevents the UTXO from being reused in future transactions, thus preventing double spending.

After generating the membership and non-membership proofs, the user compiles the transaction using the zkVM. The zkVM is responsible for generating the necessary ZKPs, -which validate the transaction without revealing sensitive details. The compiled transaction, along with the proofs, is then submitted to the sequencer for verification.

  1. The role of the sequencer: Once the transaction is submitted, the sequencer verifies the ZKPs to confirm that the transaction is valid. If the proofs are verified +This makes deshielded executions ideal for cases where transparency is needed, but complete privacy is still a priority for certain elements of the transaction.

    +

    Summary of UTXO consumption in NSSA​

    +
    +

    In NSSA, consuming UTXOs is a critical step in maintaining the security and integrity of the blockchain by preventing double spending. +When a UTXO is consumed, it is used as an input in a transaction, effectively marking it as spent. This ensures that the UTXO cannot be reused, preserving the integrity of the blockchain.

    +
      +
    1. The process of consuming UTXOs: The process of consuming a UTXO begins when a user selects a UTXO from their private state. The user verifies the UTXO’s existence and +ownership using their viewing key, ensuring that they are the legitimate owner of the UTXO. Once verified, the user generates two key cryptographic proofs: +
        +
      • Membership proof: This proof confirms that the UTXO exists within the hashed UTXO tree, ensuring its validity within the system.
        • +
      • Non-membership proof: This proof ensures that the UTXO has not been previously consumed by checking its absence in the nullifier tree, which tracks spent UTXOs.
        • +
      +
      2. +
    +

    To mark the UTXO as spent, a nullifier is generated. This nullifier is a unique cryptographic hash derived from the UTXO, which is then added to the nullifier tree in the public state. +Adding the nullifier to the tree prevents the UTXO from being reused in future transactions, thus preventing double spending.

    +

    After generating the membership and non-membership proofs, the user compiles the transaction using the zkVM. The zkVM is responsible for generating the necessary ZKPs, +which validate the transaction without revealing sensitive details. The compiled transaction, along with the proofs, is then submitted to the sequencer for verification.

    +
      +
    1. The role of the sequencer: Once the transaction is submitted, the sequencer verifies the ZKPs to confirm that the transaction is valid. If the proofs are verified successfully, the sequencer updates both the private and public states to reflect the transaction. This includes updating the nullifier tree with the newly generated nullifier, -ensuring that the UTXO is marked as spent and cannot be reused.

    Example: Alice sending tokens to Bob​

    Consider an example where Alice wants to send 5 Nescience tokens to Bob using a private execution. Alice selects a UTXO from her private state that contains 5 Nescience tokens. +ensuring that the UTXO is marked as spent and cannot be reused.

    2. +
+

Example: Alice sending tokens to Bob​

+
+

Consider an example where Alice wants to send 5 Nescience tokens to Bob using a private execution. Alice selects a UTXO from her private state that contains 5 Nescience tokens. She generates the necessary membership and non-membership proofs, ensuring that her UTXO exists in the system and has not been previously spent. Alice then creates a nullifier by -hashing the UTXO and compiles the transaction with the zkVM.

Once Alice submits the transaction, the sequencer verifies the proofs and updates the blockchain by adding the nullifier to the nullifier tree and recording the transaction details. -This ensures that Alice’s UTXO is marked as spent and cannot be used again, while Bob receives the 5 tokens.

The importance of nullifiers​

Nullifiers are a key mechanism in preventing double spending. By marking consumed UTXOs as spent and tracking them in the nullifier tree, NSSA ensures that +hashing the UTXO and compiles the transaction with the zkVM.

+

Once Alice submits the transaction, the sequencer verifies the proofs and updates the blockchain by adding the nullifier to the nullifier tree and recording the transaction details. +This ensures that Alice’s UTXO is marked as spent and cannot be used again, while Bob receives the 5 tokens.

+

The importance of nullifiers​

+
+

Nullifiers are a key mechanism in preventing double spending. By marking consumed UTXOs as spent and tracking them in the nullifier tree, NSSA ensures that once a UTXO is used in a transaction, it cannot be reused in any future transactions. This process is fundamental to maintaining the integrity and security of the blockchain, -as it guarantees that assets are only spent once and prevents potential attacks on the system.

In conclusion, the process of consuming UTXOs in NSSA combines cryptographic proofs, nullifiers, and ZKPs to ensure that transactions -are secure, confidential, and free from the risks of double spending.

C. Cryptographic primitives in NSSA

In the NSSA, cryptographic primitives are the foundational elements that ensure the security, privacy, and efficiency of the state separation model. +as it guarantees that assets are only spent once and prevents potential attacks on the system.

+

In conclusion, the process of consuming UTXOs in NSSA combines cryptographic proofs, nullifiers, and ZKPs to ensure that transactions +are secure, confidential, and free from the risks of double spending.

+

C. Cryptographic primitives in NSSA

+

In the NSSA, cryptographic primitives are the foundational elements that ensure the security, privacy, and efficiency of the state separation model. These cryptographic tools enable private transactions, secure data management, and robust verification processes across both public and private states. The architecture leverages a wide range of cryptographic mechanisms, including advanced hash functions, key management systems, tree structures, and ZKPs, -to safeguard user data and maintain the integrity of transactions.

Cryptographic hash functions play a pivotal role in concealing UTXO details, generating nullifiers, and constructing sparse Merkle trees, which organize and verify +to safeguard user data and maintain the integrity of transactions.

+

Cryptographic hash functions play a pivotal role in concealing UTXO details, generating nullifiers, and constructing sparse Merkle trees, which organize and verify data efficiently within the network. Key management and address generation further enhance the security of user assets and identity, ensuring that only authorized -users can access and control their holdings.

The architecture also relies on specialized tree structures for organizing data, verifying the existence of UTXOs, and tracking nullifiers, which prevent double spending. +users can access and control their holdings.

+

The architecture also relies on specialized tree structures for organizing data, verifying the existence of UTXOs, and tracking nullifiers, which prevent double spending. Additionally, Nescience features a privacy-preserving zero-knowledge virtual machine (zk-zkVM), which allows users to prove the correctness of an execution without -disclosing sensitive information. This enables private transactions and maintains confidentiality across the network.

As Nescience evolves, optional cryptographic mechanisms such as multi-party computation (MPC) may be integrated to enhance synchronization across privacy levels. +disclosing sensitive information. This enables private transactions and maintains confidentiality across the network.

+

As Nescience evolves, optional cryptographic mechanisms such as multi-party computation (MPC) may be integrated to enhance synchronization across privacy levels. This MPC-based synchronization mechanism is still under development and under review for potential inclusion in the system. Together, these cryptographic primitives -form the backbone of Nescience’s security architecture, ensuring that users can transact and interact privately, securely, and efficiently.

In the following sections, we will explore each of these cryptographic components in detail, beginning with the role of hash functions.

a) Hash functions in Nescience​

Hash functions are a foundational element of Nescience’s cryptographic framework, serving multiple critical roles that ensure the security, privacy, and efficiency of the system. +form the backbone of Nescience’s security architecture, ensuring that users can transact and interact privately, securely, and efficiently.

+

In the following sections, we will explore each of these cryptographic components in detail, beginning with the role of hash functions.

+

a) Hash functions in Nescience​

+
+

Hash functions are a foundational element of Nescience’s cryptographic framework, serving multiple critical roles that ensure the security, privacy, and efficiency of the system. One of the primary uses of hash functions in Nescience is to conceal sensitive details of UTXOs by converting them into fixed-size hashes. This process allows UTXO details to remain private, ensuring that sensitive information is not directly exposed on the blockchain, while still enabling their existence and integrity to be verified. Hashing -the UTXO details allows the actual data to remain confidential, with the hashes stored in a global tree structure for efficient management and retrieval.

Additionally, hash functions are essential for generating nullifiers, which play a crucial role in preventing double spending. Nullifiers are created by hashing UTXOs +the UTXO details allows the actual data to remain confidential, with the hashes stored in a global tree structure for efficient management and retrieval.

+

Additionally, hash functions are essential for generating nullifiers, which play a crucial role in preventing double spending. Nullifiers are created by hashing UTXOs and are used to mark them as spent, ensuring that they cannot be reused in subsequent transactions. These nullifiers are stored in a nullifier tree, and each transaction must prove that its UTXO’s nullifier is not already present in the tree before it can be processed. This ensures that the UTXO has not been spent before, maintaining the -integrity of the transaction process.

Hash functions are also vital in the construction of sparse Merkle trees, which provide an efficient and secure method for verifying data within the blockchain. +integrity of the transaction process.

+

Hash functions are also vital in the construction of sparse Merkle trees, which provide an efficient and secure method for verifying data within the blockchain. Sparse Merkle trees enable quick and reliable proofs of membership and non-membership, making them essential for verifying both UTXOs and nullifiers. By using hash functions -to build these trees, Nescience can ensure the integrity of the data, as any tampering with the data would result in a change in the hash, making the manipulation detectable.

Another critical consideration in Nescience is the compatibility of hash functions with ZKPs. ZK-friendly hash functions are optimized for efficient +to build these trees, Nescience can ensure the integrity of the data, as any tampering with the data would result in a change in the hash, making the manipulation detectable.

+

Another critical consideration in Nescience is the compatibility of hash functions with ZKPs. ZK-friendly hash functions are optimized for efficient computation within the constraints of ZK circuits, ensuring that they do not become a bottleneck in the proof generation or verification process. These hash functions maintain strong cryptographic security properties while enabling efficient computations in ZKP systems, which is essential for maintaining privacy and -integrity within the ZK framework.

The primary advantage of using hash functions in Nescience is their ability to ensure that transaction details remain private while still allowing for verification +integrity within the ZK framework.

+

The primary advantage of using hash functions in Nescience is their ability to ensure that transaction details remain private while still allowing for verification of their validity. Furthermore, by integrating hash functions into Merkle trees, the blockchain data becomes tamper-proof, enabling quick and efficient verification -processes that uphold the system’s security and privacy standards.

Use case: How to use the Pedersen hash to create the UTXO commitment​

As mentioned in the UTXOs in private executions section, the user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs +processes that uphold the system’s security and privacy standards.

+

Use case: How to use the Pedersen hash to create the UTXO commitment​

+
+

As mentioned in the UTXOs in private executions section, the user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. The Pedersen hash is used to create the UTXO commitment. The Pedersen hash is a homomorphic commitment scheme that allows secure commitments -while maintaining privacy and enabling proofs of correctness in transactions. The commitment formula is as follows:

Commitment=C(UTXO,RCK)=gUTXO⋅hRCKCommitment = C(UTXO,RCK) =g^{UTXO}⋅h^{RCK}

In this formula, gg and hh are two generators of a cryptographic group where no known relationship exists between them. This ensures that the commitment is secure -and computationally infeasible to reverse or manipulate without knowing the original UTXO components. The random number RCKRCK adds an additional layer of security -by blinding the UTXO's contents, ensuring that the commitment doesn't leak any information about the underlying data.

Importance of homomorphic commitments

It is essential to use a homomorphic commitment like the Pedersen commitment for UTXOs because it allows for the verification of important properties in transactions, +while maintaining privacy and enabling proofs of correctness in transactions. The commitment formula is as follows:

+

Commitment=C(UTXO,RCK)=gUTXO⋅hRCKCommitment = C(UTXO,RCK) =g^{UTXO}⋅h^{RCK}

+

In this formula, gg and hh are two generators of a cryptographic group where no known relationship exists between them. This ensures that the commitment is secure +and computationally infeasible to reverse or manipulate without knowing the original UTXO components. The random number RCKRCK adds an additional layer of security +by blinding the UTXO's contents, ensuring that the commitment doesn't leak any information about the underlying data.

+

Importance of homomorphic commitments

+

It is essential to use a homomorphic commitment like the Pedersen commitment for UTXOs because it allows for the verification of important properties in transactions, such as ensuring that the total input value of a transaction equals the total output value. This balance is crucial for preventing the unauthorized creation of funds or d discrepancies in transactions. A homomorphic commitment enables these proofs because of its additive properties. Specifically, the exponents in the commitment formula are additive, -meaning that commitments can be combined and verified without revealing the individual components. For instance, if you have two UTXOs with commitments C(UTXO1,RCK1)C(UTXO_1,RCK_1) -and C(UTXO2,RCK2)C(UTXO_2,RCK_2), you can combine them and verify that the resulting commitment is valid without exposing the actual amounts.

This capability is leveraged through a modified version of the Schnorr protocol, which is used in conjunction with the Pedersen hash to verify the correctness of transactions. -The Schnorr protocol allows users to prove, without revealing the actual values, that the sum of inputs equals the sum of outputs, ensuring that no funds are created or lost in the transaction.

Limitations of standard cryptographic hashes

Standard cryptographic hash functions, such as SHA-256, are not suitable for this purpose because they lack the algebraic structure needed for homomorphic properties. +meaning that commitments can be combined and verified without revealing the individual components. For instance, if you have two UTXOs with commitments C(UTXO1,RCK1)C(UTXO_1,RCK_1) +and C(UTXO2,RCK2)C(UTXO_2,RCK_2), you can combine them and verify that the resulting commitment is valid without exposing the actual amounts.

+

This capability is leveraged through a modified version of the Schnorr protocol, which is used in conjunction with the Pedersen hash to verify the correctness of transactions. +The Schnorr protocol allows users to prove, without revealing the actual values, that the sum of inputs equals the sum of outputs, ensuring that no funds are created or lost in the transaction.

+

Limitations of standard cryptographic hashes

+

Standard cryptographic hash functions, such as SHA-256, are not suitable for this purpose because they lack the algebraic structure needed for homomorphic properties. In particular, while SHA-256 provides strong security for general hashing purposes, it does not allow the additive properties that are required to perform the type of ZKPs used in Nescience for UTXO commitments. This is why the Pedersen hash is preferred, as it enables the secure and private execution of transactions -while allowing for balance verification and other critical proofs.

Conclusion

By using homomorphic commitments like the Pedersen hash, NSSA ensures that UTXOs can be securely committed and validated without exposing sensitive information. +while allowing for balance verification and other critical proofs.

+

Conclusion

+

By using homomorphic commitments like the Pedersen hash, NSSA ensures that UTXOs can be securely committed and validated without exposing sensitive information. The random component (RCK) adds an additional layer of security, and the additive properties of the Pedersen commitment enable powerful ZKPs that maintain the -integrity of the system.

b) Key management and addresses in Nescience​

NSSA utilizes different cryptographic schemes, such as public key encryption and digital signatures, to ensure secure private executions through +integrity of the system.

+

b) Key management and addresses in Nescience​

+
+

NSSA utilizes different cryptographic schemes, such as public key encryption and digital signatures, to ensure secure private executions through the exchange of UTXOs. These schemes rely on a structured set of cryptographic keys, each serving a specific purpose in maintaining privacy, security, and control over assets. -Here's a breakdown of the keys used in Nescience:

I. Spending key​

The spending key is the fundamental secret key in NSSA, acting as the primary control mechanism for a user’s UTXOs and other digital assets. -It plays a critical role in the cryptographic security of the system, ensuring that only the rightful owner can authorize and spend their assets.

  • Role of the spending key: The spending key is responsible for generating the user’s private keys, which are used in various cryptographic operations such as +Here's a breakdown of the keys used in Nescience:

    +

    I. Spending key​

    +

    The spending key is the fundamental secret key in NSSA, acting as the primary control mechanism for a user’s UTXOs and other digital assets. +It plays a critical role in the cryptographic security of the system, ensuring that only the rightful owner can authorize and spend their assets.

    +
      +
    • +

      Role of the spending key: The spending key is responsible for generating the user’s private keys, which are used in various cryptographic operations such as signing transactions and creating commitments. This hierarchical relationship means that the spending key sits at the root of a user’s key structure, safeguarding access to all associated private keys and, consequently, to the user’s assets. In Nescience’s privacy-focused model, the spending key is never exposed or shared outside the user’s control. Unlike other keys, it does not interact with the public state, kernel circuits, or even the ZKP system. This isolation ensures that the spending key remains completely private and inaccessible to external entities. By keeping the spending key separate from the operational aspects of the network, -Nescience minimizes the risk of key leakage or compromise.

    • Generation and security of the spending key: The spending key is generated randomly from the scalar field, a large mathematical space that ensures uniqueness +Nescience minimizes the risk of key leakage or compromise.

      +
      • +
    • +

      Generation and security of the spending key: The spending key is generated randomly from the scalar field, a large mathematical space that ensures uniqueness and cryptographic strength. This randomness is crucial because it prevents attackers from predicting or replicating the key, thereby safeguarding the user’s assets from unauthorized access: it is computationally infeasible for an attacker to guess or brute-force the key. Once the spending key is generated, it is securely stored -by the user, typically in a hardware wallet or another secure storage mechanism that prevents unauthorized access.

    • Spending UTXOs with the spending key: The spending key’s primary function is to authorize the spending of UTXOs in private transactions. When a user initiates +by the user, typically in a hardware wallet or another secure storage mechanism that prevents unauthorized access.

      +
      • +
    • +

      Spending UTXOs with the spending key: The spending key’s primary function is to authorize the spending of UTXOs in private transactions. When a user initiates a transaction, the spending key is used to generate the necessary cryptographic proofs and signatures, ensuring that the transaction is valid and originates from the rightful owner. However, even though the spending key generates these proofs, it is never directly exposed during the transaction process. Instead, derived private keys handle the operational aspects while the spending key remains secure in the background. For example, when Alice decides to spend a UTXO in a private execution, her spending key generates the required private keys that will sign the transaction and ensure its validity. However, the spending key itself -never appears in any public state or transaction data, preserving its confidentiality.

    • Ensuring security through isolation: One of the key security principles of the spending key is its isolation from the network. Since it never interacts with +never appears in any public state or transaction data, preserving its confidentiality.

      +
      • +
    • +

      Ensuring security through isolation: One of the key security principles of the spending key is its isolation from the network. Since it never interacts with public-facing elements, such as the public state or kernel circuits, the risk of exposure is significantly reduced. This isolation ensures that even if other parts -of the cryptographic infrastructure are compromised, the spending key remains protected, preventing unauthorized spending of UTXOs.

    In summary, the spending key in Nescience is a powerful and carefully guarded element of the cryptographic system. It is the root key from which other private keys +of the cryptographic infrastructure are compromised, the spending key remains protected, preventing unauthorized spending of UTXOs.

    +
    • +
+

In summary, the spending key in Nescience is a powerful and carefully guarded element of the cryptographic system. It is the root key from which other private keys are derived, allowing users to spend their UTXOs securely and privately. Its isolation from the public state and its random generation from a secure scalar field ensures -that the spending key remains protected, making it a cornerstone of security in NSSA.

II. Private keys​

In Nescience, the private key is an essential cryptographic element responsible for facilitating various secure operations, such as generating commitments and signing +that the spending key remains protected, making it a cornerstone of security in NSSA.

+

II. Private keys​

+

In Nescience, the private key is an essential cryptographic element responsible for facilitating various secure operations, such as generating commitments and signing transactions. While the spending key plays a foundational role in safeguarding access to UTXOs and assets, the private keys handle the operational aspects of transactions -and cryptographic proofs. The private key consists of three critical components: privatekey.rsd{private}_{key}.rsd, privatekey.rcm{private}_{key}.rcm, and privatekey.sig{private}_{key}.sig, each serving a -distinct purpose within the Nescience cryptographic framework.

  1. privatekey.rsd{private}_{key}.rsd (random seed): The random seed (privatekey.rsd{private}_{key}.rsd) is the first and foundational component of the private key. It is a value randomly chosen from the scalar field, which ensures +and cryptographic proofs. The private key consists of three critical components: privatekey.rsd{private}_{key}.rsd, privatekey.rcm{private}_{key}.rcm, and privatekey.sig{private}_{key}.sig, each serving a +distinct purpose within the Nescience cryptographic framework.

    +
      +
    1. +

      privatekey.rsd{private}_{key}.rsd (random seed): The random seed (privatekey.rsd{private}_{key}.rsd) is the first and foundational component of the private key. It is a value randomly chosen from the scalar field, which ensures its cryptographic security and unpredictability. This seed is generated using a random number generator, making it virtually impossible to predict or replicate. The random seed is essential because it is used to derive the other two components of the private key. By leveraging a secure random seed, Nescience ensures that the entire private key structure is rooted in randomness, preventing external entities from guessing or deriving the key through brute-force attacks. -The strength of the random seed ensures the overall security of the private key and, consequently, the integrity of the user's transactions and commitments.

    2. privatekey.rcm{private}_{key}.rcm (random commitment): The random commitment component (privatekey.rcm{private}_{key}.rcm) is a crucial part of the private key used specifically in the commitment scheme. It acts as a blinding factor, -adding a layer of security to commitments made by the user. The privatekey.rcm{private}_{key}.rcm value is also drawn from the scalar field and is used to ensure that the commitment -to any UTXO or other sensitive data remains confidential. The commitment scheme in Nescience requires the use of privatekey.rcm{private}_{key}.rcm to create cryptographic commitments -that bind the user to specific data (such as UTXO details) without revealing the actual data. The role of privatekey.rcm{private}_{key}.rcm is to ensure that these commitments are -non-malleable and secure, preventing anyone from modifying the committed data without detection. For instance, when Alice commits to a UTXO, privatekey.rcm{private}_{key}.rcm is used +The strength of the random seed ensures the overall security of the private key and, consequently, the integrity of the user's transactions and commitments.

      +
      3. +
    3. +

      privatekey.rcm{private}_{key}.rcm (random commitment): The random commitment component (privatekey.rcm{private}_{key}.rcm) is a crucial part of the private key used specifically in the commitment scheme. It acts as a blinding factor, +adding a layer of security to commitments made by the user. The privatekey.rcm{private}_{key}.rcm value is also drawn from the scalar field and is used to ensure that the commitment +to any UTXO or other sensitive data remains confidential. The commitment scheme in Nescience requires the use of privatekey.rcm{private}_{key}.rcm to create cryptographic commitments +that bind the user to specific data (such as UTXO details) without revealing the actual data. The role of privatekey.rcm{private}_{key}.rcm is to ensure that these commitments are +non-malleable and secure, preventing anyone from modifying the committed data without detection. For instance, when Alice commits to a UTXO, privatekey.rcm{private}_{key}.rcm is used to generate a Pedersen commitment that ensures the UTXO details are hidden but can still be verified cryptographically. This means that even though the actual UTXO details -are concealed, their existence and integrity can be proven.

    4. privatekey.sig{private}_{key}.sig (signing key for transactions): The signing key (privatekey.sig{private}_{key}.sig) is the third and final component of the private key, used primarily for signing transactions. One possible approach is that -Nescience employs Schnorr signatures, a cryptographic protocol known for its efficiency and security. In this case, the privatekey.sig{private}_{key}.sig component would generate +are concealed, their existence and integrity can be proven.

      +
      5. +
    5. +

      privatekey.sig{private}_{key}.sig (signing key for transactions): The signing key (privatekey.sig{private}_{key}.sig) is the third and final component of the private key, used primarily for signing transactions. One possible approach is that +Nescience employs Schnorr signatures, a cryptographic protocol known for its efficiency and security. In this case, the privatekey.sig{private}_{key}.sig component would generate Schnorr signatures that are used to authenticate transactions, ensuring that only the rightful owner of the private key can authorize the spending of UTXOs. Schnorr signatures are important as they provide a secure and non-repudiable method of verifying that a transaction was initiated by the legitimate owner of the assets. -When Alice signs a transaction using her privatekey.sig{private}_{key}.sig, the corresponding public key allows others to verify that the transaction was indeed signed by Alice, +When Alice signs a transaction using her privatekey.sig{private}_{key}.sig, the corresponding public key allows others to verify that the transaction was indeed signed by Alice, without revealing her private key. This verification process ensures that all transactions are legitimate and prevents unauthorized entities from forging transactions -or spending assets they do not control. Even if an attacker gains access to the signed transaction, they cannot reverse engineer the privatekey.sig{private}_{key}.sig, ensuring -the security of Alice's future transactions.

    Robustness of private keys in Nescience

    Despite the critical role of the private key in the operation of NSSA, the system is designed to maintain security even in the event that the +or spending assets they do not control. Even if an attacker gains access to the signed transaction, they cannot reverse engineer the privatekey.sig{private}_{key}.sig, ensuring +the security of Alice's future transactions.

    +
    2. +
+

Robustness of private keys in Nescience

+

Despite the critical role of the private key in the operation of NSSA, the system is designed to maintain security even in the event that the private key is compromised. This resilience is achieved through the integrity of the spending key, which is never exposed in the process of signing or committing. The spending key acts as the ultimate safeguard, ensuring that even if a private key component is compromised, the attacker cannot access or spend the user's assets -without control over the spending key.

The architecture’s design, where private keys handle operational tasks but rely on the spending key for ultimate control, ensures a layered approach to security. -This way, the system can mitigate the damage of a compromised private key by maintaining the inviolability of the user's assets.

Conclusion

In summary, the private key in Nescience consists of three interrelated components that together ensure secure transaction signing, commitment creation, and the -protection of user data. The privatekey.rsd{private}_{key}.rsd serves as the root from which the other key components are derived, ensuring randomness and security. -The privatekey.rcm{private}_{key}.rcm plays a crucial role in generating commitments, while privatekey.sig{private}_{key}.sig provides the signing capability needed for transaction authentication. -Together, these components enable users to engage in private, secure transactions while preserving the integrity of their assets, even in the face of potential key compromise.

III. Public keys​

Public keys in Nescience serve as the user's interface with the network, allowing for secure interaction and verification without exposing the user's private keys. +without control over the spending key.

+

The architecture’s design, where private keys handle operational tasks but rely on the spending key for ultimate control, ensures a layered approach to security. +This way, the system can mitigate the damage of a compromised private key by maintaining the inviolability of the user's assets.

+

Conclusion

+

In summary, the private key in Nescience consists of three interrelated components that together ensure secure transaction signing, commitment creation, and the +protection of user data. The privatekey.rsd{private}_{key}.rsd serves as the root from which the other key components are derived, ensuring randomness and security. +The privatekey.rcm{private}_{key}.rcm plays a crucial role in generating commitments, while privatekey.sig{private}_{key}.sig provides the signing capability needed for transaction authentication. +Together, these components enable users to engage in private, secure transactions while preserving the integrity of their assets, even in the face of potential key compromise.

+

III. Public keys​

+

Public keys in Nescience serve as the user's interface with the network, allowing for secure interaction and verification without exposing the user's private keys. Derived directly from the user's private keys, public keys play a crucial role in enabling cryptographic operations such as transaction verification, commitment schemes, and deterministic computations. The public key components correspond to their private key counterparts and ensure that transactions and commitments are securely processed -and validated across the network.

  1. publickey.sig{public}_{key}.sig (verifying Schnorr signatures):

The publickey.sig{public}_{key}.sig is derived from the signing component of the private key (privatekey.sig{private}_{key}.sig) and is used for verifying Schnorr signatures. +and validated across the network.

+
    +
  1. publickey.sig{public}_{key}.sig (verifying Schnorr signatures):
    2. +
+

The publickey.sig{public}_{key}.sig is derived from the signing component of the private key (privatekey.sig{private}_{key}.sig) and is used for verifying Schnorr signatures. Schnorr signatures are used to authenticate transactions, ensuring that they have been signed by the legitimate owner of the private key. This public key is essentially a verification key, allowing others in the network to confirm that a specific transaction was indeed authorized by the user. When a transaction is -broadcast to the network, publickey.sig{public}_{key}.sig enables any participant to verify that the transaction’s signature matches the user’s private key without +broadcast to the network, publickey.sig{public}_{key}.sig enables any participant to verify that the transaction’s signature matches the user’s private key without needing access to the private key itself. This mechanism prevents forgeries as only the legitimate owner with access to the private key can generate a valid Schnorr signature. -For example, if Alice sends a transaction, she signs it with her private key (privatekey.sig{private}_{key}.sig). Bob, or any other network participant, can use Alice’s publickey.sig{public}_{key}.sig -to verify the signature. If the signature is valid, Bob can be confident that the transaction was authorized by Alice and not by an imposter.

  1. publickey.rcm{public}_{key}.rcm (commitment schemes)

The publickey.rcm{public}_{key}.rcm is derived from the commitment component of the private key (privatekey.rcm{private}_{key}.rcm). It is used in the commitment schemes +For example, if Alice sends a transaction, she signs it with her private key (privatekey.sig{private}_{key}.sig). Bob, or any other network participant, can use Alice’s publickey.sig{public}_{key}.sig +to verify the signature. If the signature is valid, Bob can be confident that the transaction was authorized by Alice and not by an imposter.

+
    +
  1. publickey.rcm{public}_{key}.rcm (commitment schemes)
    2. +
+

The publickey.rcm{public}_{key}.rcm is derived from the commitment component of the private key (privatekey.rcm{private}_{key}.rcm). It is used in the commitment schemes that underpin Nescience’s privacy-preserving architecture. Commitments are a crucial cryptographic technique that allows users to commit to a piece of data (such as a UTXO) -without revealing the actual data, while still enabling proof of its integrity and existence. In Nescience, the publickey.rcm{public}_{key}.rcm is used as part of the Pedersen commitment scheme, +without revealing the actual data, while still enabling proof of its integrity and existence. In Nescience, the publickey.rcm{public}_{key}.rcm is used as part of the Pedersen commitment scheme, where it functions as a public commitment to certain transaction details. Even though the actual values are hidden (thanks to the private key component), the commitment can -still be verified by other network participants using publickey.rcm{public}_{key}.rcm. This enables secure and private transactions while maintaining the ability to verify that commitments -are consistent with the original data. For instance, when Alice commits to a UTXO, she uses her private key to generate the commitment, and the publickey.rcm{public}_{key}.rcm is available -to others to verify the commitment’s validity without revealing the underlying details.

  1. publickey.sk(prf){public}_{key}.sk(prf) (pseudorandom function)

The publickey.sk(prf){public}_{key}.sk(prf) is derived from a random field element within the private key and is used to generate the pseudorandom function (PRF) associated with the user's account. +still be verified by other network participants using publickey.rcm{public}_{key}.rcm. This enables secure and private transactions while maintaining the ability to verify that commitments +are consistent with the original data. For instance, when Alice commits to a UTXO, she uses her private key to generate the commitment, and the publickey.rcm{public}_{key}.rcm is available +to others to verify the commitment’s validity without revealing the underlying details.

+
    +
  1. publickey.sk(prf){public}_{key}.sk(prf) (pseudorandom function)
    2. +
+

The publickey.sk(prf){public}_{key}.sk(prf) is derived from a random field element within the private key and is used to generate the pseudorandom function (PRF) associated with the user's account. This PRF is essential for producing deterministic outputs based on the user’s keys and transaction data while ensuring that these outputs are unique to the user and cannot be predicted or replicated by others. The PRF is crucial in scenarios where the user needs to derive unique identifiers or values that are tied to their specific account, -ensuring that these values remain consistent across different transactions or interactions without revealing sensitive information. For example, publickey.sk(prf){public}_{key}.sk(prf) may be -used in generating deterministic yet secure addresses or transaction references, which can be linked to the user’s activity in a controlled manner. By using publickey.sk(prf){public}_{key}.sk(prf), +ensuring that these values remain consistent across different transactions or interactions without revealing sensitive information. For example, publickey.sk(prf){public}_{key}.sk(prf) may be +used in generating deterministic yet secure addresses or transaction references, which can be linked to the user’s activity in a controlled manner. By using publickey.sk(prf){public}_{key}.sk(prf), Nescience ensures that certain operations, like generating addresses or computing deterministic transaction outcomes, remain both private and cryptographically secure. The public key’s -role in this process is to maintain consistency in these outputs while preventing unauthorized parties from reverse engineering the associated private keys or transaction data.

Summary

Public keys in Nescience are essential for secure interactions within the network. "publickey.sig{public}_{key}.sig" allows others to verify that transactions were signed by the legitimate owner, -ensuring the authenticity of every operation. "publickey.rcm{public}_{key}.rcm" enables secure and private commitment schemes, allowing participants to commit to transaction details without -revealing sensitive information. Finally, "publickey.sk(prf){public}_{key}.sk(prf)" powers deterministic outputs through a pseudorandom function, ensuring that user-specific data remains consistent -and secure throughout various transactions. Together, these public key components facilitate privacy, security, and trust within NSSA, enabling seamless interactions while safeguarding user data.

IV. Viewing key​

The viewing key in NSSA is a specialized cryptographic key that allows a user to decrypt both incoming and outgoing transactions associated with their account. +role in this process is to maintain consistency in these outputs while preventing unauthorized parties from reverse engineering the associated private keys or transaction data.

+

Summary

+

Public keys in Nescience are essential for secure interactions within the network. "publickey.sig{public}_{key}.sig" allows others to verify that transactions were signed by the legitimate owner, +ensuring the authenticity of every operation. "publickey.rcm{public}_{key}.rcm" enables secure and private commitment schemes, allowing participants to commit to transaction details without +revealing sensitive information. Finally, "publickey.sk(prf){public}_{key}.sk(prf)" powers deterministic outputs through a pseudorandom function, ensuring that user-specific data remains consistent +and secure throughout various transactions. Together, these public key components facilitate privacy, security, and trust within NSSA, enabling seamless interactions while safeguarding user data.

+

IV. Viewing key​

+

The viewing key in NSSA is a specialized cryptographic key that allows a user to decrypt both incoming and outgoing transactions associated with their account. This key is designed to offer a degree of transparency to the user, enabling them to view the details of their transactions without compromising the security of their assets or granting -control over those assets.

  • Role of the viewing key: The primary function of the viewing key is to provide visibility into transaction details while maintaining the integrity of private, shielded, +control over those assets.

    +
      +
    • +

      Role of the viewing key: The primary function of the viewing key is to provide visibility into transaction details while maintaining the integrity of private, shielded, or deshielded transactions. It enables the user to see the specifics of the transactions they are involved in—such as amounts transferred, asset types, and metadata—without exposing the sensitive transaction data to the broader network. For instance, if Alice has executed a private transaction with Bob, her viewing key allows her to decrypt and review the details of the transaction, ensuring that everything was processed correctly. This ability to audit her own transactions helps Alice maintain confidence in the integrity -of her private interactions on the blockchain.

    • Security considerations: Despite its utility, the viewing key must be handled with care as its exposure could potentially compromise the user’s privacy. +of her private interactions on the blockchain.

      +
      • +
    • +

      Security considerations: Despite its utility, the viewing key must be handled with care as its exposure could potentially compromise the user’s privacy. Although possessing the viewing key does not provide the ability to spend or sign transactions (that authority remains strictly with the spending key and private keys), it does allow anyone with access to the viewing key to decrypt the details of the user’s private transactions. This means that if the viewing key is leaked or stolen, the privacy guarantees of Nescience’s private, shielded, and deshielded executions could be undermined. Specifically, the viewing key could be used to link various transactions, breaking the unlinkability of private transactions. For example, an attacker with access to the viewing key could decrypt past and future transactions, exposing the relationships between different parties and transaction flows. To mitigate this risk, Nescience recommends that users treat their viewing key with the same level of protection as their private keys. -It should be stored securely in encrypted hardware wallets or other secure storage solutions to prevent unauthorized access.

    • Balancing privacy and transparency: The viewing key provides an essential balance between privacy and transparency in NSSA. While it ensures that users +It should be stored securely in encrypted hardware wallets or other secure storage solutions to prevent unauthorized access.

      +
      • +
    • +

      Balancing privacy and transparency: The viewing key provides an essential balance between privacy and transparency in NSSA. While it ensures that users can monitor their transaction history and verify the details of their private transactions, it does so without compromising the control of their funds. This allows users to maintain a transparent view of their interactions while keeping their assets secure. For example, if Alice is using shielded execution to transfer assets, her viewing key enables her to audit the transaction without allowing anyone else, including Bob or external observers, to see the specific details unless they also have access to the viewing key. Moreover, since the viewing key does not grant signing or spending authority, even if it were exposed, an attacker would still not be able to manipulate the user’s assets. However, -to maintain the unlinkability and confidentiality of private transactions, the viewing key must be kept secure at all times.

    • Protecting transaction unlinkability: In private transactions, unlinkability is one of the core privacy guarantees. This property ensures that individual +to maintain the unlinkability and confidentiality of private transactions, the viewing key must be kept secure at all times.

      +
      • +
    • +

      Protecting transaction unlinkability: In private transactions, unlinkability is one of the core privacy guarantees. This property ensures that individual transactions cannot be correlated with each other or linked to the same user unless that user chooses to reveal the connection. The viewing key must be carefully protected to preserve this unlinkability, as its compromise could allow someone to map out a user’s private transaction history. For instance, in deshielded transactions, the viewing key allows the user to see which private UTXOs were consumed and how the public state was modified. If the viewing key is compromised, an attacker could potentially -link private UTXOs across multiple transactions, unraveling the user’s privacy.

    Conclusion

    The viewing key in Nescience is a powerful tool for providing insight into both incoming and outgoing transactions without granting control over assets. It allows users +link private UTXOs across multiple transactions, unraveling the user’s privacy.

    +
    • +
+

Conclusion

+

The viewing key in Nescience is a powerful tool for providing insight into both incoming and outgoing transactions without granting control over assets. It allows users to decrypt and verify their transaction details, maintaining transparency in their interactions. However, due to its potential to compromise privacy if exposed, the viewing key must be handled with great care. Proper security measures are necessary to protect the viewing key, ensuring that the unlinkability of private, shielded, and deshielded -transactions remains intact. In this way, the viewing key offers a crucial balance between privacy and transparency within the Nescience ecosystem.

V. Ephemeral key​

The ephemeral key is generated using a combination of the sender’s spending key and the UTXO's nullifier, ensuring that the key is unique to each transaction. -The process can be informally described as follows:

  1. Ephemeral key generation
    -Let ρ\rho denote the nullifier of the UTXO being consumed in the transaction. The sender uses the receiver’s public key component publickey.sk(prf){public}_{key}.sk(prf), -which is derived from the receiver’s private key, to compute an ephemeral secret key (eskesk). The computation is based on the nullifier ρ\rho and a base value:

    esk=publickey.sk(prf((0,0,0,0)∣∣ρ)esk = {public}_{key}.sk(prf((0,0,0,0) || \rho) -This formula binds the secret key to the specific transaction, leveraging the receiver’s cryptographic identity and the unique properties of the UTXO being spent.

  2. Deriving the ephemeral public key
    -After computing the ephemeral secret key (eskesk), the next step is to derive the corresponding ephemeral public key (epk). This is done using the Key Agreement -Protocol's DerivePublic algorithm, which generates the public key associated with the shared secret key. The ephemeral public key is computed as:

    epk=KA.DerivePublic(esk,gd)epk = KA.DerivePublic(esk, gd)

Here, (gdgd) is the diversifier address associated with the receiver’s account. The diversifier address is computed from the receiver’s -account using the DiversifierHash function:

gd=receiver.DiversifierHash(d)gd = receiver.DiversifierHash(d)

The diversifier (dd) is a random value selected by the sender to add randomness to the process. This diversifier ensures that even if a single receiver is involved -in multiple transactions, the derived keys remain distinct for each transaction. The value (dd) is included in the transaction note for transparency and reproducibility.

  1. Establishing the shared secret
    The shared secret, used to encrypt the transaction details, is derived from the key agreement between the sender’s ephemeral key and the receiver’s viewing key. +transactions remains intact. In this way, the viewing key offers a crucial balance between privacy and transparency within the Nescience ecosystem.

    +

    V. Ephemeral key​

    +

    The ephemeral key is generated using a combination of the sender’s spending key and the UTXO's nullifier, ensuring that the key is unique to each transaction. +The process can be informally described as follows:

    +
      +
    1. Ephemeral key generation
      +Let ρ\rho denote the nullifier of the UTXO being consumed in the transaction. The sender uses the receiver’s public key component publickey.sk(prf){public}_{key}.sk(prf), +which is derived from the receiver’s private key, to compute an ephemeral secret key (eskesk). The computation is based on the nullifier ρ\rho and a base value:
      2. +
    +

    esk=publickey.sk(prf((0,0,0,0)∣∣ρ)esk = {public}_{key}.sk(prf((0,0,0,0) || \rho) +This formula binds the secret key to the specific transaction, leveraging the receiver’s cryptographic identity and the unique properties of the UTXO being spent.

    +
      +
    1. Deriving the ephemeral public key
      +After computing the ephemeral secret key (eskesk), the next step is to derive the corresponding ephemeral public key (epk). This is done using the Key Agreement +Protocol's DerivePublic algorithm, which generates the public key associated with the shared secret key. The ephemeral public key is computed as:
      2. +
    +

    epk=KA.DerivePublic(esk,gd)epk = KA.DerivePublic(esk, gd)

    +

    Here, (gdgd) is the diversifier address associated with the receiver’s account. The diversifier address is computed from the receiver’s +account using the DiversifierHash function:

    +

    gd=receiver.DiversifierHash(d)gd = receiver.DiversifierHash(d)

    +

    The diversifier (dd) is a random value selected by the sender to add randomness to the process. This diversifier ensures that even if a single receiver is involved +in multiple transactions, the derived keys remain distinct for each transaction. The value (dd) is included in the transaction note for transparency and reproducibility.

    +
      +
    1. Establishing the shared secret
      +The shared secret, used to encrypt the transaction details, is derived from the key agreement between the sender’s ephemeral key and the receiver’s viewing key. Any party possessing the receiver’s viewing key can use it in conjunction with the ephemeral key to compute the shared secret, which is then used to decrypt the transaction. -This ensures that only the intended recipient (or anyone with their viewing key) can access the transaction details.

    Key components and protocol

    The formal protocol for generating ephemeral keys closely follows this informal description but involves additional intermediate steps for converting values to +This ensures that only the intended recipient (or anyone with their viewing key) can access the transaction details.

    2. +
+

Key components and protocol

+

The formal protocol for generating ephemeral keys closely follows this informal description but involves additional intermediate steps for converting values to binary sequences to fit implementation requirements. These steps are essential for ensuring compatibility with cryptographic algorithms used in NSSA. -The protocol uses the following key components:

  • Nullifier (ρ\rho): Ensures that the ephemeral key is tied to the specific UTXO being consumed, preventing reuse of the key in future transactions.
  • Receiver’s public key (publickey.sk(prf){public}_{key}.sk(prf): Establishes the receiver's identity in the key generation process, ensuring that the shared secret can -only be derived by the intended party.
  • Diversifier (dd): Adds randomness to the transaction, ensuring that keys remain unique across different transactions involving the same receiver.

The end result is an ephemeral key system that provides strong cryptographic guarantees for transaction privacy, leveraging key agreement protocols and secure -cryptographic primitives to prevent unauthorized access to sensitive transaction data.

Conclusion

The ephemeral key in Nescience is a critical element for maintaining transaction confidentiality. It facilitates a secure key agreement between the sender and the receiver, +The protocol uses the following key components:

+
    +
  • Nullifier (ρ\rho): Ensures that the ephemeral key is tied to the specific UTXO being consumed, preventing reuse of the key in future transactions.
    • +
  • Receiver’s public key (publickey.sk(prf){public}_{key}.sk(prf): Establishes the receiver's identity in the key generation process, ensuring that the shared secret can +only be derived by the intended party.
    • +
  • Diversifier (dd): Adds randomness to the transaction, ensuring that keys remain unique across different transactions involving the same receiver.
    • +
+

The end result is an ephemeral key system that provides strong cryptographic guarantees for transaction privacy, leveraging key agreement protocols and secure +cryptographic primitives to prevent unauthorized access to sensitive transaction data.

+

Conclusion

+

The ephemeral key in Nescience is a critical element for maintaining transaction confidentiality. It facilitates a secure key agreement between the sender and the receiver, allowing for the encryption of transaction details with a shared secret that can only be derived by the intended recipient. By incorporating the nullifier, receiver's public key, and diversifier address, the ephemeral key ensures that transaction privacy is preserved while preventing unauthorized access to transaction information, even in a complex, -multi-party blockchain environment.

VI. Nescience addresses​

Nescience’s dual address system is a core component of its privacy-focused architecture, designed to balance transparency and confidentiality across different types of transactions. -The architecture provides each user or smart contract with both public addresses and private addresses, allowing them to participate in both open and confidential activities on the blockchain.

a) Public addresses​

Public addresses in Nescience are visible to all participants on the network and reside within the public state. These addresses are essential for engaging in +multi-party blockchain environment.

+

VI. Nescience addresses​

+

Nescience’s dual address system is a core component of its privacy-focused architecture, designed to balance transparency and confidentiality across different types of transactions. +The architecture provides each user or smart contract with both public addresses and private addresses, allowing them to participate in both open and confidential activities on the blockchain.

+

a) Public addresses​

+

Public addresses in Nescience are visible to all participants on the network and reside within the public state. These addresses are essential for engaging in transparent and verifiable interactions, such as sending tokens or invoking smart contracts that are meant to be publicly auditable. Public addresses serve as -the interface for users who need to engage with the transparent elements of the system, including public transactions or smart contracts that require public access.

They are analogous to traditional blockchain addresses seen in systems like Ethereum or Bitcoin, where every participant can see the address and the transactions associated with it. +the interface for users who need to engage with the transparent elements of the system, including public transactions or smart contracts that require public access.

+

They are analogous to traditional blockchain addresses seen in systems like Ethereum or Bitcoin, where every participant can see the address and the transactions associated with it. For example, when Alice wants to receive tokens from Bob in a public transaction, she can provide her public address, allowing Bob to send the tokens transparently. -Anyone on the network can verify the transaction, providing accountability and trust in the public state.

Because public addresses are visible and auditable, they are typically used for interactions where privacy is not a concern or where transparency is desirable. +Anyone on the network can verify the transaction, providing accountability and trust in the public state.

+

Because public addresses are visible and auditable, they are typically used for interactions where privacy is not a concern or where transparency is desirable. This could include simple token transfers, public contract calls, or interactions with dapps that require public accountability, -such as voting or governance systems.

b) Private addresses​

In contrast, private addresses are designed for confidentiality and are not visible onchain. These addresses are used exclusively for private transactions and executions, +such as voting or governance systems.

+

b) Private addresses​

+

In contrast, private addresses are designed for confidentiality and are not visible onchain. These addresses are used exclusively for private transactions and executions, ensuring that sensitive details—such as the sender, receiver, or amount transferred—remain hidden from the public state. Private addresses are a key feature of -Nescience’s private, shielded, and deshielded execution models, where preserving the confidentiality of participants is crucial.

Users can generate an unlimited number of private addresses using their private keys. This flexibility allows users to compartmentalize their interactions, +Nescience’s private, shielded, and deshielded execution models, where preserving the confidentiality of participants is crucial.

+

Users can generate an unlimited number of private addresses using their private keys. This flexibility allows users to compartmentalize their interactions, giving them the ability to provide different private addresses to different parties. For instance, Alice could create a unique private address for each entity she interacts with, thereby ensuring that her transactions remain isolated and difficult to trace. This feature enhances privacy by preventing any direct linkage -between different transactions or activities associated with a single user.

Private addresses are not tied to the public state and are only accessible through the user’s private key infrastructure. Transactions involving private addresses +between different transactions or activities associated with a single user.

+

Private addresses are not tied to the public state and are only accessible through the user’s private key infrastructure. Transactions involving private addresses are conducted within the confines of the private state and are only decrypted by the intended participants. For example, when Alice sends tokens to Bob using -a private address, the details of that transaction remain confidential, accessible only to Alice and Bob, unless they choose to reveal it.

Role of the viewing key in private addresses: A key feature of Nescience’s private address system is the viewing key, which allows users to decrypt any transaction +a private address, the details of that transaction remain confidential, accessible only to Alice and Bob, unless they choose to reveal it.

+

Role of the viewing key in private addresses: A key feature of Nescience’s private address system is the viewing key, which allows users to decrypt any transaction involving their private addresses. This capability provides oversight and transparency into the user’s private transactions, ensuring that they can monitor their own activity without exposing the details to the public. The viewing key does not compromise the security of the user's assets as it does not grant spending or signing authority. However, it does allow the user to audit and verify the accuracy of their private transactions, ensuring that everything proceeds as expected. For instance, Alice can use her viewing key to review the details of a private transaction she conducted with Bob, ensuring that the correct amount was transferred and that the transaction was properly processed. This functionality is critical for users who want to maintain control over their private interactions while still benefiting from transparency into their transaction history. -The ability to generate multiple private addresses and decrypt them with the viewing key ensures that users can maintain compartmentalized privacy without sacrificing oversight.

Summary

Nescience’s dual address system—comprising public and private addresses—provides users with the flexibility to engage in both transparent and confidential transactions. +The ability to generate multiple private addresses and decrypt them with the viewing key ensures that users can maintain compartmentalized privacy without sacrificing oversight.

+

Summary

+

Nescience’s dual address system—comprising public and private addresses—provides users with the flexibility to engage in both transparent and confidential transactions. Public addresses are visible onchain and are used for open, public interactions that require accountability and auditability. In contrast, private addresses are -invisible onchain and are used for confidential transactions, enhancing privacy and security.

By allowing users to generate multiple private addresses, Nescience gives individuals control over the visibility of their transactions. Combined with the viewing +invisible onchain and are used for confidential transactions, enhancing privacy and security.

+

By allowing users to generate multiple private addresses, Nescience gives individuals control over the visibility of their transactions. Combined with the viewing key’s ability to decrypt transactions involving private addresses, the system ensures that users can maintain transparency over their private transactions without exposing sensitive information to the public state. This dual-address approach enables users to seamlessly switch between public and private interactions depending on their needs, -providing a robust framework for both privacy and transparency in NSSA.

VII. Conclusion​

Key management in NSSA is a carefully designed system that strikes an optimal balance between security, privacy, and flexibility. +providing a robust framework for both privacy and transparency in NSSA.

+

VII. Conclusion​

+

Key management in NSSA is a carefully designed system that strikes an optimal balance between security, privacy, and flexibility. The architecture’s hierarchical structure, with distinct roles for the spending key, private keys, and public keys, ensures that users retain full control over their assets while maintaining the integrity of their transactions. The spending key, as the root of security, provides unassailable control over the user's UTXOs and assets, ensuring that only the rightful owner can authorize spending. Private keys, derived from the spending key, enable users to engage -in cryptographic operations such as signing transactions and generating commitments without exposing sensitive information to the network.

The viewing key adds another layer of transparency, allowing users to decrypt and review their transactions without compromising their authority over their assets. -While it provides a window into transaction history, the viewing key does not grant spending power, preserving the critical separation between visibility and control.

The dual system of public and private addresses gives users the flexibility to navigate between open, transparent transactions and confidential, privacy-protected activities. +in cryptographic operations such as signing transactions and generating commitments without exposing sensitive information to the network.

+

The viewing key adds another layer of transparency, allowing users to decrypt and review their transactions without compromising their authority over their assets. +While it provides a window into transaction history, the viewing key does not grant spending power, preserving the critical separation between visibility and control.

+

The dual system of public and private addresses gives users the flexibility to navigate between open, transparent transactions and confidential, privacy-protected activities. Public addresses allow users to engage in verifiable, public interactions while private addresses enable compartmentalized, secure transactions that remain hidden from the public eye. This dual-address framework ensures that users can seamlessly adapt to different privacy requirements, whether they are participating in public -dapps or conducting sensitive financial operations.

Overall, Nescience’s cryptographic infrastructure is designed to empower users to engage confidently in both transparent and confidential activities. +dapps or conducting sensitive financial operations.

+

Overall, Nescience’s cryptographic infrastructure is designed to empower users to engage confidently in both transparent and confidential activities. By providing flexible, secure key management and address systems, Nescience ensures that users can fully participate in the blockchain ecosystem without compromising their privacy or control. The architecture supports the nuanced needs of modern blockchain users, who require both the transparency of public -interactions and the security of private transactions, all while maintaining the integrity and confidentiality of their assets.

c) Trees in NSSA​

Trees in NSSA serve as verifiable databases, essential for maintaining privacy and security. Different types of trees are used for various purposes:

  1. Global state tree: The global state tree is a single, public tree that holds all public assets and storage information. It acts as a central repository for all -publicly accessible data on the blockchain. By organizing this data in a Merkle tree structure, the Global State Tree allows for efficient and secure verification of public information.

  2. Hashed UTXO tree: The hashed UTXO tree is a public tree that contains hashes of all created UTXOs. When users wish to consume a UTXO, they provide a membership +interactions and the security of private transactions, all while maintaining the integrity and confidentiality of their assets.

    +

    c) Trees in NSSA​

    +

    Trees in NSSA serve as verifiable databases, essential for maintaining privacy and security. Different types of trees are used for various purposes:

    +
      +
    1. +

      Global state tree: The global state tree is a single, public tree that holds all public assets and storage information. It acts as a central repository for all +publicly accessible data on the blockchain. By organizing this data in a Merkle tree structure, the Global State Tree allows for efficient and secure verification of public information.

      +
      2. +
    2. +

      Hashed UTXO tree: The hashed UTXO tree is a public tree that contains hashes of all created UTXOs. When users wish to consume a UTXO, they provide a membership proof to demonstrate that the UTXO exists within this tree. This process ensures that only valid and existing UTXOs can be spent, maintaining the integrity of transactions. In fact, users generate membership proofs that verify the presence of specific UTXOs in the tree without revealing their actual data. The benefit here is that the Merkle -tree structure allows for quick and efficient verification of UTXO existence.

    3. UTXO trees (private states): Each user or smart contract has its private state stored in UTXO trees. These trees are kept as plaintext on the client’s +tree structure allows for quick and efficient verification of UTXO existence.

      +
      4. +
    4. +

      UTXO trees (private states): Each user or smart contract has its private state stored in UTXO trees. These trees are kept as plaintext on the client’s local system (off-chain), ensuring privacy as sensitive information remains confidential. The private state includes all UTXOs owned by the user or the smart contract, and these -are not directly exposed to the public blockchain. For instance, users have full control over their private state, which is not visible to other participants in the network.

    In conclusion, the tree structures enable efficient verification of transaction validity without compromising privacy. By using Merkle trees, +are not directly exposed to the public blockchain. For instance, users have full control over their private state, which is not visible to other participants in the network.

    +
    3. +
+

In conclusion, the tree structures enable efficient verification of transaction validity without compromising privacy. By using Merkle trees, Nescience ensures that any tampering with the data can be easily detected. The efficient structure of these trees supports the scalability of the architecture, allowing it to handle a large number of transactions and data entries. By leveraging different types of trees, Nescience ensures efficient and secure management -of both public and private states.

## d) Nullifier tree in Nescience

The nullifier tree is a fundamental component of NSSA, designed to prevent double spending by securely tracking all consumed UTXOs. -This tree acts as a public ledger of spent UTXOs, ensuring that once a UTXO is consumed in a transaction, it cannot be reused in future transactions.

The primary function of the nullifier Tree is to store the nullifiers of all consumed UTXOs. By recording the nullifiers in a public tree, -the system ensures that each UTXO is spent only once, thereby safeguarding the integrity of the entire network.

  • Ensuring non-membership and preventing double spending +of both public and private states.

    +

    ## d) Nullifier tree in Nescience

    +

    The nullifier tree is a fundamental component of NSSA, designed to prevent double spending by securely tracking all consumed UTXOs. +This tree acts as a public ledger of spent UTXOs, ensuring that once a UTXO is consumed in a transaction, it cannot be reused in future transactions.

    +

    The primary function of the nullifier Tree is to store the nullifiers of all consumed UTXOs. By recording the nullifiers in a public tree, +the system ensures that each UTXO is spent only once, thereby safeguarding the integrity of the entire network.

    +
      +
    • +

      Ensuring non-membership and preventing double spending Before a user can consume a UTXO in a transaction, they must provide a non-membership proof. This proof demonstrates that the UTXO’s nullifier does not already exist in the Nullifier Tree, proving that the UTXO has not been spent before. If the UTXO’s nullifier is found in the tree, the system will reject the transaction, preventing double spending. The non-membership proof ensures that users cannot attempt to spend the same UTXO in multiple transactions. This mechanism is critical for maintaining the security and reliability of NSSA. The tree structure, which is typically built using a cryptographic tree like a Merkle tree, allows for efficient verification of nullifiers. -Verifiers can quickly check whether a nullifier is present or absent in the tree, ensuring that each UTXO is only spent once.

    • Nullifier tree structure and operation +Verifiers can quickly check whether a nullifier is present or absent in the tree, ensuring that each UTXO is only spent once.

      +
      • +
    • +

      Nullifier tree structure and operation The nullifier tree is likely structured as a Merkle tree, which is a cryptographic binary tree where each node represents the hash of its child nodes. This structure allows for efficient storage and verification of large sets of nullifiers as only the root hash of the tree needs to be stored on the blockchain. When a new nullifier is added to the tree, the tree is recalculated, and the root hash is updated. This process ensures that all consumed UTXOs are securely recorded. @@ -479,57 +1002,134 @@ Each time a transaction consumes a UTXO, the nullifier is added to the Nullifier UTXO has not been double spent, verifiers can use the tree’s root hash and a proof of inclusion or exclusion (membership or non-membership proof) to check whether the nullifier is present in the tree. For example, if Alice wants to spend a UTXO, she must prove that the nullifier associated with that UTXO is not already in the Nullifier Tree. She generates a non-membership proof that shows her nullifier is not recorded in the tree, and the transaction is allowed to proceed. Once the transaction is completed, -the nullifier is added to the tree, ensuring that the UTXO cannot be used again.

    Conclusion +the nullifier is added to the tree, ensuring that the UTXO cannot be used again.

    +
    • +
+

Conclusion The Nullifier Tree is a crucial element of Nescience's security. By recording all consumed UTXOs and ensuring that nullifiers are unique, the tree prevents double spending and maintains the integrity of the blockchain. The non-membership proof mechanism guarantees that every transaction is validated against the tree. This structure supports -the scalability and security of NSSA, providing a reliable method for verifying the validity of transactions while preventing malicious behavior.

e) Recursive-friendly privacy-preserving zk-zkVM​

The development of the zk-zkVM in Nescience is a work in progress, as the architecture continues to evolve to support privacy-preserving transactions +the scalability and security of NSSA, providing a reliable method for verifying the validity of transactions while preventing malicious behavior.

+

e) Recursive-friendly privacy-preserving zk-zkVM​

+

The development of the zk-zkVM in Nescience is a work in progress, as the architecture continues to evolve to support privacy-preserving transactions and efficient ZKP generation. The goal of the zk-zkVM is to seamlessly integrate with the Nescience state-separation architecture, -ensuring that private transactions remain confidential while allowing the network to verify their validity without compromising privacy.

Currently, we are exploring and testing several existing zkVMs to identify the most suitable platform for our needs. Our focus is on finding a zkVM +ensuring that private transactions remain confidential while allowing the network to verify their validity without compromising privacy.

+

Currently, we are exploring and testing several existing zkVMs to identify the most suitable platform for our needs. Our focus is on finding a zkVM that not only supports the core features of Nescience, such as state separation and privacy, but also provides the efficiency and scalability required for a decentralized system. Once a suitable zkVM is chosen, we will begin implementing advanced privacy features on top of it, including support for -confidential transactions, selective disclosure, and recursive proof aggregation.

The integration of these privacy-preserving features with an existing zkVM will enable Nescience to fully employ its state-separation architecture, +confidential transactions, selective disclosure, and recursive proof aggregation.

+

The integration of these privacy-preserving features with an existing zkVM will enable Nescience to fully employ its state-separation architecture, ensuring that users can conduct private transactions with robust security and scalability. This approach will allow us to leverage the strengths of -proven zkVM technologies while enhancing them with the unique privacy and state-separation capabilities that Nescience requires.

  • Privacy-preserving features: At its core, the zk-zkVM is designed with privacy in mind. One of the zk-zkVM’s standout privacy features is selective disclosure, +proven zkVM technologies while enhancing them with the unique privacy and state-separation capabilities that Nescience requires.

    +
      +
    • +

      Privacy-preserving features: At its core, the zk-zkVM is designed with privacy in mind. One of the zk-zkVM’s standout privacy features is selective disclosure, which allows users to reveal only specific details of a transaction as needed. For example, a user could disclose the transaction amount while concealing the identities of the participants. The zk-zkVM employs advanced encryption techniques to protect this sensitive data. All transaction data is encrypted before being stored on the blockchain, so even if the data is intercepted, it cannot be deciphered without the appropriate decryption keys. Another of the crucial privacy-preserving features is the support for confidential transactions. Only the parties involved in the transaction can access the encrypted data. Furthermore, the zk-zkVM supports verifiable encryption, a powerful capability that allows encrypted data to be included in ZKPs without needing to decrypt it. This ensures that transaction details remain private -while their correctness can still be proven.

    • Lightweight design for accessibility: The zk-zkVM is being designed to be lightweight and efficient, enabling it to run on standard consumer-grade hardware. -This makes it accessible to a wide range of users without requiring specialized equipment or significant computational resources.

    • Faster proving time: To maintain a seamless user experience, especially during high transaction volumes, the zk-zkVM is being optimized for fast proving times. -Fast proof generation is particularly important for ensuring that the system remains usable during periods of peak activity, preventing bottlenecks and maintaining the fluidity of the network.

    • Recursive-friendly operations: One of the most advanced features of the zk-zkVM will be its support for recursive operations. Recursion enables the aggregation -of multiple proofs into a single proof, improving efficiency on both the client and sequencer sides of the network.

    • Client-side recursion (batch processing): When a single transaction involves multiple executions, each requiring its own ZKP, these individual +while their correctness can still be proven.

      +
      • +
    • +

      Lightweight design for accessibility: The zk-zkVM is being designed to be lightweight and efficient, enabling it to run on standard consumer-grade hardware. +This makes it accessible to a wide range of users without requiring specialized equipment or significant computational resources.

      +
      • +
    • +

      Faster proving time: To maintain a seamless user experience, especially during high transaction volumes, the zk-zkVM is being optimized for fast proving times. +Fast proof generation is particularly important for ensuring that the system remains usable during periods of peak activity, preventing bottlenecks and maintaining the fluidity of the network.

      +
      • +
    • +

      Recursive-friendly operations: One of the most advanced features of the zk-zkVM will be its support for recursive operations. Recursion enables the aggregation +of multiple proofs into a single proof, improving efficiency on both the client and sequencer sides of the network.

      +
      • +
    • +

      Client-side recursion (batch processing): When a single transaction involves multiple executions, each requiring its own ZKP, these individual proofs can be recursively aggregated before being sent to the sequencer. This reduces the overall data transmitted, enhancing the efficiency of the transaction process -by compressing multiple proofs into a single package. -

    • Sequencer-side recursion (reduced redundancy): The sequencer, which is responsible for processing transactions and creating verifiable blocks, collects transactions +by compressing multiple proofs into a single package.

      +
      • +
    • +

      Sequencer-side recursion (reduced redundancy): The sequencer, which is responsible for processing transactions and creating verifiable blocks, collects transactions containing aggregated proofs. These proofs are further merged into a single comprehensive proof, ensuring that all transactions within a block are validated collectively. -This process reduces redundancy and optimizes the blockchain’s efficiency by minimizing the size and complexity of the proofs required for verification.

    • Developer-friendly language: To foster widespread adoption and innovation within the Nescience ecosystem, the zk-zkVM would include a developer-friendly language. +This process reduces redundancy and optimizes the blockchain’s efficiency by minimizing the size and complexity of the proofs required for verification.

      +
      • +
    • +

      Developer-friendly language: To foster widespread adoption and innovation within the Nescience ecosystem, the zk-zkVM would include a developer-friendly language. This high-level language simplifies the process of building applications that leverage state separation and privacy-preserving transactions. The language should offer extensive support for modular design, APIs, and SDKs, enabling developers to integrate their applications with the zk-zkVM more easily. By lowering the barrier to entry, Nescience encourages -innovation and helps expand the range of privacy-preserving applications that can be built on its platform.

    Conclusion

    The zk-zkVM in Nescience is a powerful and versatile virtual machine that embodies the principles of privacy, efficiency, and scalability. By supporting ZKPs +innovation and helps expand the range of privacy-preserving applications that can be built on its platform.

    +
    • +
+

Conclusion

+

The zk-zkVM in Nescience is a powerful and versatile virtual machine that embodies the principles of privacy, efficiency, and scalability. By supporting ZKPs and integrating with advanced privacy technologies like homomorphic encryption. Its lightweight design allows it to run efficiently on standard hardware, promoting decentralization, and its recursive operations further enhance the system's scalability. With its developer-friendly language and fast proving times, the zk-zkVM is positioned as a key component in -fostering the growth and adoption of privacy-preserving blockchain applications.

f) MPC-based synchronization mechanism (under review)​

Nescience is developing an MPC-based synchronization mechanism to balance privacy and fairness between public and private execution types. +fostering the growth and adoption of privacy-preserving blockchain applications.

+

f) MPC-based synchronization mechanism (under review)​

+

Nescience is developing an MPC-based synchronization mechanism to balance privacy and fairness between public and private execution types. This mechanism extracts common information from encrypted UTXOs without revealing private details, ensuring privacy and preventing UTXO linkage to users or specific transactions. -It guarantees that public and private executions remain equitable, with the total input equaling the public output.

The mechanism employs MPC protocols to perform computations privately, ZKPs to verify correctness, and cryptographic protocols +It guarantees that public and private executions remain equitable, with the total input equaling the public output.

+

The mechanism employs MPC protocols to perform computations privately, ZKPs to verify correctness, and cryptographic protocols to secure data during synchronization. This ensures a consistent and fair environment for all users, regardless of their chosen privacy level. Currently, -this feature is under development and review for potential inclusion depending on the research output and compatibility.

D. Future plans for Nescience

Nescience is committed to continuously evolving its architecture to ensure scalability, privacy, and security in a growing blockchain landscape. +this feature is under development and review for potential inclusion depending on the research output and compatibility.

+

D. Future plans for Nescience

+

Nescience is committed to continuously evolving its architecture to ensure scalability, privacy, and security in a growing blockchain landscape. One of the primary goals is to integrate the zk-zkVM and the Nescience state-separation architecture into a fully functioning node, -enabling efficient private transactions while maintaining network integrity.

  • Addressing scalability challenges: A key challenge facing Nescience is the increasing size of nullifier and hashed UTXO trees, which could impact -network performance and scalability over time. To mitigate this, Nescience plans to adopt state-of-the-art scalable privacy techniques such as:
    • Mutator sets: Dynamically adjusting data structures to manage the growth of the nullifier set efficiently.
    • SNARK-based accumulators: Compressing data in a verifiable way to ensure that only relevant information is stored while maintaining cryptographic security.
    • Pruning techniques: Periodically trimming unnecessary data from trees to maintain optimal size and performance, ensuring that the network scales logarithmically -rather than exponentially as more transactions occur.

By implementing these approaches, Nescience aims to keep the size of its data structures manageable, ensuring that scalability does not come at the cost of performance or privacy.

  • Enhanced key management: Another critical focus for Nescience is improving key management to streamline operations and enhance security. +enabling efficient private transactions while maintaining network integrity.

    +
      +
    • Addressing scalability challenges: A key challenge facing Nescience is the increasing size of nullifier and hashed UTXO trees, which could impact +network performance and scalability over time. To mitigate this, Nescience plans to adopt state-of-the-art scalable privacy techniques such as: +
        +
      • Mutator sets: Dynamically adjusting data structures to manage the growth of the nullifier set efficiently.
        • +
      • SNARK-based accumulators: Compressing data in a verifiable way to ensure that only relevant information is stored while maintaining cryptographic security.
        • +
      • Pruning techniques: Periodically trimming unnecessary data from trees to maintain optimal size and performance, ensuring that the network scales logarithmically +rather than exponentially as more transactions occur.
        • +
      +
      • +
    +

    By implementing these approaches, Nescience aims to keep the size of its data structures manageable, ensuring that scalability does not come at the cost of performance or privacy.

    +
      +
    • +

      Enhanced key management: Another critical focus for Nescience is improving key management to streamline operations and enhance security. The plan is to integrate the different keys used for signatures, addresses, UTXO encryption, and SNARK verification into a unified system. This integration will simplify key management for users while reducing the risk of security breaches caused by complex, disparate key systems. Nescience also plans to implement Hierarchical Deterministic (HD) keys, which allow users to derive multiple keys from a single seed, enhancing both security and usability. This approach reduces the complexity of managing multiple keys across various functions and provides an additional layer of protection for private transactions. Additionally, multi-signature schemes will be introduced, requiring multiple parties to authorize transactions. -This feature increases security by reducing the likelihood of unauthorized access, ensuring that a single compromised key cannot lead to malicious transactions.

    • Integrating advanced cryptographic techniques: Nescience will integrate advanced cryptographic techniques, enhancing both privacy and scalability. Among these are:

      • Homomorphic encryption: Allowing computations to be performed on encrypted data without the need to decrypt it, preserving privacy while enabling secure, complex data processing.
      • Zero-knowledge rollups: Bundling multiple transactions into a single proof to reduce the amount of data processed and stored on the blockchain, -significantly improving scalability without sacrificing security.

    These cryptographic enhancements will ensure that Nescience can support a growing network while continuing to protect user privacy and maintaining high transaction throughput.

    • Long-term vision

    The ultimate goal for Nescience is to deploy a fully operational node powered by zk-zkVM and the Nescience state-separation architecture. +This feature increases security by reducing the likelihood of unauthorized access, ensuring that a single compromised key cannot lead to malicious transactions.

    +
    • +
  • +

    Integrating advanced cryptographic techniques: Nescience will integrate advanced cryptographic techniques, enhancing both privacy and scalability. Among these are:

    +
      +
    • Homomorphic encryption: Allowing computations to be performed on encrypted data without the need to decrypt it, preserving privacy while enabling secure, complex data processing.
      • +
    • Zero-knowledge rollups: Bundling multiple transactions into a single proof to reduce the amount of data processed and stored on the blockchain, +significantly improving scalability without sacrificing security.
      • +
    +
    • +
+

These cryptographic enhancements will ensure that Nescience can support a growing network while continuing to protect user privacy and maintaining high transaction throughput.

+
    +
  • Long-term vision
    • +
+

The ultimate goal for Nescience is to deploy a fully operational node powered by zk-zkVM and the Nescience state-separation architecture. This node will handle complex, private transactions at scale while integrating all of the advanced cryptographic techniques outlined in the roadmap. -Nescience aims to provide users with an infrastructure that balances privacy, security, and efficiency, ensuring the network remains resilient and capable of handling future demands.

By pursuing these future plans, Nescience is poised to not only address current challenges around scalability and key management but also lead the way in +Nescience aims to provide users with an infrastructure that balances privacy, security, and efficiency, ensuring the network remains resilient and capable of handling future demands.

+

By pursuing these future plans, Nescience is poised to not only address current challenges around scalability and key management but also lead the way in applying advanced cryptography to decentralized systems. This vision will help secure the long-term integrity and performance of the Nescience state-separation -model as the blockchain grows and evolves.

References

[1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

[2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrieved from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

[3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

[4] Wuille, P. (2012). Bitcoin Improvement Proposal (BIP) 32. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

[5] Sin7y Tech Review (29): Design Principles of Private Transactions in Aleo & Zcash. Retrieved from https://hackmd.io/@sin7y/rkxFXLkgs

[6] Sin7y Tech Review (33): Principles of private transactions and regulatory compliance issues. Retrieved from https://hackmd.io/@sin7y/S16RyFzZn

[7] Zcash Protocol Specification. Retrieved from https://zips.z.cash/protocol/protocol.pdf

[8] Anatomy of a Zcash Transaction. Retrieved from https://electriccoin.co/blog/anatomy-of-zcash

[9] The Penumbra Protocol: Notes, Nullifiers, and Trees. Retrieved from https://protocol.penumbra.zone/main/concepts/notes_nullifiers_trees.html

[10] Zero-knowledge Virtual Machine (ZKVM). Retrieved from https://medium.com/@abhilashkrish/zero-knowledge-virtual-machine-zkvm-95adc2082cfd

[11] What's a Sparse Merkle tree?. Retrieved from https://medium.com/@kelvinfichter/whats-a-sparse-merkle-tree-acda70aeb837

[12] Lecture 10: Accounts Model and Merkle Trees. Retrieved from https://web.stanford.edu/class/ee374/lec_notes/lec10.pdf

[13] The UTXO vs Account Model. Retrieved from https://www.horizen.io/academy/utxo-vs-account-model/

[14] Addresses and Value Pools in Zcash. Retrieved from https://zcash.readthedocs.io/en/latest/rtd_pages/addresses.html

- - +model as the blockchain grows and evolves.

+

References

+

[1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

+

[2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrieved from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

+

[3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

+

[4] Wuille, P. (2012). Bitcoin Improvement Proposal (BIP) 32. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

+

[5] Sin7y Tech Review (29): Design Principles of Private Transactions in Aleo & Zcash. Retrieved from https://hackmd.io/@sin7y/rkxFXLkgs

+

[6] Sin7y Tech Review (33): Principles of private transactions and regulatory compliance issues. Retrieved from https://hackmd.io/@sin7y/S16RyFzZn

+

[7] Zcash Protocol Specification. Retrieved from https://zips.z.cash/protocol/protocol.pdf

+

[8] Anatomy of a Zcash Transaction. Retrieved from https://electriccoin.co/blog/anatomy-of-zcash

+

[9] The Penumbra Protocol: Notes, Nullifiers, and Trees. Retrieved from https://protocol.penumbra.zone/main/concepts/notes_nullifiers_trees.html

+

[10] Zero-knowledge Virtual Machine (ZKVM). Retrieved from https://medium.com/@abhilashkrish/zero-knowledge-virtual-machine-zkvm-95adc2082cfd

+

[11] What's a Sparse Merkle tree?. Retrieved from https://medium.com/@kelvinfichter/whats-a-sparse-merkle-tree-acda70aeb837

+

[12] Lecture 10: Accounts Model and Merkle Trees. Retrieved from https://web.stanford.edu/class/ee374/lec_notes/lec10.pdf

+

[13] The UTXO vs Account Model. Retrieved from https://www.horizen.io/academy/utxo-vs-account-model/

+

[14] Addresses and Value Pools in Zcash. Retrieved from https://zcash.readthedocs.io/en/latest/rtd_pages/addresses.html

\ No newline at end of file diff --git a/rlog/archive/index.html b/rlog/archive/index.html index 6b09aca2..0965d0d9 100644 --- a/rlog/archive/index.html +++ b/rlog/archive/index.html @@ -1,26 +1,23 @@ - + - -Archive | Vac Research + +Archive | Vac Research - - - - + + + + -
-

Archive

Archive

2019

2020

2021

2022

2023

2024

2025

- - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

Archive

Archive

2025​

2024​

2023​

2022​

2021​

2020​

2019​

\ No newline at end of file diff --git a/rlog/atom.xml b/rlog/atom.xml index 35bf9ca7..e855aa0a 100644 --- a/rlog/atom.xml +++ b/rlog/atom.xml @@ -15,20 +15,25 @@ This article introduces MDSECheck method — a novel approach to checking square MDS matrices for unconditional security -as the components of affine permutation layers of P-SP-networks.

Introduction​

Maximum distance separable (MDS) matrices play a significant role +as the components of affine permutation layers of P-SP-networks.

+ +

Introduction​

+

Maximum distance separable (MDS) matrices play a significant role in algebraic coding theory and symmetric cryptography. In particular, square MDS matrices are commonly used in affine permutation layers of partial substitution-permutation networks (P-SPNs). These are widespread designs of the modern symmetric ciphers and hash functions. -A classic example of the latter is Poseidon [1], -a well-known hash function used in zk-SNARK proving systems.

Square MDS matrices differ in terms of security +A classic example of the latter is Poseidon [1], +a well-known hash function used in zk-SNARK proving systems.

+

Square MDS matrices differ in terms of security that they are able to provide for P-SPNs. The use of some such matrices in certain P-SPNs may result in existence of infinitely long subspace trails of small period for the latter, -which make them vulnerable to differential cryptanalysis [2].

Two methods for security checking of square MDS matrices for P-SPNs -have been proposed in [2]. +which make them vulnerable to differential cryptanalysis [2].

+

Two methods for security checking of square MDS matrices for P-SPNs +have been proposed in [2]. The first one, which is referred to as the three tests method in the rest of the article, is aimed at security checking for a specified structure of the substitution layer of a P-SPN. @@ -37,145 +42,180 @@ has been designed to determine whether a square MDS matrix satisfies a sufficient condition of being secure regardless of the structure of a P-SPN substitution layer, i.e. to check whether the matrix belongs to the class of square MDS matrices, which are referred to -as unconditionally secure in the current article.

This article aims to introduce MDSECheck method — +as unconditionally secure in the current article.

+

This article aims to introduce MDSECheck method — a novel approach to checking square MDS matrices for unconditional security, which has already been implemented in the Rust programming language as -the library crate [3]. +the library crate [3]. The next sections explain the notions mentioned above, describe the MDSECheck method as well as its mathematical foundations, provide a brief overview of the MDSECheck library crate -and outline possible future research directions.

MDS matrix: how to define and construct​

An mm x nn matrix MM over a finite field is called MDS, -if and only if for distinct nn-dimensional column vectors v1v_1 and v2v_2 -the column vectors v1 ∣ Mv1v_1 \: | \: M v_1 and v2 ∣ Mv2v_2 \: | \: M v_2, -where ∣| stands for vertical concatenation, -do not coincide in nn or more components. -The set of all possible column vectors v ∣ Mvv \: | \: M v for -some fixed matrix MM is a systematic MDS code, i.e. +and outline possible future research directions.

+

MDS matrix: how to define and construct​

+

An mm x nn matrix MM over a finite field is called MDS, +if and only if for distinct nn-dimensional column vectors v1v_1 and v2v_2 +the column vectors v1 ∣ Mv1v_1 \: | \: M v_1 and v2 ∣ Mv2v_2 \: | \: M v_2, +where ∣| stands for vertical concatenation, +do not coincide in nn or more components. +The set of all possible column vectors v ∣ Mvv \: | \: M v for +some fixed matrix MM is a systematic MDS code, i.e. a linear code, which contains input symbols on their original positions and achieves the Singleton bound. -The latter property results in good error-correction capability.

There are several equivalent definitions of MDS matrices, +The latter property results in good error-correction capability.

+

There are several equivalent definitions of MDS matrices, but the next one is especially useful for constructing them directly by means of algebraic methods. A matrix over a finite field is called MDS, if and only if all its square submatrices are nonsingular. -The matrix entries and the matrix itself are also considered submatrices.

One of the most efficient and straightforward methods to directly construct -an MDS matrix is generating a Cauchy matrix [4]. -Such an mm x nn matrix is defined using -mm-dimensional vector xx and nn-dimensional vector yy, -for which all entries in the concatenation of xx and yy are distinct. +The matrix entries and the matrix itself are also considered submatrices.

+

One of the most efficient and straightforward methods to directly construct +an MDS matrix is generating a Cauchy matrix [4]. +Such an mm x nn matrix is defined using +mm-dimensional vector xx and nn-dimensional vector yy, +for which all entries in the concatenation of xx and yy are distinct. The entries of the Cauchy matrix are described by the formula -Mi,j=1 / (xi−yj)M_{i, j} = 1 \: / \: (x_i - y_j). +Mi,j=1 / (xi−yj)M_{i, j} = 1 \: / \: (x_i - y_j). It is obvious that any submatrix of a Cauchy matrix is also a Cauchy matrix. -The Cauchy determinant formula [5] implies that +The Cauchy determinant formula [5] implies that every square Cauchy matrix is nonsingular. -Thus, Cauchy matrices satisfy the second definition of MDS matrices.

Partial substitution-permutation networks​

Describing SPNs in algebraic terms is convenient, +Thus, Cauchy matrices satisfy the second definition of MDS matrices.

+

Partial substitution-permutation networks​

+

Describing SPNs in algebraic terms is convenient, so this approach has been chosen for this article. SPNs are designs of the symmetric cryptoprimitives, which operate on an internal state, which is represented -as an nn-dimensional vector over some finite field, +as an nn-dimensional vector over some finite field, and update this state iteratively by means of -the round transformations described below.

Each round begins with an optional update of the internal state by +the round transformations described below.

+

Each round begins with an optional update of the internal state by adding to its components some input data or extraction of some of these components as the output data. This optional step depends on the specific cryptoprimitive and the current round number. The next step is called the nonlinear substitution layer -and lies in replacing the ii-th component of the internal state -with Si(c)S_i(c) for each i∈[1..n]i \in [1..n], where cc is the component value -and Si(x)S_i(x) is a nonlinear invertible function over the finite field. -The function Si(x)S_i(x) is specific to the cryptoprimitive and called an S-Box. +and lies in replacing the ii-th component of the internal state +with Si(c)S_i(c) for each i∈[1..n]i \in [1..n], where cc is the component value +and Si(x)S_i(x) is a nonlinear invertible function over the finite field. +The function Si(x)S_i(x) is specific to the cryptoprimitive and called an S-Box. The final step, which is known as the affine permutation layer, -replaces the internal state with MX+cM X + c, -where XX is the current internal state, -MM is a nonsingular square matrix and -cc is the vector of the round constants. -The value of cc is specific to the cryptoprimitive +replaces the internal state with MX+cM X + c, +where XX is the current internal state, +MM is a nonsingular square matrix and +cc is the vector of the round constants. +The value of cc is specific to the cryptoprimitive and the current round number, -while MM typically depends only on the cryptoprimitive. -The data flow diagram for an SPN is given below.

..................................                         
   │        │        │        │                           
   ▼        ▼        ▼        ▼                           
┌────────────────────────────────┐                         
│ Optional addition / extraction │ <─────> Input / output
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
│S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
└──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
   ▼        ▼        ▼        ▼                            
┌────────────────────────────────┐                         
│       Affine permutation       │                         
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
┌────────────────────────────────┐                         
│ Optional addition / extraction │ <─────> Input / output
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
│S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
└──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
   ▼        ▼        ▼        ▼                            
┌────────────────────────────────┐                         
│       Affine permutation       │                         
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
..................................

Partial SPNs are modifications of SPNs, +while MM typically depends only on the cryptoprimitive. +The data flow diagram for an SPN is given below.

+
..................................                         
   │        │        │        │                           
   ▼        ▼        ▼        ▼                           
┌────────────────────────────────┐                         
│ Optional addition / extraction │ <─────> Input / output
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
│S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
└──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
   ▼        ▼        ▼        ▼                            
┌────────────────────────────────┐                         
│       Affine permutation       │                         
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
┌────────────────────────────────┐                         
│ Optional addition / extraction │ <─────> Input / output
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
│S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
└──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
   ▼        ▼        ▼        ▼                            
┌────────────────────────────────┐                         
│       Affine permutation       │                         
└──┬────────┬────────┬────────┬──┘                         
   ▼        ▼        ▼        ▼                            
..................................
+

Partial SPNs are modifications of SPNs, where for certain rounds some S-Boxes are replaced with -the identity functions to reduce computational efforts [2]. +the identity functions to reduce computational efforts [2]. For example, the nonlinear substitution layers of the partial rounds of -Poseidon update only the first internal state component [1]. -In the case of P-SPNs, security considerations commonly demand to choose MM +Poseidon update only the first internal state component [1]. +In the case of P-SPNs, security considerations commonly demand to choose MM as a square MDS matrix, because these matrices provide -perfect diffusion property for the affine permutation layer [6]. +perfect diffusion property for the affine permutation layer [6]. Possessing this property means ensuring that -any two nn-dimensional internal states, -which differ in exactly tt components, +any two nn-dimensional internal states, +which differ in exactly tt components, are mapped by the affine permutation layer to -two new internal states that differ in at least n−t+1n - t + 1 components.

Square MDS matrix security check in the context of P-SPNs​

Certain square MDS matrices should not be used in certain P-SPNs +two new internal states that differ in at least n−t+1n - t + 1 components.

+

Square MDS matrix security check in the context of P-SPNs​

+

Certain square MDS matrices should not be used in certain P-SPNs to avoid making them vulnerable to differential cryptanalysis, since it may exploit the existence of infinitely long subspace trails -of small period for vulnerable P-SPNs. [2]. -Such matrices are called insecure with respect to particular P-SPNs.

An infinitely long subspace trail of period ll exists for a P-SPN, +of small period for vulnerable P-SPNs. [2]. +Such matrices are called insecure with respect to particular P-SPNs.

+

An infinitely long subspace trail of period ll exists for a P-SPN, if and only if there is a proper subspace of differences of internal state vectors, such that if for a pair of initial internal states the difference belongs to this subspace, then the difference for the new internal states, which are obtained from the initial ones -by means of the same ll-round transformation, -also belongs to this subspace [2].

Two methods for checking square MDS matrices for suitability for P-SPNs +by means of the same ll-round transformation, +also belongs to this subspace [2].

+

Two methods for checking square MDS matrices for suitability for P-SPNs in terms of existence of infinitely long subspace trails -have been proposed in [2]. +have been proposed in [2]. The three tests method is aimed at checking whether using a specified matrix for a P-SPN with a specified structure of the substitution layer -leads to existence of infinitely long subspace trails of period pp -for this P-SPN for all pp no larger than a given ll. +leads to existence of infinitely long subspace trails of period pp +for this P-SPN for all pp no larger than a given ll. The sufficient test method has been designed to determine whether a square MDS matrix satisfies a sufficient condition -of non-existence of infinitely long subspace trails of period pp -for P-SPNs using this matrix for all pp no larger than a specified ll.

The sufficient test method is a direct consequence of -Theorem 8 in [2] and consists in checking that -the minimal polynomial of the pp-th power of the tested matrix -has maximum degree and is irreducible for all p∈[1..l]p \in [1..l]. +of non-existence of infinitely long subspace trails of period pp +for P-SPNs using this matrix for all pp no larger than a specified ll.

+

The sufficient test method is a direct consequence of +Theorem 8 in [2] and consists in checking that +the minimal polynomial of the pp-th power of the tested matrix +has maximum degree and is irreducible for all p∈[1..l]p \in [1..l]. The aforesaid sufficient non-existence condition is satisfied by the matrix, -if and only if all the checks yield positive results.

It is convenient to define +if and only if all the checks yield positive results.

+

It is convenient to define the unconditional P-SPN security level of the square MDS matrix as follows: -this level is ll for the matrix MM, -if and only if the minimal polynomials of MM, M2M^2, ......, MlM^l +this level is ll for the matrix MM, +if and only if the minimal polynomials of MM, M2M^2, ......, MlM^l have maximum degree and are irreducible, -but for Ml + 1M^{l \: + \: 1} the minimal polynomial does not have this property. +but for Ml + 1M^{l \: + \: 1} the minimal polynomial does not have this property. Using this definition, the purpose of the sufficient test method can be described as checking whether the unconditional P-SPN security level of the specified matrix -is no less than a given bound.

MDSECheck method: getting rid of the matrix powers​

The MDSECheck method, whose name is derived from +is no less than a given bound.

+

MDSECheck method: getting rid of the matrix powers​

+

The MDSECheck method, whose name is derived from the words "MDS", "security", "elaborated" and "check", has the same purpose as the sufficient test method, but achieves it differently. The differences of the first method from the latter -and approaches to implementing them can be described as follows:

  1. Computation and verification of minimal polynomials -of M2M^2, M3M^3, ......, MlM^l, where MM is -the tested nn x nn matrix over GF(q)GF(q) -and ll is the security level bound, +and approaches to implementing them can be described as follows:

    +
      +
    1. +

      Computation and verification of minimal polynomials +of M2M^2, M3M^3, ......, MlM^l, where MM is +the tested nn x nn matrix over GF(q)GF(q) +and ll is the security level bound, has been replaced with checks for the corresponding powers -of a root of the characteristic polynomial of MM -for non-presence in nontrivial subfields of GF(qn)GF(q^n).

      1. The non-presence check is performed without -straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n). +of a root of the characteristic polynomial of MM +for non-presence in nontrivial subfields of GF(qn)GF(q^n).

        +
          +
        1. +

          The non-presence check is performed without +straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n). The root is checked only for non-presence in the subfields -GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, -GF(qn / pd)GF(q^{n \: / \: p_d}), where p1p_1, p2p_2, ......, pdp_d -are all prime divisors of nn.

        2. The non-presence check reuses some data computed during -the checking for irreducibility the minimal polynomial of MM, -which in this case coincides with f(y)f(y) -designating the characteristic polynomial of MM. -The values of yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y) are saved -for each j∈[1..d]j \in [1..d] during the irreducibility check +GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, +GF(qn / pd)GF(q^{n \: / \: p_d}), where p1p_1, p2p_2, ......, pdp_d +are all prime divisors of nn.

          +
          3. +
        3. +

          The non-presence check reuses some data computed during +the checking for irreducibility the minimal polynomial of MM, +which in this case coincides with f(y)f(y) +designating the characteristic polynomial of MM. +The values of yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y) are saved +for each j∈[1..d]j \in [1..d] during the irreducibility check to replace exponentiations with sequential computations -of (yi)qn / pjmod  f(y)(y^i)^{q^{n \: / \: p_j}} \mod f(y) from -(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) -as its product with yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y).

      2. The check of the minimal polynomial of MM +of (yi)qn / pjmod  f(y)(y^i)^{q^{n \: / \: p_j}} \mod f(y) from +(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) +as its product with yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y).

        +
        3. +
      +
      2. +
    2. +

      The check of the minimal polynomial of MM for irreducibility and maximum degree is performed without unconditional computation of this polynomial. This computation has been replaced with the Krylov method fragment, which consists in building and solving -only one system of linear equations over GF(q)GF(q). -If MM has an irreducible minimal polynomial of maximum degree, +only one system of linear equations over GF(q)GF(q). +If MM has an irreducible minimal polynomial of maximum degree, then its coefficients are trivially determined from the system solution. If the system is degenerate, -then the minimal polynomial of MM does not have such properties.

    The correctness of the first distinctive feature can be proven as follows. +then the minimal polynomial of MM does not have such properties.

    +
    2. +
+

The correctness of the first distinctive feature can be proven as follows. Verifying that the minimal polynomial of a matrix is of maximum degree and irreducible is equivalent to verifying that @@ -183,145 +223,195 @@ the characteristic polynomial of this matrix is irreducible, because the minimal polynomial divides the characteristic one. Also, it is trivially proven that for a matrix with such a minimal polynomial it is equal to the characteristic polynomial. -Thus, the required checks for the matrices M2M^2, M3M^3, ......, MlM^l -can be done by checking their characteristic polynomials for irreducibility.

Let MM be nn x nn matrix over GF(q)GF(q), +Thus, the required checks for the matrices M2M^2, M3M^3, ......, MlM^l +can be done by checking their characteristic polynomials for irreducibility.

+

Let MM be nn x nn matrix over GF(q)GF(q), whose minimal polynomial is of maximum degree and irreducible. -The statements in the previous paragraph imply that f(y)f(y), -which is the nn-degree characteristic polynomial of MM, is irreducible. -Consider MM over the extension field GF(qn)GF(q^n), -which is the splitting field of f(y)f(y). -Let α∈GF(qn)α \in GF(q^n) be a root of f(y)f(y). +The statements in the previous paragraph imply that f(y)f(y), +which is the nn-degree characteristic polynomial of MM, is irreducible. +Consider MM over the extension field GF(qn)GF(q^n), +which is the splitting field of f(y)f(y). +Let α∈GF(qn)α \in GF(q^n) be a root of f(y)f(y). According to standard results from the Galois field theory, -αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}} -are distinct roots of f(y)f(y) [7]. -Thus, these powers of αα are nn distinct eigenvalues of MM. -Hence, due to matrix similarity properties, there is some matrix SS -such that SMS−1=DS M S^{-1} = D, where DD is the diagonal matrix, +αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}} +are distinct roots of f(y)f(y) [7]. +Thus, these powers of αα are nn distinct eigenvalues of MM. +Hence, due to matrix similarity properties, there is some matrix SS +such that SMS−1=DS M S^{-1} = D, where DD is the diagonal matrix, whose nonzero elements are -αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}}. -Therefore, SMiS−1=DiS M^i S^{-1} = D^i, -so the roots of the characteristic polynomial of MiM^i -are αiα^i, (αq)i(α^q)^i, (αq2)i(α^{q^2})^i, ......, (αqn − 1)i(α^{q^{n \: - \: 1}})^i. -If the minimal polynomial of αiα^i has degree less than nn, -then the characteristic polynomial of MiM^i is divisible +αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}}. +Therefore, SMiS−1=DiS M^i S^{-1} = D^i, +so the roots of the characteristic polynomial of MiM^i +are αiα^i, (αq)i(α^q)^i, (αq2)i(α^{q^2})^i, ......, (αqn − 1)i(α^{q^{n \: - \: 1}})^i. +If the minimal polynomial of αiα^i has degree less than nn, +then the characteristic polynomial of MiM^i is divisible by this minimal polynomial, -while αiα^i lies in some nontrivial subfield of GF(qn)GF(q^n). +while αiα^i lies in some nontrivial subfield of GF(qn)GF(q^n). One of the fields isomorphic to this subfield is a residue class ring -of polynomials modulo the minimal polynomial of αiα^i [7]. -If the minimal polynomial of αiα^i is of degree nn, -then the characteristic polynomial of MiM^i +of polynomials modulo the minimal polynomial of αiα^i [7]. +If the minimal polynomial of αiα^i is of degree nn, +then the characteristic polynomial of MiM^i equals this minimal polynomial and therefore is irreducible, -while αiα^i does not lie in any nontrivial subfield of GF(qn)GF(q^n). -In this case, 11, αiα^i, (αi)2(α^i)^2, ......, (αi)n − 1(α^i)^{n \: - \: 1} +while αiα^i does not lie in any nontrivial subfield of GF(qn)GF(q^n). +In this case, 11, αiα^i, (αi)2(α^i)^2, ......, (αi)n − 1(α^i)^{n \: - \: 1} are linearly independent as distinct roots of -an irreducible polynomial over a finite field [7], -so any field containing αiα^i has at least qnq^n elements -and therefore cannot be a trivial subfield of GF(qn)GF(q^n). +an irreducible polynomial over a finite field [7], +so any field containing αiα^i has at least qnq^n elements +and therefore cannot be a trivial subfield of GF(qn)GF(q^n). Thus, checking the characteristic polynomials of -the matrices M2M^2, M3M^3, ......, MlM^l for irreducibility -is equivalent to verifying that α2α^2, α3α^3, ......, αlα^l -do not lie in any nontrivial subfield of GF(qn)GF(q^n).

The last sentences of the two previous paragraphs imply the following: -verifying that the minimal polynomials of M2M^2, M3M^3, ......, MlM^l +the matrices M2M^2, M3M^3, ......, MlM^l for irreducibility +is equivalent to verifying that α2α^2, α3α^3, ......, αlα^l +do not lie in any nontrivial subfield of GF(qn)GF(q^n).

+

The last sentences of the two previous paragraphs imply the following: +verifying that the minimal polynomials of M2M^2, M3M^3, ......, MlM^l are of maximum degree and irreducible can be performed by verifying that the corresponding powers of a root of -the characteristic polynomial of the nn x nn matrix MM over GF(q)GF(q) -do not belong to any nontrivial subfield of GF(qn)GF(q^n). ■\blacksquare

The approaches to implementing the first distinctive feature +the characteristic polynomial of the nn x nn matrix MM over GF(q)GF(q) +do not belong to any nontrivial subfield of GF(qn)GF(q^n). ■\blacksquare

+

The approaches to implementing the first distinctive feature can be explained and proven to be correct as follows. -Since GF(qw)GF(q^w) is a nontrivial subfield of GF(qu)GF(q^u) -if and only if ww divides uu and w<uw < u [7], -the presence of some εε in GF(qh)GF(q^h), -which is a nontrivial subfield of GF(qn)GF(q^n), -implies that ε∈GF(qn / ν)ε \in GF(q^{n \: / \: ν}) for some prime νν dividing nn, -because hh divides the quotient of nn and some of its prime factors. +Since GF(qw)GF(q^w) is a nontrivial subfield of GF(qu)GF(q^u) +if and only if ww divides uu and w<uw < u [7], +the presence of some εε in GF(qh)GF(q^h), +which is a nontrivial subfield of GF(qn)GF(q^n), +implies that ε∈GF(qn / ν)ε \in GF(q^{n \: / \: ν}) for some prime νν dividing nn, +because hh divides the quotient of nn and some of its prime factors. Thus, checking that some value does not belong to subfields -GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, -GF(qn / pd)GF(q^{n \: / \: p_d}), -where p1p_1, p2p_2, ......, pdp_d are all prime divisors of nn, +GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, +GF(qn / pd)GF(q^{n \: / \: p_d}), +where p1p_1, p2p_2, ......, pdp_d are all prime divisors of nn, is equivalent to checking this value for -non-presence in nontrivial subfields of GF(qn)GF(q^n).

Checking for irreducibility the minimal polynomial of MM -is performed by means of Algorithm 2.2.9 in [8] +non-presence in nontrivial subfields of GF(qn)GF(q^n).

+

Checking for irreducibility the minimal polynomial of MM +is performed by means of Algorithm 2.2.9 in [8] and consists in sequential computation of -ypmod  f(y)y^p \mod f(y), yp2mod  f(y)y^{p^2} \mod f(y), ......, -yp⌊n / 2⌋mod  f(y)y^{p^{\lfloor n \: / \: 2 \rfloor}} \mod f(y) -and checking that GCD(ypimod  f(y) − y,f(y))=1GCD(y^{p^i} \mod f(y) \: - \: y, f(y)) = 1 -for each i∈[1..⌊n / 2⌋]i \in [1..\lfloor n \: / \: 2 \rfloor], -where f(y)f(y) is the characteristic polynomial of MM +ypmod  f(y)y^p \mod f(y), yp2mod  f(y)y^{p^2} \mod f(y), ......, +yp⌊n / 2⌋mod  f(y)y^{p^{\lfloor n \: / \: 2 \rfloor}} \mod f(y) +and checking that GCD(ypimod  f(y) − y,f(y))=1GCD(y^{p^i} \mod f(y) \: - \: y, f(y)) = 1 +for each i∈[1..⌊n / 2⌋]i \in [1..\lfloor n \: / \: 2 \rfloor], +where f(y)f(y) is the characteristic polynomial of MM and coincides with the minimal polynomial in this case. The optimized root non-presence check is performed -by checking that for each i∈[2..l]i \in [2..l] for each j∈[1..d]j \in [1..d] -the value of ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) is nonzero. +by checking that for each i∈[2..l]i \in [2..l] for each j∈[1..d]j \in [1..d] +the value of ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) is nonzero. This approach is based on the following standard results -from the Galois field theory [7]:

  • GF(qn)GF(q^n) is isomorphic to the residue class ring of -univariate polynomials in yy modulo f(y)f(y), -because at this point f(y)f(y) is known to be irreducible, -and some root of f(y)f(y) is mapped by this isomorphism to -the residue class the polynomial yy in this ring.

  • All elements of a finite field GF(qw)GF(q^w) and only they -are roots of yqw−yy^{q^w} - y.

The expression ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) -can be rewritten as ((yqn / pj)i−yi)mod  f(y)((y^{q^{n \: / \: p_j}})^i - y^i) \mod f(y), +from the Galois field theory [7]:

+
    +
  • +

    GF(qn)GF(q^n) is isomorphic to the residue class ring of +univariate polynomials in yy modulo f(y)f(y), +because at this point f(y)f(y) is known to be irreducible, +and some root of f(y)f(y) is mapped by this isomorphism to +the residue class the polynomial yy in this ring.

    +
    • +
  • +

    All elements of a finite field GF(qw)GF(q^w) and only they +are roots of yqw−yy^{q^w} - y.

    +
    • +
+

The expression ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) +can be rewritten as ((yqn / pj)i−yi)mod  f(y)((y^{q^{n \: / \: p_j}})^i - y^i) \mod f(y), which can be computed without exponentiation as the product of -(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) and -yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y), -which has been saved during the irreducibility check.

The second distinctive feature can be +(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) and +yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y), +which has been saved during the irreducibility check.

+

The second distinctive feature can be explained and proven to be correct in following way. -The nn x nn matrix MM does not have a minimal polynomial of maximum degree, -if some Krylov subspace of order nn for it is not nn-dimensional. +The nn x nn matrix MM does not have a minimal polynomial of maximum degree, +if some Krylov subspace of order nn for it is not nn-dimensional. Indeed, the minimal polynomial of the matrix is divisible by the minimal polynomial of the restriction of this linear operator to an arbitrary subspace, -and in the considered case the latter polynomial has degree less than nn, +and in the considered case the latter polynomial has degree less than nn, because the degree of the minimal polynomial of a linear operator cannot exceed the dimension of the subspace the operator acts on. -Thus, an unconditional computation of the minimal polynomial of MM +Thus, an unconditional computation of the minimal polynomial of MM is not required to determine whether this polynomial is irreducible and has maximum degree. Using this computation has been replaced with the Krylov method fragment, -which consists in choosing any nonzero nn-dimensional column vector vv -and solving the system of linear equations AX=bA X = b, -where AA is an nn x nn matrix, -whose columns are vv, MvM v, M2vM^2 v, ......, Mn − 1vM^{n \: - \: 1} v, -and bb is MnvM^n v. -If AA is singular, -the minimal polynomial of MM is reducible or does not have maximum degree, -so checking MM has been accomplished; -otherwise, f(y)f(y), which is the minimal and characteristic polynomial of MM, -can be expressed as yn−Xn − 1yn − 1−Xn − 2yn − 2−…−X1y−X0y^n - X_{n \: - \: 1} y^{n \: - \: 1} - X_{n \: - \: 2} y^{n \: - \: 2} - … - X_1 y - X_0.

The steps of MDSECheck method can be summarized as follows:

  1. The square MDS matrix MM over GF(q)GF(q) -and the unconditional P-SPN security level bound ll are received as inputs.

  2. The Krylov method fragment is used to compute the minimal polynomial of MM. -If the computation fails, then MM is not unconditionally secure, -so the check of MM is complete. +which consists in choosing any nonzero nn-dimensional column vector vv +and solving the system of linear equations AX=bA X = b, +where AA is an nn x nn matrix, +whose columns are vv, MvM v, M2vM^2 v, ......, Mn − 1vM^{n \: - \: 1} v, +and bb is MnvM^n v. +If AA is singular, +the minimal polynomial of MM is reducible or does not have maximum degree, +so checking MM has been accomplished; +otherwise, f(y)f(y), which is the minimal and characteristic polynomial of MM, +can be expressed as yn−Xn − 1yn − 1−Xn − 2yn − 2−…−X1y−X0y^n - X_{n \: - \: 1} y^{n \: - \: 1} - +X_{n \: - \: 2} y^{n \: - \: 2} - … - X_1 y - X_0.

    +

    The steps of MDSECheck method can be summarized as follows:

    +
      +
    1. +

      The square MDS matrix MM over GF(q)GF(q) +and the unconditional P-SPN security level bound ll are received as inputs.

      +
      2. +
    2. +

      The Krylov method fragment is used to compute the minimal polynomial of MM. +If the computation fails, then MM is not unconditionally secure, +so the check of MM is complete. If it succeeds, then the minimal polynomial has maximum degree -and, therefore, coincides with the characteristic polynomial of MM.

    3. Algorithm 2.2.9 is used -to check for irreducibility the minimal polynomial of MM, -which is also the characteristic polynomial of MM in this case. +and, therefore, coincides with the characteristic polynomial of MM.

      +
      4. +
    4. +

      Algorithm 2.2.9 is used +to check for irreducibility the minimal polynomial of MM, +which is also the characteristic polynomial of MM in this case. Some data computed during this step is saved to be reused at the next one. -If the polynomial is reducible, then the check of MM is complete, -because MM has been found to be not unconditionally secure.

    5. The values of α2α^2, α3α^3, ......, αlα^l, -where αα is a root of the characteristic polynomial of MM, +If the polynomial is reducible, then the check of MM is complete, +because MM has been found to be not unconditionally secure.

      +
      6. +
    6. +

      The values of α2α^2, α3α^3, ......, αlα^l, +where αα is a root of the characteristic polynomial of MM, are sequentially checked for non-presence in -nontrivial subfields of GF(qn)GF(q^n) as described above. -If αiα^i belongs to some nontrivial subfield of GF(qn)GF(q^n), -then the unconditional P-SPN security level of MM is i − 1i \: - \: 1, -so the check of MM is complete. +nontrivial subfields of GF(qn)GF(q^n) as described above. +If αiα^i belongs to some nontrivial subfield of GF(qn)GF(q^n), +then the unconditional P-SPN security level of MM is i − 1i \: - \: 1, +so the check of MM is complete. If all the values do not belong to such a subfield, -then the unconditional P-SPN security level is at least ll.

    MDSECheck library crate: implementation in Rust​

    The library crate [3] provides tools for +then the unconditional P-SPN security level is at least ll.

    +
    3. +
+

MDSECheck library crate: implementation in Rust​

+

The library crate [3] provides tools for generating random square Cauchy MDS matrices over prime finite fields and applying the MDSECheck method to check such matrices for unconditional security. The used data types of field elements and polynomials are provided by -the crates ark-ff [9] and ark-poly [10]. -The auxiliary tools in the crate modules are accessible as well.

Generating by means of this crate a 10 x 10 MDS matrix, -which is defined over the BN254 scalar field [11] +the crates ark-ff [9] and ark-poly [10]. +The auxiliary tools in the crate modules are accessible as well.

+

Generating by means of this crate a 10 x 10 MDS matrix, +which is defined over the BN254 scalar field [11] and has unconditional P-SPN security level is 1000, takes less than 60 milliseconds on average for the laptop with the processor Intel® Core™ i9-14900HX, -whose maximum clock frequency is 5.8 GHz.

Conclusion​

The MDSECheck method proposed in this article is a novel approach +whose maximum clock frequency is 5.8 GHz.

+

Conclusion​

+

The MDSECheck method proposed in this article is a novel approach to checking square MDS matrices for unconditional security as the components of affine permutation layers of P-SPNs. It has been implemented as a practical library crate for generating unconditionally secure square MDS matrices -for P-SPNs over prime finite fields.

The future research directions may include +for P-SPNs over prime finite fields.

+

The future research directions may include theoretical and experimental studies of performance of approaches, which use the MDSECheck method -to generate unconditionally secure square MDS matrices for P-SPNs.

References​

  1. L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, M. Schofnegger. "POSEIDON: A New Hash Function for Zero-Knowledge Proof Systems (Updated Version)".
  2. L. Grassi, C. Rechberger, M. Schofnegger. "Proving Resistance Against Infinitely Long Subspace Trails: How to Choose the Linear Layer".
  3. The page "mdsecheck" on crates.io.
  4. Y. Kumar, P. Mishra, S. Samanta, K. Chand Gupta, A. Gaur. "Construction of all MDS and involutory MDS matrices".
  5. The page "Value of Cauchy Determinant" on proofwiki.org.
  6. T. Silva, R. Dahab "MDS Matrices for Cryptography".
  7. S. Huczynska, M. Neunhöffer. "Finite Fields"
  8. R. Crandall, C. Pomerance. "Prime Numbers: A Computational Perspective" (2nd edition).
  9. The page "ark-ff" on crates.io.
  10. The page "ark-poly" on crates.io.
  11. The page "ark-bn254" on crates.io.
]]> +to generate unconditionally secure square MDS matrices for P-SPNs.

+

References​

+
    +
  1. L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, M. Schofnegger. "POSEIDON: A New Hash Function for Zero-Knowledge Proof Systems (Updated Version)".
    2. +
  2. L. Grassi, C. Rechberger, M. Schofnegger. "Proving Resistance Against Infinitely Long Subspace Trails: How to Choose the Linear Layer".
    3. +
  3. The page "mdsecheck" on crates.io.
    4. +
  4. Y. Kumar, P. Mishra, S. Samanta, K. Chand Gupta, A. Gaur. "Construction of all MDS and involutory MDS matrices".
    5. +
  5. The page "Value of Cauchy Determinant" on proofwiki.org.
    6. +
  6. T. Silva, R. Dahab "MDS Matrices for Cryptography".
    7. +
  7. S. Huczynska, M. Neunhöffer. "Finite Fields"
    8. +
  8. R. Crandall, C. Pomerance. "Prime Numbers: A Computational Perspective" (2nd edition).
    9. +
  9. The page "ark-ff" on crates.io.
    10. +
  10. The page "ark-poly" on crates.io.
    11. +
  11. The page "ark-bn254" on crates.io.
    12. +
]]> Aleksei Vambol @@ -332,45 +422,223 @@ to generate unconditionally secure square MDS matrices for P-SPNs.

2025-01-09T18:30:00.000Z - In this post, we recap Vac's achievements in 2024 and look forward to 2025.

With 2024 now behind us and a new year ahead, -Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem.

Vac comprises various subteams and service units, each with its own focus. -Below, we celebrate each unit's achievements and look forward to its 2025 plans.

Nescience​

Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.

Highlights​

This year, the Nescience state separation architecture moved from exploration to real progress, + In this post, we recap Vac's achievements in 2024 and look forward to 2025.

+ +

With 2024 now behind us and a new year ahead, +Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem.

+

Vac comprises various subteams and service units, each with its own focus. +Below, we celebrate each unit's achievements and look forward to its 2025 plans.

+

Nescience​

+

Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.

+

Highlights​

+

This year, the Nescience state separation architecture moved from exploration to real progress, taking significant steps towards building a functional and reliable system. The team focused on turning ideas into something real, testing the proposed architecture, -and understanding its strengths and weaknesses.

We also made progress on the essential parts of NSSA’s system, including:

  • Key protocol for secure key management
  • Execution types and circuits for reliable computation
  • UTXO specification to manage state transitions effectively
  • Cryptography module to ensure privacy and security

Looking forward​

In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use.

  • Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement.
  • Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness).
  • Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem.
  • Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact.
  • Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.

Token Economics (TKE)​

The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.

Highlights​

  • Formalized and implemented Codex economic incentives in the Litepaper and simulations
  • Orchestrated Status Network incentive structure and smart contract implementation
  • Started building Nomos’s economic model
  • Consulted and provided analysis of incentives for the Logos Operators ordinals project
  • Drove discussions on the economic sustainability of Waku; -helped define RLN membership and its payment mechanism

Looking forward​

In 2025, TKE will continue to support IFT portfolio projects, +and understanding its strengths and weaknesses.

+
    +
  • ZkVM exploration and benchmarks + +
    • +
  • Defined the NSSA architecture + +
    • +
  • Built the sandboxed testnet +
      +
    • Designed the first version of the node specification
      • +
    • All core components (execution types, UTXOs, cryptographic primitives) implemented and being tested
      • +
    • Testing the performance of all execution types in various scenarios
      • +
    +
    • +
+

We also made progress on the essential parts of NSSA’s system, including:

+
    +
  • Key protocol for secure key management
    • +
  • Execution types and circuits for reliable computation
    • +
  • UTXO specification to manage state transitions effectively
    • +
  • Cryptography module to ensure privacy and security
    • +
+

Looking forward​

+

In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use.

+
    +
  • Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement.
    • +
  • Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness).
    • +
  • Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem.
    • +
  • Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact.
    • +
  • Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.
    • +
+

Token Economics (TKE)​

+

The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.

+

Highlights​

+
    +
  • Formalized and implemented Codex economic incentives in the Litepaper and simulations
    • +
  • Orchestrated Status Network incentive structure and smart contract implementation
    • +
  • Started building Nomos’s economic model
    • +
  • Consulted and provided analysis of incentives for the Logos Operators ordinals project
    • +
  • Drove discussions on the economic sustainability of Waku; +helped define RLN membership and its payment mechanism
    • +
+

Looking forward​

+

In 2025, TKE will continue to support IFT portfolio projects, working toward economic sustainability while strengthening relationships within the organization. -Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.

Quality Assurance (QA)​

The QA Service Unit focuses on the development and execution of comprehensive test plans, -including implementing unit and interoperability testing.

Highlights​

  • Matured Waku interoperability testing framework with coverage for all major protocols and features
  • Began collaboration with Nomos, contributing to unit and integration testing
  • Partnered with the Status team to test message reliability under unstable network conditions

Looking forward​

  • Extend collaboration with the Waku team on go-waku bindings and message reliability testing
  • Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation
  • Work closely with Status’s QA team to enhance the functional testing framework
  • Continue work on nim-libp2p testing
  • Expand collaboration to additional projects

RFC​

The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. -The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.

Highlights​

  • Working to implement RFC culture across the IFT ecosystem
  • Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex.
  • Reworked our standards with regard to writing RFCs to a consensus-oriented specification system

Looking forward​

  • Continue to implement RFC culture across the IFT ecosystem
  • Broaden the number of RFCs produced -– particularly for IFT portfolio projects nearing public releases
  • Include new projects with the rfc-index
  • Encourage external projects requiring RFCs to establish relationships with the service unit

Applied Cryptography and ZK (ACZ)​

The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, +Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.

+

Quality Assurance (QA)​

+

The QA Service Unit focuses on the development and execution of comprehensive test plans, +including implementing unit and interoperability testing.

+

Highlights​

+
    +
  • Matured Waku interoperability testing framework with coverage for all major protocols and features
    • +
  • Began collaboration with Nomos, contributing to unit and integration testing
    • +
  • Partnered with the Status team to test message reliability under unstable network conditions
    • +
+

Looking forward​

+
    +
  • Extend collaboration with the Waku team on go-waku bindings and message reliability testing
    • +
  • Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation
    • +
  • Work closely with Status’s QA team to enhance the functional testing framework
    • +
  • Continue work on nim-libp2p testing
    • +
  • Expand collaboration to additional projects
    • +
+

RFC​

+

The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. +The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.

+

Highlights​

+
    +
  • Working to implement RFC culture across the IFT ecosystem
    • +
  • Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex.
    • +
  • Reworked our standards with regard to writing RFCs to a consensus-oriented specification system
    • +
+

Looking forward​

+
    +
  • Continue to implement RFC culture across the IFT ecosystem
    • +
  • Broaden the number of RFCs produced +– particularly for IFT portfolio projects nearing public releases
    • +
  • Include new projects with the rfc-index
    • +
  • Encourage external projects requiring RFCs to establish relationships with the service unit
    • +
+

Applied Cryptography and ZK (ACZ)​

+

The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, enhancing the security, privacy, and trustworthiness of IFT portfolio projects -and contributing to the overall integrity and resilience of the decentralized web ecosystem.

Highlights​

  • Researched a libp2p mix protocol and first proof-of-concept implementation (including ping and GossipSub over mix)
  • Researched a decentralized version of MLS (message layer security) with a first proof of concept
  • Released Zerokit v0.6.0 -and v0.5.0
  • Added gnark RLN implementation
  • Released Stealth Address Kit v0.3.1, +and contributing to the overall integrity and resilience of the decentralized web ecosystem.

    +

    Highlights​

    +

    Looking forward​

    • Ensure libp2p mix protocol is production-ready -and support with the publishing of a paper and blog posts
    • Ensure decentralized MLS is production-ready -and support with the publishing of a paper and blog posts
    • Begin explorations of additional research topics
    • Release Zerokit v0.7 and future versions

    P2P​

    The P2P Service Unit specializes in peer-to-peer technologies -and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.

    Highlights​

    Looking forward​

    • Add new features to nim-libp2p: -QUIC transport, web transport
    • Update specifications for libp2p GossipSub, -aiming to significantly improve its performance

    Distributed Systems Testing (DST)​

    The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. -By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.

    Highlights​

    +

    Looking forward​

    +
      +
    • Ensure libp2p mix protocol is production-ready +and support with the publishing of a paper and blog posts
      • +
    • Ensure decentralized MLS is production-ready +and support with the publishing of a paper and blog posts
      • +
    • Begin explorations of additional research topics
      • +
    • Release Zerokit v0.7 and future versions
      • +
    +

    P2P​

    +

    The P2P Service Unit specializes in peer-to-peer technologies +and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.

    +

    Highlights​

    + +

    Looking forward​

    +
      +
    • Add new features to nim-libp2p: +QUIC transport, web transport
      • +
    • Update specifications for libp2p GossipSub, +aiming to significantly improve its performance
      • +
    +

    Distributed Systems Testing (DST)​

    +

    The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. +By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.

    +

    Highlights​

    +
      +
    • DST compute resources transitioned from a hosted environment to a dedicated Vac Lab, enabling better customization of resources and adding significantly more compute power -– enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control.
    • Maintained monthly regression simulations for both Waku and Nim-libp2p, -helping us to detect several issues and ensure that future versions do not introduce new ones.
    • Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.

    Looking forward​

    • More testing and simulations for Codex and Nomos
    • Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard

    Nim​

    Several IFT portfolio projects use the Nim ecosystem for its efficiency. -The Nim Service Unit is responsible for the development and maintenance of Nim tooling.

    Highlights​

    • Released Nim-libp2p (v1.7.1, +– enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control.
      • +
    • Maintained monthly regression simulations for both Waku and Nim-libp2p, +helping us to detect several issues and ensure that future versions do not introduce new ones.
      • +
    • Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.
      • +
    +

    Looking forward​

    +
      +
    • More testing and simulations for Codex and Nomos
      • +
    • Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard
      • +
    +

    Nim​

    +

    Several IFT portfolio projects use the Nim ecosystem for its efficiency. +The Nim Service Unit is responsible for the development and maintenance of Nim tooling.

    +

    Highlights​

    +
      +
    • Released Nim-libp2p (v1.7.1, v1.7.0, v1.6.0, v1.5.0, v1.4.0, v1.3.0, -v1.2.0)
    • Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution
    • Nim VSCode Extension
    • Stabilized Nim Language Server

    Smart Contracts (SC)​

    Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.

    Highlights​

    • Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network
    • Wrote specifications for Codex's architectural components and Status's staking contracts
    • Delivered several learn-up sessions on a variety of topics for IFT contributors, including:
      • Stealth addresses
      • Tokenized vaults
      • Rental NFTs
      • Merkle trees
      • Account abstraction
      • EVM deep dive

    Looking forward​

    • Deploy the SNT staking protocol on the Status Network testnet
    • Encourage community security audits via contests
    • Provide smart contract consultation services for IFT portfolio products
    • Engage in more learn-up sessions to promote org-wide knowledge sharing.

    Heading into 2025​

    This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. -The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit.

    As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. +v1.2.0)

    • +
  • Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution
    • +
  • Nim VSCode Extension
    • +
  • Stabilized Nim Language Server
    • +
+

Smart Contracts (SC)​

+

Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.

+

Highlights​

+
    +
  • Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network
    • +
  • Wrote specifications for Codex's architectural components and Status's staking contracts
    • +
  • Delivered several learn-up sessions on a variety of topics for IFT contributors, including: +
      +
    • Stealth addresses
      • +
    • Tokenized vaults
      • +
    • Rental NFTs
      • +
    • Merkle trees
      • +
    • Account abstraction
      • +
    • EVM deep dive
      • +
    +
    • +
+

Looking forward​

+
    +
  • Deploy the SNT staking protocol on the Status Network testnet
    • +
  • Encourage community security audits via contests
    • +
  • Provide smart contract consultation services for IFT portfolio products
    • +
  • Engage in more learn-up sessions to promote org-wide knowledge sharing.
    • +
+

Heading into 2025​

+

This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. +The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit.

+

As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. Our goal is to serve as an internal conduit of expertise within the organization, supported by a strong RFC culture, maintaining a repository of internal knowledge creation, and identifying and facilitating IFT project synergies. -Such an approach should lead to greater efficiencies across the organization.

We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. +Such an approach should lead to greater efficiencies across the organization.

+

We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. In the final quarter of 2024, Vac stepped up its collaboration with the libp2p community and made a concerted effort to engage the community on the Vac forum. -In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems.

We look forward to continuing our journey with you!

Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.

]]> +In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems.

+

We look forward to continuing our journey with you!

+

Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.

]]> Vac @@ -381,94 +649,192 @@ In 2025, we aim to continue working closely with those communities to which we a 2024-12-30T12:00:00.000Z - In this post, we introduce a crucial data structure used throughout web3.

Introduction​

A large amount of data is swapped between users on a blockchain in the form of transactions. + In this post, we introduce a crucial data structure used throughout web3.

+ +

Introduction​

+

A large amount of data is swapped between users on a blockchain in the form of transactions. Over the entire life of a blockchain, the storage space required to maintain a copy of every transaction becomes untenable for most users. However, the integrity of a blockchain relies on a large pool of users that can validate the blockchain's history from its inception to its present state. The data representing the blockchain's state is compressed. -This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users.

Data compression alone is not the end goal. +This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users.

+

Data compression alone is not the end goal. As mentioned, it is essential for users to be able to validate the blockchain's history. The property of compression and validation was solved in Bitcoin by the use of Merkle trees. Merkle trees were introduced first by Ralph Merkle in his dissertation [1]. A Merkle tree is a data structure that compresses a digest of data to a constant size while still providing a method for proving membership of elements of the digest. -A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.

Tree structure​

A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. +A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.

+

Tree structure​

+

A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. The trees that we consider can be arranged in layers with multiple nodes (children) merged into a single node (parent) in the preceding layer. A single node exists in the base layer; this special node is called the root node. -The highest level of the tree consists of childless nodes called leaves.

Image 1

A binary tree has one additional property: +The highest level of the tree consists of childless nodes called leaves.

+

Image 1

+

A binary tree has one additional property: each nonleaf node has exactly two children nodes. That is, we assume that nodes in a binary tree are either a parent node with two children or a leaf. -As strange as it sounds, each child node has exactly one parental node.

Image 2

A binary tree with 2n2^n leaves consists of n+1n+1 layers. -Additionally, such a tree has 2n+1−12^{n+1}-1 nodes.

Merkle trees​

A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. +As strange as it sounds, each child node has exactly one parental node.

+

Image 2

+

A binary tree with 2n2^n leaves consists of n+1n+1 layers. +Additionally, such a tree has 2n+1−12^{n+1}-1 nodes.

+

Merkle trees​

+

A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. Merkle trees are usually taken to have a binary tree structure. -As such, the presentation we provide in this section will be for binary trees.

Construction​

In this section, we show how Merkle trees are constructed to compress a digest DD. -Suppose that the digest DD consists of 2n2^n entries; -we assume that the digest DD has this many entries since a Merkle tree is a binary tree. -Additionally, each digest can be padded to ensure that DD has the desired number of entries.

Each leaf of the Merkle tree contains the hash of a digest entry. +As such, the presentation we provide in this section will be for binary trees.

+

Construction​

+

In this section, we show how Merkle trees are constructed to compress a digest DD. +Suppose that the digest DD consists of 2n2^n entries; +we assume that the digest DD has this many entries since a Merkle tree is a binary tree. +Additionally, each digest can be padded to ensure that DD has the desired number of entries.

+

Each leaf of the Merkle tree contains the hash of a digest entry. Each parent node contains the hash of the concatenation of their child nodes. Through this iterative construction, we reach the root of the tree. The value contained in the root node is called the root hash. -The root hash is a compressed representation of the digest DD.

Image 3

Each node in the Merkle tree is computed by taking a hash. -Since a binary tree with 2n2^n leaves has 2n+1−12^{n+1}-1 nodes, -then we need to evaluate 2n+1−12^{n+1}-1 hashes to construct the Merkle tree.

Merkle tree intregrity​

A large quantity of data can be compressed to a single hash value. +The root hash is a compressed representation of the digest DD.

+

Image 3

+

Each node in the Merkle tree is computed by taking a hash. +Since a binary tree with 2n2^n leaves has 2n+1−12^{n+1}-1 nodes, +then we need to evaluate 2n+1−12^{n+1}-1 hashes to construct the Merkle tree.

+

Merkle tree intregrity​

+

A large quantity of data can be compressed to a single hash value. A natural question to ask is: could a clever party find another digest that yields a Merkle tree with the same root hash? If possible, this would compromise the ledger since the blockchain's history could be altered. Fortunately, Merkle trees are quite secure. -In fact, Merkle trees can be used to both bind and hide a digest.

The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). -A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value.

This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on.

In certain applications, +In fact, Merkle trees can be used to both bind and hide a digest.

+

The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). +A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value.

+

This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on.

+

In certain applications, it may be desirable for the digest of a Merkle tree to be kept confidential. This is achieved with the preimage resistant property of hash functions. A hash function is preimage resistant provided that it is difficult to reverse the hashing operation. -It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest.

Now, we see that Merkle trees are secured structures that are tamper resistant.

Proof of membership​

An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. -This can be done with logarithmic storage and logarithmic computation time.

Suppose that we want to show that data ℓ\ell is part of the Merkle tree's digest. -Additionally, suppose that hash\mathsf{hash} is the hash function used to construct the tree. -We assume that the hash function hash\mathsf{hash} can be computed in constant-time for any input.

Suppose that a prover provides data ℓ\ell to a verifier, -and tells the verifier that ℓ\ell corresponds to the iith leaf of the Merkle tree. -For the verifier to be convinced that ℓ\ell is part of the digest, he needs to be able to construct the tree's root hash using hash\mathsf{hash}, ii, ℓ\ell and some additional information from the prover. +It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest.

+

Now, we see that Merkle trees are secured structures that are tamper resistant.

+

Proof of membership​

+

An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. +This can be done with logarithmic storage and logarithmic computation time.

+

Suppose that we want to show that data ℓ\ell is part of the Merkle tree's digest. +Additionally, suppose that hash\mathsf{hash} is the hash function used to construct the tree. +We assume that the hash function hash\mathsf{hash} can be computed in constant-time for any input.

+

Suppose that a prover provides data ℓ\ell to a verifier, +and tells the verifier that ℓ\ell corresponds to the iith leaf of the Merkle tree. +For the verifier to be convinced that ℓ\ell is part of the digest, he needs to be able to construct the tree's root hash using hash\mathsf{hash}, ii, ℓ\ell and some additional information from the prover. Specifically, the prover must provide the sibling hashes for each value that the verifier can compute. This enables the verifier to compute the parents of the siblings that the prover provides and the values that he was able to produce himself. -The last of the computed parents is the root.

The leaf index ii indicates whether a hash value provided by the prover is a left or right sibling. -This is done by looking at the binary expansion of ii.

The verifier can compute the leaf h0=hash(ℓ)h_0 = \mathsf{hash}(\ell). -Next, using h0h_0's sibling, h0′h'_0, provided by the prover, -the verifier can compute h1=hash(h0∥h0′)h_1 = \mathsf{hash}(h_0 \|h'_0) or h1=hash(h0′∥h0)h_1 = \mathsf{hash}(h'_0 \| h_0) -depending on whether h0′h'_0 is a left or right sibling. -This pathing continues until the verifier either successfully computes the root hash (in n+1n+1 hashes) or fails to do so.

The prover has to provide nn sibling nodes for the proof of membership.

There is a key detail that is essential for the proof of membership to be secure. -The root hash has to be provided to the verifier prior to the selection of data ℓ\ell. -Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership.

Capped proof of membership​

Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. +The last of the computed parents is the root.

+

The leaf index ii indicates whether a hash value provided by the prover is a left or right sibling. +This is done by looking at the binary expansion of ii.

+

The verifier can compute the leaf h0=hash(ℓ)h_0 = \mathsf{hash}(\ell). +Next, using h0h_0's sibling, h0′h'_0, provided by the prover, +the verifier can compute h1=hash(h0∥h0′)h_1 = \mathsf{hash}(h_0 \|h'_0) or h1=hash(h0′∥h0)h_1 = \mathsf{hash}(h'_0 \| h_0) +depending on whether h0′h'_0 is a left or right sibling. +This pathing continues until the verifier either successfully computes the root hash (in n+1n+1 hashes) or fails to do so.

+

The prover has to provide nn sibling nodes for the proof of membership.

+

There is a key detail that is essential for the proof of membership to be secure. +The root hash has to be provided to the verifier prior to the selection of data ℓ\ell. +Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership.

+

Capped proof of membership​

+

Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. A specific layer of the Merkle tree is published instead of just the root hash. -By doing this, a capped proof of membership is just the path from leaf to the published layer.

Extensions of Merkle trees​

Merkle trees can be extended in multiple ways. -In this section, we explore a select few of these extensions.

Sparse Merkle trees​

A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. +By doing this, a capped proof of membership is just the path from leaf to the published layer.

+

Extensions of Merkle trees​

+

Merkle trees can be extended in multiple ways. +In this section, we explore a select few of these extensions.

+

Sparse Merkle trees​

+

A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. Specifically, each digest entry has a particular leaf index. For simplicity, we assume that the index value is computed by taking the hash of the entry. -We note that this is a sorted SMT.

Let nn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n leaves.

An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\mathsf{null} marker inserted in. +We note that this is a sorted SMT.

+

Let nn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n leaves.

+

An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\mathsf{null} marker inserted in. This means that we can prove membership in the way described. -However, we can also prove nonmembership of an element by showing that null\mathsf{null} is located in the element's hash location. -The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership.

We can take advantage of the sparse nature of SMTs to provide shortened proofs. +However, we can also prove nonmembership of an element by showing that null\mathsf{null} is located in the element's hash location. +The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership.

+

We can take advantage of the sparse nature of SMTs to provide shortened proofs. Specifically, it is unlikely for entries to cluster together. -Thus, it is efficient to maintain a list of values:

Null values
d0:=Hash(null),d_0 := \mathsf{Hash(null)},
d1:=Hash(d0∥∥d0),d_1 := \mathsf{Hash(d_0 \|\| d_0)},
d2:=Hash(d1∥∥d1),d_2 := \mathsf{Hash(d_1 \|\| d_1)},
⋮\vdots
dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \mathsf{Hash(d_{n-2} \|\| d_{n-2})}.

Each of the did_i's represents the root hash of a Merkle tree with 2i2^i leaves containing null\mathsf{null}. -These values can be used to shorten the time needed to construct an SMT and the length of proofs.

Proof of nonmembership​

In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. +Thus, it is efficient to maintain a list of values:

+
Null values
d0:=Hash(null),d_0 := \mathsf{Hash(null)},
d1:=Hash(d0∥∥d0),d_1 := \mathsf{Hash(d_0 \|\| d_0)},
d2:=Hash(d1∥∥d1),d_2 := \mathsf{Hash(d_1 \|\| d_1)},
⋮\vdots
dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \mathsf{Hash(d_{n-2} \|\| d_{n-2})}.
+

Each of the did_i's represents the root hash of a Merkle tree with 2i2^i leaves containing null\mathsf{null}. +These values can be used to shorten the time needed to construct an SMT and the length of proofs.

+

Proof of nonmembership​

+

In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. However, the proof of membership may result in false positives due to collisions. This would affect nonmembership proofs as well. -Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest.

Why is sorting essential? +Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest.

+

Why is sorting essential? The sorting mechanism of data can be arbitrarily chosen. However, it is essential that there are no gaps in the ordering. The maximum number of elements that could ever exist in the digest must be known. A simple method for this is to use a hash function to provide fingerprints to the data. Each hash using either SHA-256 or Keccak has 256-bits. -Our entire digest could consist of a maximum of 22562^{256} entries. -This assumes that our digest does not contain collisions.

The fingerprint of a piece of data ℓ\ell indicates which leaf of the SMT it is contained in. -This means that a nonmembership of ℓ\ell in the SMT becomes a matter of proving that null\mathsf{null} is contained in ℓ\ell's location.

It is crucial for the SMT to be sorted. -Otherwise, a malicious party can append the entry ℓ\ell to a random location. +Our entire digest could consist of a maximum of 22562^{256} entries. +This assumes that our digest does not contain collisions.

+

The fingerprint of a piece of data ℓ\ell indicates which leaf of the SMT it is contained in. +This means that a nonmembership of ℓ\ell in the SMT becomes a matter of proving that null\mathsf{null} is contained in ℓ\ell's location.

+

It is crucial for the SMT to be sorted. +Otherwise, a malicious party can append the entry ℓ\ell to a random location. This allows for the malicious party to provide contradictory proofs that prove both membership and nonmembership. We note that the requirement that an SMT is sorted may be too strong of an assumption in centralized cases. -However, sortedness is a necessary property of SMTs for decentralized systems.

Verkle Trees​

A proof of membership grows in length as the Merkle tree grows. +However, sortedness is a necessary property of SMTs for decentralized systems.

+

Verkle Trees​

+

A proof of membership grows in length as the Merkle tree grows. The most obvious approach to remedy this scalability issue is to use Merkle trees in which each node has more than two children. However, this does not fix the issue. -A proof of membership in a kk-nary Merkle tree [6] (each node has kk children) has a proof size log⁡k(n)(k−1)\log_k(n)(k-1). -The multiple k−1k-1 is the number of silbings that a node has on each layer. -Hence, the proof size grows faster than a logarithmic function of the digest size.

An alternate approach is to use a different data structure: Verkle trees [6]. +A proof of membership in a kk-nary Merkle tree [6] (each node has kk children) has a proof size log⁡k(n)(k−1)\log_k(n)(k-1). +The multiple k−1k-1 is the number of silbings that a node has on each layer. +Hence, the proof size grows faster than a logarithmic function of the digest size.

+

An alternate approach is to use a different data structure: Verkle trees [6]. A Verkle tree replaces hash functions with polynomial commitments [7, 8]. -We will explore Verkle trees in a future Vac 101 edition.

References​

]]> +We will explore Verkle trees in a future Vac 101 edition.

+

References​

+]]> Marvin @@ -479,9 +845,13 @@ We will explore Verkle trees in a future Vac 101 edition.

2024-10-31T12:00:00.000Z - Large Message Handling in GossipSub: Potential Improvements

Motivation​

The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). + Large Message Handling in GossipSub: Potential Improvements

+ +

Motivation​

+

The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). It is particularly relevant for applications that require on-time, network-wide dissemination of large messages, -such as Ethereum and Waku [1,2].

This matter has been extensively discussed in the libp2p community [3, +such as Ethereum and Waku [1,2].

+

This matter has been extensively discussed in the libp2p community [3, 4, 5, 6, @@ -491,164 +861,350 @@ and numerous improvements have been considered (or even incorporated) for the Go [3, 7, 9, -10].

Problem description​

Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\lceil \log_D(N) \rceil rounds, -with approximately (D−1)X−1×D(D-1)^{X-1} \times D transmissions in each round, -where X,DX, D represent the round number and mesh size.

Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \times (F+D), -where FF represents the number of peers included in floodpublish.

This arrangement works fine for relatively small/moderate message sizes. -However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen.

Interestingly, a higher DD or FF can also degrade performance in this situation.

Several aspects contribute to this behavior:

  1. Ideally, a message transmission to a single peer concludes in τ1=LR+P\tau_1 = \frac {L}{R}+P (ignoring any message processing time), -where L,R,PL, R, P represent message size, data rate, and link latency. +10].

    +

    Problem description​

    +

    Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\lceil \log_D(N) \rceil rounds, +with approximately (D−1)X−1×D(D-1)^{X-1} \times D transmissions in each round, +where X,DX, D represent the round number and mesh size.

    +

    Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \times (F+D), +where FF represents the number of peers included in floodpublish.

    +

    This arrangement works fine for relatively small/moderate message sizes. +However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen.

    +

    Interestingly, a higher DD or FF can also degrade performance in this situation.

    +

    Several aspects contribute to this behavior:

    +
      +
    1. +

      Ideally, a message transmission to a single peer concludes in τ1=LR+P\tau_1 = \frac {L}{R}+P (ignoring any message processing time), +where L,R,PL, R, P represent message size, data rate, and link latency. Therefore, the time required for sending a message on a 100Mbps link with 100ms latency -jumps from τ110KB=100.8ms\tau_1^{10KB} = 100.8ms for a 10KB message to τ11MB=180ms\tau_1^{1MB} = 180ms for a 1MB message. -For DD peers, the transmission time increases to τD1MB=(80×D)+100ms\tau_D^{1MB} = (80 \times D) + 100ms, -triggering additional queuing delays (proportional to the transmission queue size) during each transmission round.

    2. In practice, τ11MB\tau_1^{1MB} sometimes rises to several hundred milliseconds, +jumps from τ110KB=100.8ms\tau_1^{10KB} = 100.8ms for a 10KB message to τ11MB=180ms\tau_1^{1MB} = 180ms for a 1MB message. +For DD peers, the transmission time increases to τD1MB=(80×D)+100ms\tau_D^{1MB} = (80 \times D) + 100ms, +triggering additional queuing delays (proportional to the transmission queue size) during each transmission round.

      +
      3. +
    3. +

      In practice, τ11MB\tau_1^{1MB} sometimes rises to several hundred milliseconds, further exaggerating the abovementioned queuing delays. This rise is because TCP congestion avoidance algorithms usually limit maximum in-flight bytes in a round trip time (RTT) -based on the congestion window (CwndC_{wnd}) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \times MSS}, -with CwndC_{wnd} rising with the data transfer for each flow. +based on the congestion window (CwndC_{wnd}) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \times MSS}, +with CwndC_{wnd} rising with the data transfer for each flow. Consequently, sending the same message through a newly established (cold) connection takes longer. -The message transfer time lowers as the CwndC_{wnd} grows. +The message transfer time lowers as the CwndC_{wnd} grows. Therefore, performance-improvement practices such as floodpublish, frequent mesh adjustment, and lazy sending typically result in longer than expected message dissemination times for large messages (due to cold connections). -It is also worth mentioning that some TCP variants reset their CwndC_{wnd} after different periods of inactivity.

    4. Theoretically, the message transmission time to D peers (τD)(\tau_D) remains the same +It is also worth mentioning that some TCP variants reset their CwndC_{wnd} after different periods of inactivity.

      +
      5. +
    5. +

      Theoretically, the message transmission time to D peers (τD)(\tau_D) remains the same even if the message is relayed sequentially to all peers or simultaneous transmissions are carried out, -i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i +i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i However, sequential transmissions finish early for individual peers, allowing them to relay early. -This may result in quicker network-wide message dissemination.

    1. A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). +This may result in quicker network-wide message dissemination.

      +
      2. +
    2. +

      A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). As the message disseminates, it's not uncommon for some peers to receive it much earlier than others. Early gossip (IHAVE announcements) may bring in many IWANT requests to the early receivers (even from peers already receiving the same message), -which adds to their workload.

    3. A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. +which adds to their workload.

      +
      4. +
    4. +

      A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. As a result, already scheduled messages are prioritized over those published by the peer itself, introducing a significant initial delay to the locally published messages. Enqueuing IWANT replies to the outgoing message queue can further exaggerate the problem. The lack of adaptiveness and standardization in outgoing message prioritization are key factors that can lead to noticeable inconsistency -in message dissemination latency at each hop, even in similar network conditions.

    5. Message size directly contributes to peers' workloads in terms of processing and transmission time. +in message dissemination latency at each hop, even in similar network conditions.

      +
      6. +
    6. +

      Message size directly contributes to peers' workloads in terms of processing and transmission time. It also raises the probability of simultaneous redundant transmissions to the same peer, resulting in bandwidth wastage, congestion, and slow message propagation to the network. -Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers.

    7. Most use cases necessitate validating received messages before forwarding them to the next-hop peers. -For a higher message transfer time (τ)(\tau ), this store-and-forward delay accumulates across the hops traveled by the message.

    Possible improvements​

    1. Minimizing transfer time for large messages​

    The impact of message size and achievable data rate on message transmit time τ\tau is crucial -as this time accumulates due to the store-and-forward delay introduced at intermediate hops.

    Some possible improvements to minimize overall message dissemination latency include:

    a. Message fragmentation​

    In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) -can be simplified to roughly δ≈δTx+Ph\delta \approx \delta_{Tx} + P_h, where δTx\delta_{Tx} represents accumulative message transmit time denoted as δTx=SR×h\delta_{Tx} = \frac{S}{R} \times h, -with S,RS, R being the data size and data rate, -and h,Phh, P_h being the number of hops in the longest path and message propagation time through the longest path.

    Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\frac{\delta_{Tx}}{n}. +Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers.

    +
    2. +
  2. +

    Most use cases necessitate validating received messages before forwarding them to the next-hop peers. +For a higher message transfer time (τ)(\tau ), this store-and-forward delay accumulates across the hops traveled by the message.

    +
    3. +
+

Possible improvements​

+

1. Minimizing transfer time for large messages​

+

The impact of message size and achievable data rate on message transmit time τ\tau is crucial +as this time accumulates due to the store-and-forward delay introduced at intermediate hops.

+

Some possible improvements to minimize overall message dissemination latency include:

+

a. Message fragmentation​

+

In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) +can be simplified to roughly δ≈δTx+Ph\delta \approx \delta_{Tx} + P_h, where δTx\delta_{Tx} represents accumulative message transmit time denoted as δTx=SR×h\delta_{Tx} = \frac{S}{R} \times h, +with S,RS, R being the data size and data rate, +and h,Phh, P_h being the number of hops in the longest path and message propagation time through the longest path.

+

Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\frac{\delta_{Tx}}{n}. As a received fragment can be immediately relayed by the receiver (while the sender is still transmitting the remaining fragments), -it reduces the transmit time to δTx=SR×2h−1n\delta_{Tx} = \frac{S}{R} \times \frac{2h-1}{n}.

This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions.

However, it is worth noting that many applications require each fragment to be individually verifiable. +it reduces the transmit time to δTx=SR×2h−1n\delta_{Tx} = \frac{S}{R} \times \frac{2h-1}{n}.

+

This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions.

+

However, it is worth noting that many applications require each fragment to be individually verifiable. At the same time, message fragmentation allows a malicious peer to never relay some fragments of a message, -which can lead to a significant rise in the application's receive buffer size.

Therefore, message fragmentation requires a careful tradeoff analysis between time and risks.

b. Message staggering​

Considering the same bandwidth, the time τD\tau_D required for sending a message to D peers stays the same, -even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i.

However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\tau_1 \approx \frac{\tau_D}{D}) due to bandwidth concentration for a particular peer. -So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers.

As a result, after every τDD\frac{\tau_D}{D} milliseconds, -the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\ \forall\ X \in \{0, D-1\} and by ∑k=X−DX−1λk ∀ X≥D\sum_{k=X-D}^{X-1} \lambda_k\ \forall\ X \geq D. -Here, XX represents message transmission round X=i⋅τDD∣i∈N0X = i \cdot \frac{\tau_D}{D} \mid i \in \mathbb{N}_0, and λk\lambda_k represents the number of peers that received the message in round kk.

It is worth noting that a realistic network imposes certain constraints on staggering for peers. +which can lead to a significant rise in the application's receive buffer size.

+

Therefore, message fragmentation requires a careful tradeoff analysis between time and risks.

+

b. Message staggering​

+

Considering the same bandwidth, the time τD\tau_D required for sending a message to D peers stays the same, +even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i.

+

However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\tau_1 \approx \frac{\tau_D}{D}) due to bandwidth concentration for a particular peer. +So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers.

+

As a result, after every τDD\frac{\tau_D}{D} milliseconds, +the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\ \forall\ X \in \{0, D-1\} and by ∑k=X−DX−1λk ∀ X≥D\sum_{k=X-D}^{X-1} \lambda_k\ \forall\ X \geq D. +Here, XX represents message transmission round X=i⋅τDD∣i∈N0X = i \cdot \frac{\tau_D}{D} \mid i \in \mathbb{N}_0, and λk\lambda_k represents the number of peers that received the message in round kk.

+

It is worth noting that a realistic network imposes certain constraints on staggering for peers. For instance, in a network with dissimilar peer capabilities, placing a slow peer (also in cases where many senders simultaneously select the same peer) at the head of the transmission queue -may result in head-of-line blocking for the message queue.

At the same time, early receivers get many IWANT requests, increasing their workload.

c. Message prioritization for slow senders​

A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. +may result in head-of-line blocking for the message queue.

+

At the same time, early receivers get many IWANT requests, increasing their workload.

+

c. Message prioritization for slow senders​

+

A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. Any new message transmission at this stage (especially the locally published messages) gets delayed. -Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers.

For instance, any GossipSub peer will likely receive every message from multiple senders, +Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers.

+

For instance, any GossipSub peer will likely receive every message from multiple senders, leading to redundant transmissions [11]. Implementing efficient strategies (only for slow senders) like lazy sending and prioritizing locally published messages/IWANT replies over already queued messages -can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers.

A peer can identify itself as a slow peer by using any bandwidth estimation approach -or simply setting an outgoing message queue threshold for all mesh members.

Eliminating/deprioritizing some messages can lower a peer's score, +can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers.

+

A peer can identify itself as a slow peer by using any bandwidth estimation approach +or simply setting an outgoing message queue threshold for all mesh members.

+

Eliminating/deprioritizing some messages can lower a peer's score, but it also earns the peer an overall better score by achieving some early message transfers.
For instance, sending many near-first messages can only save a peer from a deficit penalty. On the other hand, sending only one message (assuming MeshMessageDeliveriesThreshold defaults to 1) -as the first delivered message can add to the accumulative peer score.

2. Mitigating transport issues​

Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time -as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd}) size.

Rapid adaptation of CwndC_{wnd} to the available network conditions can help lower message dissemination latency.

Therefore, selecting a more suitable TCP variant like BBR, +as the first delivered message can add to the accumulative peer score.

+

2. Mitigating transport issues​

+

Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time +as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd}) size.

+

Rapid adaptation of CwndC_{wnd} to the available network conditions can help lower message dissemination latency.

+

Therefore, selecting a more suitable TCP variant like BBR, which is known for its ability to dynamically adjust the congestion window based on network conditions, -can significantly enhance GossipSub's performance.

At the same time, parameters like receive window scaling and initial CwndC_{wnd} also impact message transfer time, -but these are usually OS-specific system-wide choices.

One possible solution is to raise CwndC_{wnd} by exchanging data over the newly established connection. +can significantly enhance GossipSub's performance.

+

At the same time, parameters like receive window scaling and initial CwndC_{wnd} also impact message transfer time, +but these are usually OS-specific system-wide choices.

+

One possible solution is to raise CwndC_{wnd} by exchanging data over the newly established connection. This data may involve useful details like peer exchange information and gossip to build initial trust, -or GossipSub can use some dummy data to raise CwndC_{wnd} to a reasonable level.

It's important to understand that some TCP variants reset CwndC_{wnd} after specific periods of inactivity [12]. +or GossipSub can use some dummy data to raise CwndC_{wnd} to a reasonable level.

+

It's important to understand that some TCP variants reset CwndC_{wnd} after specific periods of inactivity [12]. This can lead to a decline in TCP's performance for applications -that generate traffic after intervals long enough to trigger the resetting of the congestion window.

Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13].

The limitations faced with CwndC_{wnd} scaling also impact some performance optimizations in GossipSub. -For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency.

However, a small CwndC_{wnd} value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time +that generate traffic after intervals long enough to trigger the resetting of the congestion window.

+

Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13].

+

The limitations faced with CwndC_{wnd} scaling also impact some performance optimizations in GossipSub. +For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency.

+

However, a small CwndC_{wnd} value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time [4]. -Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth.

The same is the case with IWANT replies.

Maintaining a bigger mesh (with warm TCP connections) and relaying to DD peers can be a better alternative to this problem.

3. Eliminating redundant transmissions​

For every received packet, a peer makes roughly DD transmissions to contribute its fair share to the spread of messages. +Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth.

+

The same is the case with IWANT replies.

+

Maintaining a bigger mesh (with warm TCP connections) and relaying to DD peers can be a better alternative to this problem.

+

3. Eliminating redundant transmissions​

+

For every received packet, a peer makes roughly DD transmissions to contribute its fair share to the spread of messages. However, the fact that many recipients had already received the message (from some other peer) -makes this message propagation inefficient.

Although the DD-spread is attributed to quicker dissemination and resilience against non-conforming peers, +makes this message propagation inefficient.

+

Although the DD-spread is attributed to quicker dissemination and resilience against non-conforming peers, many potential solutions can still minimize redundant transmissions -while preserving the resilience of GossipSub.

These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, +while preserving the resilience of GossipSub.

+

These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, not only address the issue of redundancy but also provide an opportunity for bandwidth optimization, -especially for resource-constrained peers.

For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], -can significantly reduce redundant transmissions.

It allows any node to notify its mesh members that it has already received a message, +especially for resource-constrained peers.

+

For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], +can significantly reduce redundant transmissions.

+

It allows any node to notify its mesh members that it has already received a message, thereby preventing them from resending the same message. -This functionality is useful when a node receives a message larger than a specified threshold.

In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages.

It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, -i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message.

This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience.

However, the use of IDONTWANT messages alone has an inherent limitation. -For instance, a peer can only send an IDONTWANT after receiving the complete message.

A large message transmission consumes significant time. -For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd} and latency).

As a result, other mesh members may also start transmitting the same message during this interval. -A few potential solutions include:

a. Staggering with IDONTWANT messages​

As previously discussed, staggering can significantly reduce network-wide message dissemination latency. -This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach.

Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. +This functionality is useful when a node receives a message larger than a specified threshold.

+

In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages.

+

It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, +i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message.

+

This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience.

+

However, the use of IDONTWANT messages alone has an inherent limitation. +For instance, a peer can only send an IDONTWANT after receiving the complete message.

+

A large message transmission consumes significant time. +For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd} and latency).

+

As a result, other mesh members may also start transmitting the same message during this interval. +A few potential solutions include:

+

a. Staggering with IDONTWANT messages​

+

As previously discussed, staggering can significantly reduce network-wide message dissemination latency. +This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach.

+

Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. This is because a node only saturates its bandwidth for a small subset of mesh peers, -leading to early transmissions and prompt IDONTWANT message notifications to the mesh members.

It is worth highlighting that staggering can be implemented in various ways.

For example, it can be applied to peers (peer staggering) -where a node sequentially relays the same message to all peers one by one.

Alternatively, a node can send a different message to every peer (message staggering or rotational sending), +leading to early transmissions and prompt IDONTWANT message notifications to the mesh members.

+

It is worth highlighting that staggering can be implemented in various ways.

+

For example, it can be applied to peers (peer staggering) +where a node sequentially relays the same message to all peers one by one.

+

Alternatively, a node can send a different message to every peer (message staggering or rotational sending), allowing IDONTWANTs for other messages to arrive during this time. -The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time.

As the peers in staggered sending are sequentially covered -(with a faster speed due to bandwidth concentration), this leads to another problem.

The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. +The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time.

+

As the peers in staggered sending are sequentially covered +(with a faster speed due to bandwidth concentration), this leads to another problem.

+

The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. IHAVE announcements for newly received large messages trigger IWANTs from nodes (including those already receiving the same message), -leading to an additional workload for early receivers [14].

Potential solutions to mitigate these problems include:

1) Defering IHAVE announcements for large messages.

Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. +leading to an additional workload for early receivers [14].

+

Potential solutions to mitigate these problems include:

+
    +
  1. Defering IHAVE announcements for large messages.
    2. +
+

Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. However, deciding on a suitable deferred interval is crucial for optimal performance. -One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers.

2) Defering IWANT requests for messages that are currently being received.

This requires prior knowledge of msgIDs for the messages under reception. +One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers.

+
    +
  1. Defering IWANT requests for messages that are currently being received.
    2. +
+

This requires prior knowledge of msgIDs for the messages under reception. Knowing the message length is also essential in deciding a suitable defer interval -to handle situations where a sender starts sending a message and never completes the transmission.

3) Not issuing IWANT for a message if at least KK peers have transmitted IDONTWANT for the same message -(as this indicates that these peers will eventually relay this message).

However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. -A delayed IWANT, along with negative peer scoring, can remedy this problem.

b. IMReceiving message​

A peer can issue an IDONTWANT only after it has received the entire message. +to handle situations where a sender starts sending a message and never completes the transmission.

+
    +
  1. Not issuing IWANT for a message if at least KK peers have transmitted IDONTWANT for the same message +(as this indicates that these peers will eventually relay this message).
    2. +
+

However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. +A delayed IWANT, along with negative peer scoring, can remedy this problem.

+

b. IMReceiving message​

+

A peer can issue an IDONTWANT only after it has received the entire message. However, a large message transmission may take several hundred milliseconds to complete. -During this time, many other mesh members may start relaying the same message.

Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, -significantly compromising the effectiveness of IDONTWANT messages.

Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. +During this time, many other mesh members may start relaying the same message.

+

Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, +significantly compromising the effectiveness of IDONTWANT messages.

+

Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. If a receiver is already receiving the same message from another sender, -the receiver can request to defer this transmission by sending a brief IMReceiving message.

An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval.

However, waiting for IMReceiving after sending the preamble can delay the message transmission. +the receiver can request to defer this transmission by sending a brief IMReceiving message.

+

An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval.

+

However, waiting for IMReceiving after sending the preamble can delay the message transmission. On the other hand, proceeding with message transfer (after sending the preamble) leads to another problem: -it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message.

To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), +it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message.

+

To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), urging other mesh peers to defer sending the same message [15, -16].

The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. -This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages.

While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, -it's crucial to be aware of the potential risks.

A malicious user can exploit this approach to disrupt message transmission -either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers.

This could ultimately result in network-wide slow message propagation.

However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks.

c. IDONTWANT message with reduced forwarding​

It is common for slow peers to pile up outgoing message queues, +16].

+

The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. +This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages.

+

While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, +it's crucial to be aware of the potential risks.

+

A malicious user can exploit this approach to disrupt message transmission +either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers.

+

This could ultimately result in network-wide slow message propagation.

+

However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks.

+

c. IDONTWANT message with reduced forwarding​

+

It is common for slow peers to pile up outgoing message queues, especially for large message transfers. This results in a significant queuing delay for outgoing messages. -Reduced message forwarding can help decrease the workload of slower peers.

On receiving a message longer than the specified threshold, -a slow peer can relay it to only K∈DK \in D peers and send an IDONTWANT message to all the peers in DD.

In this arrangement, the IDONTWANT message serves an additional purpose: -to promptly announce data availability, reinforcing redundancy in the presence of adversaries.

When a peer receives an IDONTWANT for an unseen message, +Reduced message forwarding can help decrease the workload of slower peers.

+

On receiving a message longer than the specified threshold, +a slow peer can relay it to only K∈DK \in D peers and send an IDONTWANT message to all the peers in DD.

+

In this arrangement, the IDONTWANT message serves an additional purpose: +to promptly announce data availability, reinforcing redundancy in the presence of adversaries.

+

When a peer receives an IDONTWANT for an unseen message, it learns about the new message and can request it by sending an IWANT request without waiting for the heartbeat (gossip) interval. -As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network.

This approach conserves peer bandwidth by minimizing redundant transmissions -while ensuring GossipSub resilience at the cost of one RTT (for missing peers).

Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages).

However, finding an appropriate value for KK is crucial for optimal performance. -A smaller KK saves peer bandwidth, while a larger KK achieves quicker spread until outgoing message queues pile up. -Setting K=DlowK = D_{low} can be one option.

It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-K peers). +As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network.

+

This approach conserves peer bandwidth by minimizing redundant transmissions +while ensuring GossipSub resilience at the cost of one RTT (for missing peers).

+

Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages).

+

However, finding an appropriate value for KK is crucial for optimal performance. +A smaller KK saves peer bandwidth, while a larger KK achieves quicker spread until outgoing message queues pile up. +Setting K=DlowK = D_{low} can be one option.

+

It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-K peers). However, a minimized workload enables early message dissemination to the remaining peers. -These early transmissions and randomized KK set selection can help achieve an overall better peer score.

4. Message prioritization​

Despite the standardized specifications of the GossipSub protocol, -the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization.

It is worth mentioning that every node is responsible for transmitting different types of packets, -including control messages, locally published messages, messages received from mesh members, IWANT replies, etc.

As long as traffic volume is lower than the available data rate, -the message forwarding mechanisms yield similar results due to negligible queuing delays.

However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), -the outgoing message queue(s) sizes rise, potentially impacting the network's performance.

In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, +These early transmissions and randomized KK set selection can help achieve an overall better peer score.

+

4. Message prioritization​

+

Despite the standardized specifications of the GossipSub protocol, +the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization.

+

It is worth mentioning that every node is responsible for transmitting different types of packets, +including control messages, locally published messages, messages received from mesh members, IWANT replies, etc.

+

As long as traffic volume is lower than the available data rate, +the message forwarding mechanisms yield similar results due to negligible queuing delays.

+

However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), +the outgoing message queue(s) sizes rise, potentially impacting the network's performance.

+

In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, introducing a queuing delay proportional to the queue size. -The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc.

On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. -One possible solution is to use weighted queues for a fair spread of messages.

Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time -and allow for customizable message handling.

For example, staggering between peers and messages can be better managed by using priority queues. +The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc.

+

On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. +One possible solution is to use weighted queues for a fair spread of messages.

+

Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time +and allow for customizable message handling.

+

For example, staggering between peers and messages can be better managed by using priority queues. However, it is important to note that message prioritization also introduces additional complexity to the system, -necessitating sophisticated algorithms for better message handling.

5. Maximizing benefits from IWANT messages​

During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. +necessitating sophisticated algorithms for better message handling.

+

5. Maximizing benefits from IWANT messages​

+

During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. These peers can use IWANT messages to request any missing messages. -A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval.

The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. +A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval.

+

The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. They bridge the information gap between nearby and far-off peers, ensuring that information can be disseminated to peers outside the mesh. -This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks.

However, it is essential to understand that high transmission times for large messages -require careful due diligence when using IWANT messages for reasons not limited to:

1) A large message reception may take several hundred milliseconds to complete. -During this time, an IHAVE message announcing the same message ID will trigger an IWANT request.

2) A peer can send IWANT requests for the same message to multiple nodes, -leading to simultaneous transmissions of the same message.

3) Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, -resulting in lower peer scores and slower message propagation.

A few possible solutions to mitigate this problem may include:

1) Issuing IHAVE announcements only after the message is delivered to many mesh peers.

2) Allocating a volume-based budget to service IWANT requests during each heartbeat interval.

3) Deferring IWANT requests for messages that are currently being received.

4) Deferring IWANT requests if at least KK IDONTWANTs are received for the same message.

5) A large message transmission can yield high CwndC_{wnd}; preferring such peers during mesh maintenance can be helpful.

Summary​

This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages.

We delve into multiple factors, +This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks.

+

However, it is essential to understand that high transmission times for large messages +require careful due diligence when using IWANT messages for reasons not limited to:

+
    +
  1. +

    A large message reception may take several hundred milliseconds to complete. +During this time, an IHAVE message announcing the same message ID will trigger an IWANT request.

    +
    2. +
  2. +

    A peer can send IWANT requests for the same message to multiple nodes, +leading to simultaneous transmissions of the same message.

    +
    3. +
  3. +

    Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, +resulting in lower peer scores and slower message propagation.

    +
    4. +
+

A few possible solutions to mitigate this problem may include:

+
    +
  1. +

    Issuing IHAVE announcements only after the message is delivered to many mesh peers.

    +
    2. +
  2. +

    Allocating a volume-based budget to service IWANT requests during each heartbeat interval.

    +
    3. +
  3. +

    Deferring IWANT requests for messages that are currently being received.

    +
    4. +
  4. +

    Deferring IWANT requests if at least KK IDONTWANTs are received for the same message.

    +
    5. +
  5. +

    A large message transmission can yield high CwndC_{wnd}; preferring such peers during mesh maintenance can be helpful.

    +
    6. +
+

Summary​

+

This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages.

+

We delve into multiple factors, such as increased message transmit times, store-and-forward delays, congestion avoidance mechanisms, and prioritization between messages, -to establish a comprehensive understanding of the problem.

The study also explores the performance of optimization efforts -like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions.

A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. +to establish a comprehensive understanding of the problem.

+

The study also explores the performance of optimization efforts +like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions.

+

A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. Coupling this constraint with increased message transmission times -results in notable store-and-forward delays accumulating at each hop.

Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation -by utilizing a considerable share of available bandwidth on redundant transmissions.

Therefore, approaches focused on eliminating redundant transmissions +results in notable store-and-forward delays accumulating at each hop.

+

Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation +by utilizing a considerable share of available bandwidth on redundant transmissions.

+

Therefore, approaches focused on eliminating redundant transmissions (IDONTWANT, IMReceiving, lazy sending, etc.) can prove helpful. At the same time, strategies aimed at reducing store-and-forward delays -(fragmentation, staggering, prioritization, etc.) can prove beneficial.

It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. +(fragmentation, staggering, prioritization, etc.) can prove beneficial.

+

It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. Some of these have already been explored and discussed to some extent [5, 17, 18]. -We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings.

Please feel free to join the discussion and leave feedback regarding this post in the -VAC forum.

References​

[1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844

[2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/

[3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666

[4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383

[5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw

[6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850

[7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911

[8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859

[9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh

[10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52

[11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921

[12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1

[13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558

[14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101

[15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281

[16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283

[17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366

[18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/

]]> +We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings.

+

Please feel free to join the discussion and leave feedback regarding this post in the +VAC forum.

+

References​

+

[1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844

+

[2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/

+

[3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666

+

[4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383

+

[5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw

+

[6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850

+

[7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911

+

[8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859

+

[9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh

+

[10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52

+

[11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921

+

[12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1

+

[13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558

+

[14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101

+

[15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281

+

[16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283

+

[17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366

+

[18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/

]]> Umar Farooq @@ -659,33 +1215,76 @@ We are nearing the completion of a comprehensive performance evaluation of these 2024-10-28T12:00:00.000Z - This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.

Overview​

IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. -Cutting down on duplicates can potentially render two significant advantages:

  1. Reducing bandwidth requirements

  2. Reducing message dissemination time (latency)

For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message.

Duplicates investigation reveals that -the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network.

This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, + This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.

+ +

Overview​

+

IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. +Cutting down on duplicates can potentially render two significant advantages:

+
    +
  1. +

    Reducing bandwidth requirements

    +
    2. +
  2. +

    Reducing message dissemination time (latency)

    +
    3. +
+

For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message.

+

Duplicates investigation reveals that +the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network.

+

This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, showing a potential for a significant drop in bandwidth utilization. -On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.

Experiments​

We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) -to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time.

The experiments are performed on nim-libp2p using the shadow simulator. -The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages.

In all experiments, ten messages are transmitted to the network, i.e., +On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.

+

Experiments​

+

We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) +to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time.

+

The experiments are performed on nim-libp2p using the shadow simulator. +The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages.

+

In all experiments, ten messages are transmitted to the network, i.e., ten peers (publishers) are selected as the message transmitters. Every publisher transmits exactly one message, and inter-packet spacing (delay) is set to four seconds for each published message. -For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class.

At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}. -These messages are not included in latency computations.

The simulation details are presented in the table below.

ParameterValueParameterValue
Peers2000Publishers10
Peer bandwidth50-150 MbpsLink latency40-130 ms
Message size1KB, 50KB, 500KB, 1MBDD8
Heartbeat interval700ms, 1000ms, 1500msDlowD_{low}6
FloodPublishFalseDhighD_{high}12
Gossip factor0.05Muxeryamux

Findings​

We use bandwidth utilization and latency as evaluation metrics. +For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class.

+

At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}. +These messages are not included in latency computations.

+

The simulation details are presented in the table below.

+
ParameterValueParameterValue
Peers2000Publishers10
Peer bandwidth50-150 MbpsLink latency40-130 ms
Message size1KB, 50KB, 500KB, 1MBDD8
Heartbeat interval700ms, 1000ms, 1500msDlowD_{low}6
FloodPublishFalseDhighD_{high}12
Gossip factor0.05Muxeryamux
+

Findings​

+

We use bandwidth utilization and latency as evaluation metrics. Bandwidth utilization represents total network-wide traffic (including gossip and other control messages). Latency refers to network-wide message dissemination time. -The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights.

Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. +The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights.

+

Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. A higher drop is seen with a higher heartbeat interval. Interestingly, a relatively low bandwidth reduction (12-20%) is seen for 1MB messages, -compared to 500KB messages (18-21%).

Image 1
Image 2
Image 3

This is because downloading a large message may consume several hundred milliseconds. +compared to 500KB messages (18-21%).

+
Image 1
Image 2
Image 3
+

This is because downloading a large message may consume several hundred milliseconds. During this time, a receiver will likely generate multiple IWANT requests -for the same message, increasing bandwidth utilization.

Moreover, a peer can generate +for the same message, increasing bandwidth utilization.

+

Moreover, a peer can generate IDONTWANTs only after it has finished downloading the message. -A longer download time will result in simultaneous reception of the same message from other mesh members.

Image 1
Image 2

These IWANT requests mainly overwhelm early message receivers, +A longer download time will result in simultaneous reception of the same message from other mesh members.

+
Image 1
Image 2
+

These IWANT requests mainly overwhelm early message receivers, which can negatively impact message dissemination time on some occasions. -Therefore, a similar message dissemination time is seen with and without IDONTWANT messages.

Image 1
Image 2
Image 3

Similar results are seen on our large-scale deployment runs -(running Waku nodes in Kubernetes).

Please feel free to join the discussion and leave feedback regarding this post in the -VAC forum.

References​

]]> +Therefore, a similar message dissemination time is seen with and without IDONTWANT messages.

+
Image 1
Image 2
Image 3
+

Similar results are seen on our large-scale deployment runs +(running Waku nodes in Kubernetes).

+

Please feel free to join the discussion and leave feedback regarding this post in the +VAC forum.

+

References​

+]]> Umar Farooq @@ -696,14 +1295,18 @@ Therefore, a similar message dissemination time is seen with and without IDONTWA 2024-10-15T12:00:00.000Z - In this post, we introduce a common technique used to convert interactive protocols to their noninteractive variant.

Introduction​

The set of interactive protocols form a class of protocols that consist of communication between two parties: the Prover and the Verifier. + In this post, we introduce a common technique used to convert interactive protocols to their noninteractive variant.

+ +

Introduction​

+

The set of interactive protocols form a class of protocols that consist of communication between two parties: the Prover and the Verifier. The Prover tries to convince the Verifier of a given claim. For example, the Prover may want to convince the Verifier that she owns a specific Unspent Transaction Output (UTXO); that is, the Prover possesses the ability to spend the UTXO. In many instances, there is information that the Prover does not wish to reveal to the Verifier. In our example, it is critical that the Prover does not provide the Verifier with the spending key associated with her UTXO. In addition to the Prover's claim and secret data, there is additional data, public parameters, that the claimed statement is expressed in terms of. -The public parameters can be thought of as the basis for all similar claims.

In an interactive protocol, the Prover and the Verifier are in active communication. +The public parameters can be thought of as the basis for all similar claims.

+

In an interactive protocol, the Prover and the Verifier are in active communication. Specifically, the Prover and the Verifier exchange messages so that the Verifier can validate the Prover's claim. However, this communication is not practical for many applications. It is necessary that any party can verify the Prover's claim in decentralized systems. @@ -712,84 +1315,210 @@ Instead, it is desirable for the Prover to generate a proof on their own that ca To achieve this, it is necessary for the Prover to generate the Verifier's messages in such a way that the Prover cannot manipulate the Verifier's messages for her benefit. The Fiat-Shamir heuristic 1 is used for this purpose. -Even though much of our discussion will focus on Σ\Sigma-protocols, -the Fiat-Shamir heuristic is not limited to Σ\Sigma-protocols. +Even though much of our discussion will focus on Σ\Sigma-protocols, +the Fiat-Shamir heuristic is not limited to Σ\Sigma-protocols. The Fiat-Shamir heuristic has been applied to zk-SNARKs, but the security in this setting has been the subject of discussion and research in recent years. -Block et al. 2 provide the first formal analysis of Fiat-Shamir heuristic in zk-SNARKs.

Sigma Protocols​

A Σ\Sigma-protocol is a family of interactive protocols that consists of three publicly transmitted messages between the Prover and the Verifier. -In particular, the protocol has the following framework:

ProverVerifier
⟶commitment\stackrel{\mathsf{commitment}}{\longrightarrow}
⟵challenge\stackrel{\mathsf{challenge}}{\longleftarrow}
⟶response\stackrel{\mathsf{response}}{\longrightarrow}

These three messages form the protocol's transcript: (commitment,challenge,response)(\mathsf{commitment}, \mathsf{challenge}, \mathsf{response}). +Block et al. 2 provide the first formal analysis of Fiat-Shamir heuristic in zk-SNARKs.

+

Sigma Protocols​

+

A Σ\Sigma-protocol is a family of interactive protocols that consists of three publicly transmitted messages between the Prover and the Verifier. +In particular, the protocol has the following framework:

+
ProverVerifier
⟶commitment\stackrel{\mathsf{commitment}}{\longrightarrow}
⟵challenge\stackrel{\mathsf{challenge}}{\longleftarrow}
⟶response\stackrel{\mathsf{response}}{\longrightarrow}
+

These three messages form the protocol's transcript: (commitment,challenge,response)(\mathsf{commitment}, \mathsf{challenge}, \mathsf{response}). The Verifier uses all three of these messages to validate the Prover's original claim. The Verifier's challenge should be selected uniform random from all possible challenges. -Based on this selection, a dishonest Prover can only convince the Verifier with a negligible probability.

The Schnorr Protocol​

The Schnorr protocol 3 is usually the first Σ\Sigma-protocol that one studies. +Based on this selection, a dishonest Prover can only convince the Verifier with a negligible probability.

+

The Schnorr Protocol​

+

The Schnorr protocol 3 is usually the first Σ\Sigma-protocol that one studies. Additionally, the Schnorr protocol can be used as an efficient signature scheme. -The Schnorr protocol provides a framework that enables the Prover to convince the Verifier that: for group elements gg and XX, -the Prover knows the power to raise gg to obtain XX. -Specifically, the Prover possesses some integer xx so that X=gxX = g^x. +The Schnorr protocol provides a framework that enables the Prover to convince the Verifier that: for group elements gg and XX, +the Prover knows the power to raise gg to obtain XX. +Specifically, the Prover possesses some integer xx so that X=gxX = g^x. Cryptographic resources may use either multiplicative or additive notation for groups; we will use multiplicative notation. -Briefly, the element gg being combined with itself in multiplicative notation is g⋅g=g2g \cdot g = g^2, -while in additive notation it is g+g=2gg + g = 2g. -We assume that our group is of prime order pp, and is sufficiently large to satisfy the discrete logarithm assumption.

The Schnorr protocol proceeds as follows:

ProverVerifier
t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t⟶T\stackrel{T}{\longrightarrow}
⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
output 1 provided gz=?TXcg^z \stackrel{?}{=} T X^c

Chaum-Pedersen protocol​

A tuple of group elements (U,V,W)(U,V,W) is a DH-triple if and only if there exists some x∈Zpx \in \mathbb{Z}_p so that V=gxV = g^x and W=UxW = U^x. -The Chaum-Pedersen protocol provides a framework that enables a Prover to convince a Verifier that she possesses such a xx for a claimed DH-triple (U,V,W)(U,V,W). -The Chaum-Pedersen protocol proceeds as follows:

ProverVerifier
t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t, S:=UtS := U^t⟶T,S\stackrel{T,S}{\longrightarrow}
⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
output 1 provided gz=?TVcg^z \stackrel{?}{=} T V^c and Uz=?SWcU^z \stackrel{?}{=} SW^c

Hash Functions​

Cryptographic hash functions serve as the backbone to the Fiat-Shamir heuristic. -A hash function, Hash\mathsf{Hash}, is a special function that takes in an arbitrary binary string and outputs a binary string of a predetermined fixed length. +Briefly, the element gg being combined with itself in multiplicative notation is g⋅g=g2g \cdot g = g^2, +while in additive notation it is g+g=2gg + g = 2g. +We assume that our group is of prime order pp, and is sufficiently large to satisfy the discrete logarithm assumption.

+

The Schnorr protocol proceeds as follows:

+
ProverVerifier
t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t⟶T\stackrel{T}{\longrightarrow}
⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
output 1 provided gz=?TXcg^z \stackrel{?}{=} T X^c
+

Chaum-Pedersen protocol​

+

A tuple of group elements (U,V,W)(U,V,W) is a DH-triple if and only if there exists some x∈Zpx \in \mathbb{Z}_p so that V=gxV = g^x and W=UxW = U^x. +The Chaum-Pedersen protocol provides a framework that enables a Prover to convince a Verifier that she possesses such a xx for a claimed DH-triple (U,V,W)(U,V,W). +The Chaum-Pedersen protocol proceeds as follows:

+
ProverVerifier
t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t, S:=UtS := U^t⟶T,S\stackrel{T,S}{\longrightarrow}
⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
output 1 provided gz=?TVcg^z \stackrel{?}{=} T V^c and Uz=?SWcU^z \stackrel{?}{=} SW^c
+

Hash Functions​

+

Cryptographic hash functions serve as the backbone to the Fiat-Shamir heuristic. +A hash function, Hash\mathsf{Hash}, is a special function that takes in an arbitrary binary string and outputs a binary string of a predetermined fixed length. Specifically, -Hash:{0,1}∗→{0,1}n\mathsf{Hash} : \{0,1\}^* \rightarrow \{0,1\}^n.

The security of cryptographic hash functions will rely on certain tasks being computationally infeasible. -A task is computationally infeasible provided that there is no deterministic algorithm that can conclude the task in polynomial-time.

A cryptographic hash function satisfies the following properties:

  • Succinct: The hash function should be easy to compute; the hash Hash(b)\mathsf{Hash}({\bf{b}}) can be efficiently computed for any binary string b{\bf{b}}.
  • Preimage Resistance: It should be computationally infeasible to work backwards given the output of a hash function. Let y{\bf{y}} be a binary string of length nn. -It should be 'impossible' to find some binary string x{\bf{x}} so that y=Hash(x){\bf{y}} = \mathsf{Hash}({\bf{x}}).
  • Collision Resistance: It should be difficult to find two strings that hash to the same value. -It should be computationally infeasible to find two binary strings x1{\bf{x}_1} and x2{\bf{x}_2} so that Hash(x1)=Hash(x2).\mathsf{Hash}({\bf{x}_1}) = \mathsf{Hash}({\bf{x}_2}).

A related class of functions is one-way functions. +Hash:{0,1}∗→{0,1}n\mathsf{Hash} : \{0,1\}^* \rightarrow \{0,1\}^n.

+

The security of cryptographic hash functions will rely on certain tasks being computationally infeasible. +A task is computationally infeasible provided that there is no deterministic algorithm that can conclude the task in polynomial-time.

+

A cryptographic hash function satisfies the following properties:

+
    +
  • Succinct: The hash function should be easy to compute; the hash Hash(b)\mathsf{Hash}({\bf{b}}) can be efficiently computed for any binary string b{\bf{b}}.
    • +
  • Preimage Resistance: It should be computationally infeasible to work backwards given the output of a hash function. Let y{\bf{y}} be a binary string of length nn. +It should be 'impossible' to find some binary string x{\bf{x}} so that y=Hash(x){\bf{y}} = \mathsf{Hash}({\bf{x}}).
    • +
  • Collision Resistance: It should be difficult to find two strings that hash to the same value. +It should be computationally infeasible to find two binary strings x1{\bf{x}_1} and x2{\bf{x}_2} so that Hash(x1)=Hash(x2).\mathsf{Hash}({\bf{x}_1}) = \mathsf{Hash}({\bf{x}_2}).
    • +
+

A related class of functions is one-way functions. A one-way function satisfies the first two conditions of a cryptographic hash function (succinct and preimage resistance). All cryptographic hash functions are a one-way functions. However, one-way functions do not necessarily satisfy collision-resistance. We will simply refer to cryptographic hash functions as hash functions for the rest of this blog. Commonly used hash functions include SHA-256 5, -Keccak 6, and Poseidon 7.

The Fiat-Shamir heuristic​

The Fiat-Shamir heuristic is the technique used to convert an interactive protocol to a noninteractive protocol. +Keccak 6, and Poseidon 7.

+

The Fiat-Shamir heuristic​

+

The Fiat-Shamir heuristic is the technique used to convert an interactive protocol to a noninteractive protocol. This is done by replacing each of the Verifier's messages with a hashed value. -Specifically, the Prover generates the Verifier's message by evaluating the hash function Hash\mathsf{Hash} +Specifically, the Prover generates the Verifier's message by evaluating the hash function Hash\mathsf{Hash} with the concatenation of all public values that appear in the protocol thus far. -If m0,…,mtm_0, \dots, m_t denote the public values in the protocol thus far, -then the Verifier's message is computed as mt+1:=Hash(m0∣∣⋯∣∣mt)m_{t+1} := \mathsf{Hash}(m_0|| \cdots || m_t).

Since Hash\mathsf{Hash} can be efficiently computed, and the messages m0,…,mtm_0, \dots, m_t are public, then any verifying party can compute mt+1m_{t+1}. -Critically, since Hash\mathsf{Hash} is preimage resistant and collision resistant, -the Prover cannot manipulate her choices of the messages m0,…,mtm_0,\dots, m_t to influence the message mt+1m_{t+1}. -Hence, verifying parties can trust that mt+1m_{t+1} is sufficiently random with respect to the preceding messages.

There are two variants of the Fiat-Shamir heuristic: weak and strong. +If m0,…,mtm_0, \dots, m_t denote the public values in the protocol thus far, +then the Verifier's message is computed as mt+1:=Hash(m0∣∣⋯∣∣mt)m_{t+1} := \mathsf{Hash}(m_0|| \cdots || m_t).

+

Since Hash\mathsf{Hash} can be efficiently computed, and the messages m0,…,mtm_0, \dots, m_t are public, then any verifying party can compute mt+1m_{t+1}. +Critically, since Hash\mathsf{Hash} is preimage resistant and collision resistant, +the Prover cannot manipulate her choices of the messages m0,…,mtm_0,\dots, m_t to influence the message mt+1m_{t+1}. +Hence, verifying parties can trust that mt+1m_{t+1} is sufficiently random with respect to the preceding messages.

+

There are two variants of the Fiat-Shamir heuristic: weak and strong. The weak variant uses all of the publicly traded messages in computing the Verifier's messages but does not include the public parameters. However, in the strong variant all of the publicly traded messages and public parameters are used to compute the Verifier's messages. -We will provide a discussion on issues that can arise from using the weak Fiat-Shamir heuristic.

Schnorr Protocol with the strong Fiat-Shamir​

When the strong Fiat-Shamir heuristic is applied to the Schnorr protocol, the message c=Hash(g∣∣X∣∣T)c = \mathsf{Hash}(g||X||T). -This choice of cc provides security since it should be computationally infeasible to find collisions for the outputs of Hash\mathsf{Hash}. -Thus, cc fixes the group elements gg, XX and TT.

The elements that would be omitted in the hash by applying weak Fiat-Shamir heuristic are gg and XX.

Chaum-Pedersen Protocol with the strong Fiat-Shamir​

The message c=Hash(g∣∣U∣∣V∣∣W∣∣T∣∣S)c = Hash(g||U||V||W||T||S) when the Prover applies the strong Fiat-Shamir heuristic to the Chaum-Pedersen protocol. -The properties of Hash\mathsf{Hash} fixes the generator gg and the Prover's statement (U,V,W)(U,V,W).

Improper use of the Fiat-Shamir heuristic​

The Fiat-Shamir heuristic appears to be a fairly straightforward technique to implement. +We will provide a discussion on issues that can arise from using the weak Fiat-Shamir heuristic.

+

Schnorr Protocol with the strong Fiat-Shamir​

+

When the strong Fiat-Shamir heuristic is applied to the Schnorr protocol, the message c=Hash(g∣∣X∣∣T)c = \mathsf{Hash}(g||X||T). +This choice of cc provides security since it should be computationally infeasible to find collisions for the outputs of Hash\mathsf{Hash}. +Thus, cc fixes the group elements gg, XX and TT.

+

The elements that would be omitted in the hash by applying weak Fiat-Shamir heuristic are gg and XX.

+

Chaum-Pedersen Protocol with the strong Fiat-Shamir​

+

The message c=Hash(g∣∣U∣∣V∣∣W∣∣T∣∣S)c = Hash(g||U||V||W||T||S) when the Prover applies the strong Fiat-Shamir heuristic to the Chaum-Pedersen protocol. +The properties of Hash\mathsf{Hash} fixes the generator gg and the Prover's statement (U,V,W)(U,V,W).

+

Improper use of the Fiat-Shamir heuristic​

+

The Fiat-Shamir heuristic appears to be a fairly straightforward technique to implement. However, a subtle but serious issue that can occur in the application of the Fiat-Shamir heuristic has been a point of discussion for the past few years. The issue concerns what messages are included in the hash. -In particular, are the public parameters used to compute the hash value?

Bernhard et al. 8 provide a discussion of Fiat-Shamir heuristic restricted to Σ\Sigma-protocols. +In particular, are the public parameters used to compute the hash value?

+

Bernhard et al. 8 provide a discussion of Fiat-Shamir heuristic restricted to Σ\Sigma-protocols. In particular, Bernhard et al. discuss the pitfalls of the weak Fiat-Shamir heuristic. Recall that the strong Fiat-Shamir heuristic requires that the public parameters are included in the calculations of the Verifier's messages while the weak version does not. The inclusion of the public parameters in the hash evaluations fixes these public values for the entire protocol. -This means that the Prover cannot retroactively change them.

The issues with the differences in the variants of the Fiat-Shamir heuristics has persisted since Bernhard et al.'s paper. +This means that the Prover cannot retroactively change them.

+

The issues with the differences in the variants of the Fiat-Shamir heuristics has persisted since Bernhard et al.'s paper. In recent years, auditors from Trail of Bits and OpenZeppelin have released blogs (9, 10, 11, 12, 13) and papers (14, 15) -describing specific vulnerabilities in zero-knowledge papers and repositories associated with the use of the weak Fiat-Shamir heuristic.

Trail of Bits coined the term FROZEN Heart to describe the use of weak Fiat-Shamir heuristic. +describing specific vulnerabilities in zero-knowledge papers and repositories associated with the use of the weak Fiat-Shamir heuristic.

+

Trail of Bits coined the term FROZEN Heart to describe the use of weak Fiat-Shamir heuristic. Frozen comes from the phrase "FoRging Of ZEro kNowledge proofs", -and Fiat-Shamir is the "heart" of transforming an interactive protocol to noninteractive protocol.

Now, we examine how weak Fiat-Shamir affects the Schnorr protocol and Chaum-Pedersen protocol.

Schnorr protocol with the weak Fiat-Shamir heuristic​

For Schnorr, we will examine two variants: -the first where we only include the Prover's claim XX but not the public parameter gg, and -the second where we include the public parameter gg but not the Prover's claim XX.

Since we omit the generator g∈Gg \in \mathbb{G} from the computation for the message cc in our first approach, -then c=Hash(X∣∣T)c = \mathsf{Hash}(X||T).

Now, a malicious Prover can complete the transcript for the Schnorr protocol by selecting any z∈RZpz \in_R \mathbb{Z}_p. -Since, gg is not fixed as it was not included in the computation of cc. -But, the malicious Prover needs the transcript (T,c,z)(T,c,z) to satisfy gz=TXcg^z = TX^c. -Hence, the malicious Prover can compute the generator g=(TXc)z−1.g = (TX^c)^{z^{-1}}.

In our second approach, we omit the group element X∈GX \in \mathbb{G} from the computation for the challenge cc. -That is, c=Hash(g∣∣T)c = \mathsf{Hash}(g||T).

As with the previous example, the malicious Prover takes a Schnorr transcript (T,c,z)(T,c,z) where z∈RZpz \in_R \mathbb{Z}_p. -It is necessary for the malicious Prover to find a value XX so that gz=TXcg^z = TX^c. -This can be acheived by computing X=(gzT−1)c−1X = (g^z T^{-1})^{c^{-1}}.

Chaum-Pedersen protocol with the Fiat-Shamir heuristic​

The Verifier's message c=Hash(T,S)c = Hash(T,S) when weak Fiat-Shamir heuristic is applied. -The Prover's triple (U,V,W)(U,V,W) and the generator gg are not fixed by cc. -As such, a malicious Prover can generate values for U,V,WU,V,W, and gg that satisfy the Verifier's identity checks. -In the case of a malicious Prover, TT and SS are randomly group elements instead of being computed using a value tt that the Prover selected. -This means a malicious Prover must randomly select the value zz as well.

Given the values that have been fixed so far, each of the Verifier's identities consists of two unknowns. +and Fiat-Shamir is the "heart" of transforming an interactive protocol to noninteractive protocol.

+

Now, we examine how weak Fiat-Shamir affects the Schnorr protocol and Chaum-Pedersen protocol.

+

Schnorr protocol with the weak Fiat-Shamir heuristic​

+

For Schnorr, we will examine two variants: +the first where we only include the Prover's claim XX but not the public parameter gg, and +the second where we include the public parameter gg but not the Prover's claim XX.

+

Since we omit the generator g∈Gg \in \mathbb{G} from the computation for the message cc in our first approach, +then c=Hash(X∣∣T)c = \mathsf{Hash}(X||T).

+

Now, a malicious Prover can complete the transcript for the Schnorr protocol by selecting any z∈RZpz \in_R \mathbb{Z}_p. +Since, gg is not fixed as it was not included in the computation of cc. +But, the malicious Prover needs the transcript (T,c,z)(T,c,z) to satisfy gz=TXcg^z = TX^c. +Hence, the malicious Prover can compute the generator g=(TXc)z−1.g = (TX^c)^{z^{-1}}.

+

In our second approach, we omit the group element X∈GX \in \mathbb{G} from the computation for the challenge cc. +That is, c=Hash(g∣∣T)c = \mathsf{Hash}(g||T).

+

As with the previous example, the malicious Prover takes a Schnorr transcript (T,c,z)(T,c,z) where z∈RZpz \in_R \mathbb{Z}_p. +It is necessary for the malicious Prover to find a value XX so that gz=TXcg^z = TX^c. +This can be acheived by computing X=(gzT−1)c−1X = (g^z T^{-1})^{c^{-1}}.

+

Chaum-Pedersen protocol with the Fiat-Shamir heuristic​

+

The Verifier's message c=Hash(T,S)c = Hash(T,S) when weak Fiat-Shamir heuristic is applied. +The Prover's triple (U,V,W)(U,V,W) and the generator gg are not fixed by cc. +As such, a malicious Prover can generate values for U,V,WU,V,W, and gg that satisfy the Verifier's identity checks. +In the case of a malicious Prover, TT and SS are randomly group elements instead of being computed using a value tt that the Prover selected. +This means a malicious Prover must randomly select the value zz as well.

+

Given the values that have been fixed so far, each of the Verifier's identities consists of two unknowns. Hence, it is necessary to select one of these unknowns from each identity so that a malicious Prover can compute the last value. -For instances, suppose that the malicious Prover randomly selects VV and WW. -The malicious Prover can compute g=(TVc)1/zg = (T V^c)^{1/z} and V=(SWc)1/zV = (SW^c)^{1/z}. -Thus, the malicious Prover has a claimed statement (U,V,W)(U,V,W) for generator gg that passes the Verifier's identities using weak Fiat-Shamir heuristic.

The omission of any of the values U,V,W,U,V,W, and gg in the computation of cc allows a malicious Prover to forge a proof.

Conclusion​

The Fiat-Shamir heuristic is an essential technique to convert an interactive protocol to a variant that does not require communication. -Additionally, careful application of this technique is necessary to maintain the integrity of the system.

References​

]]> +For instances, suppose that the malicious Prover randomly selects VV and WW. +The malicious Prover can compute g=(TVc)1/zg = (T V^c)^{1/z} and V=(SWc)1/zV = (SW^c)^{1/z}. +Thus, the malicious Prover has a claimed statement (U,V,W)(U,V,W) for generator gg that passes the Verifier's identities using weak Fiat-Shamir heuristic.

+

The omission of any of the values U,V,W,U,V,W, and gg in the computation of cc allows a malicious Prover to forge a proof.

+

Conclusion​

+

The Fiat-Shamir heuristic is an essential technique to convert an interactive protocol to a variant that does not require communication. +Additionally, careful application of this technique is necessary to maintain the integrity of the system.

+

References​

+]]> Marvin @@ -799,61 +1528,281 @@ Additionally, careful application of this technique is necessary to maintain the https://vac.dev/rlog/zkVM-testing 2024-09-26T12:00:00.000Z - - Introduction

Following our initial exploration of zkVMs in our previous blog post [1], +

+ Following our initial exploration of zkVMs in our previous blog post [1], we have conducted a series of tests to identify the most suitable zkVM for the Nescience architecture [2]. This post outlines the testing process, results, and conclusions. Additionally, the full test suite and scripts can be found on our GitHub page [3], allowing others to replicate the results or explore the candidates further. Please note that we chose not to use hardware acceleration in our benchmarks, as our project is aimed at a broad audience. -Particularly, we cannot assume AVX512 support by default, as it is typically available only in high-end CPUs.

We've shortlisted the following zkVMs for testing:

  • SP1 [4]
  • RISC0 [5]
  • Nexus [6]
  • ZkMIPS [7]
  • ZkWASM [8]
  • Valida [9]

Why these candidates?

When narrowing down the zkVMs, we focused on key factors:

  • True zero-knowledge functionality: The zkVMs had to demonstrate or be close to demonstrating the ability to generate and verify zero-knowledge proofs.
  • Performance baselines: We sought zkVMs with solid benchmarks in performance, particularly in speed and efficiency.
  • Specific functionalities: For Nescience, functionalities like lookup tables, precompiles, and recursive capabilities are critical.

We need a zkVM that supports these to enable robust project development.

Preliminary information on the candidates

  1. SP1 is a performant, open-source zkVM that verifies the execution of arbitrary Rust (or any LLVM-compiled language) programs. +Particularly, we cannot assume AVX512 support by default, as it is typically available only in high-end CPUs.

    +

    We've shortlisted the following zkVMs for testing:

    +
      +
    • SP1 [4]
      • +
    • RISC0 [5]
      • +
    • Nexus [6]
      • +
    • ZkMIPS [7]
      • +
    • ZkWASM [8]
      • +
    • Valida [9]
      • +
    +

    Why these candidates?

    +

    When narrowing down the zkVMs, we focused on key factors:

    +
      +
    • True zero-knowledge functionality: The zkVMs had to demonstrate or be close to demonstrating the ability to generate and verify zero-knowledge proofs.
      • +
    • Performance baselines: We sought zkVMs with solid benchmarks in performance, particularly in speed and efficiency.
      • +
    • Specific functionalities: For Nescience, functionalities like lookup tables, precompiles, and recursive capabilities are critical.
      • +
    +

    We need a zkVM that supports these to enable robust project development.

    +

    Preliminary information on the candidates

    +
      +
    1. +

      SP1 is a performant, open-source zkVM that verifies the execution of arbitrary Rust (or any LLVM-compiled language) programs. SP1 utilizes Plonky3, enabling recursive proofs and supporting a wide range of cryptographic algorithms, including ECC-based ones like Groth16. -While it supports aggregation, it appears not to support zero knowledge in a conventional manner.

    2. RISC0 zkVM allows one to prove the correct execution of arbitrary Rust code. Built on a RISC-V architecture, it is inherently adaptable +While it supports aggregation, it appears not to support zero knowledge in a conventional manner.

      +
      3. +
    3. +

      RISC0 zkVM allows one to prove the correct execution of arbitrary Rust code. Built on a RISC-V architecture, it is inherently adaptable for implementing standard cryptographic hash functions such as SHA-256 and ECDSA. RISC0 employs STARKs, providing a security level of 98 bits. -It supports multiple programming languages, including C and Rust, thanks to its compatibility with LLVM and WASM.

    4. Nexus is a modular, extensible, open-source, highly parallelized, prover-optimized, and contributor-friendly zkVM written in Rust. +It supports multiple programming languages, including C and Rust, thanks to its compatibility with LLVM and WASM.

      +
      5. +
    5. +

      Nexus is a modular, extensible, open-source, highly parallelized, prover-optimized, and contributor-friendly zkVM written in Rust. It focuses on performance and security, using the Nova folding scheme, which is particularly effective for recursive proofs. -Nexus also supports precompiles and targeted compilation, and besides Rust, it offers C++ support.

    6. ZkMIPS is a general verifiable computing infrastructure based on Plonky2 and the MIPS microarchitecture, aiming to empower Ethereum -as a global settlement layer. It can run arbitrary Rust code as well. Notably, zkMIPS is the only zkVM in this list that utilizes the MIPS opcode set.

    7. ZkWASM adheres to and supports the unmodified standard WASM bytecode specification. Since Rust code can be compiled to WASM bytecode, -one could theoretically run any Rust code on a zkWASM machine, providing flexibility and broad language support.

    8. Valida is a STARK-based virtual machine aiming to improve upon the state of the art in several categories:

      • Code reuse: The VM has a RISC-inspired instruction set, simplifying the targeting of conventional programming languages. +Nexus also supports precompiles and targeted compilation, and besides Rust, it offers C++ support.

        +
        • +
      • +

        ZkMIPS is a general verifiable computing infrastructure based on Plonky2 and the MIPS microarchitecture, aiming to empower Ethereum +as a global settlement layer. It can run arbitrary Rust code as well. Notably, zkMIPS is the only zkVM in this list that utilizes the MIPS opcode set.

        +
        • +
      • +

        ZkWASM adheres to and supports the unmodified standard WASM bytecode specification. Since Rust code can be compiled to WASM bytecode, +one could theoretically run any Rust code on a zkWASM machine, providing flexibility and broad language support.

        +
        • +
      • +

        Valida is a STARK-based virtual machine aiming to improve upon the state of the art in several categories:

        +
          +
        • Code reuse: The VM has a RISC-inspired instruction set, simplifying the targeting of conventional programming languages. A backend compiler is being developed to compile LLVM IR to the Valida ISA, enabling the proving of programs written in Rust, -Go, C++, and others with minimal to no changes in source code.
        • Prover performance: Engineered to maximize prover performance, Valida is compatible with a 31-bit field, restricted to degree 3 constraints, +Go, C++, and others with minimal to no changes in source code.
          • +
        • Prover performance: Engineered to maximize prover performance, Valida is compatible with a 31-bit field, restricted to degree 3 constraints, and features minimal instruction decoding. It operates directly on memory without general-purpose registers or a dedicated stack, -utilizing newer lookup arguments to reduce trace overhead involved in cross-chip communication.
        • Extensibility: Designed to be customizable, Valida can easily be extended to include an arbitrary number of user-defined instructions. -Procedural macros are used to construct the desired machine at compile time, avoiding any runtime penalties.

    Valida appears to be in the early stages of development but already showcases respectable performance metrics.

    Testing plan

    To thoroughly evaluate each zkVM, we devised a two-stage testing process:

    • Stage 1: Arithmetic operations

      The first phase focused on evaluating the zkVMs’ ability to handle basic arithmetic operations: addition, subtraction, multiplication, +utilizing newer lookup arguments to reduce trace overhead involved in cross-chip communication.

      • +
    • Extensibility: Designed to be customizable, Valida can easily be extended to include an arbitrary number of user-defined instructions. +Procedural macros are used to construct the desired machine at compile time, avoiding any runtime penalties.
      • +
    +
    2. +
+

Valida appears to be in the early stages of development but already showcases respectable performance metrics.

+

Testing plan

+

To thoroughly evaluate each zkVM, we devised a two-stage testing process:

+
    +
  • +

    Stage 1: Arithmetic operations

    +

    The first phase focused on evaluating the zkVMs’ ability to handle basic arithmetic operations: addition, subtraction, multiplication, division, modulus division, and square root calculations. We designed the test around heptagonal numbers, which required zkVMs to process multiple arithmetic operations simultaneously. By using this method, we could measure efficiency and speed in handling complex mathematical calculations – -a crucial element for zkVM performance.

  • Stage 2: Memory consumption

    For the second phase, we evaluated each zkVM’s ability to manage memory under heavy loads. We tested several data structures, including lists, -hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent tests for the following operations:

    • Insert: How quickly can the zkVM add data to structures?
    • Delete: Does the zkVM handle memory release effectively?
    • Append: Can the zkVM efficiently grow data structures?
    • Search: How fast and efficient is the zkVM when retrieving stored data?

The purpose of this stage was to identify any memory bottlenecks and to determine whether a zkVM could manage high-intensity tasks efficiently, -something vital for the Nescience project’s complex, data-heavy processes.

Machine specifications

The tests were conducted on the following hardware configuration:

  • CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
  • RAM: 600GiB DDR4 3200MHz ECC RAM, distributed across 16 DIMMs
  • Host OS: Proxmox 8.3
  • Hypervisor: KVM
  • Network layer: Open vSwitch
  • Machine model: Supermicro AS-2024US-TRT

Results

1. SP1​

SP1 does not provide zero-knowledge capability in its proofs but delivers respectable performance, though slightly behind its main competitor. +a crucial element for zkVM performance.

+ +
  • +

    Stage 2: Memory consumption

    +

    For the second phase, we evaluated each zkVM’s ability to manage memory under heavy loads. We tested several data structures, including lists, +hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent tests for the following operations:

    +
      +
    • Insert: How quickly can the zkVM add data to structures?
      • +
    • Delete: Does the zkVM handle memory release effectively?
      • +
    • Append: Can the zkVM efficiently grow data structures?
      • +
    • Search: How fast and efficient is the zkVM when retrieving stored data?
      • +
    +
    • + +

    The purpose of this stage was to identify any memory bottlenecks and to determine whether a zkVM could manage high-intensity tasks efficiently, +something vital for the Nescience project’s complex, data-heavy processes.

    +

    Machine specifications

    +

    The tests were conducted on the following hardware configuration:

    +
      +
    • CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
      • +
    • RAM: 600GiB DDR4 3200MHz ECC RAM, distributed across 16 DIMMs
      • +
    • Host OS: Proxmox 8.3
      • +
    • Hypervisor: KVM
      • +
    • Network layer: Open vSwitch
      • +
    • Machine model: Supermicro AS-2024US-TRT
      • +
    +

    Results

    +

    1. SP1​

    +

    SP1 does not provide zero-knowledge capability in its proofs but delivers respectable performance, though slightly behind its main competitor. Memory leaks were minimal, staying below the 700 KB threshold. Interestingly, SP1 consumed more RAM during the basic arithmetic test than in memory allocation tests, showcasing the team's effective handling of memory under load. In the basic test, -allocations were primarily in the 9-16 B, 33-64 B, and 65-128 B ranges. For memory allocations, most fell into the 129-256 B range.

    • Stage 1: Hept 100 test
      • Proof size: 3.108 MB
      • Proof time: 16.95 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 3.17 MB
      • Proof time: 20.85 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    2. RISC0​

    RISC0 stands out with exceptional performance in proof size and generation time, ranking among the best +allocations were primarily in the 9-16 B, 33-64 B, and 65-128 B ranges. For memory allocations, most fell into the 129-256 B range.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 3.108 MB
        • +
      • Proof time: 16.95 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 3.17 MB
        • +
      • Proof time: 20.85 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    2. RISC0​

    +

    RISC0 stands out with exceptional performance in proof size and generation time, ranking among the best (with the exception of Valida and zkWASM's basic test). It also handles memory well, with minor leaks under 0.5 MB -and controlled RAM consumption staying below 2.2 GB. RISC0's memory allocations were consistent, primarily in the 17-32 B and 33-64 B ranges.

    • Stage 1: Hept 100 test
      • Proof size: 217.4 KB
      • Proof time: 9.73 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 217.4 KB
      • Proof time: 16.63 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    Based on these results, RISC0 is a solid candidate for Nescience.

    3. Nexus​

    Nexus' performance offers interesting insights into folding scheme-based zkVMs. Surprisingly, proof sizes remained consistent +and controlled RAM consumption staying below 2.2 GB. RISC0's memory allocations were consistent, primarily in the 17-32 B and 33-64 B ranges.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 217.4 KB
        • +
      • Proof time: 9.73 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 217.4 KB
        • +
      • Proof time: 16.63 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +

    Based on these results, RISC0 is a solid candidate for Nescience.

    +
    +

    3. Nexus​

    +

    Nexus' performance offers interesting insights into folding scheme-based zkVMs. Surprisingly, proof sizes remained consistent regardless of workload, with no significant memory leaks (under 700 KB). However, while RAM consumption increased slightly with higher -workloads (up to 1.2 GB), Nexus performed poorly during memory allocation tests, making it unsuitable for our use case.

    • Allocation details:

      • Basic test: Most allocations concentrated in 65-128 B
      • Memory-heavy test: Allocations in the 129-256 B range

    • Stage 1: Hept 100 test

      • Proof size: 46 MB
      • Proof time: 12.06 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 46 MB
      • Proof time: 56 minutes
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    4. ZkMIPS​

    ZkMIPS presents an intriguing case. While it shows good results in terms of proof size and generation time during the basic test, +workloads (up to 1.2 GB), Nexus performed poorly during memory allocation tests, making it unsuitable for our use case.

    +
      +
    • +

      Allocation details:

      +
        +
      • Basic test: Most allocations concentrated in 65-128 B
        • +
      • Memory-heavy test: Allocations in the 129-256 B range
        • +
      +
      • +
    • +

      Stage 1: Hept 100 test

      +
        +
      • Proof size: 46 MB
        • +
      • Proof time: 12.06 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 46 MB
        • +
      • Proof time: 56 minutes
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    4. ZkMIPS​

    +

    ZkMIPS presents an intriguing case. While it shows good results in terms of proof size and generation time during the basic test, these come at the cost of significant RAM usage and memory leaks. The memory allocation test revealed a concerning 6.7 GB memory leak, with 0.5 GB leaked during the basic test. Despite this, RAM consumption (while high at 17+ GB) remains stable under higher workloads. -Allocation sizes are spread across several ranges, with notable concentrations in the 17-32 B, 65-128 B, and 257-512 B slots.

    • Stage 1: Hept 100 test
      • Proof size: 4.3 MB
      • Proof time: 9.32 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 4.898 MB
      • Proof time: 42.57 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    This zkVM provides mixed results with strong proof generation but concerning memory management issues.

    5. ZkWASM​

    ZkWASM, unfortunately, performed poorly in both stages regarding proof size and generation time. RAM consumption was particularly high, +Allocation sizes are spread across several ranges, with notable concentrations in the 17-32 B, 65-128 B, and 257-512 B slots.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 4.3 MB
        • +
      • Proof time: 9.32 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 4.898 MB
        • +
      • Proof time: 42.57 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +

    This zkVM provides mixed results with strong proof generation but concerning memory management issues.

    +
    +

    5. ZkWASM​

    +

    ZkWASM, unfortunately, performed poorly in both stages regarding proof size and generation time. RAM consumption was particularly high, exceeding 7 GB in the basic test, and an astounding 57 GB during memory allocation tests. Despite its impressive memory usage, the proof sizes were relatively large at 18 KB and 334 KB respectively. Allocation sizes were mainly concentrated in the 33-64 B range, -with neighboring slots contributing small but notable amounts.

    • Stage 1: Hept 100 test
      • Proof size: 18 KB
      • Proof time: 42.7 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 334 KB
      • Proof time: 323 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    6. Valida​

    Valida delivered impressive results in proof generation speed and size, with a proof size of 280 KB and a proof time of < 1 second. +with neighboring slots contributing small but notable amounts.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 18 KB
        • +
      • Proof time: 42.7 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 334 KB
        • +
      • Proof time: 323 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    6. Valida​

    +

    Valida delivered impressive results in proof generation speed and size, with a proof size of 280 KB and a proof time of < 1 second. However, profiling was not possible due to Valida's limited Rust support. Valida currently compiles Rust using the LLVM backend, transpiling LLVM IR to leverage its C/C++ implementation, which leads to errors when handling Rust-specific data structures or dependencies. As a result, complex memory interactions couldn't be tested, and using Valida with Rust code is currently not advisable. -A GitHub issue has been opened to address this.

    Summary table​

    Stage 1​

    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP116.95 s3.108 MB2.1 GB656.8 KB
    RISC09.73 s217.4 KB1.9 GB470.5 KB
    Nexus12.06 s46 MB9.7 MB646.5 KB
    ZkMIPS9.32 s4.3 MB17.3 GB453.8 MB
    ZkWASM42.7 s18 KB8.2 GB259.4 KB
    Valida< 1 s280 KBN/AN/A

    Stage 2​

    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP120.85 s3.17 MB1.9 GB616 KB
    RISC016.63 s217.4 KB2.3 GB485.3 KB
    Nexus56 m46 MB1.9 GB616 KB
    ZkMIPS42.57 s4.898 MB18.9 GB6.9 GB
    ZkWASM323 s334 KB58.8 GB259.4 KB
    ValidaN/AN/AN/AN/A

    Summary

    After an extensive evaluation of six zkVM candidates for the Nescience project, RISC0 emerged as the top choice. +A GitHub issue has been opened to address this.

    +
    +

    Summary table​

    +

    Stage 1​

    +
    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP116.95 s3.108 MB2.1 GB656.8 KB
    RISC09.73 s217.4 KB1.9 GB470.5 KB
    Nexus12.06 s46 MB9.7 MB646.5 KB
    ZkMIPS9.32 s4.3 MB17.3 GB453.8 MB
    ZkWASM42.7 s18 KB8.2 GB259.4 KB
    Valida< 1 s280 KBN/AN/A
    +
    +

    Stage 2​

    +
    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP120.85 s3.17 MB1.9 GB616 KB
    RISC016.63 s217.4 KB2.3 GB485.3 KB
    Nexus56 m46 MB1.9 GB616 KB
    ZkMIPS42.57 s4.898 MB18.9 GB6.9 GB
    ZkWASM323 s334 KB58.8 GB259.4 KB
    ValidaN/AN/AN/AN/A
    +
    +

    Summary

    +

    After an extensive evaluation of six zkVM candidates for the Nescience project, RISC0 emerged as the top choice. It excels in both proof generation time and size while maintaining a reasonable memory footprint. With strong zero-knowledge proof capabilities and support for multiple programming languages, it aligns well with our project's needs for privacy, -performance, and flexibility. Its overall balance between performance and efficiency makes it the most viable zkVM at this stage.

    Valida, while promising with its potential for high prover performance, is still in early development and suffers from Rust integration issues. +performance, and flexibility. Its overall balance between performance and efficiency makes it the most viable zkVM at this stage.

    +

    Valida, while promising with its potential for high prover performance, is still in early development and suffers from Rust integration issues. The current LLVM IR transpilation limitations mean it cannot handle complex memory interactions, disqualifying it for now. -However, once its development matures, Valida could become a strong alternative, and we plan to revisit it as it evolves.

    SP1, though initially interesting, failed to meet the zero-knowledge proof requirement. Its performance in arithmetic operations was -respectable but insufficient to justify further consideration given its lack of ZK functionality – critical for our privacy-first objectives.

    Nexus demonstrated consistent proof sizes and manageable memory usage, but its lackluster performance during memory-intensive tasks and +However, once its development matures, Valida could become a strong alternative, and we plan to revisit it as it evolves.

    +

    SP1, though initially interesting, failed to meet the zero-knowledge proof requirement. Its performance in arithmetic operations was +respectable but insufficient to justify further consideration given its lack of ZK functionality – critical for our privacy-first objectives.

    +

    Nexus demonstrated consistent proof sizes and manageable memory usage, but its lackluster performance during memory-intensive tasks and its proof size (especially for larger workloads) disqualified it from being a top contender. While zkMIPS delivered solid proof times, -the memory issues were too significant to ignore, making it unsuitable.

    Finally, zkWASM exhibited the poorest results, struggling both in proof size and generation time. Despite its potential for WASM bytecode support, -the excessive RAM consumption (up to 57 GB in the memory test) rendered it impractical for Nescience’s use case.

    In conclusion, RISC0 is the best fit for Nescience at this stage, but Valida remains a future candidate as its development progresses.

    In the future, we plan to compare RISC0 and SP1 with CUDA acceleration. Ideally, by that time, more zkVMs will include similar acceleration capabilities, -enabling a fairer and more comprehensive comparison across platforms.

    We’d love to hear your thoughts on our zkVM testing process and results! Do you agree with our conclusions, or do you think we missed a promising zkVM? -We’re always open to feedback, insights, and suggestions from the community.

    Join the discussion and share your perspectives on +the memory issues were too significant to ignore, making it unsuitable.

    +

    Finally, zkWASM exhibited the poorest results, struggling both in proof size and generation time. Despite its potential for WASM bytecode support, +the excessive RAM consumption (up to 57 GB in the memory test) rendered it impractical for Nescience’s use case.

    +

    In conclusion, RISC0 is the best fit for Nescience at this stage, but Valida remains a future candidate as its development progresses.

    +

    In the future, we plan to compare RISC0 and SP1 with CUDA acceleration. Ideally, by that time, more zkVMs will include similar acceleration capabilities, +enabling a fairer and more comprehensive comparison across platforms.

    +

    We’d love to hear your thoughts on our zkVM testing process and results! Do you agree with our conclusions, or do you think we missed a promising zkVM? +We’re always open to feedback, insights, and suggestions from the community.

    +

    Join the discussion and share your perspectives on our forum or try out the -tests yourself through our GitHub page!

    References

    [1] Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? Retrieved from https://vac.dev/rlog/zkVM-explorations/

    [2] Nescience: A User-Centric State-Separation Architecture. Retrieved from https://vac.dev/rlog/Nescience-state-separation-architecture

    [3] Our GitHub Page for zkVM Testing. Retrieved from https://github.com/vacp2p/nescience-zkvm-testing

    [4] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    [5] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    [6] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    [7] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    [9] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    ]]>     +tests yourself through our GitHub page!

    +

    References

    +

    [1] Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? Retrieved from https://vac.dev/rlog/zkVM-explorations/

    +

    [2] Nescience: A User-Centric State-Separation Architecture. Retrieved from https://vac.dev/rlog/Nescience-state-separation-architecture

    +

    [3] Our GitHub Page for zkVM Testing. Retrieved from https://github.com/vacp2p/nescience-zkvm-testing

    +

    [4] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    +

    [5] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    +

    [6] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    +

    [7] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    +

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    +

    [9] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    ]]>     Moudy @@ -863,58 +1812,308 @@ tests yourself through our 2024-08-27T12:00:00.000Z - - Introduction

    The blockchain space is rapidly evolving, and with it, new technologies are emerging that promise enhanced privacy, scalability, and security. +

    + The blockchain space is rapidly evolving, and with it, new technologies are emerging that promise enhanced privacy, scalability, and security. As decentralized systems grow in complexity and usage, the need for secure and private computation has never been greater. Zero-knowledge virtual machines (zkVMs) are one such innovation, allowing for computations to be proven correct without revealing the underlying data. ZkVMs have enormous implications for privacy-preserving applications, decentralized finance (DeFi), and other blockchain-based use cases. -However, as the term "zkVM" becomes more widely adopted, it is critical to distinguish between projects that truly satisfy the stringent requirements of a zkVM and those that do not.

    What is a zkVM?

    A zkVM is a virtual machine that combines the principles of cryptographic proof generation and privacy preservation with the computational model +However, as the term "zkVM" becomes more widely adopted, it is critical to distinguish between projects that truly satisfy the stringent requirements of a zkVM and those that do not.

    +

    What is a zkVM?

    +

    A zkVM is a virtual machine that combines the principles of cryptographic proof generation and privacy preservation with the computational model of traditional virtual machines. Essentially, a zkVM enables the execution of arbitrary programs while generating cryptographic proofs—specifically, zero-knowledge proofs (ZKPs)—that can verify the correctness of these computations without revealing any sensitive information. This ensures that computations can be trusted while protecting the privacy of the data involved. -The key characteristics of a zkVM include:

    • Proof generation: The ability to produce ZKPs that verify the correct execution of programs. There are several types of cryptographic techniques used in zkVMs to +The key characteristics of a zkVM include:

      +
        +
      • Proof generation: The ability to produce ZKPs that verify the correct execution of programs. There are several types of cryptographic techniques used in zkVMs to generate these proofs, such as zk-SNARKs, zk-STARKs, and recursive proofs. A zkVM’s ability to generate these proofs determines how effectively it can ensure the integrity of computations -in a privacy-preserving manner.
      • Privacy preservation: The system must maintain privacy, ensuring that only the proof is revealed, not the underlying computation or data. Privacy-preserving zkVMs allow users to maintain +in a privacy-preserving manner.
        • +
      • Privacy preservation: The system must maintain privacy, ensuring that only the proof is revealed, not the underlying computation or data. Privacy-preserving zkVMs allow users to maintain confidentiality without compromising the security or verifiability of their operations. However, not all zkVMs achieve the same level of privacy. Some may focus more on proof generation -and scalability while deprioritizing privacy features, which can limit their use in certain privacy-sensitive applications.
      • Scalability and performance: zkVMs should offer scalable and efficient computation, leveraging advanced cryptographic techniques like zk-SNARKs, zk-STARKs, or recursive proofs. -A zkVM's performance must also be measured in terms of latency (time to generate and verify a proof) and throughput (number of computations processed within a certain time frame).
      • Verifiable computation: The zkVM should be able to prove the execution of arbitrary programs in a secure and verifiable manner. Verifiable computation ensures that zkVMs can be deployed -across a wide range of applications, from DeFi to private data-sharing platforms and more.

      Why zkVMs matter

      The rise of zkVMs is a crucial development for the future of blockchain and decentralized technologies. As more systems require the ability to scale while maintaining privacy and trust, -zkVMs provide a powerful solution. They offer the potential to reshape the way decentralized applications (dapps) handle sensitive information, enabling them to be both efficient and private.

      It is essential to distinguish between projects that fully realize the potential of zkVMs and those that do not. In the remainder of this post, we evaluate several zkVM projects, analyzing -whether they satisfy the criteria for being classified as zkVMs based on our research.

      Our methodology

      We analyzed each project’s documentation, source code, and available benchmarks to determine whether they meet the definition of a zkVM. -Our criteria focus on the key capabilities of zkVMs—proof generation, privacy, scalability, and integration with existing systems.

      ZkVM project analysis

      1. [SP1]​

      • Overview: SP1 [1] is a developer-friendly zkVM designed to enable ZKP execution for LLVM-based languages like C, C++, Rust, and others. It supports a RISC-V-like instruction set architecture (ISA), -which makes it compatible with various programming languages compiled through LLVM.
      • Main focus: The main focus of SP1 is scalability, open-source contributions, and accessibility for developers. It prioritizes performance over privacy, -making it a good fit for environments where privacy isn't the primary concern.
      • Privacy: Not explicitly mentioned, making it less suitable for privacy-preserving applications.
      • Performance: SP1 has demonstrated up to 5.4x better performance than similar zkVMs like RISC0 for specific computations such as Fibonacci sequence generation.
      • Integration: SP1 is highly adaptable for rollups, light client verifiers, oracles, and even web2 projects like verifying the originality of images.
      • Conclusion: Yes, SP1 is a zkVM, but it does not prioritize zero-knowledge privacy, focusing more on scalability and performance.

      2. [Nexus]​

      • Overview: Nexus [2] is a highly modular zkVM designed to process up to a trillion CPU cycles per second. It relies on RISC-V instructions for computation, making it extensible and scalable. -However, it currently lacks full ZKP capabilities due to its use of Spartan proofs.
      • Main focus: Nexus focuses on high performance and scalability, aiming to create an efficient execution environment for computationally intensive tasks.
      • Privacy: Although zero-knowledge privacy isn't the primary feature of Nexus, the project hints at potential privacy enhancements in the future.
      • Performance: Nexus has a high theoretical throughput, but it has yet to demonstrate benchmarks on zero-knowledge privacy.
      • Integration: Nexus is a good fit for high-performance environments that do not necessarily require full privacy.
      • Conclusion: Yes, Nexus qualifies as a zkVM in terms of scalability and proof generation, but it does not yet achieve full zero-knowledge privacy.

      3. [RISC0]​

      • Overview: Risc0 [3] is a general-purpose zkVM with strong developer support. It allows for the execution of Rust and C code on a RISC-V virtual machine -and generates zk-SNARK and zk-STARK proofs for these computations.
      • Main focus: Risc0 is focused on ease of use for developers by abstracting away the complexities of circuit generation, making it accessible for a wide range of use cases.
      • Privacy: Full zero-knowledge privacy is supported via zk-SNARK and zk-STARK proofs, with Groth16 used for constant-size proof generation.
      • Performance: Risc0 offers strong benchmarks across different hardware setups, making it one of the most versatile zkVMs in terms of performance and scalability.
      • Integration: Risc0 integrates with several ecosystems, including Ethereum, and supports verifiable execution of Rust-based programs.
      • Conclusion: Yes, Risc0 qualifies as a zkVM, offering a balance of developer usability, scalability, and privacy.

      4. [Powdr]​

      • Overview: Powdr [4] is a toolkit for creating custom zkVMs. It allows developers to select from various front-end and back-end components to create zkVMs tailored to specific needs.
      • Main focus: Powdr is focused on providing a modular architecture for zkVM creation. It enables flexibility by allowing the combination of different ZK-proof backends like Halo2 or Valida.
      • Privacy: Powdr itself does not generate ZKPs, but it facilitates the creation of zkVMs that do.
      • Performance: The performance depends on the components chosen by the developer, as Powdr itself is more of a framework.
      • Integration: Powdr is highly customizable and can integrate with existing zkVM frameworks to extend their capabilities.
      • Conclusion: No, Powdr is not a zkVM itself, but it is a powerful tool for building customized zkVMs with different privacy and performance needs.

      5. [ZkMIPS]​

      • Overview: ZkMIPS [5] uses zk-STARKs to ensure privacy during computation, ensuring that private inputs are preserved while still proving correctness.
      • Performance: ZkMIPS is built for scalability, though explicit benchmarks are not widely published.
      • Integration: ZkMIPS can be integrated into systems that rely on MIPS architecture, making it versatile for legacy codebases that require privacy.
      • Conclusion: Yes, zkMIPS is a zkVM focused on scalability and privacy for MIPS-based architectures.

      6. [Valida]​

      • Overview: Valida [6] is a performance-oriented zkVM that generates proofs for programs using a custom ISA designed to optimize zkVM implementation. -It uses Plonky3 for its proof system.
      • Main focus: Valida is centered around optimizing prover performance and extensibility, making it a valuable tool for generating proofs efficiently.
      • Privacy: While Valida is focused on performance, it does not prioritize zero-knowledge privacy as much as other zkVMs.
      • Performance: Valida has benchmarks indicating its performance advantages in proving computations quickly, particularly through parallel processing.
      • Integration: Valida is specialized and may not integrate as seamlessly into general-purpose systems, as it is optimized for performance over broad applicability.
      • Conclusion: Yes, Valida qualifies as a zkVM based on proof generation, but its lack of focus on privacy makes it less suitable for privacy-first use cases.

      7. [Jolt]​

      • Overview: Jolt [7] is a zkVM built to optimize prover performance using a modified Hyrax polynomial commitment system. It relies on RISC-V instructions for computation -but falls short of full zero-knowledge capabilities.
      • Main focus: Jolt's main goal is to optimize the speed of proving program execution, making it suitable for high-performance applications where privacy isn't the primary concern.
      • Privacy: Jolt does not fully achieve zero-knowledge privacy due to the choice of polynomial commitment schemes.
      • Performance: Jolt offers strong performance, with benchmarks highlighting its ability to process proofs efficiently.
      • Integration: Jolt can be integrated with systems that prioritize speed over privacy, particularly where rapid proof generation is essential.
      • Conclusion: Yes, Jolt qualifies as a zkVM based on proof generation, though it does not provide full zero-knowledge privacy.

      8. [ZkWASM]​

      • Overview: ZkWASM [8] is a zkVM designed to execute WebAssembly (WASM) code in a privacy-preserving and scalable manner. It uses zk-SNARKs to prove the correctness of WASM -program execution while ensuring privacy.
      • Main focus: ZkWASM focuses on scalability and privacy for WebAssembly, making it ideal for dapps that require verifiable computation without compromising privacy.
      • Privacy: Full zero-knowledge privacy is provided through zk-SNARKs, ensuring that the execution of WASM programs remains confidential.
      • Performance: ZkWASM is optimized for running WASM programs efficiently, with offchain computation and onchain verification to enhance performance.
      • Integration: ZkWASM is ideal for dapps, particularly those that use WebAssembly and require verifiable execution.
      • Conclusion: Yes, zkWASM qualifies as a zkVM, providing strong privacy, scalability, and verifiable execution for WebAssembly code.

      9. [Aleo]​

      • Overview: Aleo's [9] snarkVM converts code into Aleo instructions, which are then compiled into bytecode executable on its zkVM. Aleo emphasizes building private, scalable dapps.
      • Main focus: Aleo prioritizes privacy and scalability for dapps, providing a robust framework for developers building private dapps.
      • Privacy: Aleo offers full privacy through zk-SNARK proofs, making it suitable for building fully private applications.
      • Performance: Aleo focuses on scalability through efficient proof systems, though detailed performance benchmarks are not widely available.
      • Integration: Aleo is built for privacy-first dapps and integrates with other zkVM-based systems.
      • Conclusion: Yes, Aleo qualifies as a zkVM, offering a comprehensive solution for private and scalable dapps.

      10. [Ola]​

      • Overview: Ola [10] is a ZK-friendly, high-performance layer-2 (L2) rollup platform that is still under development. It is designed to execute computations offchain while generating -validity proofs for these computations, ensuring that they are correctly executed without compromising security.
      • Privacy: Ola does not specifically prioritize privacy in the same way that zkVMs do. While it leverages ZKPs for scalability, its focus is on proving the correctness of -transactions and computations rather than ensuring that the data remains private.
      • Performance: Ola is designed to achieve high performance, particularly in terms of transaction throughput.
      • Integration: Ola is designed to be interoperable with various layer-1 blockchains. The platform supports a hybrid ZK-rollup architecture and is expected to include bridges for cross-chain -interoperability, enabling assets and data to move seamlessly between the layer-1 blockchain and the Ola rollup.
      • Conclusion: No, Ola is not a zkVM. While it leverages ZKPs (in the form of ZK-rollups) to ensure the validity of offchain computations, its primary focus is on scalability and performance +and scalability while deprioritizing privacy features, which can limit their use in certain privacy-sensitive applications.
        • +
      • Scalability and performance: zkVMs should offer scalable and efficient computation, leveraging advanced cryptographic techniques like zk-SNARKs, zk-STARKs, or recursive proofs. +A zkVM's performance must also be measured in terms of latency (time to generate and verify a proof) and throughput (number of computations processed within a certain time frame).
        • +
      • Verifiable computation: The zkVM should be able to prove the execution of arbitrary programs in a secure and verifiable manner. Verifiable computation ensures that zkVMs can be deployed +across a wide range of applications, from DeFi to private data-sharing platforms and more.
        • +
      +

      Why zkVMs matter

      +

      The rise of zkVMs is a crucial development for the future of blockchain and decentralized technologies. As more systems require the ability to scale while maintaining privacy and trust, +zkVMs provide a powerful solution. They offer the potential to reshape the way decentralized applications (dapps) handle sensitive information, enabling them to be both efficient and private.

      +

      It is essential to distinguish between projects that fully realize the potential of zkVMs and those that do not. In the remainder of this post, we evaluate several zkVM projects, analyzing +whether they satisfy the criteria for being classified as zkVMs based on our research.

      +

      Our methodology

      +

      We analyzed each project’s documentation, source code, and available benchmarks to determine whether they meet the definition of a zkVM. +Our criteria focus on the key capabilities of zkVMs—proof generation, privacy, scalability, and integration with existing systems.

      +

      ZkVM project analysis

      +

      1. [SP1]​

      +
        +
      • Overview: SP1 [1] is a developer-friendly zkVM designed to enable ZKP execution for LLVM-based languages like C, C++, Rust, and others. It supports a RISC-V-like instruction set architecture (ISA), +which makes it compatible with various programming languages compiled through LLVM.
        • +
      • Main focus: The main focus of SP1 is scalability, open-source contributions, and accessibility for developers. It prioritizes performance over privacy, +making it a good fit for environments where privacy isn't the primary concern.
        • +
      • Privacy: Not explicitly mentioned, making it less suitable for privacy-preserving applications.
        • +
      • Performance: SP1 has demonstrated up to 5.4x better performance than similar zkVMs like RISC0 for specific computations such as Fibonacci sequence generation.
        • +
      • Integration: SP1 is highly adaptable for rollups, light client verifiers, oracles, and even web2 projects like verifying the originality of images.
        • +
      • Conclusion: Yes, SP1 is a zkVM, but it does not prioritize zero-knowledge privacy, focusing more on scalability and performance.
        • +
      +

      2. [Nexus]​

      +
        +
      • Overview: Nexus [2] is a highly modular zkVM designed to process up to a trillion CPU cycles per second. It relies on RISC-V instructions for computation, making it extensible and scalable. +However, it currently lacks full ZKP capabilities due to its use of Spartan proofs.
        • +
      • Main focus: Nexus focuses on high performance and scalability, aiming to create an efficient execution environment for computationally intensive tasks.
        • +
      • Privacy: Although zero-knowledge privacy isn't the primary feature of Nexus, the project hints at potential privacy enhancements in the future.
        • +
      • Performance: Nexus has a high theoretical throughput, but it has yet to demonstrate benchmarks on zero-knowledge privacy.
        • +
      • Integration: Nexus is a good fit for high-performance environments that do not necessarily require full privacy.
        • +
      • Conclusion: Yes, Nexus qualifies as a zkVM in terms of scalability and proof generation, but it does not yet achieve full zero-knowledge privacy.
        • +
      +

      3. [RISC0]​

      +
        +
      • Overview: Risc0 [3] is a general-purpose zkVM with strong developer support. It allows for the execution of Rust and C code on a RISC-V virtual machine +and generates zk-SNARK and zk-STARK proofs for these computations.
        • +
      • Main focus: Risc0 is focused on ease of use for developers by abstracting away the complexities of circuit generation, making it accessible for a wide range of use cases.
        • +
      • Privacy: Full zero-knowledge privacy is supported via zk-SNARK and zk-STARK proofs, with Groth16 used for constant-size proof generation.
        • +
      • Performance: Risc0 offers strong benchmarks across different hardware setups, making it one of the most versatile zkVMs in terms of performance and scalability.
        • +
      • Integration: Risc0 integrates with several ecosystems, including Ethereum, and supports verifiable execution of Rust-based programs.
        • +
      • Conclusion: Yes, Risc0 qualifies as a zkVM, offering a balance of developer usability, scalability, and privacy.
        • +
      +

      4. [Powdr]​

      +
        +
      • Overview: Powdr [4] is a toolkit for creating custom zkVMs. It allows developers to select from various front-end and back-end components to create zkVMs tailored to specific needs.
        • +
      • Main focus: Powdr is focused on providing a modular architecture for zkVM creation. It enables flexibility by allowing the combination of different ZK-proof backends like Halo2 or Valida.
        • +
      • Privacy: Powdr itself does not generate ZKPs, but it facilitates the creation of zkVMs that do.
        • +
      • Performance: The performance depends on the components chosen by the developer, as Powdr itself is more of a framework.
        • +
      • Integration: Powdr is highly customizable and can integrate with existing zkVM frameworks to extend their capabilities.
        • +
      • Conclusion: No, Powdr is not a zkVM itself, but it is a powerful tool for building customized zkVMs with different privacy and performance needs.
        • +
      +

      5. [ZkMIPS]​

      +
        +
      • Overview: ZkMIPS [5] uses zk-STARKs to ensure privacy during computation, ensuring that private inputs are preserved while still proving correctness.
        • +
      • Performance: ZkMIPS is built for scalability, though explicit benchmarks are not widely published.
        • +
      • Integration: ZkMIPS can be integrated into systems that rely on MIPS architecture, making it versatile for legacy codebases that require privacy.
        • +
      • Conclusion: Yes, zkMIPS is a zkVM focused on scalability and privacy for MIPS-based architectures.
        • +
      +

      6. [Valida]​

      +
        +
      • Overview: Valida [6] is a performance-oriented zkVM that generates proofs for programs using a custom ISA designed to optimize zkVM implementation. +It uses Plonky3 for its proof system.
        • +
      • Main focus: Valida is centered around optimizing prover performance and extensibility, making it a valuable tool for generating proofs efficiently.
        • +
      • Privacy: While Valida is focused on performance, it does not prioritize zero-knowledge privacy as much as other zkVMs.
        • +
      • Performance: Valida has benchmarks indicating its performance advantages in proving computations quickly, particularly through parallel processing.
        • +
      • Integration: Valida is specialized and may not integrate as seamlessly into general-purpose systems, as it is optimized for performance over broad applicability.
        • +
      • Conclusion: Yes, Valida qualifies as a zkVM based on proof generation, but its lack of focus on privacy makes it less suitable for privacy-first use cases.
        • +
      +

      7. [Jolt]​

      +
        +
      • Overview: Jolt [7] is a zkVM built to optimize prover performance using a modified Hyrax polynomial commitment system. It relies on RISC-V instructions for computation +but falls short of full zero-knowledge capabilities.
        • +
      • Main focus: Jolt's main goal is to optimize the speed of proving program execution, making it suitable for high-performance applications where privacy isn't the primary concern.
        • +
      • Privacy: Jolt does not fully achieve zero-knowledge privacy due to the choice of polynomial commitment schemes.
        • +
      • Performance: Jolt offers strong performance, with benchmarks highlighting its ability to process proofs efficiently.
        • +
      • Integration: Jolt can be integrated with systems that prioritize speed over privacy, particularly where rapid proof generation is essential.
        • +
      • Conclusion: Yes, Jolt qualifies as a zkVM based on proof generation, though it does not provide full zero-knowledge privacy.
        • +
      +

      8. [ZkWASM]​

      +
        +
      • Overview: ZkWASM [8] is a zkVM designed to execute WebAssembly (WASM) code in a privacy-preserving and scalable manner. It uses zk-SNARKs to prove the correctness of WASM +program execution while ensuring privacy.
        • +
      • Main focus: ZkWASM focuses on scalability and privacy for WebAssembly, making it ideal for dapps that require verifiable computation without compromising privacy.
        • +
      • Privacy: Full zero-knowledge privacy is provided through zk-SNARKs, ensuring that the execution of WASM programs remains confidential.
        • +
      • Performance: ZkWASM is optimized for running WASM programs efficiently, with offchain computation and onchain verification to enhance performance.
        • +
      • Integration: ZkWASM is ideal for dapps, particularly those that use WebAssembly and require verifiable execution.
        • +
      • Conclusion: Yes, zkWASM qualifies as a zkVM, providing strong privacy, scalability, and verifiable execution for WebAssembly code.
        • +
      +

      9. [Aleo]​

      +
        +
      • Overview: Aleo's [9] snarkVM converts code into Aleo instructions, which are then compiled into bytecode executable on its zkVM. Aleo emphasizes building private, scalable dapps.
        • +
      • Main focus: Aleo prioritizes privacy and scalability for dapps, providing a robust framework for developers building private dapps.
        • +
      • Privacy: Aleo offers full privacy through zk-SNARK proofs, making it suitable for building fully private applications.
        • +
      • Performance: Aleo focuses on scalability through efficient proof systems, though detailed performance benchmarks are not widely available.
        • +
      • Integration: Aleo is built for privacy-first dapps and integrates with other zkVM-based systems.
        • +
      • Conclusion: Yes, Aleo qualifies as a zkVM, offering a comprehensive solution for private and scalable dapps.
        • +
      +

      10. [Ola]​

      +
        +
      • Overview: Ola [10] is a ZK-friendly, high-performance layer-2 (L2) rollup platform that is still under development. It is designed to execute computations offchain while generating +validity proofs for these computations, ensuring that they are correctly executed without compromising security.
        • +
      • Privacy: Ola does not specifically prioritize privacy in the same way that zkVMs do. While it leverages ZKPs for scalability, its focus is on proving the correctness of +transactions and computations rather than ensuring that the data remains private.
        • +
      • Performance: Ola is designed to achieve high performance, particularly in terms of transaction throughput.
        • +
      • Integration: Ola is designed to be interoperable with various layer-1 blockchains. The platform supports a hybrid ZK-rollup architecture and is expected to include bridges for cross-chain +interoperability, enabling assets and data to move seamlessly between the layer-1 blockchain and the Ola rollup.
        • +
      • Conclusion: No, Ola is not a zkVM. While it leverages ZKPs (in the form of ZK-rollups) to ensure the validity of offchain computations, its primary focus is on scalability and performance rather than privacy or verifiable execution of arbitrary programs. Ola is more accurately described as a ZK-rollup platform aimed at improving transaction throughput and reducing transaction costs on -layer-1 blockchains.

      11. [Miden]​

      • Overview: Miden zkVM [11] is a zk-STARK-based virtual machine that converts code into Miden VM instructions and proves the execution of these instructions with zero-knowledge privacy.
      • Main focus: Miden focuses on scalability and privacy for ZK-rollups, offering efficient proof generation for dapps.
      • Privacy: Miden ensures privacy for transactions and programs via zk-STARK proofs, making it suitable for private dapps.
      • Performance: Miden is optimized for scalability, with benchmarks showing its ability to handle up to 1,000 transactions per second (TPS).
      • Integration: Miden integrates well with ZK-rollup solutions, making it ideal for L2 scaling solutions on blockchains like Ethereum.
      • Conclusion: Yes, Miden qualifies as a zkVM, providing strong privacy and scalability for dapps and ZK-rollups.

      12. [ZkOS]​

      • Overview: ZkOS [12] is a verifiable operating system focused on running zkApps in a decentralized manner. It is built on the RISC-V architecture and aims to create -a world computer where all untrusted executions can be verified.
      • Main focus: ZkOS is primarily designed to offer a proof-of-concept operating system where all executions can be verified in a trustless manner. -However, its focus is more on the infrastructure for verifiable applications rather than being a traditional zkVM.
      • Privacy: ZkOS does not focus on privacy guarantees such as those found in zkVMs that generate ZKPs.
      • Performance: ZkOS focuses on the efficient execution of dapps, but performance benchmarks specific to ZKP generation are not provided.
      • Integration: ZkOS supports the execution of zkApps, but it is more of a verifiable operating system rather than a zkVM, making it distinct in its functionality.
      • Conclusion: No, zkOS is not a zkVM. It is a verifiable operating system focused on the infrastructure to support zkApps but does not directly generate ZKPs or focus on privacy preservation.

      13. [Triton]​

      • Overview: Triton [13] is a domain-specific language (DSL) and compiler designed primarily for high-performance GPU kernels, particularly those used in deep learning applications.
      • Main focus: The primary goal of Triton is to optimize computation for machine learning and GPU workloads. It is focused on enhancing performance and efficiency in processing data -rather than on ZKPs or verifiable computation.
      • Privacy: Triton does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on high-performance computation rather than cryptographic verifiability.
      • Performance: Triton is highly optimized for GPU execution, offering significant improvements in performance for computationally intensive tasks such as those found in deep learning.
      • Integration: Triton is integrated with GPU-based computation environments and is highly specialized for optimizing low-level operations on hardware rather -than being a general-purpose virtual machine.
      • Conclusion: No, Triton is not a zkVM. It is a specialized tool for optimizing GPU workloads, focusing on performance rather than privacy or ZKPs.

      14. [Cairo]​

      • Overview: Cairo zkVM [14] uses a custom language that compiles to an optimized STARK-based proof system, ensuring verifiable computation. It is primarily used in systems like Starknet.
      • Main focus: Cairo focuses on scalability and performance, using zk-STARK proofs to ensure the verifiable and secure execution of programs.
      • Privacy: Cairo provides privacy through zk-STARKs, but it focuses more on scalability and performance than privacy-first use cases.
      • Performance: Cairo is highly optimized for performance, making it well-suited for scalable applications on Starknet.
      • Integration: Cairo integrates deeply with systems like Starknet, supporting verifiable computation in a highly scalable and efficient manner.
      • Conclusion: Yes, Cairo qualifies as a zkVM, focusing on performance and verifiable execution while being ZK-friendly.

      15. [SnarkOS]​

      • Overview: SnarkOS [15] is a decentralized operating system designed to power Aleo's network, enabling secure and private dapps. -It manages transactions and consensus, making it a critical infrastructure component for Aleo's zkVM-based ecosystem.
      • Main focus: SnarkOS primarily focuses on securing Aleo's network through consensus mechanisms and privacy-preserving transactions rather than acting as a -zkVM that directly proves program execution.
      • Privacy: SnarkOS supports zero-knowledge privacy through its integration with Aleo's zkVM, but the operating system itself does not generate ZKPs for arbitrary computations.
      • Performance: SnarkOS is optimized for managing dapps on the Aleo network and handling private transactions, but its focus is more on infrastructure -and consensus than on proof generation.
      • Integration: SnarkOS integrates seamlessly with Aleo's zkVM to support private dapps and transactions, but its primary role is as a consensus layer.
      • Conclusion: No, SnarkOS is not a zkVM. It serves as an operating system for Aleo's decentralized network, focusing on privacy and consensus rather than on generating ZKPs for computations.

      16. [Lurk]​

      • Overview: Lurk [16] is a Turing-complete programming language designed for recursive zk-SNARKs. It focuses on enabling developers to build complex, -recursive ZKPs efficiently through a custom language tailored for verifiable computation.
      • Main focus: Lurk is centered around recursive proof generation rather than serving as a traditional virtual machine. Its purpose is to facilitate the creation of complex zk-SNARK-based proofs, -making it a specialized tool for cryptographic proofs rather than general-purpose computation.
      • Privacy: Lurk is built for generating zk-SNARKs, which inherently provide privacy. However, Lurk itself is a language and not a zkVM that executes arbitrary programs and generates ZKPs for them.
      • Performance: Lurk is optimized for recursive zk-SNARK generation, but specific performance metrics are tied to its proof-generation capabilities rather than traditional execution environments.
      • Integration: Lurk is specialized for zk-SNARKs and may not easily integrate with other general-purpose systems, as it focuses on specific cryptographic tasks.
      • Conclusion: No, Lurk is not a zkVM. It is a programming language designed for recursive zk-SNARKs and focuses on proof generation rather than program execution in a virtual machine environment.

      17. [Piecrust]​

      • Overview: Piecrust [17] is a WASM-based zkVM designed to run on the Dusk Network. It supports concurrent execution and focuses on providing privacy and scalability for smart contracts.
      • Main focus: Piecrust is designed to provide private and efficient execution of smart contracts through the use of ZKPs.
      • Privacy: Piecrust supports ZK-friendly computations and enhances privacy through cryptographic primitives such as Merkle trees.
      • Performance: Piecrust is designed to be scalable and concurrent, allowing multiple sessions to run simultaneously, which improves overall performance.
      • Integration: Piecrust integrates with the Dusk Network and supports private smart contracts, making it ideal for dapps.
      • Conclusion: Yes, Piecrust qualifies as a zkVM, offering scalability, privacy, and support for succinct proof generation.

      18. [Ceno]​

      • Overview: Ceno [18] is a zkVM that provides a theoretical framework for reducing proving time by grouping common portions of code together. It uses recursive proofs to enhance prover efficiency.
      • Main focus: Ceno aims to optimize prover performance through recursive proofs, making it a powerful tool for handling complex computations efficiently.
      • Privacy: Ceno supports zero-knowledge privacy through recursive proofs and is designed to handle large-scale computations securely.
      • Performance: Ceno's recursive proof framework ensures that it can efficiently prove the execution of programs, reducing the time required for proof generation.
      • Integration: Ceno can be integrated into systems that require high efficiency and privacy, particularly those handling complex, repeated computations.
      • Conclusion: Yes, Ceno qualifies as a zkVM, providing efficient and private computation through the use of recursive proofs.

      19. [Stellar]​

      • Overview: Stellar [19] is a decentralized protocol designed to facilitate cross-border transactions between digital and fiat currencies.
      • Main focus: Stellar's primary goal is to improve financial transactions by enabling decentralized, low-cost currency transfers. It does not aim to provide ZKPs or run verifiable computations -like a zkVM.
      • Privacy: Stellar focuses on confidentiality and security for financial transactions, but it does not employ ZKPs in the way zkVMs do for verifying computation without revealing data.
      • Performance: Stellar prioritizes the performance of financial transactions, ensuring low latency and high throughput across its decentralized network. -However, this performance focus is specific to transactions rather than general-purpose program execution.
      • Integration: Stellar is designed for integration with financial systems, enabling currency conversions and transfers, but it is not built for executing smart contracts or verifiable computations.
      • Conclusion: No, Stellar is not a zkVM. It is a decentralized financial protocol focused on facilitating cross-border payments rather than verifiable or privacy-preserving computation.

      20. [NovaNet]​

      • Overview: NovaNet [20] is an open peer-to-peer network that aims to build upon concepts of non-uniform incremental verifiable computation.
      • Main focus: NovaNet's focus is on peer-to-peer networking and decentralized computing rather than on proving the execution of programs in a zero-knowledge manner.
      • Privacy: NovaNet does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on decentralized networking and computation.
      • Performance: NovaNet prioritizes efficient decentralized computation but does not focus on privacy or performance benchmarks related to ZKPs.
      • Integration: NovaNet is built for decentralized networks but is not designed to integrate with systems requiring verifiable computation or ZKP generation.
      • Conclusion: No, NovaNet is not a zkVM. It is a decentralized peer-to-peer network focused on distributed computing rather than zero-knowledge computation.

      21. [ZkLLVM]​

      • Overview: ZkLLVM [21] is a compiler that transforms C++ or Rust code into circuits for use in zk-SNARK or zk-STARK systems. Its primary purpose is to bridge high-level programming -languages with ZKP systems by compiling code into arithmetic circuits that can be used to generate and verify proofs.
      • Main focus: ZkLLVM focuses on making ZKPs accessible to developers by enabling them to write code in familiar languages (C++, Rust) and then compile that code into ZK circuits.
      • Privacy: ZkLLVM enables the generation of ZKPs by compiling high-level code into ZK-compatible circuits. It plays a crucial role in privacy-preserving applications but does not act -as a zkVM itself.
      • Performance: ZkLLVM allows for the performance of ZKPs to be closely tied to the complexity of the compiled circuits. The performance depends on the underlying -zk-SNARK or zk-STARK system used.
      • Integration: ZkLLVM integrates with zk-SNARK and zk-STARK proof systems, making it useful for a variety of privacy-focused applications, but it does not serve as a zkVM -for general-purpose computation.
      • Conclusion: No, zkLLVM is not a zkVM. It is a compiler that transforms high-level code into ZK circuits, enabling ZKPs but not acting as a virtual machine for executing and proving programs.

      22. [ZkMove]​

      • Overview: ZkMove [22] is a zkVM designed to execute smart contracts written in the Move language. It utilizes ZKPs to ensure that the execution of these contracts remains verifiable and secure.
      • Main focus: ZkMove focuses on privacy and verifiable execution for Move-based smart contracts, providing a framework for ZK-friendly computation.
      • Privacy: ZkMove ensures that smart contract execution remains private through ZKPs, making it suitable for privacy-preserving applications.
      • Performance: ZkMove is optimized for verifiable execution, ensuring that contracts can be proven correct while preserving privacy.
      • Integration: ZkMove integrates well with systems that use the Move language, particularly in environments that require private smart contract execution.
      • Conclusion: Yes, zkMove qualifies as a zkVM, offering ZK-friendly execution and privacy for smart contracts written in the Move language.

      23. [O1VM]​

      • Overview: O1VM [23] is a general-purpose zkVM developed by o1Labs. It is designed to prove the execution of MIPS programs efficiently through a combination of zk-SNARKs -and specialized techniques like folding schemes and RAMLookups.
      • Main focus: O1VM focuses on scalability and verifiable computation for MIPS-based programs, making it a strong contender for executing and proving complex programs efficiently.
      • Privacy: O1VM ensures privacy through zk-SNARK proofs, keeping the details of the computation private while proving its correctness.
      • Performance: O1VM is optimized for handling long execution traces and complex computations, making it highly scalable.
      • Integration: O1VM integrates well with MIPS-based architectures and systems that require privacy-preserving computation.
      • Conclusion: Yes, o1VM qualifies as a zkVM, providing privacy, scalability, and strong proof generation for MIPS programs.

      Summary of findings

      Project nameZkVM statusZero knowledgeReasoning/comments
      SP1YesNoProves execution of LLVM-based programs but lacks privacy features.
      NexusYesNoStrong proof generation but lacks zero-knowledge privacy due to Spartan.
      Risc0YesYesSupports full ZKP generation for Rust programs.
      PowdrNoYesToolkit for creating custom zkVMs, not a zkVM itself.
      ZkMIPSYesYesSupports MIPS-like architecture with full zero-knowledge and proof generation.
      ValidaYesNoPerformance-focused zkVM, lacks privacy guarantees.
      JoltYesNoPerformance-focused zkVM, does not achieve zero-knowledge privacy.
      ZkWASMYesYesFull zero-knowledge and verifiable execution of WebAssembly code.
      AleoYesYesFully private and scalable dapps.
      OlaNoNoPrimarily a ZK-rollup platform, not a zkVM, focusing on scalability and performance rather than privacy.
      MidenYesYesZk-STARK-based zkVM with strong privacy and scalability.
      ZkOSNoNoVerifiable operating system focused on zkApps, not a zkVM.
      TritonNoNoOptimizes GPU workloads but not designed for ZKPs.
      CairoYesZK-friendlyCustom Rust-based language with zk-STARK proof generation.
      SnarkOSNoYesDecentralized OS for Aleo's network, focuses on consensus rather than verifiable computation.
      LurkNoNoProgramming language for recursive zk-SNARKs, not a zkVM.
      PiecrustYesZK-friendlyZkVM with recursive SNARK capabilities, focused on succinct proof generation.
      CenoYesYesTheoretical zkVM improving prover efficiency through recursive proofs.
      StellarNoNoFocuses on cross-border transactions, not ZK-proof generation or verifiable computation.
      NovaNetNoNoPeer-to-peer network focused on distributed computing, not zero-knowledge computation.
      ZkLLVMNoYes, in some casesCompiler for generating ZK-circuits, not a zkVM.
      ZkMoveYesZK-friendlyZkVM supporting Move language with ZKP execution.
      O1VMYesYesMIPS-based zkVM with strong privacy, scalability, and proof generation.

      Insights and conclusions

      Our analysis reveals that many of the projects labeled as zkVMs do meet the core criteria for zkVMs, offering verifiable computation and proof generation +layer-1 blockchains.

      • +
    +

    11. [Miden]​

    +
      +
    • Overview: Miden zkVM [11] is a zk-STARK-based virtual machine that converts code into Miden VM instructions and proves the execution of these instructions with zero-knowledge privacy.
      • +
    • Main focus: Miden focuses on scalability and privacy for ZK-rollups, offering efficient proof generation for dapps.
      • +
    • Privacy: Miden ensures privacy for transactions and programs via zk-STARK proofs, making it suitable for private dapps.
      • +
    • Performance: Miden is optimized for scalability, with benchmarks showing its ability to handle up to 1,000 transactions per second (TPS).
      • +
    • Integration: Miden integrates well with ZK-rollup solutions, making it ideal for L2 scaling solutions on blockchains like Ethereum.
      • +
    • Conclusion: Yes, Miden qualifies as a zkVM, providing strong privacy and scalability for dapps and ZK-rollups.
      • +
    +

    12. [ZkOS]​

    +
      +
    • Overview: ZkOS [12] is a verifiable operating system focused on running zkApps in a decentralized manner. It is built on the RISC-V architecture and aims to create +a world computer where all untrusted executions can be verified.
      • +
    • Main focus: ZkOS is primarily designed to offer a proof-of-concept operating system where all executions can be verified in a trustless manner. +However, its focus is more on the infrastructure for verifiable applications rather than being a traditional zkVM.
      • +
    • Privacy: ZkOS does not focus on privacy guarantees such as those found in zkVMs that generate ZKPs.
      • +
    • Performance: ZkOS focuses on the efficient execution of dapps, but performance benchmarks specific to ZKP generation are not provided.
      • +
    • Integration: ZkOS supports the execution of zkApps, but it is more of a verifiable operating system rather than a zkVM, making it distinct in its functionality.
      • +
    • Conclusion: No, zkOS is not a zkVM. It is a verifiable operating system focused on the infrastructure to support zkApps but does not directly generate ZKPs or focus on privacy preservation.
      • +
    +

    13. [Triton]​

    +
      +
    • Overview: Triton [13] is a domain-specific language (DSL) and compiler designed primarily for high-performance GPU kernels, particularly those used in deep learning applications.
      • +
    • Main focus: The primary goal of Triton is to optimize computation for machine learning and GPU workloads. It is focused on enhancing performance and efficiency in processing data +rather than on ZKPs or verifiable computation.
      • +
    • Privacy: Triton does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on high-performance computation rather than cryptographic verifiability.
      • +
    • Performance: Triton is highly optimized for GPU execution, offering significant improvements in performance for computationally intensive tasks such as those found in deep learning.
      • +
    • Integration: Triton is integrated with GPU-based computation environments and is highly specialized for optimizing low-level operations on hardware rather +than being a general-purpose virtual machine.
      • +
    • Conclusion: No, Triton is not a zkVM. It is a specialized tool for optimizing GPU workloads, focusing on performance rather than privacy or ZKPs.
      • +
    +

    14. [Cairo]​

    +
      +
    • Overview: Cairo zkVM [14] uses a custom language that compiles to an optimized STARK-based proof system, ensuring verifiable computation. It is primarily used in systems like Starknet.
      • +
    • Main focus: Cairo focuses on scalability and performance, using zk-STARK proofs to ensure the verifiable and secure execution of programs.
      • +
    • Privacy: Cairo provides privacy through zk-STARKs, but it focuses more on scalability and performance than privacy-first use cases.
      • +
    • Performance: Cairo is highly optimized for performance, making it well-suited for scalable applications on Starknet.
      • +
    • Integration: Cairo integrates deeply with systems like Starknet, supporting verifiable computation in a highly scalable and efficient manner.
      • +
    • Conclusion: Yes, Cairo qualifies as a zkVM, focusing on performance and verifiable execution while being ZK-friendly.
      • +
    +

    15. [SnarkOS]​

    +
      +
    • Overview: SnarkOS [15] is a decentralized operating system designed to power Aleo's network, enabling secure and private dapps. +It manages transactions and consensus, making it a critical infrastructure component for Aleo's zkVM-based ecosystem.
      • +
    • Main focus: SnarkOS primarily focuses on securing Aleo's network through consensus mechanisms and privacy-preserving transactions rather than acting as a +zkVM that directly proves program execution.
      • +
    • Privacy: SnarkOS supports zero-knowledge privacy through its integration with Aleo's zkVM, but the operating system itself does not generate ZKPs for arbitrary computations.
      • +
    • Performance: SnarkOS is optimized for managing dapps on the Aleo network and handling private transactions, but its focus is more on infrastructure +and consensus than on proof generation.
      • +
    • Integration: SnarkOS integrates seamlessly with Aleo's zkVM to support private dapps and transactions, but its primary role is as a consensus layer.
      • +
    • Conclusion: No, SnarkOS is not a zkVM. It serves as an operating system for Aleo's decentralized network, focusing on privacy and consensus rather than on generating ZKPs for computations.
      • +
    +

    16. [Lurk]​

    +
      +
    • Overview: Lurk [16] is a Turing-complete programming language designed for recursive zk-SNARKs. It focuses on enabling developers to build complex, +recursive ZKPs efficiently through a custom language tailored for verifiable computation.
      • +
    • Main focus: Lurk is centered around recursive proof generation rather than serving as a traditional virtual machine. Its purpose is to facilitate the creation of complex zk-SNARK-based proofs, +making it a specialized tool for cryptographic proofs rather than general-purpose computation.
      • +
    • Privacy: Lurk is built for generating zk-SNARKs, which inherently provide privacy. However, Lurk itself is a language and not a zkVM that executes arbitrary programs and generates ZKPs for them.
      • +
    • Performance: Lurk is optimized for recursive zk-SNARK generation, but specific performance metrics are tied to its proof-generation capabilities rather than traditional execution environments.
      • +
    • Integration: Lurk is specialized for zk-SNARKs and may not easily integrate with other general-purpose systems, as it focuses on specific cryptographic tasks.
      • +
    • Conclusion: No, Lurk is not a zkVM. It is a programming language designed for recursive zk-SNARKs and focuses on proof generation rather than program execution in a virtual machine environment.
      • +
    +

    17. [Piecrust]​

    +
      +
    • Overview: Piecrust [17] is a WASM-based zkVM designed to run on the Dusk Network. It supports concurrent execution and focuses on providing privacy and scalability for smart contracts.
      • +
    • Main focus: Piecrust is designed to provide private and efficient execution of smart contracts through the use of ZKPs.
      • +
    • Privacy: Piecrust supports ZK-friendly computations and enhances privacy through cryptographic primitives such as Merkle trees.
      • +
    • Performance: Piecrust is designed to be scalable and concurrent, allowing multiple sessions to run simultaneously, which improves overall performance.
      • +
    • Integration: Piecrust integrates with the Dusk Network and supports private smart contracts, making it ideal for dapps.
      • +
    • Conclusion: Yes, Piecrust qualifies as a zkVM, offering scalability, privacy, and support for succinct proof generation.
      • +
    +

    18. [Ceno]​

    +
      +
    • Overview: Ceno [18] is a zkVM that provides a theoretical framework for reducing proving time by grouping common portions of code together. It uses recursive proofs to enhance prover efficiency.
      • +
    • Main focus: Ceno aims to optimize prover performance through recursive proofs, making it a powerful tool for handling complex computations efficiently.
      • +
    • Privacy: Ceno supports zero-knowledge privacy through recursive proofs and is designed to handle large-scale computations securely.
      • +
    • Performance: Ceno's recursive proof framework ensures that it can efficiently prove the execution of programs, reducing the time required for proof generation.
      • +
    • Integration: Ceno can be integrated into systems that require high efficiency and privacy, particularly those handling complex, repeated computations.
      • +
    • Conclusion: Yes, Ceno qualifies as a zkVM, providing efficient and private computation through the use of recursive proofs.
      • +
    +

    19. [Stellar]​

    +
      +
    • Overview: Stellar [19] is a decentralized protocol designed to facilitate cross-border transactions between digital and fiat currencies.
      • +
    • Main focus: Stellar's primary goal is to improve financial transactions by enabling decentralized, low-cost currency transfers. It does not aim to provide ZKPs or run verifiable computations +like a zkVM.
      • +
    • Privacy: Stellar focuses on confidentiality and security for financial transactions, but it does not employ ZKPs in the way zkVMs do for verifying computation without revealing data.
      • +
    • Performance: Stellar prioritizes the performance of financial transactions, ensuring low latency and high throughput across its decentralized network. +However, this performance focus is specific to transactions rather than general-purpose program execution.
      • +
    • Integration: Stellar is designed for integration with financial systems, enabling currency conversions and transfers, but it is not built for executing smart contracts or verifiable computations.
      • +
    • Conclusion: No, Stellar is not a zkVM. It is a decentralized financial protocol focused on facilitating cross-border payments rather than verifiable or privacy-preserving computation.
      • +
    +

    20. [NovaNet]​

    +
      +
    • Overview: NovaNet [20] is an open peer-to-peer network that aims to build upon concepts of non-uniform incremental verifiable computation.
      • +
    • Main focus: NovaNet's focus is on peer-to-peer networking and decentralized computing rather than on proving the execution of programs in a zero-knowledge manner.
      • +
    • Privacy: NovaNet does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on decentralized networking and computation.
      • +
    • Performance: NovaNet prioritizes efficient decentralized computation but does not focus on privacy or performance benchmarks related to ZKPs.
      • +
    • Integration: NovaNet is built for decentralized networks but is not designed to integrate with systems requiring verifiable computation or ZKP generation.
      • +
    • Conclusion: No, NovaNet is not a zkVM. It is a decentralized peer-to-peer network focused on distributed computing rather than zero-knowledge computation.
      • +
    +

    21. [ZkLLVM]​

    +
      +
    • Overview: ZkLLVM [21] is a compiler that transforms C++ or Rust code into circuits for use in zk-SNARK or zk-STARK systems. Its primary purpose is to bridge high-level programming +languages with ZKP systems by compiling code into arithmetic circuits that can be used to generate and verify proofs.
      • +
    • Main focus: ZkLLVM focuses on making ZKPs accessible to developers by enabling them to write code in familiar languages (C++, Rust) and then compile that code into ZK circuits.
      • +
    • Privacy: ZkLLVM enables the generation of ZKPs by compiling high-level code into ZK-compatible circuits. It plays a crucial role in privacy-preserving applications but does not act +as a zkVM itself.
      • +
    • Performance: ZkLLVM allows for the performance of ZKPs to be closely tied to the complexity of the compiled circuits. The performance depends on the underlying +zk-SNARK or zk-STARK system used.
      • +
    • Integration: ZkLLVM integrates with zk-SNARK and zk-STARK proof systems, making it useful for a variety of privacy-focused applications, but it does not serve as a zkVM +for general-purpose computation.
      • +
    • Conclusion: No, zkLLVM is not a zkVM. It is a compiler that transforms high-level code into ZK circuits, enabling ZKPs but not acting as a virtual machine for executing and proving programs.
      • +
    +

    22. [ZkMove]​

    +
      +
    • Overview: ZkMove [22] is a zkVM designed to execute smart contracts written in the Move language. It utilizes ZKPs to ensure that the execution of these contracts remains verifiable and secure.
      • +
    • Main focus: ZkMove focuses on privacy and verifiable execution for Move-based smart contracts, providing a framework for ZK-friendly computation.
      • +
    • Privacy: ZkMove ensures that smart contract execution remains private through ZKPs, making it suitable for privacy-preserving applications.
      • +
    • Performance: ZkMove is optimized for verifiable execution, ensuring that contracts can be proven correct while preserving privacy.
      • +
    • Integration: ZkMove integrates well with systems that use the Move language, particularly in environments that require private smart contract execution.
      • +
    • Conclusion: Yes, zkMove qualifies as a zkVM, offering ZK-friendly execution and privacy for smart contracts written in the Move language.
      • +
    +

    23. [O1VM]​

    +
      +
    • Overview: O1VM [23] is a general-purpose zkVM developed by o1Labs. It is designed to prove the execution of MIPS programs efficiently through a combination of zk-SNARKs +and specialized techniques like folding schemes and RAMLookups.
      • +
    • Main focus: O1VM focuses on scalability and verifiable computation for MIPS-based programs, making it a strong contender for executing and proving complex programs efficiently.
      • +
    • Privacy: O1VM ensures privacy through zk-SNARK proofs, keeping the details of the computation private while proving its correctness.
      • +
    • Performance: O1VM is optimized for handling long execution traces and complex computations, making it highly scalable.
      • +
    • Integration: O1VM integrates well with MIPS-based architectures and systems that require privacy-preserving computation.
      • +
    • Conclusion: Yes, o1VM qualifies as a zkVM, providing privacy, scalability, and strong proof generation for MIPS programs.
      • +
    +

    Summary of findings

    +
    Project nameZkVM statusZero knowledgeReasoning/comments
    SP1YesNoProves execution of LLVM-based programs but lacks privacy features.
    NexusYesNoStrong proof generation but lacks zero-knowledge privacy due to Spartan.
    Risc0YesYesSupports full ZKP generation for Rust programs.
    PowdrNoYesToolkit for creating custom zkVMs, not a zkVM itself.
    ZkMIPSYesYesSupports MIPS-like architecture with full zero-knowledge and proof generation.
    ValidaYesNoPerformance-focused zkVM, lacks privacy guarantees.
    JoltYesNoPerformance-focused zkVM, does not achieve zero-knowledge privacy.
    ZkWASMYesYesFull zero-knowledge and verifiable execution of WebAssembly code.
    AleoYesYesFully private and scalable dapps.
    OlaNoNoPrimarily a ZK-rollup platform, not a zkVM, focusing on scalability and performance rather than privacy.
    MidenYesYesZk-STARK-based zkVM with strong privacy and scalability.
    ZkOSNoNoVerifiable operating system focused on zkApps, not a zkVM.
    TritonNoNoOptimizes GPU workloads but not designed for ZKPs.
    CairoYesZK-friendlyCustom Rust-based language with zk-STARK proof generation.
    SnarkOSNoYesDecentralized OS for Aleo's network, focuses on consensus rather than verifiable computation.
    LurkNoNoProgramming language for recursive zk-SNARKs, not a zkVM.
    PiecrustYesZK-friendlyZkVM with recursive SNARK capabilities, focused on succinct proof generation.
    CenoYesYesTheoretical zkVM improving prover efficiency through recursive proofs.
    StellarNoNoFocuses on cross-border transactions, not ZK-proof generation or verifiable computation.
    NovaNetNoNoPeer-to-peer network focused on distributed computing, not zero-knowledge computation.
    ZkLLVMNoYes, in some casesCompiler for generating ZK-circuits, not a zkVM.
    ZkMoveYesZK-friendlyZkVM supporting Move language with ZKP execution.
    O1VMYesYesMIPS-based zkVM with strong privacy, scalability, and proof generation.
    +

    Insights and conclusions

    +

    Our analysis reveals that many of the projects labeled as zkVMs do meet the core criteria for zkVMs, offering verifiable computation and proof generation as foundational features. However, a number of these projects fall short of delivering full zero-knowledge privacy. Projects like Risc0, Aleo, and Miden stand out as leading zkVM frameworks -that balance proof generation, privacy, and scalability, offering strong platforms for developers seeking to build privacy-preserving applications.

    Conversely, projects like SP1 and Nexus excel in generating verifiable proofs but currently lack comprehensive zero-knowledge privacy mechanisms. These platforms are excellent for -scenarios where proof generation and scalability are paramount, but privacy is not a primary concern.

    As zkVM technology continues to evolve, we expect to see more projects integrating enhanced privacy-preserving mechanisms while simultaneously improving performance and scalability. +that balance proof generation, privacy, and scalability, offering strong platforms for developers seeking to build privacy-preserving applications.

    +

    Conversely, projects like SP1 and Nexus excel in generating verifiable proofs but currently lack comprehensive zero-knowledge privacy mechanisms. These platforms are excellent for +scenarios where proof generation and scalability are paramount, but privacy is not a primary concern.

    +

    As zkVM technology continues to evolve, we expect to see more projects integrating enhanced privacy-preserving mechanisms while simultaneously improving performance and scalability. This ongoing development will likely broaden the application of zkVMs across the blockchain ecosystem, particularly in privacy-sensitive sectors such as finance, data security, -and decentralized applications.

    What are your thoughts on our zkVM analysis? Do you agree with our findings, or do you know of other zkVM projects that should be on our radar? We would love to hear your insights, questions, -or suggestions! Feel free to join the discussion on our forum.

    References

    [1] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    [2] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    [3] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    [4] Powdr. Retrieved from https://docs.powdr.org/

    [5] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    [6] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    [7] Building Jolt: A fast, easy-to-use zkVM. Retrieved from https://a16zcrypto.com/posts/article/building-jolt/

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    [9] Aleo. Retrieved from https://aleo.org/blog/

    [10] OlaVM Whitepaper V2. Retrieved from https://github.com/Sin7Y/olavm-whitepaper-v2/tree/master

    [11] Polygon Miden VM. Retrieved from https://0xpolygonmiden.github.io/miden-vm/intro/main.html

    [12] The Adventures of OS: Making a RISC-V Operating System using Rust. Retrieved from https://osblog.stephenmarz.com/index.html

    [13] Triton VM. Retrieved from https://triton-vm.org/spec/

    [14] How does the original Cairo VM work?. Retrieved from https://github.com/lambdaclass/cairo-vm/blob/main/docs/python_vm/README.md

    [15] Aleo completes security audits of snarkOS & snarkVM. Retrieved from https://aleo.org/post/aleo-completes-security-audits-of-snarkos-and-snarkvm/

    [16] Lurk zkVM. Retrieved from https://github.com/lurk-lab

    [17] Piecrust VM. Retrieved from https://docs.rs/piecrust/latest/piecrust/

    [18] Ceno: Non-uniform, Segment and Parallel Zero-knowledge Virtual Machine. Retrieved from https://eprint.iacr.org/2024/387

    [19] ZkVM: a new design for fast, confidential smart contracts. Retrieved from https://stellar.org/blog/developers/zkvm-a-new-design-for-fast-confidential-smart-contracts

    [20] Novanet. Retrieved from https://www.novanet.xyz/blog

    [21] ZKLLVM. Retrieved from https://github.com/NilFoundation/zkLLVM

    [22] zkMove 0.2.0 - Achieving Full Bytecode Compatibility with Move. Retrieved from https://www.zkmove.net/2023-06-20-zkMove-0.2.0-Achieving-Full-Bytecode-Compatibility-with-Move/

    [23] O1VM. Retrieved from https://github.com/o1-labs/proof-systems/tree/master/o1vm

    ]]> +and decentralized applications.

    +

    What are your thoughts on our zkVM analysis? Do you agree with our findings, or do you know of other zkVM projects that should be on our radar? We would love to hear your insights, questions, +or suggestions! Feel free to join the discussion on our forum.

    +

    References

    +

    [1] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    +

    [2] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    +

    [3] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    +

    [4] Powdr. Retrieved from https://docs.powdr.org/

    +

    [5] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    +

    [6] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    +

    [7] Building Jolt: A fast, easy-to-use zkVM. Retrieved from https://a16zcrypto.com/posts/article/building-jolt/

    +

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    +

    [9] Aleo. Retrieved from https://aleo.org/blog/

    +

    [10] OlaVM Whitepaper V2. Retrieved from https://github.com/Sin7Y/olavm-whitepaper-v2/tree/master

    +

    [11] Polygon Miden VM. Retrieved from https://0xpolygonmiden.github.io/miden-vm/intro/main.html

    +

    [12] The Adventures of OS: Making a RISC-V Operating System using Rust. Retrieved from https://osblog.stephenmarz.com/index.html

    +

    [13] Triton VM. Retrieved from https://triton-vm.org/spec/

    +

    [14] How does the original Cairo VM work?. Retrieved from https://github.com/lambdaclass/cairo-vm/blob/main/docs/python_vm/README.md

    +

    [15] Aleo completes security audits of snarkOS & snarkVM. Retrieved from https://aleo.org/post/aleo-completes-security-audits-of-snarkos-and-snarkvm/

    +

    [16] Lurk zkVM. Retrieved from https://github.com/lurk-lab

    +

    [17] Piecrust VM. Retrieved from https://docs.rs/piecrust/latest/piecrust/

    +

    [18] Ceno: Non-uniform, Segment and Parallel Zero-knowledge Virtual Machine. Retrieved from https://eprint.iacr.org/2024/387

    +

    [19] ZkVM: a new design for fast, confidential smart contracts. Retrieved from https://stellar.org/blog/developers/zkvm-a-new-design-for-fast-confidential-smart-contracts

    +

    [20] Novanet. Retrieved from https://www.novanet.xyz/blog

    +

    [21] ZKLLVM. Retrieved from https://github.com/NilFoundation/zkLLVM

    +

    [22] zkMove 0.2.0 - Achieving Full Bytecode Compatibility with Move. Retrieved from https://www.zkmove.net/2023-06-20-zkMove-0.2.0-Achieving-Full-Bytecode-Compatibility-with-Move/

    +

    [23] O1VM. Retrieved from https://github.com/o1-labs/proof-systems/tree/master/o1vm

    ]]> Moudy @@ -925,459 +2124,983 @@ or suggestions! Feel free to join the 2024-08-23T12:00:00.000Z - Nescience: A user-centric state-separation architecture.

    Disclaimer: This content is a work in progress. Some components may be updated, changed, or expanded as new research findings become available.

    A. Introduction

    In blockchain applications, privacy settings are typically predefined by developers, leaving users with limited control. This traditional, + Nescience: A user-centric state-separation architecture.

    + +

    Disclaimer: This content is a work in progress. Some components may be updated, changed, or expanded as new research findings become available.

    +

    In blockchain applications, privacy settings are typically predefined by developers, leaving users with limited control. This traditional, one-size-fits-all approach often leads to inefficiencies and potential privacy concerns as it fails to cater to the diverse needs of individual users. The Nescience state-separation architecture (NSSA) aims to address these issues by shifting privacy control from developers to users. NSSA introduces a flexible, user-centric approach that allows for customized privacy settings to better meet individual needs. This blog post will delve into the details of NSSA, -including its different execution types, cryptographic foundations, and unique challenges.

    Introducing NSSA: A user-centric approach​

    NSSA gives users control over their privacy settings by introducing shielded (which creates a layer of privacy for the outputs, and only the necessary details are shared) +including its different execution types, cryptographic foundations, and unique challenges.

    +

    Introducing NSSA: A user-centric approach​

    +

    NSSA gives users control over their privacy settings by introducing shielded (which creates a layer of privacy for the outputs, and only the necessary details are shared) and deshielded (which reveal private details, making them publicly visible) execution types in addition to the traditional public and private modes. This flexibility allows users to customize their privacy settings to match their unique needs, whether they require high levels of confidentiality or more transparency. In NSSA, the system is divided into two states: public and private. The public state uses an account-based model while the private state employs a UTXO-based (unspent transaction output) model. Private executions within NSSA utilize UTXO exchanges, ensuring that transaction details remain confidential. The sequencer verifies these exchanges without accessing specific details, enhancing privacy by unlinking sender and receiver identities. Zero-knowledge proofs (ZKPs) allow users to prove transaction validity without revealing data, maintaining the integrity and confidentiality of private transactions. UTXOs contain assets such as balances, NFTs, or private storage data, and are stored in plaintext within Sparse Merkle trees (SMTs) in the private state and -as hashes in the public state. This dual-storage approach keeps UTXO details confidential while allowing public verification through hashes, achieving a balance between privacy and transparency.

    Implementing NSSA introduces unique challenges, particularly in cryptographic implementation and maintaining the integrity of private executions. These challenges are addressed +as hashes in the public state. This dual-storage approach keeps UTXO details confidential while allowing public verification through hashes, achieving a balance between privacy and transparency.

    +

    Implementing NSSA introduces unique challenges, particularly in cryptographic implementation and maintaining the integrity of private executions. These challenges are addressed through various solutions such as ZKPs, which ensure transaction validity without compromising privacy, and the dual-storage approach, which maintains confidentiality while enabling public verification. By allowing users to customize their privacy settings, NSSA enhances user experience and promotes wider adoption of private execution platforms. As we move towards -a future where user-empowered privacy control is crucial, NSSA provides a flexible and user-centric solution that meets the diverse needs of blockchain users.

    Why NSSA differs from other hybrid execution platforms​

    In many existing hybrid execution platforms, privacy settings are predefined by developers, often applying a one-size-fits-all approach that does not accommodate the +a future where user-empowered privacy control is crucial, NSSA provides a flexible and user-centric solution that meets the diverse needs of blockchain users.

    +

    Why NSSA differs from other hybrid execution platforms​

    +

    In many existing hybrid execution platforms, privacy settings are predefined by developers, often applying a one-size-fits-all approach that does not accommodate the diverse privacy needs of users. These platforms blend public and private states, but control over privacy remains with the application developers. While this approach is straightforward for developers (who bear the responsibility for any potential privacy leaks), it leaves users with no control over their own privacy settings. This rigidity becomes problematic as user needs evolve over time, or as new regulations necessitate changes to privacy configurations. In such cases, -updates to decentralized applications are required to adjust privacy settings, which can disrupt the user experience and create friction.

    NSSA addresses these limitations by introducing a groundbreaking concept: selective privacy. Unlike traditional platforms where privacy +updates to decentralized applications are required to adjust privacy settings, which can disrupt the user experience and create friction.

    +

    NSSA addresses these limitations by introducing a groundbreaking concept: selective privacy. Unlike traditional platforms where privacy is static and determined by developers, selective privacy empowers users to dynamically choose their own privacy levels based on their unique needs and sensitivity. -This flexibility is critical in a decentralized ecosystem where the diversity of users and use cases demands a more adaptable privacy solution.

    In the NSSA model, users have the autonomy to select how they interact with decentralized applications (dapps) by choosing from four types of transaction executions: public, +This flexibility is critical in a decentralized ecosystem where the diversity of users and use cases demands a more adaptable privacy solution.

    +

    In the NSSA model, users have the autonomy to select how they interact with decentralized applications (dapps) by choosing from four types of transaction executions: public, private, shielded, and deshielded. This model allows users to tailor their privacy settings on a per-transaction basis, selecting the most appropriate execution type for each specific interaction. For instance, a user concerned about data confidentiality might opt for a fully private transaction while another user, wary of privacy but seeking transparency, -might choose a public execution.

    While selective privacy may appear complex, especially for users who are not technically inclined, Nescience mitigates this by allowing the community or developers to +might choose a public execution.

    +

    While selective privacy may appear complex, especially for users who are not technically inclined, Nescience mitigates this by allowing the community or developers to establish best practices and recommended approaches. These guidelines provide users with an informed starting point, and over time, users can adjust their privacy settings as their preferences and trust in the platform evolve. Importantly, selective privacy gives users the right to alter their privacy level at any point in the future, -ensuring that their privacy settings remain aligned with their needs as they change.

    This approach not only empowers users but also facilitates greater adoption of dapps. Users who are skeptical about privacy concerns can initially engage with transparent +ensuring that their privacy settings remain aligned with their needs as they change.

    +

    This approach not only empowers users but also facilitates greater adoption of dapps. Users who are skeptical about privacy concerns can initially engage with transparent transactions and gradually shift towards more private executions as they gain confidence in the system and vice versa for users who start with privacy but later find transparency beneficial for certain transactions. In this way, selective privacy bridges the gap between privacy and transparency, allowing for an optimal balance to emerge from the community’s -collective preferences.

    To liken this to open-source projects: in traditional systems, developers fix privacy rules much like immutable code—users must comply with these fixed settings. -In contrast, with selective privacy, the rules are malleable and shaped by the users’ preferences, enabling the community to find the ideal balance between privacy and efficiency over time.

    NSSA is distinct from traditional zero-knowledge (ZK) rollups in several key ways. One of the unique features of NSSA is its public execution type, which does not +collective preferences.

    +

    To liken this to open-source projects: in traditional systems, developers fix privacy rules much like immutable code—users must comply with these fixed settings. +In contrast, with selective privacy, the rules are malleable and shaped by the users’ preferences, enabling the community to find the ideal balance between privacy and efficiency over time.

    +

    NSSA is distinct from traditional zero-knowledge (ZK) rollups in several key ways. One of the unique features of NSSA is its public execution type, which does not require ZKPs or a zero-knowledge virtual machine (zkVM). This provides a significant advantage in terms of scalability and efficiency as users can choose public executions for -transactions that do not require enhanced privacy, avoiding the overhead associated with ZKP generation and verification.

    Moreover, NSSA introduces two additional execution types—shielded and deshielded—which further distinguish it from traditional privacy-preserving rollups. +transactions that do not require enhanced privacy, avoiding the overhead associated with ZKP generation and verification.

    +

    Moreover, NSSA introduces two additional execution types—shielded and deshielded—which further distinguish it from traditional privacy-preserving rollups. These execution types allow for more nuanced control over privacy, giving users the ability to shield certain aspects of a transaction while deshielding others. -This flexibility sets NSSA apart as a more adaptable and user-centric platform, catering to a wide range of privacy needs without imposing a one-size-fits-all solution.

    By combining selective privacy with a flexible execution model, NSSA offers a more robust and adaptable framework for decentralized applications, -ensuring that users maintain control over their privacy while benefiting from the security and efficiency of blockchain technology.

    How Nescience state-separation architecture can be used​

    NSSA offers a flexible, privacy-preserving add-on that can be applied to existing dapps. +This flexibility sets NSSA apart as a more adaptable and user-centric platform, catering to a wide range of privacy needs without imposing a one-size-fits-all solution.

    +

    By combining selective privacy with a flexible execution model, NSSA offers a more robust and adaptable framework for decentralized applications, +ensuring that users maintain control over their privacy while benefiting from the security and efficiency of blockchain technology.

    +

    How Nescience state-separation architecture can be used​

    +

    NSSA offers a flexible, privacy-preserving add-on that can be applied to existing dapps. One of the emerging trends in the blockchain space is that each dapp is expected to have its own rollup for efficiency, and it is estimated that Ethereum could see the deployment of different rollups in the near future. A key question arises: how many of these rollups will incorporate privacy? For dapp developers who want to offer flexible, -user-centric privacy features, NSSA provides a solution through selective privacy.

    Use case: Adding privacy to existing dapps​

    Consider a dapp running on a transparent network that offers no inherent privacy to its users. Converting this dapp to a privacy-preserving architecture from scratch would +user-centric privacy features, NSSA provides a solution through selective privacy.

    +

    Use case: Adding privacy to existing dapps​

    +

    Consider a dapp running on a transparent network that offers no inherent privacy to its users. Converting this dapp to a privacy-preserving architecture from scratch would require significant effort, restructuring, and a deep understanding of cryptographic frameworks. However, with NSSA, the dapp does not need to undergo extensive changes. -Instead, the Nescience state-separation model can be deployed as an add-on, offering selective privacy as an option for the dapp’s users.

    This allows the dapp to retain its existing functionality while providing users with a choice between the traditional, transparent version and a new version with selective privacy features. +Instead, the Nescience state-separation model can be deployed as an add-on, offering selective privacy as an option for the dapp’s users.

    +

    This allows the dapp to retain its existing functionality while providing users with a choice between the traditional, transparent version and a new version with selective privacy features. With NSSA, the privacy settings are flexible, meaning users can tailor their level of privacy according to their individual needs while the dapp operates on its current infrastructure. -This contrasts sharply with the typical approach, where dapps are either entirely transparent or fully private, with no flexibility for users to select their own privacy preferences.

    Key advantage: Decoupling from the host chain​

    A key feature of NSSA is that it operates independently of the privacy characteristics of the host blockchain. Whether the host chain is fully transparent or fully private, +This contrasts sharply with the typical approach, where dapps are either entirely transparent or fully private, with no flexibility for users to select their own privacy preferences.

    +

    Key advantage: Decoupling from the host chain​

    +

    A key feature of NSSA is that it operates independently of the privacy characteristics of the host blockchain. Whether the host chain is fully transparent or fully private, the Nescience state-separation architecture can be deployed on top of it, offering users the ability to choose their own privacy settings. -This decoupling from the host chain’s inherent privacy model is critical as it allows users to benefit from selective privacy even in environments that were not originally designed to offer it.

    In fully private chains, NSSA allows users to selectively reveal transaction details when compliance with regulations or other requirements is necessary. -In fully transparent chains, NSSA allows users to maintain privacy for specific transactions, offering flexibility that would not otherwise be possible.

    Conclusion​

    NSSA provides a powerful tool for dapp developers who want to offer selective privacy to their users without the need for a complete overhaul of their existing systems. +This decoupling from the host chain’s inherent privacy model is critical as it allows users to benefit from selective privacy even in environments that were not originally designed to offer it.

    +

    In fully private chains, NSSA allows users to selectively reveal transaction details when compliance with regulations or other requirements is necessary. +In fully transparent chains, NSSA allows users to maintain privacy for specific transactions, offering flexibility that would not otherwise be possible.

    +

    Conclusion​

    +

    NSSA provides a powerful tool for dapp developers who want to offer selective privacy to their users without the need for a complete overhaul of their existing systems. By deploying NSSA as an add-on, dapps can give users the ability to choose their own privacy settings whether they are operating on -transparent or private blockchains. This flexibility makes NSSA a valuable option for any dapp looking to provide enhanced privacy options while maintaining efficiency and ease of use.

    B. Design

    In this section, we will delve into the core design components of the Nescience state-separation architecture, covering its key structural elements and the mechanisms -that drive its functionality. We will explore the following topics:

    1. Architecture's components: An in-depth look at the foundational building blocks of NSSA, including the public and private states, UTXO structures, zkVM, and smart contracts. -These components work together to facilitate secure, flexible, and scalable transactions within the architecture.

    2. General execution overview: We will outline the overall flow of transaction execution within NSSA, describing how users interact with the system and how the architecture -supports various types of executions—public, private, shielded, and deshielded—while preserving privacy and efficiency.

    3. Execution processes and UTXO management: This section will focus on the lifecycle of UTXOs within the architecture, from their generation to consumption. -We will also cover the processes involved in managing UTXOs, including proof generation, state transitions, and ensuring transaction validity.

    These topics will provide a comprehensive understanding of how NSSA enables flexible and secure interactions within dapps.

    1. Architecture's components​

    NSSA introduces an advanced prototype execution framework designed to enhance privacy and security in blockchain applications. -This framework integrates several essential components: the public state, private state, zkVM, various execution types, Nescience users, and smart contracts.

    a) Public state​

    The public state in the NSSA is a fundamental component designed to hold all publicly accessible information within +transparent or private blockchains. This flexibility makes NSSA a valuable option for any dapp looking to provide enhanced privacy options while maintaining efficiency and ease of use.

    +

    B. Design

    +

    In this section, we will delve into the core design components of the Nescience state-separation architecture, covering its key structural elements and the mechanisms +that drive its functionality. We will explore the following topics:

    +
      +
    1. +

      Architecture's components: An in-depth look at the foundational building blocks of NSSA, including the public and private states, UTXO structures, zkVM, and smart contracts. +These components work together to facilitate secure, flexible, and scalable transactions within the architecture.

      +
      2. +
    2. +

      General execution overview: We will outline the overall flow of transaction execution within NSSA, describing how users interact with the system and how the architecture +supports various types of executions—public, private, shielded, and deshielded—while preserving privacy and efficiency.

      +
      3. +
    3. +

      Execution processes and UTXO management: This section will focus on the lifecycle of UTXOs within the architecture, from their generation to consumption. +We will also cover the processes involved in managing UTXOs, including proof generation, state transitions, and ensuring transaction validity.

      +
      4. +
    +

    These topics will provide a comprehensive understanding of how NSSA enables flexible and secure interactions within dapps.

    +

    1. Architecture's components​

    +
    +

    NSSA introduces an advanced prototype execution framework designed to enhance privacy and security in blockchain applications. +This framework integrates several essential components: the public state, private state, zkVM, various execution types, Nescience users, and smart contracts.

    +

    a) Public state​

    +
    +

    The public state in the NSSA is a fundamental component designed to hold all publicly accessible information within the blockchain network. This state is organized as a single Merkle tree structure, a sophisticated data structure that ensures efficient and secure data verification. -The public state includes critical information such as user balances and the public storage data of smart contracts.

    In an account-based model, the public state operates by storing each account or smart contract's public data as individual leaf nodes within the Merkle tree. +The public state includes critical information such as user balances and the public storage data of smart contracts.

    +

    In an account-based model, the public state operates by storing each account or smart contract's public data as individual leaf nodes within the Merkle tree. When transactions occur, they directly modify the state by updating these leaf nodes. This direct modification ensures that the most current state of the network -is always reflected accurately.

    The Merkle tree structure is essential for maintaining data integrity. Each leaf node contains a hash of a data block, and each non-leaf node contains the +is always reflected accurately.

    +

    The Merkle tree structure is essential for maintaining data integrity. Each leaf node contains a hash of a data block, and each non-leaf node contains the hash of its child nodes. This hierarchical arrangement means that any change in the data will result in a change in the corresponding hash, making it easy to detect any tampering. The root hash, or Merkle root, is stored on the blockchain, providing a cryptographic guarantee of the data's integrity. This root hash serves as a single, -concise representation of the entire state, enabling quick and reliable verification by any network participant.

    Transparency is a key feature of the public state. All data stored within this state is openly accessible and verifiable by any participant in the network. +concise representation of the entire state, enabling quick and reliable verification by any network participant.

    +

    Transparency is a key feature of the public state. All data stored within this state is openly accessible and verifiable by any participant in the network. This openness ensures that all transactions and state changes are visible and auditable, fostering trust and accountability. For example, user balances are publicly viewable, which helps ensure transparency and trust in the system. Similarly, public smart contract storage can be accessed and verified by anyone, -making it suitable for applications that require public scrutiny and auditability, such as public record updates and some financial transactions.

    The workflow of managing the public state involves several steps to ensure data integrity and transparency. When a user initiates a transaction involving public data, +making it suitable for applications that require public scrutiny and auditability, such as public record updates and some financial transactions.

    +

    The workflow of managing the public state involves several steps to ensure data integrity and transparency. When a user initiates a transaction involving public data, the relevant changes are proposed and applied to the public state tree. The transaction details, such as transferring funds between accounts or updating smart contract storage, update the corresponding leaf nodes in the Merkle tree. Following this, the hashes of the affected nodes are recalculated up to the root, ensuring that the entire tree accurately reflects the new state of the network. The updated Merkle root is then recorded on the blockchain, allowing all network participants to verify the integrity -of the public state. Any discrepancy in the data will result in a mismatched root hash, signaling potential tampering or errors.

    In summary, the public state in NSSA leverages the robustness of the Merkle tree structure to provide a secure, transparent, and verifiable environment for publicly +of the public state. Any discrepancy in the data will result in a mismatched root hash, signaling potential tampering or errors.

    +

    In summary, the public state in NSSA leverages the robustness of the Merkle tree structure to provide a secure, transparent, and verifiable environment for publicly accessible information. By operating on an account-based model and maintaining rigorous data integrity checks, the public state ensures that all transactions are -transparent and trustworthy, laying a strong foundation for a reliable blockchain network.

    b) Private state​

    The private state in the NSSA is a sophisticated system designed to maintain user privacy while ensuring transaction integrity. +transparent and trustworthy, laying a strong foundation for a reliable blockchain network.

    +

    b) Private state​

    +
    +

    The private state in the NSSA is a sophisticated system designed to maintain user privacy while ensuring transaction integrity. Each user has their own individual Merkle tree, which holds their private information such as balances and storage data. This structure is distinct from the public state, which uses an account-based model. Instead, the private state employs a UTXO-based model. In this model, each transaction output is a discrete -unit that can be independently spent in future transactions. This design provides users with granular control over their transaction outputs.

    A key aspect of maintaining privacy within the private state is the use of ZKPs. ZKPs allow transactions to be validated without revealing any +unit that can be independently spent in future transactions. This design provides users with granular control over their transaction outputs.

    +

    A key aspect of maintaining privacy within the private state is the use of ZKPs. ZKPs allow transactions to be validated without revealing any underlying private data. This means that while the system can verify that a transaction is legitimate, the details of the transaction remain confidential. Only parties -with the appropriate viewing key can access and reconstruct the user’s list of UTXOs, ensuring that sensitive information is protected.

    The private state also employs a dual-storage approach to balance privacy and transparency. UTXOs are stored in plaintext within SMTs in the private state, +with the appropriate viewing key can access and reconstruct the user’s list of UTXOs, ensuring that sensitive information is protected.

    +

    The private state also employs a dual-storage approach to balance privacy and transparency. UTXOs are stored in plaintext within SMTs in the private state, providing detailed and accessible records for the user. In contrast, the public state only holds hashes of these UTXOs. This method ensures that while the public can verify -the existence and integrity of private transactions through these hashes, they cannot access the specific details.

    The workflow for a transaction in the private state begins with the user initiating a transaction involving their private data, such as transferring a private balance or +the existence and integrity of private transactions through these hashes, they cannot access the specific details.

    +

    The workflow for a transaction in the private state begins with the user initiating a transaction involving their private data, such as transferring a private balance or updating private smart contract storage. The transaction involves spending existing UTXOs, represented as leaves in the Merkle tree, and creating new UTXOs, which are then appended to the user’s private list. The zkVM generates a ZKP to validate the transaction without revealing -any private data, ensuring the transaction adheres to the system's rules.

    Once the proof is generated, it is submitted to the sequencer, which verifies the transaction’s validity. Upon successful verification, the nullifier is added to the nullifier set, -preventing double spending of the same UTXO. The use of ZKPs and nullifiers ensures that the private state maintains both security and privacy.

    In summary, the private state in NSSA is meticulously designed to provide users with control over their private information while ensuring the security and integrity of transactions. +any private data, ensuring the transaction adheres to the system's rules.

    +

    Once the proof is generated, it is submitted to the sequencer, which verifies the transaction’s validity. Upon successful verification, the nullifier is added to the nullifier set, +preventing double spending of the same UTXO. The use of ZKPs and nullifiers ensures that the private state maintains both security and privacy.

    +

    In summary, the private state in NSSA is meticulously designed to provide users with control over their private information while ensuring the security and integrity of transactions. By utilizing a UTXO-based model, individual Merkle trees, ZKPs, and a dual-storage system, NSSA achieves a balance between confidentiality and verifiability, -making it a robust solution for managing private blockchain transactions.

    c) ZkVM (zero-knowledge virtual machine)​

    The zkVM is a pivotal component in NSSA, designed to uphold the highest standards +making it a robust solution for managing private blockchain transactions.

    +

    c) ZkVM (zero-knowledge virtual machine)​

    +
    +

    The zkVM is a pivotal component in NSSA, designed to uphold the highest standards of privacy and security in blockchain transactions. Its primary function is to generate and aggregate ZKPs, enabling users to validate the correctness of their transactions without disclosing any underlying details. This capability is crucial for maintaining the confidentiality and integrity of sensitive -data within the blockchain network.

    ZKPs are sophisticated cryptographic protocols that allow one party, the prover, to convince another party, the verifier, that a certain statement is true, +data within the blockchain network.

    +

    ZKPs are sophisticated cryptographic protocols that allow one party, the prover, to convince another party, the verifier, that a certain statement is true, without revealing any information beyond the validity of the statement itself. In the context of the zkVM, this means users can prove their transactions are valid without -exposing transaction specifics, such as amounts or parties involved. This process is essential for transactions within the private state, where maintaining confidentiality is paramount.

    The generation of ZKPs involves intricate cryptographic computations. When a user initiates a transaction, the zkVM processes the transaction inputs and produces a proof +exposing transaction specifics, such as amounts or parties involved. This process is essential for transactions within the private state, where maintaining confidentiality is paramount.

    +

    The generation of ZKPs involves intricate cryptographic computations. When a user initiates a transaction, the zkVM processes the transaction inputs and produces a proof that the transaction adheres to the protocol's rules. This proof must be robust enough to convince the verifier of the transaction's validity while preserving the privacy -of the transaction details.

    Performance optimization is another critical function of the zkVM. In a typical blockchain scenario, verifying multiple individual proofs can be computationally intensive +of the transaction details.

    +

    Performance optimization is another critical function of the zkVM. In a typical blockchain scenario, verifying multiple individual proofs can be computationally intensive and time consuming, potentially leading to network congestion and delays. To address this, the zkVM can aggregate multiple ZKPs into a single, consolidated proof. This aggregation significantly reduces the verification overhead as the verifier needs to check only one comprehensive proof rather than multiple individual ones. -This efficiency is vital for maintaining high throughput and low latency in the blockchain network, ensuring that the system can handle a large volume of transactions swiftly and securely.

    Furthermore, the zkVM's role extends beyond mere proof generation and aggregation. It also ensures that all transactions meet the required privacy and security standards +This efficiency is vital for maintaining high throughput and low latency in the blockchain network, ensuring that the system can handle a large volume of transactions swiftly and securely.

    +

    Furthermore, the zkVM's role extends beyond mere proof generation and aggregation. It also ensures that all transactions meet the required privacy and security standards before they are recorded on the blockchain. By interacting seamlessly with other components such as the public and private states, the zkVM ensures that any transaction, -whether it involves public data, private data, or a mix of both, is thoroughly validated and secured.

    In summary, the zkVM is essential for the NSSA, providing the cryptographic backbone necessary to support secure and private transactions. Its ability to generate and +whether it involves public data, private data, or a mix of both, is thoroughly validated and secured.

    +

    In summary, the zkVM is essential for the NSSA, providing the cryptographic backbone necessary to support secure and private transactions. Its ability to generate and aggregate ZKPs not only preserves the confidentiality of user data but also enhances the overall efficiency and scalability of the blockchain network. -By ensuring that all transactions are validated without revealing sensitive information, the zkVM upholds the integrity and trustworthiness of the Nescience blockchain system.

    d) Execution types in NSSA​

    NSSA incorporates multiple execution types to cater to varying levels of privacy and security requirements. -These execution types—public, private, shielded, and deshielded—are designed to provide users with flexible options for managing their transactions based on their specific privacy needs.

    1. Public executions​

    Public executions are straightforward transactions that involve reading from and writing to the public state. In this model, data is openly accessible and verifiable +By ensuring that all transactions are validated without revealing sensitive information, the zkVM upholds the integrity and trustworthiness of the Nescience blockchain system.

    +

    d) Execution types in NSSA​

    +
    +

    NSSA incorporates multiple execution types to cater to varying levels of privacy and security requirements. +These execution types—public, private, shielded, and deshielded—are designed to provide users with flexible options for managing their transactions based on their specific privacy needs.

    +

    1. Public executions​

    +

    Public executions are straightforward transactions that involve reading from and writing to the public state. In this model, data is openly accessible and verifiable by all participants in the network. Public executions do not require ZKPs since transparency is the primary goal. This execution type is ideal -for non-sensitive transactions where public visibility is beneficial, such as updating public records, performing open financial transactions, or interacting with public smart contracts.

    The workflow for a public execution starts with a user initiating a transaction that modifies public data. The transaction details are then used to update the relevant +for non-sensitive transactions where public visibility is beneficial, such as updating public records, performing open financial transactions, or interacting with public smart contracts.

    +

    The workflow for a public execution starts with a user initiating a transaction that modifies public data. The transaction details are then used to update the relevant leaf nodes in the Merkle tree. As changes are made, the hashes of affected nodes are recalculated up to the root, ensuring that the entire tree reflects the most recent state. -Finally, the updated Merkle root is recorded on the blockchain, making the new state publicly verifiable.

    2. Private executions​

    Private executions are designed for confidential transactions, reading from and writing to the private state. These transactions require ZKPs to ensure that while the +Finally, the updated Merkle root is recorded on the blockchain, making the new state publicly verifiable.

    +

    2. Private executions​

    +

    Private executions are designed for confidential transactions, reading from and writing to the private state. These transactions require ZKPs to ensure that while the transaction details are validated, the actual data remains private. This execution type is suitable for scenarios where privacy is crucial, such as private financial -transactions or sensitive data management within smart contracts.

    In a private execution, the user initiates a transaction involving private data. The transaction spends existing UTXOs and creates new ones, all of which are represented as +transactions or sensitive data management within smart contracts.

    +

    In a private execution, the user initiates a transaction involving private data. The transaction spends existing UTXOs and creates new ones, all of which are represented as leaves in the Merkle tree. The zkVM generates a ZKP to validate the transaction without revealing private data. This proof is submitted to the sequencer, which verifies the proof to ensure the transaction's validity. Upon successful verification, the nullifier is added to the nullifier set, and the private state is updated -with the new Merkle root.

    3. Shielded executions​

    Shielded executions create a layer of privacy for the outputs by allowing interactions between the public and private states. When a transaction occurs in a shielded execution, +with the new Merkle root.

    +

    3. Shielded executions​

    +

    Shielded executions create a layer of privacy for the outputs by allowing interactions between the public and private states. When a transaction occurs in a shielded execution, details of the transaction are processed within the private state, ensuring that sensitive information remains confidential. Only the necessary details are shared with the public state, -often in a masked or encrypted form. This approach allows for the validation of the transaction without revealing critical data, thus preserving the privacy of the involved parties.

    The workflow for shielded executions begins with the user initiating a transaction that reads from the public state and prepares to write to the private state. Public data is accessed, +often in a masked or encrypted form. This approach allows for the validation of the transaction without revealing critical data, thus preserving the privacy of the involved parties.

    +

    The workflow for shielded executions begins with the user initiating a transaction that reads from the public state and prepares to write to the private state. Public data is accessed, and the private state is prepared to receive new data. The zkVM generates a ZKP to hide the receiver’s identity. This proof is submitted to the sequencer, which verifies the proof to ensure the transaction's validity. If valid, the private state is updated with the new data while the public state reflects the change without revealing private details. -This type of execution is particularly useful for scenarios where the receiver’s identity needs to be hidden, such as in anonymous donation systems or confidential data storage.

    4. Deshielded executions​

    Deshielded executions operate in the opposite manner of shielded executions, where data is read from the private state and written to the public state. This execution type is useful -in situations where the sender's identity needs to be kept confidential while making the transaction results publicly visible.

    In a deshielded execution, the user initiates a transaction that reads from the private state and prepares to write to the public state. Private data is accessed, +This type of execution is particularly useful for scenarios where the receiver’s identity needs to be hidden, such as in anonymous donation systems or confidential data storage.

    +

    4. Deshielded executions​

    +

    Deshielded executions operate in the opposite manner of shielded executions, where data is read from the private state and written to the public state. This execution type is useful +in situations where the sender's identity needs to be kept confidential while making the transaction results publicly visible.

    +

    In a deshielded execution, the user initiates a transaction that reads from the private state and prepares to write to the public state. Private data is accessed, and the transaction details are prepared. The zkVM generates a ZKP to hide the sender’s identity. This proof is then submitted to the sequencer, which verifies the proof to ensure the transaction's validity. Once verified, the public state is updated with the new data, reflecting the change while keeping the sender’s details confidential. This can be useful when transparency is needed, such as when auditing or proving certain aspects of a transaction to a wider audience. By selectively deshielding certain transactions, users can control what information is shared publicly, thus maintaining a balance between privacy and transparency -as required by their specific use case.

    Table of execution types​

    TypeRead fromWrite toZKP requiredUse caseDescription
    PublicPublic statePublic stateNoNon-sensitive transactions requiring transparency.Ideal for transactions that do not require privacy, ensuring full transparency.
    PrivatePrivate statePrivate stateYesConfidential transactions needing privacy.Suitable for transactions that require confidentiality. Ensures that transaction details remain private through the use of ZKPs.
    ShieldedPublic statePrivate stateYesTransactions where the receiver’s identity needs to be hidden.Hides the identity of the receiver while keeping the transaction details private. Suitable for anonymous donations or confidential data storage.
    DeshieldedPrivate statePublic stateYesTransactions where the sender’s identity needs to be hidden.Ensures the sender’s identity remains confidential while making the transaction results public. Suitable for confidential disbursements or anonymized data publication.

    By supporting a range of execution types, NSSA provides a flexible and robust framework for managing privacy and security in blockchain transactions. +as required by their specific use case.

    +

    Table of execution types​

    +
    TypeRead fromWrite toZKP requiredUse caseDescription
    PublicPublic statePublic stateNoNon-sensitive transactions requiring transparency.Ideal for transactions that do not require privacy, ensuring full transparency.
    PrivatePrivate statePrivate stateYesConfidential transactions needing privacy.Suitable for transactions that require confidentiality. Ensures that transaction details remain private through the use of ZKPs.
    ShieldedPublic statePrivate stateYesTransactions where the receiver’s identity needs to be hidden.Hides the identity of the receiver while keeping the transaction details private. Suitable for anonymous donations or confidential data storage.
    DeshieldedPrivate statePublic stateYesTransactions where the sender’s identity needs to be hidden.Ensures the sender’s identity remains confidential while making the transaction results public. Suitable for confidential disbursements or anonymized data publication.
    +
    +

    By supporting a range of execution types, NSSA provides a flexible and robust framework for managing privacy and security in blockchain transactions. Whether the need is for complete transparency, total privacy, or a balanced approach, NSSA's execution types allow users to select the level of confidentiality -that best fits their requirements. This flexibility enhances the overall utility of the blockchain, making it suitable for a wide array of applications and use cases.

    e) Nescience users​

    Nescience users are integral to the architecture, managing balances and assets within the blockchain network and invoking smart contracts with various privacy options. -They can choose the appropriate execution type—public, private, shielded, or deshielded—based on their specific privacy and security needs.

    Users handle both public and private balances. Public balances are visible to all network participants and suitable for non-sensitive transactions, +that best fits their requirements. This flexibility enhances the overall utility of the blockchain, making it suitable for a wide array of applications and use cases.

    +

    e) Nescience users​

    +
    +

    Nescience users are integral to the architecture, managing balances and assets within the blockchain network and invoking smart contracts with various privacy options. +They can choose the appropriate execution type—public, private, shielded, or deshielded—based on their specific privacy and security needs.

    +

    Users handle both public and private balances. Public balances are visible to all network participants and suitable for non-sensitive transactions, while private balances are confidential and used for transactions requiring privacy. Digital wallets provide a user-friendly interface for managing -these balances, assets, and transactions, allowing users to select the desired execution type seamlessly.

    Security is ensured through the use of cryptographic keys, which authenticate and verify transactions. ZKPs maintain privacy -by validating transaction correctness without revealing underlying data, ensuring sensitive information remains confidential even during verification.

    The workflow for users involves initiating a transaction, preparing inputs, interacting with smart contracts, generating proofs if needed, +these balances, assets, and transactions, allowing users to select the desired execution type seamlessly.

    +

    Security is ensured through the use of cryptographic keys, which authenticate and verify transactions. ZKPs maintain privacy +by validating transaction correctness without revealing underlying data, ensuring sensitive information remains confidential even during verification.

    +

    The workflow for users involves initiating a transaction, preparing inputs, interacting with smart contracts, generating proofs if needed, and submitting the transaction to the sequencer for verification and state update. This flexible approach supports various use cases, -from financial transactions and decentralized applications to data privacy management, allowing users to maintain control over their privacy settings.

    By offering this high degree of flexibility and security, Nescience enables users to tailor their privacy settings to their specific needs, +from financial transactions and decentralized applications to data privacy management, allowing users to maintain control over their privacy settings.

    +

    By offering this high degree of flexibility and security, Nescience enables users to tailor their privacy settings to their specific needs, ensuring sensitive transactions remain confidential while non-sensitive ones are transparent. This integration of cryptographic keys and ZKPs -provides a robust framework for a wide range of blockchain applications, enhancing both utility and trust within the network.

    f) Smart contracts in NSSA​

    Smart contracts are a core feature of NSSA, providing a way to automate and execute predefined actions based on coded rules. +provides a robust framework for a wide range of blockchain applications, enhancing both utility and trust within the network.

    +

    f) Smart contracts in NSSA​

    +
    +

    Smart contracts are a core feature of NSSA, providing a way to automate and execute predefined actions based on coded rules. Once deployed on the blockchain, these contracts become immutable, meaning their behavior cannot be altered. This ensures that they perform exactly as intended without the risk of tampering. Because the state and data of the contract are stored permanently on the blockchain, all interactions are fully -transparent and auditable, creating a reliable and trustworthy environment.

    One of the key strengths of smart contracts is their ability to automate processes. They are designed to automatically execute when specific conditions are met, +transparent and auditable, creating a reliable and trustworthy environment.

    +

    One of the key strengths of smart contracts is their ability to automate processes. They are designed to automatically execute when specific conditions are met, reducing the need for manual oversight or intermediaries. For example, a smart contract might transfer funds when a certain deadline is reached or update a record -once a task is completed. This self-executing nature makes them efficient and minimizes human error.

    Smart contracts operate deterministically, meaning they will always produce the same result given the same inputs. This predictability is crucial for ensuring reliability, -especially in complex systems. Additionally, they run in isolated environments on the blockchain, which enhances security by preventing unintended interactions with other processes.

    Security is another critical feature of smart contracts. They leverage the underlying cryptographic protections of the blockchain, ensuring that every interaction +once a task is completed. This self-executing nature makes them efficient and minimizes human error.

    +

    Smart contracts operate deterministically, meaning they will always produce the same result given the same inputs. This predictability is crucial for ensuring reliability, +especially in complex systems. Additionally, they run in isolated environments on the blockchain, which enhances security by preventing unintended interactions with other processes.

    +

    Security is another critical feature of smart contracts. They leverage the underlying cryptographic protections of the blockchain, ensuring that every interaction is secure and authenticated. Before deployment, the contract code can be audited and verified to ensure it functions correctly. Once on the blockchain, -the immutable nature of the code prevents unauthorized modifications, further ensuring the integrity of the system.

    Running smart contracts requires computational resources, which are compensated through gas fees. These fees vary depending on the complexity of the operations within the contract. -More resource-intensive contracts incur higher fees, which helps manage the computational load on the blockchain network.

    The workflow of a smart contract begins with its development, where developers code the contract using languages like Rust. Once the code is compiled and deployed to the blockchain, +the immutable nature of the code prevents unauthorized modifications, further ensuring the integrity of the system.

    +

    Running smart contracts requires computational resources, which are compensated through gas fees. These fees vary depending on the complexity of the operations within the contract. +More resource-intensive contracts incur higher fees, which helps manage the computational load on the blockchain network.

    +

    The workflow of a smart contract begins with its development, where developers code the contract using languages like Rust. Once the code is compiled and deployed to the blockchain, it becomes a permanent part of the network. Users can then interact with the contract by sending transactions that invoke specific functions. The contract checks whether the -required conditions are met, and if so, it automatically executes the specified actions, such as transferring tokens or updating data on the blockchain.

    The benefits of smart contracts are numerous. They eliminate the need for intermediaries by providing a system where trust is built into the code itself. +required conditions are met, and if so, it automatically executes the specified actions, such as transferring tokens or updating data on the blockchain.

    +

    The benefits of smart contracts are numerous. They eliminate the need for intermediaries by providing a system where trust is built into the code itself. This not only reduces costs but also increases efficiency by automating repetitive processes. The inherent security of smart contracts, combined with their transparency—where every action is recorded and visible on the blockchain—makes them a powerful tool for ensuring accountability and trust in decentralized systems. -They can be ideal for managing decentralized autonomous organizations (DAOs), where governance decisions are automated through coded rules.

    By integrating smart contracts, NSSA offers a highly versatile, secure, and transparent framework that can support a wide range of applications -across various industries, from finance to governance, supply chains, and more.

    2. General execution overview​

    This section explains the execution process within NSSA, providing an overview of how it works from start to finish. -It outlines the steps involved in each execution type, guiding the reader through the entire process from user interaction to completion.

    The process begins when a user initiates a transaction by invoking a smart contract. This invocation involves selecting at least one of +They can be ideal for managing decentralized autonomous organizations (DAOs), where governance decisions are automated through coded rules.

    +

    By integrating smart contracts, NSSA offers a highly versatile, secure, and transparent framework that can support a wide range of applications +across various industries, from finance to governance, supply chains, and more.

    +

    2. General execution overview​

    +
    +

    This section explains the execution process within NSSA, providing an overview of how it works from start to finish. +It outlines the steps involved in each execution type, guiding the reader through the entire process from user interaction to completion.

    +

    The process begins when a user initiates a transaction by invoking a smart contract. This invocation involves selecting at least one of the four execution types: public, private, shielded, or deshielded. The choice of execution type determines how data will be read from and written to the blockchain, affecting the transaction's privacy and security levels. Each execution type caters to different privacy needs, allowing the user to tailor the transaction according -to their specific requirements, whether it be full transparency or complete confidentiality.

    general

    User actions​

    Step 1: Smart contract selection and input creation

    • Smart contract selection: The user selects a smart contract they wish to invoke.
    • Input creation: The user creates a set of inputs required for the invocation by reading the necessary data from both the public and private states. This includes:
      • Public data such as current account balances, public keys, and smart contract states.
      • Private data such as private account balances and UTXOs.

    Step 2: Choosing execution type

    • Execution type selection: The user selects the type of execution based on their privacy needs. The options include:
      • Public execution: Suitable for transactions where transparency is desired.
      • Private execution: Used when transaction details need to be confidential.
      • Shielded execution: Hides the receiver's identity.
      • Deshielded execution: Hides the sender's identity.
    • ZkVM requirement: If the execution involves private, shielded, or deshielded types, the user must call the zkVM to handle these confidential transactions. -For purely public executions, the zkVM is not needed, and the user can directly transmit the transaction code to the sequencer.

    Step 3: Calling zkVM for proof generation

    • ZkVM compilation: The user calls the zkVM to compile the smart contract with both public and private inputs.
      • Kernel circuit proofs: The zkVM generates individual proofs for each execution type through kernel circuits.
      • Proof aggregation: The zkVM aggregates these individual proofs into a single comprehensive proof, combining both private and public inputs.

    Step 4: Transmitting public inputs and retaining private inputs

    • Retaining private inputs: The user keeps the private inputs secure and does not transmit them.
    • Revealing public inputs: The user transmits the following public inputs to the sequencer:
      • Public inputs of the recursive proof
      • Hashes of UTXOs
      • Updates to the public state
      • Transaction signature
      • Nullifiers (to prevent double spending)

    After completing these steps, the user's part of the execution is done, and the sequencer takes over the process.

    Sequencer actions​

    Step 5: Proof verification

    • Proof and data reception: The sequencer receives the proof and public inputs from the user.
    • Verification process:
      • For private, shielded, and deshielded executions, the sequencer verifies the proof using the provided public data.
      • For public executions, the sequencer reruns the smart contract code with the provided inputs to check the results.
    • Validation: If both the zkVM proofs and public execution results are verified successfully, the sequencer collects the proof and public data to proceed. -If verification fails, the process is aborted, and the transaction is rejected.

    Step 6: Aggregating proofs and finalizing the block

    • Proof aggregation: The sequencer calls the zkVM again to aggregate all received proofs into one comprehensive proof to finalize the block.
    • Finalizing the block:
      • Public state update: The sequencer updates the public state with the new transaction data.
      • Nullifier tree update: Updates the nullifier tree to reflect the new state and prevent double spending.
      • Synchronization mechanism: Runs synchronization mechanisms to ensure fairness and consistency across the network.
      • UTXO validation: Validates the exchanged UTXOs to complete the transaction process.

    This comprehensive process ensures that transactions are executed securely, with the appropriate level of privacy and state updates synchronized across the network.

    Below, we outline the execution process of the four different execution types within NSSA:

    • Public execution:

    public

    • Private execution:

    private

    • Shielded execution:

    shielded

    • Deshielded execution:

    deshielded

    3. Execution processes and UTXO management​

    In Nescience state-separation architecture, UTXOs are key components for managing private data and assets. They serve as private entities that hold both storage and assets, +to their specific requirements, whether it be full transparency or complete confidentiality.

    +

    general

    +

    User actions​

    +
    +

    Step 1: Smart contract selection and input creation

    +
      +
    • Smart contract selection: The user selects a smart contract they wish to invoke.
      • +
    • Input creation: The user creates a set of inputs required for the invocation by reading the necessary data from both the public and private states. This includes: +
        +
      • Public data such as current account balances, public keys, and smart contract states.
        • +
      • Private data such as private account balances and UTXOs.
        • +
      +
      • +
    +

    Step 2: Choosing execution type

    +
      +
    • Execution type selection: The user selects the type of execution based on their privacy needs. The options include: +
        +
      • Public execution: Suitable for transactions where transparency is desired.
        • +
      • Private execution: Used when transaction details need to be confidential.
        • +
      • Shielded execution: Hides the receiver's identity.
        • +
      • Deshielded execution: Hides the sender's identity.
        • +
      +
      • +
    • ZkVM requirement: If the execution involves private, shielded, or deshielded types, the user must call the zkVM to handle these confidential transactions. +For purely public executions, the zkVM is not needed, and the user can directly transmit the transaction code to the sequencer.
      • +
    +

    Step 3: Calling zkVM for proof generation

    +
      +
    • ZkVM compilation: The user calls the zkVM to compile the smart contract with both public and private inputs. +
        +
      • Kernel circuit proofs: The zkVM generates individual proofs for each execution type through kernel circuits.
        • +
      • Proof aggregation: The zkVM aggregates these individual proofs into a single comprehensive proof, combining both private and public inputs.
        • +
      +
      • +
    +

    Step 4: Transmitting public inputs and retaining private inputs

    +
      +
    • Retaining private inputs: The user keeps the private inputs secure and does not transmit them.
      • +
    • Revealing public inputs: The user transmits the following public inputs to the sequencer: +
        +
      • Public inputs of the recursive proof
        • +
      • Hashes of UTXOs
        • +
      • Updates to the public state
        • +
      • Transaction signature
        • +
      • Nullifiers (to prevent double spending)
        • +
      +
      • +
    +

    After completing these steps, the user's part of the execution is done, and the sequencer takes over the process.

    +

    Sequencer actions​

    +
    +

    Step 5: Proof verification

    +
      +
    • Proof and data reception: The sequencer receives the proof and public inputs from the user.
      • +
    • Verification process: +
        +
      • For private, shielded, and deshielded executions, the sequencer verifies the proof using the provided public data.
        • +
      • For public executions, the sequencer reruns the smart contract code with the provided inputs to check the results.
        • +
      +
      • +
    • Validation: If both the zkVM proofs and public execution results are verified successfully, the sequencer collects the proof and public data to proceed. +If verification fails, the process is aborted, and the transaction is rejected.
      • +
    +

    Step 6: Aggregating proofs and finalizing the block

    +
      +
    • Proof aggregation: The sequencer calls the zkVM again to aggregate all received proofs into one comprehensive proof to finalize the block.
      • +
    • Finalizing the block: +
        +
      • Public state update: The sequencer updates the public state with the new transaction data.
        • +
      • Nullifier tree update: Updates the nullifier tree to reflect the new state and prevent double spending.
        • +
      • Synchronization mechanism: Runs synchronization mechanisms to ensure fairness and consistency across the network.
        • +
      • UTXO validation: Validates the exchanged UTXOs to complete the transaction process.
        • +
      +
      • +
    +

    This comprehensive process ensures that transactions are executed securely, with the appropriate level of privacy and state updates synchronized across the network.

    +

    Below, we outline the execution process of the four different execution types within NSSA:

    +
      +
    • Public execution:
      • +
    +

    public

    +
      +
    • Private execution:
      • +
    +

    private

    +
      +
    • Shielded execution:
      • +
    +

    shielded

    +
      +
    • Deshielded execution:
      • +
    +

    deshielded

    +

    3. Execution processes and UTXO management​

    +
    +

    In Nescience state-separation architecture, UTXOs are key components for managing private data and assets. They serve as private entities that hold both storage and assets, facilitating secure and confidential transactions. UTXOs are utilized in three of the four execution types within NSSA: private, shielded, -and deshielded executions. This section explores the lifecycle of UTXOs, detailing their generation, transfer, encryption, and eventual consumption within the private execution framework.

    a) Components of a Nescience UTXO​

    A Nescience UTXO is a critical and versatile component of the private state in the Nescience state-separation architecture. +and deshielded executions. This section explores the lifecycle of UTXOs, detailing their generation, transfer, encryption, and eventual consumption within the private execution framework.

    +

    a) Components of a Nescience UTXO​

    +
    +

    A Nescience UTXO is a critical and versatile component of the private state in the Nescience state-separation architecture. It carries essential information that ensures its proper functionality within private execution, such as the owner, value, private storage slot, non-fungibles, -and other cryptographic components. Below is a detailed breakdown of each component and its role in maintaining the integrity, security, and privacy of the system:

    • Owner: +and other cryptographic components. Below is a detailed breakdown of each component and its role in maintaining the integrity, security, and privacy of the system:

      +
        +
      • +

        Owner: The owner component represents the public key of the entity that controls the UTXO. Only the owner can spend this UTXO, ensuring its security and privacy through public key cryptography. This means that the UTXO remains secure as only the rightful owner, using their private key, can generate valid signatures to authorize the transaction. For example, if Alice owns a UTXO linked to her public key, she must sign any transaction to spend it using her private key. This cryptographic protection ensures that only Alice can authorize -spending the UTXO and transfer it to someone else, such as Bob.

      • Value: +spending the UTXO and transfer it to someone else, such as Bob.

        +
        • +
      • +

        Value: The value in a UTXO represents the balance or asset contained within it. This could be cryptocurrency, tokens, or other digital assets. The value ensures accurate accounting, preventing double spending and maintaining the overall integrity of the system. For instance, if Alice's UTXO has a value of 10 tokens, this represents her ownership of that amount -within the network, and when spent, this value will be deducted from her UTXO and transferred accordingly.

      • Private storage slot: +within the network, and when spent, this value will be deducted from her UTXO and transferred accordingly.

        +
        • +
      • +

        Private storage slot: The private storage slot is an arbitrary and flexible storage space within the UTXO for Nescience applications. It allows users and smart contracts to store additional private data that is only accessible by the owner. This could be used to hold metadata, smart contract states, or user-specific information. For example, if a smart contract is holding private user data, -this information is securely stored in the private storage slot and can only be accessed or modified by the owner, ensuring privacy and security.

      • Non-fungibles: +this information is securely stored in the private storage slot and can only be accessed or modified by the owner, ensuring privacy and security.

        +
        • +
      • +

        Non-fungibles: Non-fungibles within the UTXO represent unique assets, such as NFTs (Non-Fungible Tokens). Each non-fungible asset is assigned a unique serial number or identifier within the UTXO, ensuring its distinctiveness and traceability. For example, if Alice owns a digital artwork represented as an NFT, the non-fungible component of the UTXO will store the unique identifier -for this NFT, preventing duplication or forgery of the digital asset.

      • Random commitment key: +for this NFT, preventing duplication or forgery of the digital asset.

        +
        • +
      • +

        Random commitment key: The random commitment key (RCK) is a randomly generated number used to create a cryptographic commitment to the contents of the UTXO. This commitment ensures the integrity of the data without revealing any private information. By generating a random key for the commitment, the system ensures that even if someone observes the commitment, they cannot infer any details -about the underlying UTXO. For example, RCK helps maintain confidentiality in the system while still allowing the verification of transactions.

      • Nullifier key: +about the underlying UTXO. For example, RCK helps maintain confidentiality in the system while still allowing the verification of transactions.

        +
        • +
      • +

        Nullifier key: The Nullifier key is another randomly generated number, used to ensure that a UTXO is only spent once. When a UTXO is spent, its nullifier key is recorded in a nullifier set to prevent double spending. This key guarantees that once a UTXO is spent, it cannot be reused in another transaction, effectively nullifying it from future use. This mechanism is crucial for -maintaining the security and integrity of the system, as it ensures that no UTXO can be spent more than once.

      b) UTXO lifecycle: From generation to consumption​

      UTXOs in NSSA are created when a transaction outputs a specific value, asset, or data intended for future use. Once generated, these UTXOs become private entities -owned by specific users, containing sensitive information such as balances, private data, or unique assets like NFTs.

      To maintain the required level of confidentiality, UTXOs are encrypted and transferred anonymously across the network. This encryption process ensures that the data within each UTXO +maintaining the security and integrity of the system, as it ensures that no UTXO can be spent more than once.

      +
      • +
    +

    b) UTXO lifecycle: From generation to consumption​

    +
    +

    UTXOs in NSSA are created when a transaction outputs a specific value, asset, or data intended for future use. Once generated, these UTXOs become private entities +owned by specific users, containing sensitive information such as balances, private data, or unique assets like NFTs.

    +

    To maintain the required level of confidentiality, UTXOs are encrypted and transferred anonymously across the network. This encryption process ensures that the data within each UTXO remains hidden from network participants, including the sequencer, while still allowing for verification and validation through ZKPs. These proofs enable the network -to ensure that UTXOs are valid, prevent double spending, and maintain security, all without revealing any sensitive information.

    When a user wishes to spend or transfer a UTXO, the lifecycle progresses towards its consumption. The user must prove ownership and validity of the UTXO through a ZKP, +to ensure that UTXOs are valid, prevent double spending, and maintain security, all without revealing any sensitive information.

    +

    When a user wishes to spend or transfer a UTXO, the lifecycle progresses towards its consumption. The user must prove ownership and validity of the UTXO through a ZKP, which is then verified by the sequencer. This process occurs in private, shielded, and deshielded executions, where confidentiality is a priority. Once the proof is validated, -the UTXO is consumed, meaning it is marked as spent and cannot be reused, ensuring the integrity of the transaction and preventing double spending.

    UTXOs are central to the private, shielded, and deshielded execution types in Nescience. In private executions, UTXOs are transferred securely between parties without revealing any +the UTXO is consumed, meaning it is marked as spent and cannot be reused, ensuring the integrity of the transaction and preventing double spending.

    +

    UTXOs are central to the private, shielded, and deshielded execution types in Nescience. In private executions, UTXOs are transferred securely between parties without revealing any details to the public state. In shielded executions, UTXOs are used to receive assets from the public state while keeping the recipient's identity confidential. Finally, -in deshielded executions, UTXOs are used to send assets from the private state to the public state, while preserving the sender's anonymity.

    Since UTXOs are not exchanged in public executions, this lifecycle analysis is focused solely on private, shielded, and deshielded executions, where privacy and confidentiality are essential. +in deshielded executions, UTXOs are used to send assets from the private state to the public state, while preserving the sender's anonymity.

    +

    Since UTXOs are not exchanged in public executions, this lifecycle analysis is focused solely on private, shielded, and deshielded executions, where privacy and confidentiality are essential. In these contexts, the careful management and transfer of UTXOs ensure that the users' private data and assets remain secure, while still allowing for seamless and confidential transactions -within the network.

    At this point, it's crucial to introduce two key components that will play a significant role in the next section: the ephemeral key and the nillifier.

    • Ephemeral key: The ephemeral key is embedded in the transaction message and plays a crucial role in maintaining privacy. It is used by the sender, alongside the receiver's public key, +within the network.

      +

      At this point, it's crucial to introduce two key components that will play a significant role in the next section: the ephemeral key and the nillifier.

      +
        +
      • +

        Ephemeral key: The ephemeral key is embedded in the transaction message and plays a crucial role in maintaining privacy. It is used by the sender, alongside the receiver's public key, in a key agreement protocol to derive a shared secret. This shared secret is then employed to encrypt the transaction details, ensuring that only those with the receiver's viewing key can decrypt the transaction. By using the ephemeral key, the receiver can regenerate the shared secret, granting access to the transaction's contents. The sender generates the ephemeral key -using their spending key and the UTXO's nullifier, reinforcing the security of the transaction. (more details in key management and addresses section)

      • Nullifier: A nullifier is a unique value tied to a specific UTXO, ensuring that it has not been spent before. Its uniqueness is essential, as a nullifier must never correspond to more +using their spending key and the UTXO's nullifier, reinforcing the security of the transaction. (more details in key management and addresses section)

        +
        • +
      • +

        Nullifier: A nullifier is a unique value tied to a specific UTXO, ensuring that it has not been spent before. Its uniqueness is essential, as a nullifier must never correspond to more than one UTXO—otherwise, even if both UTXOs are valid, only one could be spent. This would undermine the integrity of the system. To spend a UTXO, a proof must be provided showing that the nullifier does not already exist in the Nullifier Tree. Once the transaction is confirmed and included in the blockchain, the nullifier is added to the Nullifier Tree, preventing any future reuse of the same UTXO. A UTXO's nullifier is generated by combining the receiver's nullifier key with the transaction note's commitment, further ensuring its distinctiveness -and security. (More details in nullifier tree section.)

      I) UTXOs in private executions​

      In private executions within NSSA, transactions are handled ensuring maximum privacy by concealing all transaction details from the public state. +and security. (More details in nullifier tree section.)

      +
      • +
    +

    I) UTXOs in private executions​

    +
    +

    In private executions within NSSA, transactions are handled ensuring maximum privacy by concealing all transaction details from the public state. This approach is particularly useful for confidential payments, where the identities of the sender and receiver, as well as the transaction amounts, must remain hidden. -The process is powered by ZKPs, ensuring that only the involved parties have access to the transaction details while maintaining the integrity of the network.

    • Stages of private execution: Private executions operate in two key stages: UTXO consumption and UTXO creation. In the first stage, UTXOs from the private state are used +The process is powered by ZKPs, ensuring that only the involved parties have access to the transaction details while maintaining the integrity of the network.

      +
        +
      • +

        Stages of private execution: Private executions operate in two key stages: UTXO consumption and UTXO creation. In the first stage, UTXOs from the private state are used as inputs for the transaction. In the second stage, new UTXOs are generated as outputs and stored back in the private state. Throughout this process, the details of the -transaction are kept confidential and only shared between the sender and receiver.

      • Private transaction workflow (transaction initialization): The user initiates a private transaction by selecting the input UTXOs that will be spent and determining the +transaction are kept confidential and only shared between the sender and receiver.

        +
        • +
      • +

        Private transaction workflow (transaction initialization): The user initiates a private transaction by selecting the input UTXOs that will be spent and determining the output UTXOs to be created. This involves specifying the amounts to be transferred and the recipient’s private address (a divestified address that hides the recipient's public address from the network). The nullifier key and random number for commitments (RCK) are also generated at this stage to define how these UTXOs can be spent or nullified in the -future by the receiver.

      • Proof generation and verification: Next, the zkVM generates a ZKP to validate the transaction. This proof includes both a membership proof for the input UTXOs, +future by the receiver.

        +
        • +
      • +

        Proof generation and verification: Next, the zkVM generates a ZKP to validate the transaction. This proof includes both a membership proof for the input UTXOs, confirming their presence in the hashed UTXO tree, and a non-membership proof to ensure that the input UTXOs have not already been spent (i.e., they are not in the nullifier tree). -The proof also confirms that the total input value matches the total output value, ensuring no discrepancies. The user then submits the proof, along with the necessary metadata, to the sequencer.

      • Shared secret and encryption: To maintain confidentiality, the sender uses the receiver’s divestified address to generate an ephemeral public key. +The proof also confirms that the total input value matches the total output value, ensuring no discrepancies. The user then submits the proof, along with the necessary metadata, to the sequencer.

        +
        • +
      • +

        Shared secret and encryption: To maintain confidentiality, the sender uses the receiver’s divestified address to generate an ephemeral public key. This allows the creation of a shared secret between the sender and receiver. Using a key derivation function, a symmetric encryption key is generated from the shared secret. -The input and output UTXOs are then encrypted using this symmetric key, ensuring that only the intended recipient can decrypt the data.

      • Broadcasting the transaction: The user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. -These committed UTXOs are sent to the sequencer, which updates the hashed UTXO tree without knowing the transaction details.

      • Decryption by the receiver: After the broadcast, the receiver attempts to decrypt the broadcast UTXOs using their symmetric key, derived from the ephemeral public key. +The input and output UTXOs are then encrypted using this symmetric key, ensuring that only the intended recipient can decrypt the data.

        +
        • +
      • +

        Broadcasting the transaction: The user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. +These committed UTXOs are sent to the sequencer, which updates the hashed UTXO tree without knowing the transaction details.

        +
        • +
      • +

        Decryption by the receiver: After the broadcast, the receiver attempts to decrypt the broadcast UTXOs using their symmetric key, derived from the ephemeral public key. If the receiver successfully decrypts a UTXO, it confirms ownership of that UTXO. The receiver then computes the nullifier for the UTXO and verifies its presence in the hashed -UTXO tree and its absence from the nullifier tree, ensuring it has not been spent. Finally, the new UTXO is added to the receiver’s locally stored UTXO tree for future transactions.

      Throughout the private execution process, the identities of both the sender and receiver, as well as all transaction details, remain hidden from the public. +UTXO tree and its absence from the nullifier tree, ensuring it has not been spent. Finally, the new UTXO is added to the receiver’s locally stored UTXO tree for future transactions.

      +
      • +
    +

    Throughout the private execution process, the identities of both the sender and receiver, as well as all transaction details, remain hidden from the public. The use of ZKPs ensures that the integrity of the transaction is verified without revealing any sensitive information. At the end of the process, -the network guarantees that no participant, aside from the sender and receiver, can deduce any details about the transaction or the involved parties.

    II) UTXOs in shielded executions​

    In shielded executions, the interaction between public and private states provides a hybrid privacy model that balances transparency and confidentiality. +the network guarantees that no participant, aside from the sender and receiver, can deduce any details about the transaction or the involved parties.

    +

    II) UTXOs in shielded executions​

    +
    +

    In shielded executions, the interaction between public and private states provides a hybrid privacy model that balances transparency and confidentiality. This model is suitable for scenarios where the initial step, such as a public transaction, requires visibility, while subsequent actions, such as private asset management, need to remain confidential. One common use case is asset conversion—where a public token is converted into a private token. The conversion is visible on the public ledger, -but subsequent transactions remain private.

    a) How shielded executions work​

    Shielded executions operate in two distinct stages: first, there is a modification of the public state, and then new UTXOs are created and stored in the private state. +but subsequent transactions remain private.

    +
    a) How shielded executions work​
    +

    Shielded executions operate in two distinct stages: first, there is a modification of the public state, and then new UTXOs are created and stored in the private state. Importantly, shielded executions do not consume UTXOs but instead mint them, as new UTXOs are created to reflect the changes in the private state. This structure demands ZKPs to ensure that the newly minted UTXOs are consistent with the modifications in the public state. Here’s a step-by-step breakdown of how the shielded -execution process unfolds:

    1. Transaction initiation: The user initiates a transaction that modifies the public state, such as converting a public token to a private token. -The transaction alters the public state (e.g., balances or smart contract storage) while simultaneously preparing to mint new UTXOs in the private state.

    2. Generating UTXOs: After modifying the public state, the system mints new UTXOs in the private state. These UTXOs must be securely created, ensuring their integrity -and consistency with the initial public state modification. A ZKP is generated by the user to prove that these new UTXOs align with the changes made in the public state.

    3. Key setup for privacy: The sender retrieves the receiver's address and uses it to create a shared secret through an ephemeral public key. This shared secret is then used -to derive a symmetric key, which encrypts the output UTXOs. This encryption ensures that only the intended receiver can decrypt and access the UTXOs.

    4. Broadcasting and verifying UTXOs: After encrypting the UTXOs, the sender broadcasts them to the network. The new hashed UTXOs are sent to the sequencer, +execution process unfolds:

      +
        +
      1. +

        Transaction initiation: The user initiates a transaction that modifies the public state, such as converting a public token to a private token. +The transaction alters the public state (e.g., balances or smart contract storage) while simultaneously preparing to mint new UTXOs in the private state.

        +
        2. +
      2. +

        Generating UTXOs: After modifying the public state, the system mints new UTXOs in the private state. These UTXOs must be securely created, ensuring their integrity +and consistency with the initial public state modification. A ZKP is generated by the user to prove that these new UTXOs align with the changes made in the public state.

        +
        3. +
      3. +

        Key setup for privacy: The sender retrieves the receiver's address and uses it to create a shared secret through an ephemeral public key. This shared secret is then used +to derive a symmetric key, which encrypts the output UTXOs. This encryption ensures that only the intended receiver can decrypt and access the UTXOs.

        +
        4. +
      4. +

        Broadcasting and verifying UTXOs: After encrypting the UTXOs, the sender broadcasts them to the network. The new hashed UTXOs are sent to the sequencer, which verifies the validity of the UTXOs and attaches them to the hashed UTXO tree within the private state. The public inputs for the ZKP circuits consist of the -Pedersen-hashed UTXOs and the modifications in the public state.

      5. Receiver's role: Once the UTXOs are broadcast, the receiver attempts to decrypt each UTXO using the symmetric key derived from the shared secret. If the decryption is successful, -the UTXO belongs to the receiver. The receiver then verifies the UTXO’s validity by checking its inclusion in the hashed UTXO tree and ensuring that its nullifier has not yet been used.

      6. Nullifier check and integration: To prevent double spending, the receiver computes the nullifier for the received UTXO and verifies that it is not already present in the nullifier tree. -Once verified, the receiver adds the UTXO to their locally stored UTXO tree for future use in private transactions.

      While shielded executions offer privacy, certain information is still exposed to the public state, such as the sender's identity. To further enhance privacy, +Pedersen-hashed UTXOs and the modifications in the public state.

      +
      5. +
    5. +

      Receiver's role: Once the UTXOs are broadcast, the receiver attempts to decrypt each UTXO using the symmetric key derived from the shared secret. If the decryption is successful, +the UTXO belongs to the receiver. The receiver then verifies the UTXO’s validity by checking its inclusion in the hashed UTXO tree and ensuring that its nullifier has not yet been used.

      +
      6. +
    6. +

      Nullifier check and integration: To prevent double spending, the receiver computes the nullifier for the received UTXO and verifies that it is not already present in the nullifier tree. +Once verified, the receiver adds the UTXO to their locally stored UTXO tree for future use in private transactions.

      +
      7. +
    +

    While shielded executions offer privacy, certain information is still exposed to the public state, such as the sender's identity. To further enhance privacy, the sender can create empty UTXOs—UTXOs that don’t belong to anyone but are included in the transaction to obfuscate the true details of the transaction. -Though this approach increases the size of the data, it adds a layer of privacy by complicating the identification of meaningful transactions.

    b) Summary of shielded execution flow​
    • Stage 1 (public modification): The user modifies public state data, such as converting tokens from public to private. This stage is visible to the public.
    • Stage 2 (UTXO minting and privacy): New UTXOs are minted in the private state, encrypted, and broadcast to the network. The transaction remains private from this point forward, -secured by ZKPs and cryptographic keys.
    • Receiver’s role: The receiver decrypts the UTXOs and verifies their validity, ensuring the UTXOs are not double spent and are ready for future transactions.

    In summary, shielded executions enable a hybrid privacy model in Nescience, balancing public transparency and private confidentiality. They are well-suited for -transactions requiring initial public visibility, such as asset conversions, while ensuring that subsequent actions remain secure and private within the network.

    III) UTXOs in deshielded executions​

    In NSSA, deshielded executions offer a unique way to move data and assets from the private state to the public state, revealing previously private +Though this approach increases the size of the data, it adds a layer of privacy by complicating the identification of meaningful transactions.

    +
    b) Summary of shielded execution flow​
    +
      +
    • Stage 1 (public modification): The user modifies public state data, such as converting tokens from public to private. This stage is visible to the public.
      • +
    • Stage 2 (UTXO minting and privacy): New UTXOs are minted in the private state, encrypted, and broadcast to the network. The transaction remains private from this point forward, +secured by ZKPs and cryptographic keys.
      • +
    • Receiver’s role: The receiver decrypts the UTXOs and verifies their validity, ensuring the UTXOs are not double spent and are ready for future transactions.
      • +
    +

    In summary, shielded executions enable a hybrid privacy model in Nescience, balancing public transparency and private confidentiality. They are well-suited for +transactions requiring initial public visibility, such as asset conversions, while ensuring that subsequent actions remain secure and private within the network.

    +

    III) UTXOs in deshielded executions​

    +
    +

    In NSSA, deshielded executions offer a unique way to move data and assets from the private state to the public state, revealing previously private information in a controlled and verifiable manner. This type of execution allows for selective disclosure, ensuring transparency when needed while still maintaining the security and privacy of critical details through cryptographic techniques like ZKPs. Deshielded executions are particularly valuable for use cases -such as regulatory compliance reporting, where specific transaction details must be revealed to meet legal requirements, while other sensitive transactions remain private.

    a) Stages of deshielded executions​

    • Stage 1 (UTXO consumption): The process begins in the private state, where UTXOs are consumed as inputs for the transaction. This involves gathering all necessary -UTXOs that contain the assets or balances to be made public, as well as any associated private data stored in memory slots.

    • Stage 2 (public state modification): After the UTXOs are consumed, the transaction details are made public by modifying the public state. This update includes changes +such as regulatory compliance reporting, where specific transaction details must be revealed to meet legal requirements, while other sensitive transactions remain private.

      +
      a) Stages of deshielded executions​
      +
        +
      • +

        Stage 1 (UTXO consumption): The process begins in the private state, where UTXOs are consumed as inputs for the transaction. This involves gathering all necessary +UTXOs that contain the assets or balances to be made public, as well as any associated private data stored in memory slots.

        +
        • +
      • +

        Stage 2 (public state modification): After the UTXOs are consumed, the transaction details are made public by modifying the public state. This update includes changes to the public balances, storage data, and any necessary public records. While the public state is updated, the sender’s identity and other sensitive information remain hidden, -thanks to the privacy-preserving properties of ZKPs.

      This model ensures that private data can be selectively revealed when needed, offering both flexibility and transparency. It is particularly useful for scenarios requiring -auditing or compliance reporting, where specific details must be made publicly verifiable without exposing the entire history or contents of private transactions.

      b) How deshielded executions work​

      The deshielded execution process starts when a user initiates a transaction using private UTXOs. The Nescience zkVM is called to generate a ZKP, -which validates the transaction without revealing sensitive details such as the sender's identity or the specifics of the Nescience application being executed.

      During the transaction, the UTXOs from the private state are consumed, meaning they are used up as inputs and will no longer be available for future transactions. +thanks to the privacy-preserving properties of ZKPs.

      +
      • +
    +

    This model ensures that private data can be selectively revealed when needed, offering both flexibility and transparency. It is particularly useful for scenarios requiring +auditing or compliance reporting, where specific details must be made publicly verifiable without exposing the entire history or contents of private transactions.

    +
    b) How deshielded executions work​
    +

    The deshielded execution process starts when a user initiates a transaction using private UTXOs. The Nescience zkVM is called to generate a ZKP, +which validates the transaction without revealing sensitive details such as the sender's identity or the specifics of the Nescience application being executed.

    +

    During the transaction, the UTXOs from the private state are consumed, meaning they are used up as inputs and will no longer be available for future transactions. Instead of generating new UTXOs, the transaction modifies the public state, updating the necessary balances or memory slots related to the transaction. -Here’s a step-by-step breakdown of how the deshielded execution process unfolds:

    1. Get receiver's public address: The sender first identifies the public address of the receiver, to which the information or assets will be made public.

    2. Determine input UTXOs and public state modifications: The sender gathers all the input UTXOs needed for the transaction and determines the public state modifications -necessary for the Nescience applications and token transfers involved.

    3. Calculate nullifiers: Nullifiers are generated for each input UTXO, ensuring that these UTXOs cannot be reused or double spent. The nullifiers are derived from the -corresponding UTXO commitments.

    4. Call zkVM with deshielded circuits: The sender invokes the zkVM with deshielded kernel circuits, which generates the proof. The proof ensures that all input UTXOs -are valid by verifying their membership in the UTXO tree and their non-membership in the nullifier tree, ensuring they haven’t been spent.

    5. Generate and submit proof: The zkVM generates a ZKP that verifies the correctness of the transaction without revealing private details. -The proof includes the nullifiers and the planned modifications to the public state.

    6. Send proof to sequencer: The sender then sends the proof and any relevant public information to the sequencer. The sequencer is responsible for verifying the proof, -updating the public state accordingly, and adding the nullifiers to the nullifier tree.

    Once the proof and public information have been broadcast to the network, the receiver does not need to take any further action. +Here’s a step-by-step breakdown of how the deshielded execution process unfolds:

    +
      +
    1. +

      Get receiver's public address: The sender first identifies the public address of the receiver, to which the information or assets will be made public.

      +
      2. +
    2. +

      Determine input UTXOs and public state modifications: The sender gathers all the input UTXOs needed for the transaction and determines the public state modifications +necessary for the Nescience applications and token transfers involved.

      +
      3. +
    3. +

      Calculate nullifiers: Nullifiers are generated for each input UTXO, ensuring that these UTXOs cannot be reused or double spent. The nullifiers are derived from the +corresponding UTXO commitments.

      +
      4. +
    4. +

      Call zkVM with deshielded circuits: The sender invokes the zkVM with deshielded kernel circuits, which generates the proof. The proof ensures that all input UTXOs +are valid by verifying their membership in the UTXO tree and their non-membership in the nullifier tree, ensuring they haven’t been spent.

      +
      5. +
    5. +

      Generate and submit proof: The zkVM generates a ZKP that verifies the correctness of the transaction without revealing private details. +The proof includes the nullifiers and the planned modifications to the public state.

      +
      6. +
    6. +

      Send proof to sequencer: The sender then sends the proof and any relevant public information to the sequencer. The sequencer is responsible for verifying the proof, +updating the public state accordingly, and adding the nullifiers to the nullifier tree.

      +
      7. +
    +

    Once the proof and public information have been broadcast to the network, the receiver does not need to take any further action. The sequencer manages the public state updates and ensures that the transaction is properly executed. By the end of the deshielded execution, specific transaction details become publicly visible, such as the identity of the receiver and the outcome of the transaction. This allows participants in the public state to extract information about the transaction, including the receiver's identity and some details about the execution. While the receiver's identity is revealed, the sender's identity and sensitive transaction details remain hidden, thanks to the use of ZKPs. -This makes deshielded executions ideal for cases where transparency is needed, but complete privacy is still a priority for certain elements of the transaction.

    Summary of UTXO consumption in NSSA​

    In NSSA, consuming UTXOs is a critical step in maintaining the security and integrity of the blockchain by preventing double spending. -When a UTXO is consumed, it is used as an input in a transaction, effectively marking it as spent. This ensures that the UTXO cannot be reused, preserving the integrity of the blockchain.

    1. The process of consuming UTXOs: The process of consuming a UTXO begins when a user selects a UTXO from their private state. The user verifies the UTXO’s existence and -ownership using their viewing key, ensuring that they are the legitimate owner of the UTXO. Once verified, the user generates two key cryptographic proofs:
      * **Membership proof:** This proof confirms that the UTXO exists within the hashed UTXO tree, ensuring its validity within the system.
      * **Non-membership proof:** This proof ensures that the UTXO has not been previously consumed by checking its absence in the nullifier tree, which tracks spent UTXOs.

    To mark the UTXO as spent, a nullifier is generated. This nullifier is a unique cryptographic hash derived from the UTXO, which is then added to the nullifier tree in the public state. -Adding the nullifier to the tree prevents the UTXO from being reused in future transactions, thus preventing double spending.

    After generating the membership and non-membership proofs, the user compiles the transaction using the zkVM. The zkVM is responsible for generating the necessary ZKPs, -which validate the transaction without revealing sensitive details. The compiled transaction, along with the proofs, is then submitted to the sequencer for verification.

    1. The role of the sequencer: Once the transaction is submitted, the sequencer verifies the ZKPs to confirm that the transaction is valid. If the proofs are verified +This makes deshielded executions ideal for cases where transparency is needed, but complete privacy is still a priority for certain elements of the transaction.

      +

      Summary of UTXO consumption in NSSA​

      +
      +

      In NSSA, consuming UTXOs is a critical step in maintaining the security and integrity of the blockchain by preventing double spending. +When a UTXO is consumed, it is used as an input in a transaction, effectively marking it as spent. This ensures that the UTXO cannot be reused, preserving the integrity of the blockchain.

      +
        +
      1. The process of consuming UTXOs: The process of consuming a UTXO begins when a user selects a UTXO from their private state. The user verifies the UTXO’s existence and +ownership using their viewing key, ensuring that they are the legitimate owner of the UTXO. Once verified, the user generates two key cryptographic proofs: +
          +
        • Membership proof: This proof confirms that the UTXO exists within the hashed UTXO tree, ensuring its validity within the system.
          • +
        • Non-membership proof: This proof ensures that the UTXO has not been previously consumed by checking its absence in the nullifier tree, which tracks spent UTXOs.
          • +
        +
        2. +
      +

      To mark the UTXO as spent, a nullifier is generated. This nullifier is a unique cryptographic hash derived from the UTXO, which is then added to the nullifier tree in the public state. +Adding the nullifier to the tree prevents the UTXO from being reused in future transactions, thus preventing double spending.

      +

      After generating the membership and non-membership proofs, the user compiles the transaction using the zkVM. The zkVM is responsible for generating the necessary ZKPs, +which validate the transaction without revealing sensitive details. The compiled transaction, along with the proofs, is then submitted to the sequencer for verification.

      +
        +
      1. The role of the sequencer: Once the transaction is submitted, the sequencer verifies the ZKPs to confirm that the transaction is valid. If the proofs are verified successfully, the sequencer updates both the private and public states to reflect the transaction. This includes updating the nullifier tree with the newly generated nullifier, -ensuring that the UTXO is marked as spent and cannot be reused.

      Example: Alice sending tokens to Bob​

      Consider an example where Alice wants to send 5 Nescience tokens to Bob using a private execution. Alice selects a UTXO from her private state that contains 5 Nescience tokens. +ensuring that the UTXO is marked as spent and cannot be reused.

      2. +
    +

    Example: Alice sending tokens to Bob​

    +
    +

    Consider an example where Alice wants to send 5 Nescience tokens to Bob using a private execution. Alice selects a UTXO from her private state that contains 5 Nescience tokens. She generates the necessary membership and non-membership proofs, ensuring that her UTXO exists in the system and has not been previously spent. Alice then creates a nullifier by -hashing the UTXO and compiles the transaction with the zkVM.

    Once Alice submits the transaction, the sequencer verifies the proofs and updates the blockchain by adding the nullifier to the nullifier tree and recording the transaction details. -This ensures that Alice’s UTXO is marked as spent and cannot be used again, while Bob receives the 5 tokens.

    The importance of nullifiers​

    Nullifiers are a key mechanism in preventing double spending. By marking consumed UTXOs as spent and tracking them in the nullifier tree, NSSA ensures that +hashing the UTXO and compiles the transaction with the zkVM.

    +

    Once Alice submits the transaction, the sequencer verifies the proofs and updates the blockchain by adding the nullifier to the nullifier tree and recording the transaction details. +This ensures that Alice’s UTXO is marked as spent and cannot be used again, while Bob receives the 5 tokens.

    +

    The importance of nullifiers​

    +
    +

    Nullifiers are a key mechanism in preventing double spending. By marking consumed UTXOs as spent and tracking them in the nullifier tree, NSSA ensures that once a UTXO is used in a transaction, it cannot be reused in any future transactions. This process is fundamental to maintaining the integrity and security of the blockchain, -as it guarantees that assets are only spent once and prevents potential attacks on the system.

    In conclusion, the process of consuming UTXOs in NSSA combines cryptographic proofs, nullifiers, and ZKPs to ensure that transactions -are secure, confidential, and free from the risks of double spending.

    C. Cryptographic primitives in NSSA

    In the NSSA, cryptographic primitives are the foundational elements that ensure the security, privacy, and efficiency of the state separation model. +as it guarantees that assets are only spent once and prevents potential attacks on the system.

    +

    In conclusion, the process of consuming UTXOs in NSSA combines cryptographic proofs, nullifiers, and ZKPs to ensure that transactions +are secure, confidential, and free from the risks of double spending.

    +

    C. Cryptographic primitives in NSSA

    +

    In the NSSA, cryptographic primitives are the foundational elements that ensure the security, privacy, and efficiency of the state separation model. These cryptographic tools enable private transactions, secure data management, and robust verification processes across both public and private states. The architecture leverages a wide range of cryptographic mechanisms, including advanced hash functions, key management systems, tree structures, and ZKPs, -to safeguard user data and maintain the integrity of transactions.

    Cryptographic hash functions play a pivotal role in concealing UTXO details, generating nullifiers, and constructing sparse Merkle trees, which organize and verify +to safeguard user data and maintain the integrity of transactions.

    +

    Cryptographic hash functions play a pivotal role in concealing UTXO details, generating nullifiers, and constructing sparse Merkle trees, which organize and verify data efficiently within the network. Key management and address generation further enhance the security of user assets and identity, ensuring that only authorized -users can access and control their holdings.

    The architecture also relies on specialized tree structures for organizing data, verifying the existence of UTXOs, and tracking nullifiers, which prevent double spending. +users can access and control their holdings.

    +

    The architecture also relies on specialized tree structures for organizing data, verifying the existence of UTXOs, and tracking nullifiers, which prevent double spending. Additionally, Nescience features a privacy-preserving zero-knowledge virtual machine (zk-zkVM), which allows users to prove the correctness of an execution without -disclosing sensitive information. This enables private transactions and maintains confidentiality across the network.

    As Nescience evolves, optional cryptographic mechanisms such as multi-party computation (MPC) may be integrated to enhance synchronization across privacy levels. +disclosing sensitive information. This enables private transactions and maintains confidentiality across the network.

    +

    As Nescience evolves, optional cryptographic mechanisms such as multi-party computation (MPC) may be integrated to enhance synchronization across privacy levels. This MPC-based synchronization mechanism is still under development and under review for potential inclusion in the system. Together, these cryptographic primitives -form the backbone of Nescience’s security architecture, ensuring that users can transact and interact privately, securely, and efficiently.

    In the following sections, we will explore each of these cryptographic components in detail, beginning with the role of hash functions.

    a) Hash functions in Nescience​

    Hash functions are a foundational element of Nescience’s cryptographic framework, serving multiple critical roles that ensure the security, privacy, and efficiency of the system. +form the backbone of Nescience’s security architecture, ensuring that users can transact and interact privately, securely, and efficiently.

    +

    In the following sections, we will explore each of these cryptographic components in detail, beginning with the role of hash functions.

    +

    a) Hash functions in Nescience​

    +
    +

    Hash functions are a foundational element of Nescience’s cryptographic framework, serving multiple critical roles that ensure the security, privacy, and efficiency of the system. One of the primary uses of hash functions in Nescience is to conceal sensitive details of UTXOs by converting them into fixed-size hashes. This process allows UTXO details to remain private, ensuring that sensitive information is not directly exposed on the blockchain, while still enabling their existence and integrity to be verified. Hashing -the UTXO details allows the actual data to remain confidential, with the hashes stored in a global tree structure for efficient management and retrieval.

    Additionally, hash functions are essential for generating nullifiers, which play a crucial role in preventing double spending. Nullifiers are created by hashing UTXOs +the UTXO details allows the actual data to remain confidential, with the hashes stored in a global tree structure for efficient management and retrieval.

    +

    Additionally, hash functions are essential for generating nullifiers, which play a crucial role in preventing double spending. Nullifiers are created by hashing UTXOs and are used to mark them as spent, ensuring that they cannot be reused in subsequent transactions. These nullifiers are stored in a nullifier tree, and each transaction must prove that its UTXO’s nullifier is not already present in the tree before it can be processed. This ensures that the UTXO has not been spent before, maintaining the -integrity of the transaction process.

    Hash functions are also vital in the construction of sparse Merkle trees, which provide an efficient and secure method for verifying data within the blockchain. +integrity of the transaction process.

    +

    Hash functions are also vital in the construction of sparse Merkle trees, which provide an efficient and secure method for verifying data within the blockchain. Sparse Merkle trees enable quick and reliable proofs of membership and non-membership, making them essential for verifying both UTXOs and nullifiers. By using hash functions -to build these trees, Nescience can ensure the integrity of the data, as any tampering with the data would result in a change in the hash, making the manipulation detectable.

    Another critical consideration in Nescience is the compatibility of hash functions with ZKPs. ZK-friendly hash functions are optimized for efficient +to build these trees, Nescience can ensure the integrity of the data, as any tampering with the data would result in a change in the hash, making the manipulation detectable.

    +

    Another critical consideration in Nescience is the compatibility of hash functions with ZKPs. ZK-friendly hash functions are optimized for efficient computation within the constraints of ZK circuits, ensuring that they do not become a bottleneck in the proof generation or verification process. These hash functions maintain strong cryptographic security properties while enabling efficient computations in ZKP systems, which is essential for maintaining privacy and -integrity within the ZK framework.

    The primary advantage of using hash functions in Nescience is their ability to ensure that transaction details remain private while still allowing for verification +integrity within the ZK framework.

    +

    The primary advantage of using hash functions in Nescience is their ability to ensure that transaction details remain private while still allowing for verification of their validity. Furthermore, by integrating hash functions into Merkle trees, the blockchain data becomes tamper-proof, enabling quick and efficient verification -processes that uphold the system’s security and privacy standards.

    Use case: How to use the Pedersen hash to create the UTXO commitment​

    As mentioned in the UTXOs in private executions section, the user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs +processes that uphold the system’s security and privacy standards.

    +

    Use case: How to use the Pedersen hash to create the UTXO commitment​

    +
    +

    As mentioned in the UTXOs in private executions section, the user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. The Pedersen hash is used to create the UTXO commitment. The Pedersen hash is a homomorphic commitment scheme that allows secure commitments -while maintaining privacy and enabling proofs of correctness in transactions. The commitment formula is as follows:

    Commitment=C(UTXO,RCK)=gUTXO⋅hRCKCommitment = C(UTXO,RCK) =g^{UTXO}⋅h^{RCK}

    In this formula, gg and hh are two generators of a cryptographic group where no known relationship exists between them. This ensures that the commitment is secure -and computationally infeasible to reverse or manipulate without knowing the original UTXO components. The random number RCKRCK adds an additional layer of security -by blinding the UTXO's contents, ensuring that the commitment doesn't leak any information about the underlying data.

    Importance of homomorphic commitments

    It is essential to use a homomorphic commitment like the Pedersen commitment for UTXOs because it allows for the verification of important properties in transactions, +while maintaining privacy and enabling proofs of correctness in transactions. The commitment formula is as follows:

    +

    Commitment=C(UTXO,RCK)=gUTXO⋅hRCKCommitment = C(UTXO,RCK) =g^{UTXO}⋅h^{RCK}

    +

    In this formula, gg and hh are two generators of a cryptographic group where no known relationship exists between them. This ensures that the commitment is secure +and computationally infeasible to reverse or manipulate without knowing the original UTXO components. The random number RCKRCK adds an additional layer of security +by blinding the UTXO's contents, ensuring that the commitment doesn't leak any information about the underlying data.

    +

    Importance of homomorphic commitments

    +

    It is essential to use a homomorphic commitment like the Pedersen commitment for UTXOs because it allows for the verification of important properties in transactions, such as ensuring that the total input value of a transaction equals the total output value. This balance is crucial for preventing the unauthorized creation of funds or d discrepancies in transactions. A homomorphic commitment enables these proofs because of its additive properties. Specifically, the exponents in the commitment formula are additive, -meaning that commitments can be combined and verified without revealing the individual components. For instance, if you have two UTXOs with commitments C(UTXO1,RCK1)C(UTXO_1,RCK_1) -and C(UTXO2,RCK2)C(UTXO_2,RCK_2), you can combine them and verify that the resulting commitment is valid without exposing the actual amounts.

    This capability is leveraged through a modified version of the Schnorr protocol, which is used in conjunction with the Pedersen hash to verify the correctness of transactions. -The Schnorr protocol allows users to prove, without revealing the actual values, that the sum of inputs equals the sum of outputs, ensuring that no funds are created or lost in the transaction.

    Limitations of standard cryptographic hashes

    Standard cryptographic hash functions, such as SHA-256, are not suitable for this purpose because they lack the algebraic structure needed for homomorphic properties. +meaning that commitments can be combined and verified without revealing the individual components. For instance, if you have two UTXOs with commitments C(UTXO1,RCK1)C(UTXO_1,RCK_1) +and C(UTXO2,RCK2)C(UTXO_2,RCK_2), you can combine them and verify that the resulting commitment is valid without exposing the actual amounts.

    +

    This capability is leveraged through a modified version of the Schnorr protocol, which is used in conjunction with the Pedersen hash to verify the correctness of transactions. +The Schnorr protocol allows users to prove, without revealing the actual values, that the sum of inputs equals the sum of outputs, ensuring that no funds are created or lost in the transaction.

    +

    Limitations of standard cryptographic hashes

    +

    Standard cryptographic hash functions, such as SHA-256, are not suitable for this purpose because they lack the algebraic structure needed for homomorphic properties. In particular, while SHA-256 provides strong security for general hashing purposes, it does not allow the additive properties that are required to perform the type of ZKPs used in Nescience for UTXO commitments. This is why the Pedersen hash is preferred, as it enables the secure and private execution of transactions -while allowing for balance verification and other critical proofs.

    Conclusion

    By using homomorphic commitments like the Pedersen hash, NSSA ensures that UTXOs can be securely committed and validated without exposing sensitive information. +while allowing for balance verification and other critical proofs.

    +

    Conclusion

    +

    By using homomorphic commitments like the Pedersen hash, NSSA ensures that UTXOs can be securely committed and validated without exposing sensitive information. The random component (RCK) adds an additional layer of security, and the additive properties of the Pedersen commitment enable powerful ZKPs that maintain the -integrity of the system.

    b) Key management and addresses in Nescience​

    NSSA utilizes different cryptographic schemes, such as public key encryption and digital signatures, to ensure secure private executions through +integrity of the system.

    +

    b) Key management and addresses in Nescience​

    +
    +

    NSSA utilizes different cryptographic schemes, such as public key encryption and digital signatures, to ensure secure private executions through the exchange of UTXOs. These schemes rely on a structured set of cryptographic keys, each serving a specific purpose in maintaining privacy, security, and control over assets. -Here's a breakdown of the keys used in Nescience:

    I. Spending key​

    The spending key is the fundamental secret key in NSSA, acting as the primary control mechanism for a user’s UTXOs and other digital assets. -It plays a critical role in the cryptographic security of the system, ensuring that only the rightful owner can authorize and spend their assets.

    • Role of the spending key: The spending key is responsible for generating the user’s private keys, which are used in various cryptographic operations such as +Here's a breakdown of the keys used in Nescience:

      +

      I. Spending key​

      +

      The spending key is the fundamental secret key in NSSA, acting as the primary control mechanism for a user’s UTXOs and other digital assets. +It plays a critical role in the cryptographic security of the system, ensuring that only the rightful owner can authorize and spend their assets.

      +
        +
      • +

        Role of the spending key: The spending key is responsible for generating the user’s private keys, which are used in various cryptographic operations such as signing transactions and creating commitments. This hierarchical relationship means that the spending key sits at the root of a user’s key structure, safeguarding access to all associated private keys and, consequently, to the user’s assets. In Nescience’s privacy-focused model, the spending key is never exposed or shared outside the user’s control. Unlike other keys, it does not interact with the public state, kernel circuits, or even the ZKP system. This isolation ensures that the spending key remains completely private and inaccessible to external entities. By keeping the spending key separate from the operational aspects of the network, -Nescience minimizes the risk of key leakage or compromise.

      • Generation and security of the spending key: The spending key is generated randomly from the scalar field, a large mathematical space that ensures uniqueness +Nescience minimizes the risk of key leakage or compromise.

        +
        • +
      • +

        Generation and security of the spending key: The spending key is generated randomly from the scalar field, a large mathematical space that ensures uniqueness and cryptographic strength. This randomness is crucial because it prevents attackers from predicting or replicating the key, thereby safeguarding the user’s assets from unauthorized access: it is computationally infeasible for an attacker to guess or brute-force the key. Once the spending key is generated, it is securely stored -by the user, typically in a hardware wallet or another secure storage mechanism that prevents unauthorized access.

      • Spending UTXOs with the spending key: The spending key’s primary function is to authorize the spending of UTXOs in private transactions. When a user initiates +by the user, typically in a hardware wallet or another secure storage mechanism that prevents unauthorized access.

        +
        • +
      • +

        Spending UTXOs with the spending key: The spending key’s primary function is to authorize the spending of UTXOs in private transactions. When a user initiates a transaction, the spending key is used to generate the necessary cryptographic proofs and signatures, ensuring that the transaction is valid and originates from the rightful owner. However, even though the spending key generates these proofs, it is never directly exposed during the transaction process. Instead, derived private keys handle the operational aspects while the spending key remains secure in the background. For example, when Alice decides to spend a UTXO in a private execution, her spending key generates the required private keys that will sign the transaction and ensure its validity. However, the spending key itself -never appears in any public state or transaction data, preserving its confidentiality.

      • Ensuring security through isolation: One of the key security principles of the spending key is its isolation from the network. Since it never interacts with +never appears in any public state or transaction data, preserving its confidentiality.

        +
        • +
      • +

        Ensuring security through isolation: One of the key security principles of the spending key is its isolation from the network. Since it never interacts with public-facing elements, such as the public state or kernel circuits, the risk of exposure is significantly reduced. This isolation ensures that even if other parts -of the cryptographic infrastructure are compromised, the spending key remains protected, preventing unauthorized spending of UTXOs.

      In summary, the spending key in Nescience is a powerful and carefully guarded element of the cryptographic system. It is the root key from which other private keys +of the cryptographic infrastructure are compromised, the spending key remains protected, preventing unauthorized spending of UTXOs.

      +
      • +
    +

    In summary, the spending key in Nescience is a powerful and carefully guarded element of the cryptographic system. It is the root key from which other private keys are derived, allowing users to spend their UTXOs securely and privately. Its isolation from the public state and its random generation from a secure scalar field ensures -that the spending key remains protected, making it a cornerstone of security in NSSA.

    II. Private keys​

    In Nescience, the private key is an essential cryptographic element responsible for facilitating various secure operations, such as generating commitments and signing +that the spending key remains protected, making it a cornerstone of security in NSSA.

    +

    II. Private keys​

    +

    In Nescience, the private key is an essential cryptographic element responsible for facilitating various secure operations, such as generating commitments and signing transactions. While the spending key plays a foundational role in safeguarding access to UTXOs and assets, the private keys handle the operational aspects of transactions -and cryptographic proofs. The private key consists of three critical components: privatekey.rsd{private}_{key}.rsd, privatekey.rcm{private}_{key}.rcm, and privatekey.sig{private}_{key}.sig, each serving a -distinct purpose within the Nescience cryptographic framework.

    1. privatekey.rsd{private}_{key}.rsd (random seed): The random seed (privatekey.rsd{private}_{key}.rsd) is the first and foundational component of the private key. It is a value randomly chosen from the scalar field, which ensures +and cryptographic proofs. The private key consists of three critical components: privatekey.rsd{private}_{key}.rsd, privatekey.rcm{private}_{key}.rcm, and privatekey.sig{private}_{key}.sig, each serving a +distinct purpose within the Nescience cryptographic framework.

      +
        +
      1. +

        privatekey.rsd{private}_{key}.rsd (random seed): The random seed (privatekey.rsd{private}_{key}.rsd) is the first and foundational component of the private key. It is a value randomly chosen from the scalar field, which ensures its cryptographic security and unpredictability. This seed is generated using a random number generator, making it virtually impossible to predict or replicate. The random seed is essential because it is used to derive the other two components of the private key. By leveraging a secure random seed, Nescience ensures that the entire private key structure is rooted in randomness, preventing external entities from guessing or deriving the key through brute-force attacks. -The strength of the random seed ensures the overall security of the private key and, consequently, the integrity of the user's transactions and commitments.

      2. privatekey.rcm{private}_{key}.rcm (random commitment): The random commitment component (privatekey.rcm{private}_{key}.rcm) is a crucial part of the private key used specifically in the commitment scheme. It acts as a blinding factor, -adding a layer of security to commitments made by the user. The privatekey.rcm{private}_{key}.rcm value is also drawn from the scalar field and is used to ensure that the commitment -to any UTXO or other sensitive data remains confidential. The commitment scheme in Nescience requires the use of privatekey.rcm{private}_{key}.rcm to create cryptographic commitments -that bind the user to specific data (such as UTXO details) without revealing the actual data. The role of privatekey.rcm{private}_{key}.rcm is to ensure that these commitments are -non-malleable and secure, preventing anyone from modifying the committed data without detection. For instance, when Alice commits to a UTXO, privatekey.rcm{private}_{key}.rcm is used +The strength of the random seed ensures the overall security of the private key and, consequently, the integrity of the user's transactions and commitments.

        +
        3. +
      3. +

        privatekey.rcm{private}_{key}.rcm (random commitment): The random commitment component (privatekey.rcm{private}_{key}.rcm) is a crucial part of the private key used specifically in the commitment scheme. It acts as a blinding factor, +adding a layer of security to commitments made by the user. The privatekey.rcm{private}_{key}.rcm value is also drawn from the scalar field and is used to ensure that the commitment +to any UTXO or other sensitive data remains confidential. The commitment scheme in Nescience requires the use of privatekey.rcm{private}_{key}.rcm to create cryptographic commitments +that bind the user to specific data (such as UTXO details) without revealing the actual data. The role of privatekey.rcm{private}_{key}.rcm is to ensure that these commitments are +non-malleable and secure, preventing anyone from modifying the committed data without detection. For instance, when Alice commits to a UTXO, privatekey.rcm{private}_{key}.rcm is used to generate a Pedersen commitment that ensures the UTXO details are hidden but can still be verified cryptographically. This means that even though the actual UTXO details -are concealed, their existence and integrity can be proven.

      4. privatekey.sig{private}_{key}.sig (signing key for transactions): The signing key (privatekey.sig{private}_{key}.sig) is the third and final component of the private key, used primarily for signing transactions. One possible approach is that -Nescience employs Schnorr signatures, a cryptographic protocol known for its efficiency and security. In this case, the privatekey.sig{private}_{key}.sig component would generate +are concealed, their existence and integrity can be proven.

        +
        5. +
      5. +

        privatekey.sig{private}_{key}.sig (signing key for transactions): The signing key (privatekey.sig{private}_{key}.sig) is the third and final component of the private key, used primarily for signing transactions. One possible approach is that +Nescience employs Schnorr signatures, a cryptographic protocol known for its efficiency and security. In this case, the privatekey.sig{private}_{key}.sig component would generate Schnorr signatures that are used to authenticate transactions, ensuring that only the rightful owner of the private key can authorize the spending of UTXOs. Schnorr signatures are important as they provide a secure and non-repudiable method of verifying that a transaction was initiated by the legitimate owner of the assets. -When Alice signs a transaction using her privatekey.sig{private}_{key}.sig, the corresponding public key allows others to verify that the transaction was indeed signed by Alice, +When Alice signs a transaction using her privatekey.sig{private}_{key}.sig, the corresponding public key allows others to verify that the transaction was indeed signed by Alice, without revealing her private key. This verification process ensures that all transactions are legitimate and prevents unauthorized entities from forging transactions -or spending assets they do not control. Even if an attacker gains access to the signed transaction, they cannot reverse engineer the privatekey.sig{private}_{key}.sig, ensuring -the security of Alice's future transactions.

      Robustness of private keys in Nescience

      Despite the critical role of the private key in the operation of NSSA, the system is designed to maintain security even in the event that the +or spending assets they do not control. Even if an attacker gains access to the signed transaction, they cannot reverse engineer the privatekey.sig{private}_{key}.sig, ensuring +the security of Alice's future transactions.

      +
      2. +
    +

    Robustness of private keys in Nescience

    +

    Despite the critical role of the private key in the operation of NSSA, the system is designed to maintain security even in the event that the private key is compromised. This resilience is achieved through the integrity of the spending key, which is never exposed in the process of signing or committing. The spending key acts as the ultimate safeguard, ensuring that even if a private key component is compromised, the attacker cannot access or spend the user's assets -without control over the spending key.

    The architecture’s design, where private keys handle operational tasks but rely on the spending key for ultimate control, ensures a layered approach to security. -This way, the system can mitigate the damage of a compromised private key by maintaining the inviolability of the user's assets.

    Conclusion

    In summary, the private key in Nescience consists of three interrelated components that together ensure secure transaction signing, commitment creation, and the -protection of user data. The privatekey.rsd{private}_{key}.rsd serves as the root from which the other key components are derived, ensuring randomness and security. -The privatekey.rcm{private}_{key}.rcm plays a crucial role in generating commitments, while privatekey.sig{private}_{key}.sig provides the signing capability needed for transaction authentication. -Together, these components enable users to engage in private, secure transactions while preserving the integrity of their assets, even in the face of potential key compromise.

    III. Public keys​

    Public keys in Nescience serve as the user's interface with the network, allowing for secure interaction and verification without exposing the user's private keys. +without control over the spending key.

    +

    The architecture’s design, where private keys handle operational tasks but rely on the spending key for ultimate control, ensures a layered approach to security. +This way, the system can mitigate the damage of a compromised private key by maintaining the inviolability of the user's assets.

    +

    Conclusion

    +

    In summary, the private key in Nescience consists of three interrelated components that together ensure secure transaction signing, commitment creation, and the +protection of user data. The privatekey.rsd{private}_{key}.rsd serves as the root from which the other key components are derived, ensuring randomness and security. +The privatekey.rcm{private}_{key}.rcm plays a crucial role in generating commitments, while privatekey.sig{private}_{key}.sig provides the signing capability needed for transaction authentication. +Together, these components enable users to engage in private, secure transactions while preserving the integrity of their assets, even in the face of potential key compromise.

    +

    III. Public keys​

    +

    Public keys in Nescience serve as the user's interface with the network, allowing for secure interaction and verification without exposing the user's private keys. Derived directly from the user's private keys, public keys play a crucial role in enabling cryptographic operations such as transaction verification, commitment schemes, and deterministic computations. The public key components correspond to their private key counterparts and ensure that transactions and commitments are securely processed -and validated across the network.

    1. publickey.sig{public}_{key}.sig (verifying Schnorr signatures):

    The publickey.sig{public}_{key}.sig is derived from the signing component of the private key (privatekey.sig{private}_{key}.sig) and is used for verifying Schnorr signatures. +and validated across the network.

    +
      +
    1. publickey.sig{public}_{key}.sig (verifying Schnorr signatures):
      2. +
    +

    The publickey.sig{public}_{key}.sig is derived from the signing component of the private key (privatekey.sig{private}_{key}.sig) and is used for verifying Schnorr signatures. Schnorr signatures are used to authenticate transactions, ensuring that they have been signed by the legitimate owner of the private key. This public key is essentially a verification key, allowing others in the network to confirm that a specific transaction was indeed authorized by the user. When a transaction is -broadcast to the network, publickey.sig{public}_{key}.sig enables any participant to verify that the transaction’s signature matches the user’s private key without +broadcast to the network, publickey.sig{public}_{key}.sig enables any participant to verify that the transaction’s signature matches the user’s private key without needing access to the private key itself. This mechanism prevents forgeries as only the legitimate owner with access to the private key can generate a valid Schnorr signature. -For example, if Alice sends a transaction, she signs it with her private key (privatekey.sig{private}_{key}.sig). Bob, or any other network participant, can use Alice’s publickey.sig{public}_{key}.sig -to verify the signature. If the signature is valid, Bob can be confident that the transaction was authorized by Alice and not by an imposter.

    1. publickey.rcm{public}_{key}.rcm (commitment schemes)

    The publickey.rcm{public}_{key}.rcm is derived from the commitment component of the private key (privatekey.rcm{private}_{key}.rcm). It is used in the commitment schemes +For example, if Alice sends a transaction, she signs it with her private key (privatekey.sig{private}_{key}.sig). Bob, or any other network participant, can use Alice’s publickey.sig{public}_{key}.sig +to verify the signature. If the signature is valid, Bob can be confident that the transaction was authorized by Alice and not by an imposter.

    +
      +
    1. publickey.rcm{public}_{key}.rcm (commitment schemes)
      2. +
    +

    The publickey.rcm{public}_{key}.rcm is derived from the commitment component of the private key (privatekey.rcm{private}_{key}.rcm). It is used in the commitment schemes that underpin Nescience’s privacy-preserving architecture. Commitments are a crucial cryptographic technique that allows users to commit to a piece of data (such as a UTXO) -without revealing the actual data, while still enabling proof of its integrity and existence. In Nescience, the publickey.rcm{public}_{key}.rcm is used as part of the Pedersen commitment scheme, +without revealing the actual data, while still enabling proof of its integrity and existence. In Nescience, the publickey.rcm{public}_{key}.rcm is used as part of the Pedersen commitment scheme, where it functions as a public commitment to certain transaction details. Even though the actual values are hidden (thanks to the private key component), the commitment can -still be verified by other network participants using publickey.rcm{public}_{key}.rcm. This enables secure and private transactions while maintaining the ability to verify that commitments -are consistent with the original data. For instance, when Alice commits to a UTXO, she uses her private key to generate the commitment, and the publickey.rcm{public}_{key}.rcm is available -to others to verify the commitment’s validity without revealing the underlying details.

    1. publickey.sk(prf){public}_{key}.sk(prf) (pseudorandom function)

    The publickey.sk(prf){public}_{key}.sk(prf) is derived from a random field element within the private key and is used to generate the pseudorandom function (PRF) associated with the user's account. +still be verified by other network participants using publickey.rcm{public}_{key}.rcm. This enables secure and private transactions while maintaining the ability to verify that commitments +are consistent with the original data. For instance, when Alice commits to a UTXO, she uses her private key to generate the commitment, and the publickey.rcm{public}_{key}.rcm is available +to others to verify the commitment’s validity without revealing the underlying details.

    +
      +
    1. publickey.sk(prf){public}_{key}.sk(prf) (pseudorandom function)
      2. +
    +

    The publickey.sk(prf){public}_{key}.sk(prf) is derived from a random field element within the private key and is used to generate the pseudorandom function (PRF) associated with the user's account. This PRF is essential for producing deterministic outputs based on the user’s keys and transaction data while ensuring that these outputs are unique to the user and cannot be predicted or replicated by others. The PRF is crucial in scenarios where the user needs to derive unique identifiers or values that are tied to their specific account, -ensuring that these values remain consistent across different transactions or interactions without revealing sensitive information. For example, publickey.sk(prf){public}_{key}.sk(prf) may be -used in generating deterministic yet secure addresses or transaction references, which can be linked to the user’s activity in a controlled manner. By using publickey.sk(prf){public}_{key}.sk(prf), +ensuring that these values remain consistent across different transactions or interactions without revealing sensitive information. For example, publickey.sk(prf){public}_{key}.sk(prf) may be +used in generating deterministic yet secure addresses or transaction references, which can be linked to the user’s activity in a controlled manner. By using publickey.sk(prf){public}_{key}.sk(prf), Nescience ensures that certain operations, like generating addresses or computing deterministic transaction outcomes, remain both private and cryptographically secure. The public key’s -role in this process is to maintain consistency in these outputs while preventing unauthorized parties from reverse engineering the associated private keys or transaction data.

    Summary

    Public keys in Nescience are essential for secure interactions within the network. "publickey.sig{public}_{key}.sig" allows others to verify that transactions were signed by the legitimate owner, -ensuring the authenticity of every operation. "publickey.rcm{public}_{key}.rcm" enables secure and private commitment schemes, allowing participants to commit to transaction details without -revealing sensitive information. Finally, "publickey.sk(prf){public}_{key}.sk(prf)" powers deterministic outputs through a pseudorandom function, ensuring that user-specific data remains consistent -and secure throughout various transactions. Together, these public key components facilitate privacy, security, and trust within NSSA, enabling seamless interactions while safeguarding user data.

    IV. Viewing key​

    The viewing key in NSSA is a specialized cryptographic key that allows a user to decrypt both incoming and outgoing transactions associated with their account. +role in this process is to maintain consistency in these outputs while preventing unauthorized parties from reverse engineering the associated private keys or transaction data.

    +

    Summary

    +

    Public keys in Nescience are essential for secure interactions within the network. "publickey.sig{public}_{key}.sig" allows others to verify that transactions were signed by the legitimate owner, +ensuring the authenticity of every operation. "publickey.rcm{public}_{key}.rcm" enables secure and private commitment schemes, allowing participants to commit to transaction details without +revealing sensitive information. Finally, "publickey.sk(prf){public}_{key}.sk(prf)" powers deterministic outputs through a pseudorandom function, ensuring that user-specific data remains consistent +and secure throughout various transactions. Together, these public key components facilitate privacy, security, and trust within NSSA, enabling seamless interactions while safeguarding user data.

    +

    IV. Viewing key​

    +

    The viewing key in NSSA is a specialized cryptographic key that allows a user to decrypt both incoming and outgoing transactions associated with their account. This key is designed to offer a degree of transparency to the user, enabling them to view the details of their transactions without compromising the security of their assets or granting -control over those assets.

    • Role of the viewing key: The primary function of the viewing key is to provide visibility into transaction details while maintaining the integrity of private, shielded, +control over those assets.

      +
        +
      • +

        Role of the viewing key: The primary function of the viewing key is to provide visibility into transaction details while maintaining the integrity of private, shielded, or deshielded transactions. It enables the user to see the specifics of the transactions they are involved in—such as amounts transferred, asset types, and metadata—without exposing the sensitive transaction data to the broader network. For instance, if Alice has executed a private transaction with Bob, her viewing key allows her to decrypt and review the details of the transaction, ensuring that everything was processed correctly. This ability to audit her own transactions helps Alice maintain confidence in the integrity -of her private interactions on the blockchain.

      • Security considerations: Despite its utility, the viewing key must be handled with care as its exposure could potentially compromise the user’s privacy. +of her private interactions on the blockchain.

        +
        • +
      • +

        Security considerations: Despite its utility, the viewing key must be handled with care as its exposure could potentially compromise the user’s privacy. Although possessing the viewing key does not provide the ability to spend or sign transactions (that authority remains strictly with the spending key and private keys), it does allow anyone with access to the viewing key to decrypt the details of the user’s private transactions. This means that if the viewing key is leaked or stolen, the privacy guarantees of Nescience’s private, shielded, and deshielded executions could be undermined. Specifically, the viewing key could be used to link various transactions, breaking the unlinkability of private transactions. For example, an attacker with access to the viewing key could decrypt past and future transactions, exposing the relationships between different parties and transaction flows. To mitigate this risk, Nescience recommends that users treat their viewing key with the same level of protection as their private keys. -It should be stored securely in encrypted hardware wallets or other secure storage solutions to prevent unauthorized access.

      • Balancing privacy and transparency: The viewing key provides an essential balance between privacy and transparency in NSSA. While it ensures that users +It should be stored securely in encrypted hardware wallets or other secure storage solutions to prevent unauthorized access.

        +
        • +
      • +

        Balancing privacy and transparency: The viewing key provides an essential balance between privacy and transparency in NSSA. While it ensures that users can monitor their transaction history and verify the details of their private transactions, it does so without compromising the control of their funds. This allows users to maintain a transparent view of their interactions while keeping their assets secure. For example, if Alice is using shielded execution to transfer assets, her viewing key enables her to audit the transaction without allowing anyone else, including Bob or external observers, to see the specific details unless they also have access to the viewing key. Moreover, since the viewing key does not grant signing or spending authority, even if it were exposed, an attacker would still not be able to manipulate the user’s assets. However, -to maintain the unlinkability and confidentiality of private transactions, the viewing key must be kept secure at all times.

      • Protecting transaction unlinkability: In private transactions, unlinkability is one of the core privacy guarantees. This property ensures that individual +to maintain the unlinkability and confidentiality of private transactions, the viewing key must be kept secure at all times.

        +
        • +
      • +

        Protecting transaction unlinkability: In private transactions, unlinkability is one of the core privacy guarantees. This property ensures that individual transactions cannot be correlated with each other or linked to the same user unless that user chooses to reveal the connection. The viewing key must be carefully protected to preserve this unlinkability, as its compromise could allow someone to map out a user’s private transaction history. For instance, in deshielded transactions, the viewing key allows the user to see which private UTXOs were consumed and how the public state was modified. If the viewing key is compromised, an attacker could potentially -link private UTXOs across multiple transactions, unraveling the user’s privacy.

      Conclusion

      The viewing key in Nescience is a powerful tool for providing insight into both incoming and outgoing transactions without granting control over assets. It allows users +link private UTXOs across multiple transactions, unraveling the user’s privacy.

      +
      • +
    +

    Conclusion

    +

    The viewing key in Nescience is a powerful tool for providing insight into both incoming and outgoing transactions without granting control over assets. It allows users to decrypt and verify their transaction details, maintaining transparency in their interactions. However, due to its potential to compromise privacy if exposed, the viewing key must be handled with great care. Proper security measures are necessary to protect the viewing key, ensuring that the unlinkability of private, shielded, and deshielded -transactions remains intact. In this way, the viewing key offers a crucial balance between privacy and transparency within the Nescience ecosystem.

    V. Ephemeral key​

    The ephemeral key is generated using a combination of the sender’s spending key and the UTXO's nullifier, ensuring that the key is unique to each transaction. -The process can be informally described as follows:

    1. Ephemeral key generation
      -Let ρ\rho denote the nullifier of the UTXO being consumed in the transaction. The sender uses the receiver’s public key component publickey.sk(prf){public}_{key}.sk(prf), -which is derived from the receiver’s private key, to compute an ephemeral secret key (eskesk). The computation is based on the nullifier ρ\rho and a base value:

      esk=publickey.sk(prf((0,0,0,0)∣∣ρ)esk = {public}_{key}.sk(prf((0,0,0,0) || \rho) -This formula binds the secret key to the specific transaction, leveraging the receiver’s cryptographic identity and the unique properties of the UTXO being spent.

    2. Deriving the ephemeral public key
      -After computing the ephemeral secret key (eskesk), the next step is to derive the corresponding ephemeral public key (epk). This is done using the Key Agreement -Protocol's DerivePublic algorithm, which generates the public key associated with the shared secret key. The ephemeral public key is computed as:

      epk=KA.DerivePublic(esk,gd)epk = KA.DerivePublic(esk, gd)

    Here, (gdgd) is the diversifier address associated with the receiver’s account. The diversifier address is computed from the receiver’s -account using the DiversifierHash function:

    gd=receiver.DiversifierHash(d)gd = receiver.DiversifierHash(d)

    The diversifier (dd) is a random value selected by the sender to add randomness to the process. This diversifier ensures that even if a single receiver is involved -in multiple transactions, the derived keys remain distinct for each transaction. The value (dd) is included in the transaction note for transparency and reproducibility.

    1. Establishing the shared secret
      The shared secret, used to encrypt the transaction details, is derived from the key agreement between the sender’s ephemeral key and the receiver’s viewing key. +transactions remains intact. In this way, the viewing key offers a crucial balance between privacy and transparency within the Nescience ecosystem.

      +

      V. Ephemeral key​

      +

      The ephemeral key is generated using a combination of the sender’s spending key and the UTXO's nullifier, ensuring that the key is unique to each transaction. +The process can be informally described as follows:

      +
        +
      1. Ephemeral key generation
        +Let ρ\rho denote the nullifier of the UTXO being consumed in the transaction. The sender uses the receiver’s public key component publickey.sk(prf){public}_{key}.sk(prf), +which is derived from the receiver’s private key, to compute an ephemeral secret key (eskesk). The computation is based on the nullifier ρ\rho and a base value:
        2. +
      +

      esk=publickey.sk(prf((0,0,0,0)∣∣ρ)esk = {public}_{key}.sk(prf((0,0,0,0) || \rho) +This formula binds the secret key to the specific transaction, leveraging the receiver’s cryptographic identity and the unique properties of the UTXO being spent.

      +
        +
      1. Deriving the ephemeral public key
        +After computing the ephemeral secret key (eskesk), the next step is to derive the corresponding ephemeral public key (epk). This is done using the Key Agreement +Protocol's DerivePublic algorithm, which generates the public key associated with the shared secret key. The ephemeral public key is computed as:
        2. +
      +

      epk=KA.DerivePublic(esk,gd)epk = KA.DerivePublic(esk, gd)

      +

      Here, (gdgd) is the diversifier address associated with the receiver’s account. The diversifier address is computed from the receiver’s +account using the DiversifierHash function:

      +

      gd=receiver.DiversifierHash(d)gd = receiver.DiversifierHash(d)

      +

      The diversifier (dd) is a random value selected by the sender to add randomness to the process. This diversifier ensures that even if a single receiver is involved +in multiple transactions, the derived keys remain distinct for each transaction. The value (dd) is included in the transaction note for transparency and reproducibility.

      +
        +
      1. Establishing the shared secret
        +The shared secret, used to encrypt the transaction details, is derived from the key agreement between the sender’s ephemeral key and the receiver’s viewing key. Any party possessing the receiver’s viewing key can use it in conjunction with the ephemeral key to compute the shared secret, which is then used to decrypt the transaction. -This ensures that only the intended recipient (or anyone with their viewing key) can access the transaction details.

      Key components and protocol

      The formal protocol for generating ephemeral keys closely follows this informal description but involves additional intermediate steps for converting values to +This ensures that only the intended recipient (or anyone with their viewing key) can access the transaction details.

      2. +
    +

    Key components and protocol

    +

    The formal protocol for generating ephemeral keys closely follows this informal description but involves additional intermediate steps for converting values to binary sequences to fit implementation requirements. These steps are essential for ensuring compatibility with cryptographic algorithms used in NSSA. -The protocol uses the following key components:

    • Nullifier (ρ\rho): Ensures that the ephemeral key is tied to the specific UTXO being consumed, preventing reuse of the key in future transactions.
    • Receiver’s public key (publickey.sk(prf){public}_{key}.sk(prf): Establishes the receiver's identity in the key generation process, ensuring that the shared secret can -only be derived by the intended party.
    • Diversifier (dd): Adds randomness to the transaction, ensuring that keys remain unique across different transactions involving the same receiver.

    The end result is an ephemeral key system that provides strong cryptographic guarantees for transaction privacy, leveraging key agreement protocols and secure -cryptographic primitives to prevent unauthorized access to sensitive transaction data.

    Conclusion

    The ephemeral key in Nescience is a critical element for maintaining transaction confidentiality. It facilitates a secure key agreement between the sender and the receiver, +The protocol uses the following key components:

    +
      +
    • Nullifier (ρ\rho): Ensures that the ephemeral key is tied to the specific UTXO being consumed, preventing reuse of the key in future transactions.
      • +
    • Receiver’s public key (publickey.sk(prf){public}_{key}.sk(prf): Establishes the receiver's identity in the key generation process, ensuring that the shared secret can +only be derived by the intended party.
      • +
    • Diversifier (dd): Adds randomness to the transaction, ensuring that keys remain unique across different transactions involving the same receiver.
      • +
    +

    The end result is an ephemeral key system that provides strong cryptographic guarantees for transaction privacy, leveraging key agreement protocols and secure +cryptographic primitives to prevent unauthorized access to sensitive transaction data.

    +

    Conclusion

    +

    The ephemeral key in Nescience is a critical element for maintaining transaction confidentiality. It facilitates a secure key agreement between the sender and the receiver, allowing for the encryption of transaction details with a shared secret that can only be derived by the intended recipient. By incorporating the nullifier, receiver's public key, and diversifier address, the ephemeral key ensures that transaction privacy is preserved while preventing unauthorized access to transaction information, even in a complex, -multi-party blockchain environment.

    VI. Nescience addresses​

    Nescience’s dual address system is a core component of its privacy-focused architecture, designed to balance transparency and confidentiality across different types of transactions. -The architecture provides each user or smart contract with both public addresses and private addresses, allowing them to participate in both open and confidential activities on the blockchain.

    a) Public addresses​

    Public addresses in Nescience are visible to all participants on the network and reside within the public state. These addresses are essential for engaging in +multi-party blockchain environment.

    +

    VI. Nescience addresses​

    +

    Nescience’s dual address system is a core component of its privacy-focused architecture, designed to balance transparency and confidentiality across different types of transactions. +The architecture provides each user or smart contract with both public addresses and private addresses, allowing them to participate in both open and confidential activities on the blockchain.

    +

    a) Public addresses​

    +

    Public addresses in Nescience are visible to all participants on the network and reside within the public state. These addresses are essential for engaging in transparent and verifiable interactions, such as sending tokens or invoking smart contracts that are meant to be publicly auditable. Public addresses serve as -the interface for users who need to engage with the transparent elements of the system, including public transactions or smart contracts that require public access.

    They are analogous to traditional blockchain addresses seen in systems like Ethereum or Bitcoin, where every participant can see the address and the transactions associated with it. +the interface for users who need to engage with the transparent elements of the system, including public transactions or smart contracts that require public access.

    +

    They are analogous to traditional blockchain addresses seen in systems like Ethereum or Bitcoin, where every participant can see the address and the transactions associated with it. For example, when Alice wants to receive tokens from Bob in a public transaction, she can provide her public address, allowing Bob to send the tokens transparently. -Anyone on the network can verify the transaction, providing accountability and trust in the public state.

    Because public addresses are visible and auditable, they are typically used for interactions where privacy is not a concern or where transparency is desirable. +Anyone on the network can verify the transaction, providing accountability and trust in the public state.

    +

    Because public addresses are visible and auditable, they are typically used for interactions where privacy is not a concern or where transparency is desirable. This could include simple token transfers, public contract calls, or interactions with dapps that require public accountability, -such as voting or governance systems.

    b) Private addresses​

    In contrast, private addresses are designed for confidentiality and are not visible onchain. These addresses are used exclusively for private transactions and executions, +such as voting or governance systems.

    +

    b) Private addresses​

    +

    In contrast, private addresses are designed for confidentiality and are not visible onchain. These addresses are used exclusively for private transactions and executions, ensuring that sensitive details—such as the sender, receiver, or amount transferred—remain hidden from the public state. Private addresses are a key feature of -Nescience’s private, shielded, and deshielded execution models, where preserving the confidentiality of participants is crucial.

    Users can generate an unlimited number of private addresses using their private keys. This flexibility allows users to compartmentalize their interactions, +Nescience’s private, shielded, and deshielded execution models, where preserving the confidentiality of participants is crucial.

    +

    Users can generate an unlimited number of private addresses using their private keys. This flexibility allows users to compartmentalize their interactions, giving them the ability to provide different private addresses to different parties. For instance, Alice could create a unique private address for each entity she interacts with, thereby ensuring that her transactions remain isolated and difficult to trace. This feature enhances privacy by preventing any direct linkage -between different transactions or activities associated with a single user.

    Private addresses are not tied to the public state and are only accessible through the user’s private key infrastructure. Transactions involving private addresses +between different transactions or activities associated with a single user.

    +

    Private addresses are not tied to the public state and are only accessible through the user’s private key infrastructure. Transactions involving private addresses are conducted within the confines of the private state and are only decrypted by the intended participants. For example, when Alice sends tokens to Bob using -a private address, the details of that transaction remain confidential, accessible only to Alice and Bob, unless they choose to reveal it.

    Role of the viewing key in private addresses: A key feature of Nescience’s private address system is the viewing key, which allows users to decrypt any transaction +a private address, the details of that transaction remain confidential, accessible only to Alice and Bob, unless they choose to reveal it.

    +

    Role of the viewing key in private addresses: A key feature of Nescience’s private address system is the viewing key, which allows users to decrypt any transaction involving their private addresses. This capability provides oversight and transparency into the user’s private transactions, ensuring that they can monitor their own activity without exposing the details to the public. The viewing key does not compromise the security of the user's assets as it does not grant spending or signing authority. However, it does allow the user to audit and verify the accuracy of their private transactions, ensuring that everything proceeds as expected. For instance, Alice can use her viewing key to review the details of a private transaction she conducted with Bob, ensuring that the correct amount was transferred and that the transaction was properly processed. This functionality is critical for users who want to maintain control over their private interactions while still benefiting from transparency into their transaction history. -The ability to generate multiple private addresses and decrypt them with the viewing key ensures that users can maintain compartmentalized privacy without sacrificing oversight.

    Summary

    Nescience’s dual address system—comprising public and private addresses—provides users with the flexibility to engage in both transparent and confidential transactions. +The ability to generate multiple private addresses and decrypt them with the viewing key ensures that users can maintain compartmentalized privacy without sacrificing oversight.

    +

    Summary

    +

    Nescience’s dual address system—comprising public and private addresses—provides users with the flexibility to engage in both transparent and confidential transactions. Public addresses are visible onchain and are used for open, public interactions that require accountability and auditability. In contrast, private addresses are -invisible onchain and are used for confidential transactions, enhancing privacy and security.

    By allowing users to generate multiple private addresses, Nescience gives individuals control over the visibility of their transactions. Combined with the viewing +invisible onchain and are used for confidential transactions, enhancing privacy and security.

    +

    By allowing users to generate multiple private addresses, Nescience gives individuals control over the visibility of their transactions. Combined with the viewing key’s ability to decrypt transactions involving private addresses, the system ensures that users can maintain transparency over their private transactions without exposing sensitive information to the public state. This dual-address approach enables users to seamlessly switch between public and private interactions depending on their needs, -providing a robust framework for both privacy and transparency in NSSA.

    VII. Conclusion​

    Key management in NSSA is a carefully designed system that strikes an optimal balance between security, privacy, and flexibility. +providing a robust framework for both privacy and transparency in NSSA.

    +

    VII. Conclusion​

    +

    Key management in NSSA is a carefully designed system that strikes an optimal balance between security, privacy, and flexibility. The architecture’s hierarchical structure, with distinct roles for the spending key, private keys, and public keys, ensures that users retain full control over their assets while maintaining the integrity of their transactions. The spending key, as the root of security, provides unassailable control over the user's UTXOs and assets, ensuring that only the rightful owner can authorize spending. Private keys, derived from the spending key, enable users to engage -in cryptographic operations such as signing transactions and generating commitments without exposing sensitive information to the network.

    The viewing key adds another layer of transparency, allowing users to decrypt and review their transactions without compromising their authority over their assets. -While it provides a window into transaction history, the viewing key does not grant spending power, preserving the critical separation between visibility and control.

    The dual system of public and private addresses gives users the flexibility to navigate between open, transparent transactions and confidential, privacy-protected activities. +in cryptographic operations such as signing transactions and generating commitments without exposing sensitive information to the network.

    +

    The viewing key adds another layer of transparency, allowing users to decrypt and review their transactions without compromising their authority over their assets. +While it provides a window into transaction history, the viewing key does not grant spending power, preserving the critical separation between visibility and control.

    +

    The dual system of public and private addresses gives users the flexibility to navigate between open, transparent transactions and confidential, privacy-protected activities. Public addresses allow users to engage in verifiable, public interactions while private addresses enable compartmentalized, secure transactions that remain hidden from the public eye. This dual-address framework ensures that users can seamlessly adapt to different privacy requirements, whether they are participating in public -dapps or conducting sensitive financial operations.

    Overall, Nescience’s cryptographic infrastructure is designed to empower users to engage confidently in both transparent and confidential activities. +dapps or conducting sensitive financial operations.

    +

    Overall, Nescience’s cryptographic infrastructure is designed to empower users to engage confidently in both transparent and confidential activities. By providing flexible, secure key management and address systems, Nescience ensures that users can fully participate in the blockchain ecosystem without compromising their privacy or control. The architecture supports the nuanced needs of modern blockchain users, who require both the transparency of public -interactions and the security of private transactions, all while maintaining the integrity and confidentiality of their assets.

    c) Trees in NSSA​

    Trees in NSSA serve as verifiable databases, essential for maintaining privacy and security. Different types of trees are used for various purposes:

    1. Global state tree: The global state tree is a single, public tree that holds all public assets and storage information. It acts as a central repository for all -publicly accessible data on the blockchain. By organizing this data in a Merkle tree structure, the Global State Tree allows for efficient and secure verification of public information.

    2. Hashed UTXO tree: The hashed UTXO tree is a public tree that contains hashes of all created UTXOs. When users wish to consume a UTXO, they provide a membership +interactions and the security of private transactions, all while maintaining the integrity and confidentiality of their assets.

      +

      c) Trees in NSSA​

      +

      Trees in NSSA serve as verifiable databases, essential for maintaining privacy and security. Different types of trees are used for various purposes:

      +
        +
      1. +

        Global state tree: The global state tree is a single, public tree that holds all public assets and storage information. It acts as a central repository for all +publicly accessible data on the blockchain. By organizing this data in a Merkle tree structure, the Global State Tree allows for efficient and secure verification of public information.

        +
        2. +
      2. +

        Hashed UTXO tree: The hashed UTXO tree is a public tree that contains hashes of all created UTXOs. When users wish to consume a UTXO, they provide a membership proof to demonstrate that the UTXO exists within this tree. This process ensures that only valid and existing UTXOs can be spent, maintaining the integrity of transactions. In fact, users generate membership proofs that verify the presence of specific UTXOs in the tree without revealing their actual data. The benefit here is that the Merkle -tree structure allows for quick and efficient verification of UTXO existence.

      3. UTXO trees (private states): Each user or smart contract has its private state stored in UTXO trees. These trees are kept as plaintext on the client’s +tree structure allows for quick and efficient verification of UTXO existence.

        +
        4. +
      4. +

        UTXO trees (private states): Each user or smart contract has its private state stored in UTXO trees. These trees are kept as plaintext on the client’s local system (off-chain), ensuring privacy as sensitive information remains confidential. The private state includes all UTXOs owned by the user or the smart contract, and these -are not directly exposed to the public blockchain. For instance, users have full control over their private state, which is not visible to other participants in the network.

      In conclusion, the tree structures enable efficient verification of transaction validity without compromising privacy. By using Merkle trees, +are not directly exposed to the public blockchain. For instance, users have full control over their private state, which is not visible to other participants in the network.

      +
      3. +
    +

    In conclusion, the tree structures enable efficient verification of transaction validity without compromising privacy. By using Merkle trees, Nescience ensures that any tampering with the data can be easily detected. The efficient structure of these trees supports the scalability of the architecture, allowing it to handle a large number of transactions and data entries. By leveraging different types of trees, Nescience ensures efficient and secure management -of both public and private states.

    ## d) Nullifier tree in Nescience

    The nullifier tree is a fundamental component of NSSA, designed to prevent double spending by securely tracking all consumed UTXOs. -This tree acts as a public ledger of spent UTXOs, ensuring that once a UTXO is consumed in a transaction, it cannot be reused in future transactions.

    The primary function of the nullifier Tree is to store the nullifiers of all consumed UTXOs. By recording the nullifiers in a public tree, -the system ensures that each UTXO is spent only once, thereby safeguarding the integrity of the entire network.

    • Ensuring non-membership and preventing double spending +of both public and private states.

      +

      ## d) Nullifier tree in Nescience

      +

      The nullifier tree is a fundamental component of NSSA, designed to prevent double spending by securely tracking all consumed UTXOs. +This tree acts as a public ledger of spent UTXOs, ensuring that once a UTXO is consumed in a transaction, it cannot be reused in future transactions.

      +

      The primary function of the nullifier Tree is to store the nullifiers of all consumed UTXOs. By recording the nullifiers in a public tree, +the system ensures that each UTXO is spent only once, thereby safeguarding the integrity of the entire network.

      +
        +
      • +

        Ensuring non-membership and preventing double spending Before a user can consume a UTXO in a transaction, they must provide a non-membership proof. This proof demonstrates that the UTXO’s nullifier does not already exist in the Nullifier Tree, proving that the UTXO has not been spent before. If the UTXO’s nullifier is found in the tree, the system will reject the transaction, preventing double spending. The non-membership proof ensures that users cannot attempt to spend the same UTXO in multiple transactions. This mechanism is critical for maintaining the security and reliability of NSSA. The tree structure, which is typically built using a cryptographic tree like a Merkle tree, allows for efficient verification of nullifiers. -Verifiers can quickly check whether a nullifier is present or absent in the tree, ensuring that each UTXO is only spent once.

      • Nullifier tree structure and operation +Verifiers can quickly check whether a nullifier is present or absent in the tree, ensuring that each UTXO is only spent once.

        +
        • +
      • +

        Nullifier tree structure and operation The nullifier tree is likely structured as a Merkle tree, which is a cryptographic binary tree where each node represents the hash of its child nodes. This structure allows for efficient storage and verification of large sets of nullifiers as only the root hash of the tree needs to be stored on the blockchain. When a new nullifier is added to the tree, the tree is recalculated, and the root hash is updated. This process ensures that all consumed UTXOs are securely recorded. @@ -1385,56 +3108,135 @@ Each time a transaction consumes a UTXO, the nullifier is added to the Nullifier UTXO has not been double spent, verifiers can use the tree’s root hash and a proof of inclusion or exclusion (membership or non-membership proof) to check whether the nullifier is present in the tree. For example, if Alice wants to spend a UTXO, she must prove that the nullifier associated with that UTXO is not already in the Nullifier Tree. She generates a non-membership proof that shows her nullifier is not recorded in the tree, and the transaction is allowed to proceed. Once the transaction is completed, -the nullifier is added to the tree, ensuring that the UTXO cannot be used again.

      Conclusion +the nullifier is added to the tree, ensuring that the UTXO cannot be used again.

      +
      • +
    +

    Conclusion The Nullifier Tree is a crucial element of Nescience's security. By recording all consumed UTXOs and ensuring that nullifiers are unique, the tree prevents double spending and maintains the integrity of the blockchain. The non-membership proof mechanism guarantees that every transaction is validated against the tree. This structure supports -the scalability and security of NSSA, providing a reliable method for verifying the validity of transactions while preventing malicious behavior.

    e) Recursive-friendly privacy-preserving zk-zkVM​

    The development of the zk-zkVM in Nescience is a work in progress, as the architecture continues to evolve to support privacy-preserving transactions +the scalability and security of NSSA, providing a reliable method for verifying the validity of transactions while preventing malicious behavior.

    +

    e) Recursive-friendly privacy-preserving zk-zkVM​

    +

    The development of the zk-zkVM in Nescience is a work in progress, as the architecture continues to evolve to support privacy-preserving transactions and efficient ZKP generation. The goal of the zk-zkVM is to seamlessly integrate with the Nescience state-separation architecture, -ensuring that private transactions remain confidential while allowing the network to verify their validity without compromising privacy.

    Currently, we are exploring and testing several existing zkVMs to identify the most suitable platform for our needs. Our focus is on finding a zkVM +ensuring that private transactions remain confidential while allowing the network to verify their validity without compromising privacy.

    +

    Currently, we are exploring and testing several existing zkVMs to identify the most suitable platform for our needs. Our focus is on finding a zkVM that not only supports the core features of Nescience, such as state separation and privacy, but also provides the efficiency and scalability required for a decentralized system. Once a suitable zkVM is chosen, we will begin implementing advanced privacy features on top of it, including support for -confidential transactions, selective disclosure, and recursive proof aggregation.

    The integration of these privacy-preserving features with an existing zkVM will enable Nescience to fully employ its state-separation architecture, +confidential transactions, selective disclosure, and recursive proof aggregation.

    +

    The integration of these privacy-preserving features with an existing zkVM will enable Nescience to fully employ its state-separation architecture, ensuring that users can conduct private transactions with robust security and scalability. This approach will allow us to leverage the strengths of -proven zkVM technologies while enhancing them with the unique privacy and state-separation capabilities that Nescience requires.

    • Privacy-preserving features: At its core, the zk-zkVM is designed with privacy in mind. One of the zk-zkVM’s standout privacy features is selective disclosure, +proven zkVM technologies while enhancing them with the unique privacy and state-separation capabilities that Nescience requires.

      +
        +
      • +

        Privacy-preserving features: At its core, the zk-zkVM is designed with privacy in mind. One of the zk-zkVM’s standout privacy features is selective disclosure, which allows users to reveal only specific details of a transaction as needed. For example, a user could disclose the transaction amount while concealing the identities of the participants. The zk-zkVM employs advanced encryption techniques to protect this sensitive data. All transaction data is encrypted before being stored on the blockchain, so even if the data is intercepted, it cannot be deciphered without the appropriate decryption keys. Another of the crucial privacy-preserving features is the support for confidential transactions. Only the parties involved in the transaction can access the encrypted data. Furthermore, the zk-zkVM supports verifiable encryption, a powerful capability that allows encrypted data to be included in ZKPs without needing to decrypt it. This ensures that transaction details remain private -while their correctness can still be proven.

      • Lightweight design for accessibility: The zk-zkVM is being designed to be lightweight and efficient, enabling it to run on standard consumer-grade hardware. -This makes it accessible to a wide range of users without requiring specialized equipment or significant computational resources.

      • Faster proving time: To maintain a seamless user experience, especially during high transaction volumes, the zk-zkVM is being optimized for fast proving times. -Fast proof generation is particularly important for ensuring that the system remains usable during periods of peak activity, preventing bottlenecks and maintaining the fluidity of the network.

      • Recursive-friendly operations: One of the most advanced features of the zk-zkVM will be its support for recursive operations. Recursion enables the aggregation -of multiple proofs into a single proof, improving efficiency on both the client and sequencer sides of the network.

      • Client-side recursion (batch processing): When a single transaction involves multiple executions, each requiring its own ZKP, these individual +while their correctness can still be proven.

        +
        • +
      • +

        Lightweight design for accessibility: The zk-zkVM is being designed to be lightweight and efficient, enabling it to run on standard consumer-grade hardware. +This makes it accessible to a wide range of users without requiring specialized equipment or significant computational resources.

        +
        • +
      • +

        Faster proving time: To maintain a seamless user experience, especially during high transaction volumes, the zk-zkVM is being optimized for fast proving times. +Fast proof generation is particularly important for ensuring that the system remains usable during periods of peak activity, preventing bottlenecks and maintaining the fluidity of the network.

        +
        • +
      • +

        Recursive-friendly operations: One of the most advanced features of the zk-zkVM will be its support for recursive operations. Recursion enables the aggregation +of multiple proofs into a single proof, improving efficiency on both the client and sequencer sides of the network.

        +
        • +
      • +

        Client-side recursion (batch processing): When a single transaction involves multiple executions, each requiring its own ZKP, these individual proofs can be recursively aggregated before being sent to the sequencer. This reduces the overall data transmitted, enhancing the efficiency of the transaction process -by compressing multiple proofs into a single package. -

      • Sequencer-side recursion (reduced redundancy): The sequencer, which is responsible for processing transactions and creating verifiable blocks, collects transactions +by compressing multiple proofs into a single package.

        +
        • +
      • +

        Sequencer-side recursion (reduced redundancy): The sequencer, which is responsible for processing transactions and creating verifiable blocks, collects transactions containing aggregated proofs. These proofs are further merged into a single comprehensive proof, ensuring that all transactions within a block are validated collectively. -This process reduces redundancy and optimizes the blockchain’s efficiency by minimizing the size and complexity of the proofs required for verification.

      • Developer-friendly language: To foster widespread adoption and innovation within the Nescience ecosystem, the zk-zkVM would include a developer-friendly language. +This process reduces redundancy and optimizes the blockchain’s efficiency by minimizing the size and complexity of the proofs required for verification.

        +
        • +
      • +

        Developer-friendly language: To foster widespread adoption and innovation within the Nescience ecosystem, the zk-zkVM would include a developer-friendly language. This high-level language simplifies the process of building applications that leverage state separation and privacy-preserving transactions. The language should offer extensive support for modular design, APIs, and SDKs, enabling developers to integrate their applications with the zk-zkVM more easily. By lowering the barrier to entry, Nescience encourages -innovation and helps expand the range of privacy-preserving applications that can be built on its platform.

      Conclusion

      The zk-zkVM in Nescience is a powerful and versatile virtual machine that embodies the principles of privacy, efficiency, and scalability. By supporting ZKPs +innovation and helps expand the range of privacy-preserving applications that can be built on its platform.

      +
      • +
    +

    Conclusion

    +

    The zk-zkVM in Nescience is a powerful and versatile virtual machine that embodies the principles of privacy, efficiency, and scalability. By supporting ZKPs and integrating with advanced privacy technologies like homomorphic encryption. Its lightweight design allows it to run efficiently on standard hardware, promoting decentralization, and its recursive operations further enhance the system's scalability. With its developer-friendly language and fast proving times, the zk-zkVM is positioned as a key component in -fostering the growth and adoption of privacy-preserving blockchain applications.

    f) MPC-based synchronization mechanism (under review)​

    Nescience is developing an MPC-based synchronization mechanism to balance privacy and fairness between public and private execution types. +fostering the growth and adoption of privacy-preserving blockchain applications.

    +

    f) MPC-based synchronization mechanism (under review)​

    +

    Nescience is developing an MPC-based synchronization mechanism to balance privacy and fairness between public and private execution types. This mechanism extracts common information from encrypted UTXOs without revealing private details, ensuring privacy and preventing UTXO linkage to users or specific transactions. -It guarantees that public and private executions remain equitable, with the total input equaling the public output.

    The mechanism employs MPC protocols to perform computations privately, ZKPs to verify correctness, and cryptographic protocols +It guarantees that public and private executions remain equitable, with the total input equaling the public output.

    +

    The mechanism employs MPC protocols to perform computations privately, ZKPs to verify correctness, and cryptographic protocols to secure data during synchronization. This ensures a consistent and fair environment for all users, regardless of their chosen privacy level. Currently, -this feature is under development and review for potential inclusion depending on the research output and compatibility.

    D. Future plans for Nescience

    Nescience is committed to continuously evolving its architecture to ensure scalability, privacy, and security in a growing blockchain landscape. +this feature is under development and review for potential inclusion depending on the research output and compatibility.

    +

    D. Future plans for Nescience

    +

    Nescience is committed to continuously evolving its architecture to ensure scalability, privacy, and security in a growing blockchain landscape. One of the primary goals is to integrate the zk-zkVM and the Nescience state-separation architecture into a fully functioning node, -enabling efficient private transactions while maintaining network integrity.

    • Addressing scalability challenges: A key challenge facing Nescience is the increasing size of nullifier and hashed UTXO trees, which could impact -network performance and scalability over time. To mitigate this, Nescience plans to adopt state-of-the-art scalable privacy techniques such as:
      • Mutator sets: Dynamically adjusting data structures to manage the growth of the nullifier set efficiently.
      • SNARK-based accumulators: Compressing data in a verifiable way to ensure that only relevant information is stored while maintaining cryptographic security.
      • Pruning techniques: Periodically trimming unnecessary data from trees to maintain optimal size and performance, ensuring that the network scales logarithmically -rather than exponentially as more transactions occur.

    By implementing these approaches, Nescience aims to keep the size of its data structures manageable, ensuring that scalability does not come at the cost of performance or privacy.

    • Enhanced key management: Another critical focus for Nescience is improving key management to streamline operations and enhance security. +enabling efficient private transactions while maintaining network integrity.

      +
        +
      • Addressing scalability challenges: A key challenge facing Nescience is the increasing size of nullifier and hashed UTXO trees, which could impact +network performance and scalability over time. To mitigate this, Nescience plans to adopt state-of-the-art scalable privacy techniques such as: +
          +
        • Mutator sets: Dynamically adjusting data structures to manage the growth of the nullifier set efficiently.
          • +
        • SNARK-based accumulators: Compressing data in a verifiable way to ensure that only relevant information is stored while maintaining cryptographic security.
          • +
        • Pruning techniques: Periodically trimming unnecessary data from trees to maintain optimal size and performance, ensuring that the network scales logarithmically +rather than exponentially as more transactions occur.
          • +
        +
        • +
      +

      By implementing these approaches, Nescience aims to keep the size of its data structures manageable, ensuring that scalability does not come at the cost of performance or privacy.

      +
        +
      • +

        Enhanced key management: Another critical focus for Nescience is improving key management to streamline operations and enhance security. The plan is to integrate the different keys used for signatures, addresses, UTXO encryption, and SNARK verification into a unified system. This integration will simplify key management for users while reducing the risk of security breaches caused by complex, disparate key systems. Nescience also plans to implement Hierarchical Deterministic (HD) keys, which allow users to derive multiple keys from a single seed, enhancing both security and usability. This approach reduces the complexity of managing multiple keys across various functions and provides an additional layer of protection for private transactions. Additionally, multi-signature schemes will be introduced, requiring multiple parties to authorize transactions. -This feature increases security by reducing the likelihood of unauthorized access, ensuring that a single compromised key cannot lead to malicious transactions.

      • Integrating advanced cryptographic techniques: Nescience will integrate advanced cryptographic techniques, enhancing both privacy and scalability. Among these are:

        • Homomorphic encryption: Allowing computations to be performed on encrypted data without the need to decrypt it, preserving privacy while enabling secure, complex data processing.
        • Zero-knowledge rollups: Bundling multiple transactions into a single proof to reduce the amount of data processed and stored on the blockchain, -significantly improving scalability without sacrificing security.

      These cryptographic enhancements will ensure that Nescience can support a growing network while continuing to protect user privacy and maintaining high transaction throughput.

      • Long-term vision

      The ultimate goal for Nescience is to deploy a fully operational node powered by zk-zkVM and the Nescience state-separation architecture. +This feature increases security by reducing the likelihood of unauthorized access, ensuring that a single compromised key cannot lead to malicious transactions.

      +
      • +
    • +

      Integrating advanced cryptographic techniques: Nescience will integrate advanced cryptographic techniques, enhancing both privacy and scalability. Among these are:

      +
        +
      • Homomorphic encryption: Allowing computations to be performed on encrypted data without the need to decrypt it, preserving privacy while enabling secure, complex data processing.
        • +
      • Zero-knowledge rollups: Bundling multiple transactions into a single proof to reduce the amount of data processed and stored on the blockchain, +significantly improving scalability without sacrificing security.
        • +
      +
      • +
    +

    These cryptographic enhancements will ensure that Nescience can support a growing network while continuing to protect user privacy and maintaining high transaction throughput.

    +
      +
    • Long-term vision
      • +
    +

    The ultimate goal for Nescience is to deploy a fully operational node powered by zk-zkVM and the Nescience state-separation architecture. This node will handle complex, private transactions at scale while integrating all of the advanced cryptographic techniques outlined in the roadmap. -Nescience aims to provide users with an infrastructure that balances privacy, security, and efficiency, ensuring the network remains resilient and capable of handling future demands.

    By pursuing these future plans, Nescience is poised to not only address current challenges around scalability and key management but also lead the way in +Nescience aims to provide users with an infrastructure that balances privacy, security, and efficiency, ensuring the network remains resilient and capable of handling future demands.

    +

    By pursuing these future plans, Nescience is poised to not only address current challenges around scalability and key management but also lead the way in applying advanced cryptography to decentralized systems. This vision will help secure the long-term integrity and performance of the Nescience state-separation -model as the blockchain grows and evolves.

    References

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrieved from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    [4] Wuille, P. (2012). Bitcoin Improvement Proposal (BIP) 32. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    [5] Sin7y Tech Review (29): Design Principles of Private Transactions in Aleo & Zcash. Retrieved from https://hackmd.io/@sin7y/rkxFXLkgs

    [6] Sin7y Tech Review (33): Principles of private transactions and regulatory compliance issues. Retrieved from https://hackmd.io/@sin7y/S16RyFzZn

    [7] Zcash Protocol Specification. Retrieved from https://zips.z.cash/protocol/protocol.pdf

    [8] Anatomy of a Zcash Transaction. Retrieved from https://electriccoin.co/blog/anatomy-of-zcash

    [9] The Penumbra Protocol: Notes, Nullifiers, and Trees. Retrieved from https://protocol.penumbra.zone/main/concepts/notes_nullifiers_trees.html

    [10] Zero-knowledge Virtual Machine (ZKVM). Retrieved from https://medium.com/@abhilashkrish/zero-knowledge-virtual-machine-zkvm-95adc2082cfd

    [11] What's a Sparse Merkle tree?. Retrieved from https://medium.com/@kelvinfichter/whats-a-sparse-merkle-tree-acda70aeb837

    [12] Lecture 10: Accounts Model and Merkle Trees. Retrieved from https://web.stanford.edu/class/ee374/lec_notes/lec10.pdf

    [13] The UTXO vs Account Model. Retrieved from https://www.horizen.io/academy/utxo-vs-account-model/

    [14] Addresses and Value Pools in Zcash. Retrieved from https://zcash.readthedocs.io/en/latest/rtd_pages/addresses.html

    ]]> +model as the blockchain grows and evolves.

    +

    References

    +

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    +

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrieved from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    +

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    +

    [4] Wuille, P. (2012). Bitcoin Improvement Proposal (BIP) 32. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    +

    [5] Sin7y Tech Review (29): Design Principles of Private Transactions in Aleo & Zcash. Retrieved from https://hackmd.io/@sin7y/rkxFXLkgs

    +

    [6] Sin7y Tech Review (33): Principles of private transactions and regulatory compliance issues. Retrieved from https://hackmd.io/@sin7y/S16RyFzZn

    +

    [7] Zcash Protocol Specification. Retrieved from https://zips.z.cash/protocol/protocol.pdf

    +

    [8] Anatomy of a Zcash Transaction. Retrieved from https://electriccoin.co/blog/anatomy-of-zcash

    +

    [9] The Penumbra Protocol: Notes, Nullifiers, and Trees. Retrieved from https://protocol.penumbra.zone/main/concepts/notes_nullifiers_trees.html

    +

    [10] Zero-knowledge Virtual Machine (ZKVM). Retrieved from https://medium.com/@abhilashkrish/zero-knowledge-virtual-machine-zkvm-95adc2082cfd

    +

    [11] What's a Sparse Merkle tree?. Retrieved from https://medium.com/@kelvinfichter/whats-a-sparse-merkle-tree-acda70aeb837

    +

    [12] Lecture 10: Accounts Model and Merkle Trees. Retrieved from https://web.stanford.edu/class/ee374/lec_notes/lec10.pdf

    +

    [13] The UTXO vs Account Model. Retrieved from https://www.horizen.io/academy/utxo-vs-account-model/

    +

    [14] Addresses and Value Pools in Zcash. Retrieved from https://zcash.readthedocs.io/en/latest/rtd_pages/addresses.html

    ]]> Moudy @@ -1445,84 +3247,193 @@ model as the blockchain grows and evolves.

    References

    [1] 2024-07-19T12:00:00.000Z

    - We examine two data structures: Bloom filters and Cuckoo filters.

    Membership with Bloom Filters and Cuckoo Filters​

    The ability to efficiently query the membership of an element in a given data set is crucial. + We examine two data structures: Bloom filters and Cuckoo filters.

    + +

    Membership with Bloom Filters and Cuckoo Filters​

    +

    The ability to efficiently query the membership of an element in a given data set is crucial. In certain applications, it is more important to output a result quickly than to have a 'perfect' result. In particular, false positives may be an acceptable tradeoff for speed. In this blog, we examine Bloom and Cuckoo data filters. -Both of these filters are data structures that can be used for membership proofs.

    Everyone is familiar with the process of creating a new account for various websites, whether it is an e-mail account or a social media account. +Both of these filters are data structures that can be used for membership proofs.

    +

    Everyone is familiar with the process of creating a new account for various websites, whether it is an e-mail account or a social media account. Consider when you enter your desired username. Many sites provide real-time feedback, as you type, on the availability of a given string. In this scenario, it is necessary that the result is seemingly instant, regardless of the number of existing accounts. However, it is not important that the usernames that are flagged as unavailable are, in fact, in use. -That is, it is sufficient to have a probabilistic check for membership.

    Bloom filters and Cuckoo filters are data structures that can be used to accumulate data with a fixed amount of space. -The associated filter FF for a digest of data DD can be queried to determine whether an element is (possibly) a member of DD:

    • 0: The queried element is definitely not a member of digest DD.
    • 1: The entry is possibly a member of the digest DD.

    The algorithms associated with Bloom filters and Cuckoo filters, which we will discuss shortly, are deterministic. -The possibility of false positives arises from the query algorithm.

    Bloom filters​

    A Bloom filter is a data structure that can be used to accumulate an arbitrary amount of data with a fixed amount of space. +That is, it is sufficient to have a probabilistic check for membership.

    +

    Bloom filters and Cuckoo filters are data structures that can be used to accumulate data with a fixed amount of space. +The associated filter FF for a digest of data DD can be queried to determine whether an element is (possibly) a member of DD:

    +
      +
    • 0: The queried element is definitely not a member of digest DD.
      • +
    • 1: The entry is possibly a member of the digest DD.
      • +
    +

    The algorithms associated with Bloom filters and Cuckoo filters, which we will discuss shortly, are deterministic. +The possibility of false positives arises from the query algorithm.

    +

    Bloom filters​

    +

    A Bloom filter is a data structure that can be used to accumulate an arbitrary amount of data with a fixed amount of space. Bloom filters have been a popular data structure for proof of non-membership due to their small storage size. -Specifically, a Bloom filter consists of a binary string v∈{0,1}n{\bf{v}} \in \{0,1\}^n and kk hash functions {hi:{0,1}∗→{0,…,n−1}}i=0k−1\{h_i: \{0,1\}^* \rightarrow \{0,\dots,n-1\}\}_{i=0}^{k-1}. -We note that each hash function hih_i is used to determine an index of our binary string v{\bf{v}} to flip the associated bit to 1. -The binary string v{\bf{v}} is initialized with every entry as 0. -The hash functions do not need to be cryptographic hash functions.

    • Append: Suppose that we wish to add the element xx to the Bloom filter.
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=hi(x){\bf{b}}[i] := h_i(x) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
      • Update the binary string v[b[i]]←1{\bf{v}}[{\bf{b}}[i]] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
    • Query: Suppose that we wish to query the Bloom filter for element yy.
      • Return 1 provided v[hi(y)]=1{\bf{v}}[h_i(y)] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.

    The algorithm Query will output 1 for every element yy that has been added to the Bloom filter. +Specifically, a Bloom filter consists of a binary string v∈{0,1}n{\bf{v}} \in \{0,1\}^n and kk hash functions {hi:{0,1}∗→{0,…,n−1}}i=0k−1\{h_i: \{0,1\}^* \rightarrow \{0,\dots,n-1\}\}_{i=0}^{k-1}. +We note that each hash function hih_i is used to determine an index of our binary string v{\bf{v}} to flip the associated bit to 1. +The binary string v{\bf{v}} is initialized with every entry as 0. +The hash functions do not need to be cryptographic hash functions.

    +
      +
    • +

      Append: Suppose that we wish to add the element xx to the Bloom filter.

      +
        +
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=hi(x){\bf{b}}[i] := h_i(x) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      • Update the binary string v[b[i]]←1{\bf{v}}[{\bf{b}}[i]] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Bloom filter for element yy.

      +
        +
      • Return 1 provided v[hi(y)]=1{\bf{v}}[h_i(y)] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.
        • +
      +
      • +
    +

    The algorithm Query will output 1 for every element yy that has been added to the Bloom filter. This is a consequence of the Append algorithm. However, due to potential collisions over a set of hash functions, it is possible for false positives to occur. -Moreover, the possibility of collisions makes it impossible to remove elements from the Bloom filter.

    Complexity​

    The storage of a Bloom filter requires constant space. -Specifically, the Bloom filter uses nn bits regardless of the size of the digest. -So, regardless of the number of elements that we append, the Bloom filter will use nn bits. -Further, if we assume that each of the kk hash functions runs in constant time, then we can append/query an entry in O(k)O(k).

    Example​

    Suppose that k=3k = 3 and n=10n = 10. -Our Bloom filter is initialized as v=(0000000000).\bf{v} = \begin{pmatrix}0&0&0&0&0&0&0&0&0&0\end{pmatrix}. -Now, we will append the words addadd, sumsum, and equalequal. -Suppose that

    h0(add)=1h1(add)=4h2(add)=7h0(sum)=9h1(sum)=2h2(sum)=1h0(equal)=5h1(equal)=8h2(equal)=0.\begin{matrix} h_0(add) = 1 & h_1(add) = 4 & h_2(add) = 7\\ h_0(sum) = 9 & h_1(sum) = 2 & h_2(sum) = 1\\ h_0(equal) = 5 & h_1(equal) = 8 & h_2(equal) = 0. \end{matrix}

    After appending these words, the Bloom filter is v=(1110110111).\bf{v} = \begin{pmatrix}1&1&1&0&1&1&0&1&1&1\end{pmatrix}.

    Now, suppose that we query the words subtractsubtract and multiplemultiple so that

    h0(subtract)=3h1(subtract)=5h2(subtract)=1h0(multiple)=7h1(multiple)=1h2(multiple)=4\begin{matrix} h_0(subtract) = 3 & h_1(subtract) = 5 & h_2(subtract) = 1\\ h_0(multiple) = 7 & h_1(multiple) = 1 & h_2(multiple) = 4\\ \end{matrix}.

    The query for subtractsubtract returns 0 since v[3]=0{\bf{v}}[3]=0. -On the other hand, the query for multiplemultiple returns 1 since v[1]=1,v[4]=1{\bf{v}}[1]=1, {\bf{v}}[4] = 1, and v[7]=1{\bf{v}}[7]=1. -Even though multiplemultiple was not used to generate the Bloom filter v{\bf{v}}, our query returns the false positive.

    Probability of false positives​

    For our analysis, we will assume that the probabilities that arise in our analysis are independent. -However, this assumption can be removed to gain the same approximation.

    We note that for a single hash function, the probability that a specific bit is flipped to 1 is 1/n1/n. -So, the probability that the specific bit is not flipped by the hash function is 1−1/n1-1/n. -Applying our assumption that the kk hash functions are 'independent,' +Moreover, the possibility of collisions makes it impossible to remove elements from the Bloom filter.

    +

    Complexity​

    +

    The storage of a Bloom filter requires constant space. +Specifically, the Bloom filter uses nn bits regardless of the size of the digest. +So, regardless of the number of elements that we append, the Bloom filter will use nn bits. +Further, if we assume that each of the kk hash functions runs in constant time, then we can append/query an entry in O(k)O(k).

    +

    Example​

    +

    Suppose that k=3k = 3 and n=10n = 10. +Our Bloom filter is initialized as v=(0000000000).\bf{v} = \begin{pmatrix}0&0&0&0&0&0&0&0&0&0\end{pmatrix}. +Now, we will append the words addadd, sumsum, and equalequal. +Suppose that

    +

    h0(add)=1h1(add)=4h2(add)=7h0(sum)=9h1(sum)=2h2(sum)=1h0(equal)=5h1(equal)=8h2(equal)=0.\begin{matrix} +h_0(add) = 1 & h_1(add) = 4 & h_2(add) = 7\\ +h_0(sum) = 9 & h_1(sum) = 2 & h_2(sum) = 1\\ +h_0(equal) = 5 & h_1(equal) = 8 & h_2(equal) = 0. +\end{matrix}

    +

    After appending these words, the Bloom filter is v=(1110110111).\bf{v} = \begin{pmatrix}1&1&1&0&1&1&0&1&1&1\end{pmatrix}.

    +

    Now, suppose that we query the words subtractsubtract and multiplemultiple so that

    +

    h0(subtract)=3h1(subtract)=5h2(subtract)=1h0(multiple)=7h1(multiple)=1h2(multiple)=4\begin{matrix} h_0(subtract) = 3 & h_1(subtract) = 5 & h_2(subtract) = 1\\ h_0(multiple) = 7 & h_1(multiple) = 1 & h_2(multiple) = 4\\ +\end{matrix}.

    +

    The query for subtractsubtract returns 0 since v[3]=0{\bf{v}}[3]=0. +On the other hand, the query for multiplemultiple returns 1 since v[1]=1,v[4]=1{\bf{v}}[1]=1, {\bf{v}}[4] = 1, and v[7]=1{\bf{v}}[7]=1. +Even though multiplemultiple was not used to generate the Bloom filter v{\bf{v}}, our query returns the false positive.

    +

    Probability of false positives​

    +

    For our analysis, we will assume that the probabilities that arise in our analysis are independent. +However, this assumption can be removed to gain the same approximation.

    +

    We note that for a single hash function, the probability that a specific bit is flipped to 1 is 1/n1/n. +So, the probability that the specific bit is not flipped by the hash function is 1−1/n1-1/n. +Applying our assumption that the kk hash functions are 'independent,' the probability that the specific bit is not flipped by any of the hash functions is -(1−1/n)k(1-1/n)^k.

    Recall the calculus fact lim⁡∞(1−1/n)n=e−1\lim_{\infty} (1-1/n)^n = e^{-1}. -That is, as we increase the number of bits that our Bloom filter uses, the approximate probability that a given bit is not flipped by any of the kk hash functions is e−k/ne^{-k/n}.

    Suppose that ℓ\ell entries have been added to the Bloom filter. -The probability that a specific bit is still 0 after the ℓ\ell entries have been added is approximately e−ℓk/ne^{-\ell k/n}. +(1−1/n)k(1-1/n)^k.

    +

    Recall the calculus fact lim⁡∞(1−1/n)n=e−1\lim_{\infty} (1-1/n)^n = e^{-1}. +That is, as we increase the number of bits that our Bloom filter uses, the approximate probability that a given bit is not flipped by any of the kk hash functions is e−k/ne^{-k/n}.

    +

    Suppose that ℓ\ell entries have been added to the Bloom filter. +The probability that a specific bit is still 0 after the ℓ\ell entries have been added is approximately e−ℓk/ne^{-\ell k/n}. The probability that a queried element is erroneously claimed as a member of the digest is approximately -(1−e−ℓk/n)k(1-e^{-\ell k/n})^k.

    The following table provides concrete values for these approximations.

    nnkkℓ\ell(1−e−ℓk/n)k(1-e^{-\ell k/n})^k
    32330.01474
    32370.11143
    323120.30802
    323170.50595
    323280.79804

    Notice that the probability of false positives increases as the number of elements (ℓ\ell) that have been added to the digest increases.

    Sliding-Window Bloom filter​

    Our toy example and table illustrated an issue concerning Bloom filters. -The number of entries that can be added to a Bloom filter is restricted by our choice of kk and nn. +(1−e−ℓk/n)k(1-e^{-\ell k/n})^k.

    +

    The following table provides concrete values for these approximations.

    +
    nnkkℓ\ell(1−e−ℓk/n)k(1-e^{-\ell k/n})^k
    32330.01474
    32370.11143
    323120.30802
    323170.50595
    323280.79804
    +

    Notice that the probability of false positives increases as the number of elements (ℓ\ell) that have been added to the digest increases.

    +

    Sliding-Window Bloom filter​

    +

    Our toy example and table illustrated an issue concerning Bloom filters. +The number of entries that can be added to a Bloom filter is restricted by our choice of kk and nn. Not only does the probability that false positives will occur increase, -but it is possible that our vector v{\bf{v}} can be a string of all 1s. -Szepieniec and Værge proposed a modification to Bloom filters to handle this.

    Instead of having a fixed number of bits for our Bloom filter, we dynamically allot memory based on the number of entries that have been added to the filter. -Given a predetermined threshold (bb) for the number of entries, we shift our 'window' of flipping bits by ss bits. +but it is possible that our vector v{\bf{v}} can be a string of all 1s. +Szepieniec and Værge proposed a modification to Bloom filters to handle this.

    +

    Instead of having a fixed number of bits for our Bloom filter, we dynamically allot memory based on the number of entries that have been added to the filter. +Given a predetermined threshold (bb) for the number of entries, we shift our 'window' of flipping bits by ss bits. Note that this means that it is necessary to keep track of when a given entry is added to the digest. -This means that querying the Sliding-Window Bloom filter will yield different results when different timestamps are used.

    This can be done with kk hash functions as we used earlier. -Alternatively, Szepieniec and Værge proposed using the same hash function but to produce kk entries in the current window. -Specifically, we obtain the bits we wish to flip to 1s by computing h(X∣∣i)h(X || i) for each i∈{0,…,k−1}i \in \{0,\dots, k-1\} and XX as we will define next. -For Sliding-Window Bloom filters, XX is more than just the element we wish to append to the filter. -Instead, XX consists of the element xx and a timestamp tt. -The timestamp tt is used to locate the correct window for bits, as we see below:

    • Append: Suppose that we wish to add the element xx with timestamp tt to the Sliding-Window Bloom filter.
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=h(x∣∣t∣∣i){\bf{b}}[i] := h(x||t||i) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
      • Update the binary string v[b[i]+⌊t/b⌋s]←1{\bf{v}}[{\bf{b}}[i]+\lfloor t/b \rfloor s] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
    • Query: Suppose that we wish to query the Bloom filter for element yy with timestamp tt.
      • Return 1 provided v[h(y∣∣t∣∣i)+⌊t/b⌋s]=1{\bf{v}}[h(y||t||i) + \lfloor t/b \rfloor s] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.

    By incorporating a shifting window, we maintain efficient querying and appending at the cost of constant space. -However, by losing constant space, we gain 'infinite' scalability.

    Cuckoo filters​

    A Cuckoo filter is a data structure for probabilistic membership proofs based on Cuckoo hash tables. +This means that querying the Sliding-Window Bloom filter will yield different results when different timestamps are used.

    +

    This can be done with kk hash functions as we used earlier. +Alternatively, Szepieniec and Værge proposed using the same hash function but to produce kk entries in the current window. +Specifically, we obtain the bits we wish to flip to 1s by computing h(X∣∣i)h(X || i) for each i∈{0,…,k−1}i \in \{0,\dots, k-1\} and XX as we will define next. +For Sliding-Window Bloom filters, XX is more than just the element we wish to append to the filter. +Instead, XX consists of the element xx and a timestamp tt. +The timestamp tt is used to locate the correct window for bits, as we see below:

    +
      +
    • +

      Append: Suppose that we wish to add the element xx with timestamp tt to the Sliding-Window Bloom filter.

      +
        +
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=h(x∣∣t∣∣i){\bf{b}}[i] := h(x||t||i) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      • Update the binary string v[b[i]+⌊t/b⌋s]←1{\bf{v}}[{\bf{b}}[i]+\lfloor t/b \rfloor s] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Bloom filter for element yy with timestamp tt.

      +
        +
      • Return 1 provided v[h(y∣∣t∣∣i)+⌊t/b⌋s]=1{\bf{v}}[h(y||t||i) + \lfloor t/b \rfloor s] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.
        • +
      +
      • +
    +

    By incorporating a shifting window, we maintain efficient querying and appending at the cost of constant space. +However, by losing constant space, we gain 'infinite' scalability.

    +

    Cuckoo filters​

    +

    A Cuckoo filter is a data structure for probabilistic membership proofs based on Cuckoo hash tables. The specific design goal for Cuckoo filters is to address the inability to remove elements from a Bloom Filter. This is done by replacing a list of bits with a list of 'fingerprints.' A fingerprint can be thought of as the hash value for an entry in the digest. A Cuckoo filter is a fixed-length list of 'fingerprints.' -If the maximum number of entries that a Cuckoo filter can hold is nn and a fingerprint occupies ff bits, -then the Cuckoo filter occupies nfnf bits.

    Now, we describe the algorithms associated with the Cuckoo filter CC with hash function hash(X)hash(X) and fingerprint function fingerprint(X)fingerprint(X).

    • Append: Suppose that we wish to add the element xx to the Cuckoo filter.

      • If either position ix:=hash(x)i_x := hash(x) or jx:=i⊗hash(fingerprint(x))j_x := i \otimes hash(fingerprint(x)) of CC is empty, -then fingerprint(x)fingerprint(x) is inserted into an empty position.
      • If both ixi_x and jxj_x are occupied with a fingerprint that is distinct from fingerprint(x)fingerprint(x), -then we select either ixi_x or jxj_x to insert fingerprint(x)fingerprint(x). +If the maximum number of entries that a Cuckoo filter can hold is nn and a fingerprint occupies ff bits, +then the Cuckoo filter occupies nfnf bits.

        +

        Now, we describe the algorithms associated with the Cuckoo filter CC with hash function hash(X)hash(X) and fingerprint function fingerprint(X)fingerprint(X).

        +
          +
        • +

          Append: Suppose that we wish to add the element xx to the Cuckoo filter.

          +
            +
          • If either position ix:=hash(x)i_x := hash(x) or jx:=i⊗hash(fingerprint(x))j_x := i \otimes hash(fingerprint(x)) of CC is empty, +then fingerprint(x)fingerprint(x) is inserted into an empty position.
            • +
          • If both ixi_x and jxj_x are occupied with a fingerprint that is distinct from fingerprint(x)fingerprint(x), +then we select either ixi_x or jxj_x to insert fingerprint(x)fingerprint(x). The fingerprint that had previously occupied this position cannot be discarried. Instead, we insert this fingerprint into its alternate location. This reshuffling process either ends with fingerprints all having their own bucket or one that cannot be inserted. -In the case that we have a fingerprint that cannot be inserted, then the Cuckoo filter is overfilled.

        • Query: Suppose that we wish to query the Cuckoo filter for element yy.

          • Return 1 provided fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y.

        • Delete: Suppose that we wish to delete the element yy from the Cuckoo filter.

          • If yy has been added to the Cuckoo filter, then fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y. -We remove fingerprint(y)fingerprint(y) from the appropriate position.

        We note that false positives in Cuckoo filters only occur when an element shares a fingerprint and hash with a value that has already been added to the Cuckoo filter.

        Example​

        In this example, we will append the words addadd, sumsum, and equalequal to a Cuckoo filter with 8 slots.

        For each word xx, we compute two indices: -ix:=hash(x) and jx:=hash(x)⊗hash(fingerprint(x)).i_x := hash(x) \text{ and } j_x := hash(x) \otimes hash(fingerprint(x)). +In the case that we have a fingerprint that cannot be inserted, then the Cuckoo filter is overfilled.

        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Cuckoo filter for element yy.

      +
        +
      • Return 1 provided fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y.
        • +
      +
      • +
    • +

      Delete: Suppose that we wish to delete the element yy from the Cuckoo filter.

      +
        +
      • If yy has been added to the Cuckoo filter, then fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y. +We remove fingerprint(y)fingerprint(y) from the appropriate position.
        • +
      +
      • +
    +

    We note that false positives in Cuckoo filters only occur when an element shares a fingerprint and hash with a value that has already been added to the Cuckoo filter.

    +

    Example​

    +

    In this example, we will append the words addadd, sumsum, and equalequal to a Cuckoo filter with 8 slots.

    +

    For each word xx, we compute two indices: +ix:=hash(x) and jx:=hash(x)⊗hash(fingerprint(x)).i_x := hash(x) \text{ and } j_x := hash(x) \otimes hash(fingerprint(x)). Suppose that we have the following values for -our words:

    wordixi_xjxj_x
    addadd(0,1,0)(0,1,0)(1,0,0)(1,0,0)
    sumsum(1,0,1)(1,0,1)(1,1,0)(1,1,0)
    equalequal(0,1,0)(0,1,0)(1,0,1)(1,0,1)

    For clarity of the example, we append the words directly to the buckets instead of fingerprints of our data.

    01234567
    append addaddaddadd
    append sumsumaddaddsumsum

    Notice that both of the buckets (2 and 5) that equalequal can map to are occupied. -So, we select one of these buckets (say 2) to insert equalequal into. -Then, we have to insert addadd to its possible bucket (1). -This leaves us with the Cuckoo filter:

    01234567
    addaddequalequalsumsum

    Complexity​

    Notice that deletions and queries to Cuckoo filters are done in constant time. -Specifically, only two locations need to be checked for any data xx. +our words:

    +
    wordixi_xjxj_x
    addadd(0,1,0)(0,1,0)(1,0,0)(1,0,0)
    sumsum(1,0,1)(1,0,1)(1,1,0)(1,1,0)
    equalequal(0,1,0)(0,1,0)(1,0,1)(1,0,1)
    +

    For clarity of the example, we append the words directly to the buckets instead of fingerprints of our data.

    +
    01234567
    append addaddaddadd
    append sumsumaddaddsumsum
    +

    Notice that both of the buckets (2 and 5) that equalequal can map to are occupied. +So, we select one of these buckets (say 2) to insert equalequal into. +Then, we have to insert addadd to its possible bucket (1). +This leaves us with the Cuckoo filter:

    +
    01234567
    addaddequalequalsumsum
    +

    Complexity​

    +

    Notice that deletions and queries to Cuckoo filters are done in constant time. +Specifically, only two locations need to be checked for any data xx. Appends may require shuffling previously added elements to their alternate locations. -As such, the append does not run in constant time.

    Bloom filters vs Cuckoo filters​

    The design of Bloom filters is focused on space efficiency and quick query time. +As such, the append does not run in constant time.

    +

    Bloom filters vs Cuckoo filters​

    +

    The design of Bloom filters is focused on space efficiency and quick query time. Even though they occupy constant space, -Cuckoo filters require significantly more space for nn items than Bloom filters. +Cuckoo filters require significantly more space for nn items than Bloom filters. The worst-case append in a Cuckoo filter is slower than the append in a Bloom filter. However, an append that does not require any shuffling in a Cuckoo filter can be quicker than appends in Bloom filters. Cuckoo filters make up for these disadvantages with quicker query time and the ability to delete entries. -Further, the probability of false positives in Cuckoo filters is lower than the probability of false positives in Bloom filters.

    Combining Filters with RLN​

    In a series of posts (1,2,3), +Further, the probability of false positives in Cuckoo filters is lower than the probability of false positives in Bloom filters.

    +

    Combining Filters with RLN​

    +

    In a series of posts (1,2,3), various versons of rate limiting nullifiers (RLN) that are used by Waku has been discussed. RLN uses a sparse Merkle tree for the membership set. The computational power required to construct the Merkle tree prevent light clients from participating in verifying membership proofs. @@ -1531,17 +3442,31 @@ it was proposed to move the membership set on-chain so that it would not be nece Unfortunately, the naive approach is not practical as the gas limit for a single call is too restrictive for an appropriately sized tree. Instead, it was proposed to make utilize of subtrees. In this section, we provide a discussion of an alternate solution for light clients by using filters for the membership set. -The two parts of RLN that we will focus on are user registration and deletion.

    Both Bloom and Cuckoo filters support user registration as this is can be done as an append. +The two parts of RLN that we will focus on are user registration and deletion.

    +

    Both Bloom and Cuckoo filters support user registration as this is can be done as an append. The fixed size of these filters would restrict the total number of users that can register. This can be migitated by using Sliding-Window Bloom filter as this supports system growth. The Sliding-Window can be adapted to Cuckoo filters as well. In the case of a Sliding-Window filter, an user would maintain the epoch of when they registered. The registration of new users to Bloom filters can be done in constant time which is a significant improvement over appending to subtrees. -Unfortunately, the complexity of registration to Cuckoo filters cannot be as easily computed.

    A user could be slashed from the RLN by sending too many messages in a given epoch. +Unfortunately, the complexity of registration to Cuckoo filters cannot be as easily computed.

    +

    A user could be slashed from the RLN by sending too many messages in a given epoch. Unfortunately, Bloom filters do not support the deletion of members. -Luckily, Cuckoo filters allow for deletions that can performed in constant time.

    Cuckoo filter that use Sliding-Window could be used so that light clients are able to verify proofs of membership in the RLN. +Luckily, Cuckoo filters allow for deletions that can performed in constant time.

    +

    Cuckoo filter that use Sliding-Window could be used so that light clients are able to verify proofs of membership in the RLN. These proofs are not a substitute to the usual proofs that a heavy client can verify due to the allowance of false positives. -However, with the allowance of false positives, a light client can participate in verification RLN proofs in an efficient manner.

    References​

    ]]>     +However, with the allowance of false positives, a light client can participate in verification RLN proofs in an efficient manner.

    +

    References​

    +]]> Marvin @@ -1552,54 +3477,171 @@ However, with the allowance of false positives, a light client can participate i 2024-05-13T12:00:00.000Z - Improving on the previous version of RLN by allowing dynamic epoch sizes.

    Introduction​

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    The premise of RLN-v3 is to have a variable message rate per variable epoch, -which can be explained in the following way:

    • RLN-v1: “Alice can send 1 message per global epoch”

      Practically, this is 1 msg/second

    • RLN-v2: “Alice can send x messages per global epoch”

      Practically, this is x msg/second

    • RLN-v3: “Alice can send x messages within a time interval y chosen by herself. + Improving on the previous version of RLN by allowing dynamic epoch sizes.

      + +

      Introduction​

      +

      Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

      +

      The premise of RLN-v3 is to have a variable message rate per variable epoch, +which can be explained in the following way:

      +
        +
      • +

        RLN-v1: “Alice can send 1 message per global epoch”

        +

        Practically, this is 1 msg/second

        +
        • +
      • +

        RLN-v2: “Alice can send x messages per global epoch”

        +

        Practically, this is x msg/second

        +
        • +
      • +

        RLN-v3: “Alice can send x messages within a time interval y chosen by herself. The funds she has to pay are affected by both the number of messages and the chosen time interval. -Other participants can choose different time intervals fitting their specific needs.

        Practically, this is x msg/y seconds

      RLN-v3 allows higher flexibility and ease of payment/stake for users who have more predictable usage patterns and therefore, -more predictable bandwidth usage on a p2p network (Waku, etc.).

      For example:

      • An AMM that broadcasts bids, asks, and fills over Waku may require a lot of throughput in the smallest epoch possible and hence may register an RLN-v3 membership of 10000 msg/1 second. -They could do this with RLN-v2, too.
      • Alice, a casual user of a messaging app built on Waku, who messages maybe 3-4 people infrequently during the day, may register an RLN-v3 membership of 100 msg/hour, +Other participants can choose different time intervals fitting their specific needs.

        +

        Practically, this is x msg/y seconds

        +
        • +
      +

      RLN-v3 allows higher flexibility and ease of payment/stake for users who have more predictable usage patterns and therefore, +more predictable bandwidth usage on a p2p network (Waku, etc.).

      +

      For example:

      +
        +
      • An AMM that broadcasts bids, asks, and fills over Waku may require a lot of throughput in the smallest epoch possible and hence may register an RLN-v3 membership of 10000 msg/1 second. +They could do this with RLN-v2, too.
        • +
      • Alice, a casual user of a messaging app built on Waku, who messages maybe 3-4 people infrequently during the day, may register an RLN-v3 membership of 100 msg/hour, which would not be possible in RLN-v2 considering the global epoch was set to 1 second. With RLN-v2, Alice would have to register with a membership of 1 msg/sec, which would translate to 3600 msg/hour. This is much higher than her usage and would -result in her overpaying to stake into the membership set.
      • A sync service built over Waku, +result in her overpaying to stake into the membership set.
        • +
      • A sync service built over Waku, whose spec defines that it MUST broadcast a set of public keys every hour, may register an RLN-v3 membership of 1 msg/hour, -cutting down the costs to enter the membership set earlier.

      Theory​

      Modification to leaves set in the membership Merkle tree​

      To ensure that a user’s epoch size (user_epoch_limit) is included within their membership we must modify the user’s commitment/leaf in the tree to contain it. +cutting down the costs to enter the membership set earlier.

      • +
    +

    Theory​

    +

    Modification to leaves set in the membership Merkle tree​

    +

    To ensure that a user’s epoch size (user_epoch_limit) is included within their membership we must modify the user’s commitment/leaf in the tree to contain it. A user’s commitment/leaf in the tree is referred to as a rate_commitment, which was previously derived from their public key (identity_commitment) -and their variable message rate (user_message_limit).

    In RLN-v2:

    rate_commitment=poseidon([identity_commitment,user_message_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit])

    In RLN-v3:

    rate_commitment=poseidon([identity_commitment,user_message_limit,user_epoch_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit, user\_epoch\_limit])

    Modification to circuit inputs​

    To detect double signaling, +and their variable message rate (user_message_limit).

    +

    In RLN-v2:

    +rate_commitment=poseidon([identity_commitment,user_message_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit]) +

    In RLN-v3:

    +rate_commitment=poseidon([identity_commitment,user_message_limit,user_epoch_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit, user\_epoch\_limit]) +

    Modification to circuit inputs​

    +

    To detect double signaling, we make use of a circuit output nullifier, which remains the same if a user generates a proof with the same message_id and external_nullifier, -where the external_nullifier and nullifier are defined as:

    external_nullifier=poseidon([epoch,rln_identifier])nullifier=poseidon([identity_secret,external_nullifier,message_id])external\_nullifier = poseidon([epoch, rln\_identifier]) \\ nullifier = poseidon([identity\_secret, external\_nullifier, message\_id])

    Where:

    • epoch is defined as the Unix epoch timestamp with seconds precision.
    • rln_identifier uniquely identifies an application for which a user submits a proof.
    • identity_secret is the private key of the user.
    • message_id is the sequence number of the user’s message within user_message_limit in an epoch.

    In RLN-v2, the global epoch was 1 second, +where the external_nullifier and nullifier are defined as:

    +external_nullifier=poseidon([epoch,rln_identifier])nullifier=poseidon([identity_secret,external_nullifier,message_id])external\_nullifier = poseidon([epoch, rln\_identifier]) \\ +nullifier = poseidon([identity\_secret, external\_nullifier, message\_id]) +

    Where:

    +
      +
    • epoch is defined as the Unix epoch timestamp with seconds precision.
      • +
    • rln_identifier uniquely identifies an application for which a user submits a proof.
      • +
    • identity_secret is the private key of the user.
      • +
    • message_id is the sequence number of the user’s message within user_message_limit in an epoch.
      • +
    +

    In RLN-v2, the global epoch was 1 second, hence we did not need to perform any assertions to the epoch’s value inside the circuit, -and the validation of the epoch was handled off-circuit (i.e., too old, too large, bad values, etc.).

    In RLN-v3, we propose that the epoch that is passed into the circuit +and the validation of the epoch was handled off-circuit (i.e., too old, too large, bad values, etc.).

    +

    In RLN-v3, we propose that the epoch that is passed into the circuit must be a valid multiple of user_epoch_limit -since the user may pass in values of the epoch which do not directly correlate with the user_epoch_limit.

    For example:

    • A user with user_epoch_limit of 120 +since the user may pass in values of the epoch which do not directly correlate with the user_epoch_limit.

      +

      For example:

      +
        +
      • A user with user_epoch_limit of 120 passes in an epoch of 237 generates user_message_limit proofs with it, can increment the epoch by 1, and generate user_message_limit proofs with it, -thereby allowing them to bypass the message per epoch restriction.

      One could say that we could perform this validation outside of the circuit, +thereby allowing them to bypass the message per epoch restriction.

      • +
    +

    One could say that we could perform this validation outside of the circuit, but we maintain the user_epoch_limit as a private input to the circuit so that the user is not deanonymized by the anonymity set connected to that user_epoch_limit. Since user_epoch_limit is kept private, -the verifier does not have access to that value and cannot perform validation on it.

    If we ensure that the epoch is a multiple of user_epoch_limit, -we have the following scenarios:

    • A user with user_epoch_limit of 120 +the verifier does not have access to that value and cannot perform validation on it.

      +

      If we ensure that the epoch is a multiple of user_epoch_limit, +we have the following scenarios:

      +
        +
      • A user with user_epoch_limit of 120 passes in an epoch of 237. -Proof generation fails since the epoch is not a multiple of user_epoch_limit.
      • A user with user_epoch_limit of 120 +Proof generation fails since the epoch is not a multiple of user_epoch_limit.
        • +
      • A user with user_epoch_limit of 120 passes in an epoch of 240 and -can generate user_message_limit proofs without being slashed.

      Since we perform operations on the epoch, we must include it as a circuit input (previously, it was removed from the circuit inputs to RLN-v2).

      Therefore, the new circuit inputs are as follows:

      // unchanged
      private identity_secret
      private user_message_limit
      private message_id
      private pathElements[]
      private pathIndices[]
      public x // messageHash

      // new/changed
      private user_epoch_limit
      private user_epoch_quotient // epoch/user_epoch_limit to assert within circuit
      public epoch
      public rln_identifier

      The circuit outputs remain the same.

      Additional circuit constraints​

      1. Since we accept the epoch, user_epoch_quotient, and user_epoch_limit, -we must ensure that the relation between these 3 values is preserved. I.e.:

        epoch==user_epoch_limit∗user_epoch_quotientepoch == user\_epoch\_limit * user\_epoch\_quotient

      2. To ensure no overflows/underflows occur in the above multiplication, +can generate user_message_limit proofs without being slashed.

        3. +
    +

    Since we perform operations on the epoch, we must include it as a circuit input (previously, it was removed from the circuit inputs to RLN-v2).

    +

    Therefore, the new circuit inputs are as follows:

    +
    // unchanged
    private identity_secret
    private user_message_limit
    private message_id
    private pathElements[]
    private pathIndices[]
    public x // messageHash

    // new/changed
    private user_epoch_limit
    private user_epoch_quotient // epoch/user_epoch_limit to assert within circuit
    public epoch
    public rln_identifier
    +

    The circuit outputs remain the same.

    +

    Additional circuit constraints​

    +
      +
    1. +

      Since we accept the epoch, user_epoch_quotient, and user_epoch_limit, +we must ensure that the relation between these 3 values is preserved. I.e.:

      +epoch==user_epoch_limit∗user_epoch_quotientepoch == user\_epoch\_limit * user\_epoch\_quotient +
      2. +
    2. +

      To ensure no overflows/underflows occur in the above multiplication, we must constrain the inputs of epoch, user_epoch_quotient, and user_epoch_limit. -We have assumed 3600 to be the maximum valid size of the user_epoch_quotient.

    size(epoch)≤64 bitssize(user_epoch_limit)≤12 bitsuser_epoch_limit≤3600user_epoch_limit≤epochuser_epoch_quotient<user_epoch_limitsize(epoch) \leq 64\ bits \\ size(user\_epoch\_limit) \leq 12\ bits \\ user\_epoch\_limit \leq 3600 \\ user\_epoch\_limit \leq epoch \\ user\_epoch\_quotient < user\_epoch\_limit

    Modifications to external epoch validation (Waku, etc.)​

    For receivers of an RLN-v3 proof +We have assumed 3600 to be the maximum valid size of the user_epoch_quotient.

    + + +size(epoch)≤64 bitssize(user_epoch_limit)≤12 bitsuser_epoch_limit≤3600user_epoch_limit≤epochuser_epoch_quotient<user_epoch_limitsize(epoch) \leq 64\ bits \\ +size(user\_epoch\_limit) \leq 12\ bits \\ +user\_epoch\_limit \leq 3600 \\ +user\_epoch\_limit \leq epoch \\ +user\_epoch\_quotient < user\_epoch\_limit +

    Modifications to external epoch validation (Waku, etc.)​

    +

    For receivers of an RLN-v3 proof to detect if a message is too old, we must use the higher bound of the user_epoch_limit, which has been set to 3600. -The trade-off here is that we allow hour-old messages to propagate within the network.

    Modifications to double signaling detection scheme (Waku, etc.)​

    For verifiers of RLN-v1/v2 proofs, +The trade-off here is that we allow hour-old messages to propagate within the network.

    +

    Modifications to double signaling detection scheme (Waku, etc.)​

    +

    For verifiers of RLN-v1/v2 proofs, a log of nullifiers seen in the last epoch is maintained, and if there is a match with a pre-existing nullifier, -double signaling has been detected and the verifier MAY proceed to slash the spamming user.

    With the RLN-v3 scheme, +double signaling has been detected and the verifier MAY proceed to slash the spamming user.

    +

    With the RLN-v3 scheme, we need to increase the size of the nullifier log used, which previously cleared itself every second to the higher bound of the user_epoch_limit, which is 3600. -Now, the RLN proof verifier must clear the nullifier log every 3600 seconds to satisfactorily detect double signaling.

    The implementation​

    An implementation of the RLN-v3 scheme in gnark can be found here.

    Comments on performance​

    • Hardware: Macbook Air M2, 16GB RAM
    • Circuit: RLN-v3
    • Proving system: Groth16
    • Framework: gnark
    • Elliptic curve: bn254 (aka bn128) (not to be confused with the 254-bit Weierstrass curve)
    • Finite field: Prime-order subgroup of the group of points on the bn254 curve
    • Default Merkle tree height: 20
    • Hashing algorithm: Poseidon
    • Merkle tree: Sparse Indexed Merkle Tree

    Proving​

    The proving time for the RLN-v3 circuit is 90ms for a single proof.

    Verification​

    The verification time for the RLN-v3 circuit is 1.7ms for a single proof.

    Conclusion​

    The RLN-v3 scheme introduces a new epoch-based message rate-limiting scheme to the RLN protocol. -It enhances the user's flexibility in setting their message limits and cost-optimizes their stake.

    Future work​

    • Implementing the RLN-v3 scheme in Zerokit
    • Implementing the RLN-v3 scheme in Waku
    • Formal security analysis of the RLN-v3 scheme

    References​

    ]]>     +Now, the RLN proof verifier must clear the nullifier log every 3600 seconds to satisfactorily detect double signaling.

    +

    The implementation​

    +

    An implementation of the RLN-v3 scheme in gnark can be found here.

    +

    Comments on performance​

    +
      +
    • Hardware: Macbook Air M2, 16GB RAM
      • +
    • Circuit: RLN-v3
      • +
    • Proving system: Groth16
      • +
    • Framework: gnark
      • +
    • Elliptic curve: bn254 (aka bn128) (not to be confused with the 254-bit Weierstrass curve)
      • +
    • Finite field: Prime-order subgroup of the group of points on the bn254 curve
      • +
    • Default Merkle tree height: 20
      • +
    • Hashing algorithm: Poseidon
      • +
    • Merkle tree: Sparse Indexed Merkle Tree
      • +
    +

    Proving​

    +

    The proving time for the RLN-v3 circuit is 90ms for a single proof.

    +

    Verification​

    +

    The verification time for the RLN-v3 circuit is 1.7ms for a single proof.

    +

    Conclusion​

    +

    The RLN-v3 scheme introduces a new epoch-based message rate-limiting scheme to the RLN protocol. +It enhances the user's flexibility in setting their message limits and cost-optimizes their stake.

    +

    Future work​

    +
      +
    • Implementing the RLN-v3 scheme in Zerokit
      • +
    • Implementing the RLN-v3 scheme in Waku
      • +
    • Formal security analysis of the RLN-v3 scheme
      • +
    +

    References​

    +]]> Aaryamann @@ -1610,39 +3652,115 @@ It enhances the user's flexibility in setting their message limits and cost-opti 2024-05-03T12:00:00.000Z - How resource-restricted devices can verify RLN proofs fast and efficiently.

    Introduction​

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    This post expands upon ideas described in the previous post, -focusing on how resource-restricted devices can verify RLN proofs fast and efficiently.

    Previously, it was required to fetch all the memberships from the smart contract, + How resource-restricted devices can verify RLN proofs fast and efficiently.

    + +

    Introduction​

    +

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    +

    This post expands upon ideas described in the previous post, +focusing on how resource-restricted devices can verify RLN proofs fast and efficiently.

    +

    Previously, it was required to fetch all the memberships from the smart contract, construct the merkle tree locally, and derive the merkle root, -which is subsequently used to verify RLN proofs.

    This process is not feasible for resource-restricted devices since it involves a lot of RPC calls, computation and fault tolerance. -One cannot expect a mobile phone to fetch all the memberships from the smart contract and construct the merkle tree locally.

    Constraints and requirements​

    An alternative solution to the one proposed in this post is to construct the merkle tree on-chain, +which is subsequently used to verify RLN proofs.

    +

    This process is not feasible for resource-restricted devices since it involves a lot of RPC calls, computation and fault tolerance. +One cannot expect a mobile phone to fetch all the memberships from the smart contract and construct the merkle tree locally.

    +

    Constraints and requirements​

    +

    An alternative solution to the one proposed in this post is to construct the merkle tree on-chain, and have the root accessible with a single RPC call. -However, this approach increases gas costs for inserting new memberships and may not be feasible until it is optimized further with batching mechanisms, etc.

    The other methods have been explored in more depth here.

    Following are the requirements and constraints for the solution proposed in this post:

    1. Cheap membership insertions.
    2. As few RPC calls as possible to reduce startup time.
    3. Merkle root of the tree is available on-chain.
    4. No centralized services to sequence membership insertions.
    5. Map inserted commitments to the block in which they were inserted.

    Metrics on sync time for a tree with 2,653 leaves​

    The following metrics are based on the current implementation of RLN in the Waku gen0 network.

    Test bench​

    • Hardware: Macbook Air M2, 16GB RAM
    • Network: 120 Megabits/sec
    • Nwaku commit: e61e4ff
    • RLN membership set contract: 0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4
    • Deployed block number: 4,230,716
    • RLN Membership set depth: 20
    • Hash function: PoseidonT3 (which is a gas guzzler)
    • Max size of the membership set: 2^20 = 1,048,576 leaves

    Metrics​

    • Time to sync the whole tree: 4 minutes
    • RPC calls: 702
    • Number of leaves: 2,653

    One can argue that the time to sync the tree at the current state is not that bad. +However, this approach increases gas costs for inserting new memberships and may not be feasible until it is optimized further with batching mechanisms, etc.

    +

    The other methods have been explored in more depth here.

    +

    Following are the requirements and constraints for the solution proposed in this post:

    +
      +
    1. Cheap membership insertions.
      2. +
    2. As few RPC calls as possible to reduce startup time.
      3. +
    3. Merkle root of the tree is available on-chain.
      4. +
    4. No centralized services to sequence membership insertions.
      5. +
    5. Map inserted commitments to the block in which they were inserted.
      6. +
    +

    Metrics on sync time for a tree with 2,653 leaves​

    +

    The following metrics are based on the current implementation of RLN in the Waku gen0 network.

    +

    Test bench​

    +
      +
    • Hardware: Macbook Air M2, 16GB RAM
      • +
    • Network: 120 Megabits/sec
      • +
    • Nwaku commit: e61e4ff
      • +
    • RLN membership set contract: 0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4
      • +
    • Deployed block number: 4,230,716
      • +
    • RLN Membership set depth: 20
      • +
    • Hash function: PoseidonT3 (which is a gas guzzler)
      • +
    • Max size of the membership set: 2^20 = 1,048,576 leaves
      • +
    +

    Metrics​

    +
      +
    • Time to sync the whole tree: 4 minutes
      • +
    • RPC calls: 702
      • +
    • Number of leaves: 2,653
      • +
    +

    One can argue that the time to sync the tree at the current state is not that bad. However, the number of RPC calls is a concern, which scales linearly with the number of blocks since the contract was deployed This is because the implementation fetches all events from the contract, chunking 2,000 blocks at a time. This is done to avoid hitting the block limit of 10,000 events per call, -which is a limitation of popular RPC providers.

    Proposed solution​

    From a theoretical perspective, +which is a limitation of popular RPC providers.

    +

    Proposed solution​

    +

    From a theoretical perspective, one could construct the merkle tree on-chain, in a view call, in-memory. -However, this is not feasible due to the gas costs associated with it.

    To compute the root of a Merkle tree with 2202^{20} leaves it costs approximately 2 billion gas. +However, this is not feasible due to the gas costs associated with it.

    +

    To compute the root of a Merkle tree with 2202^{20} leaves it costs approximately 2 billion gas. With Infura and Alchemy capping the gas limit to 350M and 550M gas respectively, -it is not possible to compute the root of the tree in a single call.

    Acknowledging that Polygon Miden and Penumbra both make use of a tiered commitment tree, -we propose a similar approach for RLN.

    A tiered commitment tree is a tree which is sharded into multiple smaller subtrees, +it is not possible to compute the root of the tree in a single call.

    +

    Acknowledging that Polygon Miden and Penumbra both make use of a tiered commitment tree, +we propose a similar approach for RLN.

    +

    A tiered commitment tree is a tree which is sharded into multiple smaller subtrees, each of which is a tree in itself. This allows scaling in terms of the number of leaves, -as well as reducing state bloat by just storing the root of a subtree when it is full instead of all its leaves.

    Here, the question arises: -What is the maximum number of leaves in a subtree with which the root can be computed in a single call?

    It costs approximately 217M gas to compute the root of a Merkle tree with 2102^{10} leaves.

    This is a feasible number for a single call, -and hence we propose a tiered commitment tree with a maximum of 2102^{10} leaves in a subtree and the number of subtrees is 2102^{10}. -Therefore, the maximum number of leaves in the tree is 2202^{20} (the same as the current implementation).

    img

    Insertion​

    When a commitment is inserted into the tree it is first inserted into the first subtree. -When the first subtree is full the next insertions go into the second subtree and so on.

    Syncing​

    When syncing the tree, +as well as reducing state bloat by just storing the root of a subtree when it is full instead of all its leaves.

    +

    Here, the question arises: +What is the maximum number of leaves in a subtree with which the root can be computed in a single call?

    +

    It costs approximately 217M gas to compute the root of a Merkle tree with 2102^{10} leaves.

    +

    This is a feasible number for a single call, +and hence we propose a tiered commitment tree with a maximum of 2102^{10} leaves in a subtree and the number of subtrees is 2102^{10}. +Therefore, the maximum number of leaves in the tree is 2202^{20} (the same as the current implementation).

    +

    img

    +

    Insertion​

    +

    When a commitment is inserted into the tree it is first inserted into the first subtree. +When the first subtree is full the next insertions go into the second subtree and so on.

    +

    Syncing​

    +

    When syncing the tree, one only needs to fetch the roots of the subtrees. -The root of the full tree can be computed in-memory or on-chain.

    This allows us to derive the following relation:

    number_of_rpc_calls=number_of_filled_subtrees+1number\_of\_rpc\_calls = number\_of\_filled\_subtrees + 1

    This is a significant improvement over the current implementation, -which requires fetching all the memberships from the smart contract.

    Gas costs​

    The gas costs for inserting a commitment into the tree are the same as the current implementation except it consists of an extra SSTORE operation to store the shardIndex of the commitment.

    Events​

    The events emitted by the contract are the same as the current implementation, -appending the shardIndex of the commitment.

    Proof of concept​

    A proof of concept implementation of the tiered commitment tree is available here, -and is deployed on Sepolia at 0xE7987c70B54Ff32f0D5CBbAA8c8Fc1cAf632b9A5.

    It is compatible with the current implementation of the RLN verifier.

    Future work​

    1. Optimize the gas costs of the tiered commitment tree.
    2. Explore using different number of leaves under a given node in the tree (currently set to 2).

    Conclusion​

    The tiered commitment tree is a promising approach to reduce the number of RPC calls required to sync the tree and reduce the gas costs associated with computing the root of the tree. -Consequently, it allows for a more scalable and efficient RLN verifier.

    References​

    ]]>     +The root of the full tree can be computed in-memory or on-chain.

    +

    This allows us to derive the following relation:

    +number_of_rpc_calls=number_of_filled_subtrees+1number\_of\_rpc\_calls = number\_of\_filled\_subtrees + 1 +

    This is a significant improvement over the current implementation, +which requires fetching all the memberships from the smart contract.

    +

    Gas costs​

    +

    The gas costs for inserting a commitment into the tree are the same as the current implementation except it consists of an extra SSTORE operation to store the shardIndex of the commitment.

    +

    Events​

    +

    The events emitted by the contract are the same as the current implementation, +appending the shardIndex of the commitment.

    +

    Proof of concept​

    +

    A proof of concept implementation of the tiered commitment tree is available here, +and is deployed on Sepolia at 0xE7987c70B54Ff32f0D5CBbAA8c8Fc1cAf632b9A5.

    +

    It is compatible with the current implementation of the RLN verifier.

    +

    Future work​

    +
      +
    1. Optimize the gas costs of the tiered commitment tree.
      2. +
    2. Explore using different number of leaves under a given node in the tree (currently set to 2).
      3. +
    +

    Conclusion​

    +

    The tiered commitment tree is a promising approach to reduce the number of RPC calls required to sync the tree and reduce the gas costs associated with computing the root of the tree. +Consequently, it allows for a more scalable and efficient RLN verifier.

    +

    References​

    +]]> Aaryamann @@ -1653,51 +3771,158 @@ Consequently, it allows for a more scalable and efficient RLN verifier.

    2023-11-07T12:00:00.000Z - Rate Limiting Nullifiers in practice, applied to an anonymous p2p network, like Waku.

    Introduction​

    Rate Limiting Nullifier (RLN) is a zero-knowledge gadget that allows users to prove 2 pieces of information,

    1. They belong to a permissioned membership set
    2. Their rate of signaling abides by a fixed number that has been previously declared

    The "membership set" introduced above, is in the form of a sparse, indexed merkle tree. + Rate Limiting Nullifiers in practice, applied to an anonymous p2p network, like Waku.

    + +

    Introduction​

    +

    Rate Limiting Nullifier (RLN) is a zero-knowledge gadget that allows users to prove 2 pieces of information,

    +
      +
    1. They belong to a permissioned membership set
      2. +
    2. Their rate of signaling abides by a fixed number that has been previously declared
      3. +
    +

    The "membership set" introduced above, is in the form of a sparse, indexed merkle tree. This membership set can be maintained on-chain, off-chain or as a hybrid depending on the network's storage costs. Waku makes use of a hybrid membership set, where insertions are tracked in a smart contract. In addition, each Waku node maintains a local copy of the tree, -which is updated upon each insertion.

    Users register themselves with a hash of a locally generated secret, +which is updated upon each insertion.

    +

    Users register themselves with a hash of a locally generated secret, which is then inserted into the tree at the next available index. After having registered, users can prove their membership by proving their knowledge of the pre-image of the respective leaf in the tree. The leaf hashes are also referred to as commitments of the respective users. -The actual proof is done by a Merkle Inclusion Proof, which is a type of ZK proof.

    The circuit ensures that the user's secret does indeed hash to a leaf in the tree, -and that the provided Merkle proof is valid.

    After a User generates this Merkle proof, +The actual proof is done by a Merkle Inclusion Proof, which is a type of ZK proof.

    +

    The circuit ensures that the user's secret does indeed hash to a leaf in the tree, +and that the provided Merkle proof is valid.

    +

    After a User generates this Merkle proof, they can transmit it to other users, who can verify the proof. Including a message's hash within the proof generation, -additionally guarantees integrity of that message.

    A malicious user could generate multiple proofs per epoch. +additionally guarantees integrity of that message.

    +

    A malicious user could generate multiple proofs per epoch. they generate multiple proofs per epoch. However, when multiple proofs are generated per epoch, the malicious user's secret is exposed, which strongly disincentivizes this attack. -This mechanism is further described in malicious User secret interpolation mechanism

    Note: This blog post describes rln-v1, which excludes the range check in favor of a global rate limit for all users, -which is once per time window. This version is currently in use in waku-rln-relay.

    RLN Protocol parameters​

    Given below is the set of cryptographic primitives, -and constants that are used in the RLN protocol.

    1. Proving System: groth16
    2. Elliptic Curve: bn254 (aka bn128) (not to be confused with the 254 bit Weierstrass curve)
    3. Finite Field: Prime-order subgroup of the group of points on the bn254 curve
    4. Default Merkle Tree Height: 20
    5. Hashing algorithm: Poseidon
    6. Merkle Tree: Sparse Indexed Merkle Tree
    7. Messages per epoch: 1
    8. Epoch duration: 10 seconds

    Malicious User secret interpolation mechanism​

    note: all the parameters mentioned below are elements in the finite field mentioned above.

    The private inputs to the circuit are as follows: -

    identitySecret: the randomly generated secret of the user
    identityPathIndex: the index of the commitment derived from the secret
    pathElements: elements included in the path to the index of the commitment

    Following are the public inputs to the circuit -

    x: hash of the signal to the finite field
    rlnIdentifier: application-specific identifier which this proof is being generated for
    epoch: the timestamp which this proof is being generated for

    The outputs of the circuit are as follows: -

    y: result of Shamir's secret sharing calculation
    root: root of the Merkle tree obtained after applying the inclusion proof
    nullifier: uniquely identifies a message, derived from rlnIdentifier, epoch, and the user's secret

    With the above data in mind, following is the circuit pseudocode -

    identityCommitment = Poseidon([identitySecret])
    root = MerkleInclusionProof(identityCommitment, identityPathIndex, pathElements)
    externalNullifier = Poseidon([epoch, rlnIdentifier])
    a1 = Poseidon([identitySecret, externalNullifier])
    y = identitySecret + a1 * x
    nullifier = Poseidon([a1])

    To interpolate the secret of a user who has sent multiple signals during the same epoch to the same rln-based application, we may make use of the following formula -

    a1=(y1−y2)(x1−x2)a_1 = {(y_1 - y_2) \over (x_1 - x_2)}

    where x1x_1, y1y_1 and x2x_2, y2y_2 are shares from different messages

    subsequently, we may use one pair of the shares, x1x_1 and y1y_1 to obtain the identitySecret

    identitySecret=y1−a1∗xidentitySecret = y_1 - a_1 * x

    This enables RLN to be used for rate limiting with a global limit. For arbitrary limits, -please refer to an article written by @curryrasul, rln-v2.

    Waku's problem with DoS​

    In a decentralized, privacy focused messaging system like Waku, -Denial of Service (DoS) vulnerabilities are very common, and must be addressed to promote network scale and optimal bandwidth utilization.

    DoS prevention with user metadata​

    There are a couple of ways a user can be rate-limited, either -

    1. IP Logging
    2. KYC Logging

    Both IP and KYC logging prevent systems from being truly anonymous, and hence, cannot be used as a valid DoS prevention mechanism for Waku.

    RLN can be used as an alternative, which provides the best of both worlds, i.e a permissioned membership set, as well as anonymous signaling. -However, we are bound by k-anonymity rules of the membership set.

    Waku-RLN-Relay is a libp2p pubsub validator that verifies if a proof attached to a given message is valid. -In case the proof is valid, the message is relayed.

    Performance analysis​

    Test bench specs: AMD EPYC 7502P 32-Core, 4x32GB DDR4 Reg.ECC Memory

    This simulation was conducted by @alrevuelta, and is described in more detail here.

    The simulation included 100 waku nodes running in parallel.

    Proof generation times - -

    img

    Proof verification times - -

    img

    A spammer node publishes 3000 msg/epoch, which is detected by all connected nodes, and subsequently disconnect to prevent further spam - -

    img

    Security analysis​

    Barbulescu and Duquesne +This mechanism is further described in malicious User secret interpolation mechanism

    +

    Note: This blog post describes rln-v1, which excludes the range check in favor of a global rate limit for all users, +which is once per time window. This version is currently in use in waku-rln-relay.

    +

    RLN Protocol parameters​

    +

    Given below is the set of cryptographic primitives, +and constants that are used in the RLN protocol.

    +
      +
    1. Proving System: groth16
      2. +
    2. Elliptic Curve: bn254 (aka bn128) (not to be confused with the 254 bit Weierstrass curve)
      3. +
    3. Finite Field: Prime-order subgroup of the group of points on the bn254 curve
      4. +
    4. Default Merkle Tree Height: 20
      5. +
    5. Hashing algorithm: Poseidon
      6. +
    6. Merkle Tree: Sparse Indexed Merkle Tree
      7. +
    7. Messages per epoch: 1
      8. +
    8. Epoch duration: 10 seconds
      9. +
    +

    Malicious User secret interpolation mechanism​

    +
    +

    note: all the parameters mentioned below are elements in the finite field mentioned above.

    +
    +

    The private inputs to the circuit are as follows: -

    +
    identitySecret: the randomly generated secret of the user
    identityPathIndex: the index of the commitment derived from the secret
    pathElements: elements included in the path to the index of the commitment
    +

    Following are the public inputs to the circuit -

    +
    x: hash of the signal to the finite field
    rlnIdentifier: application-specific identifier which this proof is being generated for
    epoch: the timestamp which this proof is being generated for
    +

    The outputs of the circuit are as follows: -

    +
    y: result of Shamir's secret sharing calculation
    root: root of the Merkle tree obtained after applying the inclusion proof
    nullifier: uniquely identifies a message, derived from rlnIdentifier, epoch, and the user's secret
    +

    With the above data in mind, following is the circuit pseudocode -

    +
    identityCommitment = Poseidon([identitySecret])
    root = MerkleInclusionProof(identityCommitment, identityPathIndex, pathElements)
    externalNullifier = Poseidon([epoch, rlnIdentifier])
    a1 = Poseidon([identitySecret, externalNullifier])
    y = identitySecret + a1 * x
    nullifier = Poseidon([a1])
    +

    To interpolate the secret of a user who has sent multiple signals during the same epoch to the same rln-based application, we may make use of the following formula -

    +

    a1=(y1−y2)(x1−x2)a_1 = {(y_1 - y_2) \over (x_1 - x_2)}

    +

    where x1x_1, y1y_1 and x2x_2, y2y_2 are shares from different messages

    +

    subsequently, we may use one pair of the shares, x1x_1 and y1y_1 to obtain the identitySecret

    +

    identitySecret=y1−a1∗xidentitySecret = y_1 - a_1 * x

    +

    This enables RLN to be used for rate limiting with a global limit. For arbitrary limits, +please refer to an article written by @curryrasul, rln-v2.

    +

    Waku's problem with DoS​

    +

    In a decentralized, privacy focused messaging system like Waku, +Denial of Service (DoS) vulnerabilities are very common, and must be addressed to promote network scale and optimal bandwidth utilization.

    +

    DoS prevention with user metadata​

    +

    There are a couple of ways a user can be rate-limited, either -

    +
      +
    1. IP Logging
      2. +
    2. KYC Logging
      3. +
    +

    Both IP and KYC logging prevent systems from being truly anonymous, and hence, cannot be used as a valid DoS prevention mechanism for Waku.

    +

    RLN can be used as an alternative, which provides the best of both worlds, i.e a permissioned membership set, as well as anonymous signaling. +However, we are bound by k-anonymity rules of the membership set.

    +

    Waku-RLN-Relay is a libp2p pubsub validator that verifies if a proof attached to a given message is valid. +In case the proof is valid, the message is relayed.

    +

    Performance analysis​

    +
    +

    Test bench specs: AMD EPYC 7502P 32-Core, 4x32GB DDR4 Reg.ECC Memory

    +
    +

    This simulation was conducted by @alrevuelta, and is described in more detail here.

    +

    The simulation included 100 waku nodes running in parallel.

    +

    Proof generation times - +

    img

    +

    Proof verification times - +

    img

    +

    A spammer node publishes 3000 msg/epoch, which is detected by all connected nodes, and subsequently disconnect to prevent further spam - +

    img

    +

    Security analysis​

    +

    Barbulescu and Duquesne conclude that that the bn254 curve has only 100 bits of security. Since the bn254 curve has a small embedding degree, it is vulnerable to the MOV attack. However, the MOV attack is only applicable to pairings, and not to the elliptic curve itself. It is acceptable to use the bn254 curve for RLN, -since the circuit does not make use of pairings.

    An analysis on the number of rounds in the Poseidon hash function was done, -which concluded that the hashing rounds should not be reduced,

    The smart contracts have not been audited, and are not recommended for real world deployments yet.

    Storage analysis​

    commitment_size=32 bytestree_height=20total_leaves=220max_tree_size=total_leaves∗commitment_sizemax_tree_size=220∗32=33,554,432∴max_tree_size=33.55 megabytescommitment\_size = 32\ bytes \\ tree\_height =20 \\ total\_leaves = 2^{20} \\ max\_tree\_size = total\_leaves * commitment\_size \\ max\_tree\_size = 2^{20} * 32 = 33,554,432 \\ ∴max\_tree\_size = 33.55\ megabytes

    The storage overhead introduced by RLN is minimal. +since the circuit does not make use of pairings.

    +

    An analysis on the number of rounds in the Poseidon hash function was done, +which concluded that the hashing rounds should not be reduced,

    +

    The smart contracts have not been audited, and are not recommended for real world deployments yet.

    +

    Storage analysis​

    +commitment_size=32 bytestree_height=20total_leaves=220max_tree_size=total_leaves∗commitment_sizemax_tree_size=220∗32=33,554,432∴max_tree_size=33.55 megabytescommitment\_size = 32\ bytes \\ +tree\_height =20 \\ +total\_leaves = 2^{20} \\ +max\_tree\_size = total\_leaves * commitment\_size \\ +max\_tree\_size = 2^{20} * 32 = 33,554,432 \\ +∴max\_tree\_size = 33.55\ megabytes +

    The storage overhead introduced by RLN is minimal. RLN only requires 34 megabytes of storage, which poses no problem on most end-user hardware, with the exception of IoT/microcontrollers. -Still, we are working on further optimizations allowing proof generation without having to store the full tree.

    The bare minimum requirements to run RLN​

    With proof generation time in sub-second latency, along with low storage overhead for the tree, -it is possible for end users to generate and verify RLN proofs on a modern smartphone.

    Following is a demo provided by @rramos that demonstrates -waku-rln-relay used in react native.

    Warning: The react native sdk will be deprecated soon, and the above demo should serve as a PoC for RLN on mobiles

    RLN usage guide​

    Zerokit implements api's that allow users to handle operations to the tree, -as well as generate/verify RLN proofs.

    Our main implementation of RLN can be accessed via this Rust crate, +Still, we are working on further optimizations allowing proof generation without having to store the full tree.

    +

    The bare minimum requirements to run RLN​

    +

    With proof generation time in sub-second latency, along with low storage overhead for the tree, +it is possible for end users to generate and verify RLN proofs on a modern smartphone.

    +

    Following is a demo provided by @rramos that demonstrates +waku-rln-relay used in react native.

    +
    +

    Warning: The react native sdk will be deprecated soon, and the above demo should serve as a PoC for RLN on mobiles

    +
    +

    RLN usage guide​

    +

    Zerokit implements api's that allow users to handle operations to the tree, +as well as generate/verify RLN proofs.

    +

    Our main implementation of RLN can be accessed via this Rust crate, which is documented here. It can used in other langugages via the FFI API, which is documented here. The usage of RLN in Waku is detailed in our RLN Implementers guide, -which provides step-by-step instructions on how to run Waku-RLN-Relay.

    Following is a diagram that will help understand the dependency tree -

    rln-dep-tree

    Future work​

    • Optimizations to zerokit for proof generation time.
    • Incrementing tree depth from 20 to 32, to allow more memberships.
    • Optimizations to the smart contract.
    • An ability to signal validity of a message in different time windows.
    • Usage of proving systems other than Groth16.

    References​

    ]]>     +which provides step-by-step instructions on how to run Waku-RLN-Relay.

    +

    Following is a diagram that will help understand the dependency tree -

    +

    rln-dep-tree

    +

    Future work​

    +
      +
    • Optimizations to zerokit for proof generation time.
      • +
    • Incrementing tree depth from 20 to 32, to allow more memberships.
      • +
    • Optimizations to the smart contract.
      • +
    • An ability to signal validity of a message in different time windows.
      • +
    • Usage of proving systems other than Groth16.
      • +
    +

    References​

    +]]> Aaryamann @@ -1708,90 +3933,194 @@ which provides step-by-step instructions on how to run Waku-RLN-Relay.

    Fol 2023-11-06T12:00:00.000Z

    - GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

    Motivitation​

    We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, + GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

    + +

    Motivitation​

    +

    We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, as in the case of Ethereum Improvement Proposal EIP-4844. This work led to a comprehensive study of unstructured P2P networks. -The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.

    Introduction​

    Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, -query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh.

    Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. -At the same time, the underlying IP layer is unreliable.

    Therefore, tree-like overlays are not best suited for reliable information propagation. -Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches.

    Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. -Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links.

    An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DD random peers. -Publishing through a D-regular overlay triggers approximately N×DN \times D transmissions. -Reducing DD reduces the redundant transmissions but compromises reachability and latency. -Sharing metadata through a K-regular overlay (where K>DK > D) allows nodes to pull missing messages.

    GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. -Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency.

    Striking the right balance between parameters like D,KD, K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem.

    At the same time, many other aspects can significantly contribute to the network's performance and scalability. -One option is to realize peers' suitability and continuously changing capabilities while forming overlays.

    For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. -Reshuffling of peering links according to the changing network conditions can lead to superior performance.

    Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. -Grouping multiple super nodes to form virtual node(s) can solve this problem.

    Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks.

    Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. -Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn.

    This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. +The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.

    +

    Introduction​

    +

    Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, +query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh.

    +

    Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. +At the same time, the underlying IP layer is unreliable.

    +

    Therefore, tree-like overlays are not best suited for reliable information propagation. +Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches.

    +

    Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. +Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links.

    +

    An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DD random peers. +Publishing through a D-regular overlay triggers approximately N×DN \times D transmissions. +Reducing DD reduces the redundant transmissions but compromises reachability and latency. +Sharing metadata through a K-regular overlay (where K>DK > D) allows nodes to pull missing messages.

    +

    GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. +Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency.

    +

    Striking the right balance between parameters like D,KD, K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem.

    +

    At the same time, many other aspects can significantly contribute to the network's performance and scalability. +One option is to realize peers' suitability and continuously changing capabilities while forming overlays.

    +

    For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. +Reshuffling of peering links according to the changing network conditions can lead to superior performance.

    +

    Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. +Grouping multiple super nodes to form virtual node(s) can solve this problem.

    +

    Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks.

    +

    Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. +Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn.

    +

    This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. We look into different constraints imposed by application-specific needs (performance goals) and investigate various choices that can augment the network's performance. We explore overlay designs/freshness, peer selection approaches, message-relaying mechanisms, and resilience against adversaries/peer churn. We consider GossipSub a baseline protocol to explore various possibilities and decisively commit to the ones demonstrating superior performance. -We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.

    GOAL1: Low Latency Operation​

    Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. +We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.

    +

    GOAL1: Low Latency Operation​

    +

    Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. A message delivered after the imposed time bounds is considered as dropped. -An early message delivery in applications like live streaming can further enhance the viewing quality.

    The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. -Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance.

    Interestingly, shortest-path message delivery trees have many limitations:

    1) Changing network dynamics requires a quicker and continuous readjustment of the multicast tree. -2) The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion. -3) Node failure can result in partitions, making many segments unreachable. -4) Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology.

    Solutions involve creating multiple random trees to add redundancy [2]. -Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push).

    Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. -New nodes are added from the overlay on node failure, but it requires non-faulty node selection.

    GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages).

    The mesh degree DLow≤D≤DHighD_{Low} \leq D \leq D_{High} is crucial in deciding message dissemination latency. -A smaller value for DD results in higher latency due to increased rounds, whereas a higher DD reduces latency on the cost of increased bandwidth. -At the same time, keeping DD independent of the growing network size (NN) may increase network-wide message dissemination latency. -Adjusting DD with NN maintains similar latency on the cost of increased workload for peers. -Authors in [3] suggest only a logarithmic increase in DD to maintain a manageable workload for peers. -In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \ln(N) + C for an optimal operation, -where CC is a small constant.

    Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].

    GOAL2: Considering Heterogeneity In Overlay Design​

    Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. +An early message delivery in applications like live streaming can further enhance the viewing quality.

    +

    The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. +Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance.

    +

    Interestingly, shortest-path message delivery trees have many limitations:

    +
      +
    1. Changing network dynamics requires a quicker and continuous readjustment of the multicast tree.
      2. +
    2. The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion.
      3. +
    3. Node failure can result in partitions, making many segments unreachable.
      4. +
    4. Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology.
      5. +
    +

    Solutions involve creating multiple random trees to add redundancy [2]. +Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push).

    +

    Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. +New nodes are added from the overlay on node failure, but it requires non-faulty node selection.

    +

    GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages).

    +

    The mesh degree DLow≤D≤DHighD_{Low} \leq D \leq D_{High} is crucial in deciding message dissemination latency. +A smaller value for DD results in higher latency due to increased rounds, whereas a higher DD reduces latency on the cost of increased bandwidth. +At the same time, keeping DD independent of the growing network size (NN) may increase network-wide message dissemination latency. +Adjusting DD with NN maintains similar latency on the cost of increased workload for peers. +Authors in [3] suggest only a logarithmic increase in DD to maintain a manageable workload for peers. +In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \ln(N) + C for an optimal operation, +where CC is a small constant.

    +

    Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].

    +

    GOAL2: Considering Heterogeneity In Overlay Design​

    +

    Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. Most of the research on P2P networks provides simulation results assuming nodes with similar capabilities. -The aspect of dissimilar capabilities and resource-constrained nodes is less explored.

    It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg} does not exceed ln⁡(N)+C\ln(N) + C. -Enforcing all the nodes to have approximately ln⁡(N)+C\ln(N) + C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. +The aspect of dissimilar capabilities and resource-constrained nodes is less explored.

    +

    It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg} does not exceed ln⁡(N)+C\ln(N) + C. +Enforcing all the nodes to have approximately ln⁡(N)+C\ln(N) + C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. At the same time, connecting high-bandwidth nodes through a low-bandwidth node undermines the network's performance. Ideally, the workload on any node should not exceed its available resources. -A better solution involves a two-phased operation:

    1. Every node computes its available bandwidth and selects a node degree DD proportional to its available bandwidth [4]. +A better solution involves a two-phased operation:

      +
        +
      1. +

        Every node computes its available bandwidth and selects a node degree DD proportional to its available bandwidth [4]. Different bandwidth estimation approaches are suggested in literature [5,6]. Simple bandwidth estimation approaches like variable packet size probing [6] yield similar results with less complexity. It is also worth mentioning that many nodes may want to allocate only a capped share of their bandwidth to the network. -Lowering DD according to the available bandwidth can still prove helpful. +Lowering DD according to the available bandwidth can still prove helpful. Additionally, bandwidth preservation at the transport layer through approaches like µTP can be useful. -To further conform to the suggested mesh-degree average DavgD_{avg}, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}.

      2. From the available local view, every node tries connecting peers with the lowest latency until DD connections are made. -We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning.

      The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \leq D \leq D_{High} connections.

      GOAL3: Bandwidth Optimization​

      Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. +To further conform to the suggested mesh-degree average DavgD_{avg}, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}.

      +
      2. +
    2. +

      From the available local view, every node tries connecting peers with the lowest latency until DD connections are made. +We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning.

      +
      3. +
    +

    The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \leq D \leq D_{High} connections.

    +

    GOAL3: Bandwidth Optimization​

    +

    Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. This not only adds to the network-wide message dissemination latency but a significant share of the network's bandwidth is wasted on (usually) unnecessary transmissions. -It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures.

    Many efforts have been made to minimize the impact of redundant transmissions. +It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures.

    +

    Many efforts have been made to minimize the impact of redundant transmissions. These solutions include multicast delivery trees, metadata sharing to enable pull-based operation, in-network information caching, etc. [7,8]. -GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages).

    A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. +GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages).

    +

    A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. As a result, redundant message transmissions can serve as a critical advantage of the GossipSub protocol. -Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3. Assuming an XOR coding function, -we know two trivial properties: x1⊕x2⊕x2=x1x1 \oplus x2 \oplus x2 = x1 and ∣x1∣=∣x1⊕x2⊕x2∣\vert x1 \vert = \vert x1 \oplus x2 \oplus x2 \vert.

    This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. +Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3. Assuming an XOR coding function, +we know two trivial properties: x1⊕x2⊕x2=x1x1 \oplus x2 \oplus x2 = x1 and ∣x1∣=∣x1⊕x2⊕x2∣\vert x1 \vert = \vert x1 \oplus x2 \oplus x2 \vert.

    +

    This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. The receiver can reconstruct the original message from encoded segments. -As a result, fewer transmissions are sufficient for sending more messages to the network.

    However, sharing linear combinations of messages requires organizing messages in intervals, +As a result, fewer transmissions are sufficient for sending more messages to the network.

    +

    However, sharing linear combinations of messages requires organizing messages in intervals, and devising techniques to identify all messages belonging to each interval. In addition, combining messages from different publishers requires more complex arrangements, involving embedding publisher/message IDs, delayed forwarding (to accommodate more messages), and mechanisms to ensure the decoding of messages at all peers. -Careful application-specific need analysis can help decide the benefits against the added complexity.

    GOAL4: Handling Large Messages​

    Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. -This introduces two challenges:

    1) Redundant large message transmissions result in severe network congestion. -2) Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages.

    The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. +Careful application-specific need analysis can help decide the benefits against the added complexity.

    +

    GOAL4: Handling Large Messages​

    +

    Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. +This introduces two challenges:

    +
      +
    1. Redundant large message transmissions result in severe network congestion.
      2. +
    2. Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages.
      3. +
    +

    The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. Most of the work done for handling large messages involves curtailing redundant transmissions using multicast delivery trees, -reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc.

    Approaches like message aggregation also prove helpful in minimizing bandwidth wastage.

    Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions:

    1. Using IDontWant message proposal [13] and staggered sending.

      IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. +reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc.

      +

      Approaches like message aggregation also prove helpful in minimizing bandwidth wastage.

      +

      Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions:

      +
        +
      1. +

        Using IDontWant message proposal [13] and staggered sending.

        +

        IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. Staggered sending enables relaying the message to a short subset of peers in each round. We argue that simultaneously relaying a message to all peers hampers the effectiveness of the IDontWant message. -Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages.

      2. Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. -We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.

      GOAL5: Scalability​

      P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. -In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \times D bandwidth, -where RR is average data arrival rate. -It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\lceil \log_D (N) \rceil hops. -Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size.

      Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. +Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages.

      +
      2. +
    2. +

      Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. +We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.

      +
      3. +
    +

    GOAL5: Scalability​

    +

    P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. +In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \times D bandwidth, +where RR is average data arrival rate. +It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\lceil \log_D (N) \rceil hops. +Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size.

    +

    Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. Most efforts for bringing scalability to the P2P systems have focused on curtailing redundant transmissions and flat overlay adjustments. -Hierarchical overlay designs, on the other hand, are less explored.

    Placing a logical structure in unstructured P2P systems can help scale P2P networks.

    One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. -An abstract operation of such overlay design is provided below:

    1. Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. -For this purpose, every node tries connecting peers with the lowest latency until DD connections are made or the cluster limit is reached.

    2. A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. +Hierarchical overlay designs, on the other hand, are less explored.

      +

      Placing a logical structure in unstructured P2P systems can help scale P2P networks.

      +

      One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. +An abstract operation of such overlay design is provided below:

      +
        +
      1. +

        Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. +For this purpose, every node tries connecting peers with the lowest latency until DD connections are made or the cluster limit is reached.

        +
        2. +
      2. +

        A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. These super nodes form a fully connected mesh and jointly act as a virtual node, -mitigating the problem of peer churn among super nodes.

      3. Virtual nodes form a fully connected mesh to construct a hierarchical overlay. +mitigating the problem of peer churn among super nodes.

        +
        4. +
      4. +

        Virtual nodes form a fully connected mesh to construct a hierarchical overlay. Each virtual node is essentially a collection of super nodes; -a link to any of the constituent super nodes represents a link to the virtual node.

      5. One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.

      Summary​

      Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. +a link to any of the constituent super nodes represents a link to the virtual node.

      +
      3. +
    3. +

      One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.

      +
      4. +
    +

    Summary​

    +

    Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. On the other hand, a hierarchical overlay can bring scalability at the cost of increased complexity. Regardless of the overlay design, a continuous readjustment to appropriate peering links is essential for superior performance. At the same time, bandwidth preservation (through message aggregation, caching at strategic locations, metadata sharing, pull-based operation, etc.) can help minimize latency. -However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.

    References

    • [1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf
    • [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905
    • [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf
    • [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document
    • [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454
    • [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658
    • [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473
    • [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12
    • [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576
    • [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676
    • [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446
    • [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844
    • [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413
    • [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5
    • [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16
    • [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458
    ]]>     +However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.

    +

    References

    +
      +
    • [1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf
      • +
    • [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905
      • +
    • [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf
      • +
    • [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document
      • +
    • [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454
      • +
    • [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658
      • +
    • [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473
      • +
    • [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12
      • +
    • [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576
      • +
    • [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676
      • +
    • [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446
      • +
    • [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844
      • +
    • [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413
      • +
    • [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5
      • +
    • [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16
      • +
    • [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458
      • +
    ]]> Umar Farooq @@ -1802,148 +4131,714 @@ However, problems like peer churn and in-network adversaries can be best allevia 2023-08-28T12:00:00.000Z - Nescience, a privacy-first blockchain zkVM.

    Introduction​

    Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. + Nescience, a privacy-first blockchain zkVM.

    + +

    Introduction​

    +

    Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. The goals include creating a state separation architecture for public/private computation, designing a versatile virtual machine based on mainstream instruction sets, creating proofs for private state updates, implementing a kernel-based architecture for correct execution of private functions, -and implementing core DeFi protocols such as AMMs and staking from a privacy perspective.

    It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. +and implementing core DeFi protocols such as AMMs and staking from a privacy perspective.

    +

    It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. To achieve this goal, Nescience will implement a versatile virtual machine that can be used to implement existing blockchain applications, -while also enabling the development of privacy-centric protocols such as private staking and private DEXs.

    To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, -while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover.

    It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. +while also enabling the development of privacy-centric protocols such as private staking and private DEXs.

    +

    To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, +while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover.

    +

    It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. Finally, Nescience intends to implement permissive licensing, which means that the source code will be open-source, -and developers will be able to use and modify the code without any restriction.

    Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, +and developers will be able to use and modify the code without any restriction.

    +

    Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, potential solutions, and alternatives that lay ahead. Each step is a testament to our commitment to thoroughness; we systematically test various possibilities and decisively commit to the one that demonstrates paramount performance and utility. For instance, as we progress towards achieving Goal 2, we are undertaking a rigorous benchmarking of the Nova proof system against its contemporaries. Should Nova showcase superior performance metrics, we stand ready to integrate it as our proof system of choice. Through such meticulous approaches, -we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.

    Goal 1: Create a State Separation Architecture​

    The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, -employing an account-based framework for the public state and a UTXO-based structure for the private state.

    The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, +we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.

    +

    Goal 1: Create a State Separation Architecture​

    +

    The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, +employing an account-based framework for the public state and a UTXO-based structure for the private state.

    +

    The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, while the account-based paradigm assigns balances to accounts that transactions can modify. Although the UTXO model bolsters privacy by concealing comprehensive balances, the pursuit of a dual architecture mandates a meticulous synchronization of these state models, -ensuring that private transactions remain inconspicuous in the wider public network state.

    This task is further complicated by the divergent transaction processing methods intrinsic to each model, +ensuring that private transactions remain inconspicuous in the wider public network state.

    +

    This task is further complicated by the divergent transaction processing methods intrinsic to each model, necessitating a thoughtful and innovative approach to harmonize their functionality. To seamlessly bring together the dual architecture, harmonizing the account-based model for public state with the UTXO-based model for private state, -a comprehensive strategy is essential.

    The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. -It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model.

    Here's a breakdown and a potential strategy for harmonizing these models:

    Rationale Behind the Dual Architecture: ​

    • Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, -and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications.

    • UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. -By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.

    Harmonizing the Two Systems: ​

    1. Translation Layer

      • Role: Interface between UTXO and account-based states.

      • UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. +a comprehensive strategy is essential.

        +

        The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. +It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model.

        +

        Here's a breakdown and a potential strategy for harmonizing these models:

        +

        Rationale Behind the Dual Architecture: ​

        +
          +
        • +

          Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, +and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications.

          +
          • +
        • +

          UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. +By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.

          +
          • +
        +

        Harmonizing the Two Systems: ​

        +
          +
        1. +

          Translation Layer

          +
            +
          • +

            Role: Interface between UTXO and account-based states.

            +
            • +
          • +

            UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. This could involve creating a temporary 'pseudo-account' that mirrors the -UTXO's attributes.

          • Account-to-UTXO Adapter: When an account wishes to make a private transaction, -it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction.

        2. Unified Identity Management

          • Role: Maintain a unified identity (or address) system that works across both state models, -allowing users to easily manage their public and private states without requiring separate identities.

          • Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. -This ensures privacy while keeping management centralized for the user.

        1. State Commitments

          • Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions.

          • Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. -Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies.

          • Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy.

        1. Batch Processing & Anonymity Sets

          • Role: Group several UTXO-based private transactions into a single public account-based transaction. -This can provide a level of obfuscation and can make synchronization between the two models more efficient.

          • CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy.

          • Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, -providing zk-SNARKs-based private transactions.

        2. Event Hooks & Smart Contracts

          • Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. -For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model.

          • Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, -a smart contract might release funds (account-based) once a specific UTXO is spent.

          • Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, -allowing for private logic execution.

        Challenges and Solutions ​

        1. Synchronization Overhead

          • Challenge: Combining two distinct transaction models creates an inherent synchronization challenge.

          • State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. -Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required.

          • Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. -This structure helps reduce the immediate load on the primary system.

          • Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. -This can ensure consistency without constant synchronization.

        2. Double Spending

          • Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks.

          • Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements.

          • Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. -If discrepancies arise, the transaction can be halted.

          • Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending.

        3. Complexity in User Experience

          • Challenge: The dual model, while powerful, is inherently complex.

          • Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, -allowing users to make transactions without needing to understand the nuances of the dual model.

          • Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, -especially emphasizing when and why they might choose one transaction type over the other.

          • Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. -This real-time feedback can be invaluable for iterative design improvements.

        4. Security

          • Challenge: Merging two systems can introduce unforeseen vulnerabilities.

          • Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, -especially those that might exploit the interaction between the two systems.

          • Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. -Each layer can act as a fail-safe if a potential threat bypasses another.

          • Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. -If any suspicious activity is detected, they can take corrective measures or raise alerts.

        5. Gas & Fee Management:

          • Challenge: A dual model can lead to convoluted fee structures.

          • Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. -This can ensure fairness and prevent network abuse.

          • Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. -This helps users understand potential costs upfront.

          • Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, -simplifying the gas acquisition process.

        By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, -combining the strengths of both UTXO and account-based models without their standalone limitations.

        AspectDetails
        Harmony- Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures.
        Challenges- Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs.
        Proposed Solutions- Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.

        Goal 2: Virtual Machine Creation​

        The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. -Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications.

        This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. -Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures.

        WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, -and boasts seamless integration across diverse blockchain platforms.

        Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, +UTXO's attributes.

        +
        • +
      • +

        Account-to-UTXO Adapter: When an account wishes to make a private transaction, +it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction.

        +
        • +
      +
      2. +
    2. +

      Unified Identity Management

      +
        +
      • +

        Role: Maintain a unified identity (or address) system that works across both state models, +allowing users to easily manage their public and private states without requiring separate identities.

        +
        • +
      • +

        Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. +This ensures privacy while keeping management centralized for the user.

        +
        • +
      +
      3. +
    3. +

      State Commitments

      +
        +
      • +

        Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions.

        +
        • +
      • +

        Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. +Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies.

        +
        • +
      • +

        Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy.

        +
        • +
      +
      4. +
    4. +

      Batch Processing & Anonymity Sets

      +
        +
      • +

        Role: Group several UTXO-based private transactions into a single public account-based transaction. +This can provide a level of obfuscation and can make synchronization between the two models more efficient.

        +
        • +
      • +

        CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy.

        +
        • +
      • +

        Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, +providing zk-SNARKs-based private transactions.

        +
        • +
      +
      5. +
    5. +

      Event Hooks & Smart Contracts

      +
        +
      • +

        Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. +For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model.

        +
        • +
      • +

        Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, +a smart contract might release funds (account-based) once a specific UTXO is spent.

        +
        • +
      • +

        Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, +allowing for private logic execution.

        +
        • +
      +
      6. +
    +

    Challenges and Solutions ​

    +
      +
    1. +

      Synchronization Overhead

      +
        +
      • +

        Challenge: Combining two distinct transaction models creates an inherent synchronization challenge.

        +
        • +
      • +

        State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. +Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required.

        +
        • +
      • +

        Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. +This structure helps reduce the immediate load on the primary system.

        +
        • +
      • +

        Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. +This can ensure consistency without constant synchronization.

        +
        • +
      +
      2. +
    2. +

      Double Spending

      +
        +
      • +

        Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks.

        +
        • +
      • +

        Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements.

        +
        • +
      • +

        Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. +If discrepancies arise, the transaction can be halted.

        +
        • +
      • +

        Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending.

        +
        • +
      +
      3. +
    3. +

      Complexity in User Experience

      +
        +
      • +

        Challenge: The dual model, while powerful, is inherently complex.

        +
        • +
      • +

        Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, +allowing users to make transactions without needing to understand the nuances of the dual model.

        +
        • +
      • +

        Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, +especially emphasizing when and why they might choose one transaction type over the other.

        +
        • +
      • +

        Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. +This real-time feedback can be invaluable for iterative design improvements.

        +
        • +
      +
      4. +
    4. +

      Security

      +
        +
      • +

        Challenge: Merging two systems can introduce unforeseen vulnerabilities.

        +
        • +
      • +

        Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, +especially those that might exploit the interaction between the two systems.

        +
        • +
      • +

        Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. +Each layer can act as a fail-safe if a potential threat bypasses another.

        +
        • +
      • +

        Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. +If any suspicious activity is detected, they can take corrective measures or raise alerts.

        +
        • +
      +
      5. +
    5. +

      Gas & Fee Management:

      +
        +
      • +

        Challenge: A dual model can lead to convoluted fee structures.

        +
        • +
      • +

        Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. +This can ensure fairness and prevent network abuse.

        +
        • +
      • +

        Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. +This helps users understand potential costs upfront.

        +
        • +
      • +

        Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, +simplifying the gas acquisition process.

        +
        • +
      +
      6. +
    +

    By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, +combining the strengths of both UTXO and account-based models without their standalone limitations.

    +
    AspectDetails
    Harmony- Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures.
    Challenges- Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs.
    Proposed Solutions- Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.
    +

    Goal 2: Virtual Machine Creation​

    +

    The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. +Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications.

    +

    This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. +Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures.

    +

    WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, +and boasts seamless integration across diverse blockchain platforms.

    +

    Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, elevating the levels of security and privacy. However, it is crucial to acknowledge that employing WASM or RISC-V might present challenges, -given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs).

    Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, +given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs).

    +

    Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, while potentially addressing optimization concerns, necessitate extensive research and testing due to their relatively nascent status within the field. -This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance.

    The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. +This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance.

    +

    The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. The combination of two renowned instruction sets, WASM and RISC-V, in tandem with ZK, is an innovation that could redefine privacy standards in blockchain. -Let's dissect the challenges and possibilities inherent in this goal:

    1. Established Mainstream Instruction Sets - WASM and RISC-V

      • Strengths:

        • WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. -Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape.

        • RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. +Let's dissect the challenges and possibilities inherent in this goal:

          +
            +
          1. +

            Established Mainstream Instruction Sets - WASM and RISC-V

            +
              +
            • +

              Strengths:

              +
                +
              • +

                WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. +Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape.

                +
                • +
              • +

                RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. Its adaptability to both UTXO and account-based structures coupled with its compatibility with trusted execution environments like SGX and TEE augments its appeal, -especially in domains that prioritize security and privacy.

            • Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, -they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts.

          1. Pioneering a New, Specialized Instruction Set

            • Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, -potentially offering unmatched performance and optimizations tailored to the project's requirements.

            • Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. -It would also need to garner community trust and support over time.

          1. Contemporary Proof Systems - (Super)Nova, STARKs, Sangria

            • Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, -they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization.

            • Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. +especially in domains that prioritize security and privacy.

              +
              • +
            +
            2. +
          2. +

            Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, +they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts.

            +
            3. +
        +
        • +
      • +

        Pioneering a New, Specialized Instruction Set

        +
          +
        • +

          Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, +potentially offering unmatched performance and optimizations tailored to the project's requirements.

          +
          • +
        • +

          Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. +It would also need to garner community trust and support over time.

          +
          • +
        +
        • +
      • +

        Contemporary Proof Systems - (Super)Nova, STARKs, Sangria

        +
          +
        • +

          Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, +they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization.

          +
          • +
        • +

          Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. The unknowns associated with these systems could introduce unforeseen vulnerabilities or complexities. -While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing.

    Mainstream (WASM, RISC-V)ZK-optimized (New Instruction Set)
    Existing ToolingYESNO
    Blockchain-focusedNOYES
    PerformantDEPENDSYES

    Optimization Concerns for WASM and RISC-V: ​

    • Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. -Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed.

    • Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. -Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently.

    • Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. -Fine-tuned memory management mechanisms are essential to prevent bottlenecks.

    Emerging ZKP Optimized Systems Considerations: ​

    • Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. -The trade-offs between proof size, computational efficiency, and security need to be balanced.

    • Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. -A universal system can be more versatile for diverse applications on the blockchain.

    • Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. -Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.

    Strategies for Integration: ​

    • Iterative Development: Given the complexities, an iterative development approach can be beneficial. -Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities.

    • Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization.

    • External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.

    Goal 3: Proofs Creation and Verification​

    The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. +While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing.

    + + + + +
    Mainstream (WASM, RISC-V)ZK-optimized (New Instruction Set)
    Existing ToolingYESNO
    Blockchain-focusedNOYES
    PerformantDEPENDSYES
    +

    Optimization Concerns for WASM and RISC-V: ​

    +
      +
    • +

      Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. +Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed.

      +
      • +
    • +

      Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. +Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently.

      +
      • +
    • +

      Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. +Fine-tuned memory management mechanisms are essential to prevent bottlenecks.

      +
      • +
    +

    Emerging ZKP Optimized Systems Considerations: ​

    +
      +
    • +

      Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. +The trade-offs between proof size, computational efficiency, and security need to be balanced.

      +
      • +
    • +

      Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. +A universal system can be more versatile for diverse applications on the blockchain.

      +
      • +
    • +

      Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. +Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.

      +
      • +
    +

    Strategies for Integration: ​

    +
      +
    • +

      Iterative Development: Given the complexities, an iterative development approach can be beneficial. +Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities.

      +
      • +
    • +

      Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization.

      +
      • +
    • +

      External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.

      +
      • +
    +

    Goal 3: Proofs Creation and Verification​

    +

    The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. Concurrently, the responsibility of verifying these proofs and executing public functions within the virtual machine can be effectively delegated to an external prover, a role that is incentivized to operate with utmost honesty and integrity. This intricate balance seeks to safeguard against information leakage, -preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework.

    To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. +preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework.

    +

    To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. This two-pronged approach serves as a compelling deterrent against dishonest behavior and fosters an environment of accountability. In addition to incentivization, a crucial consideration is the economic aspect of verification and execution. -The verification process has been intentionally designed to be more cost-effective than execution.

    This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. +The verification process has been intentionally designed to be more cost-effective than execution.

    +

    This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. By maintaining a cost balance that favors verification, we bolster the system’s resilience against fraudulent activities while ensuring its efficiency. In sum, our multifaceted approach endeavors to strike an intricate equilibrium between user-initiated proof creation, external verification, and incentivization. -This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability.

    As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, +This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability.

    +

    As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, and the overall integrity of the system is fortified against potential adversarial actions. To achieve the goals of user-initiated proof creation, -external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed:

    1. User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies.

      • Challenges:

        • Maintaining the quality and integrity of the proofs generated by users.

        • Ensuring that users have the tools and knowledge to produce valid proofs.

      • Solutions:

        • Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process.

        • Implement checks at the verifier's end to ensure the quality of proofs.

    1. External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine.

      • Challenges:

        • Ensuring that the external prover acts honestly.

        • Avoiding centralized points of failure.

      • Solutions:

        • Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work.

        • Use reputation systems to rank provers based on their past performances, creating a trust hierarchy.

    2. Incentivization Framework: A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task.

      • Challenges:

        • Determining the right balance of rewards and penalties.

        • Ensuring that the system cannot be gamed for undue advantage.

      • Solutions1:

        • Implement a dynamic reward system that adjusts based on network metrics and provers' performance.

        • Use a staking mechanism where provers need to lock up a certain amount of assets. -Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets.

    3. Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks.

      • Challenges:

        • Setting the right cost metrics for both verification and execution.

        • Ensuring that genuine users aren’t priced out of the system.

      • Solutions:

        • Use a dynamic pricing model, adjusting costs in real-time based on network demand.

        • Implement gas-like mechanisms to differentiate operation costs and ensure fairness.

    4. Maintaining Trustworthiness: Create a system that's transparent, holds all actors accountable, and is economically sound.

      • Challenges:

        • Keeping the balance where users feel their privacy is intact, while provers feel incentivized.

        • Ensuring the system remains resilient against adversarial attacks.

      • Solutions:

        • Implement layered checks and balances.

        • Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO).

    Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, +external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed:

    +
      +
    1. +

      User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Maintaining the quality and integrity of the proofs generated by users.

          +
          • +
        • +

          Ensuring that users have the tools and knowledge to produce valid proofs.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process.

          +
          • +
        • +

          Implement checks at the verifier's end to ensure the quality of proofs.

          +
          • +
        +
        • +
      +
      2. +
    2. +

      External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Ensuring that the external prover acts honestly.

          +
          • +
        • +

          Avoiding centralized points of failure.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work.

          +
          • +
        • +

          Use reputation systems to rank provers based on their past performances, creating a trust hierarchy.

          +
          • +
        +
        • +
      +
      3. +
    3. +

      ** Incentivization Framework:** A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Determining the right balance of rewards and penalties.

          +
          • +
        • +

          Ensuring that the system cannot be gamed for undue advantage.

          +
          • +
        +
        • +
      • +

        Solutions1:

        +
          +
        • +

          Implement a dynamic reward system that adjusts based on network metrics and provers' performance.

          +
          • +
        • +

          Use a staking mechanism where provers need to lock up a certain amount of assets. +Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets.

          +
          • +
        +
        • +
      +
      4. +
    4. +

      Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Setting the right cost metrics for both verification and execution.

          +
          • +
        • +

          Ensuring that genuine users aren’t priced out of the system.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Use a dynamic pricing model, adjusting costs in real-time based on network demand.

          +
          • +
        • +

          Implement gas-like mechanisms to differentiate operation costs and ensure fairness.

          +
          • +
        +
        • +
      +
      5. +
    5. +

      ** Maintaining Trustworthiness:** Create a system that's transparent, holds all actors accountable, and is economically sound.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Keeping the balance where users feel their privacy is intact, while provers feel incentivized.

          +
          • +
        • +

          Ensuring the system remains resilient against adversarial attacks.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Implement layered checks and balances.

          +
          • +
        • +

          Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO).

          +
          • +
        +
        • +
      +
      6. +
    +

    Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, cost dynamics, and verification integrity. A thoughtful combination of these mechanisms ensures that the system remains robust, resilient, -and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation.

    AspectDetails
    Design Principle- User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions.
    Trust & Privacy- Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage.
    Incentivization Framework- Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior.
    Economic Considerations- Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency.
    OutcomeAn ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.

    Goal 4: Kernel-based Architecture Implementation​

    This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. +and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation.

    +
    AspectDetails
    Design Principle- User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions.
    Trust & Privacy- Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage.
    Incentivization Framework- Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior.
    Economic Considerations- Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency.
    OutcomeAn ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.
    +

    Goal 4: Kernel-based Architecture Implementation​

    +

    This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. This innovative approach employs recursion to construct a call stack, which is then validated through iterative recursive computations. -At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack.

    The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. -While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies.

    The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. +At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack.

    +

    The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. +While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies.

    +

    The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. Moreover, the iterative recursive computations could potentially exhibit computational expansion as the depth of recursion increases. -This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail.

    In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. +This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail.

    +

    In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. By weaving recursion and iterative recursive computations into the fabric of our system, we aim to establish a mechanism that accentuates the trustworthiness of private function executions, -while conscientiously navigating the computational demands that ensue.

    To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, -several strategic steps and considerations can be undertaken: recursion handling and depth management.

    Recursion Handling

    • Call Stack Management:

      • Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state.

    • Proof Accumulation:

      • Design a mechanism to accumulate proof data for each function call within the call stack. -This includes cryptographic commitments, intermediate results, and cryptographic challenges.

      • Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process.

    • Intermediary SNARK Proofs:

      • Develop an intermediary SNARK proof for each function call’s correctness within the call stack. -This proof should demonstrate that the function executed correctly and produced expected outputs.

      • Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack.

    Depth management

    • Depth Limitation:

      • Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations.

      • Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth.

    • Graceful Degradation:

      • Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. -This may involve transitioning to alternative execution modes or optimization techniques.

      • Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed.

    • Resource Monitoring:

      • Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. -Provide real-time feedback to users about the cost and impact of recursive execution.

    • Dynamic Depth Adjustment:

      • Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources.

      • Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints.

    • Fallback Mechanisms:

      • Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. -These mechanisms could involve alternative verification methods or delayed execution.

    • User Notifications:

      • Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage.

    Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. +while conscientiously navigating the computational demands that ensue.

    +

    To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, +several strategic steps and considerations can be undertaken: recursion handling and depth management.

    + Recursion Handling +
      +
    • +

      Call Stack Management:

      +
        +
      • Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state.
        • +
      +
      • +
    • +

      _Proof Accumulation: _

      +
        +
      • +

        Design a mechanism to accumulate proof data for each function call within the call stack. +This includes cryptographic commitments, intermediate results, and cryptographic challenges.

        +
        • +
      • +

        Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process.

        +
        • +
      +
      • +
    • +

      Intermediary SNARK Proofs:

      +
        +
      • +

        Develop an intermediary SNARK proof for each function call’s correctness within the call stack. +This proof should demonstrate that the function executed correctly and produced expected outputs.

        +
        • +
      • +

        Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack.

        +
        • +
      +
      • +
    + Depth management +
      +
    • +

      Depth Limitation:

      +
        +
      • +

        Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations.

        +
        • +
      • +

        Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth.

        +
        • +
      +
      • +
    • +

      Graceful Degradation:

      +
        +
      • +

        Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. +This may involve transitioning to alternative execution modes or optimization techniques.

        +
        • +
      • +

        Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed.

        +
        • +
      +
      • +
    • +

      Resource Monitoring:

      +
        +
      • Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. +Provide real-time feedback to users about the cost and impact of recursive execution.
        • +
      +
      • +
    • +

      Dynamic Depth Adjustment:

      +
        +
      • +

        Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources.

        +
        • +
      • +

        Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints.

        +
        • +
      +
      • +
    • +

      Fallback Mechanisms:

      +
        +
      • Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. +These mechanisms could involve alternative verification methods or delayed execution.
        • +
      +
      • +
    • +

      User Notifications:

      +
        +
      • Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage.
        • +
      +
      • +
    +

    Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. While the approach promises robust outcomes, it's pivotal to maneuver through its intricacies with astute strategies, ensuring computational efficiency and economic viability. -By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.

    Goal 5: Seamless Interaction Design​

    Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. -This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions.

    A notable challenge in this endeavor lies in the intricate interplay between public and private states, -wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage.

    The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. -This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality.

    A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. +By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.

    +

    Goal 5: Seamless Interaction Design​

    +

    Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. +This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions.

    +

    A notable challenge in this endeavor lies in the intricate interplay between public and private states, +wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage.

    +

    The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. +This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality.

    +

    A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. The solution involves devising strategies to mitigate the risk of revealing private transaction details when connected to corresponding public actions. -By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability.

    Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, -without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design.

    To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. -In the table below, a comprehensive list of solutions that address these objectives:

    Solution CategoryDescription
    Layer 2 SolutionsEmploy zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs.
    Intermediary Smart ContractsCraft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially.
    Decentralized Identity & PseudonymityImplement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs.
    Confidential Sidechains & Cross-ChainSet up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains.
    Temporal Data StructuresCreate chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy.
    Homomorphic Encryption & MPCApply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers.
    Commit-Reveal SchemesIntroduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions.
    Auditability & VerifiabilityUse on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation.
    Data Fragmentation & ShardingFragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography.
    Ring Signatures & CoinJoinIncorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.

    Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework​

    The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. +By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability.

    +

    Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, +without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design.

    +

    To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. +In the table below, a comprehensive list of solutions that address these objectives:

    +
    Solution CategoryDescription
    Layer 2 SolutionsEmploy zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs.
    Intermediary Smart ContractsCraft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially.
    Decentralized Identity & PseudonymityImplement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs.
    Confidential Sidechains & Cross-ChainSet up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains.
    Temporal Data StructuresCreate chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy.
    Homomorphic Encryption & MPCApply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers.
    Commit-Reveal SchemesIntroduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions.
    Auditability & VerifiabilityUse on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation.
    Data Fragmentation & ShardingFragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography.
    Ring Signatures & CoinJoinIncorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.
    +

    Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework​

    +

    The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. This endeavor comes with inherent challenges, especially considering the heterogeneity of existing DeFi protocols, predominantly built on Ethereum. These variations in programming languages and VMs exacerbate the quest for interoperability. Furthermore, the success and functionality of DeFi protocols is closely tied to liquidity, -which in turn is influenced by user engagement and the amount of funds locked into the system.

    Strategic Roadmap for Goal 6 ​

    1. Pioneering Privacy-Centric DeFi Models: Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity.

    2. Specialized Smart Contracts with Privacy: Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape.

    3. Optimized User Interfaces: Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security.

    4. Tackling Interoperability:

      • Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments.

      • Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly.

    1. Enhancing and Sustaining Liquidity:

      • Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space.

      • Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity.

      • Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies.

    1. Amplifying Community Engagement: Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, -thereby nurturing a vibrant, privacy-advocating DeFi community.

    Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, -all while effectively overcoming the obstacles related to interoperability and liquidity concerns.

    Summary of the Architecture​

    In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). +which in turn is influenced by user engagement and the amount of funds locked into the system.

    +

    Strategic Roadmap for Goal 6 ​

    +
      +
    1. +

      ** Pioneering Privacy-Centric DeFi Models: ** Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity.

      +
      2. +
    2. +

      ** Specialized Smart Contracts with Privacy: ** Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape.

      +
      3. +
    3. +

      ** Optimized User Interfaces: ** Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security.

      +
      4. +
    4. +

      ** Tackling Interoperability: **

      +
        +
      • +

        Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments.

        +
        • +
      • +

        Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly.

        +
        • +
      +
      5. +
    5. +

      ** Enhancing and Sustaining Liquidity: **

      +
        +
      • +

        Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space.

        +
        • +
      • +

        Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity.

        +
        • +
      • +

        Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies.

        +
        • +
      +
      6. +
    6. +

      ** Amplifying Community Engagement:** Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, +thereby nurturing a vibrant, privacy-advocating DeFi community.

      +
      7. +
    +

    Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, +all while effectively overcoming the obstacles related to interoperability and liquidity concerns.

    +

    Summary of the Architecture​

    +

    In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). These proofs ensure that while private state data remains undisclosed, public state transitions can still be carried out and subsequently verified by third parties. This blend of public and private state is envisaged to be achieved through a state tree representing the public state, while the encrypted state leaves stand for the private state. Each user's private state indicates validity through the absence of a corresponding nullifier. A nullifier is a unique cryptographic value generated in privacy-preserving blockchain transactions to prevent double-spending, -ensuring that each private transaction is spent only once without revealing its details.

    Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. +ensuring that each private transaction is spent only once without revealing its details.

    +

    Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. For validating a singular private function call, we're leaning into the kernel-based model inspired by the ZEXE protocol. Defined as kernel circuits, these functions validate the correct execution of each private function call. Due to their recursive circuit structure, a succession of private function calls can be executed by calculating proofs in an iterative manner. -Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs.

    Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. -The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities.

    We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. +Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs.

    +

    Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. +The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities.

    +

    We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. Instead of the ZEXE protocol's structure, our approach pivots to a method anchored in the security of a secret combined with the uniqueness from hashing with the contract address. The underlying rationale is straightforward: the sender, equipped with a unique nonce and salt for the transaction, hashes the secret, payload, nonce, and salt. This result is then hashed with the contract address for the final value. The hash function's unidirectional nature ensures that the input cannot be deduced easily from its output. A specific concern, however, is the potential repetition of secret and payload values across transactions, which could jeopardize privacy. -Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation.

    Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. +Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation.

    +

    Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. Fully Homomorphic Encryption (FHE) offers another pathway, enabling function execution on encrypted data, or Multi-Party Computation (MPC) which guarantees non-disclosure of function or inputs. Yet, integrating ZKPs with either FHE or MPC presents a challenge. Combining cryptographic functions like SHA-3 and BLAKE2 can also bolster security and uniqueness. -It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.

    Current State​

    Our aim is to revolutionize the privacy and security paradigms through Nescience. +It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.

    +

    Current State​

    +

    Our aim is to revolutionize the privacy and security paradigms through Nescience. As we strive to set milestones and achieve groundbreaking advancements, -our current focus narrows onto the realization of Goal 2 and Goal 3.

    Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. +our current focus narrows onto the realization of Goal 2 and Goal 3.

    +

    Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. We believe that integrating the right proof system is pivotal to our project's success, which brings us to Nova [8]. In our project journey, we have opted to integrate the Nova proof system, recognizing its potential alignment with our overarching goals. However, as part of our meticulous approach to innovation and optimization, we acknowledge the need to thoroughly examine Nova’s performance capabilities, -particularly due to its status as a pioneering and relatively unexplored proof system.

    This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], +particularly due to its status as a pioneering and relatively unexplored proof system.

    +

    This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], pitting Nova against other prominent proof systems in the field, including Halo2 [10], Plonky2 [11], and Starky [12]. This ongoing and methodical initiative is designed to ensure a fair and impartial assessment, enabling us to draw meaningful conclusions about Nova’s strengths and limitations in relation to its counterparts. @@ -1951,17 +4846,52 @@ By leveraging the Poseidon recursion technique, we are poised to conduct an exha Through this testing framework, we aim to discern whether Nova possesses the potential to outshine its contemporaries in terms of efficiency, scalability, and overall performance. The outcome of this rigorous evaluation will be pivotal in shaping our strategic decisions moving forward. Armed with a comprehensive understanding of Nova’s performance metrics vis-à-vis other proof systems, -we can confidently chart a course that maximizes the benefits of our project’s optimization efforts.

    Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, +we can confidently chart a course that maximizes the benefits of our project’s optimization efforts.

    +

    Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, incentivizing honest behaviour, and ensuring the cost-effective verification of transactions. At the heart of this endeavor is our drive to empower users by allowing them the autonomy of generating proofs for private state updates, thereby reducing dependencies and enhancing privacy. We would like to actively work on providing comprehensive documentation, user-friendly tools, -and tutorials to aid users in this intricate process.

    Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. +and tutorials to aid users in this intricate process.

    +

    Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. Our commitment is further cemented by our efforts to introduce a dynamic reward system that adjusts based on network metrics and prover performance. -This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.

    References

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    [4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    [5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md

    [6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin

    [7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui

    [8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370

    [9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations

    [10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/

    [11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2

    [12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582

    1. Incentive Mechanisms:

      • Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. -This incentivizes participation and encourages integrity.

      • Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. -Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded.

      • Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, -aligning incentives with the blockchain’s broader consensus mechanism.

      ↩

    ]]>     +This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.

    +

    References

    +

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    +

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    +

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    +

    [4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    +

    [5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md

    +

    [6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin

    +

    [7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui

    +

    [8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370

    +

    [9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations

    +

    [10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/

    +

    [11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2

    +

    [12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582

    + +

    Footnotes​

    +
      +
    1. +

      Incentive Mechanisms:

      +
        +
      • +

        Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. +This incentivizes participation and encourages integrity.

        +
        • +
      • +

        Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. +Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded.

        +
        • +
      • +

        Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, +aligning incentives with the blockchain’s broader consensus mechanism.

        +
        • +
      +↩ +
      2. +
    +
    ]]> Moudy @@ -1972,7 +4902,43 @@ aligning incentives with the blockchain’s broader consensus mechanism.

    2023-04-24T12:00:00.000Z - Device pairing and secure message exchange using Waku and noise protocol.

    As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku.

    Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security.

    This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography.

    One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information

    Device Pairing​

    In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access.

    To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake.

    The process works as follows:

    Actors:

    • Alice the initiator
    • Bob the responder
    1. The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated
    2. Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process.
    3. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully
    4. Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker.

    The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.

    Conclusion​

    With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out!

    Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better.

    Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!

    References​

    ]]>     + Device pairing and secure message exchange using Waku and noise protocol.

    + +

    As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku.

    +

    Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security.

    +

    This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography.

    +

    One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information

    +

    Device Pairing​

    +

    In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access.

    +

    To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake.

    +

    The process works as follows:

    +

    Actors:

    +
      +
    • Alice the initiator
      • +
    • Bob the responder
      • +
    +
      +
    1. The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated
      2. +
    2. Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process.
      3. +
    3. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully
      4. +
    4. Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker.
      5. +
    +

    The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.

    +

    Conclusion​

    +

    With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out!

    +

    Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better.

    +

    Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!

    +

    References​

    +]]>     Richard @@ -1983,44 +4949,96 @@ aligning incentives with the blockchain’s broader consensus mechanism.

    2023-04-03T00:00:00.000Z - Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like.

    Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an + Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like.

    + +

    Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an overview of recent discussions and research outputs, aiming to give you a better understanding of how the Waku network -may look like in terms of scaling and incentivization.

    DOS Mitigation for Status Communities​

    Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN +may look like in terms of scaling and incentivization.

    +

    DOS Mitigation for Status Communities​

    +

    Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN (Rate Limiting Nullifiers) remains the go-to DOS protection solution due to its privacy-preserving and censorship-resistant properties, there is still more work to be done. We are excited to collaborate with PSE -(Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.

    A Heterogeneous Waku Network​

    As we noted in a previous forum post, Waku's protocol +(Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.

    +

    A Heterogeneous Waku Network​

    +

    As we noted in a previous forum post, Waku's protocol incentivization model needs to be flexible to accommodate various business models. Flexibility ensures that projects -can choose how they want to use Waku based on their specific needs.

    Reversing the Incentivization Question​

    Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to -reframe the question and instead ask, "How do we pay for the infrastructure?"

    Waku does not intend to offer a free lunch. +can choose how they want to use Waku based on their specific needs.

    +

    Reversing the Incentivization Question​

    +

    Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to +reframe the question and instead ask, "How do we pay for the infrastructure?"

    +

    Waku does not intend to offer a free lunch. Ethereum's infrastructure is supported by transaction fees and inflation, with validators receiving rewards from both sources. However, this model does not suit a communication network like Waku. Users and platforms would not want to pay for every single message they send. Additionally, Waku aims to support instant -ephemeral messages that do not require consensus or long-term storage.

    Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or +ephemeral messages that do not require consensus or long-term storage.

    +

    Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or inter-wallet communication, may have different value extraction models. Some users might provide services for the project and expect to receive value by running nodes, while others may pay for the product or run infrastructure to contribute back. Waku aims to support each of these use cases, which means there will be various ways to "pay for the -infrastructure."

    In his talk, Oskar addressed two strategies: RLN and service credentials.

    RLN and Service Credentials​

    RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, -and you can send messages.

    Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in -stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community.

    Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, +infrastructure."

    +

    In his talk, Oskar addressed two strategies: RLN and service credentials.

    +

    RLN and Service Credentials​

    +

    RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, +and you can send messages.

    +

    Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in +stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community.

    +

    Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, a community owner could pay for more storage or bandwidth, similar to Slack's model. Alternatively, individual users could contribute financially, akin to Discord's Server Boost, or by sharing their own -resources with their community.

    We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.

    Waku Network: Ethereum or Cosmos?​

    Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos.

    For those not familiar with the difference between both, in a very concise manner:

    • Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure
    • Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties

    We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. +resources with their community.

    +

    We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.

    +

    Waku Network: Ethereum or Cosmos?​

    +

    Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos.

    +

    For those not familiar with the difference between both, in a very concise manner:

    +
      +
    • Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure
      • +
    • Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties
      • +
    +

    We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. If each application has to deploy its own network, we will not achieve this goal. Therefore, we aim Waku to be not only an open source set of protocols, but also a shared infrastructure that anyone can leverage to build applications on top, with some guarantees in terms of decentralization and anonymity. This approach is closer in spirit to Ethereum than Cosmos. -Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network.

    Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. +Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network.

    +

    Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. We currently assume that there will be separate gossipsub networks with different funding models. -Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property.

    Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.

    Scalability and Discovery Protocols​

    To enable scalability, the flow of messages in the Waku network will be divided in shards, +Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property.

    +

    Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.

    +

    Scalability and Discovery Protocols​

    +

    To enable scalability, the flow of messages in the Waku network will be divided in shards, so that not every node has to forward every message of the whole network. -Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in.

    Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between).

    Marketplace protocols may also be developed to help operators understand how they can best support the network and where -their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.

    Open Problems​

    Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. -This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues.

    However, it does not cap traffic. -There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection.

    We have several ongoing initiatives:

    • Analyzing the Status Community protocol to confirm efficient usage of Waku [4]
    • Simulating the Waku Network to measure actual bandwidth usage [5]
    • Segregating chat messages from control and media messages [6]

    The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.

    The Evolving Waku Network​

    The definition of the "Waku Network" will likely change over time. In the near future, it will transition from a single +Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in.

    +

    Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between).

    +

    Marketplace protocols may also be developed to help operators understand how they can best support the network and where +their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.

    +

    Open Problems​

    +

    Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. +This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues.

    +

    However, it does not cap traffic. +There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection.

    +

    We have several ongoing initiatives:

    +
      +
    • Analyzing the Status Community protocol to confirm efficient usage of Waku [4]
      • +
    • Simulating the Waku Network to measure actual bandwidth usage [5]
      • +
    • Segregating chat messages from control and media messages [6]
      • +
    +

    The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.

    +

    The Evolving Waku Network​

    +

    The definition of the "Waku Network" will likely change over time. In the near future, it will transition from a single gossipsub network to a sharded set of networks unified by a common discovery layer. This change will promote scalability -and allow various payment models to coexist within the Waku ecosystem.

    In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly +and allow various payment models to coexist within the Waku ecosystem.

    +

    In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly maintaining its core principles. As Waku continues to evolve, we expect it to accommodate a diverse range of use cases and business models, all while preserving privacy, resisting censorship, avoiding surveillance, and remaining accessible -to devices with limited resources.

    References​

    1. WAKU2-RELAY-SHARDING
    2. 57/STATUS-Simple-Scaling
    3. RLN-V2
    4. Scaling Status Communities: Potential Problems
    5. Waku Network Testing
    6. WAKU2-RELAY-SHARDING: Control Message Shards
    ]]>     +to devices with limited resources.

    +

    References​

    +
      +
    1. WAKU2-RELAY-SHARDING
      2. +
    2. 57/STATUS-Simple-Scaling
      3. +
    3. RLN-V2
      4. +
    4. Scaling Status Communities: Potential Problems
      5. +
    5. Waku Network Testing
      6. +
    6. WAKU2-RELAY-SHARDING: Control Message Shards
      7. +
    ]]> Franck @@ -2035,49 +5053,147 @@ to devices with limited resources.

    Background​

    We have built Waku to be the communication layer for Web3. Waku is a collection of protocols to chose from for your +and infrastructure projects. We have prioritized our effort to make Waku usable on various platforms and environments.

    + +

    Background​

    +

    We have built Waku to be the communication layer for Web3. Waku is a collection of protocols to chose from for your messaging needs. It enables secure, censorship-resistant, privacy-preserving, spam-protected communication for its user. -It is designed to run on any device, from mobile to the cloud.

    Waku is available on many systems and environments and used by several applications and SDKs for decentralized communications.

    This involved research efforts in various domains: conversational security, protocol incentivization, zero-knowledge, -etc.

    Waku uses novel technologies. Hence, we knew that early dogfooding of Waku was necessary. Even if research -was still in progress [1]. Thus, as soon as Waku protocols and software were usable, we started to push -for the adoption of Waku. This started back in 2021.

    Waku is the communication component of the Web3 trifecta. This trifecta was Ethereum (contracts), Swarm +It is designed to run on any device, from mobile to the cloud.

    +

    Waku is available on many systems and environments and used by several applications and SDKs for decentralized communications.

    +

    This involved research efforts in various domains: conversational security, protocol incentivization, zero-knowledge, +etc.

    +

    Waku uses novel technologies. Hence, we knew that early dogfooding of Waku was necessary. Even if research +was still in progress [1]. Thus, as soon as Waku protocols and software were usable, we started to push +for the adoption of Waku. This started back in 2021.

    +

    Waku is the communication component of the Web3 trifecta. This trifecta was Ethereum (contracts), Swarm (storage) and Whisper (communication). Hence, it made sense to first target dApps which already uses one of the pillars: -Ethereum.

    As most dApps are web apps, we started the development of js-waku for the browser.

    Once ready, we reached out to dApps to integrate Waku, added prizes to hackathons -and gave talks.

    We also assumed we would see patterns in the usage of Waku, that we would facilitate with the help of -SDKs.

    Finally, we created several web apps: +Ethereum.

    +

    As most dApps are web apps, we started the development of js-waku for the browser.

    +

    Once ready, we reached out to dApps to integrate Waku, added prizes to hackathons +and gave talks.

    +

    We also assumed we would see patterns in the usage of Waku, that we would facilitate with the help of +SDKs.

    +

    Finally, we created several web apps: examples -and PoCs.

    By discussing with Waku users and watching it being used, we learned a few facts:

    1. The potential use cases for Waku are varied and many:
    • Wallet <> dApp communication: WalletConnect, XMTP
    • Off-chain (and private) marketplace: +and PoCs.

      +

      By discussing with Waku users and watching it being used, we learned a few facts:

      +
        +
      1. The potential use cases for Waku are varied and many:
        2. +
      +
      1. Many projects are interested in having an embedded chat in their dApp,
      2. There are complex applications that need Waku as a solution. Taking RAILGUN as an example:
      • Web wallet
      • + React Native mobile wallet
      • + NodeJS node/backend.

      (1) means that it is not that easy to create SDKs for common use cases.

      (2) was a clear candidate for an SDK. Yet, building a chat app is a complex task. Hence, the Status app team tackled -this in the form of Status Web.

      Finally, (3) was the most important lesson. We learned that multi-tier applications need Waku for decentralized and +Decentralized Uber

      • +
    • Signature exchange for a multi-sign wallet: Gnosis Safe x Waku
      • +
    • Off-chain Game moves/actions: Super Card Game (EthOnline 2021)
      • +
    • Decentralized Pastebin: Debin
      • +
    +
      +
    1. Many projects are interested in having an embedded chat in their dApp,
      2. +
    2. There are complex applications that need Waku as a solution. Taking RAILGUN as an example:
      3. +
    +
      +
    • Web wallet
      • +
    • + React Native mobile wallet
      • +
    • + NodeJS node/backend.
      • +
    +

    (1) means that it is not that easy to create SDKs for common use cases.

    +

    (2) was a clear candidate for an SDK. Yet, building a chat app is a complex task. Hence, the Status app team tackled +this in the form of Status Web.

    +

    Finally, (3) was the most important lesson. We learned that multi-tier applications need Waku for decentralized and censorship-resistant communications. For these projects, js-waku is simply not enough. They need Waku to work in their -Golang backend, Unity desktop game and React Native mobile app.

    We understood that we should see the whole Waku software suite +Golang backend, Unity desktop game and React Native mobile app.

    +

    We understood that we should see the whole Waku software suite (js-waku, nwaku, go-waku, waku-react-native, etc) as an asset for its success. -That we should not limit outreach, marketing, documentation efforts to the web, but target all platforms.

    From a market perspective, we identified several actors:

    • platforms: Projects that uses Waku to handle communication,
    • operators: Operators run Waku nodes and are incentivized to do so,
    • developers: Developers are usually part of a platforms or solo hackers learning Web3,
    • contributors: Developers and researchers with interests in decentralization, privacy, censorship-resistance, -zero-knowledge, etc.

    Waku for All Decentralized Applications and Infrastructures​

    In 2022, we shifted our focus to make the various Waku implementations usable and used.

    We made Waku multi-plaform.

    We shifted Waku positioning to leverage all Waku implementations and better serve the user's needs:

    We are consolidating the documentation for all implementations on a single website (work in progress) -to improve developer experience.

    This year, we also started the operator outreach effort to push for users to run their own Waku nodes. We have +That we should not limit outreach, marketing, documentation efforts to the web, but target all platforms.

    +

    From a market perspective, we identified several actors:

    +
      +
    • platforms: Projects that uses Waku to handle communication,
      • +
    • operators: Operators run Waku nodes and are incentivized to do so,
      • +
    • developers: Developers are usually part of a platforms or solo hackers learning Web3,
      • +
    • contributors: Developers and researchers with interests in decentralization, privacy, censorship-resistance, +zero-knowledge, etc.
      • +
    +

    Waku for All Decentralized Applications and Infrastructures​

    +

    In 2022, we shifted our focus to make the various Waku implementations usable and used.

    +

    We made Waku multi-plaform.

    +

    We shifted Waku positioning to leverage all Waku implementations and better serve the user's needs:

    + +

    We are consolidating the documentation for all implementations on a single website (work in progress) +to improve developer experience.

    +

    This year, we also started the operator outreach effort to push for users to run their own Waku nodes. We have recently concluded our first operator trial run. Nwaku's documentation, stability and performance has improved. It is now easier to -run your own Waku node.

    Today, operator wannabes most likely run their own nodes to support or use the Waku network. +run your own Waku node.

    +

    Today, operator wannabes most likely run their own nodes to support or use the Waku network. We are dogfooding Waku RLN, our novel economic spam protection protocol, and looking at incentivizing the Waku Store protocol. -This way, we are adding reasons to run your own Waku node.

    For those who were following us in 2021, know that we are retiring the Waku Connect branding in favour of the Waku -branding.

    Waku for Your Project​

    As discussed, Waku is now available on various platforms. The question remains: How can Waku benefit your project?

    Here are a couple of use cases we recently investigated:

    Layer-2 Decentralization​

    Most ([2] [3] roll-ups use a centralized sequencer or equivalent. Running several sequencers is not as straightforward as running several execution nodes. -Waku can help:

    • Provide a neutral marketplace for a mempool: If sequencers compete for L2 tx fees, they may not be incentivized to +This way, we are adding reasons to run your own Waku node.

      +

      For those who were following us in 2021, know that we are retiring the Waku Connect branding in favour of the Waku +branding.

      +

      Waku for Your Project​

      +

      As discussed, Waku is now available on various platforms. The question remains: How can Waku benefit your project?

      +

      Here are a couple of use cases we recently investigated:

      +

      Layer-2 Decentralization​

      +

      Most ([2] [3] roll-ups use a centralized sequencer or equivalent. Running several sequencers is not as straightforward as running several execution nodes. +Waku can help:

      +
        +
      • Provide a neutral marketplace for a mempool: If sequencers compete for L2 tx fees, they may not be incentivized to share transactions with other sequencers. Waku nodes can act as a neutral network to enable all sequences to access -transactions.
      • Enable censorship-resistant wallet<>L2 communication,
      • Provide rate limiting mechanism for spam protection: Using RLN to prevent DDOS.

      Device pairing and communication​

      With Waku Device Pairing, a user can setup a secure encrypted communication channel +transactions.

      • +
    • Enable censorship-resistant wallet<>L2 communication,
      • +
    • Provide rate limiting mechanism for spam protection: Using RLN to prevent DDOS.
      • +
    +

    Device pairing and communication​

    +

    With Waku Device Pairing, a user can setup a secure encrypted communication channel between their devices. As this channel would operate over Waku, it would be censorship-resistant and privacy preserving. -These two devices could be:

    • Ethereum node and mobile phone to access a remote admin panel,
    • Alice's phone and Bob's phone for any kind of secure communication,
    • Mobile wallet and desktop/browser dApp for transaction and signature exchange.

    Check js-waku#950 for the latest update on this.

    Get Involved​

    Developer? Grab any of the Waku implementations and integrate it in your app: https://waku.org/platform.

    Researcher? See https://vac.dev/contribute to participate in Waku research.

    Tech-savvy? Try to run your own node: https://waku.org/operator.

    Otherwise, play around with the various web examples.

    If you want to help, we are hiring!

    Moving Forward​

    What you can expect next:

    • Scalability and performance studies and improvement across Waku software,
    • New websites to easily find documentation about Waku and its implementations,
    • New Waku protocols implemented in all code bases and cross client PoCs +These two devices could be:

      +
        +
      • Ethereum node and mobile phone to access a remote admin panel,
        • +
      • Alice's phone and Bob's phone for any kind of secure communication,
        • +
      • Mobile wallet and desktop/browser dApp for transaction and signature exchange.
        • +
      +

      Check js-waku#950 for the latest update on this.

      +

      Get Involved​

      +

      Developer? Grab any of the Waku implementations and integrate it in your app: https://waku.org/platform.

      +

      Researcher? See https://vac.dev/contribute to participate in Waku research.

      +

      Tech-savvy? Try to run your own node: https://waku.org/operator.

      +

      Otherwise, play around with the various web examples.

      +

      If you want to help, we are hiring!

      +

      Moving Forward​

      +

      What you can expect next:

      +

      References​

      • [1] Waku is modular; it is a suite of protocols; hence some Waku protocols may be mature, while +waku-rln-relay, etc),
        • +
      • Easier to run your own waku node, more operator trials,
        • +
      • Dogfooding and Improvement of existing protocols (e.g. Waku Filter),
        • +
      • Continue our focus Waku portability: Browser, +Raspberry Pi Zero and other restricted-resource environments,
        • +
      • More communication & marketing effort around Waku and the Waku developer community.
        • +
      +
      +

      References​

      +
        +
      • [1] Waku is modular; it is a suite of protocols; hence some Waku protocols may be mature, while new protocols are still being designed. Which means that research continues to be ongoing while -Waku is already used in production.
      • [2] The Optimism Foundation runs the only block produce on the Optimism network.
      • [3] Top 10 L2s are documented has having a centralized operator.
      ]]> +Waku is already used in production.
      • +
    • [2] The Optimism Foundation runs the only block produce on the Optimism network.
      • +
    • [3] Top 10 L2s are documented has having a centralized operator.
      • +
    ]]> Franck @@ -2088,202 +5204,298 @@ Waku is already used in production.
  • 2022-11-04T12:00:00.000Z - What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    This write-up is based on a talk given at DevCon 6 in Bogota, a video can be found here

    Intro​

    In this write-up, we are going to talk about building privacy-protecting -infrastructure. What is it, why do we need it and how can we build it?

    We'll look at Waku, the communication layer for Web3. We'll look at how we are + What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    + +

    This write-up is based on a talk given at DevCon 6 in Bogota, a video can be found here

    +

    Intro​

    +

    In this write-up, we are going to talk about building privacy-protecting +infrastructure. What is it, why do we need it and how can we build it?

    +

    We'll look at Waku, the communication layer for Web3. We'll look at how we are using Zero Knowledge (ZK) technology to incentivize and protect the Waku network. We'll also look at Zerokit, a library we are writing to make ZKP easier -to use in different environments.

    At the end of this write-up, I hope you'll come away with an understanding of -the importance of privacy-protecting infrastructure and how we can build it.

    About​

    First, briefly about Vac. We build public good protocols for the decentralized +to use in different environments.

    +

    At the end of this write-up, I hope you'll come away with an understanding of +the importance of privacy-protecting infrastructure and how we can build it.

    +

    About​

    +

    First, briefly about Vac. We build public good protocols for the decentralized web, with a focus on privacy and communication. We do applied research based on which we build protocols, libraries and publications. We are also the custodians -of protocols that reflect a set of principles.

    Principles

    It has its origins in the Status app and trying to improve +of protocols that reflect a set of principles.

    +

    Principles

    +

    It has its origins in the Status app and trying to improve the underlying protocols and infrastructure. We build Waku, -among other things.

    Why build privacy-protecting infrastructure?​

    Privacy is the power to selectively reveal yourself. It is a requirement for -freedom and self-determination.

    Just like you need decentralization in order to get censorship-resistance, you -need privacy to enable freedom of expression.

    To build applications that are decentralized and privacy-protecting, you need -the base layer, the infrastructure itself, to have those properties.

    We see this a lot. It is easier to make trade-offs at the application layer than +among other things.

    +

    Why build privacy-protecting infrastructure?​

    +

    Privacy is the power to selectively reveal yourself. It is a requirement for +freedom and self-determination.

    +

    Just like you need decentralization in order to get censorship-resistance, you +need privacy to enable freedom of expression.

    +

    To build applications that are decentralized and privacy-protecting, you need +the base layer, the infrastructure itself, to have those properties.

    +

    We see this a lot. It is easier to make trade-offs at the application layer than doing them at the base layer. You can build custodial solutions on top of a decentralized and non-custodial network where participants control their own -keys, but you can't do the opposite.

    If you think about it, buildings can be seen as a form of privacy-protecting +keys, but you can't do the opposite.

    +

    If you think about it, buildings can be seen as a form of privacy-protecting infrastructure. It is completely normal and obvious in many ways, but when it comes to the digital realm our mental models and way of speaking about it hasn't -caught up yet for most people.

    I'm not going too much more into the need for privacy or what happens when you +caught up yet for most people.

    +

    I'm not going too much more into the need for privacy or what happens when you don't have it, but suffice to say it is an important property for any open -society.

    When we have conversations, true peer-to-peer offline conversations, we can talk -privately. If we use cash to buy things we can do commerce privately.

    On the Internet, great as it is, there are a lot of forces that makes this +society.

    +

    When we have conversations, true peer-to-peer offline conversations, we can talk +privately. If we use cash to buy things we can do commerce privately.

    +

    On the Internet, great as it is, there are a lot of forces that makes this natural state of things not the default. Big Tech has turned users into a commodity, a product, and monetized user's attention for advertising. To optimize for your attention they need to surveil your habits and activities, and hence breach your privacy. As opposed to more old-fashioned models, where someone is buying a useful service from a company and the incentives are more -aligned.

    We need to build credibly neutral infrastructure that protects your privacy at +aligned.

    +

    We need to build credibly neutral infrastructure that protects your privacy at the base layer, in order to truly enable applications that are -censorship-resistant and encourage meaningful freedom of expression.

    Web3 infrastructure​

    Infrastructure is what lies underneath. Many ways of looking at this but I'll +censorship-resistant and encourage meaningful freedom of expression.

    +

    Web3 infrastructure​

    +

    Infrastructure is what lies underneath. Many ways of looking at this but I'll keep it simple as per the original Web3 vision. You had Ethereum for compute/consensus, Swarm for storage, and Whisper for messaging. Waku has taken over the mantle from Whisper and is a lot more usable today than Whisper ever was, -for many reasons.

    Web3 Infrastructure

    On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, +for many reasons.

    +

    Web3 Infrastructure

    +

    On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, especially when you try to add privacy back "on top". It takes a lot of effort and it is easier to censor. We see this with recent action around Tornado Cash. Compare this with something like Zcash or Monero, where privacy is there by -default.

    There are also problems when it comes to the p2p networking side of things, for +default.

    +

    There are also problems when it comes to the p2p networking side of things, for example with Ethereum validator privacy and hostile actors and jurisdictions. If someone can easily find out where a certain validator is physically located, that's a problem in many parts of the world. Being able to have stronger -privacy-protection guarantees would be very useful for high-value targets.

    This doesn't begin to touch on the so called "dapps" that make a lot of +privacy-protection guarantees would be very useful for high-value targets.

    +

    This doesn't begin to touch on the so called "dapps" that make a lot of sacrifices in how they function, from the way domains work, to how websites are hosted and the reliance on centralized services for communication. We see this time and time again, where centralized, single points of failure systems work -for a while, but then eventually fail.

    In many cases an individual user might not care enough though, and for platforms +for a while, but then eventually fail.

    +

    In many cases an individual user might not care enough though, and for platforms the lure to take shortcuts is strong. That is why it is important to be principled, but also pragmatic in terms of the trade-offs that you allow on top. -We'll touch more on this in the design goals around modularity that Waku has.

    ZK for privacy-protecting infrastructure​

    ZKPs are a wonderful new tool. Just like smart contracts enables programmable +We'll touch more on this in the design goals around modularity that Waku has.

    +

    ZK for privacy-protecting infrastructure​

    +

    ZKPs are a wonderful new tool. Just like smart contracts enables programmable money, ZKPs allow us to express fundamentally new things. In line with the great tradition of trust-minimization, we can prove statement while revealing the absolute minimum information necessary. This fits the definition of privacy, the power to selectively reveal yourself, perfectly. I'm sure I don't need to tell anyone reading this but this is truly revolutionary. The technology is advancing -extremely fast and often it is our imagination that is the limit.

    Zero knowledge

    Waku​

    What is Waku? It is a set of modular protocols for p2p communication. It has a +extremely fast and often it is our imagination that is the limit.

    +

    Zero knowledge

    +

    Waku​

    +

    What is Waku? It is a set of modular protocols for p2p communication. It has a focus on privacy, security and being able to run anywhere. It is the spiritual -success to Whisper.

    By modular we mean that you can pick and choose protocols and how you use them +success to Whisper.

    +

    By modular we mean that you can pick and choose protocols and how you use them depending on constraints and trade-offs. For example, bandwidth usage vs -privacy.

    It is designed to work in resource restricted environments, such as mobile +privacy.

    +

    It is designed to work in resource restricted environments, such as mobile phones and in web browsers. It is important that infrastructure meets users where they are and supports their real-world use cases. Just like you don't need your own army and a castle to have your own private bathroom, you shouldn't need to have a powerful always-on node to get reasonable privacy and -censorship-resistance. We might call this self-sovereignty.

    Waku - adaptive nodes​

    One way of looking at Waku is as an open service network. There are nodes with +censorship-resistance. We might call this self-sovereignty.

    +

    Waku - adaptive nodes​

    +

    One way of looking at Waku is as an open service network. There are nodes with varying degrees of capabilities and requirements. For example when it comes to bandwidth usage, storage, uptime, privacy requirements, latency requirements, -and connectivity restrictions.

    We have a concept of adaptive nodes that can run a variety of protocols. A node +and connectivity restrictions.

    +

    We have a concept of adaptive nodes that can run a variety of protocols. A node operator can choose which protocols they want to run. Naturally, there'll be some nodes that do more consumption and other nodes that do more provisioning. This gives rise to the idea of a service network, where services are provided -for and consumed.

    Adaptive Nodes

    Waku - protocol interactions​

    There are many protocols that interact. Waku Relay protocol is based on libp2p +for and consumed.

    +

    Adaptive Nodes

    +

    Waku - protocol interactions​

    +

    There are many protocols that interact. Waku Relay protocol is based on libp2p GossipSub for p2p messaging. We have filter for bandwidth-restricted nodes to only receive subset of messages. Lightpush for nodes with short connection windows to push messages into network. Store for nodes that want to retrieve -historical messages.

    On the payload layer, we provide support for Noise handshakes/key-exchanges. +historical messages.

    +

    On the payload layer, we provide support for Noise handshakes/key-exchanges. This means that as a developers, you can get end-to-end encryption and expected guarantees out of the box. We have support for setting up a secure channel from scratch, and all of this paves the way for providing Signal's Double Ratchet at the protocol level much easier. We also have experimental support for multi-device usage. Similar features have existed in for example the Status app for a while, but with this we make it easier for any platform using Waku to use -it.

    There are other protocols too, related to peer discovery, topic usage, etc. See -specs for more details.

    Protocol Interactions

    Waku - Network​

    For the Waku network, there are a few problems. For example, when it comes to +it.

    +

    There are other protocols too, related to peer discovery, topic usage, etc. See +specs for more details.

    +Protocol Interactions +

    Waku - Network​

    +

    For the Waku network, there are a few problems. For example, when it comes to network spam and incentivizing service nodes. We want to address these while keeping privacy-guarantees of the base layer. I'm going to go into both of -these.

    The spam problem arises on the gossip layer when anyone can overwhelm the +these.

    +

    The spam problem arises on the gossip layer when anyone can overwhelm the network with messages. The service incentivization is a problem when nodes don't directly benefit from the provisioning of a certain service. This can happen if they are not using the protocol directly themselves as part of normal operation, or if they aren't socially inclined to provide a certain service. This depends a -lot on how an individual platform decides to use the network.

    Waku Network

    Dealing with network spam and RLN Relay​

    Since the p2p relay network is open to anyone, there is a problem with spam. If +lot on how an individual platform decides to use the network.

    +

    Waku Network

    +

    Dealing with network spam and RLN Relay​

    +

    Since the p2p relay network is open to anyone, there is a problem with spam. If we look at existing solutions for dealing with spam in traditional messaging systems, a lot of entities like Google, Facebook, Twitter, Telegram, Discord use phone number verification. While this is largely sybil-resistant, it is -centralized and not private at all.

    Historically, Whisper used PoW which isn't good for heterogenerous networks. +centralized and not private at all.

    +

    Historically, Whisper used PoW which isn't good for heterogenerous networks. Peer scoring is open to sybil attacks and doesn't directly address spam -protection in an anonymous p2p network.

    The key idea here is to use RLN for private economic spam protection using -zkSNARKs.

    I'm not going to go into too much detail of RLN here. If you are interested, I +protection in an anonymous p2p network.

    +

    The key idea here is to use RLN for private economic spam protection using +zkSNARKs.

    +

    I'm not going to go into too much detail of RLN here. If you are interested, I gave a talk in Amsterdam at Devconnect about this. We have some write-ups on RLN here by Sanaz who has been pushing a lot of this from our side. There's also another talk at Devcon by Tyler going into RLN in -more detail. Finally, here's the RLN spec.

    I'll briefly go over what it is, the interface and circuit and then talk about -how it is used in Waku.

    RLN - Overview and Flow​

    RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting +more detail. Finally, here's the RLN spec.

    +

    I'll briefly go over what it is, the interface and circuit and then talk about +how it is used in Waku.

    +

    RLN - Overview and Flow​

    +

    RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting mechanism based on zkSNARKs. By rate limiting we mean you can only send N messages in a given period. By anonymity we mean that you can't link message to a publisher. We can think of it as a voting booth, where you are only allowed to -vote once every election.

    Voting Booth

    It can be used for spam protection in p2p messaging systems, and also rate -limiting in general, such as for a decentralized captcha.

    There are three parts to it. You register somewhere, then you can signal and +vote once every election.

    +

    Voting Booth

    +

    It can be used for spam protection in p2p messaging systems, and also rate +limiting in general, such as for a decentralized captcha.

    +

    There are three parts to it. You register somewhere, then you can signal and finally there's a verification/slashing phase. You put some capital at risk, -either economic or social, and if you double signal you get slashed.

    RLN - Circuit​

    Here's what the private and public inputs to the circuit look like. The identity +either economic or social, and if you double signal you get slashed.

    +

    RLN - Circuit​

    +

    Here's what the private and public inputs to the circuit look like. The identity secret is generated locally, and we create an identity commitment that is inserted into a Merkle tree. We then use Merkle proofs to prove membership. Registered member can only signal once for a given epoch or external nullifier, for example every ten seconds in Unix time. RLN identifer is for a specific RLN -app.

    We also see what the circuit output looks like. This is calculated locally. y +app.

    +

    We also see what the circuit output looks like. This is calculated locally. y is a share of the secret equation, and the (internal) nullifier acts as a unique fingerprint for a given app/user/epoch combination. How do we calculate y and -the internal nullifier?

    // Private input
    signal input identity_secret;
    signal input path_elements[n_levels][1];
    signal input identity_path_index[n_levels];

    // Public input
    signal input x; // signal_hash
    signal input epoch; // external_nullifier
    signal input rln_identifier;

    // Circuit output
    signal output y;
    signal output root;
    signal output nullifier;

    RLN - Shamir's secret sharing​

    This is done using Shamir's secret +the internal nullifier?

    +
    // Private input
    signal input identity_secret;
    signal input path_elements[n_levels][1];
    signal input identity_path_index[n_levels];

    // Public input
    signal input x; // signal_hash
    signal input epoch; // external_nullifier
    signal input rln_identifier;

    // Circuit output
    signal output y;
    signal output root;
    signal output nullifier;
    +

    RLN - Shamir's secret sharing​

    +

    This is done using Shamir's secret sharing. Shamir’s secret sharing is based on idea of splitting a secret into shares. This is how -we enable slashing of funds.

    In this case, we have two shares. If a given identity a0 signals twice in +we enable slashing of funds.

    +

    In this case, we have two shares. If a given identity a0 signals twice in epoch/external nullifier, a1 is the same. For a given RLN app, internal_nullifier then stays the same. x is signal hash which is different, and y is public, so we can reconstruct identity_secret. With the identity -secret revealed, this gives access to e.g. financial stake.

    a_0 = identity_secret // secret S
    a_1 = poseidonHash([a0, external_nullifier])

    y = a_0 + x * a_1

    internal_nullifier = poseidonHash([a_1, rln_identifier])

    Shamir&#39;s secret sharing

    RLN Relay​

    This is how RLN is used with Relay/GossipSub protocol. A node registers and +secret revealed, this gives access to e.g. financial stake.

    +
    a_0 = identity_secret // secret S
    a_1 = poseidonHash([a0, external_nullifier])

    y = a_0 + x * a_1

    internal_nullifier = poseidonHash([a_1, rln_identifier])
    +

    Shamir&#39;s secret sharing

    +

    RLN Relay​

    +

    This is how RLN is used with Relay/GossipSub protocol. A node registers and locks up funds, and after that it can send messages. It publishes a message -containing the Zero Knowledge proof and some other details.

    Each relayer node listens to the membership contract for new members, and it +containing the Zero Knowledge proof and some other details.

    +

    Each relayer node listens to the membership contract for new members, and it also keeps track of relevant metadata and merkle tree. Metadata is needed to be -able to detect double signaling and perform slashing.

    Before forwarding a message, it does some verification checks to ensure there +able to detect double signaling and perform slashing.

    +

    Before forwarding a message, it does some verification checks to ensure there are no duplicate messages, ZKP is valid and no double signaling has occured. It is worth noting that this can be combined with peer scoring, for example for -duplicate messages or invalid ZK proofs.

    In line of Waku's goals of modularity, RLN Relay is applied on a specific subset -of pubsub and content topics. You can think of it as an extra secure channel.

    RLN Relay

    RLN Relay cross-client testnet​

    Where are we with RLN Relay deployment? We've recently launched our second +duplicate messages or invalid ZK proofs.

    +

    In line of Waku's goals of modularity, RLN Relay is applied on a specific subset +of pubsub and content topics. You can think of it as an extra secure channel.

    +

    RLN Relay

    +

    RLN Relay cross-client testnet​

    +

    Where are we with RLN Relay deployment? We've recently launched our second testnet. This is using RLN Relay with a smart contract on Goerli. It integrates with our example p2p chat application, and it does so through three different clients, nwaku, go-waku and js-waku for browsers. This is our first p2p -cross-client testnet for RLN Relay.

    Here's a video that shows a user +cross-client testnet for RLN Relay.

    +

    Here's a video that shows a user registering in a browser, signaling through JS-Waku. It then gets relayed to a nwaku node, that verifies the proof. The second video shows what happens in the spam case. when more than one message is sent in a given epoch, it detects it as spam and discards it. Slashing hasn't been implemented fully yet in the client -and is a work in progress.

    If you are curious and want to participate, you can join the effort on our Vac +and is a work in progress.

    +

    If you are curious and want to participate, you can join the effort on our Vac Discord. We also have tutorials -setup for all clients so you can play around with it.

    As part of this, and to make it work in multiple different environments, we've +setup for all clients so you can play around with it.

    +

    As part of this, and to make it work in multiple different environments, we've also been developing a new library called Zerokit. I'll talk about this a bit -later.

    Private settlement / Service credentials​

    Going back to the service network idea, let's talk about service credentials. +later.

    +

    Private settlement / Service credentials​

    +

    Going back to the service network idea, let's talk about service credentials. The idea behind service credentials and private settlement is to enable two actors to pay for and provide services without compromising their privacy. We do not want the payment to create a direct public link between the service provider -and requester.

    Recall the Waku service network illustration with adaptive nodes that choose +and requester.

    +

    Recall the Waku service network illustration with adaptive nodes that choose which protocols they want to run. Many of these protocols aren't very heavy and just work by default. For example the relay protocol is enabled by default. Other protocols are much heavier to provide, such as storing historical -messages.

    It is desirable to have additional incentives for this, especially for platforms +messages.

    +

    It is desirable to have additional incentives for this, especially for platforms that aren't community-based where some level of altruism can be assumed (e.g. -Status Communities, or WalletConnect cloud infrastructure).

    You have a node Alice that is often offline and wants to consume historical +Status Communities, or WalletConnect cloud infrastructure).

    +

    You have a node Alice that is often offline and wants to consume historical messages on some specific content topics. You have another node Bob that runs a server at home where they store historical messages for the last several weeks. Bob is happy to provide this service for free because he's excited about running privacy-preserving infrastructure and he's using it himself, but his node is getting overwhelmed by freeloaders and he feels like he should be paid something -for continuing to provide this service.

    Alice deposits some funds in a smart contract which registers it in a tree, +for continuing to provide this service.

    +

    Alice deposits some funds in a smart contract which registers it in a tree, similar to certain other private settlement mechanisms. A fee is taken or burned. In exchange, she gets a set of tokens or service credentials. When she wants to do a query with some criteria, she sends this to Bob. Bob responds with size of response, cost, and receiver address. Alice then sends a proof of delegation of a service token as a payment. Bob verifies the proof and resolves -the query.

    The end result is that Alice has consumed some service from Bob, and Bob has +the query.

    +

    The end result is that Alice has consumed some service from Bob, and Bob has received payment for this. There's no direct transaction link between Alice and Bob, and gas fees can be minimized by extending the period before settling on -chain.

    This can be complemented with altruistic service provisioning, for example by +chain.

    +

    This can be complemented with altruistic service provisioning, for example by splitting the peer pool into two slots, or only providing a few cheap queries -for free.

    The service provisioning is general, and can be generalized for any kind of -request/response service provisoning that we want to keep private.

    This isn't a perfect solution, but it is an incremental improvement on top of +for free.

    +

    The service provisioning is general, and can be generalized for any kind of +request/response service provisoning that we want to keep private.

    +

    This isn't a perfect solution, but it is an incremental improvement on top of the status quo. It can be augmented with more advanced techniques such as better -non-repudiable node reputation, proof of correct service provisioning, etc.

    We are currently in the raw spec / proof of concept stage of this. We expect to -launch a testnet of this later this year or early next year.

    Service credentials flow

    Zerokit​

    Zerokit is a set of Zero Knowledge modules, +non-repudiable node reputation, proof of correct service provisioning, etc.

    +

    We are currently in the raw spec / proof of concept stage of this. We expect to +launch a testnet of this later this year or early next year.

    +

    Service credentials flow

    +

    Zerokit​

    +

    Zerokit is a set of Zero Knowledge modules, written in Rust and designed to be used in many different environments. The initial goal is to get the best of both worlds with Circom/Solidity/JS and Rust/ZK ecosystem. This enables people to leverage Circom-based constructs from -non-JS environments.

    For the RLN module, it is using Circom circuits via ark-circom and Rust for +non-JS environments.

    +

    For the RLN module, it is using Circom circuits via ark-circom and Rust for scaffolding. It exposes a C FFI API that can be used through other system programming environments, like Nim and Go. It also exposes an experimental WASM -API that can be used through web browsers.

    Waku is p2p infrastructure running in many different environments, such as -Nim/JS/Go/Rust, so this a requirement for us.

    Circom and JS strengths are access to Dapp developers, tooling, generating +API that can be used through web browsers.

    +

    Waku is p2p infrastructure running in many different environments, such as +Nim/JS/Go/Rust, so this a requirement for us.

    +

    Circom and JS strengths are access to Dapp developers, tooling, generating verification code, circuits etc. Rust strengths is that it is systems-based and easy to interface with other language runtime such as Nim, Go, Rust, C. It also gives access to other Rust ZK ecosystems such as arkworks. This opens door for using other constructs, such as Halo2. This becomes especially relevant for constructs where you don't want to do a trusted setup or where circuits are more -complex/custom and performance requirements are higher.

    In general with Zerokit, we want to make it easy to build and use ZKP in a +complex/custom and performance requirements are higher.

    +

    In general with Zerokit, we want to make it easy to build and use ZKP in a multitude of environments, such as mobile phones and web browsers. Currently it is too complex to write privacy-protecting infrastructure with ZKPs considering all the languages and tools you have to learn, from JS, Solidity and Circom to @@ -2291,23 +5503,29 @@ Rust, WASM and FFI. And that isn't even touching on things like secure key storage or mobile dev. Luckily more and more projects are working on this, including writing DSLs etc. It'd also be exciting if we can make a useful toolstack for JS-less ZK dev to reduce cognitive overhead, similar to what we -have with something like Foundry.

    Other research​

    I also want to mention a few other things we are doing. One thing is +have with something like Foundry.

    +

    Other research​

    +

    I also want to mention a few other things we are doing. One thing is protocol specifications. We think this is very important for p2p infra, and we see a lot of other projects that claim to do it p2p infrastructure but they aren't clear about guarantees or how stable something is. That makes it hard to have multiple implementations, to collaborate across -different projects, and to analyze things objectively.

    Related to that is publishing papers. We've put +different projects, and to analyze things objectively.

    +

    Related to that is publishing papers. We've put out three so far, related to Waku and RLN-Relay. This makes it easier to interface with academia. There's a lot of good researchers out there and we want -to build a better bridge between academia and industry.

    Another thing is network +to build a better bridge between academia and industry.

    +

    Another thing is network privacy. Waku is modular with respect to privacy guarantees, and there are a lot of knobs to turn here depending on specific deployments. For example, if you are running the full relay protocol you currently have much stronger receiver anonymity than if you -are running filter protocol from a bandwidth or connectivity-restricted node.

    We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym +are running filter protocol from a bandwidth or connectivity-restricted node.

    +

    We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym come with some trade-offs but are a useful tool in the arsenal. A good mental model to keep in mind is the anonymity trilemma, where you can only pick 2/3 out -of low latency, low bandwidth usage and strong anonymity.

    We are currently exploring Dandelion-like +of low latency, low bandwidth usage and strong anonymity.

    +

    We are currently exploring Dandelion-like additions to the relay/gossip protocol, which would provide for stronger sender anonymity, especially in a multi-node/botnet attacker model. As part of this we are looking into different @@ -2315,18 +5533,24 @@ parameters choices and general possibilities for lower latency usage. This could make it more amenable for latency sensitive environments, such as validator privacy, for specific threat models. The general theme here is we want to be rigorous with the guarantees we provide, under what conditions and for what -threat models.

    Another thing mentioned earlier is Noise payload +threat models.

    +

    Another thing mentioned earlier is Noise payload encryption, and specifically things like allowing for pairing different devices with e.g. QR codes. This makes it easier for developers to provide secure messaging in many realistic scenarios in a -multi-device world.

    Other research

    Summary​

    We've gone over what privacy-protecting infrastructure is, why we want it and +multi-device world.

    +

    Other research

    +

    Summary​

    +

    We've gone over what privacy-protecting infrastructure is, why we want it and how we can build it. We've seen how ZK is a fundamental building block for this. We've looked at Waku, the communication layer for Web3, and how it uses Zero Knowledge proofs to stay private and function better. We've also looked at -Zerokit and how we can make it easier to do ZKP in different environments.

    Finally we also looked at some other research we've been doing. All of the +Zerokit and how we can make it easier to do ZKP in different environments.

    +

    Finally we also looked at some other research we've been doing. All of the things mentioned in this article, and more, is available as write-ups, specs, or -discussions on our forum or Github.

    If you find any of this exciting to work on, feel free to reach out on our +discussions on our forum or Github.

    +

    If you find any of this exciting to work on, feel free to reach out on our Discord. We are also hiring, and we have started expanding into other privacy infrastructure tech like private and provable computation with ZK-WASM.

    ]]> @@ -2340,1071 +5564,283 @@ computation with ZK-WASM.

    ]]> 2022-07-22T10:00:00.000Z - Introducing a basic threat model and privacy/anonymity analysis for the Waku v2 relay protocol.

    Waku v2 enables secure, privacy preserving communication using a set of modular P2P protocols. + Introducing a basic threat model and privacy/anonymity analysis for the Waku v2 relay protocol.

    + +

    Waku v2 enables secure, privacy preserving communication using a set of modular P2P protocols. Waku v2 also aims at protecting the user's anonymity. This post is the first in a series about Waku v2 security, privacy, and anonymity. The goal is to eventually have a full privacy and anonymity analysis for each of the Waku v2 protocols, as well as covering the interactions of various Waku v2 protocols. -This provides transparency with respect to Waku's current privacy and anonymity guarantees, and also identifies weak points that we have to address.

    In this post, we first give an informal description of security, privacy and anonymity in the context of Waku v2. +This provides transparency with respect to Waku's current privacy and anonymity guarantees, and also identifies weak points that we have to address.

    +

    In this post, we first give an informal description of security, privacy and anonymity in the context of Waku v2. For each definition, we summarize Waku's current guarantees regarding the respective property. -We also provide attacker models, an attack-based threat model, and a first anonymity analysis of Waku v2 relay within the respective models.

    Waku comprises many protocols that can be combined in a modular way. +We also provide attacker models, an attack-based threat model, and a first anonymity analysis of Waku v2 relay within the respective models.

    +

    Waku comprises many protocols that can be combined in a modular way. For our privacy and anonymity analysis, we start with the relay protocol because it is at the core of Waku v2 enabling Waku's publish subscribe approach to P2P messaging. -In its current form, Waku relay is a minor extension of libp2p GossipSub.

    Figure 1: The Waku v2 relay mesh is based on the [GossipSub mesh](https://docs.libp2p.io/concepts/publish-subscribe#types-of-peering)

    Informal Definitions: Security, Privacy, and Anonymity​

    The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.

    Security​

    Of the three, Security has the clearest agreed upon definition, -at least regarding its key concepts: confidentiality, integrity, and availability.

    • confidentiality: data is not disclosed to unauthorized entities.
    • integrity: data is not modified by unauthorized entities.
    • availability: data is available, i.e. accessible by authorized entities.

    While these are the key concepts, the definition of information security has been extended over time including further concepts, +In its current form, Waku relay is a minor extension of libp2p GossipSub.

    +

    Figure 1: The Waku v2 relay mesh is based on the GossipSub mesh

    +

    Informal Definitions: Security, Privacy, and Anonymity​

    +

    The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.

    +

    Security​

    +

    Of the three, Security has the clearest agreed upon definition, +at least regarding its key concepts: confidentiality, integrity, and availability.

    +
      +
    • confidentiality: data is not disclosed to unauthorized entities.
      • +
    • integrity: data is not modified by unauthorized entities.
      • +
    • availability: data is available, i.e. accessible by authorized entities.
      • +
    +

    While these are the key concepts, the definition of information security has been extended over time including further concepts, e.g. authentication and non-repudiation. -We might cover these in future posts.

    Privacy​

    Privacy allows users to choose which data and information

    • they want to share
    • and with whom they want to share it.

    This includes data and information that is associated with and/or generated by users. +We might cover these in future posts.

    +

    Privacy​

    +

    Privacy allows users to choose which data and information

    +
      +
    • they want to share
      • +
    • and with whom they want to share it.
      • +
    +

    This includes data and information that is associated with and/or generated by users. Protected data also comprises metadata that might be generated without users being aware of it. This means, no further information about the sender or the message is leaked. Metadata that is protected as part of the privacy-preserving property does not cover protecting the identities of sender and receiver. -Identities are protected by the anonymity property.

    Often privacy is realized by the confidentiality property of security. +Identities are protected by the anonymity property.

    +

    Often privacy is realized by the confidentiality property of security. This neither makes privacy and security the same, nor the one a sub category of the other. While security is abstract itself (its properties can be realized in various ways), privacy lives on a more abstract level using security properties. Privacy typically does not use integrity and availability. -An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.

    Waku offers confidentiality via secure channels set up with the help of the Noise Protocol Framework. +An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.

    +

    Waku offers confidentiality via secure channels set up with the help of the Noise Protocol Framework. Using these secure channels, message content is only disclosed to the intended receivers. They also provide good metadata protection properties. However, we do not have a metadata protection analysis as of yet, -which is part of our privacy/anonymity roadmap.

    Anonymity​

    Privacy and anonymity are closely linked. +which is part of our privacy/anonymity roadmap.

    +

    Anonymity​

    +

    Privacy and anonymity are closely linked. Both the identity of a user and data that allows inferring a user's identity should be part of the privacy policy. -For the purpose of analysis, we want to have a clearer separation between these concepts.

    We define anonymity as unlinkablity of users' identities and their shared data and/or actions.

    We subdivide anonymity into receiver anonymity and sender anonymity.

    Receiver Anonymity​

    We define receiver anonymity as unlinkability of users' identities and the data they receive and/or related actions. +For the purpose of analysis, we want to have a clearer separation between these concepts.

    +

    We define anonymity as unlinkablity of users' identities and their shared data and/or actions.

    +

    We subdivide anonymity into receiver anonymity and sender anonymity.

    +

    Receiver Anonymity​

    +

    We define receiver anonymity as unlinkability of users' identities and the data they receive and/or related actions. The data transmitted via Waku relay must be a Waku message, which contains a content topic field. Because each message is associated with a content topic, and each receiver is interested in messages with specific content topics, receiver anonymity in the context of Waku corresponds to subscriber-topic unlinkability. -An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.

    The Waku message's content topic is not related to the libp2p pubsub topic. +An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.

    +

    The Waku message's content topic is not related to the libp2p pubsub topic. For now, Waku uses a single libp2p pubsub topic, which means messages are propagated via a single mesh of peers. With this, the receiver discloses its participation in Waku on the gossipsub layer. -We will leave the analysis of libp2p gossipsub to a future article within this series, and only provide a few hints and pointers here.

    Waku offers k-anonymity regarding content topic interest in the global adversary model. -K-anonymity in the context of Waku means an attacker can link receivers to content topics with a maximum certainty of 1/k1/k. -The larger kk, the less certainty the attacker gains. -Receivers basically hide in a pool of kk content topics, any subset of which could be topics they subscribed to. +We will leave the analysis of libp2p gossipsub to a future article within this series, and only provide a few hints and pointers here.

    +

    Waku offers k-anonymity regarding content topic interest in the global adversary model. +K-anonymity in the context of Waku means an attacker can link receivers to content topics with a maximum certainty of 1/k1/k. +The larger kk, the less certainty the attacker gains. +Receivers basically hide in a pool of kk content topics, any subset of which could be topics they subscribed to. The attacker does not know which of those the receiver actually subscribed to, and the receiver enjoys plausible deniability regarding content topic subscription. -Assuming there are nn Waku content topics, a receiver has nn-anonymity with respect to association to a specific content topic.

    Technically, Waku allows distributing messages over several libp2p pubsub topics. -This yields kk-anonymity, assuming kk content topics share the same pubsub topic. +Assuming there are nn Waku content topics, a receiver has nn-anonymity with respect to association to a specific content topic.

    +

    Technically, Waku allows distributing messages over several libp2p pubsub topics. +This yields kk-anonymity, assuming kk content topics share the same pubsub topic. However, if done wrongly, such sharding of pubsub topics can breach anonymity. -A formal specification of anonymity-preserving topic sharding building on the concepts of partitioned topics is part of our roadmap.

    Also, Waku is not directly concerned with 1:1 communication, so for this post, 1:1 communication is out of scope. +A formal specification of anonymity-preserving topic sharding building on the concepts of partitioned topics is part of our roadmap.

    +

    Also, Waku is not directly concerned with 1:1 communication, so for this post, 1:1 communication is out of scope. Channels for 1:1 communication can be implemented on top of Waku relay. In the future, a 1:1 communication protocol might be added to Waku. -Similar to topic sharding, it would maintain receiver anonymity leveraging partitioned topics.

    Sender Anonymity​

    We define sender anonymity as unlinkability of users' identities and the data they send and/or related actions. -Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.

    In summary, Waku offers weak sender anonymity because of Waku's strict no sign policy, +Similar to topic sharding, it would maintain receiver anonymity leveraging partitioned topics.

    +

    Sender Anonymity​

    +

    We define sender anonymity as unlinkability of users' identities and the data they send and/or related actions. +Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.

    +

    In summary, Waku offers weak sender anonymity because of Waku's strict no sign policy, which has its origins in the Ethereum consensus specs. -17/WAKU-RLN-RELAY and 18/WAKU2-SWAP mitigate replay and injection attacks.

    Waku currently does not offer sender anonymity in stronger attacker models, as well as cannot protect against targeted attacks in weaker attacker models like the single or multi node attacker. -We will cover this in more detail in later sections.

    Anonymity Trilemma​

    The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global on-net attacker model. +17/WAKU-RLN-RELAY and 18/WAKU2-SWAP mitigate replay and injection attacks.

    +

    Waku currently does not offer sender anonymity in stronger attacker models, as well as cannot protect against targeted attacks in weaker attacker models like the single or multi node attacker. +We will cover this in more detail in later sections.

    +

    Anonymity Trilemma​

    +

    The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global on-net attacker model. Waku's goal, being a modular set of protocols, is to offer any combination of two out of these three properties, as well as blends. -An example for blending is an adjustable number of pubsub topics and peers in the respective pubsub topic mesh; this allows tuning the trade-off between anonymity and bandwidth.

    Figure 2: Anonymity Trilemma: pick two.

    A fourth factor that influences the anonymity trilemma is frequency and patterns of messages. +An example for blending is an adjustable number of pubsub topics and peers in the respective pubsub topic mesh; this allows tuning the trade-off between anonymity and bandwidth.

    +

    Figure 2: Anonymity Trilemma: pick two.

    +

    A fourth factor that influences the anonymity trilemma is frequency and patterns of messages. The more messages there are, and the more randomly distributed they are, the better the anonymity protection offered by a given anonymous communication protocol. So, incentivising users to use the protocol, for instance by lowering entry barriers, helps protecting the anonymity of all users. -The frequency/patterns factor is also related to the above described k-anonymity.

    Censorship Resistance​

    Another security related property that Waku aims to offer is censorship resistance. +The frequency/patterns factor is also related to the above described k-anonymity.

    +

    Censorship Resistance​

    +

    Another security related property that Waku aims to offer is censorship resistance. Censorship resistance guarantees that users can participate even if an attacker tries to deny them access. So, censorship resistance ties into the availability aspect of security. In the context of Waku that means users should be able to send messages as well as receive all messages they are interested in, -even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.

    Currently, Waku only guarantees censorship resistance in the weak single node attacker model. +even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.

    +

    Currently, Waku only guarantees censorship resistance in the weak single node attacker model. While currently employed secure channels mitigate targeted censorship, e.g. blocking specific content topics, general censorship resistance in strong attacker models is part of our roadmap. -Among other options, we will investigate Pluggable Transports in future articles.

    Attacker Types​

    The following lists various attacker types with varying degrees of power. -The more power an attacker has, the more difficult it is to gain the respective attacker position.

    Each attacker type comes in a passive and an active variant. +Among other options, we will investigate Pluggable Transports in future articles.

    +

    Attacker Types​

    +

    The following lists various attacker types with varying degrees of power. +The more power an attacker has, the more difficult it is to gain the respective attacker position.

    +

    Each attacker type comes in a passive and an active variant. While a passive attacker can stay hidden and is not suspicious, -the respective active attacker has more (or at least the same) deanonymization power.

    We also distinguish between internal and external attackers.

    Internal​

    With respect to Waku relay, an internal attacker participates in the same pubsub topic as its victims. -Without additional measures on higher layer protocols, access to an internal position is easy to get.

    Single Node​

    This attacker controls a single node. -Because this position corresponds to normal usage of Waku relay, it is trivial to obtain.

    Multi Node​

    This attacker controls several nodes. We assume a smaller static number of controlled nodes. +the respective active attacker has more (or at least the same) deanonymization power.

    +

    We also distinguish between internal and external attackers.

    +

    Internal​

    +

    With respect to Waku relay, an internal attacker participates in the same pubsub topic as its victims. +Without additional measures on higher layer protocols, access to an internal position is easy to get.

    +

    Single Node​

    +

    This attacker controls a single node. +Because this position corresponds to normal usage of Waku relay, it is trivial to obtain.

    +

    Multi Node​

    +

    This attacker controls several nodes. We assume a smaller static number of controlled nodes. The multi node position can be achieved relatively easily by setting up multiple nodes. Botnets might be leveraged to increase the number of available hosts. Multi node attackers could use Sybil attacks to increase the number of controlled nodes. -A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.

    Linearly Scaling Nodes​

    This attacker controls a number of nodes that scales linearly with the number of nodes in the network. +A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.

    +

    Linearly Scaling Nodes​

    +

    This attacker controls a number of nodes that scales linearly with the number of nodes in the network. This attacker is especially interesting to investigate in the context of DHT security, -which Waku uses for ambient peer discovery.

    External​

    An external attacker can only see encrypted traffic (protected by a secure channel set up with WAKU2-NOISE). +which Waku uses for ambient peer discovery.

    +

    External​

    +

    An external attacker can only see encrypted traffic (protected by a secure channel set up with WAKU2-NOISE). Because an internal position can be easily obtained, in practice external attackers would mount combined attacks that leverage both internal an external attacks. -We cover this more below when describing attacks.

    Local​

    A local attacker has access to communication links in a local network segment. -This could be a rogue access point (with routing capability).

    AS​

    An AS attacker controls a single AS (autonomous system). +We cover this more below when describing attacks.

    +

    Local​

    +

    A local attacker has access to communication links in a local network segment. +This could be a rogue access point (with routing capability).

    +

    AS​

    +

    An AS attacker controls a single AS (autonomous system). A passive AS attacker can listen to traffic on arbitrary links within the AS. -An active AS attacker can drop, inject, and alter traffic on arbitrary links within the AS.

    In practice, a malicious ISP would be considered as an AS attacker. +An active AS attacker can drop, inject, and alter traffic on arbitrary links within the AS.

    +

    In practice, a malicious ISP would be considered as an AS attacker. A malicious ISP could also easily setup a set of nodes at specific points in the network, -gaining internal attack power similar to a strong multi node attacker.

    Global On-Net​

    A global on-net attacker has complete overview over the whole network. +gaining internal attack power similar to a strong multi node attacker.

    +

    Global On-Net​

    +

    A global on-net attacker has complete overview over the whole network. A passive global attacker can listen to traffic on all links, while the active global attacker basically carries the traffic: it can freely drop, inject, and alter traffic at all positions in the network. -This basically corresponds to the Dolev-Yao model.

    An entity with this power would, in practice, also have the power of the internal linearly scaling nodes attacker.

    Attack-based Threat Analysis​

    The following lists various attacks including the weakest attacker model in which the attack can be successfully performed. -The respective attack can be performed in all stronger attacker models as well.

    An attack is considered more powerful if it can be successfully performed in a weaker attacker model.

    If not stated otherwise, we look at these attacks with respect to their capability to deanonymize the message sender.

    Scope​

    In this post, we introduce a simple tightly scoped threat model for Waku v2 Relay, which will be extended in the course of this article series.

    In this first post, we will look at the relay protocol in isolation. +This basically corresponds to the Dolev-Yao model.

    +

    An entity with this power would, in practice, also have the power of the internal linearly scaling nodes attacker.

    +

    Attack-based Threat Analysis​

    +

    The following lists various attacks including the weakest attacker model in which the attack can be successfully performed. +The respective attack can be performed in all stronger attacker models as well.

    +

    An attack is considered more powerful if it can be successfully performed in a weaker attacker model.

    +

    If not stated otherwise, we look at these attacks with respect to their capability to deanonymize the message sender.

    +

    Scope​

    +

    In this post, we introduce a simple tightly scoped threat model for Waku v2 Relay, which will be extended in the course of this article series.

    +

    In this first post, we will look at the relay protocol in isolation. Even though many threats arise from layers Waku relay is based on, and layers that in turn live on top of relay, we want to first look at relay in isolation because it is at the core of Waku v2. -Addressing and trying to solve all security issues of a complex system at once is an overwhelming task, which is why we focus on the soundness of relay first.

    This also goes well with the modular design philosophy of Waku v2, as layers of varying levels of security guarantees can be built on top of relay, all of which can relay on the guarantees that Waku provides. -Instead of looking at a multiplicative explosion of possible interactions, we look at the core in this article, and cover the most relevant combinations in future posts.

    Further restricting the scope, we will look at the data field of a relay message as a black box. +Addressing and trying to solve all security issues of a complex system at once is an overwhelming task, which is why we focus on the soundness of relay first.

    +

    This also goes well with the modular design philosophy of Waku v2, as layers of varying levels of security guarantees can be built on top of relay, all of which can relay on the guarantees that Waku provides. +Instead of looking at a multiplicative explosion of possible interactions, we look at the core in this article, and cover the most relevant combinations in future posts.

    +

    Further restricting the scope, we will look at the data field of a relay message as a black box. In a second article on Waku v2 relay, we will look into the data field, which according to the specification of Waku v2 relay must be a Waku v2 message. -We only consider messages with version field 2, which indicates that the payload has to be encoded using WAKU2-NOISE.

    Prerequisite: Get a Specific Position in the Network​

    Some attacks require the attacker node(s) to be in a specific position in the network. -In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.

    In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. +We only consider messages with version field 2, which indicates that the payload has to be encoded using WAKU2-NOISE.

    +

    Prerequisite: Get a Specific Position in the Network​

    +

    Some attacks require the attacker node(s) to be in a specific position in the network. +In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.

    +

    In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. If the victim node still has open slots, the attacker gets the desired position. -This only requires the attacker to know the gossipsub multiaddress of the victim node.

    A linearly scaling nodes attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. -Waku v2 discv5 will employ countermeasures that mitigate the amplifying effect this attacker type can achieve.

    Replay Attack​

    In the scope we defined above, Waku v2 is resilient against replay attacks. +This only requires the attacker to know the gossipsub multiaddress of the victim node.

    +

    A linearly scaling nodes attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. +Waku v2 discv5 will employ countermeasures that mitigate the amplifying effect this attacker type can achieve.

    +

    Replay Attack​

    +

    In the scope we defined above, Waku v2 is resilient against replay attacks. GossipSub nodes, and by extension Waku relay nodes, feature a seen cache, and only relay messages they have not seen before. -Further, replay attacks will be punished by RLN and SWAP.

    Neighbourhood Surveillance​

    This attack can be performed by a single node attacker that is connected to all peers of the victim node vv with respect to a specific topic mesh. -The attacker also has to be connected to vv. -In this position, the attacker will receive messages mvm_v sent by vv both on the direct path from vv, and on indirect paths relayed by peers of vv. -It will also receive messages mxm_x that are not sent by vv. These messages mxm_x are relayed by both vv and the peers of vv. -Messages that are received (significantly) faster from vv than from any other of vv's peers are very likely messages that vv sent, -because for these messages the attacker is one hop closer to the source.

    The attacker can (periodically) measure latency between itself and vv, and between itself and the peers of vv to get more accurate estimates for the expected timings. -An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between vv and its well-behaving peers. -An active AS attacker could also increase the latency between vv and its peers to make the timing differences more prominent. -This, however, might lead to vv switching to other peers.

    This attack cannot (reliably) distinguish messages mvm_v sent by vv from messages mym_y relayed by peers of vv the attacker is not connected to. +Further, replay attacks will be punished by RLN and SWAP.

    +

    Neighbourhood Surveillance​

    +

    This attack can be performed by a single node attacker that is connected to all peers of the victim node vv with respect to a specific topic mesh. +The attacker also has to be connected to vv. +In this position, the attacker will receive messages mvm_v sent by vv both on the direct path from vv, and on indirect paths relayed by peers of vv. +It will also receive messages mxm_x that are not sent by vv. These messages mxm_x are relayed by both vv and the peers of vv. +Messages that are received (significantly) faster from vv than from any other of vv's peers are very likely messages that vv sent, +because for these messages the attacker is one hop closer to the source.

    +

    The attacker can (periodically) measure latency between itself and vv, and between itself and the peers of vv to get more accurate estimates for the expected timings. +An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between vv and its well-behaving peers. +An active AS attacker could also increase the latency between vv and its peers to make the timing differences more prominent. +This, however, might lead to vv switching to other peers.

    +

    This attack cannot (reliably) distinguish messages mvm_v sent by vv from messages mym_y relayed by peers of vv the attacker is not connected to. Still, there are hop-count variations that might be leveraged. -Messages mvm_v always have a hop-count of 1 on the path from vv to the attacker, while all other paths are longer. -Messages mym_y might have the same hop-count on the path from vv as well as on other paths.

    Controlled Neighbourhood​

    If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from vv.

    Observing Messages​

    If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving vv which have not been relayed to vv. -These are the messages sent by vv. -Waku relay protects against this attack by employing secure channels setup using Noise.

    Correlation​

    Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from vv. -This (alone) does not allow an external attacker to learn which message vv sent, but it allows identifying the respective traffic propagating through the network. -The more traffic in the network, the lower the success rate of this attack.

    Combined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.

    DoS​

    An active single node attacker could run a disruption attack by

    • (1) dropping messages that should be relayed
    • (2) flooding neighbours with bogus messages

    While (1) has a negative effect on availability, the impact is not significant. +Messages mvm_v always have a hop-count of 1 on the path from vv to the attacker, while all other paths are longer. +Messages mym_y might have the same hop-count on the path from vv as well as on other paths.

    +

    Controlled Neighbourhood​

    +

    If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from vv.

    +

    Observing Messages​

    +

    If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving vv which have not been relayed to vv. +These are the messages sent by vv. +Waku relay protects against this attack by employing secure channels setup using Noise.

    +

    Correlation​

    +

    Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from vv. +This (alone) does not allow an external attacker to learn which message vv sent, but it allows identifying the respective traffic propagating through the network. +The more traffic in the network, the lower the success rate of this attack.

    +

    Combined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.

    +

    DoS​

    +

    An active single node attacker could run a disruption attack by

    +
      +
    • (1) dropping messages that should be relayed
      • +
    • (2) flooding neighbours with bogus messages
      • +
    +

    While (1) has a negative effect on availability, the impact is not significant. A linearly scaling botnet attacker, however, could significantly disrupt the network with such an attack. (2) is thwarted by RLN. -Also SWAP helps mitigating DoS attacks.

    A local attacker can DoS Waku by dropping all Waku traffic within its controlled network segment. +Also SWAP helps mitigating DoS attacks.

    +

    A local attacker can DoS Waku by dropping all Waku traffic within its controlled network segment. An AS attacker can DoS Waku within its authority, while a global attacker can DoS the whole network. -A countermeasure are censorship resistance techniques like Pluggable Transports.

    Summary and Future Work​

    Currently, Waku v2 relay offers k-anonymity with respect to receiver anonymity. -This also includes k-anonymity towards legitimate members of the same topic.

    Waku v2 relay offers sender anonymity in the single node attacker model with its strict no sign policy. +A countermeasure are censorship resistance techniques like Pluggable Transports.

    +

    Summary and Future Work​

    +

    Currently, Waku v2 relay offers k-anonymity with respect to receiver anonymity. +This also includes k-anonymity towards legitimate members of the same topic.

    +

    Waku v2 relay offers sender anonymity in the single node attacker model with its strict no sign policy. Currently, Waku v2 does not guarantee sender anonymity in the multi node and stronger attacker models. However, we are working on modular anonymity-preserving protocols and building blocks as part of our privacy/anonymity roadmap. The goal is to allow tunable anonymity with respect to trade offs between strong anonymity, low bandwidth, and low latency. All of these cannot be fully guaranteed as the the anonymity trilemma states. Some applications have specific requirements, e.g. low latency, which require a compromise on anonymity. -Anonymity-preserving mechanisms we plan to investigate and eventually specify as pluggable anonymity protocols for Waku comprise

    • Dandelion++ for lightweight anonymity;
    • onion routing as a building block adding a low latency anonymization layer;
    • a mix network for providing strong anonymity (on top of onion routing) even in the strongest attacker model at the cost of higher latency.

    These pluggable anonymity-preserving protocols will form a sub-set of the Waku v2 protocol set. -As an intermediate step, we might directly employ Tor for onion-routing, and Nym as a mix-net layer.

    In future research log posts, we will cover further Waku v2 protocols and identify anonymity problems that will be added to our roadmap. -These protocols comprise

    • 13/WAKU2-STORE, which can violate receiver anonymity as it allows filtering by content topic. -A countermeasure is using the content topic exclusively for local filters.
    • 12/WAKU2-FILTER, which discloses nodes' interest in topics;
    • 19/WAKU2-LIGHTPUSH, which also discloses nodes' interest in topics and links the lightpush client as the sender of a message to the lightpush service node;
    • 21/WAKU2-FTSTORE, which discloses nodes' interest in specific time ranges allowing to infer information like online times.

    While these protocols are not necessary for the operation of Waku v2, and can be seen as pluggable features, -we aim to provide alternatives without the cost of lowering the anonymity level.

    References​

    ]]>     +Anonymity-preserving mechanisms we plan to investigate and eventually specify as pluggable anonymity protocols for Waku comprise

    +
      +
    • Dandelion++ for lightweight anonymity;
      • +
    • onion routing as a building block adding a low latency anonymization layer;
      • +
    • a mix network for providing strong anonymity (on top of onion routing) even in the strongest attacker model at the cost of higher latency.
      • +
    +

    These pluggable anonymity-preserving protocols will form a sub-set of the Waku v2 protocol set. +As an intermediate step, we might directly employ Tor for onion-routing, and Nym as a mix-net layer.

    +

    In future research log posts, we will cover further Waku v2 protocols and identify anonymity problems that will be added to our roadmap. +These protocols comprise

    +
      +
    • 13/WAKU2-STORE, which can violate receiver anonymity as it allows filtering by content topic. +A countermeasure is using the content topic exclusively for local filters.
      • +
    • 12/WAKU2-FILTER, which discloses nodes' interest in topics;
      • +
    • 19/WAKU2-LIGHTPUSH, which also discloses nodes' interest in topics and links the lightpush client as the sender of a message to the lightpush service node;
      • +
    • 21/WAKU2-FTSTORE, which discloses nodes' interest in specific time ranges allowing to infer information like online times.
      • +
    +

    While these protocols are not necessary for the operation of Waku v2, and can be seen as pluggable features, +we aim to provide alternatives without the cost of lowering the anonymity level.

    +

    References​

    +]]> Daniel - - <![CDATA[Noise handshakes as key-exchange mechanism for Waku]]> - https://vac.dev/rlog/wakuv2-noise - - 2022-05-17T10:00:00.000Z - - We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2.

    Introduction​

    In this post we will provide an overview of how Waku v2 users can adopt Noise handshakes to agree on cryptographic keys used to securely encrypt messages.

    This process belongs to the class of key-exchange mechanisms, consisting of all those protocols that, with different levels of complexity and security guarantees, allow two parties to publicly agree on a secret without letting anyone else know what this secret is.

    But why do we need key-exchange mechanisms in the first place?

    With the advent of public-key cryptography, it become possible to decouple encryption from decryption through use of two distinct cryptographic keys: one public, used to encrypt information and that can be made available to anyone, and one private (kept secret), which enables decryption of messages encrypted with its corresponding public key. The same does not happen in the case of symmetric encryption schemes where, instead, the same key is used for both encryption and decryption operations and hence cannot be publicly revealed as for public keys.

    In order to address specific application needs, many different public, symmetric and hybrid cryptographic schemes were designed: Waku v1 and Waku v2, which inherits part of their design from the Ethereum messaging protocol Whisper, provide support to both public-key primitives (ECIES, ECDSA) and symmetric primitives (AES-256-GCM, KECCAK-256), used to sign, hash, encrypt and decrypt exchanged messages.

    In principle, when communications employ public-key based encryption schemes (ECIES, in the case of Waku), there is no need for a key-agreement among parties: messages can be directly encrypted using the recipient's public-key before being sent over the network. However, public-key encryption and decryption primitives are usually very inefficient in processing large amount of data, and this may constitute a bottleneck for many of today's applications. Symmetric encryption schemes such as AES-256-GCM, on the other hand, are much more efficient, but the encryption/decryption key needs to be shared among users beforehand any encrypted messages is exchanged.

    To counter the downsides given by each of these two approaches while taking advantage of their strengths, hybrid constructions were designed. In these, public-key primitives are employed to securely agree on a secret key which, in turn, is used with a symmetric cipher for encrypting messages. In other words, such constructions specify a (public-key based) key-agreement mechanism!

    Waku, up to payload version 1, does not implement nor recommend any protocol for exchanging symmetric ciphers' keys, leaving such task to the application layer. It is important to note that the kind of key-agreement employed has a direct impact on the security properties that can be granted on later encrypted messages, while security requirements usually depend on the specific application for which encryption is needed in the first place.

    In this regard, Status, which builds on top of Waku, implements a custom version of the X3DH key-agreement protocol, in order to allow users to instantiate end-to-end encrypted communication channels. However, although such a solution is optimal when applied to (distributed) E2E encrypted chats, it is not flexible enough to fit or simplify the variety of applications Waku aims to address. -Hence, proposing and implementing one or few key-agreements which provide certain (presumably strong) security guarantees, would inevitably degrade performances of all those applications for which, given their security requirements, more tailored and efficient key-exchange mechanisms can be employed.

    Guided by different examples, in the following sections we will overview Noise, a protocol framework we are currently integrating in Waku, for building secure key-agreements between two parties. One of the great advantage of using Noise is that it is possible to add support to new key-exchanges by just specifying users' actions from a predefined list, requiring none to minimal modifications to existing implementations. Furthermore, Noise provides a framework to systematically analyze protocols' security properties and the corresponding attacker threat models. This allows not only to easily design new key-agreements eventually optimized for specific applications we want to address, but also to easily analyze or even formally verify any of such custom protocol!

    We believe that with its enormous flexibility and features, Noise represents a perfect candidate for bringing key-exchange mechanisms in Waku.

    The Diffie-Hellman Key-exchange​

    The formalization of modern public-key cryptography started with the pioneering work of Whitefield Diffie and Martin Hellman, who detailed one of the earliest known key-agreement protocols: the famous Diffie-Hellman Key-Exchange.

    Diffie-Hellman (DH) key-exchange is largely used today and represents the main cryptographic building block on which Noise handshakes' security is based.

    In turn, the security of DH is based on a mathematical problem called discrete logarithm which is believed to be hard when the agreement is practically instantiated using certain elliptic curves EE defined over finite fields Fp\mathbb{F}_p.

    Informally, a DH exchange between Alice and Bob proceeds as follows:

    • Alice picks a secret scalar sA∈Fps_A\in\mathbb{F}_p and computes, using the underlying curve's arithmetic, the point PA=sA⋅P∈E(Fp)P_A = s_A\cdot P\in E(\mathbb{F}_p) for a certain pre-agreed public generator PP of the elliptic curve E(Fp)E(\mathbb{F}_p). She then sends PAP_A to Bob.
    • Similarly, Bob picks a secret scalar sB∈Fps_B\in\mathbb{F}_p, computes PB=sB⋅P∈E(Fp)P_B = s_B\cdot P\in E(\mathbb{F}_p) and sends PBP_B to Alice.
    • By commutativity of scalar multiplication, both Alice and Bob can now compute the point PAB=sAsB⋅PP_{AB} = s_As_B\cdot P, using the elliptic curve point received from the other party and their secret scalar.

    The assumed hardness of computing discrete logarithms in the elliptic curve, ensures that it is not possible to compute sAs_A or sBs_B from PAP_A and PBP_B, respectively. Another security assumption (named Computational Diffie-Hellman assumption) ensures that it is not possible to compute PABP_{AB} from PP, PAP_A and PBP_B. Hence the point PABP_{AB} shared by Alice and Bob at the end of the above protocol cannot be efficiently computed by an attacker intercepting PAP_A and PBP_B, and can then be used to generate a secret to be later employed, for example, as a symmetric encryption key.

    On a side note, this protocol shows the interplay between two components typical to public-key based schemes: the scalars sAs_A and sBs_B can be seen as private keys associated to the public keys PAP_A and PBP_B, respectively, which allow Alice and Bob only to compute the shared secret point PABP_{AB}.

    Ephemeral and Static Public Keys​

    Although we assumed that it is practically impossible for an attacker to compute the randomly picked secret scalar from the corresponding public elliptic curve point, it may happen that such scalar gets compromised or can be guessed due to a faulty employed random number generator. In such cases, an attacker will be able to recover the final shared secret and all encryption keys eventually derived from that, with clear catastrophic consequences for the privacy of exchanged messages.

    To mitigate such issues, multiple DH operations can be combined using two different types of exchanged elliptic curve points or, better, public keys: ephemeral keys, that is random keys used only once in a DH operation, and long-term static keys, used mainly for authentication purposes since employed multiple times.

    Just to provide an example, let us suppose Alice and Bob perform the following custom DH-based key-exchange protocol:

    • Alice generates an ephemeral key EA=eA⋅PE_A=e_A\cdot P by picking a random scalar eAe_A and sends EAE_A to Bob;
    • Similarly, Bob generates an ephemeral key EB=eB⋅PE_B=e_B\cdot P and sends EBE_B to Alice;
    • Alice and Bob computes EAB=eAeB⋅PE_{AB} = e_Ae_B \cdot P and from it derive a secret encryption key kk.
    • Bob sends to Alice his static key SB=sB⋅PS_B = s_B\cdot P encrypted with kk.
    • Alice encrypts with kk her static key SA=sA⋅PS_A = s_A\cdot P and sends it to Bob.
    • Alice and Bob decrypt the received static keys, compute the secret SAB=sAsB⋅PS_{AB} = s_As_B \cdot P and use it together with EABE_{AB} to derive a new encryption key k~\tilde{k} to be later used with a symmetric cipher.

    In this protocol, if Alice's and/or Bob's static keys get compromised, it would not possible to derive the final secret key k~\tilde{k}, since at least one ephemeral key among EAE_A and EBE_B has to be compromised too in order to recover the secret EABE_{AB}. Furthermore, since Alice's and Bob's long-term static keys are encrypted, an attacker intercepting exchanged (encrypted) public keys will not be able to link such communication to Alice or Bob, unless one of the ephemeral key is compromised (and, even in such case, none of the messages encrypted under the key k~\tilde{k} can be decrypted).

    The Noise Protocol Framework​

    In previous section we gave a small intuition on how multiple DH operations over ephemeral and static users' public keys can be combined to create different key-exchange protocols.

    The Noise Protocol Framework, defines various rules for building custom key-exchange protocols while allowing easy analysis of the security properties and threat models provided given the type and order of the DH operations employed.

    In Noise terminology, a key-agreement or Noise protocol consists of one or more Noise handshakes. During a Noise handshake, Alice and Bob exchange multiple (handshake) messages containing their ephemeral keys and/or static keys. These public keys are then used to perform a handshake-dependent sequence of Diffie-Hellman operations, whose results are all hashed into a shared secret key. Similarly as we have seen above, after a handshake is complete, each party will use the derived secret key to send and receive authenticated encrypted data by employing a symmetric cipher.

    Depending on the handshake pattern adopted, different security guarantees can be provided on messages encrypted using a handshake-derived key.

    The Noise handshakes we support in Waku all provide the following security properties:

    • Confidentiality: the adversary should not be able to learn what data is being sent between Alice and Bob.
    • Strong forward secrecy: an active adversary cannot decrypt messages nor infer any information on the employed encryption key, even in the case he has access to Alice's and Bob's long-term private keys (during or after their communication).
    • Authenticity: the adversary should not be able to cause either Alice or Bob to accept messages coming from a party different than their original senders.
    • Integrity: the adversary should not be able to cause Alice or Bob to accept data that has been tampered with.
    • Identity-hiding: once a secure communication channel is established, a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient by knowing their long-term static keys.

    We refer to Noise specification for more formal security definitions and precise threat models relative to Waku supported Noise Handshake patterns.

    Message patterns​

    Noise handshakes involving DH operations over ephemeral and static keys can be succinctly sketched using the following set of handshake message tokens: e,s,ee,se,es,ss.

    Tokens employing single letters denote (the type of) users' public keys: e refers to randomly generated ephemeral key(s), while s indicates the users' long-term static key(s).

    Two letters tokens, instead, denotes DH operations over the two users' public keys the token refers to, given that the left token letter refers to the handshake initiator's public key, while the right token letter indicates the used responder's public key. Thus, if Alice started a handshake with Bob, the es token will shortly represent a DH operation among Alice's ephemeral key e and Bob's static key s.

    Since, in order to perform any DH operations users need to share (or pre-share) the corresponding public keys, Noise compactly represents messages' exchanges using the two direction -> and <-, where the -> denotes a message (arbitrary and/or DH public key) from the initiator to the responder, while <- the opposite.

    Hence a message pattern consisting of a direction and one or multiple tokens such as <- e, s, es has to be interpreted one token at a time: in this example, the responder is sending his ephemeral and static key to the initiator and is then executing a DH operation over the initiator's ephemeral key e (shared in a previously exchanged message pattern) and his static key s. On the other hand, such message indicates also that the initiator received the responder's ephemeral and static keys e and s, respectively, and performed a DH operation over his ephemeral key and the responder's just received static key s. In this way, both parties will be able to derive at the end of each message pattern processed the same shared secret, which is eventually used to update any derived symmetric encryption keys computed so far.

    In some cases, DH public keys employed in a handshake are pre-shared before the handshake itself starts. In order to chronologically separate exchanged keys and DH operations performed before and during a handshake, Noise employs the ... delimiter.

    For example, the following message patterns

    <- e
    ...
    -> e, ee

    indicates that the initiator knew the responder's ephemeral key before he sends his own ephemeral key and executes a DH operation between both parties ephemeral keys (similarly, the responder receives the initiator's ephemeral key and does a ee DH operation).

    At this point it should be clear how such notation is able to compactly represent a large variety of DH based key-agreements. Nevertheless, we can easily define additional tokens and processing rules in order to address specific applications and security requirements, such as the psk token used to process arbitrary pre-shared key material.

    As an example of Noise flexibility, the custom protocol we detailed above can be shortly represented as (Alice is on the left):

    -> e
    <- e, ee, s
    -> s, ss

    where after each DH operation an encryption key is derived (along with the secrets computed by all previously executed DH operations) in order to encrypt/decrypt any subsequent sent/received message.

    Another example is given by the possibility to replicate within Noise the well established Signal's X3DH key-agreement protocols, thus making the latter a general framework to design and study security of many practical and widespread DH-based key-exchange protocols.

    The Noise State Objects​

    We mentioned multiple times that parties derive an encryption key each time they perform a DH operation, but how does this work in more details?

    Noise defines three state object: a Handshake State, a Symmetric State and a Cipher State, each encapsulated into each other and instantiated during the execution of a handshake.

    The Handshake State object stores the user's and other party's received ephemeral and static keys (if any) and embeds a Symmetric State object.

    The Symmetric State, instead, stores a handshake hash value h, iteratively updated with any message read/received and DH secret computed, and a chaining key ck, updated using a key derivation function every time a DH secret is computed. This object further embeds a Cipher State.

    Lastly, the Cipher State stores a symmetric encryption k key and a counter n used to encrypt and decrypt messages exchanged during the handshake (not only static keys, but also arbitrary payloads). These key and counter are refreshed every time the chaining key is updated.

    While processing each handshake's message pattern token, all these objects are updated according to some specific processing rules which employ a combination of public-key primitives, hash and key-derivation functions and symmetric ciphers. It is important to note, however, that at the end of each processed message pattern, the two users will share the same Symmetric and Cipher State embedded in their respective Handshake States.

    Once a handshake is complete, users derive two new Cipher States and can then discard the Handshake State object (and, thus, the embedded Symmetric State and Cipher State objects) -employed during the handshake.

    These two Cipher states are used to encrypt and decrypt all outbound and inbound after-handshake messages, respectively, and only to these will be granted the confidentiality, authenticity, integrity and identity-hiding properties we detailed above.

    For more details on processing rules, we refer to Noise specifications.

    Supported Noise Handshakes in Waku​

    The Noise handshakes we provided support to in Waku address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:

    • Alice and Bob know each others' static key.
    • Alice knows Bob's static key;
    • Alice and Bob share no key material and they don't know each others' static key.
    • Alice and Bob share some key material, but they don't know each others' static key.

    The possibility to have handshakes based on the reciprocal knowledge parties have of each other, allows designing Noise handshakes that can quickly reach the desired level of security on exchanged encrypted messages while keeping the number of interactions between Alice and Bob minimum.

    Nonetheless, due to the pure token-based nature of handshake processing rules, implementations can easily add support to any custom handshake pattern with minor modifications, in case more specific application use-cases need to be addressed.

    On a side note, we already mentioned that identity-hiding properties can be guaranteed against a passive attacker that only reads the communication occurring between Alice and Bob. However, an active attacker who compromised one party's static key and actively interferes with the parties' exchanged messages, may lower the identity-hiding security guarantees provided by some handshake patterns. In our security model we exclude such adversary, but, for completeness, in the following we report a summary of possible de-anonymization attacks that can be performed by such an active attacker.

    For more details on supported handshakes and on how these are implemented in Waku, we refer to WAKU2-NOISE RFC.

    The K1K1 Handshake​

    If Alice and Bob know each others' static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a K1K1 handshake. In Noise notation (Alice is on the left) this can be sketched as:

     K1K1:
        ->  s
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  se

    We note that here only ephemeral keys are exchanged. This handshake is useful in case Alice needs to instantiate a new separate encrypted communication channel with Bob, e.g. opening multiple parallel connections, file transfers, etc.

    Security considerations on identity-hiding (active attacker): no static key is transmitted, but an active attacker impersonating Alice can check candidates for Bob's static key.

    The XK1 Handshake​

    Here, Alice knows how to initiate a communication with Bob and she knows his public static key: such discovery can be achieved, for example, through a publicly accessible register of users' static keys, smart contracts, or through a previous public/private advertisement of Bob's static key.

    A Noise handshake pattern that suits this scenario is XK1:

     XK1:
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  s, se

    Within this handshake, Alice and Bob reciprocally authenticate their static keys s using ephemeral keys e. We note that while Bob's static key is assumed to be known to Alice (and hence is not transmitted), Alice's static key is sent to Bob encrypted with a key derived from both parties ephemeral keys and Bob's static key.

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob's static key against recorded/accepted exchanged handshake messages.

    The XX and XXpsk0 Handshakes​

    If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an XX handshake, where each party tranXmits to the other its own static key.

    The handshake goes as follows:

     XX:
        ->  e
        <-  e, ee, s, es
        ->  s, se

    We note that the main difference with XK1 is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange.

    This handshake can be slightly changed in case both Alice and Bob pre-shares some secret psk which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called XXpsk0, goes as follow:

     XXpsk0:
        ->  psk, e
        <-  e, ee, s, es
        ->  s, se

    The main difference with XX is that Alice's and Bob's static keys, when transmitted, would be encrypted with a key derived from psk as well.

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party for both XX and XXpsk0 handshakes. In XX, Bob's static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In XXpsk0, instead, Bob's secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret psk but not through any static key), provided that psk was not previously compromised (in such case identity-hiding properties provided by the XX handshake applies).

    Session Management and Multi-Device Support​

    When two users complete a Noise handshake, an encryption/decryption session - or Noise session - consisting of two Cipher States is instantiated.

    By identifying Noise session with a session-id derived from the handshake's cryptographic material, we can take advantage of the PubSub/GossipSub protocols used by Waku for relaying messages in order to manage instantiated Noise sessions.

    The core idea is to exchange after-handshake messages (encrypted with a Cipher State specific to the Noise session), over a content topic derived from the (secret) session-id the corresponding session refers to.

    This allows to decouple the handshaking phase from the actual encrypted communication, thus improving users' identity-hiding capabilities.

    Furthermore, by publicly revealing a value derived from session-id on the corresponding session content topic, a Noise session can be marked as stale, enabling peers to save resources by discarding any eventually stored message sent to such content topic.

    One relevant aspect in today's applications is the possibility for users to employ different devices in their communications. In some cases, this is non-trivial to achieve since, for example, encrypted messages might be required to be synced on different devices which do not necessarily share the necessary key material for decryption and may be temporarily offline.

    We address this by requiring each user's device to instantiate multiple Noise sessions either with all user's other devices which, in turn, all together share a Noise session with the other party, or by directly instantiating a Noise session with all other party's devices.

    We named these two approaches N11MN11M and NMNM, respectively, which are in turn loosely based on the paper “Multi-Device for Signal” and Signal’s Sesame Algorithm.

    Informally, in the N11MN11M session management scheme, once the first Noise session between any of Alice’s and Bob’s device is instantiated, its session information is securely propagated to all other devices using previously instantiated Noise sessions. Hence, all devices are able to send and receive new messages on the content topic associated to such session.

    In the NMNM session management scheme, instead, all pairs of Alice's and Bob's devices have a distinct Noise session: a message is then sent from the currently-in-use sender’s device to all recipient’s devices, by properly encrypting and sending it to the content topics of each corresponding Noise session. If sent messages should be available on all sender’s devices as well, we require each pair of sender’s devices to instantiate a Noise session used for syncing purposes.

    For more technical details on how Noise sessions are instantiated and managed within these two mechanisms and the different trade-offs provided by the latter, we refer to WAKU2-NOISE-SESSIONS.

    Conclusions​

    In this post we provided an overview of Noise, a protocol framework for designing Diffie-Hellman based key-exchange mechanisms allowing systematic security and threat model analysis.

    The flexibility provided by Noise components allows not only to fully replicate with same security guarantees well established key-exchange primitives such as X3DH, currently employed by Status TRANSPORT-SECURITY, but enables also optimizations based on the reciprocal knowledge parties have of each other while allowing easier protocols' security analysis and (formal) verification.

    Furthermore, different handshakes can be combined and executed one after each other, a particularly useful feature to authenticate multiple static keys employed by different applications but also to ease keys revocation.

    The possibility to manage Noise sessions over multiple devices and the fact that handshakes can be concretely instantiated using modern, fast and secure cryptographic primitives such as ChaChaPoly and BLAKE2b, make Noise one of the best candidates for efficiently and securely address the many different needs of applications built on top of Waku requiring key-agreement.

    Future steps​

    The available implementation of Noise in nwaku, although mostly complete, is still in its testing phase. As future steps we would like to:

    • have an extensively tested and robust Noise implementation;
    • formalize, implement and test performances of the two proposed N11MN11M and NMNM session management mechanisms and their suitability for common use-case scenarios;
    • provide Waku network nodes a native protocol to readily support key-exchanges, strongly-encrypted communication and multi-device session management mechanisms with none-to-little interaction besides applications' connection requests.

    References​

    ]]>     - - s1fr0 - -     - - <![CDATA[Waku v2 Ambient Peer Discovery]]> - https://vac.dev/rlog/wakuv2-apd - - 2022-05-09T10:00:00.000Z - - Introducing and discussing ambient peer discovery methods currently used by Waku v2, as well as future plans in this area.

    Waku v2 comprises a set of modular protocols for secure, privacy preserving communication. -Avoiding centralization, these protocols exchange messages over a P2P network layer. -In order to build a P2P network, participating nodes first have to discover peers within this network. -This is where ambient peer discovery comes into play: -it allows nodes to find peers, making it an integral part of any decentralized application.

    In this post the term node to refers to our endpoint or the endpoint that takes action, -while the term peer refers to other endpoints in the P2P network. -These endpoints can be any device connected to the Internet: e.g. servers, PCs, notebooks, mobile devices, or applications like a browser. -As such, nodes and peers are the same. We use these terms for the ease of explanation without loss of generality.

    In Waku's modular design, ambient peer discovery is an umbrella term for mechanisms that allow nodes to find peers. -Various ambient peer discovery mechanisms are supported, and each is specified as a separate protocol. -Where do these protocols fit into Waku's protocol stack? -The P2P layer of Waku v2 builds on libp2p gossipsub. -Nodes participating in a gossipsub protocol manage a mesh network that is used for routing messages. -This mesh network is an unstructured P2P network -offering high robustness and resilience against attacks. -Gossipsub implements many improvements overcoming the shortcomings typically associated with unstructured P2P networks, e.g. inefficient flooding based routing. -The gossipsub mesh network is managed in a decentralized way, which requires each node to know other participating peers. -Waku v2 may use any combination of its ambient discovery protocols to find appropriate peers.

    Summarizing, Waku v2 comprises a peer management layer based on libp2p gossipsub, -which manages the peers of nodes, and an ambient peer discovery layer, -which provides information about peers to the peer management layer.

    We focus on ambient peer discovery methods that are in line with our goal of building a fully decentralized, generalized, privacy-preserving and censorship-resistant messaging protocol. -Some of these protocols still need adjustments to adhere to our privacy and anonymity requirements. For now, we focus on operational stability and feasibility. -However, when choosing techniques, we pay attention to selecting mechanisms that can feasibly be tweaked for privacy in future research efforts. -Because of the modular design and the fact that Waku v2 has several discovery methods at its disposal, we could even remove a protocol in case future evaluation deems it not fitting our standards.

    This post covers the current state and future considerations of ambient peer discovery for Waku v2, -and gives reason for changes and modifications we made or plan to make. -The ambient peer discovery protocols currently supported by Waku v2 are a modified version of Ethereum's Discovery v5 -and DNS-based discovery. -Waku v2 further supports gossipsub's peer exchange protocol. -In addition, we plan to introduce protocols for general peer exchange and capability discovery, respectively. -The former allows resource restricted nodes to outsource querying for peers to stronger peers, -the latter allows querying peers for their supported capabilities. -Besides these new protocols, we are working on integrating capability discovery in our existing ambient peer discovery protocols.

    Static Node Lists​

    The simplest method of learning about peers in a P2P network is via static node lists. -These can be given to nodes as start-up parameters or listed in a config-file. -They can also be provided in a script-parseable format, e.g. in JSON. -While this method of providing bootstrap nodes is very easy to implement, it requires static peers, which introduce centralized elements. -Also, updating static peer information introduces significant administrative overhead: -code and/or config files have to be updated and released. -Typically, static node lists only hold a small number of bootstrap nodes, which may lead to high load on these nodes.

    DNS-based Discovery​

    Compared to static node lists, -DNS-based discovery (specified in EIP-1459) -provides a more dynamic way of discovering bootstrap nodes. -It is very efficient, can easily be handled by resource restricted devices and provides very good availability. -In addition to a naive DNS approach, Ethereum's DNS-based discovery introduces efficient authentication leveraging Merkle trees.

    A further advantage over static node lists is the separation of code/release management and bootstrap node management. -However, changing and updating the list of bootstrap nodes still requires administrative privileges because DNS records have to be added or updated.

    While this method of discovery still requires centralized elements, -node list management can be delegated to various DNS zones managed by other entities mitigating centralization.

    Discovery V5​

    A much more dynamic method of ambient peer discovery is Discovery v5, which is Ethereum's peer discovery protocol. -It is based on the Kademlia distributed hashtable (DHT). -An introduction to discv5 and its history, and a discv5 Waku v2 feasibility study -can be found in previous posts on this research log.

    We use Discovery v5 as an ambient peer discovery method for Waku v2 because it is decentralized, efficient, actively researched, and has web3 as its main application area. -Discv5 also offers mitigation techniques for various attacks, which we cover later in this post.

    Using a DHT (structured P2P network) as a means for ambient peer discovery, while using the gossipsub mesh network (unstructured P2P network) for transmitting actual messages, -Waku v2 leverages advantages from both worlds. -One of the main benefits of DHTs is offering a global view over participating nodes. -This, in turn, allows sampling random sets of nodes which is important for equally distributing load. -Gossipsub, on the other hand, offers great robustness and resilience against attacks. -Even if discv5 discovery should not work in advent of a DoS attack, Waku v2 can still operate switching to different discovery methods.

    Discovery methods that use separate P2P networks still depend on bootstrapping, -which Waku v2 does via parameters on start-up or via DNS-based discovery. -This might raise the question of why such discovery methods are beneficial. -The answer lies in the aforementioned global view of DHTs. Without discv5 and similar methods, the bootstrap nodes are used as part of the gossipsub mesh. -This might put heavy load on these nodes and further, might open pathways to inference attacks. -Discv5, on the other hand, uses the bootstrap nodes merely as an entry to the discovery network and can provide random sets of nodes (sampled from a global view) -for bootstrapping or expanding the mesh.

    DHT Background​

    Distributed Hash Tables are a class of structured P2P overlay networks. -A DHT can be seen as a distributed node set of which each node is responsible for a part of the hash space. -In contrast to unstructured P2P networks, e.g. the mesh network maintained by gossipsub, -DHTs have a global view over the node set and the hash space (assuming the participating nodes behave well).

    DHTs are susceptible to various kinds of attacks, especially Sybil attacks -and eclipse attacks. -While security aspects have been addressed in various research papers, general practical solutions are not available. -However, discv5 introduced various practical mitigation techniques.

    Random Walk Discovery​

    While discv5 is based on the Kademlia DHT, it only uses the distributed node set aspect of DHTs. -It does not map values (items) into the distributed hash space. -This makes sense, because the main purpose of discv5 is discovering other nodes that support discv5, which are expected to be Ethereum nodes. -Ethereum nodes that want to discover other Ethereum nodes simply query the discv5 network for a random set of peers. -If Waku v2 would do the same, only a small subset of the retrieved nodes would support Waku v2.

    A first naive solution for Waku v2 discv5 discovery is

    • retrieve a random node set, which is achieved by querying for a set of randomly chosen node IDs
    • filter the returned nodes on the query path based on Waku v2 capability via the Waku v2 ENR
    • repeat until enough Waku v2 capable nodes are found

    This query process boils down to random walk discovery, which is very resilient against attacks, but also very inefficient if the number of nodes supporting the desired capability is small. -We refer to this as the needle-in-the-haystack problem.

    Random Walk Performance Estimation​

    This subsection provides a rough estimation of the overhead introduced by random walk discovery.

    Given the following parameters:

    • nn number of total nodes participating in discv5
    • pp percentage of nodes supporting Waku
    • WW the event of having at least one Waku node in a random sample
    • kk the size of a random sample (default = 16)
    • α\alpha the number of parallel queries started
    • bb bits per hop
    • qq the number of queries

    A query takes log2bnlog_{2^b}n hops to retrieve a random sample of nodes.

    P(W)=1−(1−p/100)kP(W) = 1 - (1-p/100)^k is the probability of having at least one Waku node in the sample.

    P(Wq)=1−(1−p/100)kqP(W^q) = 1 - (1-p/100)^{kq} is the probability of having at least one Waku node in the union of qq samples.

    Expressing this in terms of qq, we can write: -P(Wq)=1−(1−p/100)kq  ⟺  q=log(1−p/100)k(1−P(Wq))P(W^q) = 1 - (1-p/100)^{kq} \iff q = log_{(1-p/100)^k}(1-P(W^q))

    Figure 1 shows a log-log plot for P(Wq)=90%P(W^q) = 90\%.

    Figure 1: log-log plot showing the number of queries necessary to retrieve a Waku v2 node with a probability of 90% in relation to the Waku v2 node concentration in the network.

    Assuming p=0.1p=0.1, we would need

    0.9=1−(1−0.1/100)16q=>q≈1440.9 = 1 - (1-0.1/100)^{16q} => q \approx 144

    queries to get a Waku node with 90% probability, which leads to ≈144∗18=2592\approx 144 * 18 = 2592 overlay hops. -Choosing b=3b=3 would reduce the number to ≈144∗6=864\approx 144 * 6 = 864. -Even when choosing α=10\alpha = 10 we would have to wait at least 80 RTTs. -This effort is just for retrieving a single Waku node. Ideally, we want at least 3 Waku nodes for bootstrapping a Waku relay.

    The discv5 doc roughly estimates p=1p=1% to be the threshold for acceptably efficient random walk discovery. -This is in line with our estimation:

    0.9=1−(1−1/100)16q=>q≈140.9 = 1 - (1-1/100)^{16q} => q \approx 14

    The number of necessary queries is linearly dependent on the percentage pp of Waku nodes. -The number of hops per query is logarithmically dependent on nn. -Thus, random walk searching is inefficient for small percentages pp. -Still, random walks are more resilient against attacks.

    We can conclude that a Waku node concentration below 1% renders vanilla discv5 unfit for our needs. -Our current solution and future plans for solving this issue are covered in the next subsections.

    Simple Solution: Separate Discovery Network​

    The simple solution we currently use for Waku v2 discv5 is a separate discv5 network. -All (well behaving) nodes in this network support Waku v2, resulting in a very high query efficiency. -However, this solution reduces resilience because the difficulty of attacking a DHT scales with the number of participating nodes.

    Discv5 Topic Discovery​

    We did not base our solution on the current version of discv5 topic discovery, -because, similar to random walk discovery, it suffers from poor performance for relatively rare capabilities/topics.

    However, there is ongoing research in discv5 topic discovery which is close to ideas we explored when pondering efficient and resilient Waku discv5 solutions. -We keep a close eye on this research, give feedback, and make suggestions, as we plan to switch to this version of topic discovery in the future.

    In a nutshell, topic discovery will manage separate routing tables for each topic. -These topic specific tables are initialized with nodes from the discv5 routing table. -While the buckets of the discv5 routing table represent distance intervals from the node's node ID, the topic table buckets represent distance intervals from topic IDs.

    Nodes that want to register a topic try to register that topic at one random peer per bucket. -This leads to registering the topic at peers in closer and closer neighbourhoods around the topic ID, which -yields a very efficient and resilient compromise between random walk discovery and DHT discovery. -Peers in larger neighbourhoods around the topic ID are less efficient to discover, however more resilient against eclipse attacks and vice versa.

    Further, this works well with the overload and DoS protection discv5 employs. -Discv5 limits the amount of nodes registered per topic on a single peer. Further, discv5 enforces a waiting time before nodes can register topics at peers. -So, for popular topics, a node might fail to register the topic in a close neighbourhood. -However, because the topic is popular (has a high occurrence percentage pp), it can still be efficiently discovered.

    In the future, we also plan to integrate Waku v2 capability discovery, which will not only allow asking for nodes that support Waku v2, -but asking for Waku v2 nodes supporting specific Waku v2 protocols like filter or store. -For the store protocol we envision sub-capabilities reflecting message topics and time frames of messages. -We will also investigate related security implications.

    Attacks on DHTs​

    In this post, we only briefly describe common attacks on DHTs. -These attacks are mainly used for denial of service (DoS), -but can also used as parts of more sophisticated attacks, e.g. deanonymization attacks. -A future post on this research log will cover security aspects of ambient peer discovery with a focus on privacy and anonymity.

    Sybil Attack

    The power of an attacker in a DHT is proportional to the number of controlled nodes. -Controlling nodes comes at a high resource cost and/or requires controlling a botnet via a preliminary attack.

    In a Sybil attack, an attacker generates lots of virtual node identities. -This allows the attacker to control a large portion of the ID space in a DHT at a relatively low cost. -Sybil attacks are especially powerful when the attacker can freely choose the IDs of generated nodes, -because this allows positioning at chosen points in the DHT.

    Because Sybil attacks amplify the power of many attacks against DHTs, -making Sybil attacks as difficult as possible is the basis for resilient DHT operation. -The typical abstract mitigation approach is binding node identities to physical network interfaces. -To some extend, this can be achieved by introducing IP address based limits. -Further, generating node IDs can be bound by proof of work (PoW), -which, however, comes with a set of shortcomings, e.g. relatively high costs on resource restricted devices. -The discv5 doc -describes both Sybil and eclipse attacks, as well as concrete mitigation techniques employed by discv5.

    Eclipse Attack

    In an eclipse attack, nodes controlled by the attacker poison the routing tables of other nodes in a way that parts of the DHT become eclipsed, i.e. invisible. -When a controlled node is asked for the next step in a path, -it provides another controlled node as the next step, -effectively navigating the querying node around or away from certain areas of the DHT. -While several mitigation techniques have been researched, there is no definitive protection against eclipse attacks available as of yet. -One mitigation technique is increasing α\alpha, the number of parallel queries, and following each concurrent path independently for the lookup.

    The eclipse attack becomes very powerful in combination with a successful Sybil attack; -especially when the attacker can freely choose the position of the Sybil nodes.

    The aforementioned new topic discovery of discv5 provides a good balance between protection against eclipse attacks and query performance.

    Peer Exchange Protocol​

    While discv5 based ambient peer discovery has many desirable properties, resource restricted nodes and nodes behind restrictive NAT setups cannot run discv5 satisfactory. -With these nodes in mind, we started working on a simple peer exchange protocol based on ideas proposed here. -The peer exchange protocol will allow nodes to ask peers for additional peers. -Similar to discv5, the peer exchange protocol will also support capability discovery.

    The new peer exchange protocol can be seen as a simple replacement for the Rendezvous protocol, which Waku v2 does not support. -While the rendezvous protocol involves nodes registering at rendezvous peers, the peer exchange protocol simply allows nodes to ask any peer for a list of peers (with a certain set of capabilities). -Rendezvous tends to introduce centralized elements as rendezvous peers have a super-peer role.

    In the future, we will investigate resource usage of Waku v2 discv5 and provide suggestions for minimal resources nodes should have to run discv5 satisfactory.

    Waku v2 comprises further protocols related to ambient peer discovery. We shortly mention them for context, even though they are not strictly ambient peer discovery protocols.

    Gossipsub Peer Exchange Protocol​

    Gossipsub provides an integrated peer exchange mechanism which is also supported by Waku v2. -Gossipsub peer exchange works in a push manner. Nodes send peer lists to peers they prune from the active mesh. -This pruning is part of the gossipsub peer management, blurring the boundaries of peer management and ambient peer discovery.

    We will investigate anonymity implications of this protocol and might disable it in favour of more anonymity-preserving protocols. -Sending a list of peers discloses information about the sending node. -We consider restricting these peer lists to cached peers that are currently not used in the active gossipsub mesh.

    Capability Negotiation​

    Some of the ambient peer discovery methods used by Waku2 will support capability discovery. -This allows to narrow down the set of retrieved peers to peers that support specific capabilities. -This is efficient because it avoids establishing connections to nodes that we are not interested in.

    However, the ambient discovery interface does not require capability discovery, which will lead to nodes having peers with unknown capabilities in their peer lists. -We work on a capability negotiation protocol which allows nodes to ask peers

    • for their complete list of capabilities, and
    • whether they support a specific capability

    We will investigate security implications, especially when sending full capability lists.

    NAT traversal​

    For NAT traversal, Waku v2 currently supports the port mapping protocols UPnP and NAT-PMP / PCP.

    In the future, we plan to add support for parts of ICE, e.g. STUN. -We do not plan to support TURN because TURN relays would introduce a centralized element. -A modified decentralized version of TURN featuring incentivization might be an option in the future; -strong peers could offer a relay service similar to TURN.

    There are plans to integrate more NAT traversal into discv5, in which we might participate. -So far, the only traversal technique supported by discv5 is nodes receiving their external IP address in pong messages.

    While NAT traversal is very important, adding more NAT traversal techniques is not a priority at the moment. -Nodes behind restrictive symmetric NAT setups cannot be discovered, but they can still discover peers in less restrictive setups. -While we wish to have as many nodes as possible to be discoverable via ambient peer discovery, two nodes behind a restrictive symmetric NAT can still exchange Waku v2 messages if they discovered a shared peer. -This is one of the nice resilience related properties of flooding based routing algorithms.

    For mobile nodes, which suffer from changing IP addresses and double NAT setups, we plan using the peer exchange protocol to ask peers for more peers. -Besides saving resources on resource restricted devices, this approach works as long as peers are in less restrictive environments.

    Conclusion and Future Prospects​

    Ambient peer discovery is an integral part of decentralized applications. It allows nodes to learn about peers in the network. -As of yet, Waku v2 supports DNS-based discovery and a slightly modified version of discv5. -We are working on further protocols, including a peer exchange protocol that allows resource restricted nodes to ask stronger peers for peer lists. -Further, we are working on adding capability discovery to our ambient discovery protocols, allowing nodes to find peers with desired properties.

    These protocols can be combined in a modular way and allow Waku v2 nodes to build a strong and resilient mesh network, -even if some discovery methods are not available in a given situation.

    We will investigate security properties of these discovery mechanisms with a focus on privacy and anonymity in a future post on this research log. -As an outlook we can already state that DHT approaches typically allow inferring information about the querying node. -Further, sending peer lists allows inferring the position of a node within the mesh, and by extension information about the node. -Waku v2 already provides some mitigation, because the mesh for transmitting actual messages, and the peer discovery network are separate. -To mitigate information leakage by transmitting peer lists, we plan to only reply with lists of peers that nodes do not use in their active meshes.

    References​

    ]]>     - - Daniel - -     - - <![CDATA[Introducing nwaku]]> - https://vac.dev/rlog/introducing-nwaku - - 2022-04-12T10:00:00.000Z - - Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.

    Background​

    If you've been following our research log, -you'll know that many things have happened in the world of Waku v2 since our last general update. -In line with our long term goals, -we've introduced new protocols, -tweaked our existing protocols -and expanded our team. -We've also shown in a series of practical experiments that Waku v2 does indeed deliver on some of the theoretical advantages it was designed to have over its predecessor, Waku v1. -A sustainability and business workshop led to the formulation of a clearer vision for Vac as a team.

    From the beginning, our protocol development has been complemented by various client implementations of these protocols, -first in Nim, -but later also in JavaScript -and Go. -A follow-up post will clarify the purposes, similarities and differences between these three clients. -The Nim client, is our reference implementation, -developed by the research team in parallel with the specs -and building on a home-grown implementation of libp2p. -The Nim client is suitable to run as a standalone adaptive node, -managed by individual operators -or as an encapsulated service node in other applications. -This post looks at some recent developments within the Nim client.

    1. nim-waku is now known as nwaku​

    Pronounced NWHA-koo. -You may already have seen us refer to "nwaku" on Vac communication channels, -but it is now official: -The nim-waku Waku v2 client has been named nwaku. -Why? Well, we needed a recognizable name for our client that could easily be referred to in everyday conversations -and nim-waku just didn't roll off the tongue. -We've followed the example of the closely related nimbus project to find a punchier name -that explicitly links the client to both the Waku set of protocols and the Nim language.

    2. Improvements in stability and performance​

    The initial implementation of Waku v2 demonstrated how the suite of protocols can be applied -to form a generalized, peer-to-peer messaging network, -while addressing a wide range of adaptive requirements. -This allowed us to lift several protocol specifications from raw to draft status, -indicating that a reference implementation exists for each. -However, as internal dogfooding increased and more external applications started using nwaku, -we stepped up our focus on the client's stability and performance. -This is especially true where we want nwaku to run unsupervised in a production environment -without any degradation in the services it provides.

    Some of the more significant productionization efforts over the last couple of months included:

    1. Reworking the store implementation to maintain stable memory usage -while storing historical messages -and serving multiple clients querying history simultaneously. -Previously, a store node would see gradual service degradation -due to inefficient memory usage when responding to history queries. -Queries that often took longer than 8 mins now complete in under 100 ms.

    2. Improved peer management. -For example, filter nodes will now remove unreachable clients after a number of connection failures, -whereas they would previously keep accumulating dead peers.

    3. Improved disk usage. -nwaku nodes that persist historical messages on disk now manage their own storage size based on the --store-capacity. -This can significantly improve node start-up times.

    More stability issues may be addressed in future as nwaku matures, -but we've noticed a marked improvement in the reliability of running nwaku nodes. -These include environments where nwaku nodes are expected to run with a long uptime. -Vac currently operates two long-running fleets of nwaku nodes, wakuv2.prod and wakuv2.test, -for internal dogfooding and -to serve as experimental bootstrapping nodes. -Status has also recently deployed similar fleets for production and testing based on nwaku. -Our goal is to have nwaku be stable, performant and flexible enough -to be an attractive option for operators to run and maintain their own Waku v2 nodes. -See also the future work section below for more on our general goal of nwaku for operators.

    3. Improvements in interoperability​

    We've implemented several features that improve nwaku's usability in different environments -and its interoperability with other Waku v2 clients. -One major step forward here was adding support for both secure and unsecured WebSocket connections as libp2p transports. -This allows direct connectivity with js-waku -and paves the way for native browser usage. -We've also added support for parsing and resolving DNS-type multiaddrs, -i.e. multiaddress protocol schemes dns, dns4, dns6 and dnsaddr. -A nwaku node can now also be configured with its own IPv4 DNS domain name -allowing dynamic IP address allocation without impacting a node's reachability by its peers.

    4. Peer discovery​

    Peer discovery is the method by which nodes become aware of each other’s existence. -The question of peer discovery in a Waku v2 network has been a focus area since the protocol was first conceptualized. -Since then several different approaches to discovery have been proposed and investigated. -We've implemented three discovery mechanisms in nwaku so far:

    DNS-based discovery​

    nwaku nodes can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. -Our implementation is based on EIP-1459.

    GossipSub peer exchange​

    GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism -whereby a pruning peer may provide a pruned peer with a set of alternative peers -where it can connect to reform its mesh. -This is a very suitable mechanism to gradually discover more peers -from an initial connection to a small set of bootstrap peers. -It is enabled in a nwaku node by default.

    Waku Node Discovery Protocol v5​

    This is a DHT-based discovery mechanism adapted to store and relay node records. -Our implementation is based on Ethereum's Discovery v5 protocol -with some minor modifications to isolate our discovery network from that of Ethereum. -The decision to separate the Waku Discovery v5 network from Ethereum's was made on considerations of lookup efficiency. -This comes at a possible tradeoff in network resilience. -We are considering merging with the Ethereum Discovery v5 network in future, -or even implement a hybrid solution. -This post explains the decision and future steps.

    5. Spam protection using RLN​

    An early addition to our suite of protocols was an extension of 11/WAKU-RELAY -that provided spam protection using Rate Limiting Nullifiers (RLN). -The nwaku client now contains a working demonstration and integration of RLN relay. -Check out this tutorial to see the protocol in action using a toy chat application built on nwaku. -We'd love for people to join us in dogfooding RLN spam protection as part of our operator incentive testnet. -Feel free to join our Vac Discord server -and head to the #rln channel for more information.

    Future work​

    As we continue working towards our goal of a fully decentralized, generalized and censorship-resistant messaging protocol, -these are some of the current and future focus areas for nwaku:

    Reaching out to operators:​

    We are starting to push for operators to run and maintain their own Waku v2 nodes, -preferably contributing to the default Waku v2 network as described by the default pubsub topic (/waku/2/default-waku/proto). -Amongst other things, a large fleet of stable operator-run Waku v2 nodes will help secure the network, -provide valuable services to a variety of applications -and ensure the future sustainability of both Vac as a research organization and the Waku suite of protocols.

    We are targeting nwaku as the main option for operator-run nodes.
    -Specifically, we aim to provide through nwaku:

    1. a lightweight and robust Waku v2 client. -This client must be first in line to support innovative and new Waku v2 protocols, -but configurable enough to serve the adaptive needs of various operators.
    2. an easy-to-follow guide for operators to configure, -set up and maintain their own nodes
    3. a set of operator-focused tools to monitor and maintain a running node

    Better conversational security layer guarantees​

    Conversational security guarantees in Waku v2 are currently designed around the Status application. -Developers building their own applications on top of Waku would therefore -either have to reimplement a set of tools similar to Status -or build their own security solutions on the application layer above Waku. -We are working on a set of features built into Waku -that will provide the general security properties Waku users may desire -and do so in a modern and simple way. -This is useful for applications outside of Status that want similar security guarantees. -As a first step, we've already made good progress toward integrating noise handshakes as a key exchange mechanism in Waku v2.

    Protocol incentivization​

    We want to design incentivization around our protocols to encourage desired behaviors in the Waku network, -rewarding nodes providing costly services -and punishing adversarial actions. -This will increase the overall security of the network -and encourage operators to run their own Waku nodes. -In turn, the sustainability of Vac as an organization will be better guaranteed. -As such, protocol incentivization was a major focus in our recent Vac Sustainability and Business Workshop. -Our first step here is to finish integrating RLN relay into Waku -with blockchain interaction to manage members, -punish spammers -and reward spam detectors. -After this, we want to design monetary incentivization for providers of store, lightpush and filter services. -This may also tie into a reputation mechanism for service nodes based on a network-wide consensus on service quality. -A big challenge for protocol incentivization is doing it in a private fashion, -so we can keep similar metadata protection guarantees as the Waku base layer. -This ties into our focus on Zero Knowledge tech.

    Improved store capacity​

    The nwaku store currently serves as an efficient in-memory store for historical messages, -dimensioned by the maximum number of messages the store node is willing to keep. -This makes the nwaku store appropriate for keeping history over a short term -without any time-based guarantees, -but with the advantage of providing fast responses to history queries. -Some applications, such as Status, require longer-term historical message storage -with time-based dimensioning -to guarantee that messages will be stored for a specified minimum period. -Because of the relatively high cost of memory compared to disk space, -a higher capacity store, with time guarantees, should operate as a disk-only database of historical messages. -This is an ongoing effort.

    Multipurpose discovery​

    In addition to the three discovery methods already implemented in nwaku, -we are working on improving discovery on at least three fronts:

    Capability discovery:​

    Waku v2 nodes may be interested in peers with specific capabilities, for example:

    1. peers within a specific pubsub topic mesh,
    2. peers with store capability,
    3. store peers with x days of history for a specific content topic, etc.

    Capability discovery entails mechanisms by which such capabilities can be advertised and discovered/negotiated. -One major hurdle to overcome is the increased complexity of finding a node with specific capabilities within the larger network (a needle in a haystack). -See the original problem statement for more.

    Improvements in Discovery v5​

    Of the implemented discovery methods, -Discovery v5 best addresses our need for a decentralized and scalable discovery mechanism. -With the basic implementation done, -there are some improvements planned for Discovery v5, -including methods to increase security such as merging with the Ethereum Discovery v5 network, -introducing explicit NAT traversal -and utilizing topic advertisement. -The Waku v2 Discovery v5 Roadmap contains more details.

    Generalized peer exchange​

    nwaku already implements GossipSub peer exchange. -We now need a general request-response mechanism outside of GossipSub -by which a node may learn about other Waku v2 nodes -by requesting and receiving a list of peers from a neighbor. -This could, for example, be a suitable way for resource-restricted devices to request a stronger peer -to perform a random Discovery v5 lookup on their behalf -or simply to be informed of a subset of the peers known to that neighbor. -See this issue for more.

    This concludes a general outline of some of the main recent developments in the nwaku client -and a summary of the current and future focus areas. -Much more is happening behind the scenes, of course, -so for more information, or to join the conversation, -feel free to join our Vac Discord server -or to check out the nwaku repo on Github. -You can also view the changelog for past releases here.

    References​

    ]]>     - - Hanno Cornelius - -     - - <![CDATA[Opinion: Pseudo-ethics in the Surveillance Tech Industry]]> - https://vac.dev/rlog/ethics-surveillance-tech - - 2021-12-03T10:00:00.000Z - - A look at typical ethical shortfalls in the global surveillance tech industry.

    This is an opinion piece by pseudonymous contributor, circe.

    Preface​

    The Vac team aims to provide a public good in the form of freely available, open source tools and protocols for decentralized communication. -As such, we value our independence and the usefulness of our protocols for a wide range of applications. -At the same time, we realize that all technical development, including ours, has a moral component. -As a diverse team we are guided by a shared devotion to the principles of human rights and liberty. -This explains why we place such a high premium on security, censorship-resistance and privacy - -a stance we share with the wider Status Network. -The post below takes a different approach from our usual more technical analyses, -by starting to peel back the curtain on the ethical shortfalls of the global surveillance tech industry.

    Spotlight on an industry​

    Apple's announcement of their lawsuit against Israel's NSO Group -marks the latest in a series of recent setbacks for the surveillance tech company. -In early November, the United States blacklisted the firm, -citing concerns about the use of their spyware by foreign governments targeting civilians such as "journalists, businesspeople, activists" and more. -The company is already embroiled in a lawsuit with Whatsapp -over their exploit of the chat app's video calling service to install malware on target devices. -NSO Group's most infamous product, Pegasus, operates as a hidden exploit installed on victims' mobile phones, -sometimes without even requiring as much as an unguarded click on a malicious link. -It has the potential to lay bare, and report to its owners, everything within the reach of the infected device. -For most people this amounts to a significant portion of their private lives and thoughts. -Pegasus can read your private messages (even encrypted), collect your passwords, record calls, track your location and access your device's microphone and camera. -No activity or application on an infected phone would be hidden.

    The latest controversies are perhaps less because of the novelty of the revelations - -the existence of Pegasus has been known to civil activists since at least 2016. -Rather, the public was reminded again of the potential scope of surveillance tech -in the indiscriminate use of Pegasus on private citizens. -This has far-reaching implications for human freedoms worldwide. -Earlier this year, a leaked list of over 50,000 targets, or possible targets, of Pegasus included -the phone numbers of human rights advocates, independent journalists, lawyers and political activists. -This should have come as no surprise. -The type of autocratically inclined agents, and governments, who would venture to buy and use such invasive cyber-arms often target those they find politically inconvenient. -Pegasus, and similar technologies, simply extend the reach and capacity of such individuals and governments - -no border or distance, no political rank or social advantage, no sanctity of profession or regard for dignity, -provide any indemnity from becoming a victim. -Your best hope is to remain uninteresting enough to escape consideration.

    The NSO Group has, of course, denied allegations of culpability and questions the authenticity of the list. -At this stage, the latter is almost beside the point: -Amnesty International's cybersecurity team, Security Lab, did find forensic evidence of Pegasus on the phones of several volunteers whose numbers appeared on the original list, -including those of journalists and human rights activists. -(Security Lab has since opened up their infection finding tool to the public.) -French intelligence has similarly inspected and confirmed infection of at least three devices belonging to journalists. -The phones of several people who were close to the Saudi-American journalist, Jamal Khashoggi, were confirmed hacked -both before and after Khashoggi's brutal murder at the Saudi embassy in Istanbul in 2018. -More reports of confirmed Pegasus hacks are still published with some regularity. -It is now an open secret that many authoritarian governments have bought Pegasus. -It's not difficult to extrapolate from existing reports and such clients' track records -what the potential injuries to human freedoms are that they can inflict with access to such a powerful cyberweapon.

    A typical response​

    NSO's response to the allegations follows a textbook approach -of avoiding earnest ethical introspection on the manufacturing, and selling, of cyber-arms. -Firstly, shift ethical responsibility to a predetermined process, a list of checkboxes of your own making. -The Group, for example, claims to sell only to "vetted governments", following a classification process -of which they have now published some procedural details but no tangible criteria. -The next step is to reaffirm continuously, and repetitively, your dedication to the legal combat against crime, -"legitimate law enforcement agencies" (note the almost tautological phrasing), -adherence to international arms trade laws, -compliance clauses in customer contracts, etc. -Thirdly, having been absolved of any moral suspicions that might exist about product and process, -from conception to engineering to trade, -distance yourself from the consequences of its use in the world. -"NSO does not operate its technology, does not collect, nor possesses, nor has any access to any kind of data of its customers." -It is interesting that directly after this statement they claim with contradictory confidence that -their "technology was not associated in any way with the heinous murder of Jamal Khashoggi". -The unapologetic tone seems hardly appropriate when the same document confirms that the Group had to -shut down customers' systems due to "confirmed misuse" and have had to do so "multiple times" in the past. -Given all this, the response manages to evade any serious interrogation of the "vetting" process itself, -which forced the company to reject "approximately 15% of potential new opportunities for Pegasus" in one year. -Courageous.

    We have heard this all before. -There exists a multi-billion dollar industry of private companies and engineering firms thriving on proceeds from -selling surveillance tools and cyber-arms to dubious agencies and foreign governments. -In turn, the most power-hungry and oppressive regimes often rely on such technological innovations - -for which they lack the in-country engineering expertise - -to maintain control, suppress uprisings, intimidate opposing journalists, and track their citizens. -It's a lucrative business opportunity, and resourceful companies have sprung up everywhere to supply this demand, -often in countries where citizens, including employees of the company, would be horrified if they were similarly subject to the oppressions of their own products. -When, in 2014, Italy's HackingTeam were pulsed by the United Nations about their (then alleged) selling of spyware to Sudan, -which would have been a contravention of the UN's weapon export ban, -they simply replied that their product was not controlled as a weapon and therefore not subject to such scrutiny. -They remained within their legal bounds, technically. -Furthermore, they similarly shifted ethical responsibility to external standards of legitimacy, -claiming their "software is not sold to governments that are blacklisted by the EU, the US, NATO, and similar international organizations". -When the company themselves were hacked in 2015, -revelations (confirmations, that is) of widespread misuse by repressive governments were damaging enough to force them to disappear and rebrand as Memento Labs. -Their website boasts an impressive list of statutes, regulations, procedures, export controls and legal frameworks, -all of which the rebranded hackers proudly comply with. -Surely no further ethical scrutiny is necessary?

    Ethics != the law​

    The law is trailing behind​

    Such recourse to the legality of your action as ethical justification is moot for several reasons. -The first is glaringly obvious - -our laws are ill-equipped to address the implications of modern technology. -Legal systems are a cumbersome inheritance built over generations. -This is especially true of the statutes and regulations governing international trade, behind which these companies so often hide. -Our best legal systems are trailing miles behind the technology for which we seek guidelines. -Legislators are still struggling to make sense of technologies like face recognition, -the repercussions of smart devices acting "on their own" and biases in algorithms. -To claim you are performing ethical due diligence by resorting to an outdated and incomplete system of legal codes is disingenuous.

    The law depends on ethics​

    The second reason is more central to my argument, -and an important flaw in these sleight of hand justifications appearing from time to time in the media. -Ethics can in no way be confused as synonymous with legality or legitimacy. -These are incommensurable concepts. -In an ideal world, of course, the law is meant to track the minimum standards of ethical conduct in a society. -Laws are often drafted exactly from some ethical, and practical, impulse to minimize harmful conduct -and provide for corrective and punitive measures where transgressions do occur. -The law, however, has a much narrower scope than ethics. -It can be just or unjust. -In fact, it is in need of ethics to constantly reform. -Ethics and values are born out of collective self-reflection. -It develops in our conversation with ourselves and others about the type of society we strive for. -As such, an ethical worldview summarizes our deepest intuitions about how we should live and measure our impact on the world. -For this reason, ethics is primarily enforced by social and internal pressures, not legal boundaries - -our desire to do what ought to be done, however we define that. -Ethics is therefore a much grander scheme than global legal systems -and the diplomatic frameworks that grants legitimacy to governments. -These are but one limited outflow of the human aspiration to form societies in accordance with our ideologies and ethics.

    International law is vague and exploitable​

    Of course, the cyber-arms trade has a favorite recourse, international law, which is even more limited. -Since such products are seldomly sold to governments and agencies within the country of production, -it enables a further distancing from consequences. -Many private surveillance companies are based in fairly liberal societies with (seemingly) strict emphases on human rights in their domestic laws. -International laws are much more complicated - for opportunists a synonym for "more grey areas in which to hide". -Company conduct can now be governed, and excused, by a system that follows -the whims of autocrats with exploitative intent and vastly different ethical conceptions from the company's purported aims. -International law, and the ways it is most often enforced by way of, say, UN-backed sanctions, -have long been shaped by the compromises of international diplomacy. -To be blunt: these laws are weak and subject to exactly the sort of narrow interests behind which mercenaries have always hidden. -The surveillance tech industry is no exception.

    Conclusion​

    My point is simple: -selling cyber-arms with the potential to become vast tools of oppression to governments and bodies with blatant histories of human rights violations, -and all but the publicly announced intention to continue operating in this way, -is categorically unconscionable. -This seems obvious no matter what ethics system you argue from, -provided it harbors any consideration for human dignity and freedom. -It is a sign of poor moral discourse that such recourses to law and legitimacy are often considered synonymous with ethical justification. -"I have acted within the bounds of law", "We supply only to legitimate law enforcement agencies", etc. are no substitutes. -Ethical conduct requires an honest evaluation of an action against some conception of "the good", -however you define that. -Too often the surveillance tech industry precisely sidesteps this question, -both in internal processes and external rationalisations to a concerned public.

    John Locke, he of the life-liberty-and-property, articulated the idea that government exists solely through the consent of the governed. -Towards the end of the 17th century, he wrote in his Second Treatise on Civil Government, -"[w]henever legislators endeavor to take away, -and destroy the property of the people, or to reduce them to slavery under arbitrary power, -they put themselves in a state of war with the people, who are thereupon absolved from any further obedience". -The inference is straightforward and humanist in essence: -legitimacy is not something that is conferred by governments and institutions. -Rather, they derive their legitimacy from us, their citizens, holding them to standards of ethics and societal ideals. -This legitimacy only remains in tact as long as this mandate is honored and continuously extended by a well-informed public. -This is the principle of informed consent on which all reciprocal ethics is based.

    The surveillance tech industry may well have nothing more or less noble in mind than profit-making within legal bounds -when developing and selling their products. -However, when such companies are revealed again and again to have supplied tools of gross human rights violations to known human rights violators, -they will do well to remember that ethics always precedes requirements of legality and legitimacy. -It is a fallacy to take normative guidance from the concept of "legitimacy" -if the concept itself depends on such normative guidelines for definition. -Without examining the ethical standards by which institutions, governments, and laws, were created, -no value-judgements about their legitimacy can be made. -Hiding behind legal compliance as substitute for moral justification is not enough. -Targets of increasingly invasive governmental snooping are too often chosen precisely to suppress the mechanisms from which the legitimacy of such governments flow - -the consent of ordinary civilians. -Free and fair elections, free speech, free media, freedom of thought are all at risk.

    References​

    ]]>     - - Circe - -     - - <![CDATA[Waku v1 vs Waku v2: Bandwidth Comparison]]> - https://vac.dev/rlog/waku-v1-v2-bandwidth-comparison - - 2021-11-03T10:00:00.000Z - - A local comparison of bandwidth profiles showing significantly improved scalability in Waku v2 over Waku v1.

    Background​

    The original plan for Waku v2 suggested theoretical improvements in resource usage over Waku v1, -mainly as a result of the improved amplification factors provided by GossipSub. -In its turn, Waku v1 proposed improvements over its predecessor, Whisper.

    Given that Waku v2 is aimed at resource restricted environments, -we are specifically interested in its scalability and resource usage characteristics. -However, the theoretical performance improvements of Waku v2 over Waku v1, -has never been properly benchmarked and tested.

    Although we're working towards a full performance evaluation of Waku v2, -this would require significant planning and resources, -if it were to simulate "real world" conditions faithfully and measure bandwidth and resource usage across different network connections, -robustness against attacks/losses, message latencies, etc. -(There already exists a fairly comprehensive evaluation of GossipSub v1.1, -on which 11/WAKU2-RELAY is based.)

    As a starting point, -this post contains a limited and local comparison of the bandwidth profile (only) between Waku v1 and Waku v2. -It reuses and adapts existing network simulations for Waku v1 and Waku v2 -and compares bandwidth usage for similar message propagation scenarios.

    Theoretical improvements in Waku v2​

    Messages are propagated in Waku v1 using flood routing. -This means that every peer will forward every new incoming message to all its connected peers (except the one it received the message from). -This necessarily leads to unnecessary duplication (termed amplification factor), -wasting bandwidth and resources. -What's more, we expect this effect to worsen the larger the network becomes, -as each connection will receive a copy of each message, -rather than a single copy per peer.

    Message routing in Waku v2 follows the libp2p GossipSub protocol, -which lowers amplification factors by only sending full message contents to a subset of connected peers. -As a Waku v2 network grows, each peer will limit its number of full-message ("mesh") peerings - -libp2p suggests a maximum of 12 such connections per peer. -This allows much better scalability than a flood-routed network. -From time to time, a Waku v2 peer will send metadata about the messages it has seen to other peers ("gossip" peers).

    See this explainer for a more detailed discussion.

    Methodology​

    The results below contain only some scenarios that provide an interesting contrast between Waku v1 and Waku v2. -For example, star network topologies do not show a substantial difference between Waku v1 and Waku v2. -This is because each peer relies on a single connection to the central node for every message, -which barely requires any routing: -each connection receives a copy of every message for both Waku v1 and Waku v2. -Hybrid topologies similarly show only a difference between Waku v1 and Waku v2 for network segments with mesh-like connections, -where routing decisions need to be made.

    For this reason, the following approach applies to all iterations:

    1. Simulations are run locally. -This limits the size of possible scenarios due to local resource constraints, -but is a way to quickly get an approximate comparison.
    2. Nodes are treated as a blackbox for which we only measure bandwidth, -using an external bandwidth monitoring tool. -In other words, we do not consider differences in the size of the envelope (for v1) or the message (for v2).
    3. Messages are published at a rate of 50 new messages per second to each network, -except where explicitly stated otherwise.
    4. Each message propagated in the network carries 8 bytes of random payload, which is encrypted. -The same symmetric key cryptographic algorithm (with the same keys) are used in both Waku v1 and v2.
    5. Traffic in each network is generated from 10 nodes (randomly-selected) and published in a round-robin fashion to 10 topics (content topics for Waku v2). -In practice, we found no significant difference in average bandwidth usage when tweaking these two parameters (the number of traffic generating nodes and the number of topics).
    6. Peers are connected in a decentralized full mesh topology, -i.e. each peer is connected to every other peer in the network. -Waku v1 is expected to flood all messages across all existing connections. -Waku v2 gossipsub will GRAFT some of these connections for full-message peerings, -with the rest being gossip-only peerings.
    7. After running each iteration, we verify that messages propagated to all peers (comparing the number of published messages to the metrics logged by each peer).

    For Waku v1, nodes are configured as "full" nodes (i.e. with full bloom filter), -while Waku v2 nodes are relay nodes, all subscribing and publishing to the same PubSub topic.

    Network size comparison​

    Iteration 1: 10 nodes​

    Let's start with a small network of 10 nodes only and see how Waku v1 bandwidth usage compares to that of Waku v2. -At this small scale we don't expect to see improved bandwidth usage in Waku v2 over Waku v1, -since all connections, for both Waku v1 and Waku v2, will be full-message connections. -The number of connections is low enough that Waku v2 nodes will likely GRAFT all connections to full-message peerings, -essentially flooding every message on every connection in a similar fashion to Waku v1. -If our expectations are confirmed, it helps validate our methodology, -showing that it gives more or less equivalent results between Waku v1 and Waku v2 networks.

    Sure enough, the figure shows that in this small-scale setup, -Waku v1 actually has a lower per-peer bandwidth usage than Waku v2. -One reason for this may be the larger overall proportion of control messages in a gossipsub-routed network such as Waku v2. -These play a larger role when the total network traffic is comparatively low, as in this iteration. -Also note that the average bandwidth remains more or less constant as long as the rate of published messages remains stable.

    Iteration 2: 30 nodes​

    Now, let's run the same scenario for a larger network of highly-connected nodes, this time consisting of 30 nodes. -At this point, the Waku v2 nodes will start pruning some connections to limit the number of full-message peerings (to a maximum of 12), -while the Waku v1 nodes will continue flooding messages to all connected peers. -We therefore expect to see a somewhat improved bandwidth usage in Waku v2 over Waku v1.

    Bandwidth usage in Waku v2 has increased only slightly from the smaller network of 10 nodes (hovering between 2000 and 3000 kbps). -This is because there are only a few more full-message peerings than before. -Compare this to the much higher increase in bandwidth usage for Waku v1, which now requires more than 4000 kbps on average.

    Iteration 3: 50 nodes​

    For an even larger network of 50 highly connected nodes, -the divergence between Waku v1 and Waku v2 is even larger. -The following figure shows comparative average bandwidth usage for a throughput of 50 messages per second.

    Average bandwidth usage (for the same message rate) has remained roughly the same for Waku v2 as it was for 30 nodes, -indicating that the number of full-message peerings per node has not increased.

    Iteration 4: 85 nodes​

    We already see a clear trend in the bandwidth comparisons above, -so let's confirm by running the test once more for a network of 85 nodes. -Due to local resource constraints, the effective throughput for Waku v1 falls to below 50 messages per second, -so the v1 results below have been normalized and are therefore approximate. -The local Waku v2 simulation maintains the message throughput rate without any problems.

    Iteration 5: 150 nodes​

    Finally, we simulate message propagation in a network of 150 nodes. -Due to local resource constraints, we run this simulation at a lower rate - -35 messages per second - -and for a shorter amount of time.

    Notice how the Waku v1 bandwidth usage is now more than 10 times worse than that of Waku v2. -This is to be expected, as each Waku v1 node will try to flood each new message to 149 other peers, -while the Waku v2 nodes limit their full-message peerings to no more than 12.

    Discussion​

    Let's summarize average bandwidth growth against network growth for a constant message propagation rate. -Since we are particularly interested in how Waku v1 compares to Waku v2 in terms of bandwidth usage, -the results are normalised to the Waku v2 average bandwidth usage for each network size.

    Extrapolation is a dangerous game, -but it's safe to deduce that the divergence will only grow for even larger network topologies. -Although control signalling contributes more towards overall bandwidth for Waku v2 networks, -this effect becomes less noticeable for larger networks. -For network segments with more than ~18 densely connected nodes, -the advantage of using Waku v2 above Waku v1 becomes clear.

    Network traffic comparison​

    The analysis above controls the average message rate while network size grows. -In reality, however, active users (and therefore message rates) are likely to grow in conjunction with the network. -This will have an effect on bandwidth for both Waku v1 and Waku v2, though not in equal measure. -Consider the impact of an increasing rate of messages in a network of constant size:

    The rate of increase in bandwidth for Waku v2 is slower than that for Waku v1 for a corresponding increase in message propagation rate. -In fact, for a network of 30 densely-connected nodes, -if the message propagation rate increases by 1 per second, -Waku v1 requires an increased average bandwidth of almost 70kbps at each node. -A similar traffic increase in Waku v2 requires on average 40kbps more bandwidth per peer, just over half that of Waku v1.

    Conclusions​

    • Waku v2 scales significantly better than Waku v1 in terms of average bandwidth usage, -especially for densely connected networks.
    • E.g. for a network consisting of 150 or more densely connected nodes, -Waku v2 provides more than 10x better average bandwidth usage rates than Waku v1.
    • As the network continues to scale, both in absolute terms (number of nodes) and in network traffic (message rates) the disparity between Waku v2 and Waku v1 becomes even larger.

    Future work​

    Now that we've confirmed that Waku v2's bandwidth improvements over its predecessor matches theory, -we can proceed to a more in-depth characterisation of Waku v2's resource usage. -Some questions that we want to answer include:

    • What proportion of Waku v2's bandwidth usage is used to propagate payload versus bandwidth spent on control messaging to maintain the mesh?
    • To what extent is message latency (time until a message is delivered to its destination) affected by network size and message rate?
    • How reliable is message delivery in Waku v2 for different network sizes and message rates?
    • What are the resource usage profiles of other Waku v2 protocols (e.g.12/WAKU2-FILTER and 19/WAKU2-LIGHTPUSH)?

    Our aim is to get ever closer to a "real world" understanding of Waku v2's performance characteristics, -identify and fix vulnerabilities -and continually improve the efficiency of our suite of protocols.

    References​

    ]]>     - - Hanno Cornelius - -     - - <![CDATA[[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging]]> - https://vac.dev/rlog/waku-v2-ethereum-coscup - - 2021-08-06T12:00:00.000Z - - Learn more about Waku v2, its origins, goals, protocols, implementation and ongoing research. Understand how it is used and how it can be useful for messaging in Ethereum.

    This is the English version of a talk originally given in Chinese at COSCUP in Taipei.

    video recording with Chinese and English subtitles.

    Introduction​

    Hi everyone!

    Today I'll talk to you about Waku v2. What it is, what problems it is solving, -and how it can be useful for things such as messaging in Ethereum. First, let me -start with some brief background.

    Brief history and background​

    Back when Ethereum got started, there used to be this concept of the "holy -trinity". You had Ethereum for compute/consensus, Swarm for storage, and Whisper -for messaging. This is partly where the term Web3 comes from.

    Status started out as an app with the goal of being a window onto Ethereum and -a secure messenger. As one of the few, if not the only, apps using Whisper in -production, not to mention on a mobile phone, we quickly realized there were -problems with the underlying protocols and infrastructure. Protocols such as -Whisper weren't quite ready for prime time yet when it came to things such as -scalability and working in the real world.

    As we started addressing some of these challenges, and moved from app -developement to focusing on protocols, research and infrastructure, we created -Vac. Vac is an r&d unit doing protocol research focused on creating modular p2p -messaging protocols for private, secure, censorship resistant communication.

    I won't go into too much detail on the issues with Whisper, if you are -interested in this check out this talk -here or this -article.

    In a nutshell, we forked Whisper to address immediate shortcomings and this -became Waku v1. Waku v2 is complete re-thought implementation from scratch on top -of libp2p. This will be the subject of today's talk.

    Waku v2​

    Overview​

    Waku v2 is a privacy-preserving peer-to-peer messaging protocol for resource -restricted devices. We can look at Waku v2 as several things:

    • Set of protocols
    • Set of implementations
    • Network of nodes

    Let's first look at what the goals are.

    Goals​

    Waku v2 provides a PubSub based messaging protocol with the following -characteristics:

    1. Generalized messaging. Applications that require a messaging protocol to -communicate human to human, machine to machine, or a mix.
    2. Peer-to-peer. For applications that require a p2p solution.
    3. Resource restricted. For example, running with limited bandwidth, being -mostly-offline, or in a browser.
    4. Privacy. Applications that have privacy requirements, such as pseudonymity, -metadata protection, etc.

    And to provide these properties in a modular fashion, where applications can -choose their desired trade-offs.

    Protocols​

    Waku v2 consists of several protocols. Here we highlight a few of the most -important ones:

    • 10/WAKU2 - main specification, details how all the pieces fit together
    • 11/RELAY - thin layer on top of GossipSub for message dissemination
    • 13/STORE - fetching of historical messages
    • 14/MESSAGE - message payload

    This is the recommended subset for a minimal Waku v2 client.

    In addition to this there are many other types of specifications at various -stages of maturity, such as: content based filtering, bridge mode to Waku v1, -JSON RPC API, zkSNARKS based spam protection with RLN, accounting and -settlements with SWAP, fault-tolerant store nodes, recommendations around topic -usage, and more.

    See https://rfc.vac.dev/ for a full overview.

    Implementations​

    Waku v2 consists of multiple implementations. This allows for client diversity, -makes it easier to strengthen the protocols, and allow people to use Waku v2 in -different contexts.

    • nim-waku - the reference client written in Nim, most full-featured.
    • js-waku - allow usage of Waku v2 from browsers, focus on interacting with dapps.
    • go-waku - subset of Waku v2 to ease integration into the Status app.

    Testnet Huilong and dogfooding​

    In order to test the protocol we have setup a testnet across all implementations -called Huilong. Yes, that's the Taipei subway station!

    Among us core devs we have disabled the main #waku Discord channel used for -development, and people run their own node connected to this toy chat application.

    Feel free to join and say hi! Instructions can be found here:

    Research​

    While Waku v2 is being used today, we are actively researching improvements. -Since the design is modular, we can gracefully introduce new capabilities. Some -of these research areas are:

    • Privacy-preserving spam protection using zkSNARKs and RLN
    • Accounting and settlement of resource usage to incentivize nodes to provide services with SWAP
    • State synchronization for store protocol to make it easier to run a store node without perfect uptime
    • Better node discovery
    • More rigorous privacy analysis
    • Improving interaction with wallets and dapp

    Use cases​

    Let's look at where Waku v2 is and can be used.

    Prelude: Topics in Waku v2​

    To give some context, there are two different types of topics in Waku v2. One is -a PubSub topic, for routing. The other is a content topic, which is used for -content based filtering. Here's an example of the default PubSub topic:

    /waku/2/default-waku/proto

    This is recommended as it increases privacy for participants and it is stored by -default, however this is up to the application.

    The second type of topic is a content topic, which is application specific. For -example, here's the content topic used in our testnet:

    /toychat/2/huilong/proto

    For more on topics, see https://rfc.vac.dev/waku/informational/23/topics

    Status app​

    In the Status protocol, content topics - topics in Whisper/Waku v1 - are used for several things:

    • Contact code topic to discover X3DH bundles for perfect forward secrecy
      • Partitioned into N (currently 5000) content topics to balance privacy with efficiency
    • Public chats correspond to hash of the plaintext name
    • Negotiated topic for 1:1 chat with DHKE derived content topic

    See more here http://rfc.vac.dev/status/deprecated/waku-usage

    Currently, Status app is in the process of migrating to and testing Waku v2.

    DappConnect: Ethereum messaging​

    It is easy to think of Waku as being for human messaging, since that's how it is -primarily used in the Status app, but the goal is to be useful for generalized -messaging, which includes Machine-To-Machine (M2M) messaging.

    Recall the concept of the holy trinity with Ethereum/Swarm/Whisper and Web3 that -we mentioned in the beginning. Messaging can be used as a building block for -dapps, wallets, and users to communicate with each other. It can be used for -things such as:

    • Multisig and DAO vote transactions only needing one on-chain operation
    • Giving dapps ability to send push notifications to users
    • Giving users ability to directly respond to requests from dapps
    • Decentralized WalletConnect
    • Etc

    Basically anything that requires communication and doesn't have to be on-chain.

    WalletConnect v2​

    WalletConnect is an open protocol for connecting dapps to wallets with a QR -code. Version 2 is using Waku v2 as a communication channel to do so in a -decentralized and private fashion.

    See for more: https://docs.walletconnect.org/v/2.0/tech-spec

    WalletConnect v2 is currently in late alpha using Waku v2.

    More examples​

    • Gasless voting and vote aggregation off-chain
    • Dapp games using Waku as player discovery mechanism
    • Send encrypted message to someone with an Ethereum key
    • <Your dapp here>

    These are all things that are in progress / proof of concept stage.

    Contribute​

    We'd love to see contributions of any form!

    Conclusion​

    In this talk we've gone over the original vision for Web3 and how Waku came to -be. We've also looked at what Waku v2 aims to do. We looked at its protocols, -implementations, the current testnet as well as briefly on some ongoing -research for Vac.

    We've also looked at some specific use cases for Waku. First we looked at how -Status uses it with different topics. Then we looked at how it can be useful for -messaging in Ethereum, including for things like WalletConnect.

    I hope this talk gives you a better idea of what Waku is, why it exists, and -that it inspires you to contribute, either to Waku itself or by using it in your -own project!

    ]]>     - - Oskar - -     - - <![CDATA[Presenting JS-Waku: Waku v2 in the Browser]]> - https://vac.dev/rlog/presenting-js-waku - - 2021-06-04T12:00:00.000Z - - JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!

    For the past 3 months, we have been working on bringing Waku v2 to the browser. -Our aim is to empower dApps with Waku v2, and it led to the creation of a new library. -We believe now is good time to introduce it!

    Waku v2​

    First, let's review what Waku v2 is and what problem it is trying to solve.

    Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised -communications on resource restricted devices (i.e., mobile phones).

    The Status chat feature was initially built over Whisper. -However, Whisper has a number of caveats which makes it inefficient for mobile phones. -For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.

    To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced. -To further enable web and restricted resource environments, Waku v2 was created based on libp2p. -The migration of the Status chat feature to Waku v2 is currently in progress.

    We see the need of such solution in the broader Ethereum ecosystem, beyond Status. -This is why we are building Waku v2 as a decentralised communication platform for all to use and build on. -If you want to read more about Waku v2 and what it aims to achieve, -checkout What's the Plan for Waku v2?.

    Since last year, we have been busy defining and implementing Waku v2 protocols in nim-waku, -from which you can build wakunode2. -Wakunode2 is an adaptive and modular Waku v2 node, -it allows users to run their own node and use the Waku v2 protocols they need. -The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.

    Waku v2 in the browser​

    We believe that dApps and wallets can benefit from the Waku network in several ways. -For some dApps, it makes sense to enable peer-to-peer communications. -For others, machine-to-machine communications would be a great asset. -For example, in the case of a DAO, -Waku could be used for gas-less voting. -Enabling the DAO to notify their users of a new vote, -and users to vote without interacting with the blockchain and spending gas.

    Murmur was the first attempt to bring Whisper to the browser, -acting as a bridge between devp2p and libp2p. -Once Waku v2 was started and there was a native implementation on top of libp2p, -a chat POC was created to demonstrate the potential of Waku v2 -in web environment. -It showed how using js-libp2p with few modifications enabled access to the Waku v2 network. -There was still some unresolved challenges. -For example, nim-waku only support TCP connections which are not supported by browser applications. -Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets, -which in turn could connect to wakunode2 via TCP.

    However, to enable dApp and Wallet developers to easily integrate Waku in their product, -we need to give them a library that is easy to use and works out of the box: -introducing JS-Waku.

    JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network. -It is available right now on npm:

    npm install js-waku.

    As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.

    Key Waku v2 protocols are already available: -message, store, relay and light push, -enabling your dApp to:

    • Send and receive near-instant messages on the Waku network (relay),
    • Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
    • Send messages with confirmations (light push).

    JS-Waku needs to operate in the same context from which Waku v2 was born: -a restricted environment were connectivity or uptime are not guaranteed; -JS-Waku brings Waku v2 to the browser.

    Achievements so far​

    We focused the past month on developing a ReactJS Chat App. -The aim was to create enough building blocks in JS-Waku to enable this showcase web app that -we now use for dogfooding purposes.

    Most of the effort was on getting familiar with the js-libp2p library -that we heavily rely on. -JS-Waku is the second implementation of Waku v2 protocol, -so a lot of effort on interoperability was needed. -For example, to ensure compatibility with the nim-waku reference implementation, -we run our tests against wakunode2 as part of the CI.

    This interoperability effort helped solidify the current Waku v2 specifications: -By clarifying the usage of topics -(#327, #383), -fix discrepancies between specs and nim-waku -(#418, #419) -and fix small nim-waku & nim-libp2p bugs -(#411, #439).

    To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes. -A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser. -Unfortunately websocket support is not yet available in nim-libp2p so -we ended up deploying websockify alongside wakunode2 instances.

    As we built the web chat app, -we were able to fine tune the API to provide a simple and succinct interface. -You can start a node, connect to other nodes and send a message in less than ten lines of code:

    import { Waku } from 'js-waku'

    const waku = await Waku.create({})

    const nodes = await getStatusFleetNodes()
    await Promise.all(nodes.map((addr) => waku.dial(addr)))

    const msg = WakuMessage.fromUtf8String(
      'Here is a message!',
      '/my-cool-app/1/my-use-case/proto',
    )
    await waku.relay.send(msg)

    We have also put a bounty at 0xHack for using JS-Waku -and running a workshop. -We were thrilled to have a couple of hackers create new software using our libraries. -One of the projects aimed to create a decentralised, end-to-end encrypted messenger app, -similar to what the ETH-DM protocol aims to achieve. -Another project was a decentralised Twitter platform. -Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.

    As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network -while preserving this out of the box experience.

    What's next?​

    Next, we are directing our attention towards Developer Experience. -We already have documentation available but we want to provide more: -Tutorials, various examples -and showing how JS-Waku can be used with Web3.

    By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets. -We think JS-Waku builds a strong case for machine-to-machine (M2M) communications. -The first use cases we are looking into are dApp notifications: -Enabling dApp to notify their user directly in their wallets! -Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified -of events such as DAO voting.

    We already have some POC in the pipeline to enable voting and polling on the Waku network, -allowing users to save gas by not broadcasting each individual vote on the blockchain.

    To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples -of signing, validating, encrypting and decrypting messages using Web3. -Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.

    As you can read, we have grand plans for JS-Waku and Waku v2. -There is a lot to do, and we would love some help so feel free to -check out the new role in our team: -js-waku: Wallet & Dapp Integration Developer. -We also have a number of positions open to work on Waku protocol and nim-waku.

    If you are as excited as us by JS-Waku, why not build a dApp with it? -You can find documentation on the npmjs page.

    Whether you are a developer, you can come chat with us using WakuJS Web Chat -or chat2. -You can get support in #dappconnect-support on Vac Discord or Telegram. -If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.

    ]]>     - - Franck - -     - - <![CDATA[Privacy-preserving p2p economic spam protection in Waku v2]]> - https://vac.dev/rlog/rln-relay - - 2021-03-05T12:00:00.000Z - - This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.

    Introduction​

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol2 through Rate-Limiting Nullifiers3 4 or RLN for short.

    Let me give a little background about Waku(v2)1. Waku is a privacy-preserving peer-to-peer (p2p) messaging protocol for resource-restricted devices. Being p2p means that Waku relies on No central server. Instead, peers collaboratively deliver messages in the network. Waku uses GossipSub16 as the underlying routing protocol (as of the writeup of this post). At a high level, GossipSub is based on publisher-subscriber architecture. That is, peers, congregate around topics they are interested in and can send messages to topics. Each message gets delivered to all peers subscribed to the topic. In GossipSub, a peer has a constant number of direct connections/neighbors. In order to publish a message, the author forwards its message to a subset of neighbors. The neighbors proceed similarly till the message gets propagated in the network of the subscribed peers. The message publishing and routing procedures are part of the Waku Relay17 protocol. -

    Figure 1: An overview of privacy-preserving p2p economic spam protection in Waku v2 RLN-Relay protocol.

    What do we mean by spamming?​

    In centralized messaging systems, a spammer usually indicates an entity that uses the messaging system to send an unsolicited message (spam) to large numbers of recipients. However, in Waku with a p2p architecture, spam messages not only affect the recipients but also all the other peers involved in the routing process as they have to spend their computational power/bandwidth/storage capacity on processing spam messages. As such, we define a spammer as an entity that uses the messaging system to publish a large number of messages in a short amount of time. The messages issued in this way are called spam. In this definition, we disregard the intention of the spammer as well as the content of the message and the number of recipients.

    Possible Solutions​

    Has the spamming issue been addressed before? Of course yes! Here is an overview of the spam protection techniques with their trade-offs and use-cases. In this overview, we distinguish between protection techniques that are targeted for centralized messaging systems and those for p2p architectures.

    Centralized Messaging Systems​

    In traditional centralized messaging systems, spam usually signifies unsolicited messages sent in bulk or messages with malicious content like malware. Protection mechanisms include

    • authentication through some piece of personally identifiable information e.g., phone number
    • checksum-based filtering to protect against messages sent in bulk
    • challenge-response systems
    • content filtering on the server or via a proxy application

    These methods exploit the fact that the messaging system is centralized and a global view of the users' activities is available based on which spamming patterns can be extracted and defeated accordingly. Moreover, users are associated with an identifier e.g., a username which enables the server to profile each user e.g., to detect suspicious behavior like spamming. Such profiling possibility is against the user's anonymity and privacy.

    Among the techniques enumerated above, authentication through phone numbers is a some-what economic-incentive measure as providing multiple valid phone numbers will be expensive for the attacker. Notice that while using an expensive authentication method can reduce the number of accounts owned by a single spammer, cannot address the spam issue entirely. This is because the spammer can still send bulk messages through one single account. For this approach to be effective, a centralized mediator is essential. That is why such a solution would not fit the p2p environments where no centralized control exists.

    P2P Systems​

    What about spam prevention in p2p messaging platforms? There are two techniques, namely Proof of Work8 deployed by Whisper9 and Peer scoring6 method (namely reputation-based approach) adopted by LibP2P. However, each of these solutions has its own shortcomings for real-life use-cases as explained below.

    Proof of work​

    The idea behind the Proof Of Work i.e., POW8 is to make messaging a computationally costly operation hence lowering the messaging rate of all the peers including the spammers. In specific, the message publisher has to solve a puzzle and the puzzle is to find a nonce such that the hash of the message concatenated with the nonce has at least z leading zeros. z is known as the difficulty of the puzzle. Since the hash function is one-way, peers have to brute-force to find a nonce. Hashing is a computationally-heavy operation so is the brute-force. While solving the puzzle is computationally expensive, it is comparatively cheap to verify the solution.

    POW is also used as the underlying mining algorithm in Ethereum and Bitcoin blockchain. There, the goal is to contain the mining speed and allow the decentralized network to come to a consensus, or agree on things like account balances and the order of transactions.

    While the use of POW makes perfect sense in Ethereum / Bitcoin blockchain, it shows practical issues in heterogeneous p2p messaging systems with resource-restricted peers. Some peers won't be able to carry the designated computation and will be effectively excluded. Such exclusion showed to be practically an issue in applications like Status, which used to rely on POW for spam-protection, to the extent that the difficulty level had to be set close to zero.

    Peer Scoring​

    The peer scoring method6 that is utilized by libp2p is to limit the number of messages issued by a peer in connection to another peer. That is each peer monitors all the peers to which it is directly connected and adjusts their messaging quota i.e., to route or not route their messages depending on their past activities. For example, if a peer detects its neighbor is sending more than x messages per month, can drop its quota to z.x where z is less than one. The shortcoming of this solution is that scoring is based on peers' local observations and the concept of the score is defined in relation to one single peer. This leaves room for an attack where a spammer can make connections to k peers in the system and publishes k.(x-1) messages by exploiting all of its k connections. Another attack scenario is through botnets consisting of a large number of e.g., a million bots. The attacker rents a botnet and inserts each of them as a legitimate peer to the network and each can publish x-1 messages per month7.

    Economic-Incentive Spam protection​

    Is this the end of our spam-protection journey? Shall we simply give up and leave spammers be? Certainly not! -Waku RLN-Relay gives us a p2p spam-protection method which:

    • suits p2p systems and does not rely on any central entity.
    • is efficient i.e., with no unreasonable computational, storage, memory, and bandwidth requirement! as such, it fits the network of heterogeneous peers.
    • respects users privacy unlike reputation-based and centralized methods.
    • deploys economic-incentives to contain spammers' activity. Namely, there is a financial sacrifice for those who want to spam the system. How? follow along ...

    We devise a general rule to save everyone's life and that is

    No one can publish more than M messages per epoch without being financially charged!

    We set M to 1 for now, but this can be any arbitrary value. You may be thinking "This is too restrictive! Only one per epoch?". Don't worry, we set the epoch to a reasonable value so that it does not slow down the communication of innocent users but will make the life of spammers harder! Epoch here can be every second, as defined by UTC date-time +-20s.

    The remainder of this post is all about the story of how to enforce this limit on each user's messaging rate as well as how to impose the financial cost when the limit gets violated. This brings us to the Rate Limiting Nullifiers and how we integrate this technique into Waku v2 (in specific the Waku Relay protocol) to protect our valuable users against spammers.

    Technical Terms​

    Zero-knowledge proof: Zero-knowledge proof (ZKP)14 allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do the trust-minimized computation that is also privacy-preserving. As a basic example, instead of showing your ID when going to a bar you simply give them proof that you are over 18, without showing the doorman your id. In this write-up, by ZKP we essentially mean zkSNARK15 which is one of the many types of ZKPs.

    Threshold Secret Sharing Scheme: (m,n) Threshold secret-sharing is a method by which you can split a secret value s into n pieces in a way that the secret s can be reconstructed by having m pieces (m <= n). The economic-incentive spam protection utilizes a (2,n) secret sharing realized by Shamir Secret Sharing Scheme13.

    Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers​

    Context: We started the idea of economic-incentive spam protection more than a year ago and conducted a feasibility study to identify blockers and unknowns. The results are published in our prior post. Since then major progress has been made and the prior identified blockers that are listed below are now addressed. Kudos to Barry WhiteHat, Onur Kilic, Koh Wei Jie for all of their hard work, research, and development which made this progress possible.

    • the proof time22 which was initially in the order of minutes ~10 mins and now is almost 0.5 seconds
    • the prover key size21 which was initially ~110MB and now is ~3.9MB
    • the lack of Shamir logic19 which is now implemented and part of the RLN repository4
    • the concern regarding the potential multi-party computation for the trusted setup of zkSNARKs which got resolved20
    • the lack of end-to-end integration that now we made it possible, have it implemented, and are going to present it in this post. New blockers are also sorted out during the e2e integration which we will discuss in the Feasibility and Open Issues section.

    Now that you have more context, let's see how the final solution works. The fundamental point is to make it economically costly to send more than your share of messages and to do so in a privacy-preserving and e2e fashion. To do that we have the following components:

    • 1- Group: We manage all the peers inside a large group (later we can split peers into smaller groups, but for now consider only one). The group management is done via a smart contract which is devised for this purpose and is deployed on the Ethereum blockchain.
    • 2- Membership: To be able to send messages and in specific for the published messages to get routed by all the peers, publishing peers have to register to the group. Membership involves setting up public and private key pairs (think of it as the username and password). The private key remains at the user side but the public key becomes a part of the group information on the contract (publicly available) and everyone has access to it. Public keys are not human-generated (like usernames) and instead they are random numbers, as such, they do not reveal any information about the owner (think of public keys as pseudonyms). Registration is mandatory for the users who want to publish a message, however, users who only want to listen to the messages are more than welcome and do not have to register in the group.
    • Membership fee: Membership is not for free! each peer has to lock a certain amount of funds during the registration (this means peers have to have an Ethereum account with sufficient balance for this sake). This fund is safely stored on the contract and remains intact unless the peer attempts to break the rules and publish more than one message per epoch.
    • Zero-knowledge Proof of membership: Do you want your message to get routed to its destination, fine, but you have to prove that you are a member of the group (sorry, no one can escape the registration phase!). Now, you may be thinking that should I attach my public key to my message to prove my membership? Absolutely Not! we said that our solution respects privacy! membership proofs are done in a zero-knowledge manner that is each message will carry cryptographic proof asserting that "the message is generated by one of the current members of the group", so your identity remains private and your anonymity is preserved!
    • Slashing through secret sharing: Till now it does not seem like we can catch spammers, right? yes, you are right! now comes the exciting part, detecting spammers and slashing them. The core idea behind the slashing is that each publishing peer (not routing peers!) has to integrate a secret share of its private key inside the message. The secret share is deterministically computed over the private key and the current epoch. The content of this share is harmless for the peer's privacy (it looks random) unless the peer attempts to publish more than one message in the same epoch hence disclosing more than one secret share of its private key. Indeed two distinct shares of the private key under the same epoch are enough to reconstruct the entire private key. Then what should you do with the recovered private key? hurry up! go to the contract and withdraw the private key and claim its fund and get rich!! Are you thinking what if spammers attach junk values instead of valid secret shares? Of course, that wouldn't be cool! so, there is a zero-knowledge proof for this sake as well where the publishing peer has to prove that the secret shares are generated correctly.

    A high-level overview of the economic spam protection is shown in Figure 1.

    Flow​

    In this section, we describe the flow of the economic-incentive spam detection mechanism from the viewpoint of a single peer. An overview of this flow is provided in Figure 3.

    Setup and Registration​

    A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members' public keys. An overview of registration is illustrated in Figure 2.

    For the registration, a peer creates a transaction that sends x amount of Ether to the contract. The peer who has the "private key" sk associated with that deposit would be able to withdraw x Ether by providing valid proof. Note that sk is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch. -The following relation holds between the sk and pk i.e., pk = H(sk) where H denotes a hash function. -

    Figure 2: Registration

    Maintaining the membership Merkle Tree​

    The ZKP of membership that we mentioned before relies on the representation of the entire group as a Merkle Tree. The tree construction and maintenance is delegated to the peers (the initial idea was to keep the tree on the chain as part of the contract, however, the cost associated with member deletion and insertion was high and unreasonable, please see Feasibility and Open Issues for more details). As such, each peer needs to build the tree locally and sync itself with the contract updates (peer insertion and deletion) to mirror them on its tree. -Two pieces of information of the tree are important as they enable peers to generate zero-knowledge proofs. One is the root of the tree and the other is the membership proof (or the authentication path). The tree root is public information whereas the membership proof is private data (or more precisely the index of the peer in the tree).

    Publishing​

    In order to publish at a given epoch, each message must carry a proof i.e., a zero-knowledge proof signifying that the publishing peer is a registered member, and has not exceeded the messaging rate at the given epoch.

    Recall that the enforcement of the messaging rate was through associating a secret shared version of the peer's sk into the message together with a ZKP that the secret shares are constructed correctly. As for the secret sharing part, the peer generates the following data:

    1. shareX
    2. shareY
    3. nullifier

    The pair (shareX, shareY) is the secret shared version of sk that are generated using Shamir secret sharing scheme. Having two such pairs for an identical nullifier results in full disclosure of peer's sk and hence burning the associated deposit. Note that the nullifier is a deterministic value derived from sk and epoch therefore any two messages issued by the same peer (i.e., using the same sk) for the same epoch are guaranteed to have identical nullifiers.

    Finally, the peer generates a zero-knowledge proof zkProof asserting the membership of the peer in the group and the correctness of the attached secret share (shareX, shareY) and the nullifier. In order to generate a valid proof, the peer needs to have two private inputs i.e., its sk and its authentication path. Other inputs are the tree root, epoch, and the content of the message.

    Privacy Hint: Note that the authentication path of each peer depends on the recent list of members (hence changes when new peers register or leave). As such, it is recommended (and necessary for privacy/anonymity) that the publisher updates her authentication path based on the latest status of the group and attempts the proof using the updated version.

    An overview of the publishing procedure is provided in Figure 3.

    Routing​

    Upon the receipt of a message, the routing peer needs to decide whether to route it or not. This decision relies on the following factors:

    1. If the epoch value attached to the message has a non-reasonable gap with the routing peer's current epoch then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs).
    2. The message MUST contain valid proof that gets verified by the routing peer. -If the preceding checks are passed successfully, then the message is relayed. In case of an invalid proof, the message is dropped. If spamming is detected, the publishing peer gets slashed (see Spam Detection and Slashing).

    An overview of the routing procedure is provided in Figure 3.

    Spam Detection and Slashing​

    In order to enable local spam detection and slashing, routing peers MUST record the nullifier, shareX, and shareY of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.

    1. The routing peer first verifies the zkProof and drops the message if not verified.
    2. Otherwise, it checks whether a message with an identical nullifier has already been relayed.
      • a) If such message exists and its shareX and shareY components are different from the incoming message, then slashing takes place (if the shareX and shareY fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).
      • b) If none found, then the message gets relayed.

    An overview of the slashing procedure is provided in Figure 3. -

    Figure 3: Publishing, Routing and Slashing workflow.

    Feasibility and Open Issues​

    We've come a long way since a year ago, blockers resolved, now we have implemented it end-to-end. We learned lot and could identify further issues and unknowns some of which are blocking getting to production. The summary of the identified issues are presented below.

    Storage overhead per peer​

    Currently, peers are supposed to maintain the entire tree locally and it imposes storage overhead which is linear in the size of the group (see this issue11 for more details). One way to cope with this is to use the light-node and full-node paradigm in which only a subset of peers who are more resourceful retain the tree whereas the light nodes obtain the necessary information by interacting with the full nodes. Another way to approach this problem is through a more storage efficient method (as described in this research issue12) where peers store a partial view of the tree instead of the entire tree. Keeping the partial view lowers the storage complexity to O(log(N)) where N is the size of the group. There are still unknown unknowns to this solution, as such, it must be studied further to become fully functional.

    Cost-effective way of member insertion and deletion​

    Currently, the cost associated with RLN-Relay membership is around 30 USD10. We aim at finding a more cost-effective approach. Please feel free to share with us your solution ideas in this regard in this issue.

    Exceeding the messaging rate via multiple registrations​

    While the economic-incentive solution has an economic incentive to discourage spamming, we should note that there is still expensive attack(s)23 that a spammer can launch to break the messaging rate limit. That is, the attacker can pay for multiple legit registrations e.g., k, hence being able to publish k messages per epoch. We believe that the higher the membership fee is, the less probable would be such an attack, hence a stronger level of spam-protection can be achieved. Following this argument, the high fee associated with the membership (which we listed above as an open problem) can indeed be contributing to a better protection level.

    Conclusion and Future Steps​

    As discussed in this post, Waku RLN Relay can achieve a privacy-preserving economic spam protection through rate-limiting nullifiers. The idea is to financially discourage peers from publishing more than one message per epoch. In specific, exceeding the messaging rate results in a financial charge. Those who violate this rule are called spammers and their messages are spam. The identification of spammers does not rely on any central entity. Also, the financial punishment of spammers is cryptographically guaranteed. -In this solution, privacy is guaranteed since: 1) Peers do not have to disclose any piece of personally identifiable information in any phase i.e., neither in the registration nor in the messaging phase 2) Peers can prove that they have not exceeded the messaging rate in a zero-knowledge manner and without leaving any trace to their membership accounts. -Furthermore, all the computations are light hence this solution fits the heterogenous p2p messaging system. Note that the zero-knowledge proof parts are handled through zkSNARKs and the benchmarking result can be found in the RLN benchmark report5.

    Future steps:

    We are still at the PoC level, and the development is in progress. As our future steps,

    • we would like to evaluate the running time associated with the Merkle tree operations. Indeed, the need to locally store Merkle tree on each peer was one of the unknowns discovered during this PoC and yet the concrete benchmarking result in this regard is not available.
    • We would also like to pursue our storage-efficient Merkle Tree maintenance solution in order to lower the storage overhead of peers.
    • In line with the storage optimization, the full-node light-node structure is another path to follow.
    • Another possible improvement is to replace the membership contract with a distributed group management scheme e.g., through distributed hash tables. This is to address possible performance issues that the interaction with the Ethereum blockchain may cause. For example, the registration transactions are subject to delay as they have to be mined before being visible in the state of the membership contract. This means peers have to wait for some time before being able to publish any message.

    Acknowledgement​

    Thanks to Onur Kılıç for his explanation and pointers and for assisting with development and runtime issues. Also thanks to Barry Whitehat for his time and insightful comments. Special thanks to Oskar Thoren for his constructive comments and his guides during the development of this PoC and the writeup of this post.

    References​

    1. RLN-Relay specification: https://rfc.vac.dev/waku/standards/core/17/rln-relay↩
    2. RLN documentation: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both↩
    3. RLN repositories: https://github.com/kilic/RLN and https://github.com/kilic/rlnapp↩
    4. Waku v2: https://rfc.vac.dev/waku/standards/core/10/waku2↩
    5. GossipSub: https://docs.libp2p.io/concepts/publish-subscribe/↩
    6. Waku Relay: https://rfc.vac.dev/waku/standards/core/11/relay↩
    7. Proof of work: http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf and https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf↩
    8. EIP-627 Whisper: https://eips.ethereum.org/EIPS/eip-627↩
    9. Peer Scoring: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring↩
    10. Peer scoring security issues: https://github.com/vacp2p/research/issues/44↩
    11. Zero Knowledge Proof: https://dl.acm.org/doi/abs/10.1145/3335741.3335750 and https://en.wikipedia.org/wiki/Zero-knowledge_proof↩
    12. zkSNARKs: https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11 and https://coinpare.io/whitepaper/zcash.pdf↩
    13. Shamir Secret Sharing Scheme: https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing↩
    14. zkSNARKs proof time: https://github.com/vacp2p/research/issues/7↩
    15. Prover key size: https://github.com/vacp2p/research/issues/8↩
    16. The lack of Shamir secret sharing in zkSNARKs: https://github.com/vacp2p/research/issues/10↩
    17. The MPC required for zkSNARKs trusted setup: https://github.com/vacp2p/research/issues/9↩
    18. Storage overhead per peer: https://github.com/vacp2p/research/issues/57↩
    19. Storage-efficient Merkle Tree maintenance: https://github.com/vacp2p/research/pull/54↩
    20. Cost-effective way of member insertion and deletion: https://github.com/vacp2p/research/issues/56↩
    21. Attack on the messaging rate: https://github.com/vacp2p/specs/issues/251↩
    22. RLN Benchmark: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks↩
    ]]>     - - Sanaz - -     - - <![CDATA[[Talk] Vac, Waku v2 and Ethereum Messaging]]> - https://vac.dev/rlog/waku-v2-ethereum-messaging - - 2020-11-10T12:00:00.000Z - - Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging.

    The following post is a transcript of the talk given at the Taipei Ethereum meetup, November 5. There is also a video recording.

    0. Introduction​

    Hi! My name is Oskar and I'm the protocol research lead at Vac. This talk will be divided into two parts. First I'll talk about the journey from Whisper, to Waku v1 and now to Waku v2. Then I'll talk about messaging in Ethereum. After this talk, you should have an idea of what Waku v2 is, the problems it is trying to solve, as well as where it can be useful for messaging in Ethereum.

    PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2​

    1. Vac intro​

    First, what is Vac? Vac grew out of our efforts Status to create a window on to Ethereum and secure messenger. Vac is modular protocol stack for p2p secure messaging, paying special attention to resource restricted devices, privacy and censorship resistance.

    Today we are going to talk mainly about Waku v2, which is the transport privacy / routing aspect of the Vac protocol stack. It sits "above" the p2p overlay, such as libp2p dealing with transports etc, and below a conversational security layer dealing with messaging encryption, such as using Double Ratchet etc.

    2. Whisper to Waku v1​

    In the beginning, there was Whisper. Whisper was part of the holy trinity of Ethereum. You had Ethereum for consensus/computation, Whisper for messaging, and Swarm for storage.

    However, for various reasons, Whisper didn't get the attention it deserved. Development dwindled, it promised too much and it suffered from many issues, such as being extremely inefficient and not being suitable for running on e.g. mobile phone. Despite this, Status used it in its app from around 2017 to 2019. As far as I know, it was one of very few, if not the only, production uses of Whisper.

    In an effort to solve some of its immediate problems, we forked Whisper into Waku and formalized it with a proper specification. This solved immediate bandwidth issues for light nodes, introduced rate limiting for better spam protection, improved historical message support, etc.

    If you are interested in this journey, checkout the EthCC talk Dean and I gave in Paris earlier this year.

    Status upgraded to Waku v1 early 2020. What next?

    3. Waku v1 to v2​

    We were far from done. The changes we had made were quite incremental and done in order to get tangible improvements as quickly as possible. This meant we couldn't address more fundamental issues related to full node routing scalability, running with libp2p for more transports, better security, better spam protection and incentivization.

    This kickstarted Waku v2 efforts, which is what we've been working on since July. This work was and is initally centered around a few pieces:

    (a) Moving to libp2p

    (b) Better routing

    (c) Accounting and user-run nodes

    The general theme was: making the Waku network more scalable and robust.

    We also did a scalability study to show at what point the network would run into issues, due to the inherent lack of routing that Whisper and Waku v1 provided.

    You can read more about this here.

    3.5 Waku v2 - Design goals​

    Taking a step back, what problem does Waku v2 attempt to solve compared to all the other solutions that exists out there? What type of applications should use it and why? We have the following design goals:

    1. Generalized messaging. Many applications requires some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix.

    2. Peer-to-peer. These applications sometimes have requirements that make them suitable for peer-to-peer solutions.

    3. Resource restricted. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:

      • limited bandwidth, CPU, memory, disk, battery, etc
      • not being publicly connectable
      • only being intermittently connected; mostly-offline

    4. Privacy. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.

    As well as to do so in a modular fashion. Meaning you can find a reasonable trade-off depending on your exact requirements. For example, you usually have to trade off some bandwidth to get metadata protection, and vice versa.

    The concept of designing for resource restricted devices also leads to the concept of adaptive nodes, where you have more of a continuum between full nodes and light nodes. For example, if you switch your phone from mobile data to WiFi you might be able to handle more bandwidth, and so on.

    4. Waku v2 - Breakdown​

    Where is Waku v2 at now, and how is it structured?

    It is running over libp2p and we had our second internal testnet last week or so. As a side note, we name our testnets after subway stations in Taipei, the first one being Nangang, and the most recent one being Dingpu.

    The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:

    • Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.
    • Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.
    • Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.
    • Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.
    • Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.
    • Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.

    Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.

    You can read more about the breakdown in this update though some progress has been made since then, as well was in the main Waku v2 spec.

    5. Waku v2 - Upcoming​

    What's coming up next? There are a few things.

    For Status to use it in production, it needs to be integrated into the main app using the Nim Node API. The bridge also needs to be implemented and tested.

    For other users, we are currently overhauling the API to allow usage from a browser, e.g. To make this experience great, there are also a few underlying infrastructure things that we need in nim-libp2p, such as a more secure HTTP server in Nim, Websockets and WebRTC support.

    There are also some changes we made to at what level content encryption happens, and this needs to be made easier to use in the API. This means you can use a node without giving your keys to it, which is useful in some environments.

    More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:

    • Better scaling, by using topic sharding.
    • Accounting and user-run nodes, to account for and incentives full nodes.
    • Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.
    • Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.

    As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next.

    PART 2 - ETHEREUM MESSAGING​

    A lot of what follows is inspired by exploratory work that John Lea has done at Status, previously Head of UX Architecture at Ubuntu.

    6. Ethereum Messaging - Why?​

    It is easy to think that Waku v2 is only for human to human messaging, since that's how Waku is currently primarily used in the Status app. However, the goal is to be useful for generalized messaging, which includes other type of information as well as machine to machine messaging.

    What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:

    • Reducing on-chain transactions
    • Reduce latency for operations
    • Decentralize centrally coordinated services (like WalletConnect)
    • Improve UX of dapps
    • Broadcast live information
    • A message transport layer for state channels

    And so on.

    7. Ethereum Messaging - Why? (Cont)​

    What are some examples of practical things Waku as used for Ethereum Messaging could solve?

    • Multisig transfers only needing one on chain transaction
    • DAO votes only needing one one chain transaction
    • Giving dapps ability to direct push notifications to users
    • Giving users ability to directly respond to requests from daps
    • Decentralized Wallet Connect

    Etc.

    8. What's needed to deliver this?​

    We can break it down into our actors:

    • Decentralized M2M messaging system (Waku)
    • Native wallets (Argent, Metamask, Status, etc)
    • Dapps that benefit from M2M messaging
    • Users whose problems are being solved

    Each of these has a bunch of requirements in turn. The messaging system needs to be decentralized, scalable, robust, etc. Wallets need support for messaging layer, dapps need to integrate this, etc.

    This is a lot! Growing adoption is a challenge. There is a catch 22 in terms of justifying development efforts for wallets, when no dapps need it, and likewise for dapps when no wallets support Waku. In addition to this, there must be proven usage of Waku before it can be relied on, etc. How can we break this up into smaller pieces of work?

    9. Breaking up the problem and a high level roadmap​

    We can start small. It doesn't and need to be used for critical features first. A more hybrid approach can be taken where it acts more as nice-to-haves.

    1. Forking Whisper and solving scalablity, spam etc issues with it. -This is a work in progress. What we talked about in part 1.
    2. Expose messaging API for Dapp developers.
    3. Implement decentralized version of WalletConnect. -Currently wallets connect ot dapps with centralized service. Great UX.
    4. Solve DAO/Multi-Sig coordination problem. -E.g. send message to wallet-derived key when it is time to sign a transaction.
    5. Extend dapp-to-user and user-to-dapp communication to more dapps. -Use lessons learned and examples to drive adoptation for wallets/dapps.

    And then build up from there.

    10. We are hiring!​

    A lot of this will happen in Javascript and browsers, since that's the primarily environment for a lot of wallets and dapps. We are currently hiring for a Waku JS Wallet integration lead to help push this effort further.

    Come talk to me after or apply here.

    That's it! You can find us on Status, Telegram, vac.dev. I'm on twitter here.

    Questions?

    ]]>     - - Oskar - -     - - <![CDATA[Waku v2 Update]]> - https://vac.dev/rlog/waku-v2-update - - 2020-09-28T12:00:00.000Z - - A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?

    It has been a while since the last post. It is time for an update on Waku v2. Aside from getting more familiar with libp2p (specifically nim-libp2p) and some vacation, what have we been up to? In this post we'll talk about what we've gotten done since last time, and briefly talk about immediate next steps and future. But first, a recap.

    Recap​

    In the last post (Waku v2 plan) we explained the rationale of Waku v2 - the current Waku network is fragile and doesn't scale. To solve this, Waku v2 aims to reduce amplification factors and get more user run nodes. We broke the work down into three separate tracks.

    1. Track 1 - Move to libp2p
    2. Track 2 - Better routing
    3. Track 3 - Accounting and user-run nodes

    As well as various rough components for each track. The primary initial focus is track 1. This means things like: moving to FloodSub, simplify the protocol, core integration, topic interest behavior, historical message caching, and Waku v1<>v2 bridge.

    Current state​

    Let's talk about the state of specs and our main implementation nim-waku. Then we'll go over our recent testnet, Nangang, and finish off with a Web PoC.

    Specs​

    After some back and forth on how to best structure things, we ended up breaking down the specs into a few pieces. While Waku v2 is best thought of as a cohesive whole in terms of its capabilities, it is made up of several protocols. Here's a list of the current specs and their status:

    Raw means there is not yet an implementation that corresponds fully to the spec, and draft means there is an implementation that corresponds to the spec. In the interest of space, we won't go into too much detail on the specs here except to note a few things:

    • The relay spec is essentially a thin wrapper on top of PubSub/FloodSub/GossipSub
    • The filter protocol corresponds to previous light client mode in Waku v1
    • The store protocol corresponds to the previous mailserver construct in Waku v1

    The filter and store protocol allow for adaptive nodes, i.e. nodes that have various capabilities. For example, a node being mostly offline, or having limited bandwidth capacity. The bridge spec outlines how to bridge the Waku v1 and v2 networks.

    Implementation​

    The main implementation we are working on is nim-waku. This builds on top of libraries such as nim-libp2p and others that the Nimbus team have been working on as part of their Ethereum 2.0 client.

    Currently nim-waku implements the relay protocol, and is close to implementing filter and store protocol. It also exposes a Nim Node API that allows libraries such as nim-status to use it. Additionally, there is also a rudimentary JSON RPC API for command line scripting.

    Nangang testnet​

    Last week we launched a very rudimentary internal testnet called Nangang. The goal was to test basic connectivity and make sure things work end to end. It didn't have things like: client integration, encryption, bridging, multiple clients, store/filter protocol, or even a real interface. What it did do is allow Waku developers to "chat" via RPC calls and looking in the log output. Doing this meant we exposed and fixed a few blockers, such as connection issues, deployment, topic subscription management, protocol and node integration, and basic scripting/API usage. After this, we felt confident enough to upgrade the main and relay spec to "draft" status.

    Waku Web PoC​

    As a bonus, we wanted to see what it'd take to get Waku running in a browser. This is a very powerful capability that enables a lot of use cases, and something that libp2p enables with its multiple transport support.

    Using the current stack, with nim-waku, would require quite a lot of ground work with WASM, WebRTC, Websockets support etc. Instead, we decided to take a shortcut and hack together a JS implementation called Waku Web Chat. This quick hack wouldn't be possible without the people behind js-libp2p-examples and js-libp2p and all its libraries. These are people like Jacob Heun, Vasco Santos, and Cayman Nava. Thanks!

    It consists of a brower implementation, a NodeJS implementation and a bootstrap server that acts as a signaling server for WebRTC. It is largely a bastardized version of GossipSub, and while it isn't completely to spec, it does allow messages originating from a browser to eventually end up at a nim-waku node, and vice versa. Which is pretty cool.

    Coming up​

    Now that we know what the current state is, what is still missing? what are the next steps?

    Things that are missing​

    While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope:

    1. Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level.

    2. Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status (status-nim has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings -us too...

    3. Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 (1, 2). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc).

    4. Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented.

    5. General tightening up of specs and implementation.

    While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope.

    Upcoming​

    Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details.

    1. Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors.

    2. Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner.

    3. Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet -format can improve things like unlinkability.

    4. zkSnarks RLN for spam protection and incentivization. We studied this last year and recent developments have made this relevant to study again. Create an experimental spec/PoC as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this!

    5. Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we'd like to deliver on that.

    As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates.

    If you are feeling adventurous and want to use early stage alpha software, check out the docs. If you want to read the specs, head over to Waku spec. If you want to talk with us, join us on Status or on Telegram (they are bridged).

    ]]>     - - Oskar - -     - - <![CDATA[What's the Plan for Waku v2?]]> - https://vac.dev/rlog/waku-v2-plan - - 2020-07-01T12:00:00.000Z - - Read about our plans for Waku v2, moving to libp2p, better routing, adaptive nodes and accounting!

    tldr: The Waku network is fragile and doesn't scale. Here's how to solve it.

    NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.

    Problem​

    The Waku network is fragile and doesn't scale.

    As Status is moving into a user-acquisition phase and is improving retention rates for users they need the infrastructure to keep up, specifically when it comes to messaging.

    Based on user acquisition models, the initial goal is to support 100k DAU in September, with demand growing from there.

    With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.

    **1. Connection limits**. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.

    **2. Bandwidth as a bottleneck**. We notice that memory usage appears to not be -the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.

    **3. Amplification factors**. Reducing amplification factors with better routing, would have a high impact, but it is likely we'd need additional measures as well, such as topic sharding or similar. See Fig 8-13.

    Figure 1-5:

    -
    -
    -
    -

    See https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0 for the full report.

    What we need to do is:

    1. Reduce amplification factors
    2. Get more user-run full nodes

    Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:

    • keep increasing requirements for Internet speed for full nodes
    • are vulnerable to censorship and attacks
    • have to control the topology in an artifical manner to keep up with load
    • basically re-invent a traditional centralized client-server app with extra steps
    • deliberately ignore most of our principles
    • risk the network being shut down when we run out of cash

    Appetite​

    Our initial risk appetite for this is 6 weeks for a small team.

    The idea is that we want to make tangible progress towards the goal in a limited period of time, as opposed to getting bogged down in trying to find a theoretically perfect generalized solution. Fixed time, variable scope.

    It is likely some elements of a complete solution will be done separately. See later sections for that.

    Solution​

    There are two main parts of the solution. One is to reduce amplification factors, and the other is incentivization to get more user run full nodes with desktop, etc.

    What does a full node provide? It provides connectivity to the network, can act as a bandwidth "barrier" and be high or reasonably high availability. What this means right now is essentially topic interest and storing historical messages.

    The goal is here to improve the status quo, not get a perfect solution from the get go. All of this can be iterated on further, for stronger guarantees, as well as replaced by other new modules.

    Let's first look at the baseline, and then go into some of the tracks and their phases. Track 1 is best done first, after which track 2 and 3 can be executed in parallel. Track 1 gives us more options for track 2 and 3. The work in track 1 is currently more well-defined, so it is likely the specifics of track 2 and 3 will get refined at a later stage.

    Baseline​

    Here's where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can't easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.

    Track 1 - Move to libp2p​

    Moving to PubSub over libp2p wouldn't improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.

    See more on libp2p PubSub here: https://docs.libp2p.io/concepts/publish-subscribe/

    As part of this move, there are a few individual pieces that are needed.

    1. FloodSub​

    This is essentially what Waku over libp2p would look like in its most basic form.

    One difference that is worth noting is that the app topics would not be the same as Waku topics. Why? In Waku we currently don't use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.

    Why can't we use Waku topics for routing directly? PubSub over libp2p isn't built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.

    Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.

    Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: https://github.com/status-im/nim-waku

    2. Simplify the protocol​

    Due to Waku's origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (https://rfc.vac.dev/waku/standards/legacy/6/waku1/). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?

    Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.

    The essential characteristic that has to be maintained is that we don't need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.

    3. Core integration​

    As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.

    4. Topic interest behavior​

    While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (https://vac.dev/fixing-whisper-with-waku). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.

    5. Historical message caching​

    Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there's no reason to reimplement a poor API if we don't have to.

    Also see section below on adaptive nodes and capabilities.

    6. Waku v1 <> Libp2p bridge​

    To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.

    Track 2 - Better routing​

    This is where we improve the amplification factors.

    1. GossipSub​

    This is a subprotocol of FloodSub in the libp2p world. Moving to GossipSub would allow traffic between full nodes to go from an amplification factor of ~25 to ~6. This basically creates a mesh of stable bidirectional connections, together with some gossiping capabilities outside of this view.

    Explaining how GossipSub works is out of scope of this document. It is implemented in nim-libp2p and used by Nimbus as part of Eth2. You can read the specs here in more detail if you are interested: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md and https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md

    -
    -
    -

    While we technically could implement this over existing Waku, we'd have to re-implement it, and we'd lose out on all the other benefits libp2p would provide, as well as the ecosystem of people and projects working on improving the scalability and security of these protocols.

    2. Topic sharding​

    This one is slightly more speculative in terms of its ultimate impact. The basic idea is to split the application topic into N shards, say 10, and then each full node can choose which shards to listen to. This can reduce amplification factors by another factor of 10.

    -

    Note that this means a light node that listens to several topics would have to be connected to more full nodes to get connectivity. For a more exotic version of this, see https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47

    This is orthogonal from the choice of FloodSub or GossipSub, but due to GossipSub's more dynamic nature it is likely best combined with it.

    3. Other factors​

    Not a primary focus, but worth a look. Looking at the scaling model, there might be other easy wins to improve overall bandwidth consumption between full nodes. For example, can we reduce envelope size by a significant factor?

    Track 3 - Accounting and user-run nodes​

    This is where we make sure the network isn't fragile, become a true p2p app, get our users excited and engaged, and allow us to scale the network without creating an even bigger cluster.

    To work in practice, this has a soft dependency on node discovery such as DNS based discovery (https://eips.ethereum.org/EIPS/eip-1459) or Discovery v5 (https://vac.dev/feasibility-discv5).

    1. Adaptive nodes and capabilities​

    We want to make the gradation between light nodes, full nodes, storing (partial set of) historical messages, only acting for a specific shard, etc more flexible and explicit. This is required to identify and discover the nodes you want. See https://github.com/vacp2p/specs/issues/87

    Depending on how the other tracks come together, this design should allow for a desktop node to identify as a full relaying node for some some app topic shard, but also express waku topic interest and retrieve historical messages itself.

    E.g. Disc v5 can be used to supply node properties through ENR.

    2. Accounting​

    This is based on a few principles:

    1. Some nodes contribute a lot more than other nodes in the network
    2. We can account for the difference in contribution in some fashion
    3. We want to incentivize nodes to tell the true, and be incentivized not to lie

    Accounting here is a stepping stone, where accounting is the raw data upon which some settlement later occurs. It can have various forms of granularity. See https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31 for discussion.

    We also note that in GossipSub, the mesh is bidrectional. Additionally, it doesn't appears to be a high priority issue in terms of nodes misreporting. What is an issue is having people run full nodes in the first place. There are a few points to that. It has to be possible in the end-user UX, nodes have to be discovered, and it has to be profitable/visible that you are contributing. UX and discovery are out of scope for this work, whereas visibility/accounting is part of this scope. Settlement is a stretch goal here.

    The general shape of the solution is inspired by the Swarm model, where we do accounting separate from settlement. It doesn't require any specific proofs, but nodes are incentivized to tell the truth in the following way:

    1. Both full node and light node do accounting in a pairwise, local fashion
    2. If a light node doesn't ultimately pay or lie about reporting, they get disconnected (e.g.)
    3. If a full node doesn't provide its service the light node may pick another full node (e.g.)

    While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:

    • end to end delivery
    • online time
    • completeness of storage

    This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See https://discuss.status.im/t/network-incentivisation-first-draft/1037 for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51 for using validity proofs and removing liveness requirement for settlement.

    All of this is optional at this stage, because our goal here is to improve the status quo for user run nodes. Accounting at this stage should be visible and correspond to the net benefit a node provides to another.

    As a concrete example: a light node has some topic interest and cares about historical messages on some topic. A full node communicates envelopes as they come in, communicates their high availability (online time) and stores/forward stored messages. Both nodes have this information, and if they agree settlement (initially just a mock message) can be sending a payment to an address at some time interval / over some defined volume. See future sections for how this can be improved upon.

    Also see below in section 4, using constructs such as eigentrust as a local reputation mechanism.

    3. Relax high availability requirement​

    If we want desktop nodes to participate in the storing of historical messages, high availability is a problem. It is a problem for any node, especially if they lie about it, but assuming they are honest it is still an issue.

    By being connected to multiple nodes, we can get an overlapping online window. Then these can be combined together to get consistency. This is obviously experimental and would need to be tested before being deployed, but if it works it'd be very useful.

    Additionally or alternatively, instead of putting a high requirement on message availability, focus on detection of missing information. This likely requires re-thinking how we do data sync / replication.

    4. Incentivize light and full nodes to tell the truth (policy, etc)​

    In accounting phase it is largely assumed nodes are honest. What happens when they lie, and how do we incentivize them to be honest? In the case of Bittorrent this is done with tit-for-tat, however this is a different kind of relationship. What follows are some examples of how this can be done.

    For light nodes:

    • if they don't, they get disconnected
    • prepayment (especially to "high value" nodes)

    For full nodes:

    • multiple nodes reporting to agree, where truth becomes a shelling point
    • use eigentrust
    • staking for discovery visibility with slashing

    5. Settlement PoC​

    Can be done after phase 2 if so desired. Basically integrate payments based on accounting and policy.

    Out of scope​

    1. We assume the Status Base model requirements are accurate.
    2. We assume Core will improve retention rates.
    3. We assume the Stimbus production team will enable integration of nim-waku.
    4. We assume Discovery mechanisms such as DNS and Discovery v5 will be worked on separately.
    5. We assume Core will, at some point, provide an UX for integrating payment of services.
    6. We assume the desktop client is sufficiently usable.
    7. We assume Core and Infra will investigate ways of improving MaxPeers.
    ]]>     - - Oskar - -     - - <![CDATA[Feasibility Study: Discv5]]> - https://vac.dev/rlog/feasibility-discv5 - - 2020-04-27T12:00:00.000Z - - Looking at discv5 and the theoretical numbers behind finding peers.

    Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers.

    This post serves as a more authoritative overview of the discv5 study, for a discussionary post providing more context make sure to check out the corresponding discuss post. Additionally, if you are unfamiliar with discv5, check out my previous write-up: "From Kademlia to Discv5".

    Motivating Problem​

    The discovery method currently used by Status, is made up of various components and grew over time to solve a mix of problems. We want to simplify this while maintaining some of the properties we currently have.

    Namely, we want to ensure censorship resistance to state-level adversaries. One of the issues Status had which caused us them add to their discovery method was the fact that addresses from providers like AWS and GCP were blocked both in Russia and China. Additionally, one of the main factors required is the ability to function on resource restricted devices.

    Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider:

    • Battery consumption - constant connections like websockets consume a lot of battery life.
    • CPU usage - certain discovery methods may be CPU incentive, slowing an app down and making it unusable.
    • Bandwidth consumption - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans.
    • Short connection windows - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time.
    • Not publicly connectable - There is a good chance that most resource restricted devices are not publicly connectable.

    For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node.

    With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation.

    In order to answer our questions, we formulated the problem with a simple method for testing. The "needle in a haystack" problem was formulated to figure out how easily a specific node can be found within a given network. This issue was fully formulated in vacp2p/research#15.

    Overview​

    The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer:

    • We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.
    • We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through topic tables.

    Feasbility​

    To be able to investigate the feasibility of discv5, we used various methods including rough calculations which can be found in the notebook, and a simulation isolated in vacp2p/research#19.

    CPU & Memory Usage​

    The experimental discv5 has already been used within Status, however what was noticed was that the CPU and memory usage was rather high. It therefore should be investiaged if this is still the case, and if it is, it should be isolated where this stems from. Additionally it is worth looking at whether or not this is the case with both the go and nim implementation.

    See details: vacp2p/research#31

    NAT on Cellular Data​

    If a peer is not publically connectable it can not participate in the DHT both ways. A lot of mobile phones are behind symmetric NATs which UDP hole-punching close to impossible. It should be investigated whether or not mobile phones will be able to participate both ways and if there are good methods for doing hole-punching.

    See details: vacp2p/research#29

    Topic Tables​

    Topic Tables allow us the ability to efficiently find nodes given a specific topic. However, they are not implemented in the status-im/nim-eth implementation nor are they fully finalized in the spec. These are important if the network grows past a size where the concentration of specific nodes is relatively low making them hard to find.

    See details: vacp2p/research#26

    Finding a node​

    It is important to note, that given a network is relatively small sized, eg 100-500 nodes, then finding a node given a specific address is relatively managable. Additionally, if the concentration of a specific capability in a network is reasonable, then finding a node advertising its capabilities using an ENR rather than the topic table is also managable. A reasonable concentration for example would be 10%, which would give us an 80% chance of getting a node with that capability in the first lookup request. This can be explored more using our discv5 notebook.

    Results​

    Research has shown that finding a node in the DHT has a relatively low effect on bandwidth, both inbound and outbound. For example when trying to find a node in a network of 100 nodes, it would take roughly 5668 bytes total. Additionally if we assume 100ms latency per request it would range at ≈ 300ms latency, translating to 3 requests to find a specific node.

    General Thoughts​

    One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more "features" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.

    Acknowledgements​

    • Oskar Thoren
    • Dmitry Shmatko
    • Kim De Mey
    • Corey Petty
    ]]>     - - Dean - -     - - <![CDATA[What Would a WeChat Replacement Need?]]> - https://vac.dev/rlog/wechat-replacement-need - - 2020-04-16T12:00:00.000Z - - What would a self-sovereign, private, censorship-resistant and open alternative to WeChat look like?

    What would it take to replace WeChat? More specifically, what would a self-sovereign, private, censorship-resistant and open alternative look like? One that allows people to communicate, coordinate and transact freely.

    Background​

    What WeChat provides to the end-user​

    Let's first look at some of the things that WeChat providers. It is a lot:

    • Messaging: 1:1 and group chat. Text, as well as voice and video. Post gifs. Share location.
    • Group chat: Limited to 500 people; above 100 people people need to verify with a bank account. Also has group video chat and QR code to join a group.
    • Timeline/Moments: Post comments with attachments and have people like/comment on it.
    • Location Discovery: See WeChat users that are nearby.
    • Profile: Nickname and profile picture; can alias people.
    • "Broadcast" messages: Send one message to many contacts, up to 200 people (spam limited).
    • Contacts: Max 5000 contacts (people get around it with multiple accounts and sim cards).
    • App reach: Many diferent web apps, extensions, native apps, etc. Scan QR code to access web app from phone.
    • Selective posting: Decide who can view your posts and who can view your comments on other people's post.
    • Transact: Send money gifts through red envelopes.
    • Transact: Use WeChat pay to transfer money to friends and businesses; linked account with Alipay that is connected to your bank account.
    • Services: Find taxis and get notifications; book flights, train tickets, hotels etc.
    • Mini apps: API for all kinds of apps that allow you to provide services etc.
    • Picture in picture: allowing you to have a video call while using the app.

    And much more. Not going to through it all in detail, and there are probably many things I don't know about WeChat since I'm not a heavy user living in mainland China.

    How WeChat works - a toy model​

    This is an overly simplistic model of how WeChat works, but it is sufficient for our purposes. This general design applies to most traditional client-server apps today.

    To sign up for account you need a phone number or equivalent. To get access to some features you need to verify your identity further, for example with official ID and/or bank account.

    When you signup this creates an entry in the WeChat server, from now on treated as a black box. You authenticate with that box, and thats where you get your messages from. If you go online the app asks that box for messages you have received while you were offline. If you login from a different app your contacts and conversations are synced from that box.

    The box gives you an account, it deals with routing to your contacts, it stores messages and attachments and gives access to mini apps that people have uploaded. For transacting money, there is a partnership with a different company that has a different box which talks to your bank account.

    This is done in a such a way that they can support a billion users with the features above, no sweat.

    Whoever controls that box can sees who you are talking with and what the content of those messages are. There is no end to end encryption. If WeChat/Tencent disagrees with you for some reason they can ban you. This means you can't interact with the box under that name anymore.

    What do we want?​

    We want something that is self-sovereign, private, censorship-resistant and open that allows individuals and groups of people to communicate and transact freely. To explore what this means in more detail, without getting lost in the weeds, we provide the following list of properties. A lot of these are tied together, and some fall out of the other requirements. Some of them stand in slight opposition to each other.

    Self-sovereignity identity. Exercises authority within your own sphere. If you aren't harming anyone, you should be able to have an account and communicate with other people.

    Pseudonymity, and ideally total anonymity. Not having your identity tied to your real name (e.g. through phone number, bank account, ID, etc). This allows people to act more freely without being overly worried about censorship and coercion in the real world. While total anonymity is even more desirable - especially to break multiple hops to a true-name action - real-world constraints sometimes makes this more challenging.

    Private and secure communication. Your communication and who you transact with should be for your eyes only. This includes transactions (transfer of value) as a form of communication.

    Censorship-resistance. Not being able to easily censor individuals on the platform. Both at an individual, group and collective level. Not having single points of failure that allow service to be disrupted.

    Decentralization. Partly falls out of censorship-resistance and other properties. If infrastructure isn't decentralized it means there's a single point of failure that can be disrupted. This is more of a tool than a goal on its own, but it is an important tool.

    Built for mass adoption. Includes scalabiltiy, UX (latency, reliability, bandwidth consumption, UI etc), and allowing for people to stick around. One way of doing this is to allow users to discover people they want to talk to.

    Scalability. Infrastructure needs to support a lot of users to be a viabile alternative. Like, a billion of them (eventually).

    Fundamentals in place to support great user experience. To be a viable alternative, aside from good UI and distribution, fundamentals such as latency, bandwidth usage, consistency etc must support great UX to be a viable alternative.

    Works for resource restricted devices, including smartphones. Most people will use a smartphone to use this. This means it has to work well on them and similar devices, without becoming a second-class citizen where we ignore properties such as censorship-resistance and privacy. Some concession to reality will be necessary due to additional constraints, which leads us to...

    Adaptive nodes. Nodes will have different capabilities, and perhaps at different times. To maintain a lot of the properties described here it is desirable if as many participants as possible are first-class citizens. If a phone is switching from a limited data plan to a WiFi network or from battery to AC power it can do more useful work, and so on. Likewise for a laptop with a lot of free disk space and spare compute power, etc.

    Sustainable. If there's no centralized, top down ad-driven model, this means all the infrastructure has to be sustainable somehow. Since these are individual entitites, this means it has to be paid for. While altruistic modes and similar can be used, this likely requires some form of incentivization scheme for useful services provided in the network. Related: free rider problem.

    Spam resistant. Relates to sustainability, scalability and built for mass adoption. Made more difficult by pseudonymous identity due to whitewashing attacks.

    Trust-minimized. To know that properties are provided for and aren't compromised, various ways of minimizing trust requirements are useful. This also related to mass adoption and social cohesion. Examples include: open and audited protocols, open source, reproducible builds, etc. This also relates to how mini apps are provided for, since we may not know their source but want to be able to use them anyway.

    Open source. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn't been compromised, e.g. by uploading private data to a third party.

    Some of these are graded and a bit subtle, i.e.:

    • Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible.
    • Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience.
    • Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals.
    • "Discover people" and striving for "total anonymity" might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy.
    • Users often want some form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement.

    Counterpoint 1​

    We could refute the above by saying that the design goals are undesirable. We want a system where people can censor others, and where everyone is tied to their real identity. Or we could say something like, freedom of speech is a general concept, and it doesn't apply to Internet companies, even if they provide a vital service. You can survive without it and you should've read the terms of service. This roughly charactericizes the mainstream view.

    Additional factor here is the idea that a group of people know more about what's good for you then you do, so they are protecting you.

    Counterpoint 2​

    We could agree with all these design goals, but think they are too extreme in terms of their requirements. For example, we could operate as a non profit, take donations and volunteers, and then host the whole infrastructure ourselves. We could say we are in a friendly legislation, so we won't be a single point of failure. Since we are working on this and maybe even our designs are open, you can trust us and we'll provide service and infrastructure that gives you what you want without having to pay for it or solve all these complex decentralized computation and so on problems. If you don't trust us for some reason, you shouldn't use us regardless. Also, this is better than status quo. And we are more likely to survive by doing this, either by taking shortcuts or by being less ambituous in terms of scope.

    Principal components​

    There are many ways to skin a cat, but this is one way of breaking down the problem. We have a general direction with the properties listed above, together with some understanding of how WeChat works for the everday user. Now the question is, what infrastructure do we need to support this? How do we achieve the above properties, or at least get closer to them? We want to figure out the necessary building blocks, and one of doing this is to map out likely necessary components.

    Background: Ethereum and Web3 stack​

    It is worth noting that a lot of the required infrastructure has been developed, at least as concepts, in the original Ethereum / Web3 vision. In it there is Ethereum for consensus/compute/transact, storage through Swarm, and communication through Whisper. That said, the main focus has been on the Ethereum blockchain itself, and a lot of things have happened in the last 5y+ with respect to technology around privacy and scalabilty. It is worth revisiting things from a fresh point of view, with the WeChat alternative in mind as a clear use case.

    Account - self-sovereign identity and the perils of phone numbers​

    Starting from the most basic: what is an account and how do you get one? With most internet services today, WeChat and almost all popular messaging apps included, you need to signup with some centralized authority. Usually you also have to verify this with some data that ties this account to you as an individual. E.g. by requiring a phone number, which in most jurisdictions 1 means giving out your real ID. This also means you can be banned from using the service by a somewhat arbitrary process, with no due process.

    Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this.

    Justifications for requiring phone numbers are usually centered around three claims:

    1. Avoiding spam
    2. Tying your account to your real name, for various reasons
    3. Using as a commonly shared identifier as a social network discovery mechanism

    Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on.

    In contrast, a self-sovereign system would allow you to "create an account" completely on your own. This can easily be done with public key cryptograpy, and it also paves the way for end-to-end encryption to make your messages private.

    The main issue with this that you need to get more creative about avoiding spam (e.g. through white washing attacks), and ideally there is some other form of social discovery mechanism.

    Just having a public key as an account isn't enough though. If it goes through a central server, then nothing is stopping that server from arbitrarly blocking requests related to that public key. Of course, this also depends on how transparent such requests are. Fundamentally, lest we rely completely on goodwill, there needs to be multiple actors by which you can use the service. This naturally points to decentralization as a requirement. See counterpoint.

    Even so, if the system is closed source we don't know what it is doing. Perhaps the app communicating is also uploading data to another place, or somehow making it possible to see who is who and act accordingly.

    You might notice that just one simple property, self-sovereign identity, leads to a slew of other requirements and properties. You might also notice that WeChat is far from alone in this, even if their identity requirements might be a bit stringent than, say, Telegram. Their control aspects are also a bit more extreme, at least for someone with western sensibilities 2.

    Most user facing applications have similar issues, Google Apps/FB/Twitter etc. For popular tools that have this built in, we can look at git - which is truly decentralized and have keypair at the bottom. It is for a very specific technical domain, and even then people rely on Github. Key management is fairly difficult even for technical people, and for normal people even more so. Banks are generally far behind on this tech, relying on arcane procedures and special purpose hardware for 2FA. That's another big issue.

    Let's shift gears a bit and talk about some other functional requirements.

    Routing - packets from A to B​

    In order to get a lot of the features WeChat provides, we need the ability to do three things: communicate, store data, and transact with people. We need a bit more than that, but let's focus on this for now.

    To communicate with people, in the base case, we need to go from one phone to another phone that is separated by a large distance. This requires some form of routing. The most natural platform to build this on is the existing Internet, though not the only one. Most phones are resource restricted, and are only "on" for brief periods of time. This is needed to preserve battery and bandwidth. Additionally, Internet uses IPs as endpoints, which change as a phones move through space. NAT punching etc isn't always perfect either. This means we need a way to get a message from one public key to another, and through some intermediate nodes. We can think of these nodes as a form of service network. Similar to how a power grid works, or phone lines, or collection of ISPs.

    One important property here is to ensure we don't end up in a situation like the centralized capture scenario above, something we've seen with centralized ISPs 3 4 where they can choose which traffic is good and which is bad. We want to allow the use of different service nodes, just like if a restaurant gives you food poisioning you can go to the one next door and then the first one goes out of business after a while. And the circle of life continues.

    We shouldn't be naive though, and think that this is something nodes are likely to do for free. They need to be adequately compensated for their services, in some of incentivization scheme. That can either be monetary, or as in the case of Bittorrent, more of a barter situation where you use game theory to coordinate with strangers 5, and some form of reputation attached to it (for private trackers).

    There are many ways of doing routing, and we won't go into too much technical detail here. Suffice to say is that you likely want both a structured and unstructured alternative, and that these comes with several trade-offs when it comes to efficiency, metadata protection, ability to incentivize, compatibility with existing topologies, and suitability for mobilephones (mostly offline, bandwidth restricted, not directly connectable). Expect more on this in a future article.

    Some of these considerations naturally leads us into the storage and transaction components.

    Storage - available and persistant for later​

    If mobile phones are mostly offline, we need some way to store these messages so they can be retrieved when online again. The same goes for various kinds attachments as well, and for when people are switching devices. A user might control their timeline, but in the WeChat case that timeline is stored on Tencent's servers, and queried from there as well. This naturally needs to happen by some other service nodes. In the WeChat case, and for most IMs, the way these servers are paid for is through some indirect ad mechanism. The entity controlling these ads and so on is the same one as the one operating the servers for storage. A more direct model with different entities would see these services being compensated for their work.

    We also need storage for attachments, mini-apps, as well as a way of understanding the current state of consensus when it comes to the compute/transact module. In the WeChat case, this state is completely handled by the bank institution or one of their partners, such as Alibaba. When it comes to bearer instruments like cash, no state needs to be kept as that's a direct exchange in the physical world. This isn't directly compatible with transfering value over a distance.

    All of this state requires availability and persistance. It should be done in a trust minimized fashion and decentralized, which requires some form of incentivization for keeping data around. If it isn't, you are relying on social cohesion which breaks down at very large scales.

    Since data will be spread out across multiple nodes, you need a way to sync data and transfer it in the network. As well as being able to add and query data from it. All of this requires a routing component.

    To make it more censorship resistant it might be better to keep it as a general-purpose store, i.e. individuals don't need to know what they storing. Otherwise, you naturally end up in a situation where individual nodes can be pressured to not store certain content.

    Messaging - from me to you to all of us (not them)​

    This builds on top of routing, but it has a slightly different focus. The goal is to allow for individuals and groups to communicate in a private, secure and censorship-resistant manner.

    It also needs to provide a decent interface to the end user, in terms of dealing seamlessly with offline messages, providing reliable and timely messaging.

    In order to get closer to the ideal of total anonymity, it is useful to be able to hide metadata of who is talking to whom. This applies to both normal communication as well as for transactions. Ideally, no one but the parties involved can see who is taking part in a conversation. This can be achieved through various techniques such as mixnets, anonymous credentials, private information retrieval, and so on. Many of these techniques have a fundamental trade-off with latency and bandwidth, something that is a big concern for mobilephones. Being able to do some form of tuning, in an adaptive node manner, depending on your threat model and current capabilities is useful here.

    The baseline here is pseudonymity, and having tools to allow individuals to "cut off" ties to their real world identity and transactions. People act different in different circles in the real world, and this should be mimicked online as well. Your company, family or government shouldn't be able to know what exactly you use your paycheck for, and who you are talking to.

    Compute - transact, contract and settle​

    The most immediate need here is transaction from A to B. Direct exchange. There is also a more indirect need for private lawmaking and contracting.

    We talked about routing and storage and how they likely need to be incentivized to work properly. How are they going to be compensated? While this could in theory work via existing banking system and so on, this would be rather heavy. It'd also very likely require tying your identifier to your legal name, something that goes against what we want to achieve. What we want is something that acts more as right-to-access, similar to the way cash functions in a society 6. I pay for a fruit with something that is valuable to you and then I'm on my way.

    While there might be other candidates, such as pre-paid debit cards and so on, this transaction mode pretty much requires a cryptocurrency component. The alternative is to do it on a reputation basis, which might work for small communities, due to social cohesion, but quickly detoriates for large ones 7. Ad hoc models like private Bittorrent trackers are centralized and easy to censor.

    Now, none of the existing cryptocurrency models are ideal. They also all suffer from lack of widespread use, and it is difficult to get onboarded to them in the first place. Transactions in Bitcoin are slow. Ethereum is faster and has more capabilities, but it still suffers from linking payments over time, which makes the privacy part of this more difficult. Zcash, Monero and similar are interesting, but also require more use. For Zcash, shielded transactions appear to only account for less than 2% of all transactions in 2019 8 9.

    Another dimension is what sets general purpose cryptocurrencies like Ethereum apart. Aside from just paying from A to B, you can encode rules about when something should be paid out and not. This is very useful for doing a form of private lawmaking, contracting, for setting up service agreements with these nodes. If there's no trivial recourse as in the meatspace world, where you know someone's name and you can sue them, you need a different kind of model.

    What makes something like Zcash interesting is that it works more like digital cash. Instead of leaving a public trail for everyone, where someone can see where you got the initial money from and then trace you across various usage, for Zcash every hop is privacy preserving.

    To fulfill the general goals of being censorship resistance and secure, it is also vital that the system being used stays online and can't be easily disrupted. That points to disintermediation, as opposed to using gateways and exchanges. This is a case where something like cash, or gold, is more direct, since no one can censor this transaction without being physically present where this direct exchange is taking place. However, like before, this doesn't work over distance.

    Secure chat - just our business​

    Similar to the messaging module above. The distinction here is that we assume the network part has already taken place. Here we are interested in keeping the contents of messages private, so that means confidentiality/end-to-end encryption, integrity, authentication, as well as forward secrecy and plausible deniability. This means that even if there's some actor that gets some private key material, or confiscated your phone, there is some level of...ephemerality to your conversations. Another issue here in terms of scalable private group chat.

    Extensible mini apps​

    This relates to the compute and storage module above. Essentially we want to provide mini apps as in WeChat, but to do so in a way that is compatible with what we want to achieve more generally. This allows individuals and small businesses to create small tools for various purposes, and coordinate with strangers. E.g. booking a cab or getting an insurance, and so on.

    This has a higher dependency on the contracting/general computation aspect. I.e. often it isn't only a transaction, but you might want to encode some specific rules here that strangers can abide by without having too high trust requirements. As a simple example: escrows.

    This also needs an open API that anyone can use. It should be properly secured, so using one doesn't compromise the rest of the system it is operating in. To be censorship resistant it requires the routing and storage component to work properly.

    Where are we now?​

    Let's look back at some of desirable properties we set out in the beginning and see how close we are to building out the necessary components. Is it realistic at all or just a pipe dream? We'll see that there are many building blocks in place, and there's reason for hope.

    Self-sovereignity identity. Public key crypto and web of trust like constructs makes this possible.

    Pseudonymity, and ideally total anonymity. Pseudonymity can largely be achieved with public key crypto and open systems that allow for permissionless participation. For transactions, pseudonymity exists in most cryptocurrencies. The challenge is linkage across time, especially when interfacing with other "legacy" system. There are stronger constructs that are actively being worked on and are promising here, such as mixnets (Nym), mixers (Wasabi Wallet, Tornado.Cash) and zero knowledge proofs (Zcash, Ethereum, Starkware). This area of applied research has exploded over the last few years.

    Private and secure communication. Signal has pioneered a lot of this, following OTR. Double Ratchet, X3DH. E2EE is minimum these days, and properties like PFS and PD are getting better. For metadata protection, you have Tor, with its faults, and more active research on mixnets and private information retrieval, etc.

    Censorship-resistance. This covers a lot of ground across the spectrum. You have technologies like Bittorrent, Bitcoin/Ethereum, Tor obfuscated transports, E2EE by default, partial mesh networks in production, abilit to move/replicate host machines more quickly have all made this more of a reality than it used to be. this easier. Of course, techniques such as deep packet inspection and internet shutdowns have increased.

    Decentralization. Cryptocurrencies, projects like libp2p and IPFS. Need to be mindful here of many projects that claim decentralization but are still vulnerable to single points of failures, such as relying on gateways.

    Built for mass adoption. This one is more subjective. There's definitely a lot of work to be done here, both when it comes to fundamental performance, key management and things like social discoverability. Directionally these things are improving and becoming easier for the average person but there is a lot ot be done here.

    Scalability. With projects like Ethereum 2.0 and IPFS more and more resources are a being put into this, both at the consensus/compute layer as well as networking (gossip, scalable Kademlia) layer. Also various layer 2 solutions for transactions.

    Fundamentals in place to support great user experience. Similar to built for mass adoption. As scalability becomes more important, more applied research is being done in the p2p area to improve things like latency, bandwidth.

    Works for resource restricted devices, including smartphones. Work in progress and not enough focus here, generally an after thought. Also have stateless clients etc.

    Adaptive nodes. See above. With subprotocols and capabilities in Ethereum and libp2p, this is getting easier.

    Sustainable. Token economics is a thing. While a lot of it won't stay around, there are many more projects working on making themselves dispensable. Being open source, having an engaged community and enabling users run their own infrastructure. Users as stakeholders.

    Spam resistant. Tricky problem if you want to be pseudonymous, but some signs of hope with incentivization mechanisms, zero knowledge based signaling, etc. Together with various forms of rate limiting and better controlling of topology and network amplification. And just generally being battle-tested by real world attacks, such as historical Ethereum DDoS attacks.

    Trust minimized. Bitcoin. Zero knowledge provable computation. Open source. Reproducible builds. Signed binaries. Incentive compatible structures. Independent audits. Still a lot of work, but getting better.

    Open source. Big and only getting bigger. Including mainstream companies.

    What's next?​

    We've look at what WeChat provides and what we'd like an alternative to look like. We've also seen a few principal modules that are necessary to achieve those goals. To achieve all of this is a daunting task, and one might call it overly ambitiuous. We've also seen how far we've come with some of the goals, and how a lot of the pieces are there, in one form or another. Then it is a question of putting them all together in the right mix.

    The good news is that a lot of people are working all these building blocks and thinking about these problems. Compared to a few years ago we've come quite far when it comes to p2p infrastructure, privacy, security, scalability, and general developer mass and mindshare. If you want to join us in building some of these building blocks, and assembling them, check out our forum.

    PS. We are hiring protocol engineers. DS

    Acknowledgements​

    Corey, Dean, Jacek.

    References​

    1. Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws↩
    2. On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/↩
    3. Net Neutrality: https://www.eff.org/issues/net-neutrality↩
    4. ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/↩
    5. Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf↩
    6. The Case for Electronic Cash: https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf↩
    7. Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html↩
    8. Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions↩
    9. Shielded transactions usage (stats page 404s): https://z.cash/support/faq/↩
    ]]>     - - Oskar - -     - - <![CDATA[From Kademlia to Discv5]]> - https://vac.dev/rlog/kademlia-to-discv5 - - 2020-04-09T16:00:00.000Z - - A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.

    If you've been working on Ethereum or adjacent technologies you've probably heard of discv4 or discv5. But what are they actually? How do they work and what makes them different? To answer these questions, we need to start at the beginning, so this post will assume that there is little knowledge on the subject so the post should be accessible for anyone.

    The Beginning​

    Let's start right at the beginning: the problem of discovery and organization of nodes in peer-to-peer networks.

    Early P2P file sharing technologies, such as Napster, would share information about who holds what file using a single server. A node would connect to the central server and give it a list of the files it owns. Another node would then connect to that central server, find a node that has the file it is looking for and contact that node. This however was a flawed system -- it was vulnerable to attacks and left a single party open to lawsuits.

    It became clear that another solution was needed, and after years of research and experimentation, we were given the distributed hash table or DHT.

    Distributed Hash Tables​

    In 2001 4 new protocols for such DHTs were conceived, Tapestry, Chord, CAN and Pastry, all of which made various trade-offs and changes in their core functionality, giving them unique characteristics.

    But as said, they're all DHTs. So what is a DHT?

    A distributed hash table (DHT) is essentially a distributed key-value list. Nodes participating in the DHT can easily retrieve the value for a key.

    If we have a network with 9 key-value pairs and 3 nodes, ideally each node would store 3 (optimally 6 for redundancy) of those key-value pairs, meaning that if a key-value pair were to be updated, only part of the network would responsible for ensuring that it is. The idea is that any node in the network would know where to find the specific key-value pair it is looking for based on how things are distributed amongst the nodes.

    Kademlia​

    So now that we know what DHTs are, let's get to Kademlia, the predecessor of discv4. Kademlia was created by Petar Maymounkov and David Mazières in 2002. I will naively say that this is probably one of the most popular and most used DHT protocols. It's quite simple in how it works, so let's look at it.

    In Kademlia, nodes and values are arranged by distance (in a very mathematical definition). This distance is not a geographical one, but rather based on identifiers. It is calculated how far 2 identifiers are from eachother using some distance function.

    Kademlia uses an XOR as its distance function. An XOR is a function that outputs true only when inputs differ. Here is an example with some binary identifiers:

    XOR 10011001
        00110010
        --------
        10101011

    The top in decimal numbers means that the distance between 153 and 50 is 171.

    There are several reasons why XOR was taken:

    1. The distance from one ID to itself will be 0.
    2. Distance is symmetric, A to B is the same as B to A.
    3. Follows triangle inequality, if A, B and C are points on a triangle then the distance A to B is closer or equal to that of A to C plus the one from B to C.

    In summary, this distance function allows a node to decide what is "close" to it and make decisions based on that "closeness".

    Kademlia nodes store a routing table. This table contains multiple lists. Each subsequent list contains nodes which are a little further distanced than the ones included in the previous list. Nodes maintain detailed knowledge about nodes closest to them, and the further away a node is, the less knowledge the node maintains about it.

    So let's say I want to find a specific node. What I would do is go to any node which I already know and ask them for all their neighbours closest to my target. I repeat this process for the returned neighbours until I find my target.

    The same thing happens for values. Values have a certain distance from nodes and their IDs are structured the same way so we can calculate this distance. If I want to find a value, I simply look for the neighbours closest to that value's key until I find the one storing said value.

    For Kademlia nodes to support these functions, there are several messages with which the protocol communicates.

    • PING - Used to check whether a node is still running.
    • STORE - Stores a value with a given key on a node.
    • FINDNODE - Returns the closest nodes requested to a given ID.
    • FINDVALUE - The same as FINDNODE, except if a node stores the specific value it will return it directly.

    This is a very simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the paper or a more in-depth design specification

    Discv4​

    Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT.

    Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the FINDVALUE and STORE commands described by Kademlia.

    The lookup method previously described by Kademlia describes how a node gets its peers. A node contacts some node and asks it for the nodes closest to itself. It does so until it can no longer find any new nodes.

    Additionally, discv4 adds mutual endpoint verification. This is meant to ensure that a peer calling FINDNODE also participates in the discovery protocol.

    Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called ENRRequest.

    If you want some more details on ENRs, check out one of my posts "Network Addresses in Ethereum"

    Discv4 comes with its own range of problems however. Let's look at a few of them.

    Firstly, the way discv4 works right now, there is no way to differentiate between node sub-protocols. This means for example that an Ethereum node could add an Ethereum Classic Node, Swarm or Whisper node to its DHT without realizing that it is invalid until more communication has happened. This inability to differentiate sub-protocols makes it harder to find specific nodes, such as Ethereum nodes with light-client support.

    Next, in order to prevent replay attacks, discv4 uses timestamps. This however can lead to various issues when a host's clock is wrong. For more details, see the "Known Issues" section of the discv4 specification.

    Finally, we have an issue with the way mutual endpoint verification works. Messages can get dropped and there is no way to tell if both peers have verified eachother. This means that we could consider our peer verified while it does not consider us so making them drop the FINDNODE packet.

    Discv5​

    Finally, let's look at discv5. The next iteration of discv4 and the discovery protocol which will be used by Eth 2.0. It aims at fixing various issues present in discv4.

    The first change is the way FINDNODE works. In traditional Kademlia as well as in discv5, we pass an identifier. However, in discv5 we instead pass the logarithmic distance, meaning that a FINDNODE request gets a response containing all nodes at the specified logarithmic distance from the called node.

    Logarithmic distance means we first calculate the distance and then run it through our log base 2 function. See:

    log2(A xor B)

    And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are first in first out lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering ads on their peers.

    As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place ads on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network.

    Additionally, it is unclear how many peers a node should place these ads on and exactly which peers to place them on. For more details, check out the issue devp2p#136.

    There are a bunch more smaller changes to the protocol, but they are less important hence they were ommitted from this summary.

    Nevertheless, discv5 still does not resolve a couple issues present in discv4, such as unreliable endpoint verification. As of writing this post, there is currently no new method in discv5 to improve the endpoint verification process.

    As you can see discv5 is still a work in progress and has a few large challenges to overcome. However if it does, it will most likely be a large improvement to a more naive Kademlia implementations.

    Hopefully this article helped explain what these discovery protocols are and how they work. If you're interested in their full specifications you can find them on github.

    ]]>     - - Dean - -     - - <![CDATA[Waku Update]]> - https://vac.dev/rlog/waku-update - - 2020-02-14T12:00:00.000Z - - A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?

    Waku is our fork of Whisper where we address the shortcomings of Whisper in an iterative manner. We've seen a in previous post that Whisper doesn't scale, and why. In this post we'll talk about what the current state of Waku is, how many users it can support, and future plans.

    Current state​

    Specs:

    We released Waku spec v0.3 this week! You can see the full changelog here.

    The main change from 0.2 is making the handshake more flexible. This enables us to communicate topic interest immediately without ambiguity. We also did the following:

    • added recommendation for DNS based discovery
    • added an upgradability and compatibility policy
    • cut the spec up into several components

    We cut the spec up in several components to make Vac as modular as possible. The components right now are:

    We can probably factor these out further as the main spec is getting quite big, but this is good enough for now.

    Clients:

    There are currently two clients that implement Waku v0.3, these are Nimbus (Update: now nim-waku) in Nim and status-go in Go.

    For more details on what each client support and don't, you can follow the work in progress checklist.

    Work is currently in progress to integrate it into the Status core app. Waku is expected to be part of their upcoming 1.1 release (see Status app roadmap (link deprecated)).

    Simulation:

    We have a simulation that verifies - or rather, fails to falsify - our scalability model. More on the simulation and what it shows below.

    How many users does Waku support?​

    This is our current understanding of how many users a network running Waku can support. Specifically in the context of the Status chat app, since that's the most immediate consumer of Waku. It should generalize fairly well to most deployments.

    tl;dr (for Status app):

    • beta: 100 DAU
    • v1: 1k DAU
    • v1.1 (waku only): 10k DAU (up to x10 with deployment hotfixes)
    • v1.2 (waku+dns): 100k DAU (can optionally be folded into v1.1)

    Assuming 10 concurrent users = 100 DAU. Estimate uncertainty increases for each order of magnitude until real-world data is observed.

    As far as we know right now, these are the bottlenecks we have:

    • Immediate bottleneck - Receive bandwidth for end user clients (aka ‘Fixing Whisper with Waku’)
    • Very likely bottleneck - Nodes and cluster capacity (aka ‘DNS based node discovery’)
    • Conjecture but not unlikely to appear- Full node traffic (aka ‘the routing / partition problem’)

    We've already seen the first bottleneck being discussed in the initial post. Dean wrote a post on DNS based discovery which explains how we will address the likely second bottleneck. More on the third one in future posts.

    For more details on these bottlenecks, see Scalability estimate: How many users can Waku and the Status app support?.

    Simulation​

    The ultimate test is real-world usage. Until then, we have a simulation thanks to Kim De Mey from the Nimbus team!

    We have two network topologies, Star and full mesh. Both networks have 6 full nodes, one traditional light node with bloom filter, and one Waku light node.

    One of the full nodes sends 1 envelope over 1 of the 100 topics that the two light nodes subscribe to. After that, it sends 10000 envelopes over random topics.

    For light node, bloom filter is set to almost 10% false positive (bloom filter: n=100, k=3, m=512). It shows the number of valid and invalid envelopes received for the different nodes.

    Star network:

    DescriptionPeersValidInvalid
    Master node7100010
    Full node 13100010
    Full node 21100010
    Full node 31100010
    Full node 41100010
    Full node 51100010
    Light node28150
    Waku light node210

    Full mesh:

    DescriptionPeersValidInvalid
    Full node 071000120676
    Full node 17100019554
    Full node 251000123304
    Full node 351000111983
    Full node 451000124425
    Full node 551000123472
    Light node2803803
    Waku light node211

    Things to note:

    • Whisper light node with ~10% false positive gets ~10% of total traffic
    • Waku light node gets ~1000x less envelopes than Whisper light node
    • Full mesh results in a lot more duplicate messages, expect for Waku light node

    Run the simulation yourself here. The parameters are configurable, and it is integrated with Prometheus and Grafana.

    Difference between Waku and Whisper​

    Summary of main differences between Waku v0 spec and Whisper v6, as described in EIP-627:

    • Handshake/Status message not compatible with shh/6 nodes; specifying options as association list
    • Include topic-interest in Status handshake
    • Upgradability policy
    • topic-interest packet code
    • RLPx subprotocol is changed from shh/6 to waku/0.
    • Light node capability is added.
    • Optional rate limiting is added.
    • Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits
    • Mail Server and Mail Client functionality is now part of the specification.
    • P2P Message packet contains a list of envelopes instead of a single envelope.

    Next steps and future plans​

    Several challenges remain to make Waku a robust and suitable base -communication protocol. Here we outline a few challenges that we are addressing and will continue to work on:

    • scalability of the network
    • incentived infrastructure and spam-resistance
    • build with resource restricted devices in mind, including nodes being mostly offline

    For the third bottleneck, a likely candidate for fixing this is Kademlia routing. This is similar to what is done in Swarm's PSS. We are in the early stages of experimenting with this over libp2p in nim-libp2p. More on this in a future post!

    Acknowledgements​

    Image from "caged sky" by mh.xbhd.org is licensed under CC BY 2.0 (https://ccsearch.creativecommons.org/photos/a9168311-78de-4cb7-a6ad-f92be8361d0e)

    ]]>     - - Oskar - -     - - <![CDATA[DNS Based Discovery]]> - https://vac.dev/rlog/dns-based-discovery - - 2020-02-07T12:00:00.000Z - - A look at EIP-1459 and the benefits of DNS based discovery.

    Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as Kademlia which utilizes a distributed hash table or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data.

    Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a static list of nodes that a client can connect to.

    In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable.

    How do we do this?

    EIP 1459: Node Discovery via DNS, which is one of the strategies we are using for discovering waku nodes. EIP-1459 is a DNS-based discovery protocol that stores merkle trees in DNS records which contain connection information for nodes.

    Waku is our fork of Whisper. Oskar recently wrote an entire post explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification here

    DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be.

    > dig dnsseed.bluematt.me +short
    129.226.73.12
    107.180.78.111
    169.255.56.123
    91.216.149.28
    85.209.240.91
    66.232.124.232
    207.55.53.96
    86.149.241.168
    193.219.38.57
    190.198.210.139
    74.213.232.234
    158.181.226.33
    176.99.2.207
    202.55.87.45
    37.205.10.3
    90.133.4.73
    176.191.182.3
    109.207.166.232
    45.5.117.59
    178.211.170.2
    160.16.0.30

    The above displays the result of querying on of these DNS seeds. All the nodes are stored as A records for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the IRC bootstrapping method in v0.8.2.

    What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices.

    Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name.

    As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so.

    One thing you notice when looking at EIP-1459 is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method?

    There are multiple reasons, but the main one is security. In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in EIP-1459 where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records.

    Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party.

    So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using merkle trees as follows:

    • The root of the tree is stored in a TXT record, this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above.

      Additionally, there is also a root hash to a second tree called a link tree, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names.

      The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version.

    • Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a TXT record. The branch record contains all the hashes of the branch's leafs.

    • Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at EIP-778, or read a short blog post I wrote explaining them here.

    Below is the zone file taken from the EIP-1459, displaying how this looks in practice.

    ; name                        ttl     class type  content
    @                             60      IN    TXT   enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA
    C7HRFPF3BLGF3YR4DY5KX3SMBE    86900   IN    TXT   enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org
    JWXYDBPXYWG6FX3GMDIBFA6CJ4    86900   IN    TXT   enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24
    2XS2367YHAXJFGLZHVAWLQD4ZY    86900   IN    TXT   enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA
    H4FHT4B454P6UXFD7JCYQ5PWDY    86900   IN    TXT   enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI
    MHTDO6TMUBRIA2XWG5LUDACK24    86900   IN    TXT   enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o

    All of this has already been introduced into go-ethereum with the pull request #20094, created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on github.

    I hope this was a helpful explainer into DNS based discovery, and shows EIP-1459's benefits over more traditional DNS-based discovery schemes.

    ]]>     - - Dean - -     - - <![CDATA[Fixing Whisper with Waku]]> - https://vac.dev/rlog/fixing-whisper-with-waku - - 2019-12-03T12:00:00.000Z - - A research log. Why Whisper doesn't scale and how to fix it.

    This post will introduce Waku. Waku is a fork of Whisper that attempts to -addresses some of Whisper's shortcomings in an iterative fashion. We will also -introduce a theoretical scaling model for Whisper that shows why it doesn't -scale, and what can be done about it.

    Introduction​

    Whisper is a gossip-based communication protocol or an ephemeral key-value store -depending on which way you look at it. Historically speaking, it is the -messaging pilllar of Web3, together with -Ethereum for consensus and Swarm for storage.

    Whisper, being a somewhat esoteric protocol and with some fundamental issues, -hasn't seen a lot of usage. However, applications such as Status are using it, -and have been making minor ad hoc modifications to it to make it run on mobile -devices.

    What are these fundamental issues? In short:

    1. scalability, most immediately when it comes to bandwidth usage
    2. spam-resistance, proof of work is a poor mechanism for heterogeneous nodes
    3. no incentivized infrastructure, leading to centralized choke points
    4. lack of formal and unambiguous specification makes it hard to analyze and implement
    5. running over devp2p, which limits where it can run and how

    In this post, we'll focus on the first problem, which is scalability through bandwidth usage.

    Whisper theoretical scalability model​

    (Feel free to skip this section if you want to get right to the results).

    There's widespread implicit knowledge that Whisper "doesn't scale", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see Status Whisper usage -spec).

    Caveats​

    First, some caveats: this model likely contains bugs, has wrong assumptions, or completely misses certain dimensions. However, it acts as a form of existence proof for unscalability, with clear reasons.

    If certain assumptions are wrong, then we can challenge them and reason about them in isolation. It doesn’t mean things will definitely work as the model predicts, and that there aren’t unknown unknowns.

    The model also only deals with receiving bandwidth for end nodes, uses mostly static assumptions of averages, and doesn’t deal with spam resistance, privacy guarantees, accounting, intermediate node or network wide failures.

    Goals​

    1. Ensure network scales by being user or usage bound, as opposed to bandwidth growing in proportion to network size.
    2. Staying with in a reasonable bandwidth limit for limited data plans.
    3. Do the above without materially impacting existing nodes.

    It proceeds through various case with clear assumptions behind them, starting from the most naive assumptions. It shows results for 100 users, 10k users and 1m users.

    Model​

    Case 1. Only receiving messages meant for you [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A4. Only receiving messages meant for you.

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is 1000.0KB/day
    For  1m users, receiving bandwidth is 1000.0KB/day

    ------------------------------------------------------------

    Case 2. Receiving messages for everyone [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A5. Received messages for everyone.

    For 100 users, receiving bandwidth is   97.7MB/day
    For 10k users, receiving bandwidth is    9.5GB/day
    For  1m users, receiving bandwidth is  953.7GB/day

    ------------------------------------------------------------

    Case 3. All private messages go over one discovery topic [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A8. All private messages are received by everyone (same topic) (static).

    For 100 users, receiving bandwidth is   49.3MB/day
    For 10k users, receiving bandwidth is    4.8GB/day
    For  1m users, receiving bandwidth is  476.8GB/day

    ------------------------------------------------------------

    Case 4. All private messages are partitioned into shards [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    Case 5. 4 + Bloom filter with false positive rate

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1

    For 100 users, receiving bandwidth is   10.7MB/day
    For 10k users, receiving bandwidth is  978.0MB/day
    For  1m users, receiving bandwidth is   95.5GB/day

    NOTE: Traffic extremely sensitive to bloom false positives
    This completely dominates network traffic at scale.
    With p=1% we get 10k users ~100MB/day and 1m users ~10gb/day)

    ------------------------------------------------------------

    Case 6. Case 5 + Benign duplicate receives

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).

    For 100 users, receiving bandwidth is   21.5MB/day
    For 10k users, receiving bandwidth is    1.9GB/day
    For  1m users, receiving bandwidth is  190.9GB/day

    ------------------------------------------------------------

    Case 7. 6 + Mailserver under good conditions; small bloom fp; mostly offline

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).
    - A17. User is offline p% of the time (static) p=0.9
    - A18. No bad request, dup messages for mailservers; overlap perfect (static).
    - A19. Mailserver requests can change false positive rate to be p=0.01

    For 100 users, receiving bandwidth is    3.9MB/day
    For 10k users, receiving bandwidth is  284.8MB/day
    For  1m users, receiving bandwidth is   27.8GB/day

    ------------------------------------------------------------

    Case 8. No metadata protection w bloom filter; 1 node connected; static shard

    Aka waku mode.

    Next step up is to either only use contact code, or shard more aggressively.
    Note that this requires change of other nodes behavior, not just local node.

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    See source -for more detail on the model and its assumptions.

    Takeaways​

    1. Whisper as it currently works doesn’t scale, and we quickly run into unacceptable bandwidth usage.
    2. There are a few factors of this, but largely it boils down to noisy topics usage and use of bloom filters. Duplicate (e.g. see Whisper vs PSS) and bad envelopes are also factors, but this depends a bit more on specific deployment configurations.
    3. Waku mode (case 8) is an additional capability that doesn’t require other nodes to change, for nodes that put a premium on performance.
    4. The next bottleneck after this is the partitioned topics (app/network specific), which either needs to gracefully (and potentially quickly) grow, or an alternative way of consuming those messages needs to be deviced.

    The results are summarized in the graph above. Notice the log-log scale. The -colored backgrounds correspond to the following bandwidth usage:

    • Blue: <10mb/d (<~300mb/month)
    • Green: <30mb/d (<~1gb/month)
    • Yellow: <100mb/d (<~3gb/month)
    • Red: >100mb/d (>3gb/month)

    These ranges are somewhat arbitrary, but are based on user -requirements for users -on a limited data plan, with comparable usage for other messaging apps.

    Introducing Waku​

    Motivation for a new protocol​

    Apps such as Status will likely use something like Whisper for the forseeable -future, and we want to enable them to use it with more users on mobile devices -without bandwidth exploding with minimal changes.

    Additionally, there's not a clear cut alternative that maps cleanly to the -desired use cases (p2p, multicast, privacy-preserving, open, etc).

    We are actively researching, developing and collaborating with more greenfield -approaches. It is likely that Waku will either converge to those, or Waku will -lay the groundwork (clear specs, common issues/components) necessary to make -switching to another protocol easier. In this project we want to emphasize -iterative work with results on the order of weeks.

    Briefly on Waku mode​

    • Doesn’t impact existing clients, it’s just a separate node and capability.
    • Other nodes can still use Whisper as is, like a full node.
    • Sacrifices metadata protection and incurs higher connectivity/availability requirements for scalbility

    Requirements:

    • Exposes API to get messages from a set of list of topics (no bloom filter)
    • Way of being identified as a Waku node (e.g. through version string)
    • Option to statically encode this node in app, e.g. similar to custom bootnodes/mailserver
    • Only node that needs to be connected to, possibly as Whisper relay / mailserver hybrid

    Provides:

    • likely provides scalability of up to 10k users and beyond
    • with some enhancements to partition topic logic, can possibly scale up to 1m users (app/network specific)

    Caveats:

    • hasn’t been tested in a large-scale simulation
    • other network and intermediate node bottlenecks might become apparent (e.g. full bloom filter and private cluster capacity; can likely be dealt with in isolation using known techniques, e.g. load balancing) (deployment specific)

    Progress so far​

    In short, we have a Waku version 0 spec up as well as a PoC for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for Waku mode. See rough roadmap, project board [link deprecated] and progress thread on the Vac forum.

    The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously ad hoc implemented features, such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the geth (Go), status/whisper (Go) and nim-eth (Nim) versions, specifically around light node usage and the handshake.

    If you are interested in this effort, please check out our forum for questions, comments and proposals. We already have some discussion for better spam protection (see previous post for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement.

    ]]>     - - Oskar - -     - - <![CDATA[Feasibility Study: Semaphore rate limiting through zkSNARKs]]> - https://vac.dev/rlog/feasibility-semaphore-rate-limiting-zksnarks - - 2019-11-08T12:00:00.000Z - - A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks.

    tldr: Moon math promising for solving spam in Whisper, but to get there we need to invest more in performance work and technical upskilling.

    Motivating problem​

    In open p2p networks for messaging, one big problem is spam-resistance. Existing solutions, such as Whisper's proof of work, are insufficient, especially for heterogeneous nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.

    One possible solution is to use a right-to-access staking-based method, where a node is only able to send a message, signal, at a certain rate, and otherwise they can be slashed. One problem with this is in terms of privacy-preservation, where we specifically don't want a user to be tied to a specific payment or unique fingerprint.

    In addition to above, there are a lot of related problems that share similarities in terms of their structure and proposed solution.

    • Private transactions (Zcash, AZTEC)
    • Private voting (Semaphore)
    • Private group membership (Semaphore)
    • Layer 2 scaling, poss layer 1 (ZK Rollup; StarkWare/Eth2-3)

    Overview​

    Basic terminology​

    A zero-knowledge proof allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id.

    zkSNARKs is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more.

    Semaphore is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See current Github home for more.

    Circom is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See Github for more.

    Basic flow​

    We start with a private voting example, and then extend it to the slashable rate limiting example.

    1. A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are.

    2. When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some public outputs, and can be verified by anyone (including a smart contract).

    3. Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as "2020-01-01 vote on Foo Bar for president".

    4. Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once.

    5. As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified.

    Rate limiting example​

    In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat.

    NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same.

    1. Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s.

    2. Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key).

    3. This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them.

    4. As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it.

    Briefly on scope of 'approved users'​

    In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration.

    Technical details​

    For a fairly self-contained set of examples above, see exploration in Vac research repo. Note that the Shamir secret sharing is not inside the SNARK, but out-of-band for now.

    The current version of Semaphore is using NodeJS and Circom from Iden3 for Snarks.

    For more on rate limiting idea, see ethresearch post.

    Feasibility​

    The above repo was used to exercise the basic paths and to gain intution of feasibility. Based on it and related reading we outline a few blockers and things that require further study.

    Technical feasibility​

    Proof time​

    Prove time for Semaphore (https://github.com/kobigurk/semaphore) zKSNARKs using circom, groth and snarkjs is currently way too long. It takes on the order of ~10m to generate a proof. With Websnark, it is likely to take 30s, which might still be too long. We should experiment with native code on mobile here.

    See details.

    Proving key size​

    Prover key size is ~110mb for Semaphore. Assuming this is embedded on mobile device, it bloats the APK a lot. Current APK size is ~30mb and even that might be high for people with limited bandwidth.

    See details.

    Trusted setup​

    Using zkSNARKs a trusted setup is required to generate prover and verifier keys. As part of this setup, a toxic parameter lambda is generated. If a party gets access to this lambda, they can prove anything. This means people using zKSNARKs usually have an elaborate MPC ceremony to ensure this parameter doesn't get discovered.

    See details.

    Shamir logic in SNARK​

    For Semaphore RLN we need to embed the Shamir logic inside the SNARK in order to do slashing for spam. Currently the implementation is trusted and very hacky.

    See details.

    End to end integation​

    Currently is standalone and doesn't touch multiple users, deployed contract with merkle tree and verification, actual transactions, a mocked network, add/remove members, etc. There are bound to be edge cases and unknown unknowns here.

    See details.

    Licensing issues​

    Currently Circom uses a GPL license, which can get tricky when it comes to the App Store etc.

    See details.

    Alternative ZKPs?​

    Some of the isolated blockers for zKSNARKs (#7, #8, #9) might be mitigated by the use of other ZKP technology. However, they likely have their own issues.

    See details.

    Social feasibility​

    Technical skill​

    zkSNARKs and related technologies are quite new. To learn how they work and get an intuition for them requires individuals to dedicate a lot of time to studying them. This means we must make getting competence in these technologies if we wish to use them to our advantage.

    Time and resources​

    In order for this and related projects (such as private transaction) to get anywhere, it must be made an explicit area of focus for an extend period of time.

    General thoughts​

    Similar to Whisper, and in line with moving towards protocol and infrastructure, we need to upskill and invest resources into this. This doesn't mean developing all of the technologies ourselves, but gaining enough competence to leverage and extend existing solutions by the growing ZKP community.

    For example, this might also include leveraging largely ready made solutions such as AZTEC for private transaction; more fundamental research into ZK rollup and similar; using Semaphore for private group membership and private voting; Nim based wrapper aronud Bellman, etc.

    Acknowledgement​

    Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues.

    Peacock header image from [Tonos](<https://en.wikipedia.org/wiki/File:Flickr-lo.tangelini-Tonos(1).jpg>)._

    ]]>     - - Oskar - -     - - <![CDATA[P2P Data Sync with a Remote Log]]> - https://vac.dev/rlog/remote-log - - 2019-10-04T12:00:00.000Z - - A research log. Asynchronous P2P messaging? Remote logs to the rescue!

    A big problem when doing end-to-end data sync between mobile nodes is that most devices are offline most of the time. With a naive approach, you quickly run into issues of 'ping-pong' behavior, where messages have to be constantly retransmitted. We saw some basic calculations of what this bandwidth multiplier looks like in a previous post.

    While you could do some background processing, this is really battery-draining, and on iOS these capabilities are limited. A better approach instead is to loosen the constraint that two nodes need to be online at the same time. How do we do this? There are two main approaches, one is the store and forward model, and the other is a remote log.

    In the store and forward model, we use an intermediate node that forward messages on behalf of the recipient. In the remote log model, you instead replicate the data onto some decentralized storage, and have a mutable reference to the latest state, similar to DNS. While both work, the latter is somewhat more elegant and "pure", as it has less strict requirements of an individual node's uptime. Both act as a highly-available cache to smoothen over non-overlapping connection windows between endpoints.

    In this post we are going to describe how such a remote log schema could work. Specifically, how it enhances p2p data sync and takes care of the following requirements:

    1. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices -that are resource restricted, mostly-offline and often changing network.
    1. MAY use helper services in order to be more mobile-friendly. Examples of -helper services are decentralized file storage solutions such as IPFS and -Swarm. These help with availability and latency of data for mostly-offline -devices.

    Remote log​

    A remote log is a replication of a local log. This means a node can read data from a node that is offline.

    The spec is in an early draft stage and can be found here. A very basic spike / proof-of-concept can be found here.

    Definitions​

    TermDefinition
    CASContent-addressed storage. Stores data that can be addressed by its hash.
    NSName system. Associates mutable data to a name.
    Remote logReplication of a local log at a different location.

    Roles​

    There are four fundamental roles:

    1. Alice
    2. Bob
    3. Name system (NS)
    4. Content-addressed storage (CAS)

    The remote log is the data format of what is stored in the name system.

    "Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.

    Flow​

    Figure 1: Remote log data synchronization.

    Data format​

    The remote log lets receiving nodes know what data they are missing. Depending on the specific requirements and capabilities of the nodes and name system, the information can be referred to differently. We distinguish between three rough modes:

    1. Fully replicated log
    2. Normal sized page with CAS mapping
    3. "Linked list" mode - minimally sized page with CAS mapping

    A remote log is simply a mapping from message identifiers to their corresponding address in a CAS:

    Message Identifier (H1)CAS Hash (H2)
    H1_3H2_3
    H1_2H2_2
    H1_1H2_1
    address to next page

    The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.

    Message Identifier (H1)Content
    H1_3C3
    H1_2C2
    H1_1C1
    address to next page

    Both patterns can be used in parallel, e,g. by storing the last k messages directly and use CAS pointers for the rest. Together with the next_page page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.

    Interaction with MVDS​

    vac.mvds.Message payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.

    Future work​

    The spec is still in an early draft stage, so it is expected to change. Same with the proof of concept. More work is needed on getting a fully featured proof of concept with specific CAS and NAS instances. E.g. Swarm and Swarm Feeds, or IPFS and IPNS, or something else.

    For data sync in general:

    • Make consistency guarantees more explicit for app developers with support for sequence numbers and DAGs, as well as the ability to send non-synced messages. E.g. ephemeral typing notifications, linear/sequential history and casual consistency/DAG history
    • Better semantics and scalability for multi-user sync contexts, e.g. CRDTs and joining multiple logs together
    • Better usability in terms of application layer usage (data sync clients) and supporting more transports

    PS1. Thanks everyone who submitted great logo proposals for Vac!

    PPS2. Next week on October 10th decanus and I will be presenting Vac at Devcon, come say hi :)

    ]]>     - - Oskar - -     - - <![CDATA[Vac - A Rough Overview]]> - https://vac.dev/rlog/vac-overview - - 2019-08-02T12:00:00.000Z - - Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. What does that mean? Let's unpack it a bit.

    Basic terms​

    messaging stack. While the initial focus is on data sync, we are concerned with all layers in the stack. That means all the way from underlying transports, p2p overlays and routing, to initial trust establishment and semantics for things like group chat. The ultimate goal is to give application developers the tools they need to provide secure messaging for their users, so they can focus on their domain expertise.

    modular. Unlike many other secure messaging applications, our goal is not to have a tightly coupled set of protocols, nor is it to reinvent the wheel. Instead, we aim to provide options at each layer in the stack, and build on the shoulders of giants, putting a premimum on interoperability. It's similar in philosophy to projects such as libp2p or Substrate in that regard. Each choice comes with different trade-offs, and these look different for different applications.

    peer-to-peer. The protocols we work on are pure p2p, and aim to minimize centralization. This too is in opposition to many initiatives in the secure messaging space.

    messaging. By messaging we mean messaging in a generalized sense. This includes both human to human communication, as well machine to machine communication. By messaging we also mean something more fundamental than text messages, we also include things like transactions (state channels, etc) under this moniker.

    secure messaging. Outside of traditional notions of secure messaging, such as ensuring end to end encryption, forward secrecy, avoiding MITM-attacks, etc, we are also concerned with two other forms of secure messaging. We call these private messaging and censorship-resistance. Private messaging means viewing privacy as a security property, with all that entails. Censorship resistance ties into being p2p, but also in terms of allowing for transports and overlays that can't easily be censored by port blocking, traffic analysis, and similar.

    Vāc. Is a Vedic goddess of speech. It also hints at being a vaccine.

    Protocol stack​

    What does this stack look like? We take inspiration from core internet architecture, existing survey work and other efforts that have been done to decompose the problem into orthogonal pieces. Each layer provides their own set of properties and only interact with the layers it is adjacent to. Note that this is a rough sketch.

    Layer / ProtocolPurposeExamples
    Application layerEnd user semantics1:1 chat, group chat
    Data SyncData consistencyMVDS, BSP
    Secure TransportConfidentiality, PFS, etcDouble Ratchet, MLS
    Transport PrivacyTransport and metadata protectionWhisper, Tor, Mixnet
    P2P OverlayOverlay routing, NAT traversaldevp2p, libp2p
    Trust EstablishmentEstablishing end-to-end trustTOFU, web of trust

    As an example, end user semantics such as group chat or moderation capabilities can largely work regardless of specific choices further down the stack. Similarly, using a mesh network or Tor doesn't impact the use of Double Ratchet at the Secure Transport layer.

    Data Sync plays a similar role to what TCP does at the transport layer in a traditional Internet architecture, and for some applications something more like UDP is likely to be desirable.

    In terms of specific properties and trade-offs at each layer, we'll go deeper down into them as we study them. For now, this is best treated as a rough sketch or mental map.

    Problems and rough priorities​

    With all the pieces involved, this is quite an undertaking. Luckily, a lot of pieces are already in place and can be either incorporated as-is or iterated on. In terms of medium and long term, here's a rough sketch of priorities and open problems.

    1. Better data sync. While the current MVDS works, it is lacking in a few areas:
    • Lack of remote log for mostly-offline offline devices
    • Better scalability for multi-user chat contexts
    • Better usability in terms of application layer usage and supporting more transports
    1. Better transport layer support. Currently MVDS runs primarily over Whisper, which has a few issues:
    • scalability, being able to run with many nodes
    • spam-resistance, proof of work is a poor mechanism for heterogeneous devices
    • no incentivized infrastructure, leading to centralized choke points

    In addition to these most immediate concerns, there are other open problems. Some of these are overlapping with the above.

    1. Adaptive nodes. Better support for resource restricted devices and nodes of varying capabilities. Light connection strategy for resources and guarantees. Security games to outsource processing with guarantees.

    2. Incentivized and spam-resistant messaging. Reasons to run infrastructure and not relying on altruistic nodes. For spam resistance, in p2p multicast spam is a big attack vector due to amplification. There are a few interesting directions here, such as EigenTrust, proof of burn with micropayments, and leveraging zero-knowledge proofs.

    3. Strong privacy guarantees at transport privacy layer. More rigorous privacy guarantees and explicit trade-offs for metadata protection. Includes Mixnet.

    4. Censorship-resistant and robust P2P overlay. NAT traversal; running in the browser; mesh networks; pluggable transports for traffic obfuscation.

    5. Scalable and decentralized secure conversational security. Strong security guarantees such as forward secrecy, post compromise security, for large group chats. Includes projects such MLS and extending Double Ratchet.

    6. Better trust establishment and key handling. Avoiding MITM attacks while still enabling a good user experience. Protecting against ghost users in group chat and providing better ways to do key handling.

    There is also a set of more general problems, that touch multiple layers:

    1. Ensuring modularity and interoperability. Providing interfaces that allow for existing and new protocols to be at each layer of the stack.

    2. Better specifications. Machine-readable and formally verified specifications. More rigorous analysis of exact guarantees and behaviors. Exposing work in such a way that it can be analyzed by academics.

    3. Better simulations. Providing infrastructure and tooling to be able to test protocols in adverse environments and at scale.

    4. Enabling excellent user experience. A big reason for the lack of widespread adoption of secure messaging is the fact that more centralized, insecure methods provide a better user experience. Given that incentives can align better for users interested in secure messaging, providing an even better user experience should be doable.

    We got some work to do. Come help us if you want. See you in the next update!

    ]]>     - - Oskar - -     - - <![CDATA[P2P Data Sync for Mobile]]> - https://vac.dev/rlog/p2p-data-sync-for-mobile - - 2019-07-19T12:00:00.000Z - - A research log. Reliable and decentralized, pick two.

    Together with decanus, I've been working on the problem of data sync lately.

    In building p2p messaging systems, one problem you quickly come across is the problem of reliably transmitting data. If there's no central server with high availability guarantees, you can't meaningfully guarantee that data has been transmitted. One way of solving this problem is through a synchronization protocol.

    There are many synchronization protocols out there and I won't go into detail of how they differ with our approach here. Some common examples are Git and Bittorrent, but there are also projects like IPFS, Swarm, Dispersy, Matrix, Briar, SSB, etc.

    Problem motivation​

    Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices.

    Why p2p?​

    For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from.

    Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat.

    Why reliably sync data?​

    At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on.

    Why mobilephones?​

    Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence.

    More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc.

    Minimal Requirements​

    For requirements or design goals for a solution, here's what we came up with.

    1. MUST sync data reliably between devices. By reliably we mean having the ability to deal with messages being out of order, dropped, duplicated, or delayed.

    2. MUST NOT rely on any centralized services for reliability. By centralized services we mean any single point of failure that isn’t one of the endpoint devices.

    3. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices that are resource restricted, mostly-offline and often changing network.

    4. MAY use helper services in order to be more mobile-friendly. Examples of helper services are decentralized file storage solutions such as IPFS and Swarm. These help with availability and latency of data for mostly-offline devices.

    5. MUST have the ability to provide casual consistency. By casual consistency we mean the commonly accepted definition in distributed systems literature. This means messages that are casually related can achieve a partial ordering.

    6. MUST support ephemeral messages that don’t need replication. That is, allow for messages that don’t need to be reliabily transmitted but still needs to be transmitted between devices.

    7. MUST allow for privacy-preserving messages and extreme data loss. By privacy-preserving we mean things such as exploding messages (self-destructing messages). By extreme data loss we mean the ability for two trusted devices to recover from a, deliberate or accidental, removal of data.

    8. MUST be agnostic to whatever transport it is running on. It should not rely on specific semantics of the transport it is running on, nor be tightly coupled with it. This means a transport can be swapped out without loss of reliability between devices.

    MVDS - a minimium viable version​

    The first minimum viable version is in an alpha stage, and it has a specification, implementation and we have deployed it in a console client for end to end functionality. It's heavily inspired by Bramble Sync Protocol.

    The spec is fairly minimal. You have nodes that exchange records over some secure transport. These records are of different types, such as OFFER, MESSAGE, REQUEST, and ACK. A peer keep tracks of the state of message for each node it is interacting with. There's also logic for message retransmission with exponential delay. The positive ACK and retransmission model is quite similar to how TCP is designed.

    There are two different modes of syncing, interactive and batch mode. See sequence diagrams below.

    Interactive mode: -

    Interactive mode

    Batch mode: -

    Batch mode

    Which mode should you choose? It's a tradeoff of latency and bandwidth. If you want to minimize latency, batch mode is better. If you care about preserving bandwidth interactive mode is better. The choice is up to each node.

    Basic simulation​

    Initial ad hoc bandwidth and latency testing shows some issues with a naive approach. Running with the default simulation settings:

    • communicating nodes: 2
    • nodes using interactive mode: 2
    • interval between messages: 5s
    • time node is offine: 90%
    • nodes each node is sharing with: 2

    we notice a huge overhead. More specifically, we see a ~5 minute latency overhead and a bandwidth multiplier of x100-1000, i.e. 2-3 orders of magnitude just for receiving a message with interactive mode, without acks.

    Now, that seems terrible. A moment of reflection will reveal why that is. If each node is offline uniformly 90% of the time, that means that each record will be lost 90% of the time. Since interactive mode requires offer, request, payload (and then ack), that's three links just for Bob to receive the actual message.

    Each failed attempt implies another retransmission. That means we have (1/0.1)^3 = 1000 expected overhead to receive a message in interactive mode. The latency follows naturally from that, with the retransmission logic.

    Mostly-offline devices​

    The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost.

    There are a few ways of getting around this issue. One is having a store and forward model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status.

    Another, arguably more pure and robust, way is having a remote log, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS.

    What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are required to get reliable data transmission.

    Basic calculations for bandwidth multiplier​

    While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions:

    • two nodes exchanging a single message in batch mode
    • 10% uniformly random uptime for each node
    • in HA cache case, 100% uptime of a piece of infrastructure C
    • retransmission every epoch (with constant or exponential backoff)
    • only looking at average (p50) case

    First case, no helper services​

    A sends a message to B, and B acks it.

    A message -> B (10% chance of arrival)
    A   <- ack   B (10% chance of arrival)

    With a constant backoff, A will send messages at epoch 1, 2, 3, .... With exponential backoff and a multiplier of 2, this would be 1, 2, 4, 8, .... Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier.

    There's a difference between time to receive and time to stop sending. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore:

    1. A will send messages until it receives an ACK.
    2. B will send ACK if it receives a message.

    To get an average of one ack through, A needs to send 100 messages, and B send on average 10 acks. That's a multiplier of roughly a 100. That's roughly what we saw with the simulation above for receiving a message in interactive mode.

    Second case, high-availability caching layer​

    Let's introduce a helper node or piece of infrastructure, C. Whenever A or B sends a message, it also sends it to C. Whenever A or B comes online, it queries for messages with C.

    A message    -> B (10% chance of arrival)
    A message    -> C (100% chance of arrival)
    B <- req/res -> C (100% chance of arrival)
    A   <- ack      B (10% chance of arrival)
    C   <- ack      B (100% chance of arrival)
    A <- req/res -> C (100% chance of arrival)

    What's the probability that A's messages will arrive at B? Directly, it's still 10%. But we can assume it's 100% that C picks up the message. (Giving C a 90% chance success rate doesn't materially change the numbers).

    B will pick up A's message from C after an average of 10 epochs. Then B will send ack to A, which will also be picked up by C 100% of the time. Once A comes online again, it'll query C and receive B's ack.

    Assuming we use exponential backoff with a multiplier of 2, A will send a message directly to B at epoch 1, 2, 4, 8 (assuming it is online). At this point, epoch 10, B will be online in the average case. These direct sends will likely fail, but B will pick the message up from C and send one ack, both directly to A and to be picked up by C. Once A comes online, it'll query C and receive the ack from B, which means it won't do any more retransmits.

    How many messages have been sent? Not counting interactions with C, A sends 4 (at most) and B 1. Depending on if the interaction with C is direct or indirect (i.e. multicast), the factor for interaction with C will be ~2. This means the total bandwidth multiplier is likely to be <10, which is a lot more acceptable.

    Since the syncing semantics are end-to-end, this is without relying on the reliablity of C.

    Caveat​

    Note that both of these are probabilistic argument. They are also based on heuristics. More formal analysis would be desirable, as well as better simulations to experimentally verify them. In fact, the calculations could very well be wrong!

    Future work​

    There are many enhancements that can be made and are desirable. Let's outline a few.

    1. Data sync clients. Examples of actual usage of data sync, with more interesting domain semantics. This also includes usage of sequence numbers and DAGs to know what content is missing and ought to be synced.

    2. Remote log. As alluded to above, this is necessary. It needs a more clear specification and solid proof of concepts.

    3. More efficient ways of syncing with large number of nodes. When the number of nodes goes up, the algorithmic complexity doesn't look great. This also touches on things such as ambient content discovery.

    4. More robust simulations and real-world deployments. Exisiting simulation is ad hoc, and there are many improvements that can be made to gain more confidence and identify issues. Additionally, better formal analysis.

    5. Example usage over multiple transports. Including things like sneakernet and meshnets. The described protocol is designed to work over unstructured, structured and private p2p networks. In some cases it can leverage differences in topology, such as multicast, or direct connections.

    ]]>     - - Oskar - -     \ No newline at end of file diff --git a/rlog/authors/index.html b/rlog/authors/index.html new file mode 100644 index 00000000..56d308be --- /dev/null +++ b/rlog/authors/index.html @@ -0,0 +1,23 @@ + + + + + +Authors | Vac Research + + + + + + + + + + +

    Authors

    • Circe

      1

    • Dean

      3

    • Franck

      3

    • Hanno Cornelius

      2

    • Daniel

      2

    • Oskar

      12

    • Richard

      1

    • s1fr0

      1

    • Sanaz

      1

    • Moudy

      4

    • Aaryamann

      3

    • Umar Farooq

      3

    • Marvin

      3

    • Aleksei Vambol

      1

    • Vac

      1
    + + \ No newline at end of file diff --git a/rlog/building-privacy-protecting-infrastructure/index.html b/rlog/building-privacy-protecting-infrastructure/index.html index 969faeba..eb6462eb 100644 --- a/rlog/building-privacy-protecting-infrastructure/index.html +++ b/rlog/building-privacy-protecting-infrastructure/index.html @@ -1,220 +1,315 @@ - + - -Building Privacy-Protecting Infrastructure | Vac Research + +Building Privacy-Protecting Infrastructure | Vac Research - - - + + + -
    -

    Building Privacy-Protecting Infrastructure

    by
    19 min read

    What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    This write-up is based on a talk given at DevCon 6 in Bogota, a video can be found here

    Intro​

    In this write-up, we are going to talk about building privacy-protecting -infrastructure. What is it, why do we need it and how can we build it?

    We'll look at Waku, the communication layer for Web3. We'll look at how we are + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Building Privacy-Protecting Infrastructure

    by
    19 min read

    What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    + +

    This write-up is based on a talk given at DevCon 6 in Bogota, a video can be found here

    +

    Intro​

    +

    In this write-up, we are going to talk about building privacy-protecting +infrastructure. What is it, why do we need it and how can we build it?

    +

    We'll look at Waku, the communication layer for Web3. We'll look at how we are using Zero Knowledge (ZK) technology to incentivize and protect the Waku network. We'll also look at Zerokit, a library we are writing to make ZKP easier -to use in different environments.

    At the end of this write-up, I hope you'll come away with an understanding of -the importance of privacy-protecting infrastructure and how we can build it.

    About​

    First, briefly about Vac. We build public good protocols for the decentralized +to use in different environments.

    +

    At the end of this write-up, I hope you'll come away with an understanding of +the importance of privacy-protecting infrastructure and how we can build it.

    +

    About​

    +

    First, briefly about Vac. We build public good protocols for the decentralized web, with a focus on privacy and communication. We do applied research based on which we build protocols, libraries and publications. We are also the custodians -of protocols that reflect a set of principles.

    Principles

    It has its origins in the Status app and trying to improve +of protocols that reflect a set of principles.

    +

    Principles

    +

    It has its origins in the Status app and trying to improve the underlying protocols and infrastructure. We build Waku, -among other things.

    Why build privacy-protecting infrastructure?​

    Privacy is the power to selectively reveal yourself. It is a requirement for -freedom and self-determination.

    Just like you need decentralization in order to get censorship-resistance, you -need privacy to enable freedom of expression.

    To build applications that are decentralized and privacy-protecting, you need -the base layer, the infrastructure itself, to have those properties.

    We see this a lot. It is easier to make trade-offs at the application layer than +among other things.

    +

    Why build privacy-protecting infrastructure?​

    +

    Privacy is the power to selectively reveal yourself. It is a requirement for +freedom and self-determination.

    +

    Just like you need decentralization in order to get censorship-resistance, you +need privacy to enable freedom of expression.

    +

    To build applications that are decentralized and privacy-protecting, you need +the base layer, the infrastructure itself, to have those properties.

    +

    We see this a lot. It is easier to make trade-offs at the application layer than doing them at the base layer. You can build custodial solutions on top of a decentralized and non-custodial network where participants control their own -keys, but you can't do the opposite.

    If you think about it, buildings can be seen as a form of privacy-protecting +keys, but you can't do the opposite.

    +

    If you think about it, buildings can be seen as a form of privacy-protecting infrastructure. It is completely normal and obvious in many ways, but when it comes to the digital realm our mental models and way of speaking about it hasn't -caught up yet for most people.

    I'm not going too much more into the need for privacy or what happens when you +caught up yet for most people.

    +

    I'm not going too much more into the need for privacy or what happens when you don't have it, but suffice to say it is an important property for any open -society.

    When we have conversations, true peer-to-peer offline conversations, we can talk -privately. If we use cash to buy things we can do commerce privately.

    On the Internet, great as it is, there are a lot of forces that makes this +society.

    +

    When we have conversations, true peer-to-peer offline conversations, we can talk +privately. If we use cash to buy things we can do commerce privately.

    +

    On the Internet, great as it is, there are a lot of forces that makes this natural state of things not the default. Big Tech has turned users into a commodity, a product, and monetized user's attention for advertising. To optimize for your attention they need to surveil your habits and activities, and hence breach your privacy. As opposed to more old-fashioned models, where someone is buying a useful service from a company and the incentives are more -aligned.

    We need to build credibly neutral infrastructure that protects your privacy at +aligned.

    +

    We need to build credibly neutral infrastructure that protects your privacy at the base layer, in order to truly enable applications that are -censorship-resistant and encourage meaningful freedom of expression.

    Web3 infrastructure​

    Infrastructure is what lies underneath. Many ways of looking at this but I'll +censorship-resistant and encourage meaningful freedom of expression.

    +

    Web3 infrastructure​

    +

    Infrastructure is what lies underneath. Many ways of looking at this but I'll keep it simple as per the original Web3 vision. You had Ethereum for compute/consensus, Swarm for storage, and Whisper for messaging. Waku has taken over the mantle from Whisper and is a lot more usable today than Whisper ever was, -for many reasons.

    Web3 Infrastructure

    On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, +for many reasons.

    +

    Web3 Infrastructure

    +

    On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, especially when you try to add privacy back "on top". It takes a lot of effort and it is easier to censor. We see this with recent action around Tornado Cash. Compare this with something like Zcash or Monero, where privacy is there by -default.

    There are also problems when it comes to the p2p networking side of things, for +default.

    +

    There are also problems when it comes to the p2p networking side of things, for example with Ethereum validator privacy and hostile actors and jurisdictions. If someone can easily find out where a certain validator is physically located, that's a problem in many parts of the world. Being able to have stronger -privacy-protection guarantees would be very useful for high-value targets.

    This doesn't begin to touch on the so called "dapps" that make a lot of +privacy-protection guarantees would be very useful for high-value targets.

    +

    This doesn't begin to touch on the so called "dapps" that make a lot of sacrifices in how they function, from the way domains work, to how websites are hosted and the reliance on centralized services for communication. We see this time and time again, where centralized, single points of failure systems work -for a while, but then eventually fail.

    In many cases an individual user might not care enough though, and for platforms +for a while, but then eventually fail.

    +

    In many cases an individual user might not care enough though, and for platforms the lure to take shortcuts is strong. That is why it is important to be principled, but also pragmatic in terms of the trade-offs that you allow on top. -We'll touch more on this in the design goals around modularity that Waku has.

    ZK for privacy-protecting infrastructure​

    ZKPs are a wonderful new tool. Just like smart contracts enables programmable +We'll touch more on this in the design goals around modularity that Waku has.

    +

    ZK for privacy-protecting infrastructure​

    +

    ZKPs are a wonderful new tool. Just like smart contracts enables programmable money, ZKPs allow us to express fundamentally new things. In line with the great tradition of trust-minimization, we can prove statement while revealing the absolute minimum information necessary. This fits the definition of privacy, the power to selectively reveal yourself, perfectly. I'm sure I don't need to tell anyone reading this but this is truly revolutionary. The technology is advancing -extremely fast and often it is our imagination that is the limit.

    Zero knowledge

    Waku​

    What is Waku? It is a set of modular protocols for p2p communication. It has a +extremely fast and often it is our imagination that is the limit.

    +

    Zero knowledge

    +

    Waku​

    +

    What is Waku? It is a set of modular protocols for p2p communication. It has a focus on privacy, security and being able to run anywhere. It is the spiritual -success to Whisper.

    By modular we mean that you can pick and choose protocols and how you use them +success to Whisper.

    +

    By modular we mean that you can pick and choose protocols and how you use them depending on constraints and trade-offs. For example, bandwidth usage vs -privacy.

    It is designed to work in resource restricted environments, such as mobile +privacy.

    +

    It is designed to work in resource restricted environments, such as mobile phones and in web browsers. It is important that infrastructure meets users where they are and supports their real-world use cases. Just like you don't need your own army and a castle to have your own private bathroom, you shouldn't need to have a powerful always-on node to get reasonable privacy and -censorship-resistance. We might call this self-sovereignty.

    Waku - adaptive nodes​

    One way of looking at Waku is as an open service network. There are nodes with +censorship-resistance. We might call this self-sovereignty.

    +

    Waku - adaptive nodes​

    +

    One way of looking at Waku is as an open service network. There are nodes with varying degrees of capabilities and requirements. For example when it comes to bandwidth usage, storage, uptime, privacy requirements, latency requirements, -and connectivity restrictions.

    We have a concept of adaptive nodes that can run a variety of protocols. A node +and connectivity restrictions.

    +

    We have a concept of adaptive nodes that can run a variety of protocols. A node operator can choose which protocols they want to run. Naturally, there'll be some nodes that do more consumption and other nodes that do more provisioning. This gives rise to the idea of a service network, where services are provided -for and consumed.

    Adaptive Nodes

    Waku - protocol interactions​

    There are many protocols that interact. Waku Relay protocol is based on libp2p +for and consumed.

    +

    Adaptive Nodes

    +

    Waku - protocol interactions​

    +

    There are many protocols that interact. Waku Relay protocol is based on libp2p GossipSub for p2p messaging. We have filter for bandwidth-restricted nodes to only receive subset of messages. Lightpush for nodes with short connection windows to push messages into network. Store for nodes that want to retrieve -historical messages.

    On the payload layer, we provide support for Noise handshakes/key-exchanges. +historical messages.

    +

    On the payload layer, we provide support for Noise handshakes/key-exchanges. This means that as a developers, you can get end-to-end encryption and expected guarantees out of the box. We have support for setting up a secure channel from scratch, and all of this paves the way for providing Signal's Double Ratchet at the protocol level much easier. We also have experimental support for multi-device usage. Similar features have existed in for example the Status app for a while, but with this we make it easier for any platform using Waku to use -it.

    There are other protocols too, related to peer discovery, topic usage, etc. See -specs for more details.

    Protocol Interactions

    Waku - Network​

    For the Waku network, there are a few problems. For example, when it comes to +it.

    +

    There are other protocols too, related to peer discovery, topic usage, etc. See +specs for more details.

    +Protocol Interactions +

    Waku - Network​

    +

    For the Waku network, there are a few problems. For example, when it comes to network spam and incentivizing service nodes. We want to address these while keeping privacy-guarantees of the base layer. I'm going to go into both of -these.

    The spam problem arises on the gossip layer when anyone can overwhelm the +these.

    +

    The spam problem arises on the gossip layer when anyone can overwhelm the network with messages. The service incentivization is a problem when nodes don't directly benefit from the provisioning of a certain service. This can happen if they are not using the protocol directly themselves as part of normal operation, or if they aren't socially inclined to provide a certain service. This depends a -lot on how an individual platform decides to use the network.

    Waku Network

    Dealing with network spam and RLN Relay​

    Since the p2p relay network is open to anyone, there is a problem with spam. If +lot on how an individual platform decides to use the network.

    +

    Waku Network

    +

    Dealing with network spam and RLN Relay​

    +

    Since the p2p relay network is open to anyone, there is a problem with spam. If we look at existing solutions for dealing with spam in traditional messaging systems, a lot of entities like Google, Facebook, Twitter, Telegram, Discord use phone number verification. While this is largely sybil-resistant, it is -centralized and not private at all.

    Historically, Whisper used PoW which isn't good for heterogenerous networks. +centralized and not private at all.

    +

    Historically, Whisper used PoW which isn't good for heterogenerous networks. Peer scoring is open to sybil attacks and doesn't directly address spam -protection in an anonymous p2p network.

    The key idea here is to use RLN for private economic spam protection using -zkSNARKs.

    I'm not going to go into too much detail of RLN here. If you are interested, I +protection in an anonymous p2p network.

    +

    The key idea here is to use RLN for private economic spam protection using +zkSNARKs.

    +

    I'm not going to go into too much detail of RLN here. If you are interested, I gave a talk in Amsterdam at Devconnect about this. We have some write-ups on RLN here by Sanaz who has been pushing a lot of this from our side. There's also another talk at Devcon by Tyler going into RLN in -more detail. Finally, here's the RLN spec.

    I'll briefly go over what it is, the interface and circuit and then talk about -how it is used in Waku.

    RLN - Overview and Flow​

    RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting +more detail. Finally, here's the RLN spec.

    +

    I'll briefly go over what it is, the interface and circuit and then talk about +how it is used in Waku.

    +

    RLN - Overview and Flow​

    +

    RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting mechanism based on zkSNARKs. By rate limiting we mean you can only send N messages in a given period. By anonymity we mean that you can't link message to a publisher. We can think of it as a voting booth, where you are only allowed to -vote once every election.

    Voting Booth

    It can be used for spam protection in p2p messaging systems, and also rate -limiting in general, such as for a decentralized captcha.

    There are three parts to it. You register somewhere, then you can signal and +vote once every election.

    +

    Voting Booth

    +

    It can be used for spam protection in p2p messaging systems, and also rate +limiting in general, such as for a decentralized captcha.

    +

    There are three parts to it. You register somewhere, then you can signal and finally there's a verification/slashing phase. You put some capital at risk, -either economic or social, and if you double signal you get slashed.

    RLN - Circuit​

    Here's what the private and public inputs to the circuit look like. The identity +either economic or social, and if you double signal you get slashed.

    +

    RLN - Circuit​

    +

    Here's what the private and public inputs to the circuit look like. The identity secret is generated locally, and we create an identity commitment that is inserted into a Merkle tree. We then use Merkle proofs to prove membership. Registered member can only signal once for a given epoch or external nullifier, for example every ten seconds in Unix time. RLN identifer is for a specific RLN -app.

    We also see what the circuit output looks like. This is calculated locally. y +app.

    +

    We also see what the circuit output looks like. This is calculated locally. y is a share of the secret equation, and the (internal) nullifier acts as a unique fingerprint for a given app/user/epoch combination. How do we calculate y and -the internal nullifier?

    // Private input
    signal input identity_secret;
    signal input path_elements[n_levels][1];
    signal input identity_path_index[n_levels];

    // Public input
    signal input x; // signal_hash
    signal input epoch; // external_nullifier
    signal input rln_identifier;

    // Circuit output
    signal output y;
    signal output root;
    signal output nullifier;

    RLN - Shamir's secret sharing​

    This is done using Shamir's secret +the internal nullifier?

    +
    // Private input
    signal input identity_secret;
    signal input path_elements[n_levels][1];
    signal input identity_path_index[n_levels];

    // Public input
    signal input x; // signal_hash
    signal input epoch; // external_nullifier
    signal input rln_identifier;

    // Circuit output
    signal output y;
    signal output root;
    signal output nullifier;
    +

    RLN - Shamir's secret sharing​

    +

    This is done using Shamir's secret sharing. Shamir’s secret sharing is based on idea of splitting a secret into shares. This is how -we enable slashing of funds.

    In this case, we have two shares. If a given identity a0 signals twice in +we enable slashing of funds.

    +

    In this case, we have two shares. If a given identity a0 signals twice in epoch/external nullifier, a1 is the same. For a given RLN app, internal_nullifier then stays the same. x is signal hash which is different, and y is public, so we can reconstruct identity_secret. With the identity -secret revealed, this gives access to e.g. financial stake.

    a_0 = identity_secret // secret S
    a_1 = poseidonHash([a0, external_nullifier])

    y = a_0 + x * a_1

    internal_nullifier = poseidonHash([a_1, rln_identifier])

    Shamir&#39;s secret sharing

    RLN Relay​

    This is how RLN is used with Relay/GossipSub protocol. A node registers and +secret revealed, this gives access to e.g. financial stake.

    +
    a_0 = identity_secret // secret S
    a_1 = poseidonHash([a0, external_nullifier])

    y = a_0 + x * a_1

    internal_nullifier = poseidonHash([a_1, rln_identifier])
    +

    Shamir&#39;s secret sharing

    +

    RLN Relay​

    +

    This is how RLN is used with Relay/GossipSub protocol. A node registers and locks up funds, and after that it can send messages. It publishes a message -containing the Zero Knowledge proof and some other details.

    Each relayer node listens to the membership contract for new members, and it +containing the Zero Knowledge proof and some other details.

    +

    Each relayer node listens to the membership contract for new members, and it also keeps track of relevant metadata and merkle tree. Metadata is needed to be -able to detect double signaling and perform slashing.

    Before forwarding a message, it does some verification checks to ensure there +able to detect double signaling and perform slashing.

    +

    Before forwarding a message, it does some verification checks to ensure there are no duplicate messages, ZKP is valid and no double signaling has occured. It is worth noting that this can be combined with peer scoring, for example for -duplicate messages or invalid ZK proofs.

    In line of Waku's goals of modularity, RLN Relay is applied on a specific subset -of pubsub and content topics. You can think of it as an extra secure channel.

    RLN Relay

    RLN Relay cross-client testnet​

    Where are we with RLN Relay deployment? We've recently launched our second +duplicate messages or invalid ZK proofs.

    +

    In line of Waku's goals of modularity, RLN Relay is applied on a specific subset +of pubsub and content topics. You can think of it as an extra secure channel.

    +

    RLN Relay

    +

    RLN Relay cross-client testnet​

    +

    Where are we with RLN Relay deployment? We've recently launched our second testnet. This is using RLN Relay with a smart contract on Goerli. It integrates with our example p2p chat application, and it does so through three different clients, nwaku, go-waku and js-waku for browsers. This is our first p2p -cross-client testnet for RLN Relay.

    Here's a video that shows a user +cross-client testnet for RLN Relay.

    +

    Here's a video that shows a user registering in a browser, signaling through JS-Waku. It then gets relayed to a nwaku node, that verifies the proof. The second video shows what happens in the spam case. when more than one message is sent in a given epoch, it detects it as spam and discards it. Slashing hasn't been implemented fully yet in the client -and is a work in progress.

    If you are curious and want to participate, you can join the effort on our Vac +and is a work in progress.

    +

    If you are curious and want to participate, you can join the effort on our Vac Discord. We also have tutorials -setup for all clients so you can play around with it.

    As part of this, and to make it work in multiple different environments, we've +setup for all clients so you can play around with it.

    +

    As part of this, and to make it work in multiple different environments, we've also been developing a new library called Zerokit. I'll talk about this a bit -later.

    Private settlement / Service credentials​

    Going back to the service network idea, let's talk about service credentials. +later.

    +

    Private settlement / Service credentials​

    +

    Going back to the service network idea, let's talk about service credentials. The idea behind service credentials and private settlement is to enable two actors to pay for and provide services without compromising their privacy. We do not want the payment to create a direct public link between the service provider -and requester.

    Recall the Waku service network illustration with adaptive nodes that choose +and requester.

    +

    Recall the Waku service network illustration with adaptive nodes that choose which protocols they want to run. Many of these protocols aren't very heavy and just work by default. For example the relay protocol is enabled by default. Other protocols are much heavier to provide, such as storing historical -messages.

    It is desirable to have additional incentives for this, especially for platforms +messages.

    +

    It is desirable to have additional incentives for this, especially for platforms that aren't community-based where some level of altruism can be assumed (e.g. -Status Communities, or WalletConnect cloud infrastructure).

    You have a node Alice that is often offline and wants to consume historical +Status Communities, or WalletConnect cloud infrastructure).

    +

    You have a node Alice that is often offline and wants to consume historical messages on some specific content topics. You have another node Bob that runs a server at home where they store historical messages for the last several weeks. Bob is happy to provide this service for free because he's excited about running privacy-preserving infrastructure and he's using it himself, but his node is getting overwhelmed by freeloaders and he feels like he should be paid something -for continuing to provide this service.

    Alice deposits some funds in a smart contract which registers it in a tree, +for continuing to provide this service.

    +

    Alice deposits some funds in a smart contract which registers it in a tree, similar to certain other private settlement mechanisms. A fee is taken or burned. In exchange, she gets a set of tokens or service credentials. When she wants to do a query with some criteria, she sends this to Bob. Bob responds with size of response, cost, and receiver address. Alice then sends a proof of delegation of a service token as a payment. Bob verifies the proof and resolves -the query.

    The end result is that Alice has consumed some service from Bob, and Bob has +the query.

    +

    The end result is that Alice has consumed some service from Bob, and Bob has received payment for this. There's no direct transaction link between Alice and Bob, and gas fees can be minimized by extending the period before settling on -chain.

    This can be complemented with altruistic service provisioning, for example by +chain.

    +

    This can be complemented with altruistic service provisioning, for example by splitting the peer pool into two slots, or only providing a few cheap queries -for free.

    The service provisioning is general, and can be generalized for any kind of -request/response service provisoning that we want to keep private.

    This isn't a perfect solution, but it is an incremental improvement on top of +for free.

    +

    The service provisioning is general, and can be generalized for any kind of +request/response service provisoning that we want to keep private.

    +

    This isn't a perfect solution, but it is an incremental improvement on top of the status quo. It can be augmented with more advanced techniques such as better -non-repudiable node reputation, proof of correct service provisioning, etc.

    We are currently in the raw spec / proof of concept stage of this. We expect to -launch a testnet of this later this year or early next year.

    Service credentials flow

    Zerokit​

    Zerokit is a set of Zero Knowledge modules, +non-repudiable node reputation, proof of correct service provisioning, etc.

    +

    We are currently in the raw spec / proof of concept stage of this. We expect to +launch a testnet of this later this year or early next year.

    +

    Service credentials flow

    +

    Zerokit​

    +

    Zerokit is a set of Zero Knowledge modules, written in Rust and designed to be used in many different environments. The initial goal is to get the best of both worlds with Circom/Solidity/JS and Rust/ZK ecosystem. This enables people to leverage Circom-based constructs from -non-JS environments.

    For the RLN module, it is using Circom circuits via ark-circom and Rust for +non-JS environments.

    +

    For the RLN module, it is using Circom circuits via ark-circom and Rust for scaffolding. It exposes a C FFI API that can be used through other system programming environments, like Nim and Go. It also exposes an experimental WASM -API that can be used through web browsers.

    Waku is p2p infrastructure running in many different environments, such as -Nim/JS/Go/Rust, so this a requirement for us.

    Circom and JS strengths are access to Dapp developers, tooling, generating +API that can be used through web browsers.

    +

    Waku is p2p infrastructure running in many different environments, such as +Nim/JS/Go/Rust, so this a requirement for us.

    +

    Circom and JS strengths are access to Dapp developers, tooling, generating verification code, circuits etc. Rust strengths is that it is systems-based and easy to interface with other language runtime such as Nim, Go, Rust, C. It also gives access to other Rust ZK ecosystems such as arkworks. This opens door for using other constructs, such as Halo2. This becomes especially relevant for constructs where you don't want to do a trusted setup or where circuits are more -complex/custom and performance requirements are higher.

    In general with Zerokit, we want to make it easy to build and use ZKP in a +complex/custom and performance requirements are higher.

    +

    In general with Zerokit, we want to make it easy to build and use ZKP in a multitude of environments, such as mobile phones and web browsers. Currently it is too complex to write privacy-protecting infrastructure with ZKPs considering all the languages and tools you have to learn, from JS, Solidity and Circom to @@ -222,23 +317,29 @@ Rust, WASM and FFI. And that isn't even touching on things like secure key storage or mobile dev. Luckily more and more projects are working on this, including writing DSLs etc. It'd also be exciting if we can make a useful toolstack for JS-less ZK dev to reduce cognitive overhead, similar to what we -have with something like Foundry.

    Other research​

    I also want to mention a few other things we are doing. One thing is +have with something like Foundry.

    +

    Other research​

    +

    I also want to mention a few other things we are doing. One thing is protocol specifications. We think this is very important for p2p infra, and we see a lot of other projects that claim to do it p2p infrastructure but they aren't clear about guarantees or how stable something is. That makes it hard to have multiple implementations, to collaborate across -different projects, and to analyze things objectively.

    Related to that is publishing papers. We've put +different projects, and to analyze things objectively.

    +

    Related to that is publishing papers. We've put out three so far, related to Waku and RLN-Relay. This makes it easier to interface with academia. There's a lot of good researchers out there and we want -to build a better bridge between academia and industry.

    Another thing is network +to build a better bridge between academia and industry.

    +

    Another thing is network privacy. Waku is modular with respect to privacy guarantees, and there are a lot of knobs to turn here depending on specific deployments. For example, if you are running the full relay protocol you currently have much stronger receiver anonymity than if you -are running filter protocol from a bandwidth or connectivity-restricted node.

    We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym +are running filter protocol from a bandwidth or connectivity-restricted node.

    +

    We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym come with some trade-offs but are a useful tool in the arsenal. A good mental model to keep in mind is the anonymity trilemma, where you can only pick 2/3 out -of low latency, low bandwidth usage and strong anonymity.

    We are currently exploring Dandelion-like +of low latency, low bandwidth usage and strong anonymity.

    +

    We are currently exploring Dandelion-like additions to the relay/gossip protocol, which would provide for stronger sender anonymity, especially in a multi-node/botnet attacker model. As part of this we are looking into different @@ -246,22 +347,26 @@ parameters choices and general possibilities for lower latency usage. This could make it more amenable for latency sensitive environments, such as validator privacy, for specific threat models. The general theme here is we want to be rigorous with the guarantees we provide, under what conditions and for what -threat models.

    Another thing mentioned earlier is Noise payload +threat models.

    +

    Another thing mentioned earlier is Noise payload encryption, and specifically things like allowing for pairing different devices with e.g. QR codes. This makes it easier for developers to provide secure messaging in many realistic scenarios in a -multi-device world.

    Other research

    Summary​

    We've gone over what privacy-protecting infrastructure is, why we want it and +multi-device world.

    +

    Other research

    +

    Summary​

    +

    We've gone over what privacy-protecting infrastructure is, why we want it and how we can build it. We've seen how ZK is a fundamental building block for this. We've looked at Waku, the communication layer for Web3, and how it uses Zero Knowledge proofs to stay private and function better. We've also looked at -Zerokit and how we can make it easier to do ZKP in different environments.

    Finally we also looked at some other research we've been doing. All of the +Zerokit and how we can make it easier to do ZKP in different environments.

    +

    Finally we also looked at some other research we've been doing. All of the things mentioned in this article, and more, is available as write-ups, specs, or -discussions on our forum or Github.

    If you find any of this exciting to work on, feel free to reach out on our +discussions on our forum or Github.

    +

    If you find any of this exciting to work on, feel free to reach out on our Discord. We are also hiring, and we have started expanding into other privacy infrastructure tech like private and provable -computation with ZK-WASM.

    - - +computation with ZK-WASM.

    \ No newline at end of file diff --git a/rlog/climbing-merkle-trees/index.html b/rlog/climbing-merkle-trees/index.html index 5ab9e0e4..f23b2208 100644 --- a/rlog/climbing-merkle-trees/index.html +++ b/rlog/climbing-merkle-trees/index.html @@ -1,113 +1,208 @@ - + - -Vac 101: Climbing Merkle Trees | Vac Research + +Vac 101: Climbing Merkle Trees | Vac Research - - - - + + + + -
    -

    Vac 101: Climbing Merkle Trees

    by
    10 min read

    In this post, we introduce a crucial data structure used throughout web3.

    Introduction​

    A large amount of data is swapped between users on a blockchain in the form of transactions. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Vac 101: Climbing Merkle Trees

    by
    10 min read

    In this post, we introduce a crucial data structure used throughout web3.

    + +

    Introduction​

    +

    A large amount of data is swapped between users on a blockchain in the form of transactions. Over the entire life of a blockchain, the storage space required to maintain a copy of every transaction becomes untenable for most users. However, the integrity of a blockchain relies on a large pool of users that can validate the blockchain's history from its inception to its present state. The data representing the blockchain's state is compressed. -This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users.

    Data compression alone is not the end goal. +This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users.

    +

    Data compression alone is not the end goal. As mentioned, it is essential for users to be able to validate the blockchain's history. The property of compression and validation was solved in Bitcoin by the use of Merkle trees. Merkle trees were introduced first by Ralph Merkle in his dissertation [1]. A Merkle tree is a data structure that compresses a digest of data to a constant size while still providing a method for proving membership of elements of the digest. -A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.

    Tree structure​

    A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. +A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.

    +

    Tree structure​

    +

    A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. The trees that we consider can be arranged in layers with multiple nodes (children) merged into a single node (parent) in the preceding layer. A single node exists in the base layer; this special node is called the root node. -The highest level of the tree consists of childless nodes called leaves.

    Image 1

    A binary tree has one additional property: +The highest level of the tree consists of childless nodes called leaves.

    +

    Image 1

    +

    A binary tree has one additional property: each nonleaf node has exactly two children nodes. That is, we assume that nodes in a binary tree are either a parent node with two children or a leaf. -As strange as it sounds, each child node has exactly one parental node.

    Image 2

    A binary tree with 2n2^n leaves consists of n+1n+1 layers. -Additionally, such a tree has 2n+1−12^{n+1}-1 nodes.

    Merkle trees​

    A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. +As strange as it sounds, each child node has exactly one parental node.

    +

    Image 2

    +

    A binary tree with 2n2^n leaves consists of n+1n+1 layers. +Additionally, such a tree has 2n+1−12^{n+1}-1 nodes.

    +

    Merkle trees​

    +

    A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. Merkle trees are usually taken to have a binary tree structure. -As such, the presentation we provide in this section will be for binary trees.

    Construction​

    In this section, we show how Merkle trees are constructed to compress a digest DD. -Suppose that the digest DD consists of 2n2^n entries; -we assume that the digest DD has this many entries since a Merkle tree is a binary tree. -Additionally, each digest can be padded to ensure that DD has the desired number of entries.

    Each leaf of the Merkle tree contains the hash of a digest entry. +As such, the presentation we provide in this section will be for binary trees.

    +

    Construction​

    +

    In this section, we show how Merkle trees are constructed to compress a digest DD. +Suppose that the digest DD consists of 2n2^n entries; +we assume that the digest DD has this many entries since a Merkle tree is a binary tree. +Additionally, each digest can be padded to ensure that DD has the desired number of entries.

    +

    Each leaf of the Merkle tree contains the hash of a digest entry. Each parent node contains the hash of the concatenation of their child nodes. Through this iterative construction, we reach the root of the tree. The value contained in the root node is called the root hash. -The root hash is a compressed representation of the digest DD.

    Image 3

    Each node in the Merkle tree is computed by taking a hash. -Since a binary tree with 2n2^n leaves has 2n+1−12^{n+1}-1 nodes, -then we need to evaluate 2n+1−12^{n+1}-1 hashes to construct the Merkle tree.

    Merkle tree intregrity​

    A large quantity of data can be compressed to a single hash value. +The root hash is a compressed representation of the digest DD.

    +

    Image 3

    +

    Each node in the Merkle tree is computed by taking a hash. +Since a binary tree with 2n2^n leaves has 2n+1−12^{n+1}-1 nodes, +then we need to evaluate 2n+1−12^{n+1}-1 hashes to construct the Merkle tree.

    +

    Merkle tree intregrity​

    +

    A large quantity of data can be compressed to a single hash value. A natural question to ask is: could a clever party find another digest that yields a Merkle tree with the same root hash? If possible, this would compromise the ledger since the blockchain's history could be altered. Fortunately, Merkle trees are quite secure. -In fact, Merkle trees can be used to both bind and hide a digest.

    The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). -A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value.

    This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on.

    In certain applications, +In fact, Merkle trees can be used to both bind and hide a digest.

    +

    The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). +A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value.

    +

    This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on.

    +

    In certain applications, it may be desirable for the digest of a Merkle tree to be kept confidential. This is achieved with the preimage resistant property of hash functions. A hash function is preimage resistant provided that it is difficult to reverse the hashing operation. -It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest.

    Now, we see that Merkle trees are secured structures that are tamper resistant.

    Proof of membership​

    An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. -This can be done with logarithmic storage and logarithmic computation time.

    Suppose that we want to show that data ℓ\ell is part of the Merkle tree's digest. -Additionally, suppose that hash\mathsf{hash} is the hash function used to construct the tree. -We assume that the hash function hash\mathsf{hash} can be computed in constant-time for any input.

    Suppose that a prover provides data ℓ\ell to a verifier, -and tells the verifier that ℓ\ell corresponds to the iith leaf of the Merkle tree. -For the verifier to be convinced that ℓ\ell is part of the digest, he needs to be able to construct the tree's root hash using hash\mathsf{hash}, ii, ℓ\ell and some additional information from the prover. +It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest.

    +

    Now, we see that Merkle trees are secured structures that are tamper resistant.

    +

    Proof of membership​

    +

    An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. +This can be done with logarithmic storage and logarithmic computation time.

    +

    Suppose that we want to show that data ℓ\ell is part of the Merkle tree's digest. +Additionally, suppose that hash\mathsf{hash} is the hash function used to construct the tree. +We assume that the hash function hash\mathsf{hash} can be computed in constant-time for any input.

    +

    Suppose that a prover provides data ℓ\ell to a verifier, +and tells the verifier that ℓ\ell corresponds to the iith leaf of the Merkle tree. +For the verifier to be convinced that ℓ\ell is part of the digest, he needs to be able to construct the tree's root hash using hash\mathsf{hash}, ii, ℓ\ell and some additional information from the prover. Specifically, the prover must provide the sibling hashes for each value that the verifier can compute. This enables the verifier to compute the parents of the siblings that the prover provides and the values that he was able to produce himself. -The last of the computed parents is the root.

    The leaf index ii indicates whether a hash value provided by the prover is a left or right sibling. -This is done by looking at the binary expansion of ii.

    The verifier can compute the leaf h0=hash(ℓ)h_0 = \mathsf{hash}(\ell). -Next, using h0h_0's sibling, h0′h'_0, provided by the prover, -the verifier can compute h1=hash(h0∥h0′)h_1 = \mathsf{hash}(h_0 \|h'_0) or h1=hash(h0′∥h0)h_1 = \mathsf{hash}(h'_0 \| h_0) -depending on whether h0′h'_0 is a left or right sibling. -This pathing continues until the verifier either successfully computes the root hash (in n+1n+1 hashes) or fails to do so.

    The prover has to provide nn sibling nodes for the proof of membership.

    There is a key detail that is essential for the proof of membership to be secure. -The root hash has to be provided to the verifier prior to the selection of data ℓ\ell. -Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership.

    Capped proof of membership​

    Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. +The last of the computed parents is the root.

    +

    The leaf index ii indicates whether a hash value provided by the prover is a left or right sibling. +This is done by looking at the binary expansion of ii.

    +

    The verifier can compute the leaf h0=hash(ℓ)h_0 = \mathsf{hash}(\ell). +Next, using h0h_0's sibling, h0′h'_0, provided by the prover, +the verifier can compute h1=hash(h0∥h0′)h_1 = \mathsf{hash}(h_0 \|h'_0) or h1=hash(h0′∥h0)h_1 = \mathsf{hash}(h'_0 \| h_0) +depending on whether h0′h'_0 is a left or right sibling. +This pathing continues until the verifier either successfully computes the root hash (in n+1n+1 hashes) or fails to do so.

    +

    The prover has to provide nn sibling nodes for the proof of membership.

    +

    There is a key detail that is essential for the proof of membership to be secure. +The root hash has to be provided to the verifier prior to the selection of data ℓ\ell. +Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership.

    +

    Capped proof of membership​

    +

    Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. A specific layer of the Merkle tree is published instead of just the root hash. -By doing this, a capped proof of membership is just the path from leaf to the published layer.

    Extensions of Merkle trees​

    Merkle trees can be extended in multiple ways. -In this section, we explore a select few of these extensions.

    Sparse Merkle trees​

    A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. +By doing this, a capped proof of membership is just the path from leaf to the published layer.

    +

    Extensions of Merkle trees​

    +

    Merkle trees can be extended in multiple ways. +In this section, we explore a select few of these extensions.

    +

    Sparse Merkle trees​

    +

    A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. Specifically, each digest entry has a particular leaf index. For simplicity, we assume that the index value is computed by taking the hash of the entry. -We note that this is a sorted SMT.

    Let nn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n leaves.

    An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\mathsf{null} marker inserted in. +We note that this is a sorted SMT.

    +

    Let nn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n leaves.

    +

    An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\mathsf{null} marker inserted in. This means that we can prove membership in the way described. -However, we can also prove nonmembership of an element by showing that null\mathsf{null} is located in the element's hash location. -The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership.

    We can take advantage of the sparse nature of SMTs to provide shortened proofs. +However, we can also prove nonmembership of an element by showing that null\mathsf{null} is located in the element's hash location. +The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership.

    +

    We can take advantage of the sparse nature of SMTs to provide shortened proofs. Specifically, it is unlikely for entries to cluster together. -Thus, it is efficient to maintain a list of values:

    Null values
    d0:=Hash(null),d_0 := \mathsf{Hash(null)},
    d1:=Hash(d0∥∥d0),d_1 := \mathsf{Hash(d_0 \|\| d_0)},
    d2:=Hash(d1∥∥d1),d_2 := \mathsf{Hash(d_1 \|\| d_1)},
    ⋮\vdots
    dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \mathsf{Hash(d_{n-2} \|\| d_{n-2})}.

    Each of the did_i's represents the root hash of a Merkle tree with 2i2^i leaves containing null\mathsf{null}. -These values can be used to shorten the time needed to construct an SMT and the length of proofs.

    Proof of nonmembership​

    In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. +Thus, it is efficient to maintain a list of values:

    +
    Null values
    d0:=Hash(null),d_0 := \mathsf{Hash(null)},
    d1:=Hash(d0∥∥d0),d_1 := \mathsf{Hash(d_0 \|\| d_0)},
    d2:=Hash(d1∥∥d1),d_2 := \mathsf{Hash(d_1 \|\| d_1)},
    ⋮\vdots
    dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \mathsf{Hash(d_{n-2} \|\| d_{n-2})}.
    +

    Each of the did_i's represents the root hash of a Merkle tree with 2i2^i leaves containing null\mathsf{null}. +These values can be used to shorten the time needed to construct an SMT and the length of proofs.

    +

    Proof of nonmembership​

    +

    In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. However, the proof of membership may result in false positives due to collisions. This would affect nonmembership proofs as well. -Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest.

    Why is sorting essential? +Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest.

    +

    Why is sorting essential? The sorting mechanism of data can be arbitrarily chosen. However, it is essential that there are no gaps in the ordering. The maximum number of elements that could ever exist in the digest must be known. A simple method for this is to use a hash function to provide fingerprints to the data. Each hash using either SHA-256 or Keccak has 256-bits. -Our entire digest could consist of a maximum of 22562^{256} entries. -This assumes that our digest does not contain collisions.

    The fingerprint of a piece of data ℓ\ell indicates which leaf of the SMT it is contained in. -This means that a nonmembership of ℓ\ell in the SMT becomes a matter of proving that null\mathsf{null} is contained in ℓ\ell's location.

    It is crucial for the SMT to be sorted. -Otherwise, a malicious party can append the entry ℓ\ell to a random location. +Our entire digest could consist of a maximum of 22562^{256} entries. +This assumes that our digest does not contain collisions.

    +

    The fingerprint of a piece of data ℓ\ell indicates which leaf of the SMT it is contained in. +This means that a nonmembership of ℓ\ell in the SMT becomes a matter of proving that null\mathsf{null} is contained in ℓ\ell's location.

    +

    It is crucial for the SMT to be sorted. +Otherwise, a malicious party can append the entry ℓ\ell to a random location. This allows for the malicious party to provide contradictory proofs that prove both membership and nonmembership. We note that the requirement that an SMT is sorted may be too strong of an assumption in centralized cases. -However, sortedness is a necessary property of SMTs for decentralized systems.

    Verkle Trees​

    A proof of membership grows in length as the Merkle tree grows. +However, sortedness is a necessary property of SMTs for decentralized systems.

    +

    Verkle Trees​

    +

    A proof of membership grows in length as the Merkle tree grows. The most obvious approach to remedy this scalability issue is to use Merkle trees in which each node has more than two children. However, this does not fix the issue. -A proof of membership in a kk-nary Merkle tree [6] (each node has kk children) has a proof size log⁡k(n)(k−1)\log_k(n)(k-1). -The multiple k−1k-1 is the number of silbings that a node has on each layer. -Hence, the proof size grows faster than a logarithmic function of the digest size.

    An alternate approach is to use a different data structure: Verkle trees [6]. +A proof of membership in a kk-nary Merkle tree [6] (each node has kk children) has a proof size log⁡k(n)(k−1)\log_k(n)(k-1). +The multiple k−1k-1 is the number of silbings that a node has on each layer. +Hence, the proof size grows faster than a logarithmic function of the digest size.

    +

    An alternate approach is to use a different data structure: Verkle trees [6]. A Verkle tree replaces hash functions with polynomial commitments [7, 8]. -We will explore Verkle trees in a future Vac 101 edition.

    References​

    - - +We will explore Verkle trees in a future Vac 101 edition.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/device-pairing-in-js-waku-and-go-waku/index.html b/rlog/device-pairing-in-js-waku-and-go-waku/index.html index 511c21e8..ad976343 100644 --- a/rlog/device-pairing-in-js-waku-and-go-waku/index.html +++ b/rlog/device-pairing-in-js-waku-and-go-waku/index.html @@ -1,26 +1,59 @@ - + - -Device Pairing in Js-waku and Go-waku | Vac Research + +Device Pairing in Js-waku and Go-waku | Vac Research - - - - + + + + -
    -

    Device Pairing in Js-waku and Go-waku

    by
    5 min read

    Device pairing and secure message exchange using Waku and noise protocol.

    As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku.

    Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security.

    This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography.

    One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information

    Device Pairing​

    In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access.

    To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake.

    The process works as follows:

    Actors:

    • Alice the initiator
    • Bob the responder
    1. The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated
    2. Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process.
    3. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully
    4. Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker.

    The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.

    Conclusion​

    With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out!

    Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better.

    Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!

    References​

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Device Pairing in Js-waku and Go-waku

    by
    5 min read

    Device pairing and secure message exchange using Waku and noise protocol.

    + +

    As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku.

    +

    Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security.

    +

    This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography.

    +

    One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information

    +

    Device Pairing​

    +

    In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access.

    +

    To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake.

    +

    The process works as follows:

    +

    Actors:

    +
      +
    • Alice the initiator
      • +
    • Bob the responder
      • +
    +
      +
    1. The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated
      2. +
    2. Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process.
      3. +
    3. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully
      4. +
    4. Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker.
      5. +
    +

    The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.

    +

    Conclusion​

    +

    With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out!

    +

    Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better.

    +

    Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/dns-based-discovery/index.html b/rlog/dns-based-discovery/index.html index 0d1bd970..db925602 100644 --- a/rlog/dns-based-discovery/index.html +++ b/rlog/dns-based-discovery/index.html @@ -1,26 +1,57 @@ - + - -DNS Based Discovery | Vac Research + +DNS Based Discovery | Vac Research - - - - + + + + -
    -

    DNS Based Discovery

    by
    6 min read

    A look at EIP-1459 and the benefits of DNS based discovery.

    Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as Kademlia which utilizes a distributed hash table or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data.

    Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a static list of nodes that a client can connect to.

    In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable.

    How do we do this?

    EIP 1459: Node Discovery via DNS, which is one of the strategies we are using for discovering waku nodes. EIP-1459 is a DNS-based discovery protocol that stores merkle trees in DNS records which contain connection information for nodes.

    Waku is our fork of Whisper. Oskar recently wrote an entire post explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification here

    DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be.

    > dig dnsseed.bluematt.me +short
    129.226.73.12
    107.180.78.111
    169.255.56.123
    91.216.149.28
    85.209.240.91
    66.232.124.232
    207.55.53.96
    86.149.241.168
    193.219.38.57
    190.198.210.139
    74.213.232.234
    158.181.226.33
    176.99.2.207
    202.55.87.45
    37.205.10.3
    90.133.4.73
    176.191.182.3
    109.207.166.232
    45.5.117.59
    178.211.170.2
    160.16.0.30

    The above displays the result of querying on of these DNS seeds. All the nodes are stored as A records for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the IRC bootstrapping method in v0.8.2.

    What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices.

    Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name.

    As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so.

    One thing you notice when looking at EIP-1459 is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method?

    There are multiple reasons, but the main one is security. In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in EIP-1459 where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records.

    Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party.

    So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using merkle trees as follows:

    • The root of the tree is stored in a TXT record, this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above.

      Additionally, there is also a root hash to a second tree called a link tree, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names.

      The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version.

    • Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a TXT record. The branch record contains all the hashes of the branch's leafs.

    • Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at EIP-778, or read a short blog post I wrote explaining them here.

    Below is the zone file taken from the EIP-1459, displaying how this looks in practice.

    ; name                        ttl     class type  content
    @                             60      IN    TXT   enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA
    C7HRFPF3BLGF3YR4DY5KX3SMBE    86900   IN    TXT   enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org
    JWXYDBPXYWG6FX3GMDIBFA6CJ4    86900   IN    TXT   enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24
    2XS2367YHAXJFGLZHVAWLQD4ZY    86900   IN    TXT   enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA
    H4FHT4B454P6UXFD7JCYQ5PWDY    86900   IN    TXT   enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI
    MHTDO6TMUBRIA2XWG5LUDACK24    86900   IN    TXT   enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o

    All of this has already been introduced into go-ethereum with the pull request #20094, created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on github.

    I hope this was a helpful explainer into DNS based discovery, and shows EIP-1459's benefits over more traditional DNS-based discovery schemes.

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    DNS Based Discovery

    by
    6 min read

    A look at EIP-1459 and the benefits of DNS based discovery.

    + +

    Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as Kademlia which utilizes a distributed hash table or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data.

    +

    Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a static list of nodes that a client can connect to.

    +

    In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable.

    +

    How do we do this?

    +

    EIP 1459: Node Discovery via DNS, which is one of the strategies we are using for discovering waku nodes. EIP-1459 is a DNS-based discovery protocol that stores merkle trees in DNS records which contain connection information for nodes.

    +

    Waku is our fork of Whisper. Oskar recently wrote an entire post explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification here

    +

    DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be.

    +
    > dig dnsseed.bluematt.me +short
    129.226.73.12
    107.180.78.111
    169.255.56.123
    91.216.149.28
    85.209.240.91
    66.232.124.232
    207.55.53.96
    86.149.241.168
    193.219.38.57
    190.198.210.139
    74.213.232.234
    158.181.226.33
    176.99.2.207
    202.55.87.45
    37.205.10.3
    90.133.4.73
    176.191.182.3
    109.207.166.232
    45.5.117.59
    178.211.170.2
    160.16.0.30
    +

    The above displays the result of querying on of these DNS seeds. All the nodes are stored as A records for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the IRC bootstrapping method in v0.8.2.

    +

    What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices.

    +

    Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name.

    +

    As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so.

    +

    One thing you notice when looking at EIP-1459 is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method?

    +

    There are multiple reasons, but the main one is security. In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in EIP-1459 where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records.

    +

    Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party.

    +

    So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using merkle trees as follows:

    +
      +
    • +

      The root of the tree is stored in a TXT record, this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above.

      +

      Additionally, there is also a root hash to a second tree called a link tree, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names.

      +

      The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version.

      +
      • +
    • +

      Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a TXT record. The branch record contains all the hashes of the branch's leafs.

      +
      • +
    • +

      Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at EIP-778, or read a short blog post I wrote explaining them here.

      +
      • +
    +

    Below is the zone file taken from the EIP-1459, displaying how this looks in practice.

    +
    ; name                        ttl     class type  content
    @                             60      IN    TXT   enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA
    C7HRFPF3BLGF3YR4DY5KX3SMBE    86900   IN    TXT   enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org
    JWXYDBPXYWG6FX3GMDIBFA6CJ4    86900   IN    TXT   enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24
    2XS2367YHAXJFGLZHVAWLQD4ZY    86900   IN    TXT   enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA
    H4FHT4B454P6UXFD7JCYQ5PWDY    86900   IN    TXT   enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI
    MHTDO6TMUBRIA2XWG5LUDACK24    86900   IN    TXT   enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o
    +

    All of this has already been introduced into go-ethereum with the pull request #20094, created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on github.

    +

    I hope this was a helpful explainer into DNS based discovery, and shows EIP-1459's benefits over more traditional DNS-based discovery schemes.

    \ No newline at end of file diff --git a/rlog/ethics-surveillance-tech/index.html b/rlog/ethics-surveillance-tech/index.html index adafa2ae..bda4ef90 100644 --- a/rlog/ethics-surveillance-tech/index.html +++ b/rlog/ethics-surveillance-tech/index.html @@ -1,32 +1,37 @@ - + - -Opinion: Pseudo-ethics in the Surveillance Tech Industry | Vac Research + +Opinion: Pseudo-ethics in the Surveillance Tech Industry | Vac Research - - - - + + + + -
    -

    Opinion: Pseudo-ethics in the Surveillance Tech Industry

    by
    12 min read

    A look at typical ethical shortfalls in the global surveillance tech industry.

    This is an opinion piece by pseudonymous contributor, circe.

    Preface​

    The Vac team aims to provide a public good in the form of freely available, open source tools and protocols for decentralized communication. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Opinion: Pseudo-ethics in the Surveillance Tech Industry

    by
    12 min read

    A look at typical ethical shortfalls in the global surveillance tech industry.

    + +

    This is an opinion piece by pseudonymous contributor, circe.

    +

    Preface​

    +

    The Vac team aims to provide a public good in the form of freely available, open source tools and protocols for decentralized communication. As such, we value our independence and the usefulness of our protocols for a wide range of applications. At the same time, we realize that all technical development, including ours, has a moral component. As a diverse team we are guided by a shared devotion to the principles of human rights and liberty. This explains why we place such a high premium on security, censorship-resistance and privacy - a stance we share with the wider Status Network. The post below takes a different approach from our usual more technical analyses, -by starting to peel back the curtain on the ethical shortfalls of the global surveillance tech industry.

    Spotlight on an industry​

    Apple's announcement of their lawsuit against Israel's NSO Group +by starting to peel back the curtain on the ethical shortfalls of the global surveillance tech industry.

    +

    Spotlight on an industry​

    +

    Apple's announcement of their lawsuit against Israel's NSO Group marks the latest in a series of recent setbacks for the surveillance tech company. In early November, the United States blacklisted the firm, citing concerns about the use of their spyware by foreign governments targeting civilians such as "journalists, businesspeople, activists" and more. @@ -37,7 +42,8 @@ sometimes without even requiring as much as an unguarded click on a malicious li It has the potential to lay bare, and report to its owners, everything within the reach of the infected device. For most people this amounts to a significant portion of their private lives and thoughts. Pegasus can read your private messages (even encrypted), collect your passwords, record calls, track your location and access your device's microphone and camera. -No activity or application on an infected phone would be hidden.

    The latest controversies are perhaps less because of the novelty of the revelations - +No activity or application on an infected phone would be hidden.

    +

    The latest controversies are perhaps less because of the novelty of the revelations - the existence of Pegasus has been known to civil activists since at least 2016. Rather, the public was reminded again of the potential scope of surveillance tech in the indiscriminate use of Pegasus on private citizens. @@ -49,7 +55,8 @@ The type of autocratically inclined agents, and governments, who would venture t Pegasus, and similar technologies, simply extend the reach and capacity of such individuals and governments - no border or distance, no political rank or social advantage, no sanctity of profession or regard for dignity, provide any indemnity from becoming a victim. -Your best hope is to remain uninteresting enough to escape consideration.

    The NSO Group has, of course, denied allegations of culpability and questions the authenticity of the list. +Your best hope is to remain uninteresting enough to escape consideration.

    +

    The NSO Group has, of course, denied allegations of culpability and questions the authenticity of the list. At this stage, the latter is almost beside the point: Amnesty International's cybersecurity team, Security Lab, did find forensic evidence of Pegasus on the phones of several volunteers whose numbers appeared on the original list, including those of journalists and human rights activists. @@ -60,7 +67,9 @@ both before and after Khashoggi's brutal murder at the Saudi embassy in Ist More reports of confirmed Pegasus hacks are still published with some regularity. It is now an open secret that many authoritarian governments have bought Pegasus. It's not difficult to extrapolate from existing reports and such clients' track records -what the potential injuries to human freedoms are that they can inflict with access to such a powerful cyberweapon.

    A typical response​

    NSO's response to the allegations follows a textbook approach +what the potential injuries to human freedoms are that they can inflict with access to such a powerful cyberweapon.

    +

    A typical response​

    +

    NSO's response to the allegations follows a textbook approach of avoiding earnest ethical introspection on the manufacturing, and selling, of cyber-arms. Firstly, shift ethical responsibility to a predetermined process, a list of checkboxes of your own making. The Group, for example, claims to sell only to "vetted governments", following a classification process @@ -79,7 +88,8 @@ The unapologetic tone seems hardly appropriate when the same document confirms t shut down customers' systems due to "confirmed misuse" and have had to do so "multiple times" in the past. Given all this, the response manages to evade any serious interrogation of the "vetting" process itself, which forced the company to reject "approximately 15% of potential new opportunities for Pegasus" in one year. -Courageous.

    We have heard this all before. +Courageous.

    +

    We have heard this all before. There exists a multi-billion dollar industry of private companies and engineering firms thriving on proceeds from selling surveillance tools and cyber-arms to dubious agencies and foreign governments. In turn, the most power-hungry and oppressive regimes often rely on such technological innovations - @@ -97,7 +107,10 @@ When the company themselves were Their website boasts an impressive list of statutes, regulations, procedures, export controls and legal frameworks, all of which the rebranded hackers proudly comply with. -Surely no further ethical scrutiny is necessary?

    Ethics != the law​

    The law is trailing behind​

    Such recourse to the legality of your action as ethical justification is moot for several reasons. +Surely no further ethical scrutiny is necessary?

    +

    Ethics != the law​

    +

    The law is trailing behind​

    +

    Such recourse to the legality of your action as ethical justification is moot for several reasons. The first is glaringly obvious - our laws are ill-equipped to address the implications of modern technology. Legal systems are a cumbersome inheritance built over generations. @@ -105,7 +118,9 @@ This is especially true of the statutes and regulations governing international Our best legal systems are trailing miles behind the technology for which we seek guidelines. Legislators are still struggling to make sense of technologies like face recognition, the repercussions of smart devices acting "on their own" and biases in algorithms. -To claim you are performing ethical due diligence by resorting to an outdated and incomplete system of legal codes is disingenuous.

    The law depends on ethics​

    The second reason is more central to my argument, +To claim you are performing ethical due diligence by resorting to an outdated and incomplete system of legal codes is disingenuous.

    +

    The law depends on ethics​

    +

    The second reason is more central to my argument, and an important flaw in these sleight of hand justifications appearing from time to time in the media. Ethics can in no way be confused as synonymous with legality or legitimacy. These are incommensurable concepts. @@ -122,7 +137,9 @@ For this reason, ethics is primarily enforced by social and internal pressures, our desire to do what ought to be done, however we define that. Ethics is therefore a much grander scheme than global legal systems and the diplomatic frameworks that grants legitimacy to governments. -These are but one limited outflow of the human aspiration to form societies in accordance with our ideologies and ethics.

    International law is vague and exploitable​

    Of course, the cyber-arms trade has a favorite recourse, international law, which is even more limited. +These are but one limited outflow of the human aspiration to form societies in accordance with our ideologies and ethics.

    +

    International law is vague and exploitable​

    +

    Of course, the cyber-arms trade has a favorite recourse, international law, which is even more limited. Since such products are seldomly sold to governments and agencies within the country of production, it enables a further distancing from consequences. Many private surveillance companies are based in fairly liberal societies with (seemingly) strict emphases on human rights in their domestic laws. @@ -132,7 +149,9 @@ the whims of autocrats with exploitative intent and vastly different ethical con International law, and the ways it is most often enforced by way of, say, UN-backed sanctions, have long been shaped by the compromises of international diplomacy. To be blunt: these laws are weak and subject to exactly the sort of narrow interests behind which mercenaries have always hidden. -The surveillance tech industry is no exception.

    Conclusion​

    My point is simple: +The surveillance tech industry is no exception.

    +

    Conclusion​

    +

    My point is simple: selling cyber-arms with the potential to become vast tools of oppression to governments and bodies with blatant histories of human rights violations, and all but the publicly announced intention to continue operating in this way, is categorically unconscionable. @@ -143,7 +162,8 @@ It is a sign of poor moral discourse that such recourses to law and legitimacy a Ethical conduct requires an honest evaluation of an action against some conception of "the good", however you define that. Too often the surveillance tech industry precisely sidesteps this question, -both in internal processes and external rationalisations to a concerned public.

    John Locke, he of the life-liberty-and-property, articulated the idea that government exists solely through the consent of the governed. +both in internal processes and external rationalisations to a concerned public.

    +

    John Locke, he of the life-liberty-and-property, articulated the idea that government exists solely through the consent of the governed. Towards the end of the 17th century, he wrote in his Second Treatise on Civil Government, "[w]henever legislators endeavor to take away, and destroy the property of the people, or to reduce them to slavery under arbitrary power, @@ -152,7 +172,8 @@ The inference is straightforward and humanist in essence: legitimacy is not something that is conferred by governments and institutions. Rather, they derive their legitimacy from us, their citizens, holding them to standards of ethics and societal ideals. This legitimacy only remains in tact as long as this mandate is honored and continuously extended by a well-informed public. -This is the principle of informed consent on which all reciprocal ethics is based.

    The surveillance tech industry may well have nothing more or less noble in mind than profit-making within legal bounds +This is the principle of informed consent on which all reciprocal ethics is based.

    +

    The surveillance tech industry may well have nothing more or less noble in mind than profit-making within legal bounds when developing and selling their products. However, when such companies are revealed again and again to have supplied tools of gross human rights violations to known human rights violators, they will do well to remember that ethics always precedes requirements of legality and legitimacy. @@ -163,8 +184,24 @@ no value-judgements about their legitimacy can be made. Hiding behind legal compliance as substitute for moral justification is not enough. Targets of increasingly invasive governmental snooping are too often chosen precisely to suppress the mechanisms from which the legitimacy of such governments flow - the consent of ordinary civilians. -Free and fair elections, free speech, free media, freedom of thought are all at risk.

    References​

    - - +Free and fair elections, free speech, free media, freedom of thought are all at risk.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/feasibility-discv5/index.html b/rlog/feasibility-discv5/index.html index ca14d581..028cf435 100644 --- a/rlog/feasibility-discv5/index.html +++ b/rlog/feasibility-discv5/index.html @@ -1,26 +1,72 @@ - + - -Feasibility Study: Discv5 | Vac Research + +Feasibility Study: Discv5 | Vac Research - - - - + + + + -
    -

    Feasibility Study: Discv5

    by
    6 min read

    Looking at discv5 and the theoretical numbers behind finding peers.

    Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers.

    This post serves as a more authoritative overview of the discv5 study, for a discussionary post providing more context make sure to check out the corresponding discuss post. Additionally, if you are unfamiliar with discv5, check out my previous write-up: "From Kademlia to Discv5".

    Motivating Problem​

    The discovery method currently used by Status, is made up of various components and grew over time to solve a mix of problems. We want to simplify this while maintaining some of the properties we currently have.

    Namely, we want to ensure censorship resistance to state-level adversaries. One of the issues Status had which caused us them add to their discovery method was the fact that addresses from providers like AWS and GCP were blocked both in Russia and China. Additionally, one of the main factors required is the ability to function on resource restricted devices.

    Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider:

    • Battery consumption - constant connections like websockets consume a lot of battery life.
    • CPU usage - certain discovery methods may be CPU incentive, slowing an app down and making it unusable.
    • Bandwidth consumption - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans.
    • Short connection windows - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time.
    • Not publicly connectable - There is a good chance that most resource restricted devices are not publicly connectable.

    For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node.

    With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation.

    In order to answer our questions, we formulated the problem with a simple method for testing. The "needle in a haystack" problem was formulated to figure out how easily a specific node can be found within a given network. This issue was fully formulated in vacp2p/research#15.

    Overview​

    The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer:

    • We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.
    • We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through topic tables.

    Feasbility​

    To be able to investigate the feasibility of discv5, we used various methods including rough calculations which can be found in the notebook, and a simulation isolated in vacp2p/research#19.

    CPU & Memory Usage​

    The experimental discv5 has already been used within Status, however what was noticed was that the CPU and memory usage was rather high. It therefore should be investiaged if this is still the case, and if it is, it should be isolated where this stems from. Additionally it is worth looking at whether or not this is the case with both the go and nim implementation.

    See details: vacp2p/research#31

    NAT on Cellular Data​

    If a peer is not publically connectable it can not participate in the DHT both ways. A lot of mobile phones are behind symmetric NATs which UDP hole-punching close to impossible. It should be investigated whether or not mobile phones will be able to participate both ways and if there are good methods for doing hole-punching.

    See details: vacp2p/research#29

    Topic Tables​

    Topic Tables allow us the ability to efficiently find nodes given a specific topic. However, they are not implemented in the status-im/nim-eth implementation nor are they fully finalized in the spec. These are important if the network grows past a size where the concentration of specific nodes is relatively low making them hard to find.

    See details: vacp2p/research#26

    Finding a node​

    It is important to note, that given a network is relatively small sized, eg 100-500 nodes, then finding a node given a specific address is relatively managable. Additionally, if the concentration of a specific capability in a network is reasonable, then finding a node advertising its capabilities using an ENR rather than the topic table is also managable. A reasonable concentration for example would be 10%, which would give us an 80% chance of getting a node with that capability in the first lookup request. This can be explored more using our discv5 notebook.

    Results​

    Research has shown that finding a node in the DHT has a relatively low effect on bandwidth, both inbound and outbound. For example when trying to find a node in a network of 100 nodes, it would take roughly 5668 bytes total. Additionally if we assume 100ms latency per request it would range at ≈ 300ms latency, translating to 3 requests to find a specific node.

    General Thoughts​

    One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more "features" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.

    Acknowledgements​

    • Oskar Thoren
    • Dmitry Shmatko
    • Kim De Mey
    • Corey Petty
    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Feasibility Study: Discv5

    by
    6 min read

    Looking at discv5 and the theoretical numbers behind finding peers.

    + +
    +

    Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers.

    +
    +

    This post serves as a more authoritative overview of the discv5 study, for a discussionary post providing more context make sure to check out the corresponding discuss post. Additionally, if you are unfamiliar with discv5, check out my previous write-up: "From Kademlia to Discv5".

    +

    Motivating Problem​

    +

    The discovery method currently used by Status, is made up of various components and grew over time to solve a mix of problems. We want to simplify this while maintaining some of the properties we currently have.

    +

    Namely, we want to ensure censorship resistance to state-level adversaries. One of the issues Status had which caused us them add to their discovery method was the fact that addresses from providers like AWS and GCP were blocked both in Russia and China. Additionally, one of the main factors required is the ability to function on resource restricted devices.

    +

    Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider:

    +
      +
    • Battery consumption - constant connections like websockets consume a lot of battery life.
      • +
    • CPU usage - certain discovery methods may be CPU incentive, slowing an app down and making it unusable.
      • +
    • Bandwidth consumption - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans.
      • +
    • Short connection windows - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time.
      • +
    • Not publicly connectable - There is a good chance that most resource restricted devices are not publicly connectable.
      • +
    +

    For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node.

    +

    With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation.

    +

    In order to answer our questions, we formulated the problem with a simple method for testing. The "needle in a haystack" problem was formulated to figure out how easily a specific node can be found within a given network. This issue was fully formulated in vacp2p/research#15.

    +

    Overview​

    +

    The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer:

    +
      +
    • We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.
      • +
    • We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through topic tables.
      • +
    +

    Feasbility​

    +

    To be able to investigate the feasibility of discv5, we used various methods including rough calculations which can be found in the notebook, and a simulation isolated in vacp2p/research#19.

    +

    CPU & Memory Usage​

    +

    The experimental discv5 has already been used within Status, however what was noticed was that the CPU and memory usage was rather high. It therefore should be investiaged if this is still the case, and if it is, it should be isolated where this stems from. Additionally it is worth looking at whether or not this is the case with both the go and nim implementation.

    +

    See details: vacp2p/research#31

    +

    NAT on Cellular Data​

    +

    If a peer is not publically connectable it can not participate in the DHT both ways. A lot of mobile phones are behind symmetric NATs which UDP hole-punching close to impossible. It should be investigated whether or not mobile phones will be able to participate both ways and if there are good methods for doing hole-punching.

    +

    See details: vacp2p/research#29

    +

    Topic Tables​

    +

    Topic Tables allow us the ability to efficiently find nodes given a specific topic. However, they are not implemented in the status-im/nim-eth implementation nor are they fully finalized in the spec. These are important if the network grows past a size where the concentration of specific nodes is relatively low making them hard to find.

    +

    See details: vacp2p/research#26

    +

    Finding a node​

    +

    It is important to note, that given a network is relatively small sized, eg 100-500 nodes, then finding a node given a specific address is relatively managable. Additionally, if the concentration of a specific capability in a network is reasonable, then finding a node advertising its capabilities using an ENR rather than the topic table is also managable. A reasonable concentration for example would be 10%, which would give us an 80% chance of getting a node with that capability in the first lookup request. This can be explored more using our discv5 notebook.

    +

    Results​

    +

    Research has shown that finding a node in the DHT has a relatively low effect on bandwidth, both inbound and outbound. For example when trying to find a node in a network of 100 nodes, it would take roughly 5668 bytes total. Additionally if we assume 100ms latency per request it would range at ≈ 300ms latency, translating to 3 requests to find a specific node.

    +

    General Thoughts​

    +

    One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more "features" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.

    +

    Acknowledgements​

    +
      +
    • Oskar Thoren
      • +
    • Dmitry Shmatko
      • +
    • Kim De Mey
      • +
    • Corey Petty
      • +
    \ No newline at end of file diff --git a/rlog/feasibility-semaphore-rate-limiting-zksnarks/index.html b/rlog/feasibility-semaphore-rate-limiting-zksnarks/index.html index 67586935..96533a3c 100644 --- a/rlog/feasibility-semaphore-rate-limiting-zksnarks/index.html +++ b/rlog/feasibility-semaphore-rate-limiting-zksnarks/index.html @@ -1,26 +1,119 @@ - + - -Feasibility Study: Semaphore rate limiting through zkSNARKs | Vac Research + +Feasibility Study: Semaphore rate limiting through zkSNARKs | Vac Research - - - + + + -
    -

    Feasibility Study: Semaphore rate limiting through zkSNARKs

    by
    8 min read

    A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks.

    tldr: Moon math promising for solving spam in Whisper, but to get there we need to invest more in performance work and technical upskilling.

    Motivating problem​

    In open p2p networks for messaging, one big problem is spam-resistance. Existing solutions, such as Whisper's proof of work, are insufficient, especially for heterogeneous nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.

    One possible solution is to use a right-to-access staking-based method, where a node is only able to send a message, signal, at a certain rate, and otherwise they can be slashed. One problem with this is in terms of privacy-preservation, where we specifically don't want a user to be tied to a specific payment or unique fingerprint.

    In addition to above, there are a lot of related problems that share similarities in terms of their structure and proposed solution.

    • Private transactions (Zcash, AZTEC)
    • Private voting (Semaphore)
    • Private group membership (Semaphore)
    • Layer 2 scaling, poss layer 1 (ZK Rollup; StarkWare/Eth2-3)

    Overview​

    Basic terminology​

    A zero-knowledge proof allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id.

    zkSNARKs is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more.

    Semaphore is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See current Github home for more.

    Circom is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See Github for more.

    Basic flow​

    We start with a private voting example, and then extend it to the slashable rate limiting example.

    1. A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are.

    2. When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some public outputs, and can be verified by anyone (including a smart contract).

    3. Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as "2020-01-01 vote on Foo Bar for president".

    4. Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once.

    5. As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified.

    Rate limiting example​

    In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat.

    NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same.

    1. Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s.

    2. Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key).

    3. This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them.

    4. As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it.

    Briefly on scope of 'approved users'​

    In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration.

    Technical details​

    For a fairly self-contained set of examples above, see exploration in Vac research repo. Note that the Shamir secret sharing is not inside the SNARK, but out-of-band for now.

    The current version of Semaphore is using NodeJS and Circom from Iden3 for Snarks.

    For more on rate limiting idea, see ethresearch post.

    Feasibility​

    The above repo was used to exercise the basic paths and to gain intution of feasibility. Based on it and related reading we outline a few blockers and things that require further study.

    Technical feasibility​

    Proof time​

    Prove time for Semaphore (https://github.com/kobigurk/semaphore) zKSNARKs using circom, groth and snarkjs is currently way too long. It takes on the order of ~10m to generate a proof. With Websnark, it is likely to take 30s, which might still be too long. We should experiment with native code on mobile here.

    See details.

    Proving key size​

    Prover key size is ~110mb for Semaphore. Assuming this is embedded on mobile device, it bloats the APK a lot. Current APK size is ~30mb and even that might be high for people with limited bandwidth.

    See details.

    Trusted setup​

    Using zkSNARKs a trusted setup is required to generate prover and verifier keys. As part of this setup, a toxic parameter lambda is generated. If a party gets access to this lambda, they can prove anything. This means people using zKSNARKs usually have an elaborate MPC ceremony to ensure this parameter doesn't get discovered.

    See details.

    Shamir logic in SNARK​

    For Semaphore RLN we need to embed the Shamir logic inside the SNARK in order to do slashing for spam. Currently the implementation is trusted and very hacky.

    See details.

    End to end integation​

    Currently is standalone and doesn't touch multiple users, deployed contract with merkle tree and verification, actual transactions, a mocked network, add/remove members, etc. There are bound to be edge cases and unknown unknowns here.

    See details.

    Licensing issues​

    Currently Circom uses a GPL license, which can get tricky when it comes to the App Store etc.

    See details.

    Alternative ZKPs?​

    Some of the isolated blockers for zKSNARKs (#7, #8, #9) might be mitigated by the use of other ZKP technology. However, they likely have their own issues.

    See details.

    Social feasibility​

    Technical skill​

    zkSNARKs and related technologies are quite new. To learn how they work and get an intuition for them requires individuals to dedicate a lot of time to studying them. This means we must make getting competence in these technologies if we wish to use them to our advantage.

    Time and resources​

    In order for this and related projects (such as private transaction) to get anywhere, it must be made an explicit area of focus for an extend period of time.

    General thoughts​

    Similar to Whisper, and in line with moving towards protocol and infrastructure, we need to upskill and invest resources into this. This doesn't mean developing all of the technologies ourselves, but gaining enough competence to leverage and extend existing solutions by the growing ZKP community.

    For example, this might also include leveraging largely ready made solutions such as AZTEC for private transaction; more fundamental research into ZK rollup and similar; using Semaphore for private group membership and private voting; Nim based wrapper aronud Bellman, etc.

    Acknowledgement​

    Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues.

    Peacock header image from [Tonos](<https://en.wikipedia.org/wiki/File:Flickr-lo.tangelini-Tonos(1).jpg>)._

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Feasibility Study: Semaphore rate limiting through zkSNARKs

    by
    8 min read

    A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks.

    + +

    tldr: Moon math promising for solving spam in Whisper, but to get there we need to invest more in performance work and technical upskilling.

    +

    Motivating problem​

    +

    In open p2p networks for messaging, one big problem is spam-resistance. Existing solutions, such as Whisper's proof of work, are insufficient, especially for heterogeneous nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.

    +

    One possible solution is to use a right-to-access staking-based method, where a node is only able to send a message, signal, at a certain rate, and otherwise they can be slashed. One problem with this is in terms of privacy-preservation, where we specifically don't want a user to be tied to a specific payment or unique fingerprint.

    + +

    In addition to above, there are a lot of related problems that share similarities in terms of their structure and proposed solution.

    +
      +
    • Private transactions (Zcash, AZTEC)
      • +
    • Private voting (Semaphore)
      • +
    • Private group membership (Semaphore)
      • +
    • Layer 2 scaling, poss layer 1 (ZK Rollup; StarkWare/Eth2-3)
      • +
    +

    Overview​

    +

    Basic terminology​

    +

    A zero-knowledge proof allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id.

    +

    zkSNARKs is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more.

    +

    Semaphore is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See current Github home for more.

    +

    Circom is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See Github for more.

    +

    Basic flow​

    +

    We start with a private voting example, and then extend it to the slashable rate limiting example.

    +
      +
    1. +

      A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are.

      +
      2. +
    2. +

      When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some public outputs, and can be verified by anyone (including a smart contract).

      +
      3. +
    3. +

      Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as "2020-01-01 vote on Foo Bar for president".

      +
      4. +
    4. +

      Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once.

      +
      5. +
    5. +

      As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified.

      +
      6. +
    +

    Rate limiting example​

    +

    In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat.

    +

    NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same.

    +
      +
    1. +

      Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s.

      +
      2. +
    2. +

      Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key).

      +
      3. +
    3. +

      This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them.

      +
      4. +
    4. +

      As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it.

      +
      5. +
    +

    Briefly on scope of 'approved users'​

    +

    In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration.

    +

    Technical details​

    +

    For a fairly self-contained set of examples above, see exploration in Vac research repo. Note that the Shamir secret sharing is not inside the SNARK, but out-of-band for now.

    +

    The current version of Semaphore is using NodeJS and Circom from Iden3 for Snarks.

    +

    For more on rate limiting idea, see ethresearch post.

    +

    Feasibility​

    +

    The above repo was used to exercise the basic paths and to gain intution of feasibility. Based on it and related reading we outline a few blockers and things that require further study.

    +

    Technical feasibility​

    +

    Proof time​

    +

    Prove time for Semaphore (https://github.com/kobigurk/semaphore) zKSNARKs using circom, groth and snarkjs is currently way too long. It takes on the order of ~10m to generate a proof. With Websnark, it is likely to take 30s, which might still be too long. We should experiment with native code on mobile here.

    +

    See details.

    +

    Proving key size​

    +

    Prover key size is ~110mb for Semaphore. Assuming this is embedded on mobile device, it bloats the APK a lot. Current APK size is ~30mb and even that might be high for people with limited bandwidth.

    +

    See details.

    +

    Trusted setup​

    +

    Using zkSNARKs a trusted setup is required to generate prover and verifier keys. As part of this setup, a toxic parameter lambda is generated. If a party gets access to this lambda, they can prove anything. This means people using zKSNARKs usually have an elaborate MPC ceremony to ensure this parameter doesn't get discovered.

    +

    See details.

    +

    Shamir logic in SNARK​

    +

    For Semaphore RLN we need to embed the Shamir logic inside the SNARK in order to do slashing for spam. Currently the implementation is trusted and very hacky.

    +

    See details.

    +

    End to end integation​

    +

    Currently is standalone and doesn't touch multiple users, deployed contract with merkle tree and verification, actual transactions, a mocked network, add/remove members, etc. There are bound to be edge cases and unknown unknowns here.

    +

    See details.

    +

    Licensing issues​

    +

    Currently Circom uses a GPL license, which can get tricky when it comes to the App Store etc.

    +

    See details.

    +

    Alternative ZKPs?​

    +

    Some of the isolated blockers for zKSNARKs (#7, #8, #9) might be mitigated by the use of other ZKP technology. However, they likely have their own issues.

    +

    See details.

    +

    Social feasibility​

    +

    Technical skill​

    +

    zkSNARKs and related technologies are quite new. To learn how they work and get an intuition for them requires individuals to dedicate a lot of time to studying them. This means we must make getting competence in these technologies if we wish to use them to our advantage.

    +

    Time and resources​

    +

    In order for this and related projects (such as private transaction) to get anywhere, it must be made an explicit area of focus for an extend period of time.

    +

    General thoughts​

    +

    Similar to Whisper, and in line with moving towards protocol and infrastructure, we need to upskill and invest resources into this. This doesn't mean developing all of the technologies ourselves, but gaining enough competence to leverage and extend existing solutions by the growing ZKP community.

    +

    For example, this might also include leveraging largely ready made solutions such as AZTEC for private transaction; more fundamental research into ZK rollup and similar; using Semaphore for private group membership and private voting; Nim based wrapper aronud Bellman, etc.

    +

    Acknowledgement​

    +

    Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues.

    +

    _Peacock header image from Tonos._

    \ No newline at end of file diff --git a/rlog/fixing-whisper-with-waku/index.html b/rlog/fixing-whisper-with-waku/index.html index 3ddd2cea..a36f760a 100644 --- a/rlog/fixing-whisper-with-waku/index.html +++ b/rlog/fixing-whisper-with-waku/index.html @@ -1,47 +1,123 @@ - + - -Fixing Whisper with Waku | Vac Research + +Fixing Whisper with Waku | Vac Research - - - + + + -
    -

    Fixing Whisper with Waku

    by
    10 min read

    A research log. Why Whisper doesn't scale and how to fix it.

    This post will introduce Waku. Waku is a fork of Whisper that attempts to + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Fixing Whisper with Waku

    by
    11 min read

    A research log. Why Whisper doesn't scale and how to fix it.

    + +

    This post will introduce Waku. Waku is a fork of Whisper that attempts to addresses some of Whisper's shortcomings in an iterative fashion. We will also introduce a theoretical scaling model for Whisper that shows why it doesn't -scale, and what can be done about it.

    Introduction​

    Whisper is a gossip-based communication protocol or an ephemeral key-value store +scale, and what can be done about it.

    +

    Introduction​

    +

    Whisper is a gossip-based communication protocol or an ephemeral key-value store depending on which way you look at it. Historically speaking, it is the messaging pilllar of Web3, together with -Ethereum for consensus and Swarm for storage.

    Whisper, being a somewhat esoteric protocol and with some fundamental issues, +Ethereum for consensus and Swarm for storage.

    +

    Whisper, being a somewhat esoteric protocol and with some fundamental issues, hasn't seen a lot of usage. However, applications such as Status are using it, and have been making minor ad hoc modifications to it to make it run on mobile -devices.

    What are these fundamental issues? In short:

    1. scalability, most immediately when it comes to bandwidth usage
    2. spam-resistance, proof of work is a poor mechanism for heterogeneous nodes
    3. no incentivized infrastructure, leading to centralized choke points
    4. lack of formal and unambiguous specification makes it hard to analyze and implement
    5. running over devp2p, which limits where it can run and how

    In this post, we'll focus on the first problem, which is scalability through bandwidth usage.

    Whisper theoretical scalability model​

    (Feel free to skip this section if you want to get right to the results).

    There's widespread implicit knowledge that Whisper "doesn't scale", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see Status Whisper usage -spec).

    Caveats​

    First, some caveats: this model likely contains bugs, has wrong assumptions, or completely misses certain dimensions. However, it acts as a form of existence proof for unscalability, with clear reasons.

    If certain assumptions are wrong, then we can challenge them and reason about them in isolation. It doesn’t mean things will definitely work as the model predicts, and that there aren’t unknown unknowns.

    The model also only deals with receiving bandwidth for end nodes, uses mostly static assumptions of averages, and doesn’t deal with spam resistance, privacy guarantees, accounting, intermediate node or network wide failures.

    Goals​

    1. Ensure network scales by being user or usage bound, as opposed to bandwidth growing in proportion to network size.
    2. Staying with in a reasonable bandwidth limit for limited data plans.
    3. Do the above without materially impacting existing nodes.

    It proceeds through various case with clear assumptions behind them, starting from the most naive assumptions. It shows results for 100 users, 10k users and 1m users.

    Model​

    Case 1. Only receiving messages meant for you [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A4. Only receiving messages meant for you.

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is 1000.0KB/day
    For  1m users, receiving bandwidth is 1000.0KB/day

    ------------------------------------------------------------

    Case 2. Receiving messages for everyone [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A5. Received messages for everyone.

    For 100 users, receiving bandwidth is   97.7MB/day
    For 10k users, receiving bandwidth is    9.5GB/day
    For  1m users, receiving bandwidth is  953.7GB/day

    ------------------------------------------------------------

    Case 3. All private messages go over one discovery topic [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A8. All private messages are received by everyone (same topic) (static).

    For 100 users, receiving bandwidth is   49.3MB/day
    For 10k users, receiving bandwidth is    4.8GB/day
    For  1m users, receiving bandwidth is  476.8GB/day

    ------------------------------------------------------------

    Case 4. All private messages are partitioned into shards [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    Case 5. 4 + Bloom filter with false positive rate

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1

    For 100 users, receiving bandwidth is   10.7MB/day
    For 10k users, receiving bandwidth is  978.0MB/day
    For  1m users, receiving bandwidth is   95.5GB/day

    NOTE: Traffic extremely sensitive to bloom false positives
    This completely dominates network traffic at scale.
    With p=1% we get 10k users ~100MB/day and 1m users ~10gb/day)

    ------------------------------------------------------------

    Case 6. Case 5 + Benign duplicate receives

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).

    For 100 users, receiving bandwidth is   21.5MB/day
    For 10k users, receiving bandwidth is    1.9GB/day
    For  1m users, receiving bandwidth is  190.9GB/day

    ------------------------------------------------------------

    Case 7. 6 + Mailserver under good conditions; small bloom fp; mostly offline

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).
    - A17. User is offline p% of the time (static) p=0.9
    - A18. No bad request, dup messages for mailservers; overlap perfect (static).
    - A19. Mailserver requests can change false positive rate to be p=0.01

    For 100 users, receiving bandwidth is    3.9MB/day
    For 10k users, receiving bandwidth is  284.8MB/day
    For  1m users, receiving bandwidth is   27.8GB/day

    ------------------------------------------------------------

    Case 8. No metadata protection w bloom filter; 1 node connected; static shard

    Aka waku mode.

    Next step up is to either only use contact code, or shard more aggressively.
    Note that this requires change of other nodes behavior, not just local node.

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    See source -for more detail on the model and its assumptions.

    Takeaways​

    1. Whisper as it currently works doesn’t scale, and we quickly run into unacceptable bandwidth usage.
    2. There are a few factors of this, but largely it boils down to noisy topics usage and use of bloom filters. Duplicate (e.g. see Whisper vs PSS) and bad envelopes are also factors, but this depends a bit more on specific deployment configurations.
    3. Waku mode (case 8) is an additional capability that doesn’t require other nodes to change, for nodes that put a premium on performance.
    4. The next bottleneck after this is the partitioned topics (app/network specific), which either needs to gracefully (and potentially quickly) grow, or an alternative way of consuming those messages needs to be deviced.

    The results are summarized in the graph above. Notice the log-log scale. The -colored backgrounds correspond to the following bandwidth usage:

    • Blue: <10mb/d (<~300mb/month)
    • Green: <30mb/d (<~1gb/month)
    • Yellow: <100mb/d (<~3gb/month)
    • Red: >100mb/d (>3gb/month)

    These ranges are somewhat arbitrary, but are based on user +devices.

    +

    What are these fundamental issues? In short:

    +
      +
    1. scalability, most immediately when it comes to bandwidth usage
      2. +
    2. spam-resistance, proof of work is a poor mechanism for heterogeneous nodes
      3. +
    3. no incentivized infrastructure, leading to centralized choke points
      4. +
    4. lack of formal and unambiguous specification makes it hard to analyze and implement
      5. +
    5. running over devp2p, which limits where it can run and how
      6. +
    +

    In this post, we'll focus on the first problem, which is scalability through bandwidth usage.

    +

    Whisper theoretical scalability model​

    +

    (Feel free to skip this section if you want to get right to the results).

    +

    There's widespread implicit knowledge that Whisper "doesn't scale", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see Status Whisper usage +spec).

    +

    Caveats​

    +

    First, some caveats: this model likely contains bugs, has wrong assumptions, or completely misses certain dimensions. However, it acts as a form of existence proof for unscalability, with clear reasons.

    +

    If certain assumptions are wrong, then we can challenge them and reason about them in isolation. It doesn’t mean things will definitely work as the model predicts, and that there aren’t unknown unknowns.

    +

    The model also only deals with receiving bandwidth for end nodes, uses mostly static assumptions of averages, and doesn’t deal with spam resistance, privacy guarantees, accounting, intermediate node or network wide failures.

    +

    Goals​

    +
      +
    1. Ensure network scales by being user or usage bound, as opposed to bandwidth growing in proportion to network size.
      2. +
    2. Staying with in a reasonable bandwidth limit for limited data plans.
      3. +
    3. Do the above without materially impacting existing nodes.
      4. +
    +

    It proceeds through various case with clear assumptions behind them, starting from the most naive assumptions. It shows results for 100 users, 10k users and 1m users.

    +

    Model​

    +
    Case 1. Only receiving messages meant for you [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A4. Only receiving messages meant for you.

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is 1000.0KB/day
    For  1m users, receiving bandwidth is 1000.0KB/day

    ------------------------------------------------------------

    Case 2. Receiving messages for everyone [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A5. Received messages for everyone.

    For 100 users, receiving bandwidth is   97.7MB/day
    For 10k users, receiving bandwidth is    9.5GB/day
    For  1m users, receiving bandwidth is  953.7GB/day

    ------------------------------------------------------------

    Case 3. All private messages go over one discovery topic [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A8. All private messages are received by everyone (same topic) (static).

    For 100 users, receiving bandwidth is   49.3MB/day
    For 10k users, receiving bandwidth is    4.8GB/day
    For  1m users, receiving bandwidth is  476.8GB/day

    ------------------------------------------------------------

    Case 4. All private messages are partitioned into shards [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    Case 5. 4 + Bloom filter with false positive rate

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1

    For 100 users, receiving bandwidth is   10.7MB/day
    For 10k users, receiving bandwidth is  978.0MB/day
    For  1m users, receiving bandwidth is   95.5GB/day

    NOTE: Traffic extremely sensitive to bloom false positives
    This completely dominates network traffic at scale.
    With p=1% we get 10k users ~100MB/day and 1m users ~10gb/day)

    ------------------------------------------------------------

    Case 6. Case 5 + Benign duplicate receives

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).

    For 100 users, receiving bandwidth is   21.5MB/day
    For 10k users, receiving bandwidth is    1.9GB/day
    For  1m users, receiving bandwidth is  190.9GB/day

    ------------------------------------------------------------

    Case 7. 6 + Mailserver under good conditions; small bloom fp; mostly offline

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).
    - A17. User is offline p% of the time (static) p=0.9
    - A18. No bad request, dup messages for mailservers; overlap perfect (static).
    - A19. Mailserver requests can change false positive rate to be p=0.01

    For 100 users, receiving bandwidth is    3.9MB/day
    For 10k users, receiving bandwidth is  284.8MB/day
    For  1m users, receiving bandwidth is   27.8GB/day

    ------------------------------------------------------------

    Case 8. No metadata protection w bloom filter; 1 node connected; static shard

    Aka waku mode.

    Next step up is to either only use contact code, or shard more aggressively.
    Note that this requires change of other nodes behavior, not just local node.

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------
    +

    See source +for more detail on the model and its assumptions.

    +

    Takeaways​

    +
      +
    1. Whisper as it currently works doesn’t scale, and we quickly run into unacceptable bandwidth usage.
      2. +
    2. There are a few factors of this, but largely it boils down to noisy topics usage and use of bloom filters. Duplicate (e.g. see Whisper vs PSS) and bad envelopes are also factors, but this depends a bit more on specific deployment configurations.
      3. +
    3. Waku mode (case 8) is an additional capability that doesn’t require other nodes to change, for nodes that put a premium on performance.
      4. +
    4. The next bottleneck after this is the partitioned topics (app/network specific), which either needs to gracefully (and potentially quickly) grow, or an alternative way of consuming those messages needs to be deviced.
      5. +
    +

    +

    The results are summarized in the graph above. Notice the log-log scale. The +colored backgrounds correspond to the following bandwidth usage:

    +
      +
    • Blue: 10mb/d (~300mb/month)
      • +
    • Green: 30mb/d (~1gb/month)
      • +
    • Yellow: 100mb/d (~3gb/month)
      • +
    • Red: >100mb/d (>3gb/month)
      • +
    +

    These ranges are somewhat arbitrary, but are based on user requirements for users -on a limited data plan, with comparable usage for other messaging apps.

    Introducing Waku​

    Motivation for a new protocol​

    Apps such as Status will likely use something like Whisper for the forseeable +on a limited data plan, with comparable usage for other messaging apps.

    +

    Introducing Waku​

    +

    Motivation for a new protocol​

    +

    Apps such as Status will likely use something like Whisper for the forseeable future, and we want to enable them to use it with more users on mobile devices -without bandwidth exploding with minimal changes.

    Additionally, there's not a clear cut alternative that maps cleanly to the -desired use cases (p2p, multicast, privacy-preserving, open, etc).

    We are actively researching, developing and collaborating with more greenfield +without bandwidth exploding with minimal changes.

    +

    Additionally, there's not a clear cut alternative that maps cleanly to the +desired use cases (p2p, multicast, privacy-preserving, open, etc).

    +

    We are actively researching, developing and collaborating with more greenfield approaches. It is likely that Waku will either converge to those, or Waku will lay the groundwork (clear specs, common issues/components) necessary to make switching to another protocol easier. In this project we want to emphasize -iterative work with results on the order of weeks.

    Briefly on Waku mode​

    • Doesn’t impact existing clients, it’s just a separate node and capability.
    • Other nodes can still use Whisper as is, like a full node.
    • Sacrifices metadata protection and incurs higher connectivity/availability requirements for scalbility

    Requirements:

    • Exposes API to get messages from a set of list of topics (no bloom filter)
    • Way of being identified as a Waku node (e.g. through version string)
    • Option to statically encode this node in app, e.g. similar to custom bootnodes/mailserver
    • Only node that needs to be connected to, possibly as Whisper relay / mailserver hybrid

    Provides:

    • likely provides scalability of up to 10k users and beyond
    • with some enhancements to partition topic logic, can possibly scale up to 1m users (app/network specific)

    Caveats:

    • hasn’t been tested in a large-scale simulation
    • other network and intermediate node bottlenecks might become apparent (e.g. full bloom filter and private cluster capacity; can likely be dealt with in isolation using known techniques, e.g. load balancing) (deployment specific)

    Progress so far​

    In short, we have a Waku version 0 spec up as well as a PoC for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for Waku mode. See rough roadmap, project board [link deprecated] and progress thread on the Vac forum.

    The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously ad hoc implemented features, such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the geth (Go), status/whisper (Go) and nim-eth (Nim) versions, specifically around light node usage and the handshake.

    If you are interested in this effort, please check out our forum for questions, comments and proposals. We already have some discussion for better spam protection (see previous post for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement.

    - - +iterative work with results on the order of weeks.

    +

    Briefly on Waku mode​

    +
      +
    • Doesn’t impact existing clients, it’s just a separate node and capability.
      • +
    • Other nodes can still use Whisper as is, like a full node.
      • +
    • Sacrifices metadata protection and incurs higher connectivity/availability requirements for scalbility
      • +
    +

    Requirements:

    +
      +
    • Exposes API to get messages from a set of list of topics (no bloom filter)
      • +
    • Way of being identified as a Waku node (e.g. through version string)
      • +
    • Option to statically encode this node in app, e.g. similar to custom bootnodes/mailserver
      • +
    • Only node that needs to be connected to, possibly as Whisper relay / mailserver hybrid
      • +
    +

    Provides:

    +
      +
    • likely provides scalability of up to 10k users and beyond
      • +
    • with some enhancements to partition topic logic, can possibly scale up to 1m users (app/network specific)
      • +
    +

    Caveats:

    +
      +
    • hasn’t been tested in a large-scale simulation
      • +
    • other network and intermediate node bottlenecks might become apparent (e.g. full bloom filter and private cluster capacity; can likely be dealt with in isolation using known techniques, e.g. load balancing) (deployment specific)
      • +
    +

    Progress so far​

    +

    In short, we have a Waku version 0 spec up as well as a PoC for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for Waku mode. See rough roadmap, project board [link deprecated] and progress thread on the Vac forum.

    +

    The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously ad hoc implemented features, such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the geth (Go), status/whisper (Go) and nim-eth (Nim) versions, specifically around light node usage and the handshake.

    +

    If you are interested in this effort, please check out our forum for questions, comments and proposals. We already have some discussion for better spam protection (see previous post for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement.

    \ No newline at end of file diff --git a/rlog/future-of-waku-network/index.html b/rlog/future-of-waku-network/index.html index 71b19874..357ed883 100644 --- a/rlog/future-of-waku-network/index.html +++ b/rlog/future-of-waku-network/index.html @@ -1,63 +1,112 @@ - + - -The Future of Waku Network: Scaling, Incentivization, and Heterogeneity | Vac Research + +The Future of Waku Network: Scaling, Incentivization, and Heterogeneity | Vac Research - - - + + + -
    -

    The Future of Waku Network: Scaling, Incentivization, and Heterogeneity

    by
    6 min read

    Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like.

    Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    The Future of Waku Network: Scaling, Incentivization, and Heterogeneity

    by
    6 min read

    Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like.

    + +

    Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an overview of recent discussions and research outputs, aiming to give you a better understanding of how the Waku network -may look like in terms of scaling and incentivization.

    DOS Mitigation for Status Communities​

    Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN +may look like in terms of scaling and incentivization.

    +

    DOS Mitigation for Status Communities​

    +

    Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN (Rate Limiting Nullifiers) remains the go-to DOS protection solution due to its privacy-preserving and censorship-resistant properties, there is still more work to be done. We are excited to collaborate with PSE -(Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.

    A Heterogeneous Waku Network​

    As we noted in a previous forum post, Waku's protocol +(Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.

    +

    A Heterogeneous Waku Network​

    +

    As we noted in a previous forum post, Waku's protocol incentivization model needs to be flexible to accommodate various business models. Flexibility ensures that projects -can choose how they want to use Waku based on their specific needs.

    Reversing the Incentivization Question​

    Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to -reframe the question and instead ask, "How do we pay for the infrastructure?"

    Waku does not intend to offer a free lunch. +can choose how they want to use Waku based on their specific needs.

    +

    Reversing the Incentivization Question​

    +

    Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to +reframe the question and instead ask, "How do we pay for the infrastructure?"

    +

    Waku does not intend to offer a free lunch. Ethereum's infrastructure is supported by transaction fees and inflation, with validators receiving rewards from both sources. However, this model does not suit a communication network like Waku. Users and platforms would not want to pay for every single message they send. Additionally, Waku aims to support instant -ephemeral messages that do not require consensus or long-term storage.

    Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or +ephemeral messages that do not require consensus or long-term storage.

    +

    Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or inter-wallet communication, may have different value extraction models. Some users might provide services for the project and expect to receive value by running nodes, while others may pay for the product or run infrastructure to contribute back. Waku aims to support each of these use cases, which means there will be various ways to "pay for the -infrastructure."

    In his talk, Oskar addressed two strategies: RLN and service credentials.

    RLN and Service Credentials​

    RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, -and you can send messages.

    Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in -stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community.

    Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, +infrastructure."

    +

    In his talk, Oskar addressed two strategies: RLN and service credentials.

    +

    RLN and Service Credentials​

    +

    RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, +and you can send messages.

    +

    Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in +stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community.

    +

    Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, a community owner could pay for more storage or bandwidth, similar to Slack's model. Alternatively, individual users could contribute financially, akin to Discord's Server Boost, or by sharing their own -resources with their community.

    We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.

    Waku Network: Ethereum or Cosmos?​

    Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos.

    For those not familiar with the difference between both, in a very concise manner:

    • Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure
    • Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties

    We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. +resources with their community.

    +

    We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.

    +

    Waku Network: Ethereum or Cosmos?​

    +

    Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos.

    +

    For those not familiar with the difference between both, in a very concise manner:

    +
      +
    • Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure
      • +
    • Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties
      • +
    +

    We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. If each application has to deploy its own network, we will not achieve this goal. Therefore, we aim Waku to be not only an open source set of protocols, but also a shared infrastructure that anyone can leverage to build applications on top, with some guarantees in terms of decentralization and anonymity. This approach is closer in spirit to Ethereum than Cosmos. -Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network.

    Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. +Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network.

    +

    Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. We currently assume that there will be separate gossipsub networks with different funding models. -Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property.

    Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.

    Scalability and Discovery Protocols​

    To enable scalability, the flow of messages in the Waku network will be divided in shards, +Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property.

    +

    Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.

    +

    Scalability and Discovery Protocols​

    +

    To enable scalability, the flow of messages in the Waku network will be divided in shards, so that not every node has to forward every message of the whole network. -Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in.

    Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between).

    Marketplace protocols may also be developed to help operators understand how they can best support the network and where -their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.

    Open Problems​

    Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. -This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues.

    However, it does not cap traffic. -There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection.

    We have several ongoing initiatives:

    • Analyzing the Status Community protocol to confirm efficient usage of Waku [4]
    • Simulating the Waku Network to measure actual bandwidth usage [5]
    • Segregating chat messages from control and media messages [6]

    The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.

    The Evolving Waku Network​

    The definition of the "Waku Network" will likely change over time. In the near future, it will transition from a single +Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in.

    +

    Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between).

    +

    Marketplace protocols may also be developed to help operators understand how they can best support the network and where +their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.

    +

    Open Problems​

    +

    Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. +This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues.

    +

    However, it does not cap traffic. +There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection.

    +

    We have several ongoing initiatives:

    +
      +
    • Analyzing the Status Community protocol to confirm efficient usage of Waku [4]
      • +
    • Simulating the Waku Network to measure actual bandwidth usage [5]
      • +
    • Segregating chat messages from control and media messages [6]
      • +
    +

    The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.

    +

    The Evolving Waku Network​

    +

    The definition of the "Waku Network" will likely change over time. In the near future, it will transition from a single gossipsub network to a sharded set of networks unified by a common discovery layer. This change will promote scalability -and allow various payment models to coexist within the Waku ecosystem.

    In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly +and allow various payment models to coexist within the Waku ecosystem.

    +

    In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly maintaining its core principles. As Waku continues to evolve, we expect it to accommodate a diverse range of use cases and business models, all while preserving privacy, resisting censorship, avoiding surveillance, and remaining accessible -to devices with limited resources.

    References​

    1. WAKU2-RELAY-SHARDING
    2. 57/STATUS-Simple-Scaling
    3. RLN-V2
    4. Scaling Status Communities: Potential Problems
    5. Waku Network Testing
    6. WAKU2-RELAY-SHARDING: Control Message Shards
    - - +to devices with limited resources.

    +

    References​

    +
      +
    1. WAKU2-RELAY-SHARDING
      2. +
    2. 57/STATUS-Simple-Scaling
      3. +
    3. RLN-V2
      4. +
    4. Scaling Status Communities: Potential Problems
      5. +
    5. Waku Network Testing
      6. +
    6. WAKU2-RELAY-SHARDING: Control Message Shards
      7. +
    \ No newline at end of file diff --git a/rlog/gsub-idontwant-perf-eval/index.html b/rlog/gsub-idontwant-perf-eval/index.html index c7f5bf0c..be7c6447 100644 --- a/rlog/gsub-idontwant-perf-eval/index.html +++ b/rlog/gsub-idontwant-perf-eval/index.html @@ -1,52 +1,92 @@ - + - -Libp2p GossipSub IDONTWANT Message Performance Impact | Vac Research + +Libp2p GossipSub IDONTWANT Message Performance Impact | Vac Research - - - - + + + + -
    -

    Libp2p GossipSub IDONTWANT Message Performance Impact

    by
    4 min read

    This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.

    Overview​

    IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. -Cutting down on duplicates can potentially render two significant advantages:

    1. Reducing bandwidth requirements

    2. Reducing message dissemination time (latency)

    For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message.

    Duplicates investigation reveals that -the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network.

    This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, -showing a potential for a significant drop in bandwidth utilization. -On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.

    Experiments​

    We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) -to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time.

    The experiments are performed on nim-libp2p using the shadow simulator. -The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages.

    In all experiments, ten messages are transmitted to the network, i.e., -ten peers (publishers) are selected as the message transmitters. -Every publisher transmits exactly one message, -and inter-packet spacing (delay) is set to four seconds for each published message. -For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class.

    At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}. -These messages are not included in latency computations.

    The simulation details are presented in the table below.

    ParameterValueParameterValue
    Peers2000Publishers10
    Peer bandwidth50-150 MbpsLink latency40-130 ms
    Message size1KB, 50KB, 500KB, 1MBDD8
    Heartbeat interval700ms, 1000ms, 1500msDlowD_{low}6
    FloodPublishFalseDhighD_{high}12
    Gossip factor0.05Muxeryamux

    Findings​

    We use bandwidth utilization and latency as evaluation metrics. -Bandwidth utilization represents total network-wide traffic (including gossip and other control messages). -Latency refers to network-wide message dissemination time. -The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights.

    Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. -A higher drop is seen with a higher heartbeat interval. -Interestingly, a relatively low bandwidth reduction (12-20%) is seen for 1MB messages, -compared to 500KB messages (18-21%).

    Image 1
    Image 2
    Image 3

    This is because downloading a large message may consume several hundred milliseconds. -During this time, a receiver will likely -generate multiple IWANT requests -for the same message, increasing bandwidth utilization.

    Moreover, a peer can generate -IDONTWANTs only after it has finished downloading the message. -A longer download time will result in simultaneous reception of the same message from other mesh members.

    Image 1
    Image 2

    These IWANT requests mainly overwhelm early message receivers, -which can negatively impact message dissemination time on some occasions. -Therefore, a similar message dissemination time is seen with and without IDONTWANT messages.

    Image 1
    Image 2
    Image 3

    Similar results are seen on our large-scale deployment runs -(running Waku nodes in Kubernetes).

    Please feel free to join the discussion and leave feedback regarding this post in the -VAC forum.

    References​

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Libp2p GossipSub IDONTWANT Message Performance Impact

    by
    4 min read

    This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.

    + +

    Overview​

    +

    IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. +Cutting down on duplicates can potentially render two significant advantages:

    +
      +
    1. +

      Reducing bandwidth requirements

      +
      2. +
    2. +

      Reducing message dissemination time (latency)

      +
      3. +
    +

    For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message.

    +

    Duplicates investigation reveals that +the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network.

    +

    This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, +showing a potential for a significant drop in bandwidth utilization. +On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.

    +

    Experiments​

    +

    We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) +to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time.

    +

    The experiments are performed on nim-libp2p using the shadow simulator. +The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages.

    +

    In all experiments, ten messages are transmitted to the network, i.e., +ten peers (publishers) are selected as the message transmitters. +Every publisher transmits exactly one message, +and inter-packet spacing (delay) is set to four seconds for each published message. +For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class.

    +

    At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}. +These messages are not included in latency computations.

    +

    The simulation details are presented in the table below.

    +
    ParameterValueParameterValue
    Peers2000Publishers10
    Peer bandwidth50-150 MbpsLink latency40-130 ms
    Message size1KB, 50KB, 500KB, 1MBDD8
    Heartbeat interval700ms, 1000ms, 1500msDlowD_{low}6
    FloodPublishFalseDhighD_{high}12
    Gossip factor0.05Muxeryamux
    +

    Findings​

    +

    We use bandwidth utilization and latency as evaluation metrics. +Bandwidth utilization represents total network-wide traffic (including gossip and other control messages). +Latency refers to network-wide message dissemination time. +The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights.

    +

    Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. +A higher drop is seen with a higher heartbeat interval. +Interestingly, a relatively low bandwidth reduction (12-20%) is seen for 1MB messages, +compared to 500KB messages (18-21%).

    +
    Image 1
    Image 2
    Image 3
    +

    This is because downloading a large message may consume several hundred milliseconds. +During this time, a receiver will likely +generate multiple IWANT requests +for the same message, increasing bandwidth utilization.

    +

    Moreover, a peer can generate +IDONTWANTs only after it has finished downloading the message. +A longer download time will result in simultaneous reception of the same message from other mesh members.

    +
    Image 1
    Image 2
    +

    These IWANT requests mainly overwhelm early message receivers, +which can negatively impact message dissemination time on some occasions. +Therefore, a similar message dissemination time is seen with and without IDONTWANT messages.

    +
    Image 1
    Image 2
    Image 3
    +

    Similar results are seen on our large-scale deployment runs +(running Waku nodes in Kubernetes).

    +

    Please feel free to join the discussion and leave feedback regarding this post in the +VAC forum.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/gsub-largemsg-improvements/index.html b/rlog/gsub-largemsg-improvements/index.html index 6247e776..ec47dc8c 100644 --- a/rlog/gsub-largemsg-improvements/index.html +++ b/rlog/gsub-largemsg-improvements/index.html @@ -1,195 +1,382 @@ - + - -Large Message Handling in GossipSub: Potential Improvements | Vac Research + +Large Message Handling in GossipSub: Potential Improvements | Vac Research - - - - + + + + -
    -

    Large Message Handling in GossipSub: Potential Improvements

    by
    20 min read

    Large Message Handling in GossipSub: Potential Improvements

    Motivation​

    The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). -It is particularly relevant for applications that require on-time, network-wide dissemination of large messages, -such as Ethereum and Waku [1,2].

    This matter has been extensively discussed in the libp2p community [3, -4, -5, -6, -7, -8], -and numerous improvements have been considered (or even incorporated) for the GossipSub protocol to enable efficient large-message propagation -[3, -7, -9, -10].

    Problem description​

    Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\lceil \log_D(N) \rceil rounds, -with approximately (D−1)X−1×D(D-1)^{X-1} \times D transmissions in each round, -where X,DX, D represent the round number and mesh size.

    Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \times (F+D), -where FF represents the number of peers included in floodpublish.

    This arrangement works fine for relatively small/moderate message sizes. -However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen.

    Interestingly, a higher DD or FF can also degrade performance in this situation.

    Several aspects contribute to this behavior:

    1. Ideally, a message transmission to a single peer concludes in τ1=LR+P\tau_1 = \frac {L}{R}+P (ignoring any message processing time), -where L,R,PL, R, P represent message size, data rate, and link latency. -Therefore, the time required for sending a message on a 100Mbps link with 100ms latency -jumps from τ110KB=100.8ms\tau_1^{10KB} = 100.8ms for a 10KB message to τ11MB=180ms\tau_1^{1MB} = 180ms for a 1MB message. -For DD peers, the transmission time increases to τD1MB=(80×D)+100ms\tau_D^{1MB} = (80 \times D) + 100ms, -triggering additional queuing delays (proportional to the transmission queue size) during each transmission round.

    2. In practice, τ11MB\tau_1^{1MB} sometimes rises to several hundred milliseconds, -further exaggerating the abovementioned queuing delays. -This rise is because TCP congestion avoidance algorithms usually limit maximum in-flight bytes in a round trip time (RTT) -based on the congestion window (CwndC_{wnd}) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \times MSS}, -with CwndC_{wnd} rising with the data transfer for each flow. -Consequently, sending the same message through a newly established (cold) connection takes longer. -The message transfer time lowers as the CwndC_{wnd} grows. -Therefore, performance-improvement practices such as floodpublish, frequent mesh adjustment, and lazy sending -typically result in longer than expected message dissemination times for large messages (due to cold connections). -It is also worth mentioning that some TCP variants reset their CwndC_{wnd} after different periods of inactivity.

    3. Theoretically, the message transmission time to D peers (τD)(\tau_D) remains the same -even if the message is relayed sequentially to all peers or simultaneous transmissions are carried out, -i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i -However, sequential transmissions finish early for individual peers, allowing them to relay early. -This may result in quicker network-wide message dissemination.

    1. A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). -As the message disseminates, it's not uncommon for some peers to receive it much earlier than others. -Early gossip (IHAVE announcements) may bring in many IWANT requests to the early receivers (even from peers already receiving the same message), -which adds to their workload.

    2. A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. -As a result, already scheduled messages are prioritized over those published by the peer itself, -introducing a significant initial delay to the locally published messages. -Enqueuing IWANT replies to the outgoing message queue can further exaggerate the problem. -The lack of adaptiveness and standardization in outgoing message prioritization are key factors that can lead to noticeable inconsistency -in message dissemination latency at each hop, even in similar network conditions.

    3. Message size directly contributes to peers' workloads in terms of processing and transmission time. -It also raises the probability of simultaneous redundant transmissions to the same peer, -resulting in bandwidth wastage, congestion, and slow message propagation to the network. -Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers.

    4. Most use cases necessitate validating received messages before forwarding them to the next-hop peers. -For a higher message transfer time (τ)(\tau ), this store-and-forward delay accumulates across the hops traveled by the message.

    Possible improvements​

    1. Minimizing transfer time for large messages​

    The impact of message size and achievable data rate on message transmit time τ\tau is crucial -as this time accumulates due to the store-and-forward delay introduced at intermediate hops.

    Some possible improvements to minimize overall message dissemination latency include:

    a. Message fragmentation​

    In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) -can be simplified to roughly δ≈δTx+Ph\delta \approx \delta_{Tx} + P_h, where δTx\delta_{Tx} represents accumulative message transmit time denoted as δTx=SR×h\delta_{Tx} = \frac{S}{R} \times h, -with S,RS, R being the data size and data rate, -and h,Phh, P_h being the number of hops in the longest path and message propagation time through the longest path.

    Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\frac{\delta_{Tx}}{n}. -As a received fragment can be immediately relayed by the receiver (while the sender is still transmitting the remaining fragments), -it reduces the transmit time to δTx=SR×2h−1n\delta_{Tx} = \frac{S}{R} \times \frac{2h-1}{n}.

    This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions.

    However, it is worth noting that many applications require each fragment to be individually verifiable. -At the same time, message fragmentation allows a malicious peer to never relay some fragments of a message, -which can lead to a significant rise in the application's receive buffer size.

    Therefore, message fragmentation requires a careful tradeoff analysis between time and risks.

    b. Message staggering​

    Considering the same bandwidth, the time τD\tau_D required for sending a message to D peers stays the same, -even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i.

    However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\tau_1 \approx \frac{\tau_D}{D}) due to bandwidth concentration for a particular peer. -So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers.

    As a result, after every τDD\frac{\tau_D}{D} milliseconds, -the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\ \forall\ X \in \{0, D-1\} and by ∑k=X−DX−1λk ∀ X≥D\sum_{k=X-D}^{X-1} \lambda_k\ \forall\ X \geq D. -Here, XX represents message transmission round X=i⋅τDD∣i∈N0X = i \cdot \frac{\tau_D}{D} \mid i \in \mathbb{N}_0, and λk\lambda_k represents the number of peers that received the message in round kk.

    It is worth noting that a realistic network imposes certain constraints on staggering for peers. -For instance, in a network with dissimilar peer capabilities, -placing a slow peer (also in cases where many senders simultaneously select the same peer) at the head of the transmission queue -may result in head-of-line blocking for the message queue.

    At the same time, early receivers get many IWANT requests, increasing their workload.

    c. Message prioritization for slow senders​

    A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. -Any new message transmission at this stage (especially the locally published messages) gets delayed. -Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers.

    For instance, any GossipSub peer will likely receive every message from multiple senders, -leading to redundant transmissions [11]. -Implementing efficient strategies (only for slow senders) like lazy sending -and prioritizing locally published messages/IWANT replies over already queued messages -can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers.

    A peer can identify itself as a slow peer by using any bandwidth estimation approach -or simply setting an outgoing message queue threshold for all mesh members.

    Eliminating/deprioritizing some messages can lower a peer's score, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Large Message Handling in GossipSub: Potential Improvements

    by
    20 min read

    Large Message Handling in GossipSub: Potential Improvements

    + +

    Motivation​

    +

    The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). +It is particularly relevant for applications that require on-time, network-wide dissemination of large messages, +such as Ethereum and Waku [1,2].

    +

    This matter has been extensively discussed in the libp2p community [3, +4, +5, +6, +7, +8], +and numerous improvements have been considered (or even incorporated) for the GossipSub protocol to enable efficient large-message propagation +[3, +7, +9, +10].

    +

    Problem description​

    +

    Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\lceil \log_D(N) \rceil rounds, +with approximately (D−1)X−1×D(D-1)^{X-1} \times D transmissions in each round, +where X,DX, D represent the round number and mesh size.

    +

    Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \times (F+D), +where FF represents the number of peers included in floodpublish.

    +

    This arrangement works fine for relatively small/moderate message sizes. +However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen.

    +

    Interestingly, a higher DD or FF can also degrade performance in this situation.

    +

    Several aspects contribute to this behavior:

    +
      +
    1. +

      Ideally, a message transmission to a single peer concludes in τ1=LR+P\tau_1 = \frac {L}{R}+P (ignoring any message processing time), +where L,R,PL, R, P represent message size, data rate, and link latency. +Therefore, the time required for sending a message on a 100Mbps link with 100ms latency +jumps from τ110KB=100.8ms\tau_1^{10KB} = 100.8ms for a 10KB message to τ11MB=180ms\tau_1^{1MB} = 180ms for a 1MB message. +For DD peers, the transmission time increases to τD1MB=(80×D)+100ms\tau_D^{1MB} = (80 \times D) + 100ms, +triggering additional queuing delays (proportional to the transmission queue size) during each transmission round.

      +
      2. +
    2. +

      In practice, τ11MB\tau_1^{1MB} sometimes rises to several hundred milliseconds, +further exaggerating the abovementioned queuing delays. +This rise is because TCP congestion avoidance algorithms usually limit maximum in-flight bytes in a round trip time (RTT) +based on the congestion window (CwndC_{wnd}) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \times MSS}, +with CwndC_{wnd} rising with the data transfer for each flow. +Consequently, sending the same message through a newly established (cold) connection takes longer. +The message transfer time lowers as the CwndC_{wnd} grows. +Therefore, performance-improvement practices such as floodpublish, frequent mesh adjustment, and lazy sending +typically result in longer than expected message dissemination times for large messages (due to cold connections). +It is also worth mentioning that some TCP variants reset their CwndC_{wnd} after different periods of inactivity.

      +
      3. +
    3. +

      Theoretically, the message transmission time to D peers (τD)(\tau_D) remains the same +even if the message is relayed sequentially to all peers or simultaneous transmissions are carried out, +i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i +However, sequential transmissions finish early for individual peers, allowing them to relay early. +This may result in quicker network-wide message dissemination.

      +
      4. +
    4. +

      A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). +As the message disseminates, it's not uncommon for some peers to receive it much earlier than others. +Early gossip (IHAVE announcements) may bring in many IWANT requests to the early receivers (even from peers already receiving the same message), +which adds to their workload.

      +
      5. +
    5. +

      A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. +As a result, already scheduled messages are prioritized over those published by the peer itself, +introducing a significant initial delay to the locally published messages. +Enqueuing IWANT replies to the outgoing message queue can further exaggerate the problem. +The lack of adaptiveness and standardization in outgoing message prioritization are key factors that can lead to noticeable inconsistency +in message dissemination latency at each hop, even in similar network conditions.

      +
      6. +
    6. +

      Message size directly contributes to peers' workloads in terms of processing and transmission time. +It also raises the probability of simultaneous redundant transmissions to the same peer, +resulting in bandwidth wastage, congestion, and slow message propagation to the network. +Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers.

      +
      7. +
    7. +

      Most use cases necessitate validating received messages before forwarding them to the next-hop peers. +For a higher message transfer time (τ)(\tau ), this store-and-forward delay accumulates across the hops traveled by the message.

      +
      8. +
    +

    Possible improvements​

    +

    1. Minimizing transfer time for large messages​

    +

    The impact of message size and achievable data rate on message transmit time τ\tau is crucial +as this time accumulates due to the store-and-forward delay introduced at intermediate hops.

    +

    Some possible improvements to minimize overall message dissemination latency include:

    +

    a. Message fragmentation​

    +

    In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) +can be simplified to roughly δ≈δTx+Ph\delta \approx \delta_{Tx} + P_h, where δTx\delta_{Tx} represents accumulative message transmit time denoted as δTx=SR×h\delta_{Tx} = \frac{S}{R} \times h, +with S,RS, R being the data size and data rate, +and h,Phh, P_h being the number of hops in the longest path and message propagation time through the longest path.

    +

    Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\frac{\delta_{Tx}}{n}. +As a received fragment can be immediately relayed by the receiver (while the sender is still transmitting the remaining fragments), +it reduces the transmit time to δTx=SR×2h−1n\delta_{Tx} = \frac{S}{R} \times \frac{2h-1}{n}.

    +

    This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions.

    +

    However, it is worth noting that many applications require each fragment to be individually verifiable. +At the same time, message fragmentation allows a malicious peer to never relay some fragments of a message, +which can lead to a significant rise in the application's receive buffer size.

    +

    Therefore, message fragmentation requires a careful tradeoff analysis between time and risks.

    +

    b. Message staggering​

    +

    Considering the same bandwidth, the time τD\tau_D required for sending a message to D peers stays the same, +even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i.

    +

    However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\tau_1 \approx \frac{\tau_D}{D}) due to bandwidth concentration for a particular peer. +So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers.

    +

    As a result, after every τDD\frac{\tau_D}{D} milliseconds, +the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\ \forall\ X \in \{0, D-1\} and by ∑k=X−DX−1λk ∀ X≥D\sum_{k=X-D}^{X-1} \lambda_k\ \forall\ X \geq D. +Here, XX represents message transmission round X=i⋅τDD∣i∈N0X = i \cdot \frac{\tau_D}{D} \mid i \in \mathbb{N}_0, and λk\lambda_k represents the number of peers that received the message in round kk.

    +

    It is worth noting that a realistic network imposes certain constraints on staggering for peers. +For instance, in a network with dissimilar peer capabilities, +placing a slow peer (also in cases where many senders simultaneously select the same peer) at the head of the transmission queue +may result in head-of-line blocking for the message queue.

    +

    At the same time, early receivers get many IWANT requests, increasing their workload.

    +

    c. Message prioritization for slow senders​

    +

    A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. +Any new message transmission at this stage (especially the locally published messages) gets delayed. +Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers.

    +

    For instance, any GossipSub peer will likely receive every message from multiple senders, +leading to redundant transmissions [11]. +Implementing efficient strategies (only for slow senders) like lazy sending +and prioritizing locally published messages/IWANT replies over already queued messages +can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers.

    +

    A peer can identify itself as a slow peer by using any bandwidth estimation approach +or simply setting an outgoing message queue threshold for all mesh members.

    +

    Eliminating/deprioritizing some messages can lower a peer's score, but it also earns the peer an overall better score by achieving some early message transfers.
    -For instance, sending many near-first messages can only save a peer from a deficit penalty. -On the other hand, sending only one message (assuming MeshMessageDeliveriesThreshold defaults to 1) -as the first delivered message can add to the accumulative peer score.

    2. Mitigating transport issues​

    Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time -as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd}) size.

    Rapid adaptation of CwndC_{wnd} to the available network conditions can help lower message dissemination latency.

    Therefore, selecting a more suitable TCP variant like BBR, -which is known for its ability to dynamically adjust the congestion window based on network conditions, -can significantly enhance GossipSub's performance.

    At the same time, parameters like receive window scaling and initial CwndC_{wnd} also impact message transfer time, -but these are usually OS-specific system-wide choices.

    One possible solution is to raise CwndC_{wnd} by exchanging data over the newly established connection. -This data may involve useful details like peer exchange information and gossip to build initial trust, -or GossipSub can use some dummy data to raise CwndC_{wnd} to a reasonable level.

    It's important to understand that some TCP variants reset CwndC_{wnd} after specific periods of inactivity [12]. -This can lead to a decline in TCP's performance for applications -that generate traffic after intervals long enough to trigger the resetting of the congestion window.

    Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13].

    The limitations faced with CwndC_{wnd} scaling also impact some performance optimizations in GossipSub. -For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency.

    However, a small CwndC_{wnd} value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time -[4]. -Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth.

    The same is the case with IWANT replies.

    Maintaining a bigger mesh (with warm TCP connections) and relaying to DD peers can be a better alternative to this problem.

    3. Eliminating redundant transmissions​

    For every received packet, a peer makes roughly DD transmissions to contribute its fair share to the spread of messages. -However, the fact that many recipients had already received the message (from some other peer) -makes this message propagation inefficient.

    Although the DD-spread is attributed to quicker dissemination and resilience against non-conforming peers, -many potential solutions can still minimize redundant transmissions -while preserving the resilience of GossipSub.

    These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, -not only address the issue of redundancy but also provide an opportunity for bandwidth optimization, -especially for resource-constrained peers.

    For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], -can significantly reduce redundant transmissions.

    It allows any node to notify its mesh members that it has already received a message, -thereby preventing them from resending the same message. -This functionality is useful when a node receives a message larger than a specified threshold.

    In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages.

    It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, -i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message.

    This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience.

    However, the use of IDONTWANT messages alone has an inherent limitation. -For instance, a peer can only send an IDONTWANT after receiving the complete message.

    A large message transmission consumes significant time. -For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd} and latency).

    As a result, other mesh members may also start transmitting the same message during this interval. -A few potential solutions include:

    a. Staggering with IDONTWANT messages​

    As previously discussed, staggering can significantly reduce network-wide message dissemination latency. -This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach.

    Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. -This is because a node only saturates its bandwidth for a small subset of mesh peers, -leading to early transmissions and prompt IDONTWANT message notifications to the mesh members.

    It is worth highlighting that staggering can be implemented in various ways.

    For example, it can be applied to peers (peer staggering) -where a node sequentially relays the same message to all peers one by one.

    Alternatively, a node can send a different message to every peer (message staggering or rotational sending), -allowing IDONTWANTs for other messages to arrive during this time. -The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time.

    As the peers in staggered sending are sequentially covered -(with a faster speed due to bandwidth concentration), this leads to another problem.

    The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. -IHAVE announcements for newly received large messages trigger IWANTs from nodes -(including those already receiving the same message), -leading to an additional workload for early receivers [14].

    Potential solutions to mitigate these problems include:

    1) Defering IHAVE announcements for large messages.

    Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. -However, deciding on a suitable deferred interval is crucial for optimal performance. -One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers.

    2) Defering IWANT requests for messages that are currently being received.

    This requires prior knowledge of msgIDs for the messages under reception. -Knowing the message length is also essential in deciding a suitable defer interval -to handle situations where a sender starts sending a message and never completes the transmission.

    3) Not issuing IWANT for a message if at least KK peers have transmitted IDONTWANT for the same message -(as this indicates that these peers will eventually relay this message).

    However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. -A delayed IWANT, along with negative peer scoring, can remedy this problem.

    b. IMReceiving message​

    A peer can issue an IDONTWANT only after it has received the entire message. -However, a large message transmission may take several hundred milliseconds to complete. -During this time, many other mesh members may start relaying the same message.

    Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, -significantly compromising the effectiveness of IDONTWANT messages.

    Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. -If a receiver is already receiving the same message from another sender, -the receiver can request to defer this transmission by sending a brief IMReceiving message.

    An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval.

    However, waiting for IMReceiving after sending the preamble can delay the message transmission. -On the other hand, proceeding with message transfer (after sending the preamble) leads to another problem: -it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message.

    To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), -urging other mesh peers to defer sending the same message [15, -16].

    The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. -This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages.

    While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, -it's crucial to be aware of the potential risks.

    A malicious user can exploit this approach to disrupt message transmission -either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers.

    This could ultimately result in network-wide slow message propagation.

    However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks.

    c. IDONTWANT message with reduced forwarding​

    It is common for slow peers to pile up outgoing message queues, -especially for large message transfers. -This results in a significant queuing delay for outgoing messages. -Reduced message forwarding can help decrease the workload of slower peers.

    On receiving a message longer than the specified threshold, -a slow peer can relay it to only K∈DK \in D peers and send an IDONTWANT message to all the peers in DD.

    In this arrangement, the IDONTWANT message serves an additional purpose: -to promptly announce data availability, reinforcing redundancy in the presence of adversaries.

    When a peer receives an IDONTWANT for an unseen message, -it learns about the new message and can request it by sending an IWANT request without waiting for the heartbeat (gossip) interval. -As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network.

    This approach conserves peer bandwidth by minimizing redundant transmissions -while ensuring GossipSub resilience at the cost of one RTT (for missing peers).

    Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages).

    However, finding an appropriate value for KK is crucial for optimal performance. -A smaller KK saves peer bandwidth, while a larger KK achieves quicker spread until outgoing message queues pile up. -Setting K=DlowK = D_{low} can be one option.

    It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-K peers). -However, a minimized workload enables early message dissemination to the remaining peers. -These early transmissions and randomized KK set selection can help achieve an overall better peer score.

    4. Message prioritization​

    Despite the standardized specifications of the GossipSub protocol, -the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization.

    It is worth mentioning that every node is responsible for transmitting different types of packets, -including control messages, locally published messages, messages received from mesh members, IWANT replies, etc.

    As long as traffic volume is lower than the available data rate, -the message forwarding mechanisms yield similar results due to negligible queuing delays.

    However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), -the outgoing message queue(s) sizes rise, potentially impacting the network's performance.

    In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, -introducing a queuing delay proportional to the queue size. -The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc.

    On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. -One possible solution is to use weighted queues for a fair spread of messages.

    Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time -and allow for customizable message handling.

    For example, staggering between peers and messages can be better managed by using priority queues. -However, it is important to note that message prioritization also introduces additional complexity to the system, -necessitating sophisticated algorithms for better message handling.

    5. Maximizing benefits from IWANT messages​

    During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. -These peers can use IWANT messages to request any missing messages. -A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval.

    The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. -They bridge the information gap between nearby and far-off peers, -ensuring that information can be disseminated to peers outside the mesh. -This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks.

    However, it is essential to understand that high transmission times for large messages -require careful due diligence when using IWANT messages for reasons not limited to:

    1) A large message reception may take several hundred milliseconds to complete. -During this time, an IHAVE message announcing the same message ID will trigger an IWANT request.

    2) A peer can send IWANT requests for the same message to multiple nodes, -leading to simultaneous transmissions of the same message.

    3) Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, -resulting in lower peer scores and slower message propagation.

    A few possible solutions to mitigate this problem may include:

    1) Issuing IHAVE announcements only after the message is delivered to many mesh peers.

    2) Allocating a volume-based budget to service IWANT requests during each heartbeat interval.

    3) Deferring IWANT requests for messages that are currently being received.

    4) Deferring IWANT requests if at least KK IDONTWANTs are received for the same message.

    5) A large message transmission can yield high CwndC_{wnd}; preferring such peers during mesh maintenance can be helpful.

    Summary​

    This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages.

    We delve into multiple factors, -such as increased message transmit times, store-and-forward delays, congestion avoidance mechanisms, and prioritization between messages, -to establish a comprehensive understanding of the problem.

    The study also explores the performance of optimization efforts -like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions.

    A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. -Coupling this constraint with increased message transmission times -results in notable store-and-forward delays accumulating at each hop.

    Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation -by utilizing a considerable share of available bandwidth on redundant transmissions.

    Therefore, approaches focused on eliminating redundant transmissions -(IDONTWANT, IMReceiving, lazy sending, etc.) can prove helpful. -At the same time, strategies aimed at reducing store-and-forward delays -(fragmentation, staggering, prioritization, etc.) can prove beneficial.

    It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. -Some of these have already been explored and discussed to some extent [5, -17, -18]. -We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings.

    Please feel free to join the discussion and leave feedback regarding this post in the -VAC forum.

    References​

    [1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844

    [2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/

    [3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666

    [4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383

    [5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw

    [6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850

    [7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911

    [8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859

    [9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh

    [10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52

    [11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921

    [12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1

    [13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558

    [14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101

    [15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281

    [16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283

    [17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366

    [18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/

    - - +For instance, sending many near-first messages can only save a peer from a deficit penalty. +On the other hand, sending only one message (assuming MeshMessageDeliveriesThreshold defaults to 1) +as the first delivered message can add to the accumulative peer score.

    +

    2. Mitigating transport issues​

    +

    Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time +as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd}) size.

    +

    Rapid adaptation of CwndC_{wnd} to the available network conditions can help lower message dissemination latency.

    +

    Therefore, selecting a more suitable TCP variant like BBR, +which is known for its ability to dynamically adjust the congestion window based on network conditions, +can significantly enhance GossipSub's performance.

    +

    At the same time, parameters like receive window scaling and initial CwndC_{wnd} also impact message transfer time, +but these are usually OS-specific system-wide choices.

    +

    One possible solution is to raise CwndC_{wnd} by exchanging data over the newly established connection. +This data may involve useful details like peer exchange information and gossip to build initial trust, +or GossipSub can use some dummy data to raise CwndC_{wnd} to a reasonable level.

    +

    It's important to understand that some TCP variants reset CwndC_{wnd} after specific periods of inactivity [12]. +This can lead to a decline in TCP's performance for applications +that generate traffic after intervals long enough to trigger the resetting of the congestion window.

    +

    Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13].

    +

    The limitations faced with CwndC_{wnd} scaling also impact some performance optimizations in GossipSub. +For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency.

    +

    However, a small CwndC_{wnd} value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time +[4]. +Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth.

    +

    The same is the case with IWANT replies.

    +

    Maintaining a bigger mesh (with warm TCP connections) and relaying to DD peers can be a better alternative to this problem.

    +

    3. Eliminating redundant transmissions​

    +

    For every received packet, a peer makes roughly DD transmissions to contribute its fair share to the spread of messages. +However, the fact that many recipients had already received the message (from some other peer) +makes this message propagation inefficient.

    +

    Although the DD-spread is attributed to quicker dissemination and resilience against non-conforming peers, +many potential solutions can still minimize redundant transmissions +while preserving the resilience of GossipSub.

    +

    These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, +not only address the issue of redundancy but also provide an opportunity for bandwidth optimization, +especially for resource-constrained peers.

    +

    For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], +can significantly reduce redundant transmissions.

    +

    It allows any node to notify its mesh members that it has already received a message, +thereby preventing them from resending the same message. +This functionality is useful when a node receives a message larger than a specified threshold.

    +

    In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages.

    +

    It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, +i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message.

    +

    This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience.

    +

    However, the use of IDONTWANT messages alone has an inherent limitation. +For instance, a peer can only send an IDONTWANT after receiving the complete message.

    +

    A large message transmission consumes significant time. +For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd} and latency).

    +

    As a result, other mesh members may also start transmitting the same message during this interval. +A few potential solutions include:

    +

    a. Staggering with IDONTWANT messages​

    +

    As previously discussed, staggering can significantly reduce network-wide message dissemination latency. +This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach.

    +

    Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. +This is because a node only saturates its bandwidth for a small subset of mesh peers, +leading to early transmissions and prompt IDONTWANT message notifications to the mesh members.

    +

    It is worth highlighting that staggering can be implemented in various ways.

    +

    For example, it can be applied to peers (peer staggering) +where a node sequentially relays the same message to all peers one by one.

    +

    Alternatively, a node can send a different message to every peer (message staggering or rotational sending), +allowing IDONTWANTs for other messages to arrive during this time. +The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time.

    +

    As the peers in staggered sending are sequentially covered +(with a faster speed due to bandwidth concentration), this leads to another problem.

    +

    The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. +IHAVE announcements for newly received large messages trigger IWANTs from nodes +(including those already receiving the same message), +leading to an additional workload for early receivers [14].

    +

    Potential solutions to mitigate these problems include:

    +
      +
    1. Defering IHAVE announcements for large messages.
      2. +
    +

    Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. +However, deciding on a suitable deferred interval is crucial for optimal performance. +One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers.

    +
      +
    1. Defering IWANT requests for messages that are currently being received.
      2. +
    +

    This requires prior knowledge of msgIDs for the messages under reception. +Knowing the message length is also essential in deciding a suitable defer interval +to handle situations where a sender starts sending a message and never completes the transmission.

    +
      +
    1. Not issuing IWANT for a message if at least KK peers have transmitted IDONTWANT for the same message +(as this indicates that these peers will eventually relay this message).
      2. +
    +

    However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. +A delayed IWANT, along with negative peer scoring, can remedy this problem.

    +

    b. IMReceiving message​

    +

    A peer can issue an IDONTWANT only after it has received the entire message. +However, a large message transmission may take several hundred milliseconds to complete. +During this time, many other mesh members may start relaying the same message.

    +

    Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, +significantly compromising the effectiveness of IDONTWANT messages.

    +

    Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. +If a receiver is already receiving the same message from another sender, +the receiver can request to defer this transmission by sending a brief IMReceiving message.

    +

    An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval.

    +

    However, waiting for IMReceiving after sending the preamble can delay the message transmission. +On the other hand, proceeding with message transfer (after sending the preamble) leads to another problem: +it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message.

    +

    To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), +urging other mesh peers to defer sending the same message [15, +16].

    +

    The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. +This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages.

    +

    While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, +it's crucial to be aware of the potential risks.

    +

    A malicious user can exploit this approach to disrupt message transmission +either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers.

    +

    This could ultimately result in network-wide slow message propagation.

    +

    However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks.

    +

    c. IDONTWANT message with reduced forwarding​

    +

    It is common for slow peers to pile up outgoing message queues, +especially for large message transfers. +This results in a significant queuing delay for outgoing messages. +Reduced message forwarding can help decrease the workload of slower peers.

    +

    On receiving a message longer than the specified threshold, +a slow peer can relay it to only K∈DK \in D peers and send an IDONTWANT message to all the peers in DD.

    +

    In this arrangement, the IDONTWANT message serves an additional purpose: +to promptly announce data availability, reinforcing redundancy in the presence of adversaries.

    +

    When a peer receives an IDONTWANT for an unseen message, +it learns about the new message and can request it by sending an IWANT request without waiting for the heartbeat (gossip) interval. +As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network.

    +

    This approach conserves peer bandwidth by minimizing redundant transmissions +while ensuring GossipSub resilience at the cost of one RTT (for missing peers).

    +

    Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages).

    +

    However, finding an appropriate value for KK is crucial for optimal performance. +A smaller KK saves peer bandwidth, while a larger KK achieves quicker spread until outgoing message queues pile up. +Setting K=DlowK = D_{low} can be one option.

    +

    It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-K peers). +However, a minimized workload enables early message dissemination to the remaining peers. +These early transmissions and randomized KK set selection can help achieve an overall better peer score.

    +

    4. Message prioritization​

    +

    Despite the standardized specifications of the GossipSub protocol, +the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization.

    +

    It is worth mentioning that every node is responsible for transmitting different types of packets, +including control messages, locally published messages, messages received from mesh members, IWANT replies, etc.

    +

    As long as traffic volume is lower than the available data rate, +the message forwarding mechanisms yield similar results due to negligible queuing delays.

    +

    However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), +the outgoing message queue(s) sizes rise, potentially impacting the network's performance.

    +

    In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, +introducing a queuing delay proportional to the queue size. +The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc.

    +

    On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. +One possible solution is to use weighted queues for a fair spread of messages.

    +

    Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time +and allow for customizable message handling.

    +

    For example, staggering between peers and messages can be better managed by using priority queues. +However, it is important to note that message prioritization also introduces additional complexity to the system, +necessitating sophisticated algorithms for better message handling.

    +

    5. Maximizing benefits from IWANT messages​

    +

    During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. +These peers can use IWANT messages to request any missing messages. +A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval.

    +

    The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. +They bridge the information gap between nearby and far-off peers, +ensuring that information can be disseminated to peers outside the mesh. +This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks.

    +

    However, it is essential to understand that high transmission times for large messages +require careful due diligence when using IWANT messages for reasons not limited to:

    +
      +
    1. +

      A large message reception may take several hundred milliseconds to complete. +During this time, an IHAVE message announcing the same message ID will trigger an IWANT request.

      +
      2. +
    2. +

      A peer can send IWANT requests for the same message to multiple nodes, +leading to simultaneous transmissions of the same message.

      +
      3. +
    3. +

      Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, +resulting in lower peer scores and slower message propagation.

      +
      4. +
    +

    A few possible solutions to mitigate this problem may include:

    +
      +
    1. +

      Issuing IHAVE announcements only after the message is delivered to many mesh peers.

      +
      2. +
    2. +

      Allocating a volume-based budget to service IWANT requests during each heartbeat interval.

      +
      3. +
    3. +

      Deferring IWANT requests for messages that are currently being received.

      +
      4. +
    4. +

      Deferring IWANT requests if at least KK IDONTWANTs are received for the same message.

      +
      5. +
    5. +

      A large message transmission can yield high CwndC_{wnd}; preferring such peers during mesh maintenance can be helpful.

      +
      6. +
    +

    Summary​

    +

    This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages.

    +

    We delve into multiple factors, +such as increased message transmit times, store-and-forward delays, congestion avoidance mechanisms, and prioritization between messages, +to establish a comprehensive understanding of the problem.

    +

    The study also explores the performance of optimization efforts +like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions.

    +

    A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. +Coupling this constraint with increased message transmission times +results in notable store-and-forward delays accumulating at each hop.

    +

    Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation +by utilizing a considerable share of available bandwidth on redundant transmissions.

    +

    Therefore, approaches focused on eliminating redundant transmissions +(IDONTWANT, IMReceiving, lazy sending, etc.) can prove helpful. +At the same time, strategies aimed at reducing store-and-forward delays +(fragmentation, staggering, prioritization, etc.) can prove beneficial.

    +

    It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. +Some of these have already been explored and discussed to some extent [5, +17, +18]. +We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings.

    +

    Please feel free to join the discussion and leave feedback regarding this post in the +VAC forum.

    +

    References​

    +

    [1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844

    +

    [2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/

    +

    [3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666

    +

    [4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383

    +

    [5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw

    +

    [6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850

    +

    [7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911

    +

    [8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859

    +

    [9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh

    +

    [10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52

    +

    [11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921

    +

    [12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1

    +

    [13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558

    +

    [14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101

    +

    [15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281

    +

    [16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283

    +

    [17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366

    +

    [18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/

    \ No newline at end of file diff --git a/rlog/index.html b/rlog/index.html index 7bf4745a..a67ee53b 100644 --- a/rlog/index.html +++ b/rlog/index.html @@ -1,28 +1,25 @@ - + - -Research Blog | Vac Research + +Research Blog | Vac Research - - - - + + + + -
    -

    Research Blog

    Blog

    by
    18 min read

    This article introduces MDSECheck method — a novel approach + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Research Blog

    Blog
    - - +as the components of affine permutation layers of P-SP-networks.

    \ No newline at end of file diff --git a/rlog/introducing-nwaku/index.html b/rlog/introducing-nwaku/index.html index d371b517..4e69d62c 100644 --- a/rlog/introducing-nwaku/index.html +++ b/rlog/introducing-nwaku/index.html @@ -1,32 +1,35 @@ - + - -Introducing nwaku | Vac Research + +Introducing nwaku | Vac Research - - - - + + + + -
    -

    Introducing nwaku

    by
    11 min read

    Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.

    Background​

    If you've been following our research log, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Introducing nwaku

    by
    11 min read

    Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.

    + +

    Background​

    +

    If you've been following our research log, you'll know that many things have happened in the world of Waku v2 since our last general update. In line with our long term goals, we've introduced new protocols, tweaked our existing protocols and expanded our team. We've also shown in a series of practical experiments that Waku v2 does indeed deliver on some of the theoretical advantages it was designed to have over its predecessor, Waku v1. -A sustainability and business workshop led to the formulation of a clearer vision for Vac as a team.

    From the beginning, our protocol development has been complemented by various client implementations of these protocols, +A sustainability and business workshop led to the formulation of a clearer vision for Vac as a team.

    +

    From the beginning, our protocol development has been complemented by various client implementations of these protocols, first in Nim, but later also in JavaScript and Go. @@ -37,14 +40,18 @@ and building on a home-grown implementation of a standalone adaptive node, managed by individual operators or as an encapsulated service node in other applications. -This post looks at some recent developments within the Nim client.

    1. nim-waku is now known as nwaku​

    Pronounced NWHA-koo. +This post looks at some recent developments within the Nim client.

    +

    1. nim-waku is now known as nwaku​

    +

    Pronounced NWHA-koo. You may already have seen us refer to "nwaku" on Vac communication channels, but it is now official: The nim-waku Waku v2 client has been named nwaku. Why? Well, we needed a recognizable name for our client that could easily be referred to in everyday conversations and nim-waku just didn't roll off the tongue. We've followed the example of the closely related nimbus project to find a punchier name -that explicitly links the client to both the Waku set of protocols and the Nim language.

    2. Improvements in stability and performance​

    The initial implementation of Waku v2 demonstrated how the suite of protocols can be applied +that explicitly links the client to both the Waku set of protocols and the Nim language.

    +

    2. Improvements in stability and performance​

    +

    The initial implementation of Waku v2 demonstrated how the suite of protocols can be applied to form a generalized, peer-to-peer messaging network, while addressing a wide range of adaptive requirements. This allowed us to lift several protocol specifications from raw to draft status, @@ -52,16 +59,29 @@ indicating that a reference implementation exists for each. However, as internal dogfooding increased and more external applications started using nwaku, we stepped up our focus on the client's stability and performance. This is especially true where we want nwaku to run unsupervised in a production environment -without any degradation in the services it provides.

    Some of the more significant productionization efforts over the last couple of months included:

    1. Reworking the store implementation to maintain stable memory usage +without any degradation in the services it provides.

      +

      Some of the more significant productionization efforts over the last couple of months included:

      +
        +
      1. +

        Reworking the store implementation to maintain stable memory usage while storing historical messages and serving multiple clients querying history simultaneously. Previously, a store node would see gradual service degradation due to inefficient memory usage when responding to history queries. -Queries that often took longer than 8 mins now complete in under 100 ms.

      2. Improved peer management. +Queries that often took longer than 8 mins now complete in under 100 ms.

        +
        3. +
      3. +

        Improved peer management. For example, filter nodes will now remove unreachable clients after a number of connection failures, -whereas they would previously keep accumulating dead peers.

      4. Improved disk usage. +whereas they would previously keep accumulating dead peers.

        +
        5. +
      5. +

        Improved disk usage. nwaku nodes that persist historical messages on disk now manage their own storage size based on the --store-capacity. -This can significantly improve node start-up times.

      More stability issues may be addressed in future as nwaku matures, +This can significantly improve node start-up times.

      +
      2. +
    +

    More stability issues may be addressed in future as nwaku matures, but we've noticed a marked improvement in the reliability of running nwaku nodes. These include environments where nwaku nodes are expected to run with a long uptime. Vac currently operates two long-running fleets of nwaku nodes, wakuv2.prod and wakuv2.test, @@ -70,7 +90,9 @@ to serve as experimental bootstrapping nodes. Status has also recently deployed similar fleets for production and testing based on nwaku. Our goal is to have nwaku be stable, performant and flexible enough to be an attractive option for operators to run and maintain their own Waku v2 nodes. -See also the future work section below for more on our general goal of nwaku for operators.

    3. Improvements in interoperability​

    We've implemented several features that improve nwaku's usability in different environments +See also the future work section below for more on our general goal of nwaku for operators.

    +

    3. Improvements in interoperability​

    +

    We've implemented several features that improve nwaku's usability in different environments and its interoperability with other Waku v2 clients. One major step forward here was adding support for both secure and unsecured WebSocket connections as libp2p transports. This allows direct connectivity with js-waku @@ -78,38 +100,60 @@ and paves the way for native browser usage. We've also added support for parsing and resolving DNS-type multiaddrs, i.e. multiaddress protocol schemes dns, dns4, dns6 and dnsaddr. A nwaku node can now also be configured with its own IPv4 DNS domain name -allowing dynamic IP address allocation without impacting a node's reachability by its peers.

    4. Peer discovery​

    Peer discovery is the method by which nodes become aware of each other’s existence. +allowing dynamic IP address allocation without impacting a node's reachability by its peers.

    +

    4. Peer discovery​

    +

    Peer discovery is the method by which nodes become aware of each other’s existence. The question of peer discovery in a Waku v2 network has been a focus area since the protocol was first conceptualized. Since then several different approaches to discovery have been proposed and investigated. -We've implemented three discovery mechanisms in nwaku so far:

    DNS-based discovery​

    nwaku nodes can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. -Our implementation is based on EIP-1459.

    GossipSub peer exchange​

    GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism +We've implemented three discovery mechanisms in nwaku so far:

    +

    DNS-based discovery​

    +

    nwaku nodes can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. +Our implementation is based on EIP-1459.

    +

    GossipSub peer exchange​

    +

    GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism whereby a pruning peer may provide a pruned peer with a set of alternative peers where it can connect to reform its mesh. This is a very suitable mechanism to gradually discover more peers from an initial connection to a small set of bootstrap peers. -It is enabled in a nwaku node by default.

    Waku Node Discovery Protocol v5​

    This is a DHT-based discovery mechanism adapted to store and relay node records. +It is enabled in a nwaku node by default.

    +

    Waku Node Discovery Protocol v5​

    +

    This is a DHT-based discovery mechanism adapted to store and relay node records. Our implementation is based on Ethereum's Discovery v5 protocol with some minor modifications to isolate our discovery network from that of Ethereum. The decision to separate the Waku Discovery v5 network from Ethereum's was made on considerations of lookup efficiency. This comes at a possible tradeoff in network resilience. We are considering merging with the Ethereum Discovery v5 network in future, or even implement a hybrid solution. -This post explains the decision and future steps.

    5. Spam protection using RLN​

    An early addition to our suite of protocols was an extension of 11/WAKU-RELAY +This post explains the decision and future steps.

    +

    5. Spam protection using RLN​

    +

    An early addition to our suite of protocols was an extension of 11/WAKU-RELAY that provided spam protection using Rate Limiting Nullifiers (RLN). The nwaku client now contains a working demonstration and integration of RLN relay. Check out this tutorial to see the protocol in action using a toy chat application built on nwaku. We'd love for people to join us in dogfooding RLN spam protection as part of our operator incentive testnet. Feel free to join our Vac Discord server -and head to the #rln channel for more information.

    Future work​

    As we continue working towards our goal of a fully decentralized, generalized and censorship-resistant messaging protocol, -these are some of the current and future focus areas for nwaku:

    Reaching out to operators:​

    We are starting to push for operators to run and maintain their own Waku v2 nodes, +and head to the #rln channel for more information.

    +

    Future work​

    +

    As we continue working towards our goal of a fully decentralized, generalized and censorship-resistant messaging protocol, +these are some of the current and future focus areas for nwaku:

    +

    Reaching out to operators:​

    +

    We are starting to push for operators to run and maintain their own Waku v2 nodes, preferably contributing to the default Waku v2 network as described by the default pubsub topic (/waku/2/default-waku/proto). Amongst other things, a large fleet of stable operator-run Waku v2 nodes will help secure the network, provide valuable services to a variety of applications -and ensure the future sustainability of both Vac as a research organization and the Waku suite of protocols.

    We are targeting nwaku as the main option for operator-run nodes.
    -Specifically, we aim to provide through nwaku:

    1. a lightweight and robust Waku v2 client. +and ensure the future sustainability of both Vac as a research organization and the Waku suite of protocols.

      +

      We are targeting nwaku as the main option for operator-run nodes.
      +Specifically, we aim to provide through nwaku:

      +
        +
      1. a lightweight and robust Waku v2 client. This client must be first in line to support innovative and new Waku v2 protocols, -but configurable enough to serve the adaptive needs of various operators.
      2. an easy-to-follow guide for operators to configure, -set up and maintain their own nodes
      3. a set of operator-focused tools to monitor and maintain a running node

      Better conversational security layer guarantees​

      Conversational security guarantees in Waku v2 are currently designed around the Status application. +but configurable enough to serve the adaptive needs of various operators.

      2. +
    2. an easy-to-follow guide for operators to configure, +set up and maintain their own nodes
      3. +
    3. a set of operator-focused tools to monitor and maintain a running node
      4. +
    +

    Better conversational security layer guarantees​

    +

    Conversational security guarantees in Waku v2 are currently designed around the Status application. Developers building their own applications on top of Waku would therefore either have to reimplement a set of tools similar to Status or build their own security solutions on the application layer above Waku. @@ -117,7 +161,9 @@ We are working on integrating noise handshakes as a key exchange mechanism in Waku v2.

    Protocol incentivization​

    We want to design incentivization around our protocols to encourage desired behaviors in the Waku network, +As a first step, we've already made good progress toward integrating noise handshakes as a key exchange mechanism in Waku v2.

    +

    Protocol incentivization​

    +

    We want to design incentivization around our protocols to encourage desired behaviors in the Waku network, rewarding nodes providing costly services and punishing adversarial actions. This will increase the overall security of the network @@ -132,7 +178,9 @@ After this, we want to design monetary incentivization for providers of st This may also tie into a reputation mechanism for service nodes based on a network-wide consensus on service quality. A big challenge for protocol incentivization is doing it in a private fashion, so we can keep similar metadata protection guarantees as the Waku base layer. -This ties into our focus on Zero Knowledge tech.

    Improved store capacity​

    The nwaku store currently serves as an efficient in-memory store for historical messages, +This ties into our focus on Zero Knowledge tech.

    +

    Improved store capacity​

    +

    The nwaku store currently serves as an efficient in-memory store for historical messages, dimensioned by the maximum number of messages the store node is willing to keep. This makes the nwaku store appropriate for keeping history over a short term without any time-based guarantees, @@ -142,31 +190,77 @@ with time-based dimensioning to guarantee that messages will be stored for a specified minimum period. Because of the relatively high cost of memory compared to disk space, a higher capacity store, with time guarantees, should operate as a disk-only database of historical messages. -This is an ongoing effort.

    Multipurpose discovery​

    In addition to the three discovery methods already implemented in nwaku, -we are working on improving discovery on at least three fronts:

    Capability discovery:​

    Waku v2 nodes may be interested in peers with specific capabilities, for example:

    1. peers within a specific pubsub topic mesh,
    2. peers with store capability,
    3. store peers with x days of history for a specific content topic, etc.

    Capability discovery entails mechanisms by which such capabilities can be advertised and discovered/negotiated. +This is an ongoing effort.

    +

    Multipurpose discovery​

    +

    In addition to the three discovery methods already implemented in nwaku, +we are working on improving discovery on at least three fronts:

    +

    Capability discovery:​

    +

    Waku v2 nodes may be interested in peers with specific capabilities, for example:

    +
      +
    1. peers within a specific pubsub topic mesh,
      2. +
    2. peers with store capability,
      3. +
    3. store peers with x days of history for a specific content topic, etc.
      4. +
    +

    Capability discovery entails mechanisms by which such capabilities can be advertised and discovered/negotiated. One major hurdle to overcome is the increased complexity of finding a node with specific capabilities within the larger network (a needle in a haystack). -See the original problem statement for more.

    Improvements in Discovery v5​

    Of the implemented discovery methods, +See the original problem statement for more.

    +

    Improvements in Discovery v5​

    +

    Of the implemented discovery methods, Discovery v5 best addresses our need for a decentralized and scalable discovery mechanism. With the basic implementation done, there are some improvements planned for Discovery v5, including methods to increase security such as merging with the Ethereum Discovery v5 network, introducing explicit NAT traversal and utilizing topic advertisement. -The Waku v2 Discovery v5 Roadmap contains more details.

    Generalized peer exchange​

    nwaku already implements GossipSub peer exchange. +The Waku v2 Discovery v5 Roadmap contains more details.

    +

    Generalized peer exchange​

    +

    nwaku already implements GossipSub peer exchange. We now need a general request-response mechanism outside of GossipSub by which a node may learn about other Waku v2 nodes by requesting and receiving a list of peers from a neighbor. This could, for example, be a suitable way for resource-restricted devices to request a stronger peer to perform a random Discovery v5 lookup on their behalf or simply to be informed of a subset of the peers known to that neighbor. -See this issue for more.

    This concludes a general outline of some of the main recent developments in the nwaku client +See this issue for more.

    +
    +

    This concludes a general outline of some of the main recent developments in the nwaku client and a summary of the current and future focus areas. Much more is happening behind the scenes, of course, so for more information, or to join the conversation, feel free to join our Vac Discord server or to check out the nwaku repo on Github. -You can also view the changelog for past releases here.

    References​

    - - +You can also view the changelog for past releases here.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/kademlia-to-discv5/index.html b/rlog/kademlia-to-discv5/index.html index af569d16..53af77e6 100644 --- a/rlog/kademlia-to-discv5/index.html +++ b/rlog/kademlia-to-discv5/index.html @@ -1,26 +1,82 @@ - + - -From Kademlia to Discv5 | Vac Research + +From Kademlia to Discv5 | Vac Research - - - - + + + + -
    -

    From Kademlia to Discv5

    by
    9 min read

    A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.

    If you've been working on Ethereum or adjacent technologies you've probably heard of discv4 or discv5. But what are they actually? How do they work and what makes them different? To answer these questions, we need to start at the beginning, so this post will assume that there is little knowledge on the subject so the post should be accessible for anyone.

    The Beginning​

    Let's start right at the beginning: the problem of discovery and organization of nodes in peer-to-peer networks.

    Early P2P file sharing technologies, such as Napster, would share information about who holds what file using a single server. A node would connect to the central server and give it a list of the files it owns. Another node would then connect to that central server, find a node that has the file it is looking for and contact that node. This however was a flawed system -- it was vulnerable to attacks and left a single party open to lawsuits.

    It became clear that another solution was needed, and after years of research and experimentation, we were given the distributed hash table or DHT.

    Distributed Hash Tables​

    In 2001 4 new protocols for such DHTs were conceived, Tapestry, Chord, CAN and Pastry, all of which made various trade-offs and changes in their core functionality, giving them unique characteristics.

    But as said, they're all DHTs. So what is a DHT?

    A distributed hash table (DHT) is essentially a distributed key-value list. Nodes participating in the DHT can easily retrieve the value for a key.

    If we have a network with 9 key-value pairs and 3 nodes, ideally each node would store 3 (optimally 6 for redundancy) of those key-value pairs, meaning that if a key-value pair were to be updated, only part of the network would responsible for ensuring that it is. The idea is that any node in the network would know where to find the specific key-value pair it is looking for based on how things are distributed amongst the nodes.

    Kademlia​

    So now that we know what DHTs are, let's get to Kademlia, the predecessor of discv4. Kademlia was created by Petar Maymounkov and David Mazières in 2002. I will naively say that this is probably one of the most popular and most used DHT protocols. It's quite simple in how it works, so let's look at it.

    In Kademlia, nodes and values are arranged by distance (in a very mathematical definition). This distance is not a geographical one, but rather based on identifiers. It is calculated how far 2 identifiers are from eachother using some distance function.

    Kademlia uses an XOR as its distance function. An XOR is a function that outputs true only when inputs differ. Here is an example with some binary identifiers:

    XOR 10011001
        00110010
        --------
        10101011

    The top in decimal numbers means that the distance between 153 and 50 is 171.

    There are several reasons why XOR was taken:

    1. The distance from one ID to itself will be 0.
    2. Distance is symmetric, A to B is the same as B to A.
    3. Follows triangle inequality, if A, B and C are points on a triangle then the distance A to B is closer or equal to that of A to C plus the one from B to C.

    In summary, this distance function allows a node to decide what is "close" to it and make decisions based on that "closeness".

    Kademlia nodes store a routing table. This table contains multiple lists. Each subsequent list contains nodes which are a little further distanced than the ones included in the previous list. Nodes maintain detailed knowledge about nodes closest to them, and the further away a node is, the less knowledge the node maintains about it.

    So let's say I want to find a specific node. What I would do is go to any node which I already know and ask them for all their neighbours closest to my target. I repeat this process for the returned neighbours until I find my target.

    The same thing happens for values. Values have a certain distance from nodes and their IDs are structured the same way so we can calculate this distance. If I want to find a value, I simply look for the neighbours closest to that value's key until I find the one storing said value.

    For Kademlia nodes to support these functions, there are several messages with which the protocol communicates.

    • PING - Used to check whether a node is still running.
    • STORE - Stores a value with a given key on a node.
    • FINDNODE - Returns the closest nodes requested to a given ID.
    • FINDVALUE - The same as FINDNODE, except if a node stores the specific value it will return it directly.

    This is a very simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the paper or a more in-depth design specification

    Discv4​

    Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT.

    Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the FINDVALUE and STORE commands described by Kademlia.

    The lookup method previously described by Kademlia describes how a node gets its peers. A node contacts some node and asks it for the nodes closest to itself. It does so until it can no longer find any new nodes.

    Additionally, discv4 adds mutual endpoint verification. This is meant to ensure that a peer calling FINDNODE also participates in the discovery protocol.

    Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called ENRRequest.

    If you want some more details on ENRs, check out one of my posts "Network Addresses in Ethereum"

    Discv4 comes with its own range of problems however. Let's look at a few of them.

    Firstly, the way discv4 works right now, there is no way to differentiate between node sub-protocols. This means for example that an Ethereum node could add an Ethereum Classic Node, Swarm or Whisper node to its DHT without realizing that it is invalid until more communication has happened. This inability to differentiate sub-protocols makes it harder to find specific nodes, such as Ethereum nodes with light-client support.

    Next, in order to prevent replay attacks, discv4 uses timestamps. This however can lead to various issues when a host's clock is wrong. For more details, see the "Known Issues" section of the discv4 specification.

    Finally, we have an issue with the way mutual endpoint verification works. Messages can get dropped and there is no way to tell if both peers have verified eachother. This means that we could consider our peer verified while it does not consider us so making them drop the FINDNODE packet.

    Discv5​

    Finally, let's look at discv5. The next iteration of discv4 and the discovery protocol which will be used by Eth 2.0. It aims at fixing various issues present in discv4.

    The first change is the way FINDNODE works. In traditional Kademlia as well as in discv5, we pass an identifier. However, in discv5 we instead pass the logarithmic distance, meaning that a FINDNODE request gets a response containing all nodes at the specified logarithmic distance from the called node.

    Logarithmic distance means we first calculate the distance and then run it through our log base 2 function. See:

    log2(A xor B)

    And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are first in first out lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering ads on their peers.

    As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place ads on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network.

    Additionally, it is unclear how many peers a node should place these ads on and exactly which peers to place them on. For more details, check out the issue devp2p#136.

    There are a bunch more smaller changes to the protocol, but they are less important hence they were ommitted from this summary.

    Nevertheless, discv5 still does not resolve a couple issues present in discv4, such as unreliable endpoint verification. As of writing this post, there is currently no new method in discv5 to improve the endpoint verification process.

    As you can see discv5 is still a work in progress and has a few large challenges to overcome. However if it does, it will most likely be a large improvement to a more naive Kademlia implementations.

    Hopefully this article helped explain what these discovery protocols are and how they work. If you're interested in their full specifications you can find them on github.

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    From Kademlia to Discv5

    by
    9 min read

    A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.

    + +

    If you've been working on Ethereum or adjacent technologies you've probably heard of discv4 or discv5. But what are they actually? How do they work and what makes them different? To answer these questions, we need to start at the beginning, so this post will assume that there is little knowledge on the subject so the post should be accessible for anyone.

    +

    The Beginning​

    +

    Let's start right at the beginning: the problem of discovery and organization of nodes in peer-to-peer networks.

    +

    Early P2P file sharing technologies, such as Napster, would share information about who holds what file using a single server. A node would connect to the central server and give it a list of the files it owns. Another node would then connect to that central server, find a node that has the file it is looking for and contact that node. This however was a flawed system -- it was vulnerable to attacks and left a single party open to lawsuits.

    +

    It became clear that another solution was needed, and after years of research and experimentation, we were given the distributed hash table or DHT.

    +

    Distributed Hash Tables​

    +

    In 2001 4 new protocols for such DHTs were conceived, Tapestry, Chord, CAN and Pastry, all of which made various trade-offs and changes in their core functionality, giving them unique characteristics.

    +

    But as said, they're all DHTs. So what is a DHT?

    +

    A distributed hash table (DHT) is essentially a distributed key-value list. Nodes participating in the DHT can easily retrieve the value for a key.

    +

    If we have a network with 9 key-value pairs and 3 nodes, ideally each node would store 3 (optimally 6 for redundancy) of those key-value pairs, meaning that if a key-value pair were to be updated, only part of the network would responsible for ensuring that it is. The idea is that any node in the network would know where to find the specific key-value pair it is looking for based on how things are distributed amongst the nodes.

    +

    Kademlia​

    +

    So now that we know what DHTs are, let's get to Kademlia, the predecessor of discv4. Kademlia was created by Petar Maymounkov and David Mazières in 2002. I will naively say that this is probably one of the most popular and most used DHT protocols. It's quite simple in how it works, so let's look at it.

    +

    In Kademlia, nodes and values are arranged by distance (in a very mathematical definition). This distance is not a geographical one, but rather based on identifiers. It is calculated how far 2 identifiers are from eachother using some distance function.

    +

    Kademlia uses an XOR as its distance function. An XOR is a function that outputs true only when inputs differ. Here is an example with some binary identifiers:

    +
    XOR 10011001
        00110010
        --------
        10101011
    +

    The top in decimal numbers means that the distance between 153 and 50 is 171.

    +

    There are several reasons why XOR was taken:

    +
      +
    1. The distance from one ID to itself will be 0.
      2. +
    2. Distance is symmetric, A to B is the same as B to A.
      3. +
    3. Follows triangle inequality, if A, B and C are points on a triangle then the distance A to B is closer or equal to that of A to C plus the one from B to C.
      4. +
    +

    In summary, this distance function allows a node to decide what is "close" to it and make decisions based on that "closeness".

    +

    Kademlia nodes store a routing table. This table contains multiple lists. Each subsequent list contains nodes which are a little further distanced than the ones included in the previous list. Nodes maintain detailed knowledge about nodes closest to them, and the further away a node is, the less knowledge the node maintains about it.

    +

    So let's say I want to find a specific node. What I would do is go to any node which I already know and ask them for all their neighbours closest to my target. I repeat this process for the returned neighbours until I find my target.

    +

    The same thing happens for values. Values have a certain distance from nodes and their IDs are structured the same way so we can calculate this distance. If I want to find a value, I simply look for the neighbours closest to that value's key until I find the one storing said value.

    +

    For Kademlia nodes to support these functions, there are several messages with which the protocol communicates.

    +
      +
    • PING - Used to check whether a node is still running.
      • +
    • STORE - Stores a value with a given key on a node.
      • +
    • FINDNODE - Returns the closest nodes requested to a given ID.
      • +
    • FINDVALUE - The same as FINDNODE, except if a node stores the specific value it will return it directly.
      • +
    +

    This is a very simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the paper or a more in-depth design specification

    +

    Discv4​

    +

    Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT.

    +

    Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the FINDVALUE and STORE commands described by Kademlia.

    +

    The lookup method previously described by Kademlia describes how a node gets its peers. A node contacts some node and asks it for the nodes closest to itself. It does so until it can no longer find any new nodes.

    +

    Additionally, discv4 adds mutual endpoint verification. This is meant to ensure that a peer calling FINDNODE also participates in the discovery protocol.

    +

    Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called ENRRequest.

    +

    If you want some more details on ENRs, check out one of my posts "Network Addresses in Ethereum"

    +

    Discv4 comes with its own range of problems however. Let's look at a few of them.

    +

    Firstly, the way discv4 works right now, there is no way to differentiate between node sub-protocols. This means for example that an Ethereum node could add an Ethereum Classic Node, Swarm or Whisper node to its DHT without realizing that it is invalid until more communication has happened. This inability to differentiate sub-protocols makes it harder to find specific nodes, such as Ethereum nodes with light-client support.

    +

    Next, in order to prevent replay attacks, discv4 uses timestamps. This however can lead to various issues when a host's clock is wrong. For more details, see the "Known Issues" section of the discv4 specification.

    +

    Finally, we have an issue with the way mutual endpoint verification works. Messages can get dropped and there is no way to tell if both peers have verified eachother. This means that we could consider our peer verified while it does not consider us so making them drop the FINDNODE packet.

    +

    Discv5​

    +

    Finally, let's look at discv5. The next iteration of discv4 and the discovery protocol which will be used by Eth 2.0. It aims at fixing various issues present in discv4.

    +

    The first change is the way FINDNODE works. In traditional Kademlia as well as in discv5, we pass an identifier. However, in discv5 we instead pass the logarithmic distance, meaning that a FINDNODE request gets a response containing all nodes at the specified logarithmic distance from the called node.

    +

    Logarithmic distance means we first calculate the distance and then run it through our log base 2 function. See:

    +
    log2(A xor B)
    +

    And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are first in first out lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering ads on their peers.

    +

    As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place ads on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network.

    +

    Additionally, it is unclear how many peers a node should place these ads on and exactly which peers to place them on. For more details, check out the issue devp2p#136.

    +

    There are a bunch more smaller changes to the protocol, but they are less important hence they were ommitted from this summary.

    +

    Nevertheless, discv5 still does not resolve a couple issues present in discv4, such as unreliable endpoint verification. As of writing this post, there is currently no new method in discv5 to improve the endpoint verification process.

    +

    As you can see discv5 is still a work in progress and has a few large challenges to overcome. However if it does, it will most likely be a large improvement to a more naive Kademlia implementations.

    +
    +

    Hopefully this article helped explain what these discovery protocols are and how they work. If you're interested in their full specifications you can find them on github.

    \ No newline at end of file diff --git a/rlog/mdsecheck-method/index.html b/rlog/mdsecheck-method/index.html index 33bcdd50..786a292b 100644 --- a/rlog/mdsecheck-method/index.html +++ b/rlog/mdsecheck-method/index.html @@ -1,27 +1,29 @@ - + - -The MDSECheck method: choosing secure square MDS matrices for P-SP-networks | Vac Research + +The MDSECheck method: choosing secure square MDS matrices for P-SP-networks | Vac Research - - - - + + + + -
    -

    The MDSECheck method: choosing secure square MDS matrices for P-SP-networks

    by
    18 min read

    This article introduces MDSECheck method — a novel approach + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    The MDSECheck method: choosing secure square MDS matrices for P-SP-networks

    by
    18 min read

    This article introduces MDSECheck method — a novel approach to checking square MDS matrices for unconditional security -as the components of affine permutation layers of P-SP-networks.

    Introduction​

    Maximum distance separable (MDS) matrices play a significant role +as the components of affine permutation layers of P-SP-networks.

    + +

    Introduction​

    +

    Maximum distance separable (MDS) matrices play a significant role in algebraic coding theory and symmetric cryptography. In particular, square MDS matrices are commonly used in affine permutation layers of @@ -29,11 +31,13 @@ partial substitution-permutation networks (P-SPNs). These are widespread designs of the modern symmetric ciphers and hash functions. A classic example of the latter is Poseidon [1], -a well-known hash function used in zk-SNARK proving systems.

    Square MDS matrices differ in terms of security +a well-known hash function used in zk-SNARK proving systems.

    +

    Square MDS matrices differ in terms of security that they are able to provide for P-SPNs. The use of some such matrices in certain P-SPNs may result in existence of infinitely long subspace trails of small period for the latter, -which make them vulnerable to differential cryptanalysis [2].

    Two methods for security checking of square MDS matrices for P-SPNs +which make them vulnerable to differential cryptanalysis [2].

    +

    Two methods for security checking of square MDS matrices for P-SPNs have been proposed in [2]. The first one, which is referred to as the three tests method in the rest of the article, is aimed at security checking for @@ -43,145 +47,180 @@ has been designed to determine whether a square MDS matrix satisfies a sufficient condition of being secure regardless of the structure of a P-SPN substitution layer, i.e. to check whether the matrix belongs to the class of square MDS matrices, which are referred to -as unconditionally secure in the current article.

    This article aims to introduce MDSECheck method — +as unconditionally secure in the current article.

    +

    This article aims to introduce MDSECheck method — a novel approach to checking square MDS matrices for unconditional security, which has already been implemented in the Rust programming language as the library crate [3]. The next sections explain the notions mentioned above, describe the MDSECheck method as well as its mathematical foundations, provide a brief overview of the MDSECheck library crate -and outline possible future research directions.

    MDS matrix: how to define and construct​

    An mm x nn matrix MM over a finite field is called MDS, -if and only if for distinct nn-dimensional column vectors v1v_1 and v2v_2 -the column vectors v1 ∣ Mv1v_1 \: | \: M v_1 and v2 ∣ Mv2v_2 \: | \: M v_2, -where ∣| stands for vertical concatenation, -do not coincide in nn or more components. -The set of all possible column vectors v ∣ Mvv \: | \: M v for -some fixed matrix MM is a systematic MDS code, i.e. +and outline possible future research directions.

    +

    MDS matrix: how to define and construct​

    +

    An mm x nn matrix MM over a finite field is called MDS, +if and only if for distinct nn-dimensional column vectors v1v_1 and v2v_2 +the column vectors v1 ∣ Mv1v_1 \: | \: M v_1 and v2 ∣ Mv2v_2 \: | \: M v_2, +where ∣| stands for vertical concatenation, +do not coincide in nn or more components. +The set of all possible column vectors v ∣ Mvv \: | \: M v for +some fixed matrix MM is a systematic MDS code, i.e. a linear code, which contains input symbols on their original positions and achieves the Singleton bound. -The latter property results in good error-correction capability.

    There are several equivalent definitions of MDS matrices, +The latter property results in good error-correction capability.

    +

    There are several equivalent definitions of MDS matrices, but the next one is especially useful for constructing them directly by means of algebraic methods. A matrix over a finite field is called MDS, if and only if all its square submatrices are nonsingular. -The matrix entries and the matrix itself are also considered submatrices.

    One of the most efficient and straightforward methods to directly construct +The matrix entries and the matrix itself are also considered submatrices.

    +

    One of the most efficient and straightforward methods to directly construct an MDS matrix is generating a Cauchy matrix [4]. -Such an mm x nn matrix is defined using -mm-dimensional vector xx and nn-dimensional vector yy, -for which all entries in the concatenation of xx and yy are distinct. +Such an mm x nn matrix is defined using +mm-dimensional vector xx and nn-dimensional vector yy, +for which all entries in the concatenation of xx and yy are distinct. The entries of the Cauchy matrix are described by the formula -Mi,j=1 / (xi−yj)M_{i, j} = 1 \: / \: (x_i - y_j). +Mi,j=1 / (xi−yj)M_{i, j} = 1 \: / \: (x_i - y_j). It is obvious that any submatrix of a Cauchy matrix is also a Cauchy matrix. The Cauchy determinant formula [5] implies that every square Cauchy matrix is nonsingular. -Thus, Cauchy matrices satisfy the second definition of MDS matrices.

    Partial substitution-permutation networks​

    Describing SPNs in algebraic terms is convenient, +Thus, Cauchy matrices satisfy the second definition of MDS matrices.

    +

    Partial substitution-permutation networks​

    +

    Describing SPNs in algebraic terms is convenient, so this approach has been chosen for this article. SPNs are designs of the symmetric cryptoprimitives, which operate on an internal state, which is represented -as an nn-dimensional vector over some finite field, +as an nn-dimensional vector over some finite field, and update this state iteratively by means of -the round transformations described below.

    Each round begins with an optional update of the internal state by +the round transformations described below.

    +

    Each round begins with an optional update of the internal state by adding to its components some input data or extraction of some of these components as the output data. This optional step depends on the specific cryptoprimitive and the current round number. The next step is called the nonlinear substitution layer -and lies in replacing the ii-th component of the internal state -with Si(c)S_i(c) for each i∈[1..n]i \in [1..n], where cc is the component value -and Si(x)S_i(x) is a nonlinear invertible function over the finite field. -The function Si(x)S_i(x) is specific to the cryptoprimitive and called an S-Box. +and lies in replacing the ii-th component of the internal state +with Si(c)S_i(c) for each i∈[1..n]i \in [1..n], where cc is the component value +and Si(x)S_i(x) is a nonlinear invertible function over the finite field. +The function Si(x)S_i(x) is specific to the cryptoprimitive and called an S-Box. The final step, which is known as the affine permutation layer, -replaces the internal state with MX+cM X + c, -where XX is the current internal state, -MM is a nonsingular square matrix and -cc is the vector of the round constants. -The value of cc is specific to the cryptoprimitive +replaces the internal state with MX+cM X + c, +where XX is the current internal state, +MM is a nonsingular square matrix and +cc is the vector of the round constants. +The value of cc is specific to the cryptoprimitive and the current round number, -while MM typically depends only on the cryptoprimitive. -The data flow diagram for an SPN is given below.

    ..................................                         
       │        │        │        │                           
       ▼        ▼        ▼        ▼                           
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ..................................

    Partial SPNs are modifications of SPNs, +while MM typically depends only on the cryptoprimitive. +The data flow diagram for an SPN is given below.

    +
    ..................................                         
       │        │        │        │                           
       ▼        ▼        ▼        ▼                           
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ..................................
    +

    Partial SPNs are modifications of SPNs, where for certain rounds some S-Boxes are replaced with the identity functions to reduce computational efforts [2]. For example, the nonlinear substitution layers of the partial rounds of Poseidon update only the first internal state component [1]. -In the case of P-SPNs, security considerations commonly demand to choose MM +In the case of P-SPNs, security considerations commonly demand to choose MM as a square MDS matrix, because these matrices provide perfect diffusion property for the affine permutation layer [6]. Possessing this property means ensuring that -any two nn-dimensional internal states, -which differ in exactly tt components, +any two nn-dimensional internal states, +which differ in exactly tt components, are mapped by the affine permutation layer to -two new internal states that differ in at least n−t+1n - t + 1 components.

    Square MDS matrix security check in the context of P-SPNs​

    Certain square MDS matrices should not be used in certain P-SPNs +two new internal states that differ in at least n−t+1n - t + 1 components.

    +

    Square MDS matrix security check in the context of P-SPNs​

    +

    Certain square MDS matrices should not be used in certain P-SPNs to avoid making them vulnerable to differential cryptanalysis, since it may exploit the existence of infinitely long subspace trails of small period for vulnerable P-SPNs. [2]. -Such matrices are called insecure with respect to particular P-SPNs.

    An infinitely long subspace trail of period ll exists for a P-SPN, +Such matrices are called insecure with respect to particular P-SPNs.

    +

    An infinitely long subspace trail of period ll exists for a P-SPN, if and only if there is a proper subspace of differences of internal state vectors, such that if for a pair of initial internal states the difference belongs to this subspace, then the difference for the new internal states, which are obtained from the initial ones -by means of the same ll-round transformation, -also belongs to this subspace [2].

    Two methods for checking square MDS matrices for suitability for P-SPNs +by means of the same ll-round transformation, +also belongs to this subspace [2].

    +

    Two methods for checking square MDS matrices for suitability for P-SPNs in terms of existence of infinitely long subspace trails have been proposed in [2]. The three tests method is aimed at checking whether using a specified matrix for a P-SPN with a specified structure of the substitution layer -leads to existence of infinitely long subspace trails of period pp -for this P-SPN for all pp no larger than a given ll. +leads to existence of infinitely long subspace trails of period pp +for this P-SPN for all pp no larger than a given ll. The sufficient test method has been designed to determine whether a square MDS matrix satisfies a sufficient condition -of non-existence of infinitely long subspace trails of period pp -for P-SPNs using this matrix for all pp no larger than a specified ll.

    The sufficient test method is a direct consequence of +of non-existence of infinitely long subspace trails of period pp +for P-SPNs using this matrix for all pp no larger than a specified ll.

    +

    The sufficient test method is a direct consequence of Theorem 8 in [2] and consists in checking that -the minimal polynomial of the pp-th power of the tested matrix -has maximum degree and is irreducible for all p∈[1..l]p \in [1..l]. +the minimal polynomial of the pp-th power of the tested matrix +has maximum degree and is irreducible for all p∈[1..l]p \in [1..l]. The aforesaid sufficient non-existence condition is satisfied by the matrix, -if and only if all the checks yield positive results.

    It is convenient to define +if and only if all the checks yield positive results.

    +

    It is convenient to define the unconditional P-SPN security level of the square MDS matrix as follows: -this level is ll for the matrix MM, -if and only if the minimal polynomials of MM, M2M^2, ......, MlM^l +this level is ll for the matrix MM, +if and only if the minimal polynomials of MM, M2M^2, ......, MlM^l have maximum degree and are irreducible, -but for Ml + 1M^{l \: + \: 1} the minimal polynomial does not have this property. +but for Ml + 1M^{l \: + \: 1} the minimal polynomial does not have this property. Using this definition, the purpose of the sufficient test method can be described as checking whether the unconditional P-SPN security level of the specified matrix -is no less than a given bound.

    MDSECheck method: getting rid of the matrix powers​

    The MDSECheck method, whose name is derived from +is no less than a given bound.

    +

    MDSECheck method: getting rid of the matrix powers​

    +

    The MDSECheck method, whose name is derived from the words "MDS", "security", "elaborated" and "check", has the same purpose as the sufficient test method, but achieves it differently. The differences of the first method from the latter -and approaches to implementing them can be described as follows:

    1. Computation and verification of minimal polynomials -of M2M^2, M3M^3, ......, MlM^l, where MM is -the tested nn x nn matrix over GF(q)GF(q) -and ll is the security level bound, +and approaches to implementing them can be described as follows:

      +
        +
      1. +

        Computation and verification of minimal polynomials +of M2M^2, M3M^3, ......, MlM^l, where MM is +the tested nn x nn matrix over GF(q)GF(q) +and ll is the security level bound, has been replaced with checks for the corresponding powers -of a root of the characteristic polynomial of MM -for non-presence in nontrivial subfields of GF(qn)GF(q^n).

        1. The non-presence check is performed without -straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n). +of a root of the characteristic polynomial of MM +for non-presence in nontrivial subfields of GF(qn)GF(q^n).

          +
            +
          1. +

            The non-presence check is performed without +straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n). The root is checked only for non-presence in the subfields -GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, -GF(qn / pd)GF(q^{n \: / \: p_d}), where p1p_1, p2p_2, ......, pdp_d -are all prime divisors of nn.

          2. The non-presence check reuses some data computed during -the checking for irreducibility the minimal polynomial of MM, -which in this case coincides with f(y)f(y) -designating the characteristic polynomial of MM. -The values of yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y) are saved -for each j∈[1..d]j \in [1..d] during the irreducibility check +GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, +GF(qn / pd)GF(q^{n \: / \: p_d}), where p1p_1, p2p_2, ......, pdp_d +are all prime divisors of nn.

            +
            3. +
          3. +

            The non-presence check reuses some data computed during +the checking for irreducibility the minimal polynomial of MM, +which in this case coincides with f(y)f(y) +designating the characteristic polynomial of MM. +The values of yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y) are saved +for each j∈[1..d]j \in [1..d] during the irreducibility check to replace exponentiations with sequential computations -of (yi)qn / pjmod  f(y)(y^i)^{q^{n \: / \: p_j}} \mod f(y) from -(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) -as its product with yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y).

        2. The check of the minimal polynomial of MM +of (yi)qn / pjmod  f(y)(y^i)^{q^{n \: / \: p_j}} \mod f(y) from +(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) +as its product with yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y).

          +
          3. +
        +
        2. +
      2. +

        The check of the minimal polynomial of MM for irreducibility and maximum degree is performed without unconditional computation of this polynomial. This computation has been replaced with the Krylov method fragment, which consists in building and solving -only one system of linear equations over GF(q)GF(q). -If MM has an irreducible minimal polynomial of maximum degree, +only one system of linear equations over GF(q)GF(q). +If MM has an irreducible minimal polynomial of maximum degree, then its coefficients are trivially determined from the system solution. If the system is degenerate, -then the minimal polynomial of MM does not have such properties.

      The correctness of the first distinctive feature can be proven as follows. +then the minimal polynomial of MM does not have such properties.

      +
      2. +
    +

    The correctness of the first distinctive feature can be proven as follows. Verifying that the minimal polynomial of a matrix is of maximum degree and irreducible is equivalent to verifying that @@ -189,146 +228,194 @@ the characteristic polynomial of this matrix is irreducible, because the minimal polynomial divides the characteristic one. Also, it is trivially proven that for a matrix with such a minimal polynomial it is equal to the characteristic polynomial. -Thus, the required checks for the matrices M2M^2, M3M^3, ......, MlM^l -can be done by checking their characteristic polynomials for irreducibility.

    Let MM be nn x nn matrix over GF(q)GF(q), +Thus, the required checks for the matrices M2M^2, M3M^3, ......, MlM^l +can be done by checking their characteristic polynomials for irreducibility.

    +

    Let MM be nn x nn matrix over GF(q)GF(q), whose minimal polynomial is of maximum degree and irreducible. -The statements in the previous paragraph imply that f(y)f(y), -which is the nn-degree characteristic polynomial of MM, is irreducible. -Consider MM over the extension field GF(qn)GF(q^n), -which is the splitting field of f(y)f(y). -Let α∈GF(qn)α \in GF(q^n) be a root of f(y)f(y). +The statements in the previous paragraph imply that f(y)f(y), +which is the nn-degree characteristic polynomial of MM, is irreducible. +Consider MM over the extension field GF(qn)GF(q^n), +which is the splitting field of f(y)f(y). +Let α∈GF(qn)α \in GF(q^n) be a root of f(y)f(y). According to standard results from the Galois field theory, -αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}} -are distinct roots of f(y)f(y) [7]. -Thus, these powers of αα are nn distinct eigenvalues of MM. -Hence, due to matrix similarity properties, there is some matrix SS -such that SMS−1=DS M S^{-1} = D, where DD is the diagonal matrix, +αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}} +are distinct roots of f(y)f(y) [7]. +Thus, these powers of αα are nn distinct eigenvalues of MM. +Hence, due to matrix similarity properties, there is some matrix SS +such that SMS−1=DS M S^{-1} = D, where DD is the diagonal matrix, whose nonzero elements are -αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}}. -Therefore, SMiS−1=DiS M^i S^{-1} = D^i, -so the roots of the characteristic polynomial of MiM^i -are αiα^i, (αq)i(α^q)^i, (αq2)i(α^{q^2})^i, ......, (αqn − 1)i(α^{q^{n \: - \: 1}})^i. -If the minimal polynomial of αiα^i has degree less than nn, -then the characteristic polynomial of MiM^i is divisible +αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}}. +Therefore, SMiS−1=DiS M^i S^{-1} = D^i, +so the roots of the characteristic polynomial of MiM^i +are αiα^i, (αq)i(α^q)^i, (αq2)i(α^{q^2})^i, ......, (αqn − 1)i(α^{q^{n \: - \: 1}})^i. +If the minimal polynomial of αiα^i has degree less than nn, +then the characteristic polynomial of MiM^i is divisible by this minimal polynomial, -while αiα^i lies in some nontrivial subfield of GF(qn)GF(q^n). +while αiα^i lies in some nontrivial subfield of GF(qn)GF(q^n). One of the fields isomorphic to this subfield is a residue class ring -of polynomials modulo the minimal polynomial of αiα^i [7]. -If the minimal polynomial of αiα^i is of degree nn, -then the characteristic polynomial of MiM^i +of polynomials modulo the minimal polynomial of αiα^i [7]. +If the minimal polynomial of αiα^i is of degree nn, +then the characteristic polynomial of MiM^i equals this minimal polynomial and therefore is irreducible, -while αiα^i does not lie in any nontrivial subfield of GF(qn)GF(q^n). -In this case, 11, αiα^i, (αi)2(α^i)^2, ......, (αi)n − 1(α^i)^{n \: - \: 1} +while αiα^i does not lie in any nontrivial subfield of GF(qn)GF(q^n). +In this case, 11, αiα^i, (αi)2(α^i)^2, ......, (αi)n − 1(α^i)^{n \: - \: 1} are linearly independent as distinct roots of an irreducible polynomial over a finite field [7], -so any field containing αiα^i has at least qnq^n elements -and therefore cannot be a trivial subfield of GF(qn)GF(q^n). +so any field containing αiα^i has at least qnq^n elements +and therefore cannot be a trivial subfield of GF(qn)GF(q^n). Thus, checking the characteristic polynomials of -the matrices M2M^2, M3M^3, ......, MlM^l for irreducibility -is equivalent to verifying that α2α^2, α3α^3, ......, αlα^l -do not lie in any nontrivial subfield of GF(qn)GF(q^n).

    The last sentences of the two previous paragraphs imply the following: -verifying that the minimal polynomials of M2M^2, M3M^3, ......, MlM^l +the matrices M2M^2, M3M^3, ......, MlM^l for irreducibility +is equivalent to verifying that α2α^2, α3α^3, ......, αlα^l +do not lie in any nontrivial subfield of GF(qn)GF(q^n).

    +

    The last sentences of the two previous paragraphs imply the following: +verifying that the minimal polynomials of M2M^2, M3M^3, ......, MlM^l are of maximum degree and irreducible can be performed by verifying that the corresponding powers of a root of -the characteristic polynomial of the nn x nn matrix MM over GF(q)GF(q) -do not belong to any nontrivial subfield of GF(qn)GF(q^n). ■\blacksquare

    The approaches to implementing the first distinctive feature +the characteristic polynomial of the nn x nn matrix MM over GF(q)GF(q) +do not belong to any nontrivial subfield of GF(qn)GF(q^n). ■\blacksquare

    +

    The approaches to implementing the first distinctive feature can be explained and proven to be correct as follows. -Since GF(qw)GF(q^w) is a nontrivial subfield of GF(qu)GF(q^u) -if and only if ww divides uu and w<uw < u [7], -the presence of some εε in GF(qh)GF(q^h), -which is a nontrivial subfield of GF(qn)GF(q^n), -implies that ε∈GF(qn / ν)ε \in GF(q^{n \: / \: ν}) for some prime νν dividing nn, -because hh divides the quotient of nn and some of its prime factors. +Since GF(qw)GF(q^w) is a nontrivial subfield of GF(qu)GF(q^u) +if and only if ww divides uu and w<uw < u [7], +the presence of some εε in GF(qh)GF(q^h), +which is a nontrivial subfield of GF(qn)GF(q^n), +implies that ε∈GF(qn / ν)ε \in GF(q^{n \: / \: ν}) for some prime νν dividing nn, +because hh divides the quotient of nn and some of its prime factors. Thus, checking that some value does not belong to subfields -GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, -GF(qn / pd)GF(q^{n \: / \: p_d}), -where p1p_1, p2p_2, ......, pdp_d are all prime divisors of nn, +GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, +GF(qn / pd)GF(q^{n \: / \: p_d}), +where p1p_1, p2p_2, ......, pdp_d are all prime divisors of nn, is equivalent to checking this value for -non-presence in nontrivial subfields of GF(qn)GF(q^n).

    Checking for irreducibility the minimal polynomial of MM +non-presence in nontrivial subfields of GF(qn)GF(q^n).

    +

    Checking for irreducibility the minimal polynomial of MM is performed by means of Algorithm 2.2.9 in [8] and consists in sequential computation of -ypmod  f(y)y^p \mod f(y), yp2mod  f(y)y^{p^2} \mod f(y), ......, -yp⌊n / 2⌋mod  f(y)y^{p^{\lfloor n \: / \: 2 \rfloor}} \mod f(y) -and checking that GCD(ypimod  f(y) − y,f(y))=1GCD(y^{p^i} \mod f(y) \: - \: y, f(y)) = 1 -for each i∈[1..⌊n / 2⌋]i \in [1..\lfloor n \: / \: 2 \rfloor], -where f(y)f(y) is the characteristic polynomial of MM +ypmod  f(y)y^p \mod f(y), yp2mod  f(y)y^{p^2} \mod f(y), ......, +yp⌊n / 2⌋mod  f(y)y^{p^{\lfloor n \: / \: 2 \rfloor}} \mod f(y) +and checking that GCD(ypimod  f(y) − y,f(y))=1GCD(y^{p^i} \mod f(y) \: - \: y, f(y)) = 1 +for each i∈[1..⌊n / 2⌋]i \in [1..\lfloor n \: / \: 2 \rfloor], +where f(y)f(y) is the characteristic polynomial of MM and coincides with the minimal polynomial in this case. The optimized root non-presence check is performed -by checking that for each i∈[2..l]i \in [2..l] for each j∈[1..d]j \in [1..d] -the value of ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) is nonzero. +by checking that for each i∈[2..l]i \in [2..l] for each j∈[1..d]j \in [1..d] +the value of ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) is nonzero. This approach is based on the following standard results -from the Galois field theory [7]:

    • GF(qn)GF(q^n) is isomorphic to the residue class ring of -univariate polynomials in yy modulo f(y)f(y), -because at this point f(y)f(y) is known to be irreducible, -and some root of f(y)f(y) is mapped by this isomorphism to -the residue class the polynomial yy in this ring.

    • All elements of a finite field GF(qw)GF(q^w) and only they -are roots of yqw−yy^{q^w} - y.

    The expression ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) -can be rewritten as ((yqn / pj)i−yi)mod  f(y)((y^{q^{n \: / \: p_j}})^i - y^i) \mod f(y), +from the Galois field theory [7]:

    +
      +
    • +

      GF(qn)GF(q^n) is isomorphic to the residue class ring of +univariate polynomials in yy modulo f(y)f(y), +because at this point f(y)f(y) is known to be irreducible, +and some root of f(y)f(y) is mapped by this isomorphism to +the residue class the polynomial yy in this ring.

      +
      • +
    • +

      All elements of a finite field GF(qw)GF(q^w) and only they +are roots of yqw−yy^{q^w} - y.

      +
      • +
    +

    The expression ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) +can be rewritten as ((yqn / pj)i−yi)mod  f(y)((y^{q^{n \: / \: p_j}})^i - y^i) \mod f(y), which can be computed without exponentiation as the product of -(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) and -yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y), -which has been saved during the irreducibility check.

    The second distinctive feature can be +(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) and +yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y), +which has been saved during the irreducibility check.

    +

    The second distinctive feature can be explained and proven to be correct in following way. -The nn x nn matrix MM does not have a minimal polynomial of maximum degree, -if some Krylov subspace of order nn for it is not nn-dimensional. +The nn x nn matrix MM does not have a minimal polynomial of maximum degree, +if some Krylov subspace of order nn for it is not nn-dimensional. Indeed, the minimal polynomial of the matrix is divisible by the minimal polynomial of the restriction of this linear operator to an arbitrary subspace, -and in the considered case the latter polynomial has degree less than nn, +and in the considered case the latter polynomial has degree less than nn, because the degree of the minimal polynomial of a linear operator cannot exceed the dimension of the subspace the operator acts on. -Thus, an unconditional computation of the minimal polynomial of MM +Thus, an unconditional computation of the minimal polynomial of MM is not required to determine whether this polynomial is irreducible and has maximum degree. Using this computation has been replaced with the Krylov method fragment, -which consists in choosing any nonzero nn-dimensional column vector vv -and solving the system of linear equations AX=bA X = b, -where AA is an nn x nn matrix, -whose columns are vv, MvM v, M2vM^2 v, ......, Mn − 1vM^{n \: - \: 1} v, -and bb is MnvM^n v. -If AA is singular, -the minimal polynomial of MM is reducible or does not have maximum degree, -so checking MM has been accomplished; -otherwise, f(y)f(y), which is the minimal and characteristic polynomial of MM, -can be expressed as yn−Xn − 1yn − 1−Xn − 2yn − 2−…−X1y−X0y^n - X_{n \: - \: 1} y^{n \: - \: 1} - X_{n \: - \: 2} y^{n \: - \: 2} - … - X_1 y - X_0.

    The steps of MDSECheck method can be summarized as follows:

    1. The square MDS matrix MM over GF(q)GF(q) -and the unconditional P-SPN security level bound ll are received as inputs.

    2. The Krylov method fragment is used to compute the minimal polynomial of MM. -If the computation fails, then MM is not unconditionally secure, -so the check of MM is complete. +which consists in choosing any nonzero nn-dimensional column vector vv +and solving the system of linear equations AX=bA X = b, +where AA is an nn x nn matrix, +whose columns are vv, MvM v, M2vM^2 v, ......, Mn − 1vM^{n \: - \: 1} v, +and bb is MnvM^n v. +If AA is singular, +the minimal polynomial of MM is reducible or does not have maximum degree, +so checking MM has been accomplished; +otherwise, f(y)f(y), which is the minimal and characteristic polynomial of MM, +can be expressed as yn−Xn − 1yn − 1−Xn − 2yn − 2−…−X1y−X0y^n - X_{n \: - \: 1} y^{n \: - \: 1} - +X_{n \: - \: 2} y^{n \: - \: 2} - … - X_1 y - X_0.

      +

      The steps of MDSECheck method can be summarized as follows:

      +
        +
      1. +

        The square MDS matrix MM over GF(q)GF(q) +and the unconditional P-SPN security level bound ll are received as inputs.

        +
        2. +
      2. +

        The Krylov method fragment is used to compute the minimal polynomial of MM. +If the computation fails, then MM is not unconditionally secure, +so the check of MM is complete. If it succeeds, then the minimal polynomial has maximum degree -and, therefore, coincides with the characteristic polynomial of MM.

      3. Algorithm 2.2.9 is used -to check for irreducibility the minimal polynomial of MM, -which is also the characteristic polynomial of MM in this case. +and, therefore, coincides with the characteristic polynomial of MM.

        +
        4. +
      4. +

        Algorithm 2.2.9 is used +to check for irreducibility the minimal polynomial of MM, +which is also the characteristic polynomial of MM in this case. Some data computed during this step is saved to be reused at the next one. -If the polynomial is reducible, then the check of MM is complete, -because MM has been found to be not unconditionally secure.

      5. The values of α2α^2, α3α^3, ......, αlα^l, -where αα is a root of the characteristic polynomial of MM, +If the polynomial is reducible, then the check of MM is complete, +because MM has been found to be not unconditionally secure.

        +
        6. +
      6. +

        The values of α2α^2, α3α^3, ......, αlα^l, +where αα is a root of the characteristic polynomial of MM, are sequentially checked for non-presence in -nontrivial subfields of GF(qn)GF(q^n) as described above. -If αiα^i belongs to some nontrivial subfield of GF(qn)GF(q^n), -then the unconditional P-SPN security level of MM is i − 1i \: - \: 1, -so the check of MM is complete. +nontrivial subfields of GF(qn)GF(q^n) as described above. +If αiα^i belongs to some nontrivial subfield of GF(qn)GF(q^n), +then the unconditional P-SPN security level of MM is i − 1i \: - \: 1, +so the check of MM is complete. If all the values do not belong to such a subfield, -then the unconditional P-SPN security level is at least ll.

      MDSECheck library crate: implementation in Rust​

      The library crate [3] provides tools for +then the unconditional P-SPN security level is at least ll.

      +
      3. +
    +

    MDSECheck library crate: implementation in Rust​

    +

    The library crate [3] provides tools for generating random square Cauchy MDS matrices over prime finite fields and applying the MDSECheck method to check such matrices for unconditional security. The used data types of field elements and polynomials are provided by the crates ark-ff [9] and ark-poly [10]. -The auxiliary tools in the crate modules are accessible as well.

    Generating by means of this crate a 10 x 10 MDS matrix, +The auxiliary tools in the crate modules are accessible as well.

    +

    Generating by means of this crate a 10 x 10 MDS matrix, which is defined over the BN254 scalar field [11] and has unconditional P-SPN security level is 1000, takes less than 60 milliseconds on average for the laptop with the processor Intel® Core™ i9-14900HX, -whose maximum clock frequency is 5.8 GHz.

    Conclusion​

    The MDSECheck method proposed in this article is a novel approach +whose maximum clock frequency is 5.8 GHz.

    +

    Conclusion​

    +

    The MDSECheck method proposed in this article is a novel approach to checking square MDS matrices for unconditional security as the components of affine permutation layers of P-SPNs. It has been implemented as a practical library crate for generating unconditionally secure square MDS matrices -for P-SPNs over prime finite fields.

    The future research directions may include +for P-SPNs over prime finite fields.

    +

    The future research directions may include theoretical and experimental studies of performance of approaches, which use the MDSECheck method -to generate unconditionally secure square MDS matrices for P-SPNs.

    References​

    1. L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, M. Schofnegger. "POSEIDON: A New Hash Function for Zero-Knowledge Proof Systems (Updated Version)".
    2. L. Grassi, C. Rechberger, M. Schofnegger. "Proving Resistance Against Infinitely Long Subspace Trails: How to Choose the Linear Layer".
    3. The page "mdsecheck" on crates.io.
    4. Y. Kumar, P. Mishra, S. Samanta, K. Chand Gupta, A. Gaur. "Construction of all MDS and involutory MDS matrices".
    5. The page "Value of Cauchy Determinant" on proofwiki.org.
    6. T. Silva, R. Dahab "MDS Matrices for Cryptography".
    7. S. Huczynska, M. Neunhöffer. "Finite Fields"
    8. R. Crandall, C. Pomerance. "Prime Numbers: A Computational Perspective" (2nd edition).
    9. The page "ark-ff" on crates.io.
    10. The page "ark-poly" on crates.io.
    11. The page "ark-bn254" on crates.io.
    - - +to generate unconditionally secure square MDS matrices for P-SPNs.

    +

    References​

    +
      +
    1. L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, M. Schofnegger. "POSEIDON: A New Hash Function for Zero-Knowledge Proof Systems (Updated Version)".
      2. +
    2. L. Grassi, C. Rechberger, M. Schofnegger. "Proving Resistance Against Infinitely Long Subspace Trails: How to Choose the Linear Layer".
      3. +
    3. The page "mdsecheck" on crates.io.
      4. +
    4. Y. Kumar, P. Mishra, S. Samanta, K. Chand Gupta, A. Gaur. "Construction of all MDS and involutory MDS matrices".
      5. +
    5. The page "Value of Cauchy Determinant" on proofwiki.org.
      6. +
    6. T. Silva, R. Dahab "MDS Matrices for Cryptography".
      7. +
    7. S. Huczynska, M. Neunhöffer. "Finite Fields"
      8. +
    8. R. Crandall, C. Pomerance. "Prime Numbers: A Computational Perspective" (2nd edition).
      9. +
    9. The page "ark-ff" on crates.io.
      10. +
    10. The page "ark-poly" on crates.io.
      11. +
    11. The page "ark-bn254" on crates.io.
      12. +
    \ No newline at end of file diff --git a/rlog/p2p-data-sync-for-mobile/index.html b/rlog/p2p-data-sync-for-mobile/index.html index 11000352..4188b0bd 100644 --- a/rlog/p2p-data-sync-for-mobile/index.html +++ b/rlog/p2p-data-sync-for-mobile/index.html @@ -1,28 +1,139 @@ - + - -P2P Data Sync for Mobile | Vac Research + +P2P Data Sync for Mobile | Vac Research - - - + + + -
    -

    P2P Data Sync for Mobile

    by
    12 min read

    A research log. Reliable and decentralized, pick two.

    Together with decanus, I've been working on the problem of data sync lately.

    In building p2p messaging systems, one problem you quickly come across is the problem of reliably transmitting data. If there's no central server with high availability guarantees, you can't meaningfully guarantee that data has been transmitted. One way of solving this problem is through a synchronization protocol.

    There are many synchronization protocols out there and I won't go into detail of how they differ with our approach here. Some common examples are Git and Bittorrent, but there are also projects like IPFS, Swarm, Dispersy, Matrix, Briar, SSB, etc.

    Problem motivation​

    Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices.

    Why p2p?​

    For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from.

    Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat.

    Why reliably sync data?​

    At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on.

    Why mobilephones?​

    Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence.

    More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc.

    Minimal Requirements​

    For requirements or design goals for a solution, here's what we came up with.

    1. MUST sync data reliably between devices. By reliably we mean having the ability to deal with messages being out of order, dropped, duplicated, or delayed.

    2. MUST NOT rely on any centralized services for reliability. By centralized services we mean any single point of failure that isn’t one of the endpoint devices.

    3. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices that are resource restricted, mostly-offline and often changing network.

    4. MAY use helper services in order to be more mobile-friendly. Examples of helper services are decentralized file storage solutions such as IPFS and Swarm. These help with availability and latency of data for mostly-offline devices.

    5. MUST have the ability to provide casual consistency. By casual consistency we mean the commonly accepted definition in distributed systems literature. This means messages that are casually related can achieve a partial ordering.

    6. MUST support ephemeral messages that don’t need replication. That is, allow for messages that don’t need to be reliabily transmitted but still needs to be transmitted between devices.

    7. MUST allow for privacy-preserving messages and extreme data loss. By privacy-preserving we mean things such as exploding messages (self-destructing messages). By extreme data loss we mean the ability for two trusted devices to recover from a, deliberate or accidental, removal of data.

    8. MUST be agnostic to whatever transport it is running on. It should not rely on specific semantics of the transport it is running on, nor be tightly coupled with it. This means a transport can be swapped out without loss of reliability between devices.

    MVDS - a minimium viable version​

    The first minimum viable version is in an alpha stage, and it has a specification, implementation and we have deployed it in a console client for end to end functionality. It's heavily inspired by Bramble Sync Protocol.

    The spec is fairly minimal. You have nodes that exchange records over some secure transport. These records are of different types, such as OFFER, MESSAGE, REQUEST, and ACK. A peer keep tracks of the state of message for each node it is interacting with. There's also logic for message retransmission with exponential delay. The positive ACK and retransmission model is quite similar to how TCP is designed.

    There are two different modes of syncing, interactive and batch mode. See sequence diagrams below.

    Interactive mode: -

    Interactive mode

    Batch mode: -

    Batch mode

    Which mode should you choose? It's a tradeoff of latency and bandwidth. If you want to minimize latency, batch mode is better. If you care about preserving bandwidth interactive mode is better. The choice is up to each node.

    Basic simulation​

    Initial ad hoc bandwidth and latency testing shows some issues with a naive approach. Running with the default simulation settings:

    • communicating nodes: 2
    • nodes using interactive mode: 2
    • interval between messages: 5s
    • time node is offine: 90%
    • nodes each node is sharing with: 2

    we notice a huge overhead. More specifically, we see a ~5 minute latency overhead and a bandwidth multiplier of x100-1000, i.e. 2-3 orders of magnitude just for receiving a message with interactive mode, without acks.

    Now, that seems terrible. A moment of reflection will reveal why that is. If each node is offline uniformly 90% of the time, that means that each record will be lost 90% of the time. Since interactive mode requires offer, request, payload (and then ack), that's three links just for Bob to receive the actual message.

    Each failed attempt implies another retransmission. That means we have (1/0.1)^3 = 1000 expected overhead to receive a message in interactive mode. The latency follows naturally from that, with the retransmission logic.

    Mostly-offline devices​

    The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost.

    There are a few ways of getting around this issue. One is having a store and forward model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status.

    Another, arguably more pure and robust, way is having a remote log, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS.

    What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are required to get reliable data transmission.

    Basic calculations for bandwidth multiplier​

    While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions:

    • two nodes exchanging a single message in batch mode
    • 10% uniformly random uptime for each node
    • in HA cache case, 100% uptime of a piece of infrastructure C
    • retransmission every epoch (with constant or exponential backoff)
    • only looking at average (p50) case

    First case, no helper services​

    A sends a message to B, and B acks it.

    A message -> B (10% chance of arrival)
    A   <- ack   B (10% chance of arrival)

    With a constant backoff, A will send messages at epoch 1, 2, 3, .... With exponential backoff and a multiplier of 2, this would be 1, 2, 4, 8, .... Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier.

    There's a difference between time to receive and time to stop sending. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore:

    1. A will send messages until it receives an ACK.
    2. B will send ACK if it receives a message.

    To get an average of one ack through, A needs to send 100 messages, and B send on average 10 acks. That's a multiplier of roughly a 100. That's roughly what we saw with the simulation above for receiving a message in interactive mode.

    Second case, high-availability caching layer​

    Let's introduce a helper node or piece of infrastructure, C. Whenever A or B sends a message, it also sends it to C. Whenever A or B comes online, it queries for messages with C.

    A message    -> B (10% chance of arrival)
    A message    -> C (100% chance of arrival)
    B <- req/res -> C (100% chance of arrival)
    A   <- ack      B (10% chance of arrival)
    C   <- ack      B (100% chance of arrival)
    A <- req/res -> C (100% chance of arrival)

    What's the probability that A's messages will arrive at B? Directly, it's still 10%. But we can assume it's 100% that C picks up the message. (Giving C a 90% chance success rate doesn't materially change the numbers).

    B will pick up A's message from C after an average of 10 epochs. Then B will send ack to A, which will also be picked up by C 100% of the time. Once A comes online again, it'll query C and receive B's ack.

    Assuming we use exponential backoff with a multiplier of 2, A will send a message directly to B at epoch 1, 2, 4, 8 (assuming it is online). At this point, epoch 10, B will be online in the average case. These direct sends will likely fail, but B will pick the message up from C and send one ack, both directly to A and to be picked up by C. Once A comes online, it'll query C and receive the ack from B, which means it won't do any more retransmits.

    How many messages have been sent? Not counting interactions with C, A sends 4 (at most) and B 1. Depending on if the interaction with C is direct or indirect (i.e. multicast), the factor for interaction with C will be ~2. This means the total bandwidth multiplier is likely to be <10, which is a lot more acceptable.

    Since the syncing semantics are end-to-end, this is without relying on the reliablity of C.

    Caveat​

    Note that both of these are probabilistic argument. They are also based on heuristics. More formal analysis would be desirable, as well as better simulations to experimentally verify them. In fact, the calculations could very well be wrong!

    Future work​

    There are many enhancements that can be made and are desirable. Let's outline a few.

    1. Data sync clients. Examples of actual usage of data sync, with more interesting domain semantics. This also includes usage of sequence numbers and DAGs to know what content is missing and ought to be synced.

    2. Remote log. As alluded to above, this is necessary. It needs a more clear specification and solid proof of concepts.

    3. More efficient ways of syncing with large number of nodes. When the number of nodes goes up, the algorithmic complexity doesn't look great. This also touches on things such as ambient content discovery.

    4. More robust simulations and real-world deployments. Exisiting simulation is ad hoc, and there are many improvements that can be made to gain more confidence and identify issues. Additionally, better formal analysis.

    5. Example usage over multiple transports. Including things like sneakernet and meshnets. The described protocol is designed to work over unstructured, structured and private p2p networks. In some cases it can leverage differences in topology, such as multicast, or direct connections.

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    P2P Data Sync for Mobile

    by
    12 min read

    A research log. Reliable and decentralized, pick two.

    + +

    Together with decanus, I've been working on the problem of data sync lately.

    +

    In building p2p messaging systems, one problem you quickly come across is the problem of reliably transmitting data. If there's no central server with high availability guarantees, you can't meaningfully guarantee that data has been transmitted. One way of solving this problem is through a synchronization protocol.

    +

    There are many synchronization protocols out there and I won't go into detail of how they differ with our approach here. Some common examples are Git and Bittorrent, but there are also projects like IPFS, Swarm, Dispersy, Matrix, Briar, SSB, etc.

    +

    Problem motivation​

    +

    Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices.

    +

    Why p2p?​

    +

    For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from.

    +

    Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat.

    +

    Why reliably sync data?​

    +

    At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on.

    +

    Why mobilephones?​

    +

    Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence.

    +

    More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc.

    +

    Minimal Requirements​

    +

    For requirements or design goals for a solution, here's what we came up with.

    +
      +
    1. +

      MUST sync data reliably between devices. By reliably we mean having the ability to deal with messages being out of order, dropped, duplicated, or delayed.

      +
      2. +
    2. +

      MUST NOT rely on any centralized services for reliability. By centralized services we mean any single point of failure that isn’t one of the endpoint devices.

      +
      3. +
    3. +

      MUST allow for mobile-friendly usage. By mobile-friendly we mean devices that are resource restricted, mostly-offline and often changing network.

      +
      4. +
    4. +

      MAY use helper services in order to be more mobile-friendly. Examples of helper services are decentralized file storage solutions such as IPFS and Swarm. These help with availability and latency of data for mostly-offline devices.

      +
      5. +
    5. +

      MUST have the ability to provide casual consistency. By casual consistency we mean the commonly accepted definition in distributed systems literature. This means messages that are casually related can achieve a partial ordering.

      +
      6. +
    6. +

      MUST support ephemeral messages that don’t need replication. That is, allow for messages that don’t need to be reliabily transmitted but still needs to be transmitted between devices.

      +
      7. +
    7. +

      MUST allow for privacy-preserving messages and extreme data loss. By privacy-preserving we mean things such as exploding messages (self-destructing messages). By extreme data loss we mean the ability for two trusted devices to recover from a, deliberate or accidental, removal of data.

      +
      8. +
    8. +

      MUST be agnostic to whatever transport it is running on. It should not rely on specific semantics of the transport it is running on, nor be tightly coupled with it. This means a transport can be swapped out without loss of reliability between devices.

      +
      9. +
    +

    MVDS - a minimium viable version​

    +

    The first minimum viable version is in an alpha stage, and it has a specification, implementation and we have deployed it in a console client for end to end functionality. It's heavily inspired by Bramble Sync Protocol.

    +

    The spec is fairly minimal. You have nodes that exchange records over some secure transport. These records are of different types, such as OFFER, MESSAGE, REQUEST, and ACK. A peer keep tracks of the state of message for each node it is interacting with. There's also logic for message retransmission with exponential delay. The positive ACK and retransmission model is quite similar to how TCP is designed.

    +

    There are two different modes of syncing, interactive and batch mode. See sequence diagrams below.

    +

    Interactive mode: +

    Interactive mode

    +

    Batch mode: +

    Batch mode

    +

    Which mode should you choose? It's a tradeoff of latency and bandwidth. If you want to minimize latency, batch mode is better. If you care about preserving bandwidth interactive mode is better. The choice is up to each node.

    +

    Basic simulation​

    +

    Initial ad hoc bandwidth and latency testing shows some issues with a naive approach. Running with the default simulation settings:

    +
      +
    • communicating nodes: 2
      • +
    • nodes using interactive mode: 2
      • +
    • interval between messages: 5s
      • +
    • time node is offine: 90%
      • +
    • nodes each node is sharing with: 2
      • +
    +

    we notice a huge overhead. More specifically, we see a ~5 minute latency overhead and a bandwidth multiplier of x100-1000, i.e. 2-3 orders of magnitude just for receiving a message with interactive mode, without acks.

    +

    Now, that seems terrible. A moment of reflection will reveal why that is. If each node is offline uniformly 90% of the time, that means that each record will be lost 90% of the time. Since interactive mode requires offer, request, payload (and then ack), that's three links just for Bob to receive the actual message.

    +

    Each failed attempt implies another retransmission. That means we have (1/0.1)^3 = 1000 expected overhead to receive a message in interactive mode. The latency follows naturally from that, with the retransmission logic.

    +

    Mostly-offline devices​

    +

    The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost.

    +

    There are a few ways of getting around this issue. One is having a store and forward model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status.

    +

    Another, arguably more pure and robust, way is having a remote log, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS.

    +

    What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are required to get reliable data transmission.

    +

    Basic calculations for bandwidth multiplier​

    +

    While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions:

    +
      +
    • two nodes exchanging a single message in batch mode
      • +
    • 10% uniformly random uptime for each node
      • +
    • in HA cache case, 100% uptime of a piece of infrastructure C
      • +
    • retransmission every epoch (with constant or exponential backoff)
      • +
    • only looking at average (p50) case
      • +
    +

    First case, no helper services​

    +

    A sends a message to B, and B acks it.

    +
    A message -> B (10% chance of arrival)
    A   <- ack   B (10% chance of arrival)
    +

    With a constant backoff, A will send messages at epoch 1, 2, 3, .... With exponential backoff and a multiplier of 2, this would be 1, 2, 4, 8, .... Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier.

    +

    There's a difference between time to receive and time to stop sending. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore:

    +
      +
    1. A will send messages until it receives an ACK.
      2. +
    2. B will send ACK if it receives a message.
      3. +
    +

    To get an average of one ack through, A needs to send 100 messages, and B send on average 10 acks. That's a multiplier of roughly a 100. That's roughly what we saw with the simulation above for receiving a message in interactive mode.

    +

    Second case, high-availability caching layer​

    +

    Let's introduce a helper node or piece of infrastructure, C. Whenever A or B sends a message, it also sends it to C. Whenever A or B comes online, it queries for messages with C.

    +
    A message    -> B (10% chance of arrival)
    A message    -> C (100% chance of arrival)
    B <- req/res -> C (100% chance of arrival)
    A   <- ack      B (10% chance of arrival)
    C   <- ack      B (100% chance of arrival)
    A <- req/res -> C (100% chance of arrival)
    +

    What's the probability that A's messages will arrive at B? Directly, it's still 10%. But we can assume it's 100% that C picks up the message. (Giving C a 90% chance success rate doesn't materially change the numbers).

    +

    B will pick up A's message from C after an average of 10 epochs. Then B will send ack to A, which will also be picked up by C 100% of the time. Once A comes online again, it'll query C and receive B's ack.

    +

    Assuming we use exponential backoff with a multiplier of 2, A will send a message directly to B at epoch 1, 2, 4, 8 (assuming it is online). At this point, epoch 10, B will be online in the average case. These direct sends will likely fail, but B will pick the message up from C and send one ack, both directly to A and to be picked up by C. Once A comes online, it'll query C and receive the ack from B, which means it won't do any more retransmits.

    +

    How many messages have been sent? Not counting interactions with C, A sends 4 (at most) and B 1. Depending on if the interaction with C is direct or indirect (i.e. multicast), the factor for interaction with C will be ~2. This means the total bandwidth multiplier is likely to be <10, which is a lot more acceptable.

    +

    Since the syncing semantics are end-to-end, this is without relying on the reliablity of C.

    +

    Caveat​

    +

    Note that both of these are probabilistic argument. They are also based on heuristics. More formal analysis would be desirable, as well as better simulations to experimentally verify them. In fact, the calculations could very well be wrong!

    +

    Future work​

    +

    There are many enhancements that can be made and are desirable. Let's outline a few.

    +
      +
    1. +

      Data sync clients. Examples of actual usage of data sync, with more interesting domain semantics. This also includes usage of sequence numbers and DAGs to know what content is missing and ought to be synced.

      +
      2. +
    2. +

      Remote log. As alluded to above, this is necessary. It needs a more clear specification and solid proof of concepts.

      +
      3. +
    3. +

      More efficient ways of syncing with large number of nodes. When the number of nodes goes up, the algorithmic complexity doesn't look great. This also touches on things such as ambient content discovery.

      +
      4. +
    4. +

      More robust simulations and real-world deployments. Exisiting simulation is ad hoc, and there are many improvements that can be made to gain more confidence and identify issues. Additionally, better formal analysis.

      +
      5. +
    5. +

      Example usage over multiple transports. Including things like sneakernet and meshnets. The described protocol is designed to work over unstructured, structured and private p2p networks. In some cases it can leverage differences in topology, such as multicast, or direct connections.

      +
      6. +
    \ No newline at end of file diff --git a/rlog/page/2/index.html b/rlog/page/2/index.html index 8a415e58..98038aac 100644 --- a/rlog/page/2/index.html +++ b/rlog/page/2/index.html @@ -1,30 +1,27 @@ - + - -Research Blog | Vac Research + +Research Blog | Vac Research - - - - + + + + -
    -

    Research Blog

    Blog

    by
    7 min read

    Waku is an open communication protocol and network. Decentralized apps and infrastructure can use Waku for their + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Research Blog

    Blog

    Waku for All Decentralized Applications and Infrastructures

    by
    7 min read

    Waku is an open communication protocol and network. Decentralized apps and infrastructure can use Waku for their communication needs. It is designed to enable dApps and decentralized infrastructure projects to have secure, private, scalable communication. Waku is available in several languages and platforms, from Web to mobile to desktop to cloud. Initially, We pushed Waku adoption to the Web ecosystem, we learned that Waku is usable in a variety of complex applications -and infrastructure projects. We have prioritized our effort to make Waku usable on various platforms and environments.

    by
    19 min read

    What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    - - +and infrastructure projects. We have prioritized our effort to make Waku usable on various platforms and environments.

    Building Privacy-Protecting Infrastructure

    by
    19 min read

    What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    \ No newline at end of file diff --git a/rlog/page/3/index.html b/rlog/page/3/index.html index 6c134bd9..0ed15f12 100644 --- a/rlog/page/3/index.html +++ b/rlog/page/3/index.html @@ -1,26 +1,23 @@ - + - -Research Blog | Vac Research + +Research Blog | Vac Research - - - - + + + + -
    -

    Research Blog

    Blog

    by
    11 min read

    Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.

    by
    21 min read

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.

    by
    8 min read

    A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Research Blog

    Blog

    Introducing nwaku

    by
    11 min read

    Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.

    Privacy-preserving p2p economic spam protection in Waku v2

    by
    21 min read

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.

    Waku v2 Update

    by
    8 min read

    A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?

    \ No newline at end of file diff --git a/rlog/page/4/index.html b/rlog/page/4/index.html index 96900fe2..7934a03d 100644 --- a/rlog/page/4/index.html +++ b/rlog/page/4/index.html @@ -1,26 +1,23 @@ - + - -Research Blog | Vac Research + +Research Blog | Vac Research - - - - + + + + -
    -

    Research Blog

    Blog

    by
    9 min read

    A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.

    by
    6 min read

    A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?

    by
    6 min read

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Research Blog

    Blog

    From Kademlia to Discv5

    by
    9 min read

    A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.

    Waku Update

    by
    6 min read

    A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?

    Vac - A Rough Overview

    by
    6 min read

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.

    \ No newline at end of file diff --git a/rlog/page/5/index.html b/rlog/page/5/index.html index 935c3590..c1493d5f 100644 --- a/rlog/page/5/index.html +++ b/rlog/page/5/index.html @@ -1,26 +1,23 @@ - + - -Research Blog | Vac Research + +Research Blog | Vac Research - - - - + + + + -
    -

    Research Blog

    Blog
    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Research Blog

    Blog
    \ No newline at end of file diff --git a/rlog/presenting-js-waku/index.html b/rlog/presenting-js-waku/index.html index 6eaf6157..5c08dc9f 100644 --- a/rlog/presenting-js-waku/index.html +++ b/rlog/presenting-js-waku/index.html @@ -1,45 +1,56 @@ - + - -Presenting JS-Waku: Waku v2 in the Browser | Vac Research + +Presenting JS-Waku: Waku v2 in the Browser | Vac Research - - - + + + -
    -

    Presenting JS-Waku: Waku v2 in the Browser

    by
    7 min read

    JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!

    For the past 3 months, we have been working on bringing Waku v2 to the browser. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Presenting JS-Waku: Waku v2 in the Browser

    by
    7 min read

    JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!

    + +

    For the past 3 months, we have been working on bringing Waku v2 to the browser. Our aim is to empower dApps with Waku v2, and it led to the creation of a new library. -We believe now is good time to introduce it!

    Waku v2​

    First, let's review what Waku v2 is and what problem it is trying to solve.

    Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised -communications on resource restricted devices (i.e., mobile phones).

    The Status chat feature was initially built over Whisper. +We believe now is good time to introduce it!

    +

    Waku v2​

    +

    First, let's review what Waku v2 is and what problem it is trying to solve.

    +

    Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised +communications on resource restricted devices (i.e., mobile phones).

    +

    The Status chat feature was initially built over Whisper. However, Whisper has a number of caveats which makes it inefficient for mobile phones. -For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.

    To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced. +For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.

    +

    To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced. To further enable web and restricted resource environments, Waku v2 was created based on libp2p. -The migration of the Status chat feature to Waku v2 is currently in progress.

    We see the need of such solution in the broader Ethereum ecosystem, beyond Status. +The migration of the Status chat feature to Waku v2 is currently in progress.

    +

    We see the need of such solution in the broader Ethereum ecosystem, beyond Status. This is why we are building Waku v2 as a decentralised communication platform for all to use and build on. If you want to read more about Waku v2 and what it aims to achieve, -checkout What's the Plan for Waku v2?.

    Since last year, we have been busy defining and implementing Waku v2 protocols in nim-waku, +checkout What's the Plan for Waku v2?.

    +

    Since last year, we have been busy defining and implementing Waku v2 protocols in nim-waku, from which you can build wakunode2. Wakunode2 is an adaptive and modular Waku v2 node, it allows users to run their own node and use the Waku v2 protocols they need. -The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.

    Waku v2 in the browser​

    We believe that dApps and wallets can benefit from the Waku network in several ways. +The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.

    +

    Waku v2 in the browser​

    +

    We believe that dApps and wallets can benefit from the Waku network in several ways. For some dApps, it makes sense to enable peer-to-peer communications. For others, machine-to-machine communications would be a great asset. For example, in the case of a DAO, Waku could be used for gas-less voting. Enabling the DAO to notify their users of a new vote, -and users to vote without interacting with the blockchain and spending gas.

    Murmur was the first attempt to bring Whisper to the browser, +and users to vote without interacting with the blockchain and spending gas.

    +

    Murmur was the first attempt to bring Whisper to the browser, acting as a bridge between devp2p and libp2p. Once Waku v2 was started and there was a native implementation on top of libp2p, a chat POC was created to demonstrate the potential of Waku v2 @@ -48,59 +59,85 @@ It showed how using js-libp2p with few modifications enabled access to the Waku There was still some unresolved challenges. For example, nim-waku only support TCP connections which are not supported by browser applications. Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets, -which in turn could connect to wakunode2 via TCP.

    However, to enable dApp and Wallet developers to easily integrate Waku in their product, +which in turn could connect to wakunode2 via TCP.

    +

    However, to enable dApp and Wallet developers to easily integrate Waku in their product, we need to give them a library that is easy to use and works out of the box: -introducing JS-Waku.

    JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network. -It is available right now on npm:

    npm install js-waku.

    As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.

    Key Waku v2 protocols are already available: +introducing JS-Waku.

    +

    JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network. +It is available right now on npm:

    +

    npm install js-waku.

    +

    As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.

    +

    Key Waku v2 protocols are already available: message, store, relay and light push, -enabling your dApp to:

    • Send and receive near-instant messages on the Waku network (relay),
    • Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
    • Send messages with confirmations (light push).

    JS-Waku needs to operate in the same context from which Waku v2 was born: +enabling your dApp to:

    +
      +
    • Send and receive near-instant messages on the Waku network (relay),
      • +
    • Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
      • +
    • Send messages with confirmations (light push).
      • +
    +

    JS-Waku needs to operate in the same context from which Waku v2 was born: a restricted environment were connectivity or uptime are not guaranteed; -JS-Waku brings Waku v2 to the browser.

    Achievements so far​

    We focused the past month on developing a ReactJS Chat App. +JS-Waku brings Waku v2 to the browser.

    +

    Achievements so far​

    +

    We focused the past month on developing a ReactJS Chat App. The aim was to create enough building blocks in JS-Waku to enable this showcase web app that -we now use for dogfooding purposes.

    Most of the effort was on getting familiar with the js-libp2p library +we now use for dogfooding purposes.

    +

    Most of the effort was on getting familiar with the js-libp2p library that we heavily rely on. JS-Waku is the second implementation of Waku v2 protocol, so a lot of effort on interoperability was needed. For example, to ensure compatibility with the nim-waku reference implementation, -we run our tests against wakunode2 as part of the CI.

    This interoperability effort helped solidify the current Waku v2 specifications: +we run our tests against wakunode2 as part of the CI.

    +

    This interoperability effort helped solidify the current Waku v2 specifications: By clarifying the usage of topics (#327, #383), fix discrepancies between specs and nim-waku (#418, #419) and fix small nim-waku & nim-libp2p bugs -(#411, #439).

    To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes. +(#411, #439).

    +

    To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes. A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser. Unfortunately websocket support is not yet available in nim-libp2p so -we ended up deploying websockify alongside wakunode2 instances.

    As we built the web chat app, +we ended up deploying websockify alongside wakunode2 instances.

    +

    As we built the web chat app, we were able to fine tune the API to provide a simple and succinct interface. -You can start a node, connect to other nodes and send a message in less than ten lines of code:

    import { Waku } from 'js-waku'

    const waku = await Waku.create({})

    const nodes = await getStatusFleetNodes()
    await Promise.all(nodes.map((addr) => waku.dial(addr)))

    const msg = WakuMessage.fromUtf8String(
      'Here is a message!',
      '/my-cool-app/1/my-use-case/proto',
    )
    await waku.relay.send(msg)

    We have also put a bounty at 0xHack for using JS-Waku +You can start a node, connect to other nodes and send a message in less than ten lines of code:

    +
    import { Waku } from 'js-waku'

    const waku = await Waku.create({})

    const nodes = await getStatusFleetNodes()
    await Promise.all(nodes.map((addr) => waku.dial(addr)))

    const msg = WakuMessage.fromUtf8String(
      'Here is a message!',
      '/my-cool-app/1/my-use-case/proto',
    )
    await waku.relay.send(msg)
    +

    We have also put a bounty at 0xHack for using JS-Waku and running a workshop. We were thrilled to have a couple of hackers create new software using our libraries. One of the projects aimed to create a decentralised, end-to-end encrypted messenger app, similar to what the ETH-DM protocol aims to achieve. Another project was a decentralised Twitter platform. -Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.

    As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network -while preserving this out of the box experience.

    What's next?​

    Next, we are directing our attention towards Developer Experience. +Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.

    +

    As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network +while preserving this out of the box experience.

    +

    What's next?​

    +

    Next, we are directing our attention towards Developer Experience. We already have documentation available but we want to provide more: Tutorials, various examples -and showing how JS-Waku can be used with Web3.

    By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets. +and showing how JS-Waku can be used with Web3.

    +

    By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets. We think JS-Waku builds a strong case for machine-to-machine (M2M) communications. The first use cases we are looking into are dApp notifications: Enabling dApp to notify their user directly in their wallets! Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified -of events such as DAO voting.

    We already have some POC in the pipeline to enable voting and polling on the Waku network, -allowing users to save gas by not broadcasting each individual vote on the blockchain.

    To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples +of events such as DAO voting.

    +

    We already have some POC in the pipeline to enable voting and polling on the Waku network, +allowing users to save gas by not broadcasting each individual vote on the blockchain.

    +

    To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples of signing, validating, encrypting and decrypting messages using Web3. -Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.

    As you can read, we have grand plans for JS-Waku and Waku v2. +Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.

    +

    As you can read, we have grand plans for JS-Waku and Waku v2. There is a lot to do, and we would love some help so feel free to check out the new role in our team: js-waku: Wallet & Dapp Integration Developer. -We also have a number of positions open to work on Waku protocol and nim-waku.

    If you are as excited as us by JS-Waku, why not build a dApp with it? -You can find documentation on the npmjs page.

    Whether you are a developer, you can come chat with us using WakuJS Web Chat +We also have a number of positions open to work on Waku protocol and nim-waku.

    +

    If you are as excited as us by JS-Waku, why not build a dApp with it? +You can find documentation on the npmjs page.

    +

    Whether you are a developer, you can come chat with us using WakuJS Web Chat or chat2. You can get support in #dappconnect-support on Vac Discord or Telegram. -If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.

    - - +If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.

    \ No newline at end of file diff --git a/rlog/remote-log/index.html b/rlog/remote-log/index.html index fe6085f5..1469be9c 100644 --- a/rlog/remote-log/index.html +++ b/rlog/remote-log/index.html @@ -1,30 +1,84 @@ - + - -P2P Data Sync with a Remote Log | Vac Research + +P2P Data Sync with a Remote Log | Vac Research - - - + + + -
    -

    P2P Data Sync with a Remote Log

    by
    5 min read

    A research log. Asynchronous P2P messaging? Remote logs to the rescue!

    A big problem when doing end-to-end data sync between mobile nodes is that most devices are offline most of the time. With a naive approach, you quickly run into issues of 'ping-pong' behavior, where messages have to be constantly retransmitted. We saw some basic calculations of what this bandwidth multiplier looks like in a previous post.

    While you could do some background processing, this is really battery-draining, and on iOS these capabilities are limited. A better approach instead is to loosen the constraint that two nodes need to be online at the same time. How do we do this? There are two main approaches, one is the store and forward model, and the other is a remote log.

    In the store and forward model, we use an intermediate node that forward messages on behalf of the recipient. In the remote log model, you instead replicate the data onto some decentralized storage, and have a mutable reference to the latest state, similar to DNS. While both work, the latter is somewhat more elegant and "pure", as it has less strict requirements of an individual node's uptime. Both act as a highly-available cache to smoothen over non-overlapping connection windows between endpoints.

    In this post we are going to describe how such a remote log schema could work. Specifically, how it enhances p2p data sync and takes care of the following requirements:

    1. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices -that are resource restricted, mostly-offline and often changing network.
    1. MAY use helper services in order to be more mobile-friendly. Examples of + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

      P2P Data Sync with a Remote Log

      by
      5 min read

      A research log. Asynchronous P2P messaging? Remote logs to the rescue!

      + +

      A big problem when doing end-to-end data sync between mobile nodes is that most devices are offline most of the time. With a naive approach, you quickly run into issues of 'ping-pong' behavior, where messages have to be constantly retransmitted. We saw some basic calculations of what this bandwidth multiplier looks like in a previous post.

      +

      While you could do some background processing, this is really battery-draining, and on iOS these capabilities are limited. A better approach instead is to loosen the constraint that two nodes need to be online at the same time. How do we do this? There are two main approaches, one is the store and forward model, and the other is a remote log.

      +

      In the store and forward model, we use an intermediate node that forward messages on behalf of the recipient. In the remote log model, you instead replicate the data onto some decentralized storage, and have a mutable reference to the latest state, similar to DNS. While both work, the latter is somewhat more elegant and "pure", as it has less strict requirements of an individual node's uptime. Both act as a highly-available cache to smoothen over non-overlapping connection windows between endpoints.

      +

      In this post we are going to describe how such a remote log schema could work. Specifically, how it enhances p2p data sync and takes care of the following requirements:

      +
      +
        +
      1. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices +that are resource restricted, mostly-offline and often changing network.
        2. +
      +
      +
      +
        +
      1. MAY use helper services in order to be more mobile-friendly. Examples of helper services are decentralized file storage solutions such as IPFS and Swarm. These help with availability and latency of data for mostly-offline -devices.

      Remote log​

      A remote log is a replication of a local log. This means a node can read data from a node that is offline.

      The spec is in an early draft stage and can be found here. A very basic spike / proof-of-concept can be found here.

      Definitions​

      TermDefinition
      CASContent-addressed storage. Stores data that can be addressed by its hash.
      NSName system. Associates mutable data to a name.
      Remote logReplication of a local log at a different location.

      Roles​

      There are four fundamental roles:

      1. Alice
      2. Bob
      3. Name system (NS)
      4. Content-addressed storage (CAS)

      The remote log is the data format of what is stored in the name system.

      "Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.

      Flow​

      Figure 1: Remote log data synchronization.

      Data format​

      The remote log lets receiving nodes know what data they are missing. Depending on the specific requirements and capabilities of the nodes and name system, the information can be referred to differently. We distinguish between three rough modes:

      1. Fully replicated log
      2. Normal sized page with CAS mapping
      3. "Linked list" mode - minimally sized page with CAS mapping

      A remote log is simply a mapping from message identifiers to their corresponding address in a CAS:

      Message Identifier (H1)CAS Hash (H2)
      H1_3H2_3
      H1_2H2_2
      H1_1H2_1
      address to next page

      The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.

      Message Identifier (H1)Content
      H1_3C3
      H1_2C2
      H1_1C1
      address to next page

      Both patterns can be used in parallel, e,g. by storing the last k messages directly and use CAS pointers for the rest. Together with the next_page page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.

      Interaction with MVDS​

      vac.mvds.Message payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.

      Future work​

      The spec is still in an early draft stage, so it is expected to change. Same with the proof of concept. More work is needed on getting a fully featured proof of concept with specific CAS and NAS instances. E.g. Swarm and Swarm Feeds, or IPFS and IPNS, or something else.

      For data sync in general:

      • Make consistency guarantees more explicit for app developers with support for sequence numbers and DAGs, as well as the ability to send non-synced messages. E.g. ephemeral typing notifications, linear/sequential history and casual consistency/DAG history
      • Better semantics and scalability for multi-user sync contexts, e.g. CRDTs and joining multiple logs together
      • Better usability in terms of application layer usage (data sync clients) and supporting more transports

      PS1. Thanks everyone who submitted great logo proposals for Vac!

      PPS2. Next week on October 10th decanus and I will be presenting Vac at Devcon, come say hi :)

    - - +devices.
    • + + +

    Remote log​

    +

    A remote log is a replication of a local log. This means a node can read data from a node that is offline.

    +

    The spec is in an early draft stage and can be found here. A very basic spike / proof-of-concept can be found here.

    +

    Definitions​

    +
    TermDefinition
    CASContent-addressed storage. Stores data that can be addressed by its hash.
    NSName system. Associates mutable data to a name.
    Remote logReplication of a local log at a different location.
    +

    Roles​

    +

    There are four fundamental roles:

    +
      +
    1. Alice
      2. +
    2. Bob
      3. +
    3. Name system (NS)
      4. +
    4. Content-addressed storage (CAS)
      5. +
    +

    The remote log is the data format of what is stored in the name system.

    +

    "Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.

    +

    Flow​

    +

    Figure 1: Remote log data synchronization.

    +

    Data format​

    +

    The remote log lets receiving nodes know what data they are missing. Depending on the specific requirements and capabilities of the nodes and name system, the information can be referred to differently. We distinguish between three rough modes:

    +
      +
    1. Fully replicated log
      2. +
    2. Normal sized page with CAS mapping
      3. +
    3. "Linked list" mode - minimally sized page with CAS mapping
      4. +
    +

    A remote log is simply a mapping from message identifiers to their corresponding address in a CAS:

    +
    Message Identifier (H1)CAS Hash (H2)
    H1_3H2_3
    H1_2H2_2
    H1_1H2_1
    address to next page
    +

    The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.

    +
    Message Identifier (H1)Content
    H1_3C3
    H1_2C2
    H1_1C1
    address to next page
    +

    Both patterns can be used in parallel, e,g. by storing the last k messages directly and use CAS pointers for the rest. Together with the next_page page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.

    +

    Interaction with MVDS​

    +

    vac.mvds.Message payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.

    +

    Future work​

    +

    The spec is still in an early draft stage, so it is expected to change. Same with the proof of concept. More work is needed on getting a fully featured proof of concept with specific CAS and NAS instances. E.g. Swarm and Swarm Feeds, or IPFS and IPNS, or something else.

    +

    For data sync in general:

    +
      +
    • Make consistency guarantees more explicit for app developers with support for sequence numbers and DAGs, as well as the ability to send non-synced messages. E.g. ephemeral typing notifications, linear/sequential history and casual consistency/DAG history
      • +
    • Better semantics and scalability for multi-user sync contexts, e.g. CRDTs and joining multiple logs together
      • +
    • Better usability in terms of application layer usage (data sync clients) and supporting more transports
      • +
    +
    +

    PS1. Thanks everyone who submitted great logo proposals for Vac!

    +

    PPS2. Next week on October 10th decanus and I will be presenting Vac at Devcon, come say hi :)

    \ No newline at end of file diff --git a/rlog/rln-anonymous-dos-prevention/index.html b/rlog/rln-anonymous-dos-prevention/index.html index a7be7975..5c9ea86f 100644 --- a/rlog/rln-anonymous-dos-prevention/index.html +++ b/rlog/rln-anonymous-dos-prevention/index.html @@ -1,70 +1,174 @@ - + - -Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku | Vac Research + +Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku | Vac Research - - - - + + + + -
    -

    Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku

    by
    7 min read

    Rate Limiting Nullifiers in practice, applied to an anonymous p2p network, like Waku.

    Introduction​

    Rate Limiting Nullifier (RLN) is a zero-knowledge gadget that allows users to prove 2 pieces of information,

    1. They belong to a permissioned membership set
    2. Their rate of signaling abides by a fixed number that has been previously declared

    The "membership set" introduced above, is in the form of a sparse, indexed merkle tree. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku

    by
    7 min read

    Rate Limiting Nullifiers in practice, applied to an anonymous p2p network, like Waku.

    + +

    Introduction​

    +

    Rate Limiting Nullifier (RLN) is a zero-knowledge gadget that allows users to prove 2 pieces of information,

    +
      +
    1. They belong to a permissioned membership set
      2. +
    2. Their rate of signaling abides by a fixed number that has been previously declared
      3. +
    +

    The "membership set" introduced above, is in the form of a sparse, indexed merkle tree. This membership set can be maintained on-chain, off-chain or as a hybrid depending on the network's storage costs. Waku makes use of a hybrid membership set, where insertions are tracked in a smart contract. In addition, each Waku node maintains a local copy of the tree, -which is updated upon each insertion.

    Users register themselves with a hash of a locally generated secret, +which is updated upon each insertion.

    +

    Users register themselves with a hash of a locally generated secret, which is then inserted into the tree at the next available index. After having registered, users can prove their membership by proving their knowledge of the pre-image of the respective leaf in the tree. The leaf hashes are also referred to as commitments of the respective users. -The actual proof is done by a Merkle Inclusion Proof, which is a type of ZK proof.

    The circuit ensures that the user's secret does indeed hash to a leaf in the tree, -and that the provided Merkle proof is valid.

    After a User generates this Merkle proof, +The actual proof is done by a Merkle Inclusion Proof, which is a type of ZK proof.

    +

    The circuit ensures that the user's secret does indeed hash to a leaf in the tree, +and that the provided Merkle proof is valid.

    +

    After a User generates this Merkle proof, they can transmit it to other users, who can verify the proof. Including a message's hash within the proof generation, -additionally guarantees integrity of that message.

    A malicious user could generate multiple proofs per epoch. +additionally guarantees integrity of that message.

    +

    A malicious user could generate multiple proofs per epoch. they generate multiple proofs per epoch. However, when multiple proofs are generated per epoch, the malicious user's secret is exposed, which strongly disincentivizes this attack. -This mechanism is further described in malicious User secret interpolation mechanism

    Note: This blog post describes rln-v1, which excludes the range check in favor of a global rate limit for all users, -which is once per time window. This version is currently in use in waku-rln-relay.

    RLN Protocol parameters​

    Given below is the set of cryptographic primitives, -and constants that are used in the RLN protocol.

    1. Proving System: groth16
    2. Elliptic Curve: bn254 (aka bn128) (not to be confused with the 254 bit Weierstrass curve)
    3. Finite Field: Prime-order subgroup of the group of points on the bn254 curve
    4. Default Merkle Tree Height: 20
    5. Hashing algorithm: Poseidon
    6. Merkle Tree: Sparse Indexed Merkle Tree
    7. Messages per epoch: 1
    8. Epoch duration: 10 seconds

    Malicious User secret interpolation mechanism​

    note: all the parameters mentioned below are elements in the finite field mentioned above.

    The private inputs to the circuit are as follows: -

    identitySecret: the randomly generated secret of the user
    identityPathIndex: the index of the commitment derived from the secret
    pathElements: elements included in the path to the index of the commitment

    Following are the public inputs to the circuit -

    x: hash of the signal to the finite field
    rlnIdentifier: application-specific identifier which this proof is being generated for
    epoch: the timestamp which this proof is being generated for

    The outputs of the circuit are as follows: -

    y: result of Shamir's secret sharing calculation
    root: root of the Merkle tree obtained after applying the inclusion proof
    nullifier: uniquely identifies a message, derived from rlnIdentifier, epoch, and the user's secret

    With the above data in mind, following is the circuit pseudocode -

    identityCommitment = Poseidon([identitySecret])
    root = MerkleInclusionProof(identityCommitment, identityPathIndex, pathElements)
    externalNullifier = Poseidon([epoch, rlnIdentifier])
    a1 = Poseidon([identitySecret, externalNullifier])
    y = identitySecret + a1 * x
    nullifier = Poseidon([a1])

    To interpolate the secret of a user who has sent multiple signals during the same epoch to the same rln-based application, we may make use of the following formula -

    a1=(y1−y2)(x1−x2)a_1 = {(y_1 - y_2) \over (x_1 - x_2)}

    where x1x_1, y1y_1 and x2x_2, y2y_2 are shares from different messages

    subsequently, we may use one pair of the shares, x1x_1 and y1y_1 to obtain the identitySecret

    identitySecret=y1−a1∗xidentitySecret = y_1 - a_1 * x

    This enables RLN to be used for rate limiting with a global limit. For arbitrary limits, -please refer to an article written by @curryrasul, rln-v2.

    Waku's problem with DoS​

    In a decentralized, privacy focused messaging system like Waku, -Denial of Service (DoS) vulnerabilities are very common, and must be addressed to promote network scale and optimal bandwidth utilization.

    DoS prevention with user metadata​

    There are a couple of ways a user can be rate-limited, either -

    1. IP Logging
    2. KYC Logging

    Both IP and KYC logging prevent systems from being truly anonymous, and hence, cannot be used as a valid DoS prevention mechanism for Waku.

    RLN can be used as an alternative, which provides the best of both worlds, i.e a permissioned membership set, as well as anonymous signaling. -However, we are bound by k-anonymity rules of the membership set.

    Waku-RLN-Relay is a libp2p pubsub validator that verifies if a proof attached to a given message is valid. -In case the proof is valid, the message is relayed.

    Performance analysis​

    Test bench specs: AMD EPYC 7502P 32-Core, 4x32GB DDR4 Reg.ECC Memory

    This simulation was conducted by @alrevuelta, and is described in more detail here.

    The simulation included 100 waku nodes running in parallel.

    Proof generation times - -

    img

    Proof verification times - -

    img

    A spammer node publishes 3000 msg/epoch, which is detected by all connected nodes, and subsequently disconnect to prevent further spam - -

    img

    Security analysis​

    Barbulescu and Duquesne +This mechanism is further described in malicious User secret interpolation mechanism

    +

    Note: This blog post describes rln-v1, which excludes the range check in favor of a global rate limit for all users, +which is once per time window. This version is currently in use in waku-rln-relay.

    +

    RLN Protocol parameters​

    +

    Given below is the set of cryptographic primitives, +and constants that are used in the RLN protocol.

    +
      +
    1. Proving System: groth16
      2. +
    2. Elliptic Curve: bn254 (aka bn128) (not to be confused with the 254 bit Weierstrass curve)
      3. +
    3. Finite Field: Prime-order subgroup of the group of points on the bn254 curve
      4. +
    4. Default Merkle Tree Height: 20
      5. +
    5. Hashing algorithm: Poseidon
      6. +
    6. Merkle Tree: Sparse Indexed Merkle Tree
      7. +
    7. Messages per epoch: 1
      8. +
    8. Epoch duration: 10 seconds
      9. +
    +

    Malicious User secret interpolation mechanism​

    +
    +

    note: all the parameters mentioned below are elements in the finite field mentioned above.

    +
    +

    The private inputs to the circuit are as follows: -

    +
    identitySecret: the randomly generated secret of the user
    identityPathIndex: the index of the commitment derived from the secret
    pathElements: elements included in the path to the index of the commitment
    +

    Following are the public inputs to the circuit -

    +
    x: hash of the signal to the finite field
    rlnIdentifier: application-specific identifier which this proof is being generated for
    epoch: the timestamp which this proof is being generated for
    +

    The outputs of the circuit are as follows: -

    +
    y: result of Shamir's secret sharing calculation
    root: root of the Merkle tree obtained after applying the inclusion proof
    nullifier: uniquely identifies a message, derived from rlnIdentifier, epoch, and the user's secret
    +

    With the above data in mind, following is the circuit pseudocode -

    +
    identityCommitment = Poseidon([identitySecret])
    root = MerkleInclusionProof(identityCommitment, identityPathIndex, pathElements)
    externalNullifier = Poseidon([epoch, rlnIdentifier])
    a1 = Poseidon([identitySecret, externalNullifier])
    y = identitySecret + a1 * x
    nullifier = Poseidon([a1])
    +

    To interpolate the secret of a user who has sent multiple signals during the same epoch to the same rln-based application, we may make use of the following formula -

    +

    a1=(y1−y2)(x1−x2)a_1 = {(y_1 - y_2) \over (x_1 - x_2)}

    +

    where x1x_1, y1y_1 and x2x_2, y2y_2 are shares from different messages

    +

    subsequently, we may use one pair of the shares, x1x_1 and y1y_1 to obtain the identitySecret

    +

    identitySecret=y1−a1∗xidentitySecret = y_1 - a_1 * x

    +

    This enables RLN to be used for rate limiting with a global limit. For arbitrary limits, +please refer to an article written by @curryrasul, rln-v2.

    +

    Waku's problem with DoS​

    +

    In a decentralized, privacy focused messaging system like Waku, +Denial of Service (DoS) vulnerabilities are very common, and must be addressed to promote network scale and optimal bandwidth utilization.

    +

    DoS prevention with user metadata​

    +

    There are a couple of ways a user can be rate-limited, either -

    +
      +
    1. IP Logging
      2. +
    2. KYC Logging
      3. +
    +

    Both IP and KYC logging prevent systems from being truly anonymous, and hence, cannot be used as a valid DoS prevention mechanism for Waku.

    +

    RLN can be used as an alternative, which provides the best of both worlds, i.e a permissioned membership set, as well as anonymous signaling. +However, we are bound by k-anonymity rules of the membership set.

    +

    Waku-RLN-Relay is a libp2p pubsub validator that verifies if a proof attached to a given message is valid. +In case the proof is valid, the message is relayed.

    +

    Performance analysis​

    +
    +

    Test bench specs: AMD EPYC 7502P 32-Core, 4x32GB DDR4 Reg.ECC Memory

    +
    +

    This simulation was conducted by @alrevuelta, and is described in more detail here.

    +

    The simulation included 100 waku nodes running in parallel.

    +

    Proof generation times - +

    img

    +

    Proof verification times - +

    img

    +

    A spammer node publishes 3000 msg/epoch, which is detected by all connected nodes, and subsequently disconnect to prevent further spam - +

    img

    +

    Security analysis​

    +

    Barbulescu and Duquesne conclude that that the bn254 curve has only 100 bits of security. Since the bn254 curve has a small embedding degree, it is vulnerable to the MOV attack. However, the MOV attack is only applicable to pairings, and not to the elliptic curve itself. It is acceptable to use the bn254 curve for RLN, -since the circuit does not make use of pairings.

    An analysis on the number of rounds in the Poseidon hash function was done, -which concluded that the hashing rounds should not be reduced,

    The smart contracts have not been audited, and are not recommended for real world deployments yet.

    Storage analysis​

    commitment_size=32 bytestree_height=20total_leaves=220max_tree_size=total_leaves∗commitment_sizemax_tree_size=220∗32=33,554,432∴max_tree_size=33.55 megabytescommitment\_size = 32\ bytes \\ tree\_height =20 \\ total\_leaves = 2^{20} \\ max\_tree\_size = total\_leaves * commitment\_size \\ max\_tree\_size = 2^{20} * 32 = 33,554,432 \\ ∴max\_tree\_size = 33.55\ megabytes

    The storage overhead introduced by RLN is minimal. +since the circuit does not make use of pairings.

    +

    An analysis on the number of rounds in the Poseidon hash function was done, +which concluded that the hashing rounds should not be reduced,

    +

    The smart contracts have not been audited, and are not recommended for real world deployments yet.

    +

    Storage analysis​

    +commitment_size=32 bytestree_height=20total_leaves=220max_tree_size=total_leaves∗commitment_sizemax_tree_size=220∗32=33,554,432∴max_tree_size=33.55 megabytescommitment\_size = 32\ bytes \\ +tree\_height =20 \\ +total\_leaves = 2^{20} \\ +max\_tree\_size = total\_leaves * commitment\_size \\ +max\_tree\_size = 2^{20} * 32 = 33,554,432 \\ +∴max\_tree\_size = 33.55\ megabytes +

    The storage overhead introduced by RLN is minimal. RLN only requires 34 megabytes of storage, which poses no problem on most end-user hardware, with the exception of IoT/microcontrollers. -Still, we are working on further optimizations allowing proof generation without having to store the full tree.

    The bare minimum requirements to run RLN​

    With proof generation time in sub-second latency, along with low storage overhead for the tree, -it is possible for end users to generate and verify RLN proofs on a modern smartphone.

    Following is a demo provided by @rramos that demonstrates -waku-rln-relay used in react native.

    Warning: The react native sdk will be deprecated soon, and the above demo should serve as a PoC for RLN on mobiles

    RLN usage guide​

    Zerokit implements api's that allow users to handle operations to the tree, -as well as generate/verify RLN proofs.

    Our main implementation of RLN can be accessed via this Rust crate, +Still, we are working on further optimizations allowing proof generation without having to store the full tree.

    +

    The bare minimum requirements to run RLN​

    +

    With proof generation time in sub-second latency, along with low storage overhead for the tree, +it is possible for end users to generate and verify RLN proofs on a modern smartphone.

    +

    Following is a demo provided by @rramos that demonstrates +waku-rln-relay used in react native.

    +
    +

    Warning: The react native sdk will be deprecated soon, and the above demo should serve as a PoC for RLN on mobiles

    +
    +

    RLN usage guide​

    +

    Zerokit implements api's that allow users to handle operations to the tree, +as well as generate/verify RLN proofs.

    +

    Our main implementation of RLN can be accessed via this Rust crate, which is documented here. It can used in other langugages via the FFI API, which is documented here. The usage of RLN in Waku is detailed in our RLN Implementers guide, -which provides step-by-step instructions on how to run Waku-RLN-Relay.

    Following is a diagram that will help understand the dependency tree -

    rln-dep-tree

    Future work​

    • Optimizations to zerokit for proof generation time.
    • Incrementing tree depth from 20 to 32, to allow more memberships.
    • Optimizations to the smart contract.
    • An ability to signal validity of a message in different time windows.
    • Usage of proving systems other than Groth16.

    References​

    - - +which provides step-by-step instructions on how to run Waku-RLN-Relay.

    +

    Following is a diagram that will help understand the dependency tree -

    +

    rln-dep-tree

    +

    Future work​

    +
      +
    • Optimizations to zerokit for proof generation time.
      • +
    • Incrementing tree depth from 20 to 32, to allow more memberships.
      • +
    • Optimizations to the smart contract.
      • +
    • An ability to signal validity of a message in different time windows.
      • +
    • Usage of proving systems other than Groth16.
      • +
    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/rln-light-verifiers/index.html b/rlog/rln-light-verifiers/index.html index e4cb8e23..aabfdb6c 100644 --- a/rlog/rln-light-verifiers/index.html +++ b/rlog/rln-light-verifiers/index.html @@ -1,58 +1,131 @@ - + - -Verifying RLN Proofs in Light Clients with Subtrees | Vac Research + +Verifying RLN Proofs in Light Clients with Subtrees | Vac Research - - - - + + + + -
    -

    Verifying RLN Proofs in Light Clients with Subtrees

    by
    5 min read

    How resource-restricted devices can verify RLN proofs fast and efficiently.

    Introduction​

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    This post expands upon ideas described in the previous post, -focusing on how resource-restricted devices can verify RLN proofs fast and efficiently.

    Previously, it was required to fetch all the memberships from the smart contract, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Verifying RLN Proofs in Light Clients with Subtrees

    by
    5 min read

    How resource-restricted devices can verify RLN proofs fast and efficiently.

    + +

    Introduction​

    +

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    +

    This post expands upon ideas described in the previous post, +focusing on how resource-restricted devices can verify RLN proofs fast and efficiently.

    +

    Previously, it was required to fetch all the memberships from the smart contract, construct the merkle tree locally, and derive the merkle root, -which is subsequently used to verify RLN proofs.

    This process is not feasible for resource-restricted devices since it involves a lot of RPC calls, computation and fault tolerance. -One cannot expect a mobile phone to fetch all the memberships from the smart contract and construct the merkle tree locally.

    Constraints and requirements​

    An alternative solution to the one proposed in this post is to construct the merkle tree on-chain, +which is subsequently used to verify RLN proofs.

    +

    This process is not feasible for resource-restricted devices since it involves a lot of RPC calls, computation and fault tolerance. +One cannot expect a mobile phone to fetch all the memberships from the smart contract and construct the merkle tree locally.

    +

    Constraints and requirements​

    +

    An alternative solution to the one proposed in this post is to construct the merkle tree on-chain, and have the root accessible with a single RPC call. -However, this approach increases gas costs for inserting new memberships and may not be feasible until it is optimized further with batching mechanisms, etc.

    The other methods have been explored in more depth here.

    Following are the requirements and constraints for the solution proposed in this post:

    1. Cheap membership insertions.
    2. As few RPC calls as possible to reduce startup time.
    3. Merkle root of the tree is available on-chain.
    4. No centralized services to sequence membership insertions.
    5. Map inserted commitments to the block in which they were inserted.

    Metrics on sync time for a tree with 2,653 leaves​

    The following metrics are based on the current implementation of RLN in the Waku gen0 network.

    Test bench​

    • Hardware: Macbook Air M2, 16GB RAM
    • Network: 120 Megabits/sec
    • Nwaku commit: e61e4ff
    • RLN membership set contract: 0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4
    • Deployed block number: 4,230,716
    • RLN Membership set depth: 20
    • Hash function: PoseidonT3 (which is a gas guzzler)
    • Max size of the membership set: 2^20 = 1,048,576 leaves

    Metrics​

    • Time to sync the whole tree: 4 minutes
    • RPC calls: 702
    • Number of leaves: 2,653

    One can argue that the time to sync the tree at the current state is not that bad. +However, this approach increases gas costs for inserting new memberships and may not be feasible until it is optimized further with batching mechanisms, etc.

    +

    The other methods have been explored in more depth here.

    +

    Following are the requirements and constraints for the solution proposed in this post:

    +
      +
    1. Cheap membership insertions.
      2. +
    2. As few RPC calls as possible to reduce startup time.
      3. +
    3. Merkle root of the tree is available on-chain.
      4. +
    4. No centralized services to sequence membership insertions.
      5. +
    5. Map inserted commitments to the block in which they were inserted.
      6. +
    +

    Metrics on sync time for a tree with 2,653 leaves​

    +

    The following metrics are based on the current implementation of RLN in the Waku gen0 network.

    +

    Test bench​

    +
      +
    • Hardware: Macbook Air M2, 16GB RAM
      • +
    • Network: 120 Megabits/sec
      • +
    • Nwaku commit: e61e4ff
      • +
    • RLN membership set contract: 0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4
      • +
    • Deployed block number: 4,230,716
      • +
    • RLN Membership set depth: 20
      • +
    • Hash function: PoseidonT3 (which is a gas guzzler)
      • +
    • Max size of the membership set: 2^20 = 1,048,576 leaves
      • +
    +

    Metrics​

    +
      +
    • Time to sync the whole tree: 4 minutes
      • +
    • RPC calls: 702
      • +
    • Number of leaves: 2,653
      • +
    +

    One can argue that the time to sync the tree at the current state is not that bad. However, the number of RPC calls is a concern, which scales linearly with the number of blocks since the contract was deployed This is because the implementation fetches all events from the contract, chunking 2,000 blocks at a time. This is done to avoid hitting the block limit of 10,000 events per call, -which is a limitation of popular RPC providers.

    Proposed solution​

    From a theoretical perspective, +which is a limitation of popular RPC providers.

    +

    Proposed solution​

    +

    From a theoretical perspective, one could construct the merkle tree on-chain, in a view call, in-memory. -However, this is not feasible due to the gas costs associated with it.

    To compute the root of a Merkle tree with 2202^{20} leaves it costs approximately 2 billion gas. +However, this is not feasible due to the gas costs associated with it.

    +

    To compute the root of a Merkle tree with 2202^{20} leaves it costs approximately 2 billion gas. With Infura and Alchemy capping the gas limit to 350M and 550M gas respectively, -it is not possible to compute the root of the tree in a single call.

    Acknowledging that Polygon Miden and Penumbra both make use of a tiered commitment tree, -we propose a similar approach for RLN.

    A tiered commitment tree is a tree which is sharded into multiple smaller subtrees, +it is not possible to compute the root of the tree in a single call.

    +

    Acknowledging that Polygon Miden and Penumbra both make use of a tiered commitment tree, +we propose a similar approach for RLN.

    +

    A tiered commitment tree is a tree which is sharded into multiple smaller subtrees, each of which is a tree in itself. This allows scaling in terms of the number of leaves, -as well as reducing state bloat by just storing the root of a subtree when it is full instead of all its leaves.

    Here, the question arises: -What is the maximum number of leaves in a subtree with which the root can be computed in a single call?

    It costs approximately 217M gas to compute the root of a Merkle tree with 2102^{10} leaves.

    This is a feasible number for a single call, -and hence we propose a tiered commitment tree with a maximum of 2102^{10} leaves in a subtree and the number of subtrees is 2102^{10}. -Therefore, the maximum number of leaves in the tree is 2202^{20} (the same as the current implementation).

    img

    Insertion​

    When a commitment is inserted into the tree it is first inserted into the first subtree. -When the first subtree is full the next insertions go into the second subtree and so on.

    Syncing​

    When syncing the tree, +as well as reducing state bloat by just storing the root of a subtree when it is full instead of all its leaves.

    +

    Here, the question arises: +What is the maximum number of leaves in a subtree with which the root can be computed in a single call?

    +

    It costs approximately 217M gas to compute the root of a Merkle tree with 2102^{10} leaves.

    +

    This is a feasible number for a single call, +and hence we propose a tiered commitment tree with a maximum of 2102^{10} leaves in a subtree and the number of subtrees is 2102^{10}. +Therefore, the maximum number of leaves in the tree is 2202^{20} (the same as the current implementation).

    +

    img

    +

    Insertion​

    +

    When a commitment is inserted into the tree it is first inserted into the first subtree. +When the first subtree is full the next insertions go into the second subtree and so on.

    +

    Syncing​

    +

    When syncing the tree, one only needs to fetch the roots of the subtrees. -The root of the full tree can be computed in-memory or on-chain.

    This allows us to derive the following relation:

    number_of_rpc_calls=number_of_filled_subtrees+1number\_of\_rpc\_calls = number\_of\_filled\_subtrees + 1

    This is a significant improvement over the current implementation, -which requires fetching all the memberships from the smart contract.

    Gas costs​

    The gas costs for inserting a commitment into the tree are the same as the current implementation except it consists of an extra SSTORE operation to store the shardIndex of the commitment.

    Events​

    The events emitted by the contract are the same as the current implementation, -appending the shardIndex of the commitment.

    Proof of concept​

    A proof of concept implementation of the tiered commitment tree is available here, -and is deployed on Sepolia at 0xE7987c70B54Ff32f0D5CBbAA8c8Fc1cAf632b9A5.

    It is compatible with the current implementation of the RLN verifier.

    Future work​

    1. Optimize the gas costs of the tiered commitment tree.
    2. Explore using different number of leaves under a given node in the tree (currently set to 2).

    Conclusion​

    The tiered commitment tree is a promising approach to reduce the number of RPC calls required to sync the tree and reduce the gas costs associated with computing the root of the tree. -Consequently, it allows for a more scalable and efficient RLN verifier.

    References​

    - - +The root of the full tree can be computed in-memory or on-chain.

    +

    This allows us to derive the following relation:

    +number_of_rpc_calls=number_of_filled_subtrees+1number\_of\_rpc\_calls = number\_of\_filled\_subtrees + 1 +

    This is a significant improvement over the current implementation, +which requires fetching all the memberships from the smart contract.

    +

    Gas costs​

    +

    The gas costs for inserting a commitment into the tree are the same as the current implementation except it consists of an extra SSTORE operation to store the shardIndex of the commitment.

    +

    Events​

    +

    The events emitted by the contract are the same as the current implementation, +appending the shardIndex of the commitment.

    +

    Proof of concept​

    +

    A proof of concept implementation of the tiered commitment tree is available here, +and is deployed on Sepolia at 0xE7987c70B54Ff32f0D5CBbAA8c8Fc1cAf632b9A5.

    +

    It is compatible with the current implementation of the RLN verifier.

    +

    Future work​

    +
      +
    1. Optimize the gas costs of the tiered commitment tree.
      2. +
    2. Explore using different number of leaves under a given node in the tree (currently set to 2).
      3. +
    +

    Conclusion​

    +

    The tiered commitment tree is a promising approach to reduce the number of RPC calls required to sync the tree and reduce the gas costs associated with computing the root of the tree. +Consequently, it allows for a more scalable and efficient RLN verifier.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/rln-relay/index.html b/rlog/rln-relay/index.html index d4df4403..0387bb5b 100644 --- a/rlog/rln-relay/index.html +++ b/rlog/rln-relay/index.html @@ -1,35 +1,221 @@ - + - -Privacy-preserving p2p economic spam protection in Waku v2 | Vac Research + +Privacy-preserving p2p economic spam protection in Waku v2 | Vac Research - - - + + + -
    -

    Privacy-preserving p2p economic spam protection in Waku v2

    by
    21 min read

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.

    Introduction​

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol2 through Rate-Limiting Nullifiers3 4 or RLN for short.

    Let me give a little background about Waku(v2)1. Waku is a privacy-preserving peer-to-peer (p2p) messaging protocol for resource-restricted devices. Being p2p means that Waku relies on No central server. Instead, peers collaboratively deliver messages in the network. Waku uses GossipSub16 as the underlying routing protocol (as of the writeup of this post). At a high level, GossipSub is based on publisher-subscriber architecture. That is, peers, congregate around topics they are interested in and can send messages to topics. Each message gets delivered to all peers subscribed to the topic. In GossipSub, a peer has a constant number of direct connections/neighbors. In order to publish a message, the author forwards its message to a subset of neighbors. The neighbors proceed similarly till the message gets propagated in the network of the subscribed peers. The message publishing and routing procedures are part of the Waku Relay17 protocol. -

    Figure 1: An overview of privacy-preserving p2p economic spam protection in Waku v2 RLN-Relay protocol.

    What do we mean by spamming?​

    In centralized messaging systems, a spammer usually indicates an entity that uses the messaging system to send an unsolicited message (spam) to large numbers of recipients. However, in Waku with a p2p architecture, spam messages not only affect the recipients but also all the other peers involved in the routing process as they have to spend their computational power/bandwidth/storage capacity on processing spam messages. As such, we define a spammer as an entity that uses the messaging system to publish a large number of messages in a short amount of time. The messages issued in this way are called spam. In this definition, we disregard the intention of the spammer as well as the content of the message and the number of recipients.

    Possible Solutions​

    Has the spamming issue been addressed before? Of course yes! Here is an overview of the spam protection techniques with their trade-offs and use-cases. In this overview, we distinguish between protection techniques that are targeted for centralized messaging systems and those for p2p architectures.

    Centralized Messaging Systems​

    In traditional centralized messaging systems, spam usually signifies unsolicited messages sent in bulk or messages with malicious content like malware. Protection mechanisms include

    • authentication through some piece of personally identifiable information e.g., phone number
    • checksum-based filtering to protect against messages sent in bulk
    • challenge-response systems
    • content filtering on the server or via a proxy application

    These methods exploit the fact that the messaging system is centralized and a global view of the users' activities is available based on which spamming patterns can be extracted and defeated accordingly. Moreover, users are associated with an identifier e.g., a username which enables the server to profile each user e.g., to detect suspicious behavior like spamming. Such profiling possibility is against the user's anonymity and privacy.

    Among the techniques enumerated above, authentication through phone numbers is a some-what economic-incentive measure as providing multiple valid phone numbers will be expensive for the attacker. Notice that while using an expensive authentication method can reduce the number of accounts owned by a single spammer, cannot address the spam issue entirely. This is because the spammer can still send bulk messages through one single account. For this approach to be effective, a centralized mediator is essential. That is why such a solution would not fit the p2p environments where no centralized control exists.

    P2P Systems​

    What about spam prevention in p2p messaging platforms? There are two techniques, namely Proof of Work8 deployed by Whisper9 and Peer scoring6 method (namely reputation-based approach) adopted by LibP2P. However, each of these solutions has its own shortcomings for real-life use-cases as explained below.

    Proof of work​

    The idea behind the Proof Of Work i.e., POW8 is to make messaging a computationally costly operation hence lowering the messaging rate of all the peers including the spammers. In specific, the message publisher has to solve a puzzle and the puzzle is to find a nonce such that the hash of the message concatenated with the nonce has at least z leading zeros. z is known as the difficulty of the puzzle. Since the hash function is one-way, peers have to brute-force to find a nonce. Hashing is a computationally-heavy operation so is the brute-force. While solving the puzzle is computationally expensive, it is comparatively cheap to verify the solution.

    POW is also used as the underlying mining algorithm in Ethereum and Bitcoin blockchain. There, the goal is to contain the mining speed and allow the decentralized network to come to a consensus, or agree on things like account balances and the order of transactions.

    While the use of POW makes perfect sense in Ethereum / Bitcoin blockchain, it shows practical issues in heterogeneous p2p messaging systems with resource-restricted peers. Some peers won't be able to carry the designated computation and will be effectively excluded. Such exclusion showed to be practically an issue in applications like Status, which used to rely on POW for spam-protection, to the extent that the difficulty level had to be set close to zero.

    Peer Scoring​

    The peer scoring method6 that is utilized by libp2p is to limit the number of messages issued by a peer in connection to another peer. That is each peer monitors all the peers to which it is directly connected and adjusts their messaging quota i.e., to route or not route their messages depending on their past activities. For example, if a peer detects its neighbor is sending more than x messages per month, can drop its quota to z.x where z is less than one. The shortcoming of this solution is that scoring is based on peers' local observations and the concept of the score is defined in relation to one single peer. This leaves room for an attack where a spammer can make connections to k peers in the system and publishes k.(x-1) messages by exploiting all of its k connections. Another attack scenario is through botnets consisting of a large number of e.g., a million bots. The attacker rents a botnet and inserts each of them as a legitimate peer to the network and each can publish x-1 messages per month7.

    Economic-Incentive Spam protection​

    Is this the end of our spam-protection journey? Shall we simply give up and leave spammers be? Certainly not! -Waku RLN-Relay gives us a p2p spam-protection method which:

    • suits p2p systems and does not rely on any central entity.
    • is efficient i.e., with no unreasonable computational, storage, memory, and bandwidth requirement! as such, it fits the network of heterogeneous peers.
    • respects users privacy unlike reputation-based and centralized methods.
    • deploys economic-incentives to contain spammers' activity. Namely, there is a financial sacrifice for those who want to spam the system. How? follow along ...

    We devise a general rule to save everyone's life and that is

    No one can publish more than M messages per epoch without being financially charged!

    We set M to 1 for now, but this can be any arbitrary value. You may be thinking "This is too restrictive! Only one per epoch?". Don't worry, we set the epoch to a reasonable value so that it does not slow down the communication of innocent users but will make the life of spammers harder! Epoch here can be every second, as defined by UTC date-time +-20s.

    The remainder of this post is all about the story of how to enforce this limit on each user's messaging rate as well as how to impose the financial cost when the limit gets violated. This brings us to the Rate Limiting Nullifiers and how we integrate this technique into Waku v2 (in specific the Waku Relay protocol) to protect our valuable users against spammers.

    Technical Terms​

    Zero-knowledge proof: Zero-knowledge proof (ZKP)14 allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do the trust-minimized computation that is also privacy-preserving. As a basic example, instead of showing your ID when going to a bar you simply give them proof that you are over 18, without showing the doorman your id. In this write-up, by ZKP we essentially mean zkSNARK15 which is one of the many types of ZKPs.

    Threshold Secret Sharing Scheme: (m,n) Threshold secret-sharing is a method by which you can split a secret value s into n pieces in a way that the secret s can be reconstructed by having m pieces (m <= n). The economic-incentive spam protection utilizes a (2,n) secret sharing realized by Shamir Secret Sharing Scheme13.

    Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers​

    Context: We started the idea of economic-incentive spam protection more than a year ago and conducted a feasibility study to identify blockers and unknowns. The results are published in our prior post. Since then major progress has been made and the prior identified blockers that are listed below are now addressed. Kudos to Barry WhiteHat, Onur Kilic, Koh Wei Jie for all of their hard work, research, and development which made this progress possible.

    • the proof time22 which was initially in the order of minutes ~10 mins and now is almost 0.5 seconds
    • the prover key size21 which was initially ~110MB and now is ~3.9MB
    • the lack of Shamir logic19 which is now implemented and part of the RLN repository4
    • the concern regarding the potential multi-party computation for the trusted setup of zkSNARKs which got resolved20
    • the lack of end-to-end integration that now we made it possible, have it implemented, and are going to present it in this post. New blockers are also sorted out during the e2e integration which we will discuss in the Feasibility and Open Issues section.

    Now that you have more context, let's see how the final solution works. The fundamental point is to make it economically costly to send more than your share of messages and to do so in a privacy-preserving and e2e fashion. To do that we have the following components:

    • 1- Group: We manage all the peers inside a large group (later we can split peers into smaller groups, but for now consider only one). The group management is done via a smart contract which is devised for this purpose and is deployed on the Ethereum blockchain.
    • 2- Membership: To be able to send messages and in specific for the published messages to get routed by all the peers, publishing peers have to register to the group. Membership involves setting up public and private key pairs (think of it as the username and password). The private key remains at the user side but the public key becomes a part of the group information on the contract (publicly available) and everyone has access to it. Public keys are not human-generated (like usernames) and instead they are random numbers, as such, they do not reveal any information about the owner (think of public keys as pseudonyms). Registration is mandatory for the users who want to publish a message, however, users who only want to listen to the messages are more than welcome and do not have to register in the group.
    • Membership fee: Membership is not for free! each peer has to lock a certain amount of funds during the registration (this means peers have to have an Ethereum account with sufficient balance for this sake). This fund is safely stored on the contract and remains intact unless the peer attempts to break the rules and publish more than one message per epoch.
    • Zero-knowledge Proof of membership: Do you want your message to get routed to its destination, fine, but you have to prove that you are a member of the group (sorry, no one can escape the registration phase!). Now, you may be thinking that should I attach my public key to my message to prove my membership? Absolutely Not! we said that our solution respects privacy! membership proofs are done in a zero-knowledge manner that is each message will carry cryptographic proof asserting that "the message is generated by one of the current members of the group", so your identity remains private and your anonymity is preserved!
    • Slashing through secret sharing: Till now it does not seem like we can catch spammers, right? yes, you are right! now comes the exciting part, detecting spammers and slashing them. The core idea behind the slashing is that each publishing peer (not routing peers!) has to integrate a secret share of its private key inside the message. The secret share is deterministically computed over the private key and the current epoch. The content of this share is harmless for the peer's privacy (it looks random) unless the peer attempts to publish more than one message in the same epoch hence disclosing more than one secret share of its private key. Indeed two distinct shares of the private key under the same epoch are enough to reconstruct the entire private key. Then what should you do with the recovered private key? hurry up! go to the contract and withdraw the private key and claim its fund and get rich!! Are you thinking what if spammers attach junk values instead of valid secret shares? Of course, that wouldn't be cool! so, there is a zero-knowledge proof for this sake as well where the publishing peer has to prove that the secret shares are generated correctly.

    A high-level overview of the economic spam protection is shown in Figure 1.

    Flow​

    In this section, we describe the flow of the economic-incentive spam detection mechanism from the viewpoint of a single peer. An overview of this flow is provided in Figure 3.

    Setup and Registration​

    A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members' public keys. An overview of registration is illustrated in Figure 2.

    For the registration, a peer creates a transaction that sends x amount of Ether to the contract. The peer who has the "private key" sk associated with that deposit would be able to withdraw x Ether by providing valid proof. Note that sk is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Privacy-preserving p2p economic spam protection in Waku v2

    by
    21 min read

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.

    + +

    Introduction​

    +

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol1 through Rate-Limiting Nullifiers2 3 or RLN for short.

    +

    Let me give a little background about Waku(v2)4. Waku is a privacy-preserving peer-to-peer (p2p) messaging protocol for resource-restricted devices. Being p2p means that Waku relies on No central server. Instead, peers collaboratively deliver messages in the network. Waku uses GossipSub5 as the underlying routing protocol (as of the writeup of this post). At a high level, GossipSub is based on publisher-subscriber architecture. That is, peers, congregate around topics they are interested in and can send messages to topics. Each message gets delivered to all peers subscribed to the topic. In GossipSub, a peer has a constant number of direct connections/neighbors. In order to publish a message, the author forwards its message to a subset of neighbors. The neighbors proceed similarly till the message gets propagated in the network of the subscribed peers. The message publishing and routing procedures are part of the Waku Relay6 protocol. +

    Figure 1: An overview of privacy-preserving p2p economic spam protection in Waku v2 RLN-Relay protocol.

    +

    What do we mean by spamming?​

    +

    In centralized messaging systems, a spammer usually indicates an entity that uses the messaging system to send an unsolicited message (spam) to large numbers of recipients. However, in Waku with a p2p architecture, spam messages not only affect the recipients but also all the other peers involved in the routing process as they have to spend their computational power/bandwidth/storage capacity on processing spam messages. As such, we define a spammer as an entity that uses the messaging system to publish a large number of messages in a short amount of time. The messages issued in this way are called spam. In this definition, we disregard the intention of the spammer as well as the content of the message and the number of recipients.

    +

    Possible Solutions​

    +

    Has the spamming issue been addressed before? Of course yes! Here is an overview of the spam protection techniques with their trade-offs and use-cases. In this overview, we distinguish between protection techniques that are targeted for centralized messaging systems and those for p2p architectures.

    +

    Centralized Messaging Systems​

    +

    In traditional centralized messaging systems, spam usually signifies unsolicited messages sent in bulk or messages with malicious content like malware. Protection mechanisms include

    +
      +
    • authentication through some piece of personally identifiable information e.g., phone number
      • +
    • checksum-based filtering to protect against messages sent in bulk
      • +
    • challenge-response systems
      • +
    • content filtering on the server or via a proxy application
      • +
    +

    These methods exploit the fact that the messaging system is centralized and a global view of the users' activities is available based on which spamming patterns can be extracted and defeated accordingly. Moreover, users are associated with an identifier e.g., a username which enables the server to profile each user e.g., to detect suspicious behavior like spamming. Such profiling possibility is against the user's anonymity and privacy.

    +

    Among the techniques enumerated above, authentication through phone numbers is a some-what economic-incentive measure as providing multiple valid phone numbers will be expensive for the attacker. Notice that while using an expensive authentication method can reduce the number of accounts owned by a single spammer, cannot address the spam issue entirely. This is because the spammer can still send bulk messages through one single account. For this approach to be effective, a centralized mediator is essential. That is why such a solution would not fit the p2p environments where no centralized control exists.

    +

    P2P Systems​

    +

    What about spam prevention in p2p messaging platforms? There are two techniques, namely Proof of Work7 deployed by Whisper8 and Peer scoring9 method (namely reputation-based approach) adopted by LibP2P. However, each of these solutions has its own shortcomings for real-life use-cases as explained below.

    +

    Proof of work​

    +

    The idea behind the Proof Of Work i.e., POW7 is to make messaging a computationally costly operation hence lowering the messaging rate of all the peers including the spammers. In specific, the message publisher has to solve a puzzle and the puzzle is to find a nonce such that the hash of the message concatenated with the nonce has at least z leading zeros. z is known as the difficulty of the puzzle. Since the hash function is one-way, peers have to brute-force to find a nonce. Hashing is a computationally-heavy operation so is the brute-force. While solving the puzzle is computationally expensive, it is comparatively cheap to verify the solution.

    +

    POW is also used as the underlying mining algorithm in Ethereum and Bitcoin blockchain. There, the goal is to contain the mining speed and allow the decentralized network to come to a consensus, or agree on things like account balances and the order of transactions.

    +

    While the use of POW makes perfect sense in Ethereum / Bitcoin blockchain, it shows practical issues in heterogeneous p2p messaging systems with resource-restricted peers. Some peers won't be able to carry the designated computation and will be effectively excluded. Such exclusion showed to be practically an issue in applications like Status, which used to rely on POW for spam-protection, to the extent that the difficulty level had to be set close to zero.

    +

    Peer Scoring​

    +

    The peer scoring method9 that is utilized by libp2p is to limit the number of messages issued by a peer in connection to another peer. That is each peer monitors all the peers to which it is directly connected and adjusts their messaging quota i.e., to route or not route their messages depending on their past activities. For example, if a peer detects its neighbor is sending more than x messages per month, can drop its quota to z.x where z is less than one. The shortcoming of this solution is that scoring is based on peers' local observations and the concept of the score is defined in relation to one single peer. This leaves room for an attack where a spammer can make connections to k peers in the system and publishes k.(x-1) messages by exploiting all of its k connections. Another attack scenario is through botnets consisting of a large number of e.g., a million bots. The attacker rents a botnet and inserts each of them as a legitimate peer to the network and each can publish x-1 messages per month10.

    +

    Economic-Incentive Spam protection​

    +

    Is this the end of our spam-protection journey? Shall we simply give up and leave spammers be? Certainly not! +Waku RLN-Relay gives us a p2p spam-protection method which:

    +
      +
    • suits p2p systems and does not rely on any central entity.
      • +
    • is efficient i.e., with no unreasonable computational, storage, memory, and bandwidth requirement! as such, it fits the network of heterogeneous peers.
      • +
    • respects users privacy unlike reputation-based and centralized methods.
      • +
    • deploys economic-incentives to contain spammers' activity. Namely, there is a financial sacrifice for those who want to spam the system. How? follow along ...
      • +
    +

    We devise a general rule to save everyone's life and that is

    +

    No one can publish more than M messages per epoch without being financially charged!

    +

    We set M to 1 for now, but this can be any arbitrary value. You may be thinking "This is too restrictive! Only one per epoch?". Don't worry, we set the epoch to a reasonable value so that it does not slow down the communication of innocent users but will make the life of spammers harder! Epoch here can be every second, as defined by UTC date-time +-20s.

    +

    The remainder of this post is all about the story of how to enforce this limit on each user's messaging rate as well as how to impose the financial cost when the limit gets violated. This brings us to the Rate Limiting Nullifiers and how we integrate this technique into Waku v2 (in specific the Waku Relay protocol) to protect our valuable users against spammers.

    +

    Technical Terms​

    +

    Zero-knowledge proof: Zero-knowledge proof (ZKP)11 allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do the trust-minimized computation that is also privacy-preserving. As a basic example, instead of showing your ID when going to a bar you simply give them proof that you are over 18, without showing the doorman your id. In this write-up, by ZKP we essentially mean zkSNARK12 which is one of the many types of ZKPs.

    +

    Threshold Secret Sharing Scheme: (m,n) Threshold secret-sharing is a method by which you can split a secret value s into n pieces in a way that the secret s can be reconstructed by having m pieces (m <= n). The economic-incentive spam protection utilizes a (2,n) secret sharing realized by Shamir Secret Sharing Scheme13.

    +

    Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers​

    +

    Context: We started the idea of economic-incentive spam protection more than a year ago and conducted a feasibility study to identify blockers and unknowns. The results are published in our prior post. Since then major progress has been made and the prior identified blockers that are listed below are now addressed. Kudos to Barry WhiteHat, Onur Kilic, Koh Wei Jie for all of their hard work, research, and development which made this progress possible.

    +
      +
    • the proof time14 which was initially in the order of minutes ~10 mins and now is almost 0.5 seconds
      • +
    • the prover key size15 which was initially ~110MB and now is ~3.9MB
      • +
    • the lack of Shamir logic16 which is now implemented and part of the RLN repository3
      • +
    • the concern regarding the potential multi-party computation for the trusted setup of zkSNARKs which got resolved17
      • +
    • the lack of end-to-end integration that now we made it possible, have it implemented, and are going to present it in this post. New blockers are also sorted out during the e2e integration which we will discuss in the Feasibility and Open Issues section.
      • +
    +

    Now that you have more context, let's see how the final solution works. The fundamental point is to make it economically costly to send more than your share of messages and to do so in a privacy-preserving and e2e fashion. To do that we have the following components:

    +
      +
    • 1- Group: We manage all the peers inside a large group (later we can split peers into smaller groups, but for now consider only one). The group management is done via a smart contract which is devised for this purpose and is deployed on the Ethereum blockchain.
      • +
    • 2- Membership: To be able to send messages and in specific for the published messages to get routed by all the peers, publishing peers have to register to the group. Membership involves setting up public and private key pairs (think of it as the username and password). The private key remains at the user side but the public key becomes a part of the group information on the contract (publicly available) and everyone has access to it. Public keys are not human-generated (like usernames) and instead they are random numbers, as such, they do not reveal any information about the owner (think of public keys as pseudonyms). Registration is mandatory for the users who want to publish a message, however, users who only want to listen to the messages are more than welcome and do not have to register in the group.
      • +
    • Membership fee: Membership is not for free! each peer has to lock a certain amount of funds during the registration (this means peers have to have an Ethereum account with sufficient balance for this sake). This fund is safely stored on the contract and remains intact unless the peer attempts to break the rules and publish more than one message per epoch.
      • +
    • Zero-knowledge Proof of membership: Do you want your message to get routed to its destination, fine, but you have to prove that you are a member of the group (sorry, no one can escape the registration phase!). Now, you may be thinking that should I attach my public key to my message to prove my membership? Absolutely Not! we said that our solution respects privacy! membership proofs are done in a zero-knowledge manner that is each message will carry cryptographic proof asserting that "the message is generated by one of the current members of the group", so your identity remains private and your anonymity is preserved!
      • +
    • Slashing through secret sharing: Till now it does not seem like we can catch spammers, right? yes, you are right! now comes the exciting part, detecting spammers and slashing them. The core idea behind the slashing is that each publishing peer (not routing peers!) has to integrate a secret share of its private key inside the message. The secret share is deterministically computed over the private key and the current epoch. The content of this share is harmless for the peer's privacy (it looks random) unless the peer attempts to publish more than one message in the same epoch hence disclosing more than one secret share of its private key. Indeed two distinct shares of the private key under the same epoch are enough to reconstruct the entire private key. Then what should you do with the recovered private key? hurry up! go to the contract and withdraw the private key and claim its fund and get rich!! Are you thinking what if spammers attach junk values instead of valid secret shares? Of course, that wouldn't be cool! so, there is a zero-knowledge proof for this sake as well where the publishing peer has to prove that the secret shares are generated correctly.
      • +
    +

    A high-level overview of the economic spam protection is shown in Figure 1.

    +

    Flow​

    +

    In this section, we describe the flow of the economic-incentive spam detection mechanism from the viewpoint of a single peer. An overview of this flow is provided in Figure 3.

    +

    Setup and Registration​

    +

    A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members' public keys. An overview of registration is illustrated in Figure 2.

    +

    For the registration, a peer creates a transaction that sends x amount of Ether to the contract. The peer who has the "private key" sk associated with that deposit would be able to withdraw x Ether by providing valid proof. Note that sk is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch. The following relation holds between the sk and pk i.e., pk = H(sk) where H denotes a hash function. -

    Figure 2: Registration

    Maintaining the membership Merkle Tree​

    The ZKP of membership that we mentioned before relies on the representation of the entire group as a Merkle Tree. The tree construction and maintenance is delegated to the peers (the initial idea was to keep the tree on the chain as part of the contract, however, the cost associated with member deletion and insertion was high and unreasonable, please see Feasibility and Open Issues for more details). As such, each peer needs to build the tree locally and sync itself with the contract updates (peer insertion and deletion) to mirror them on its tree. -Two pieces of information of the tree are important as they enable peers to generate zero-knowledge proofs. One is the root of the tree and the other is the membership proof (or the authentication path). The tree root is public information whereas the membership proof is private data (or more precisely the index of the peer in the tree).

    Publishing​

    In order to publish at a given epoch, each message must carry a proof i.e., a zero-knowledge proof signifying that the publishing peer is a registered member, and has not exceeded the messaging rate at the given epoch.

    Recall that the enforcement of the messaging rate was through associating a secret shared version of the peer's sk into the message together with a ZKP that the secret shares are constructed correctly. As for the secret sharing part, the peer generates the following data:

    1. shareX
    2. shareY
    3. nullifier

    The pair (shareX, shareY) is the secret shared version of sk that are generated using Shamir secret sharing scheme. Having two such pairs for an identical nullifier results in full disclosure of peer's sk and hence burning the associated deposit. Note that the nullifier is a deterministic value derived from sk and epoch therefore any two messages issued by the same peer (i.e., using the same sk) for the same epoch are guaranteed to have identical nullifiers.

    Finally, the peer generates a zero-knowledge proof zkProof asserting the membership of the peer in the group and the correctness of the attached secret share (shareX, shareY) and the nullifier. In order to generate a valid proof, the peer needs to have two private inputs i.e., its sk and its authentication path. Other inputs are the tree root, epoch, and the content of the message.

    Privacy Hint: Note that the authentication path of each peer depends on the recent list of members (hence changes when new peers register or leave). As such, it is recommended (and necessary for privacy/anonymity) that the publisher updates her authentication path based on the latest status of the group and attempts the proof using the updated version.

    An overview of the publishing procedure is provided in Figure 3.

    Routing​

    Upon the receipt of a message, the routing peer needs to decide whether to route it or not. This decision relies on the following factors:

    1. If the epoch value attached to the message has a non-reasonable gap with the routing peer's current epoch then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs).
    2. The message MUST contain valid proof that gets verified by the routing peer. -If the preceding checks are passed successfully, then the message is relayed. In case of an invalid proof, the message is dropped. If spamming is detected, the publishing peer gets slashed (see Spam Detection and Slashing).

    An overview of the routing procedure is provided in Figure 3.

    Spam Detection and Slashing​

    In order to enable local spam detection and slashing, routing peers MUST record the nullifier, shareX, and shareY of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.

    1. The routing peer first verifies the zkProof and drops the message if not verified.
    2. Otherwise, it checks whether a message with an identical nullifier has already been relayed.
      • a) If such message exists and its shareX and shareY components are different from the incoming message, then slashing takes place (if the shareX and shareY fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).
      • b) If none found, then the message gets relayed.

    An overview of the slashing procedure is provided in Figure 3. -

    Figure 3: Publishing, Routing and Slashing workflow.

    Feasibility and Open Issues​

    We've come a long way since a year ago, blockers resolved, now we have implemented it end-to-end. We learned lot and could identify further issues and unknowns some of which are blocking getting to production. The summary of the identified issues are presented below.

    Storage overhead per peer​

    Currently, peers are supposed to maintain the entire tree locally and it imposes storage overhead which is linear in the size of the group (see this issue11 for more details). One way to cope with this is to use the light-node and full-node paradigm in which only a subset of peers who are more resourceful retain the tree whereas the light nodes obtain the necessary information by interacting with the full nodes. Another way to approach this problem is through a more storage efficient method (as described in this research issue12) where peers store a partial view of the tree instead of the entire tree. Keeping the partial view lowers the storage complexity to O(log(N)) where N is the size of the group. There are still unknown unknowns to this solution, as such, it must be studied further to become fully functional.

    Cost-effective way of member insertion and deletion​

    Currently, the cost associated with RLN-Relay membership is around 30 USD10. We aim at finding a more cost-effective approach. Please feel free to share with us your solution ideas in this regard in this issue.

    Exceeding the messaging rate via multiple registrations​

    While the economic-incentive solution has an economic incentive to discourage spamming, we should note that there is still expensive attack(s)23 that a spammer can launch to break the messaging rate limit. That is, the attacker can pay for multiple legit registrations e.g., k, hence being able to publish k messages per epoch. We believe that the higher the membership fee is, the less probable would be such an attack, hence a stronger level of spam-protection can be achieved. Following this argument, the high fee associated with the membership (which we listed above as an open problem) can indeed be contributing to a better protection level.

    Conclusion and Future Steps​

    As discussed in this post, Waku RLN Relay can achieve a privacy-preserving economic spam protection through rate-limiting nullifiers. The idea is to financially discourage peers from publishing more than one message per epoch. In specific, exceeding the messaging rate results in a financial charge. Those who violate this rule are called spammers and their messages are spam. The identification of spammers does not rely on any central entity. Also, the financial punishment of spammers is cryptographically guaranteed. +

    Figure 2: Registration

    +

    Maintaining the membership Merkle Tree​

    +

    The ZKP of membership that we mentioned before relies on the representation of the entire group as a Merkle Tree. The tree construction and maintenance is delegated to the peers (the initial idea was to keep the tree on the chain as part of the contract, however, the cost associated with member deletion and insertion was high and unreasonable, please see Feasibility and Open Issues for more details). As such, each peer needs to build the tree locally and sync itself with the contract updates (peer insertion and deletion) to mirror them on its tree. +Two pieces of information of the tree are important as they enable peers to generate zero-knowledge proofs. One is the root of the tree and the other is the membership proof (or the authentication path). The tree root is public information whereas the membership proof is private data (or more precisely the index of the peer in the tree).

    +

    Publishing​

    +

    In order to publish at a given epoch, each message must carry a proof i.e., a zero-knowledge proof signifying that the publishing peer is a registered member, and has not exceeded the messaging rate at the given epoch.

    +

    Recall that the enforcement of the messaging rate was through associating a secret shared version of the peer's sk into the message together with a ZKP that the secret shares are constructed correctly. As for the secret sharing part, the peer generates the following data:

    +
      +
    1. shareX
      2. +
    2. shareY
      3. +
    3. nullifier
      4. +
    +

    The pair (shareX, shareY) is the secret shared version of sk that are generated using Shamir secret sharing scheme. Having two such pairs for an identical nullifier results in full disclosure of peer's sk and hence burning the associated deposit. Note that the nullifier is a deterministic value derived from sk and epoch therefore any two messages issued by the same peer (i.e., using the same sk) for the same epoch are guaranteed to have identical nullifiers.

    +

    Finally, the peer generates a zero-knowledge proof zkProof asserting the membership of the peer in the group and the correctness of the attached secret share (shareX, shareY) and the nullifier. In order to generate a valid proof, the peer needs to have two private inputs i.e., its sk and its authentication path. Other inputs are the tree root, epoch, and the content of the message.

    +

    Privacy Hint: Note that the authentication path of each peer depends on the recent list of members (hence changes when new peers register or leave). As such, it is recommended (and necessary for privacy/anonymity) that the publisher updates her authentication path based on the latest status of the group and attempts the proof using the updated version.

    +

    An overview of the publishing procedure is provided in Figure 3.

    +

    Routing​

    +

    Upon the receipt of a message, the routing peer needs to decide whether to route it or not. This decision relies on the following factors:

    +
      +
    1. If the epoch value attached to the message has a non-reasonable gap with the routing peer's current epoch then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs).
      2. +
    2. The message MUST contain valid proof that gets verified by the routing peer. +If the preceding checks are passed successfully, then the message is relayed. In case of an invalid proof, the message is dropped. If spamming is detected, the publishing peer gets slashed (see Spam Detection and Slashing).
      3. +
    +

    An overview of the routing procedure is provided in Figure 3.

    +

    Spam Detection and Slashing​

    +

    In order to enable local spam detection and slashing, routing peers MUST record the nullifier, shareX, and shareY of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.

    +
      +
    1. The routing peer first verifies the zkProof and drops the message if not verified.
      2. +
    2. Otherwise, it checks whether a message with an identical nullifier has already been relayed. +
        +
      • a) If such message exists and its shareX and shareY components are different from the incoming message, then slashing takes place (if the shareX and shareY fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).
        • +
      • b) If none found, then the message gets relayed.
        • +
      +
      3. +
    +

    An overview of the slashing procedure is provided in Figure 3. +

    Figure 3: Publishing, Routing and Slashing workflow.

    +

    Feasibility and Open Issues​

    +

    We've come a long way since a year ago, blockers resolved, now we have implemented it end-to-end. We learned lot and could identify further issues and unknowns some of which are blocking getting to production. The summary of the identified issues are presented below.

    +

    Storage overhead per peer​

    +

    Currently, peers are supposed to maintain the entire tree locally and it imposes storage overhead which is linear in the size of the group (see this issue18 for more details). One way to cope with this is to use the light-node and full-node paradigm in which only a subset of peers who are more resourceful retain the tree whereas the light nodes obtain the necessary information by interacting with the full nodes. Another way to approach this problem is through a more storage efficient method (as described in this research issue19) where peers store a partial view of the tree instead of the entire tree. Keeping the partial view lowers the storage complexity to O(log(N)) where N is the size of the group. There are still unknown unknowns to this solution, as such, it must be studied further to become fully functional.

    +

    Cost-effective way of member insertion and deletion​

    +

    Currently, the cost associated with RLN-Relay membership is around 30 USD20. We aim at finding a more cost-effective approach. Please feel free to share with us your solution ideas in this regard in this issue.

    +

    Exceeding the messaging rate via multiple registrations​

    +

    While the economic-incentive solution has an economic incentive to discourage spamming, we should note that there is still expensive attack(s)21 that a spammer can launch to break the messaging rate limit. That is, the attacker can pay for multiple legit registrations e.g., k, hence being able to publish k messages per epoch. We believe that the higher the membership fee is, the less probable would be such an attack, hence a stronger level of spam-protection can be achieved. Following this argument, the high fee associated with the membership (which we listed above as an open problem) can indeed be contributing to a better protection level.

    +

    Conclusion and Future Steps​

    +

    As discussed in this post, Waku RLN Relay can achieve a privacy-preserving economic spam protection through rate-limiting nullifiers. The idea is to financially discourage peers from publishing more than one message per epoch. In specific, exceeding the messaging rate results in a financial charge. Those who violate this rule are called spammers and their messages are spam. The identification of spammers does not rely on any central entity. Also, the financial punishment of spammers is cryptographically guaranteed. In this solution, privacy is guaranteed since: 1) Peers do not have to disclose any piece of personally identifiable information in any phase i.e., neither in the registration nor in the messaging phase 2) Peers can prove that they have not exceeded the messaging rate in a zero-knowledge manner and without leaving any trace to their membership accounts. -Furthermore, all the computations are light hence this solution fits the heterogenous p2p messaging system. Note that the zero-knowledge proof parts are handled through zkSNARKs and the benchmarking result can be found in the RLN benchmark report5.

    Future steps:

    We are still at the PoC level, and the development is in progress. As our future steps,

    • we would like to evaluate the running time associated with the Merkle tree operations. Indeed, the need to locally store Merkle tree on each peer was one of the unknowns discovered during this PoC and yet the concrete benchmarking result in this regard is not available.
    • We would also like to pursue our storage-efficient Merkle Tree maintenance solution in order to lower the storage overhead of peers.
    • In line with the storage optimization, the full-node light-node structure is another path to follow.
    • Another possible improvement is to replace the membership contract with a distributed group management scheme e.g., through distributed hash tables. This is to address possible performance issues that the interaction with the Ethereum blockchain may cause. For example, the registration transactions are subject to delay as they have to be mined before being visible in the state of the membership contract. This means peers have to wait for some time before being able to publish any message.

    Acknowledgement​

    Thanks to Onur Kılıç for his explanation and pointers and for assisting with development and runtime issues. Also thanks to Barry Whitehat for his time and insightful comments. Special thanks to Oskar Thoren for his constructive comments and his guides during the development of this PoC and the writeup of this post.

    References​

    1. RLN-Relay specification: https://rfc.vac.dev/waku/standards/core/17/rln-relay↩
    2. RLN documentation: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both↩
    3. RLN repositories: https://github.com/kilic/RLN and https://github.com/kilic/rlnapp↩
    4. Waku v2: https://rfc.vac.dev/waku/standards/core/10/waku2↩
    5. GossipSub: https://docs.libp2p.io/concepts/publish-subscribe/↩
    6. Waku Relay: https://rfc.vac.dev/waku/standards/core/11/relay↩
    7. Proof of work: http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf and https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf↩
    8. EIP-627 Whisper: https://eips.ethereum.org/EIPS/eip-627↩
    9. Peer Scoring: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring↩
    10. Peer scoring security issues: https://github.com/vacp2p/research/issues/44↩
    11. Zero Knowledge Proof: https://dl.acm.org/doi/abs/10.1145/3335741.3335750 and https://en.wikipedia.org/wiki/Zero-knowledge_proof↩
    12. zkSNARKs: https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11 and https://coinpare.io/whitepaper/zcash.pdf↩
    13. Shamir Secret Sharing Scheme: https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing↩
    14. zkSNARKs proof time: https://github.com/vacp2p/research/issues/7↩
    15. Prover key size: https://github.com/vacp2p/research/issues/8↩
    16. The lack of Shamir secret sharing in zkSNARKs: https://github.com/vacp2p/research/issues/10↩
    17. The MPC required for zkSNARKs trusted setup: https://github.com/vacp2p/research/issues/9↩
    18. Storage overhead per peer: https://github.com/vacp2p/research/issues/57↩
    19. Storage-efficient Merkle Tree maintenance: https://github.com/vacp2p/research/pull/54↩
    20. Cost-effective way of member insertion and deletion: https://github.com/vacp2p/research/issues/56↩
    21. Attack on the messaging rate: https://github.com/vacp2p/specs/issues/251↩
    22. RLN Benchmark: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks↩
    - - +Furthermore, all the computations are light hence this solution fits the heterogenous p2p messaging system. Note that the zero-knowledge proof parts are handled through zkSNARKs and the benchmarking result can be found in the RLN benchmark report22.

    +

    Future steps:

    +

    We are still at the PoC level, and the development is in progress. As our future steps,

    +
      +
    • we would like to evaluate the running time associated with the Merkle tree operations. Indeed, the need to locally store Merkle tree on each peer was one of the unknowns discovered during this PoC and yet the concrete benchmarking result in this regard is not available.
      • +
    • We would also like to pursue our storage-efficient Merkle Tree maintenance solution in order to lower the storage overhead of peers.
      • +
    • In line with the storage optimization, the full-node light-node structure is another path to follow.
      • +
    • Another possible improvement is to replace the membership contract with a distributed group management scheme e.g., through distributed hash tables. This is to address possible performance issues that the interaction with the Ethereum blockchain may cause. For example, the registration transactions are subject to delay as they have to be mined before being visible in the state of the membership contract. This means peers have to wait for some time before being able to publish any message.
      • +
    +

    Acknowledgement​

    +

    Thanks to Onur Kılıç for his explanation and pointers and for assisting with development and runtime issues. Also thanks to Barry Whitehat for his time and insightful comments. Special thanks to Oskar Thoren for his constructive comments and his guides during the development of this PoC and the writeup of this post.

    +

    References​

    + +

    Footnotes​

    +
      +
    1. +

      RLN-Relay specification: https://rfc.vac.dev/waku/standards/core/17/rln-relay ↩

      +
      2. +
    2. +

      RLN documentation: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both ↩

      +
      3. +
    3. +

      RLN repositories: https://github.com/kilic/RLN and https://github.com/kilic/rlnapp ↩ ↩2

      +
      4. +
    4. +

      Waku v2: https://rfc.vac.dev/waku/standards/core/10/waku2 ↩

      +
      5. +
    5. +

      GossipSub: https://docs.libp2p.io/concepts/publish-subscribe/ ↩

      +
      6. +
    6. +

      Waku Relay: https://rfc.vac.dev/waku/standards/core/11/relay ↩

      +
      7. +
    7. +

      Proof of work: http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf and https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf ↩ ↩2

      +
      8. +
    8. +

      EIP-627 Whisper: https://eips.ethereum.org/EIPS/eip-627 ↩

      +
      9. +
    9. +

      Peer Scoring: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring ↩ ↩2

      +
      10. +
    10. +

      Peer scoring security issues: https://github.com/vacp2p/research/issues/44 ↩

      +
      11. +
    11. +

      Zero Knowledge Proof: https://dl.acm.org/doi/abs/10.1145/3335741.3335750 and https://en.wikipedia.org/wiki/Zero-knowledge_proof ↩

      +
      12. +
    12. +

      zkSNARKs: https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11 and https://coinpare.io/whitepaper/zcash.pdf ↩

      +
      13. +
    13. +

      Shamir Secret Sharing Scheme: https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing ↩

      +
      14. +
    14. +

      zkSNARKs proof time: https://github.com/vacp2p/research/issues/7 ↩

      +
      15. +
    15. +

      Prover key size: https://github.com/vacp2p/research/issues/8 ↩

      +
      16. +
    16. +

      The lack of Shamir secret sharing in zkSNARKs: https://github.com/vacp2p/research/issues/10 ↩

      +
      17. +
    17. +

      The MPC required for zkSNARKs trusted setup: https://github.com/vacp2p/research/issues/9 ↩

      +
      18. +
    18. +

      Storage overhead per peer: https://github.com/vacp2p/research/issues/57 ↩

      +
      19. +
    19. +

      Storage-efficient Merkle Tree maintenance: https://github.com/vacp2p/research/pull/54 ↩

      +
      20. +
    20. +

      Cost-effective way of member insertion and deletion: https://github.com/vacp2p/research/issues/56 ↩

      +
      21. +
    21. +

      Attack on the messaging rate: https://github.com/vacp2p/specs/issues/251 ↩

      +
      22. +
    22. +

      RLN Benchmark: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks ↩

      +
      23. +
    +
    \ No newline at end of file diff --git a/rlog/rln-v3/index.html b/rlog/rln-v3/index.html index f55882c7..f6e56379 100644 --- a/rlog/rln-v3/index.html +++ b/rlog/rln-v3/index.html @@ -1,73 +1,187 @@ - + - -RLN-v3: Towards a Flexible and Cost-Efficient Implementation | Vac Research + +RLN-v3: Towards a Flexible and Cost-Efficient Implementation | Vac Research - - - - + + + + -
    -

    RLN-v3: Towards a Flexible and Cost-Efficient Implementation

    by
    7 min read

    Improving on the previous version of RLN by allowing dynamic epoch sizes.

    Introduction​

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    The premise of RLN-v3 is to have a variable message rate per variable epoch, -which can be explained in the following way:

    • RLN-v1: “Alice can send 1 message per global epoch”

      Practically, this is 1 msg/second

    • RLN-v2: “Alice can send x messages per global epoch”

      Practically, this is x msg/second

    • RLN-v3: “Alice can send x messages within a time interval y chosen by herself. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

      RLN-v3: Towards a Flexible and Cost-Efficient Implementation

      by
      7 min read

      Improving on the previous version of RLN by allowing dynamic epoch sizes.

      + +

      Introduction​

      +

      Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

      +

      The premise of RLN-v3 is to have a variable message rate per variable epoch, +which can be explained in the following way:

      +
        +
      • +

        RLN-v1: “Alice can send 1 message per global epoch”

        +

        Practically, this is 1 msg/second

        +
        • +
      • +

        RLN-v2: “Alice can send x messages per global epoch”

        +

        Practically, this is x msg/second

        +
        • +
      • +

        RLN-v3: “Alice can send x messages within a time interval y chosen by herself. The funds she has to pay are affected by both the number of messages and the chosen time interval. -Other participants can choose different time intervals fitting their specific needs.

        Practically, this is x msg/y seconds

      RLN-v3 allows higher flexibility and ease of payment/stake for users who have more predictable usage patterns and therefore, -more predictable bandwidth usage on a p2p network (Waku, etc.).

      For example:

      • An AMM that broadcasts bids, asks, and fills over Waku may require a lot of throughput in the smallest epoch possible and hence may register an RLN-v3 membership of 10000 msg/1 second. -They could do this with RLN-v2, too.
      • Alice, a casual user of a messaging app built on Waku, who messages maybe 3-4 people infrequently during the day, may register an RLN-v3 membership of 100 msg/hour, +Other participants can choose different time intervals fitting their specific needs.

        +

        Practically, this is x msg/y seconds

        +
        • +
      +

      RLN-v3 allows higher flexibility and ease of payment/stake for users who have more predictable usage patterns and therefore, +more predictable bandwidth usage on a p2p network (Waku, etc.).

      +

      For example:

      +
        +
      • An AMM that broadcasts bids, asks, and fills over Waku may require a lot of throughput in the smallest epoch possible and hence may register an RLN-v3 membership of 10000 msg/1 second. +They could do this with RLN-v2, too.
        • +
      • Alice, a casual user of a messaging app built on Waku, who messages maybe 3-4 people infrequently during the day, may register an RLN-v3 membership of 100 msg/hour, which would not be possible in RLN-v2 considering the global epoch was set to 1 second. With RLN-v2, Alice would have to register with a membership of 1 msg/sec, which would translate to 3600 msg/hour. This is much higher than her usage and would -result in her overpaying to stake into the membership set.
      • A sync service built over Waku, +result in her overpaying to stake into the membership set.
        • +
      • A sync service built over Waku, whose spec defines that it MUST broadcast a set of public keys every hour, may register an RLN-v3 membership of 1 msg/hour, -cutting down the costs to enter the membership set earlier.

      Theory​

      Modification to leaves set in the membership Merkle tree​

      To ensure that a user’s epoch size (user_epoch_limit) is included within their membership we must modify the user’s commitment/leaf in the tree to contain it. +cutting down the costs to enter the membership set earlier.

      • +
    +

    Theory​

    +

    Modification to leaves set in the membership Merkle tree​

    +

    To ensure that a user’s epoch size (user_epoch_limit) is included within their membership we must modify the user’s commitment/leaf in the tree to contain it. A user’s commitment/leaf in the tree is referred to as a rate_commitment, which was previously derived from their public key (identity_commitment) -and their variable message rate (user_message_limit).

    In RLN-v2:

    rate_commitment=poseidon([identity_commitment,user_message_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit])

    In RLN-v3:

    rate_commitment=poseidon([identity_commitment,user_message_limit,user_epoch_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit, user\_epoch\_limit])

    Modification to circuit inputs​

    To detect double signaling, +and their variable message rate (user_message_limit).

    +

    In RLN-v2:

    +rate_commitment=poseidon([identity_commitment,user_message_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit]) +

    In RLN-v3:

    +rate_commitment=poseidon([identity_commitment,user_message_limit,user_epoch_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit, user\_epoch\_limit]) +

    Modification to circuit inputs​

    +

    To detect double signaling, we make use of a circuit output nullifier, which remains the same if a user generates a proof with the same message_id and external_nullifier, -where the external_nullifier and nullifier are defined as:

    external_nullifier=poseidon([epoch,rln_identifier])nullifier=poseidon([identity_secret,external_nullifier,message_id])external\_nullifier = poseidon([epoch, rln\_identifier]) \\ nullifier = poseidon([identity\_secret, external\_nullifier, message\_id])

    Where:

    • epoch is defined as the Unix epoch timestamp with seconds precision.
    • rln_identifier uniquely identifies an application for which a user submits a proof.
    • identity_secret is the private key of the user.
    • message_id is the sequence number of the user’s message within user_message_limit in an epoch.

    In RLN-v2, the global epoch was 1 second, +where the external_nullifier and nullifier are defined as:

    +external_nullifier=poseidon([epoch,rln_identifier])nullifier=poseidon([identity_secret,external_nullifier,message_id])external\_nullifier = poseidon([epoch, rln\_identifier]) \\ +nullifier = poseidon([identity\_secret, external\_nullifier, message\_id]) +

    Where:

    +
      +
    • epoch is defined as the Unix epoch timestamp with seconds precision.
      • +
    • rln_identifier uniquely identifies an application for which a user submits a proof.
      • +
    • identity_secret is the private key of the user.
      • +
    • message_id is the sequence number of the user’s message within user_message_limit in an epoch.
      • +
    +

    In RLN-v2, the global epoch was 1 second, hence we did not need to perform any assertions to the epoch’s value inside the circuit, -and the validation of the epoch was handled off-circuit (i.e., too old, too large, bad values, etc.).

    In RLN-v3, we propose that the epoch that is passed into the circuit +and the validation of the epoch was handled off-circuit (i.e., too old, too large, bad values, etc.).

    +

    In RLN-v3, we propose that the epoch that is passed into the circuit must be a valid multiple of user_epoch_limit -since the user may pass in values of the epoch which do not directly correlate with the user_epoch_limit.

    For example:

    • A user with user_epoch_limit of 120 +since the user may pass in values of the epoch which do not directly correlate with the user_epoch_limit.

      +

      For example:

      +
        +
      • A user with user_epoch_limit of 120 passes in an epoch of 237 generates user_message_limit proofs with it, can increment the epoch by 1, and generate user_message_limit proofs with it, -thereby allowing them to bypass the message per epoch restriction.

      One could say that we could perform this validation outside of the circuit, +thereby allowing them to bypass the message per epoch restriction.

      • +
    +

    One could say that we could perform this validation outside of the circuit, but we maintain the user_epoch_limit as a private input to the circuit so that the user is not deanonymized by the anonymity set connected to that user_epoch_limit. Since user_epoch_limit is kept private, -the verifier does not have access to that value and cannot perform validation on it.

    If we ensure that the epoch is a multiple of user_epoch_limit, -we have the following scenarios:

    • A user with user_epoch_limit of 120 +the verifier does not have access to that value and cannot perform validation on it.

      +

      If we ensure that the epoch is a multiple of user_epoch_limit, +we have the following scenarios:

      +
        +
      • A user with user_epoch_limit of 120 passes in an epoch of 237. -Proof generation fails since the epoch is not a multiple of user_epoch_limit.
      • A user with user_epoch_limit of 120 +Proof generation fails since the epoch is not a multiple of user_epoch_limit.
        • +
      • A user with user_epoch_limit of 120 passes in an epoch of 240 and -can generate user_message_limit proofs without being slashed.

      Since we perform operations on the epoch, we must include it as a circuit input (previously, it was removed from the circuit inputs to RLN-v2).

      Therefore, the new circuit inputs are as follows:

      // unchanged
      private identity_secret
      private user_message_limit
      private message_id
      private pathElements[]
      private pathIndices[]
      public x // messageHash

      // new/changed
      private user_epoch_limit
      private user_epoch_quotient // epoch/user_epoch_limit to assert within circuit
      public epoch
      public rln_identifier

      The circuit outputs remain the same.

      Additional circuit constraints​

      1. Since we accept the epoch, user_epoch_quotient, and user_epoch_limit, -we must ensure that the relation between these 3 values is preserved. I.e.:

        epoch==user_epoch_limit∗user_epoch_quotientepoch == user\_epoch\_limit * user\_epoch\_quotient

      2. To ensure no overflows/underflows occur in the above multiplication, +can generate user_message_limit proofs without being slashed.

        3. +
    +

    Since we perform operations on the epoch, we must include it as a circuit input (previously, it was removed from the circuit inputs to RLN-v2).

    +

    Therefore, the new circuit inputs are as follows:

    +
    // unchanged
    private identity_secret
    private user_message_limit
    private message_id
    private pathElements[]
    private pathIndices[]
    public x // messageHash

    // new/changed
    private user_epoch_limit
    private user_epoch_quotient // epoch/user_epoch_limit to assert within circuit
    public epoch
    public rln_identifier
    +

    The circuit outputs remain the same.

    +

    Additional circuit constraints​

    +
      +
    1. +

      Since we accept the epoch, user_epoch_quotient, and user_epoch_limit, +we must ensure that the relation between these 3 values is preserved. I.e.:

      +epoch==user_epoch_limit∗user_epoch_quotientepoch == user\_epoch\_limit * user\_epoch\_quotient +
      2. +
    2. +

      To ensure no overflows/underflows occur in the above multiplication, we must constrain the inputs of epoch, user_epoch_quotient, and user_epoch_limit. -We have assumed 3600 to be the maximum valid size of the user_epoch_quotient.

    size(epoch)≤64 bitssize(user_epoch_limit)≤12 bitsuser_epoch_limit≤3600user_epoch_limit≤epochuser_epoch_quotient<user_epoch_limitsize(epoch) \leq 64\ bits \\ size(user\_epoch\_limit) \leq 12\ bits \\ user\_epoch\_limit \leq 3600 \\ user\_epoch\_limit \leq epoch \\ user\_epoch\_quotient < user\_epoch\_limit

    Modifications to external epoch validation (Waku, etc.)​

    For receivers of an RLN-v3 proof +We have assumed 3600 to be the maximum valid size of the user_epoch_quotient.

    + + +size(epoch)≤64 bitssize(user_epoch_limit)≤12 bitsuser_epoch_limit≤3600user_epoch_limit≤epochuser_epoch_quotient<user_epoch_limitsize(epoch) \leq 64\ bits \\ +size(user\_epoch\_limit) \leq 12\ bits \\ +user\_epoch\_limit \leq 3600 \\ +user\_epoch\_limit \leq epoch \\ +user\_epoch\_quotient < user\_epoch\_limit +

    Modifications to external epoch validation (Waku, etc.)​

    +

    For receivers of an RLN-v3 proof to detect if a message is too old, we must use the higher bound of the user_epoch_limit, which has been set to 3600. -The trade-off here is that we allow hour-old messages to propagate within the network.

    Modifications to double signaling detection scheme (Waku, etc.)​

    For verifiers of RLN-v1/v2 proofs, +The trade-off here is that we allow hour-old messages to propagate within the network.

    +

    Modifications to double signaling detection scheme (Waku, etc.)​

    +

    For verifiers of RLN-v1/v2 proofs, a log of nullifiers seen in the last epoch is maintained, and if there is a match with a pre-existing nullifier, -double signaling has been detected and the verifier MAY proceed to slash the spamming user.

    With the RLN-v3 scheme, +double signaling has been detected and the verifier MAY proceed to slash the spamming user.

    +

    With the RLN-v3 scheme, we need to increase the size of the nullifier log used, which previously cleared itself every second to the higher bound of the user_epoch_limit, which is 3600. -Now, the RLN proof verifier must clear the nullifier log every 3600 seconds to satisfactorily detect double signaling.

    The implementation​

    An implementation of the RLN-v3 scheme in gnark can be found here.

    Comments on performance​

    • Hardware: Macbook Air M2, 16GB RAM
    • Circuit: RLN-v3
    • Proving system: Groth16
    • Framework: gnark
    • Elliptic curve: bn254 (aka bn128) (not to be confused with the 254-bit Weierstrass curve)
    • Finite field: Prime-order subgroup of the group of points on the bn254 curve
    • Default Merkle tree height: 20
    • Hashing algorithm: Poseidon
    • Merkle tree: Sparse Indexed Merkle Tree

    Proving​

    The proving time for the RLN-v3 circuit is 90ms for a single proof.

    Verification​

    The verification time for the RLN-v3 circuit is 1.7ms for a single proof.

    Conclusion​

    The RLN-v3 scheme introduces a new epoch-based message rate-limiting scheme to the RLN protocol. -It enhances the user's flexibility in setting their message limits and cost-optimizes their stake.

    Future work​

    • Implementing the RLN-v3 scheme in Zerokit
    • Implementing the RLN-v3 scheme in Waku
    • Formal security analysis of the RLN-v3 scheme

    References​

    - - +Now, the RLN proof verifier must clear the nullifier log every 3600 seconds to satisfactorily detect double signaling.

    +

    The implementation​

    +

    An implementation of the RLN-v3 scheme in gnark can be found here.

    +

    Comments on performance​

    +
      +
    • Hardware: Macbook Air M2, 16GB RAM
      • +
    • Circuit: RLN-v3
      • +
    • Proving system: Groth16
      • +
    • Framework: gnark
      • +
    • Elliptic curve: bn254 (aka bn128) (not to be confused with the 254-bit Weierstrass curve)
      • +
    • Finite field: Prime-order subgroup of the group of points on the bn254 curve
      • +
    • Default Merkle tree height: 20
      • +
    • Hashing algorithm: Poseidon
      • +
    • Merkle tree: Sparse Indexed Merkle Tree
      • +
    +

    Proving​

    +

    The proving time for the RLN-v3 circuit is 90ms for a single proof.

    +

    Verification​

    +

    The verification time for the RLN-v3 circuit is 1.7ms for a single proof.

    +

    Conclusion​

    +

    The RLN-v3 scheme introduces a new epoch-based message rate-limiting scheme to the RLN protocol. +It enhances the user's flexibility in setting their message limits and cost-optimizes their stake.

    +

    Future work​

    +
      +
    • Implementing the RLN-v3 scheme in Zerokit
      • +
    • Implementing the RLN-v3 scheme in Waku
      • +
    • Formal security analysis of the RLN-v3 scheme
      • +
    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/rss.xml b/rlog/rss.xml index 27db0f1e..5b4ddde2 100644 --- a/rlog/rss.xml +++ b/rlog/rss.xml @@ -16,20 +16,25 @@ This article introduces MDSECheck method — a novel approach to checking square MDS matrices for unconditional security -as the components of affine permutation layers of P-SP-networks.

    Introduction​

    Maximum distance separable (MDS) matrices play a significant role +as the components of affine permutation layers of P-SP-networks.

    + +

    Introduction​

    +

    Maximum distance separable (MDS) matrices play a significant role in algebraic coding theory and symmetric cryptography. In particular, square MDS matrices are commonly used in affine permutation layers of partial substitution-permutation networks (P-SPNs). These are widespread designs of the modern symmetric ciphers and hash functions. -A classic example of the latter is Poseidon [1], -a well-known hash function used in zk-SNARK proving systems.

    Square MDS matrices differ in terms of security +A classic example of the latter is Poseidon [1], +a well-known hash function used in zk-SNARK proving systems.

    +

    Square MDS matrices differ in terms of security that they are able to provide for P-SPNs. The use of some such matrices in certain P-SPNs may result in existence of infinitely long subspace trails of small period for the latter, -which make them vulnerable to differential cryptanalysis [2].

    Two methods for security checking of square MDS matrices for P-SPNs -have been proposed in [2]. +which make them vulnerable to differential cryptanalysis [2].

    +

    Two methods for security checking of square MDS matrices for P-SPNs +have been proposed in [2]. The first one, which is referred to as the three tests method in the rest of the article, is aimed at security checking for a specified structure of the substitution layer of a P-SPN. @@ -38,145 +43,180 @@ has been designed to determine whether a square MDS matrix satisfies a sufficient condition of being secure regardless of the structure of a P-SPN substitution layer, i.e. to check whether the matrix belongs to the class of square MDS matrices, which are referred to -as unconditionally secure in the current article.

    This article aims to introduce MDSECheck method — +as unconditionally secure in the current article.

    +

    This article aims to introduce MDSECheck method — a novel approach to checking square MDS matrices for unconditional security, which has already been implemented in the Rust programming language as -the library crate [3]. +the library crate [3]. The next sections explain the notions mentioned above, describe the MDSECheck method as well as its mathematical foundations, provide a brief overview of the MDSECheck library crate -and outline possible future research directions.

    MDS matrix: how to define and construct​

    An mm x nn matrix MM over a finite field is called MDS, -if and only if for distinct nn-dimensional column vectors v1v_1 and v2v_2 -the column vectors v1 ∣ Mv1v_1 \: | \: M v_1 and v2 ∣ Mv2v_2 \: | \: M v_2, -where ∣| stands for vertical concatenation, -do not coincide in nn or more components. -The set of all possible column vectors v ∣ Mvv \: | \: M v for -some fixed matrix MM is a systematic MDS code, i.e. +and outline possible future research directions.

    +

    MDS matrix: how to define and construct​

    +

    An mm x nn matrix MM over a finite field is called MDS, +if and only if for distinct nn-dimensional column vectors v1v_1 and v2v_2 +the column vectors v1 ∣ Mv1v_1 \: | \: M v_1 and v2 ∣ Mv2v_2 \: | \: M v_2, +where ∣| stands for vertical concatenation, +do not coincide in nn or more components. +The set of all possible column vectors v ∣ Mvv \: | \: M v for +some fixed matrix MM is a systematic MDS code, i.e. a linear code, which contains input symbols on their original positions and achieves the Singleton bound. -The latter property results in good error-correction capability.

    There are several equivalent definitions of MDS matrices, +The latter property results in good error-correction capability.

    +

    There are several equivalent definitions of MDS matrices, but the next one is especially useful for constructing them directly by means of algebraic methods. A matrix over a finite field is called MDS, if and only if all its square submatrices are nonsingular. -The matrix entries and the matrix itself are also considered submatrices.

    One of the most efficient and straightforward methods to directly construct -an MDS matrix is generating a Cauchy matrix [4]. -Such an mm x nn matrix is defined using -mm-dimensional vector xx and nn-dimensional vector yy, -for which all entries in the concatenation of xx and yy are distinct. +The matrix entries and the matrix itself are also considered submatrices.

    +

    One of the most efficient and straightforward methods to directly construct +an MDS matrix is generating a Cauchy matrix [4]. +Such an mm x nn matrix is defined using +mm-dimensional vector xx and nn-dimensional vector yy, +for which all entries in the concatenation of xx and yy are distinct. The entries of the Cauchy matrix are described by the formula -Mi,j=1 / (xi−yj)M_{i, j} = 1 \: / \: (x_i - y_j). +Mi,j=1 / (xi−yj)M_{i, j} = 1 \: / \: (x_i - y_j). It is obvious that any submatrix of a Cauchy matrix is also a Cauchy matrix. -The Cauchy determinant formula [5] implies that +The Cauchy determinant formula [5] implies that every square Cauchy matrix is nonsingular. -Thus, Cauchy matrices satisfy the second definition of MDS matrices.

    Partial substitution-permutation networks​

    Describing SPNs in algebraic terms is convenient, +Thus, Cauchy matrices satisfy the second definition of MDS matrices.

    +

    Partial substitution-permutation networks​

    +

    Describing SPNs in algebraic terms is convenient, so this approach has been chosen for this article. SPNs are designs of the symmetric cryptoprimitives, which operate on an internal state, which is represented -as an nn-dimensional vector over some finite field, +as an nn-dimensional vector over some finite field, and update this state iteratively by means of -the round transformations described below.

    Each round begins with an optional update of the internal state by +the round transformations described below.

    +

    Each round begins with an optional update of the internal state by adding to its components some input data or extraction of some of these components as the output data. This optional step depends on the specific cryptoprimitive and the current round number. The next step is called the nonlinear substitution layer -and lies in replacing the ii-th component of the internal state -with Si(c)S_i(c) for each i∈[1..n]i \in [1..n], where cc is the component value -and Si(x)S_i(x) is a nonlinear invertible function over the finite field. -The function Si(x)S_i(x) is specific to the cryptoprimitive and called an S-Box. +and lies in replacing the ii-th component of the internal state +with Si(c)S_i(c) for each i∈[1..n]i \in [1..n], where cc is the component value +and Si(x)S_i(x) is a nonlinear invertible function over the finite field. +The function Si(x)S_i(x) is specific to the cryptoprimitive and called an S-Box. The final step, which is known as the affine permutation layer, -replaces the internal state with MX+cM X + c, -where XX is the current internal state, -MM is a nonsingular square matrix and -cc is the vector of the round constants. -The value of cc is specific to the cryptoprimitive +replaces the internal state with MX+cM X + c, +where XX is the current internal state, +MM is a nonsingular square matrix and +cc is the vector of the round constants. +The value of cc is specific to the cryptoprimitive and the current round number, -while MM typically depends only on the cryptoprimitive. -The data flow diagram for an SPN is given below.

    ..................................                         
       │        │        │        │                           
       ▼        ▼        ▼        ▼                           
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ..................................

    Partial SPNs are modifications of SPNs, +while MM typically depends only on the cryptoprimitive. +The data flow diagram for an SPN is given below.

    +
    ..................................                         
       │        │        │        │                           
       ▼        ▼        ▼        ▼                           
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │ Optional addition / extraction │ <─────> Input / output
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌─────┐  ┌─────┐  ┌─────┐  ┌─────┐                         
    │S₁(x)│  │S₂(x)│  │ ... │  │Sₙ(x)│                         
    └──┬──┘  └──┬──┘  └──┬──┘  └──┬──┘                         
       ▼        ▼        ▼        ▼                            
    ┌────────────────────────────────┐                         
    │       Affine permutation       │                         
    └──┬────────┬────────┬────────┬──┘                         
       ▼        ▼        ▼        ▼                            
    ..................................
    +

    Partial SPNs are modifications of SPNs, where for certain rounds some S-Boxes are replaced with -the identity functions to reduce computational efforts [2]. +the identity functions to reduce computational efforts [2]. For example, the nonlinear substitution layers of the partial rounds of -Poseidon update only the first internal state component [1]. -In the case of P-SPNs, security considerations commonly demand to choose MM +Poseidon update only the first internal state component [1]. +In the case of P-SPNs, security considerations commonly demand to choose MM as a square MDS matrix, because these matrices provide -perfect diffusion property for the affine permutation layer [6]. +perfect diffusion property for the affine permutation layer [6]. Possessing this property means ensuring that -any two nn-dimensional internal states, -which differ in exactly tt components, +any two nn-dimensional internal states, +which differ in exactly tt components, are mapped by the affine permutation layer to -two new internal states that differ in at least n−t+1n - t + 1 components.

    Square MDS matrix security check in the context of P-SPNs​

    Certain square MDS matrices should not be used in certain P-SPNs +two new internal states that differ in at least n−t+1n - t + 1 components.

    +

    Square MDS matrix security check in the context of P-SPNs​

    +

    Certain square MDS matrices should not be used in certain P-SPNs to avoid making them vulnerable to differential cryptanalysis, since it may exploit the existence of infinitely long subspace trails -of small period for vulnerable P-SPNs. [2]. -Such matrices are called insecure with respect to particular P-SPNs.

    An infinitely long subspace trail of period ll exists for a P-SPN, +of small period for vulnerable P-SPNs. [2]. +Such matrices are called insecure with respect to particular P-SPNs.

    +

    An infinitely long subspace trail of period ll exists for a P-SPN, if and only if there is a proper subspace of differences of internal state vectors, such that if for a pair of initial internal states the difference belongs to this subspace, then the difference for the new internal states, which are obtained from the initial ones -by means of the same ll-round transformation, -also belongs to this subspace [2].

    Two methods for checking square MDS matrices for suitability for P-SPNs +by means of the same ll-round transformation, +also belongs to this subspace [2].

    +

    Two methods for checking square MDS matrices for suitability for P-SPNs in terms of existence of infinitely long subspace trails -have been proposed in [2]. +have been proposed in [2]. The three tests method is aimed at checking whether using a specified matrix for a P-SPN with a specified structure of the substitution layer -leads to existence of infinitely long subspace trails of period pp -for this P-SPN for all pp no larger than a given ll. +leads to existence of infinitely long subspace trails of period pp +for this P-SPN for all pp no larger than a given ll. The sufficient test method has been designed to determine whether a square MDS matrix satisfies a sufficient condition -of non-existence of infinitely long subspace trails of period pp -for P-SPNs using this matrix for all pp no larger than a specified ll.

    The sufficient test method is a direct consequence of -Theorem 8 in [2] and consists in checking that -the minimal polynomial of the pp-th power of the tested matrix -has maximum degree and is irreducible for all p∈[1..l]p \in [1..l]. +of non-existence of infinitely long subspace trails of period pp +for P-SPNs using this matrix for all pp no larger than a specified ll.

    +

    The sufficient test method is a direct consequence of +Theorem 8 in [2] and consists in checking that +the minimal polynomial of the pp-th power of the tested matrix +has maximum degree and is irreducible for all p∈[1..l]p \in [1..l]. The aforesaid sufficient non-existence condition is satisfied by the matrix, -if and only if all the checks yield positive results.

    It is convenient to define +if and only if all the checks yield positive results.

    +

    It is convenient to define the unconditional P-SPN security level of the square MDS matrix as follows: -this level is ll for the matrix MM, -if and only if the minimal polynomials of MM, M2M^2, ......, MlM^l +this level is ll for the matrix MM, +if and only if the minimal polynomials of MM, M2M^2, ......, MlM^l have maximum degree and are irreducible, -but for Ml + 1M^{l \: + \: 1} the minimal polynomial does not have this property. +but for Ml + 1M^{l \: + \: 1} the minimal polynomial does not have this property. Using this definition, the purpose of the sufficient test method can be described as checking whether the unconditional P-SPN security level of the specified matrix -is no less than a given bound.

    MDSECheck method: getting rid of the matrix powers​

    The MDSECheck method, whose name is derived from +is no less than a given bound.

    +

    MDSECheck method: getting rid of the matrix powers​

    +

    The MDSECheck method, whose name is derived from the words "MDS", "security", "elaborated" and "check", has the same purpose as the sufficient test method, but achieves it differently. The differences of the first method from the latter -and approaches to implementing them can be described as follows:

    1. Computation and verification of minimal polynomials -of M2M^2, M3M^3, ......, MlM^l, where MM is -the tested nn x nn matrix over GF(q)GF(q) -and ll is the security level bound, +and approaches to implementing them can be described as follows:

      +
        +
      1. +

        Computation and verification of minimal polynomials +of M2M^2, M3M^3, ......, MlM^l, where MM is +the tested nn x nn matrix over GF(q)GF(q) +and ll is the security level bound, has been replaced with checks for the corresponding powers -of a root of the characteristic polynomial of MM -for non-presence in nontrivial subfields of GF(qn)GF(q^n).

        1. The non-presence check is performed without -straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n). +of a root of the characteristic polynomial of MM +for non-presence in nontrivial subfields of GF(qn)GF(q^n).

          +
            +
          1. +

            The non-presence check is performed without +straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n). The root is checked only for non-presence in the subfields -GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, -GF(qn / pd)GF(q^{n \: / \: p_d}), where p1p_1, p2p_2, ......, pdp_d -are all prime divisors of nn.

          2. The non-presence check reuses some data computed during -the checking for irreducibility the minimal polynomial of MM, -which in this case coincides with f(y)f(y) -designating the characteristic polynomial of MM. -The values of yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y) are saved -for each j∈[1..d]j \in [1..d] during the irreducibility check +GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, +GF(qn / pd)GF(q^{n \: / \: p_d}), where p1p_1, p2p_2, ......, pdp_d +are all prime divisors of nn.

            +
            3. +
          3. +

            The non-presence check reuses some data computed during +the checking for irreducibility the minimal polynomial of MM, +which in this case coincides with f(y)f(y) +designating the characteristic polynomial of MM. +The values of yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y) are saved +for each j∈[1..d]j \in [1..d] during the irreducibility check to replace exponentiations with sequential computations -of (yi)qn / pjmod  f(y)(y^i)^{q^{n \: / \: p_j}} \mod f(y) from -(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) -as its product with yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y).

        2. The check of the minimal polynomial of MM +of (yi)qn / pjmod  f(y)(y^i)^{q^{n \: / \: p_j}} \mod f(y) from +(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) +as its product with yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y).

          +
          3. +
        +
        2. +
      2. +

        The check of the minimal polynomial of MM for irreducibility and maximum degree is performed without unconditional computation of this polynomial. This computation has been replaced with the Krylov method fragment, which consists in building and solving -only one system of linear equations over GF(q)GF(q). -If MM has an irreducible minimal polynomial of maximum degree, +only one system of linear equations over GF(q)GF(q). +If MM has an irreducible minimal polynomial of maximum degree, then its coefficients are trivially determined from the system solution. If the system is degenerate, -then the minimal polynomial of MM does not have such properties.

      The correctness of the first distinctive feature can be proven as follows. +then the minimal polynomial of MM does not have such properties.

      +
      2. +
    +

    The correctness of the first distinctive feature can be proven as follows. Verifying that the minimal polynomial of a matrix is of maximum degree and irreducible is equivalent to verifying that @@ -184,145 +224,195 @@ the characteristic polynomial of this matrix is irreducible, because the minimal polynomial divides the characteristic one. Also, it is trivially proven that for a matrix with such a minimal polynomial it is equal to the characteristic polynomial. -Thus, the required checks for the matrices M2M^2, M3M^3, ......, MlM^l -can be done by checking their characteristic polynomials for irreducibility.

    Let MM be nn x nn matrix over GF(q)GF(q), +Thus, the required checks for the matrices M2M^2, M3M^3, ......, MlM^l +can be done by checking their characteristic polynomials for irreducibility.

    +

    Let MM be nn x nn matrix over GF(q)GF(q), whose minimal polynomial is of maximum degree and irreducible. -The statements in the previous paragraph imply that f(y)f(y), -which is the nn-degree characteristic polynomial of MM, is irreducible. -Consider MM over the extension field GF(qn)GF(q^n), -which is the splitting field of f(y)f(y). -Let α∈GF(qn)α \in GF(q^n) be a root of f(y)f(y). +The statements in the previous paragraph imply that f(y)f(y), +which is the nn-degree characteristic polynomial of MM, is irreducible. +Consider MM over the extension field GF(qn)GF(q^n), +which is the splitting field of f(y)f(y). +Let α∈GF(qn)α \in GF(q^n) be a root of f(y)f(y). According to standard results from the Galois field theory, -αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}} -are distinct roots of f(y)f(y) [7]. -Thus, these powers of αα are nn distinct eigenvalues of MM. -Hence, due to matrix similarity properties, there is some matrix SS -such that SMS−1=DS M S^{-1} = D, where DD is the diagonal matrix, +αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}} +are distinct roots of f(y)f(y) [7]. +Thus, these powers of αα are nn distinct eigenvalues of MM. +Hence, due to matrix similarity properties, there is some matrix SS +such that SMS−1=DS M S^{-1} = D, where DD is the diagonal matrix, whose nonzero elements are -αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}}. -Therefore, SMiS−1=DiS M^i S^{-1} = D^i, -so the roots of the characteristic polynomial of MiM^i -are αiα^i, (αq)i(α^q)^i, (αq2)i(α^{q^2})^i, ......, (αqn − 1)i(α^{q^{n \: - \: 1}})^i. -If the minimal polynomial of αiα^i has degree less than nn, -then the characteristic polynomial of MiM^i is divisible +αα, αqα^q, αq2α^{q^2}, ......, αqn − 1α^{q^{n \: - \: 1}}. +Therefore, SMiS−1=DiS M^i S^{-1} = D^i, +so the roots of the characteristic polynomial of MiM^i +are αiα^i, (αq)i(α^q)^i, (αq2)i(α^{q^2})^i, ......, (αqn − 1)i(α^{q^{n \: - \: 1}})^i. +If the minimal polynomial of αiα^i has degree less than nn, +then the characteristic polynomial of MiM^i is divisible by this minimal polynomial, -while αiα^i lies in some nontrivial subfield of GF(qn)GF(q^n). +while αiα^i lies in some nontrivial subfield of GF(qn)GF(q^n). One of the fields isomorphic to this subfield is a residue class ring -of polynomials modulo the minimal polynomial of αiα^i [7]. -If the minimal polynomial of αiα^i is of degree nn, -then the characteristic polynomial of MiM^i +of polynomials modulo the minimal polynomial of αiα^i [7]. +If the minimal polynomial of αiα^i is of degree nn, +then the characteristic polynomial of MiM^i equals this minimal polynomial and therefore is irreducible, -while αiα^i does not lie in any nontrivial subfield of GF(qn)GF(q^n). -In this case, 11, αiα^i, (αi)2(α^i)^2, ......, (αi)n − 1(α^i)^{n \: - \: 1} +while αiα^i does not lie in any nontrivial subfield of GF(qn)GF(q^n). +In this case, 11, αiα^i, (αi)2(α^i)^2, ......, (αi)n − 1(α^i)^{n \: - \: 1} are linearly independent as distinct roots of -an irreducible polynomial over a finite field [7], -so any field containing αiα^i has at least qnq^n elements -and therefore cannot be a trivial subfield of GF(qn)GF(q^n). +an irreducible polynomial over a finite field [7], +so any field containing αiα^i has at least qnq^n elements +and therefore cannot be a trivial subfield of GF(qn)GF(q^n). Thus, checking the characteristic polynomials of -the matrices M2M^2, M3M^3, ......, MlM^l for irreducibility -is equivalent to verifying that α2α^2, α3α^3, ......, αlα^l -do not lie in any nontrivial subfield of GF(qn)GF(q^n).

    The last sentences of the two previous paragraphs imply the following: -verifying that the minimal polynomials of M2M^2, M3M^3, ......, MlM^l +the matrices M2M^2, M3M^3, ......, MlM^l for irreducibility +is equivalent to verifying that α2α^2, α3α^3, ......, αlα^l +do not lie in any nontrivial subfield of GF(qn)GF(q^n).

    +

    The last sentences of the two previous paragraphs imply the following: +verifying that the minimal polynomials of M2M^2, M3M^3, ......, MlM^l are of maximum degree and irreducible can be performed by verifying that the corresponding powers of a root of -the characteristic polynomial of the nn x nn matrix MM over GF(q)GF(q) -do not belong to any nontrivial subfield of GF(qn)GF(q^n). ■\blacksquare

    The approaches to implementing the first distinctive feature +the characteristic polynomial of the nn x nn matrix MM over GF(q)GF(q) +do not belong to any nontrivial subfield of GF(qn)GF(q^n). ■\blacksquare

    +

    The approaches to implementing the first distinctive feature can be explained and proven to be correct as follows. -Since GF(qw)GF(q^w) is a nontrivial subfield of GF(qu)GF(q^u) -if and only if ww divides uu and w<uw < u [7], -the presence of some εε in GF(qh)GF(q^h), -which is a nontrivial subfield of GF(qn)GF(q^n), -implies that ε∈GF(qn / ν)ε \in GF(q^{n \: / \: ν}) for some prime νν dividing nn, -because hh divides the quotient of nn and some of its prime factors. +Since GF(qw)GF(q^w) is a nontrivial subfield of GF(qu)GF(q^u) +if and only if ww divides uu and w<uw < u [7], +the presence of some εε in GF(qh)GF(q^h), +which is a nontrivial subfield of GF(qn)GF(q^n), +implies that ε∈GF(qn / ν)ε \in GF(q^{n \: / \: ν}) for some prime νν dividing nn, +because hh divides the quotient of nn and some of its prime factors. Thus, checking that some value does not belong to subfields -GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, -GF(qn / pd)GF(q^{n \: / \: p_d}), -where p1p_1, p2p_2, ......, pdp_d are all prime divisors of nn, +GF(qn / p1)GF(q^{n \: / \: p_1}), GF(qn / p2)GF(q^{n \: / \: p_2}), ......, +GF(qn / pd)GF(q^{n \: / \: p_d}), +where p1p_1, p2p_2, ......, pdp_d are all prime divisors of nn, is equivalent to checking this value for -non-presence in nontrivial subfields of GF(qn)GF(q^n).

    Checking for irreducibility the minimal polynomial of MM -is performed by means of Algorithm 2.2.9 in [8] +non-presence in nontrivial subfields of GF(qn)GF(q^n).

    +

    Checking for irreducibility the minimal polynomial of MM +is performed by means of Algorithm 2.2.9 in [8] and consists in sequential computation of -ypmod  f(y)y^p \mod f(y), yp2mod  f(y)y^{p^2} \mod f(y), ......, -yp⌊n / 2⌋mod  f(y)y^{p^{\lfloor n \: / \: 2 \rfloor}} \mod f(y) -and checking that GCD(ypimod  f(y) − y,f(y))=1GCD(y^{p^i} \mod f(y) \: - \: y, f(y)) = 1 -for each i∈[1..⌊n / 2⌋]i \in [1..\lfloor n \: / \: 2 \rfloor], -where f(y)f(y) is the characteristic polynomial of MM +ypmod  f(y)y^p \mod f(y), yp2mod  f(y)y^{p^2} \mod f(y), ......, +yp⌊n / 2⌋mod  f(y)y^{p^{\lfloor n \: / \: 2 \rfloor}} \mod f(y) +and checking that GCD(ypimod  f(y) − y,f(y))=1GCD(y^{p^i} \mod f(y) \: - \: y, f(y)) = 1 +for each i∈[1..⌊n / 2⌋]i \in [1..\lfloor n \: / \: 2 \rfloor], +where f(y)f(y) is the characteristic polynomial of MM and coincides with the minimal polynomial in this case. The optimized root non-presence check is performed -by checking that for each i∈[2..l]i \in [2..l] for each j∈[1..d]j \in [1..d] -the value of ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) is nonzero. +by checking that for each i∈[2..l]i \in [2..l] for each j∈[1..d]j \in [1..d] +the value of ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) is nonzero. This approach is based on the following standard results -from the Galois field theory [7]:

    • GF(qn)GF(q^n) is isomorphic to the residue class ring of -univariate polynomials in yy modulo f(y)f(y), -because at this point f(y)f(y) is known to be irreducible, -and some root of f(y)f(y) is mapped by this isomorphism to -the residue class the polynomial yy in this ring.

    • All elements of a finite field GF(qw)GF(q^w) and only they -are roots of yqw−yy^{q^w} - y.

    The expression ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) -can be rewritten as ((yqn / pj)i−yi)mod  f(y)((y^{q^{n \: / \: p_j}})^i - y^i) \mod f(y), +from the Galois field theory [7]:

    +
      +
    • +

      GF(qn)GF(q^n) is isomorphic to the residue class ring of +univariate polynomials in yy modulo f(y)f(y), +because at this point f(y)f(y) is known to be irreducible, +and some root of f(y)f(y) is mapped by this isomorphism to +the residue class the polynomial yy in this ring.

      +
      • +
    • +

      All elements of a finite field GF(qw)GF(q^w) and only they +are roots of yqw−yy^{q^w} - y.

      +
      • +
    +

    The expression ((yi)qn / pj−yi)mod  f(y)((y^i)^{q^{n \: / \: p_j}} - y^i) \mod f(y) +can be rewritten as ((yqn / pj)i−yi)mod  f(y)((y^{q^{n \: / \: p_j}})^i - y^i) \mod f(y), which can be computed without exponentiation as the product of -(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) and -yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y), -which has been saved during the irreducibility check.

    The second distinctive feature can be +(y(i − 1))qn / pjmod  f(y)(y^{(i \: - \: 1)})^{q^{n \: / \: p_j}} \mod f(y) and +yqn / pjmod  f(y)y^{q^{n \: / \: p_j}} \mod f(y), +which has been saved during the irreducibility check.

    +

    The second distinctive feature can be explained and proven to be correct in following way. -The nn x nn matrix MM does not have a minimal polynomial of maximum degree, -if some Krylov subspace of order nn for it is not nn-dimensional. +The nn x nn matrix MM does not have a minimal polynomial of maximum degree, +if some Krylov subspace of order nn for it is not nn-dimensional. Indeed, the minimal polynomial of the matrix is divisible by the minimal polynomial of the restriction of this linear operator to an arbitrary subspace, -and in the considered case the latter polynomial has degree less than nn, +and in the considered case the latter polynomial has degree less than nn, because the degree of the minimal polynomial of a linear operator cannot exceed the dimension of the subspace the operator acts on. -Thus, an unconditional computation of the minimal polynomial of MM +Thus, an unconditional computation of the minimal polynomial of MM is not required to determine whether this polynomial is irreducible and has maximum degree. Using this computation has been replaced with the Krylov method fragment, -which consists in choosing any nonzero nn-dimensional column vector vv -and solving the system of linear equations AX=bA X = b, -where AA is an nn x nn matrix, -whose columns are vv, MvM v, M2vM^2 v, ......, Mn − 1vM^{n \: - \: 1} v, -and bb is MnvM^n v. -If AA is singular, -the minimal polynomial of MM is reducible or does not have maximum degree, -so checking MM has been accomplished; -otherwise, f(y)f(y), which is the minimal and characteristic polynomial of MM, -can be expressed as yn−Xn − 1yn − 1−Xn − 2yn − 2−…−X1y−X0y^n - X_{n \: - \: 1} y^{n \: - \: 1} - X_{n \: - \: 2} y^{n \: - \: 2} - … - X_1 y - X_0.

    The steps of MDSECheck method can be summarized as follows:

    1. The square MDS matrix MM over GF(q)GF(q) -and the unconditional P-SPN security level bound ll are received as inputs.

    2. The Krylov method fragment is used to compute the minimal polynomial of MM. -If the computation fails, then MM is not unconditionally secure, -so the check of MM is complete. +which consists in choosing any nonzero nn-dimensional column vector vv +and solving the system of linear equations AX=bA X = b, +where AA is an nn x nn matrix, +whose columns are vv, MvM v, M2vM^2 v, ......, Mn − 1vM^{n \: - \: 1} v, +and bb is MnvM^n v. +If AA is singular, +the minimal polynomial of MM is reducible or does not have maximum degree, +so checking MM has been accomplished; +otherwise, f(y)f(y), which is the minimal and characteristic polynomial of MM, +can be expressed as yn−Xn − 1yn − 1−Xn − 2yn − 2−…−X1y−X0y^n - X_{n \: - \: 1} y^{n \: - \: 1} - +X_{n \: - \: 2} y^{n \: - \: 2} - … - X_1 y - X_0.

      +

      The steps of MDSECheck method can be summarized as follows:

      +
        +
      1. +

        The square MDS matrix MM over GF(q)GF(q) +and the unconditional P-SPN security level bound ll are received as inputs.

        +
        2. +
      2. +

        The Krylov method fragment is used to compute the minimal polynomial of MM. +If the computation fails, then MM is not unconditionally secure, +so the check of MM is complete. If it succeeds, then the minimal polynomial has maximum degree -and, therefore, coincides with the characteristic polynomial of MM.

      3. Algorithm 2.2.9 is used -to check for irreducibility the minimal polynomial of MM, -which is also the characteristic polynomial of MM in this case. +and, therefore, coincides with the characteristic polynomial of MM.

        +
        4. +
      4. +

        Algorithm 2.2.9 is used +to check for irreducibility the minimal polynomial of MM, +which is also the characteristic polynomial of MM in this case. Some data computed during this step is saved to be reused at the next one. -If the polynomial is reducible, then the check of MM is complete, -because MM has been found to be not unconditionally secure.

      5. The values of α2α^2, α3α^3, ......, αlα^l, -where αα is a root of the characteristic polynomial of MM, +If the polynomial is reducible, then the check of MM is complete, +because MM has been found to be not unconditionally secure.

        +
        6. +
      6. +

        The values of α2α^2, α3α^3, ......, αlα^l, +where αα is a root of the characteristic polynomial of MM, are sequentially checked for non-presence in -nontrivial subfields of GF(qn)GF(q^n) as described above. -If αiα^i belongs to some nontrivial subfield of GF(qn)GF(q^n), -then the unconditional P-SPN security level of MM is i − 1i \: - \: 1, -so the check of MM is complete. +nontrivial subfields of GF(qn)GF(q^n) as described above. +If αiα^i belongs to some nontrivial subfield of GF(qn)GF(q^n), +then the unconditional P-SPN security level of MM is i − 1i \: - \: 1, +so the check of MM is complete. If all the values do not belong to such a subfield, -then the unconditional P-SPN security level is at least ll.

      MDSECheck library crate: implementation in Rust​

      The library crate [3] provides tools for +then the unconditional P-SPN security level is at least ll.

      +
      3. +
    +

    MDSECheck library crate: implementation in Rust​

    +

    The library crate [3] provides tools for generating random square Cauchy MDS matrices over prime finite fields and applying the MDSECheck method to check such matrices for unconditional security. The used data types of field elements and polynomials are provided by -the crates ark-ff [9] and ark-poly [10]. -The auxiliary tools in the crate modules are accessible as well.

    Generating by means of this crate a 10 x 10 MDS matrix, -which is defined over the BN254 scalar field [11] +the crates ark-ff [9] and ark-poly [10]. +The auxiliary tools in the crate modules are accessible as well.

    +

    Generating by means of this crate a 10 x 10 MDS matrix, +which is defined over the BN254 scalar field [11] and has unconditional P-SPN security level is 1000, takes less than 60 milliseconds on average for the laptop with the processor Intel® Core™ i9-14900HX, -whose maximum clock frequency is 5.8 GHz.

    Conclusion​

    The MDSECheck method proposed in this article is a novel approach +whose maximum clock frequency is 5.8 GHz.

    +

    Conclusion​

    +

    The MDSECheck method proposed in this article is a novel approach to checking square MDS matrices for unconditional security as the components of affine permutation layers of P-SPNs. It has been implemented as a practical library crate for generating unconditionally secure square MDS matrices -for P-SPNs over prime finite fields.

    The future research directions may include +for P-SPNs over prime finite fields.

    +

    The future research directions may include theoretical and experimental studies of performance of approaches, which use the MDSECheck method -to generate unconditionally secure square MDS matrices for P-SPNs.

    References​

    1. L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, M. Schofnegger. "POSEIDON: A New Hash Function for Zero-Knowledge Proof Systems (Updated Version)".
    2. L. Grassi, C. Rechberger, M. Schofnegger. "Proving Resistance Against Infinitely Long Subspace Trails: How to Choose the Linear Layer".
    3. The page "mdsecheck" on crates.io.
    4. Y. Kumar, P. Mishra, S. Samanta, K. Chand Gupta, A. Gaur. "Construction of all MDS and involutory MDS matrices".
    5. The page "Value of Cauchy Determinant" on proofwiki.org.
    6. T. Silva, R. Dahab "MDS Matrices for Cryptography".
    7. S. Huczynska, M. Neunhöffer. "Finite Fields"
    8. R. Crandall, C. Pomerance. "Prime Numbers: A Computational Perspective" (2nd edition).
    9. The page "ark-ff" on crates.io.
    10. The page "ark-poly" on crates.io.
    11. The page "ark-bn254" on crates.io.
    ]]>     +to generate unconditionally secure square MDS matrices for P-SPNs.

    +

    References​

    +
      +
    1. L. Grassi, D. Khovratovich, C. Rechberger, A. Roy, M. Schofnegger. "POSEIDON: A New Hash Function for Zero-Knowledge Proof Systems (Updated Version)".
      2. +
    2. L. Grassi, C. Rechberger, M. Schofnegger. "Proving Resistance Against Infinitely Long Subspace Trails: How to Choose the Linear Layer".
      3. +
    3. The page "mdsecheck" on crates.io.
      4. +
    4. Y. Kumar, P. Mishra, S. Samanta, K. Chand Gupta, A. Gaur. "Construction of all MDS and involutory MDS matrices".
      5. +
    5. The page "Value of Cauchy Determinant" on proofwiki.org.
      6. +
    6. T. Silva, R. Dahab "MDS Matrices for Cryptography".
      7. +
    7. S. Huczynska, M. Neunhöffer. "Finite Fields"
      8. +
    8. R. Crandall, C. Pomerance. "Prime Numbers: A Computational Perspective" (2nd edition).
      9. +
    9. The page "ark-ff" on crates.io.
      10. +
    10. The page "ark-poly" on crates.io.
      11. +
    11. The page "ark-bn254" on crates.io.
      12. +
    ]]> <![CDATA[Vac 2024 Year in Review]]> @@ -330,45 +420,223 @@ to generate unconditionally secure square MDS matrices for P-SPNs.

    https://vac.dev/rlog/2024-recap Thu, 09 Jan 2025 18:30:00 GMT - In this post, we recap Vac's achievements in 2024 and look forward to 2025.

    With 2024 now behind us and a new year ahead, -Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem.

    Vac comprises various subteams and service units, each with its own focus. -Below, we celebrate each unit's achievements and look forward to its 2025 plans.

    Nescience​

    Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.

    Highlights​

    This year, the Nescience state separation architecture moved from exploration to real progress, + In this post, we recap Vac's achievements in 2024 and look forward to 2025.

    + +

    With 2024 now behind us and a new year ahead, +Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem.

    +

    Vac comprises various subteams and service units, each with its own focus. +Below, we celebrate each unit's achievements and look forward to its 2025 plans.

    +

    Nescience​

    +

    Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.

    +

    Highlights​

    +

    This year, the Nescience state separation architecture moved from exploration to real progress, taking significant steps towards building a functional and reliable system. The team focused on turning ideas into something real, testing the proposed architecture, -and understanding its strengths and weaknesses.

    We also made progress on the essential parts of NSSA’s system, including:

    • Key protocol for secure key management
    • Execution types and circuits for reliable computation
    • UTXO specification to manage state transitions effectively
    • Cryptography module to ensure privacy and security

    Looking forward​

    In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use.

    • Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement.
    • Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness).
    • Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem.
    • Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact.
    • Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.

    Token Economics (TKE)​

    The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.

    Highlights​

    • Formalized and implemented Codex economic incentives in the Litepaper and simulations
    • Orchestrated Status Network incentive structure and smart contract implementation
    • Started building Nomos’s economic model
    • Consulted and provided analysis of incentives for the Logos Operators ordinals project
    • Drove discussions on the economic sustainability of Waku; -helped define RLN membership and its payment mechanism

    Looking forward​

    In 2025, TKE will continue to support IFT portfolio projects, +and understanding its strengths and weaknesses.

    +
      +
    • ZkVM exploration and benchmarks + +
      • +
    • Defined the NSSA architecture + +
      • +
    • Built the sandboxed testnet +
        +
      • Designed the first version of the node specification
        • +
      • All core components (execution types, UTXOs, cryptographic primitives) implemented and being tested
        • +
      • Testing the performance of all execution types in various scenarios
        • +
      +
      • +
    +

    We also made progress on the essential parts of NSSA’s system, including:

    +
      +
    • Key protocol for secure key management
      • +
    • Execution types and circuits for reliable computation
      • +
    • UTXO specification to manage state transitions effectively
      • +
    • Cryptography module to ensure privacy and security
      • +
    +

    Looking forward​

    +

    In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use.

    +
      +
    • Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement.
      • +
    • Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness).
      • +
    • Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem.
      • +
    • Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact.
      • +
    • Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.
      • +
    +

    Token Economics (TKE)​

    +

    The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.

    +

    Highlights​

    +
      +
    • Formalized and implemented Codex economic incentives in the Litepaper and simulations
      • +
    • Orchestrated Status Network incentive structure and smart contract implementation
      • +
    • Started building Nomos’s economic model
      • +
    • Consulted and provided analysis of incentives for the Logos Operators ordinals project
      • +
    • Drove discussions on the economic sustainability of Waku; +helped define RLN membership and its payment mechanism
      • +
    +

    Looking forward​

    +

    In 2025, TKE will continue to support IFT portfolio projects, working toward economic sustainability while strengthening relationships within the organization. -Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.

    Quality Assurance (QA)​

    The QA Service Unit focuses on the development and execution of comprehensive test plans, -including implementing unit and interoperability testing.

    Highlights​

    • Matured Waku interoperability testing framework with coverage for all major protocols and features
    • Began collaboration with Nomos, contributing to unit and integration testing
    • Partnered with the Status team to test message reliability under unstable network conditions

    Looking forward​

    • Extend collaboration with the Waku team on go-waku bindings and message reliability testing
    • Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation
    • Work closely with Status’s QA team to enhance the functional testing framework
    • Continue work on nim-libp2p testing
    • Expand collaboration to additional projects

    RFC​

    The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. -The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.

    Highlights​

    • Working to implement RFC culture across the IFT ecosystem
    • Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex.
    • Reworked our standards with regard to writing RFCs to a consensus-oriented specification system

    Looking forward​

    • Continue to implement RFC culture across the IFT ecosystem
    • Broaden the number of RFCs produced -– particularly for IFT portfolio projects nearing public releases
    • Include new projects with the rfc-index
    • Encourage external projects requiring RFCs to establish relationships with the service unit

    Applied Cryptography and ZK (ACZ)​

    The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, +Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.

    +

    Quality Assurance (QA)​

    +

    The QA Service Unit focuses on the development and execution of comprehensive test plans, +including implementing unit and interoperability testing.

    +

    Highlights​

    +
      +
    • Matured Waku interoperability testing framework with coverage for all major protocols and features
      • +
    • Began collaboration with Nomos, contributing to unit and integration testing
      • +
    • Partnered with the Status team to test message reliability under unstable network conditions
      • +
    +

    Looking forward​

    +
      +
    • Extend collaboration with the Waku team on go-waku bindings and message reliability testing
      • +
    • Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation
      • +
    • Work closely with Status’s QA team to enhance the functional testing framework
      • +
    • Continue work on nim-libp2p testing
      • +
    • Expand collaboration to additional projects
      • +
    +

    RFC​

    +

    The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. +The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.

    +

    Highlights​

    +
      +
    • Working to implement RFC culture across the IFT ecosystem
      • +
    • Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex.
      • +
    • Reworked our standards with regard to writing RFCs to a consensus-oriented specification system
      • +
    +

    Looking forward​

    +
      +
    • Continue to implement RFC culture across the IFT ecosystem
      • +
    • Broaden the number of RFCs produced +– particularly for IFT portfolio projects nearing public releases
      • +
    • Include new projects with the rfc-index
      • +
    • Encourage external projects requiring RFCs to establish relationships with the service unit
      • +
    +

    Applied Cryptography and ZK (ACZ)​

    +

    The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, enhancing the security, privacy, and trustworthiness of IFT portfolio projects -and contributing to the overall integrity and resilience of the decentralized web ecosystem.

    Highlights​

    • Researched a libp2p mix protocol and first proof-of-concept implementation (including ping and GossipSub over mix)
    • Researched a decentralized version of MLS (message layer security) with a first proof of concept
    • Released Zerokit v0.6.0 -and v0.5.0
    • Added gnark RLN implementation
    • Released Stealth Address Kit v0.3.1, +and contributing to the overall integrity and resilience of the decentralized web ecosystem.

      +

      Highlights​

      +

      Looking forward​

      • Ensure libp2p mix protocol is production-ready -and support with the publishing of a paper and blog posts
      • Ensure decentralized MLS is production-ready -and support with the publishing of a paper and blog posts
      • Begin explorations of additional research topics
      • Release Zerokit v0.7 and future versions

      P2P​

      The P2P Service Unit specializes in peer-to-peer technologies -and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.

      Highlights​

      Looking forward​

      • Add new features to nim-libp2p: -QUIC transport, web transport
      • Update specifications for libp2p GossipSub, -aiming to significantly improve its performance

      Distributed Systems Testing (DST)​

      The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. -By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.

      Highlights​

      +

      Looking forward​

      +
        +
      • Ensure libp2p mix protocol is production-ready +and support with the publishing of a paper and blog posts
        • +
      • Ensure decentralized MLS is production-ready +and support with the publishing of a paper and blog posts
        • +
      • Begin explorations of additional research topics
        • +
      • Release Zerokit v0.7 and future versions
        • +
      +

      P2P​

      +

      The P2P Service Unit specializes in peer-to-peer technologies +and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.

      +

      Highlights​

      + +

      Looking forward​

      +
        +
      • Add new features to nim-libp2p: +QUIC transport, web transport
        • +
      • Update specifications for libp2p GossipSub, +aiming to significantly improve its performance
        • +
      +

      Distributed Systems Testing (DST)​

      +

      The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. +By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.

      +

      Highlights​

      +
        +
      • DST compute resources transitioned from a hosted environment to a dedicated Vac Lab, enabling better customization of resources and adding significantly more compute power -– enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control.
      • Maintained monthly regression simulations for both Waku and Nim-libp2p, -helping us to detect several issues and ensure that future versions do not introduce new ones.
      • Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.

      Looking forward​

      • More testing and simulations for Codex and Nomos
      • Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard

      Nim​

      Several IFT portfolio projects use the Nim ecosystem for its efficiency. -The Nim Service Unit is responsible for the development and maintenance of Nim tooling.

      Highlights​

      • Released Nim-libp2p (v1.7.1, +– enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control.
        • +
      • Maintained monthly regression simulations for both Waku and Nim-libp2p, +helping us to detect several issues and ensure that future versions do not introduce new ones.
        • +
      • Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.
        • +
      +

      Looking forward​

      +
        +
      • More testing and simulations for Codex and Nomos
        • +
      • Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard
        • +
      +

      Nim​

      +

      Several IFT portfolio projects use the Nim ecosystem for its efficiency. +The Nim Service Unit is responsible for the development and maintenance of Nim tooling.

      +

      Highlights​

      +
        +
      • Released Nim-libp2p (v1.7.1, v1.7.0, v1.6.0, v1.5.0, v1.4.0, v1.3.0, -v1.2.0)
      • Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution
      • Nim VSCode Extension
      • Stabilized Nim Language Server

      Smart Contracts (SC)​

      Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.

      Highlights​

      • Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network
      • Wrote specifications for Codex's architectural components and Status's staking contracts
      • Delivered several learn-up sessions on a variety of topics for IFT contributors, including:
        • Stealth addresses
        • Tokenized vaults
        • Rental NFTs
        • Merkle trees
        • Account abstraction
        • EVM deep dive

      Looking forward​

      • Deploy the SNT staking protocol on the Status Network testnet
      • Encourage community security audits via contests
      • Provide smart contract consultation services for IFT portfolio products
      • Engage in more learn-up sessions to promote org-wide knowledge sharing.

      Heading into 2025​

      This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. -The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit.

      As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. +v1.2.0)

      • +
    • Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution
      • +
    • Nim VSCode Extension
      • +
    • Stabilized Nim Language Server
      • +
    +

    Smart Contracts (SC)​

    +

    Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.

    +

    Highlights​

    +
      +
    • Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network
      • +
    • Wrote specifications for Codex's architectural components and Status's staking contracts
      • +
    • Delivered several learn-up sessions on a variety of topics for IFT contributors, including: +
        +
      • Stealth addresses
        • +
      • Tokenized vaults
        • +
      • Rental NFTs
        • +
      • Merkle trees
        • +
      • Account abstraction
        • +
      • EVM deep dive
        • +
      +
      • +
    +

    Looking forward​

    +
      +
    • Deploy the SNT staking protocol on the Status Network testnet
      • +
    • Encourage community security audits via contests
      • +
    • Provide smart contract consultation services for IFT portfolio products
      • +
    • Engage in more learn-up sessions to promote org-wide knowledge sharing.
      • +
    +

    Heading into 2025​

    +

    This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. +The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit.

    +

    As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. Our goal is to serve as an internal conduit of expertise within the organization, supported by a strong RFC culture, maintaining a repository of internal knowledge creation, and identifying and facilitating IFT project synergies. -Such an approach should lead to greater efficiencies across the organization.

    We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. +Such an approach should lead to greater efficiencies across the organization.

    +

    We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. In the final quarter of 2024, Vac stepped up its collaboration with the libp2p community and made a concerted effort to engage the community on the Vac forum. -In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems.

    We look forward to continuing our journey with you!

    Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.

    ]]>     +In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems.

    +

    We look forward to continuing our journey with you!

    +

    Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.

    ]]> <![CDATA[Vac 101: Climbing Merkle Trees]]> @@ -376,94 +644,192 @@ In 2025, we aim to continue working closely with those communities to which we a https://vac.dev/rlog/climbing-merkle-trees Mon, 30 Dec 2024 12:00:00 GMT - In this post, we introduce a crucial data structure used throughout web3.

    Introduction​

    A large amount of data is swapped between users on a blockchain in the form of transactions. + In this post, we introduce a crucial data structure used throughout web3.

    + +

    Introduction​

    +

    A large amount of data is swapped between users on a blockchain in the form of transactions. Over the entire life of a blockchain, the storage space required to maintain a copy of every transaction becomes untenable for most users. However, the integrity of a blockchain relies on a large pool of users that can validate the blockchain's history from its inception to its present state. The data representing the blockchain's state is compressed. -This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users.

    Data compression alone is not the end goal. +This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users.

    +

    Data compression alone is not the end goal. As mentioned, it is essential for users to be able to validate the blockchain's history. The property of compression and validation was solved in Bitcoin by the use of Merkle trees. Merkle trees were introduced first by Ralph Merkle in his dissertation [1]. A Merkle tree is a data structure that compresses a digest of data to a constant size while still providing a method for proving membership of elements of the digest. -A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.

    Tree structure​

    A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. +A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.

    +

    Tree structure​

    +

    A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. The trees that we consider can be arranged in layers with multiple nodes (children) merged into a single node (parent) in the preceding layer. A single node exists in the base layer; this special node is called the root node. -The highest level of the tree consists of childless nodes called leaves.

    Image 1

    A binary tree has one additional property: +The highest level of the tree consists of childless nodes called leaves.

    +

    Image 1

    +

    A binary tree has one additional property: each nonleaf node has exactly two children nodes. That is, we assume that nodes in a binary tree are either a parent node with two children or a leaf. -As strange as it sounds, each child node has exactly one parental node.

    Image 2

    A binary tree with 2n2^n leaves consists of n+1n+1 layers. -Additionally, such a tree has 2n+1−12^{n+1}-1 nodes.

    Merkle trees​

    A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. +As strange as it sounds, each child node has exactly one parental node.

    +

    Image 2

    +

    A binary tree with 2n2^n leaves consists of n+1n+1 layers. +Additionally, such a tree has 2n+1−12^{n+1}-1 nodes.

    +

    Merkle trees​

    +

    A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. Merkle trees are usually taken to have a binary tree structure. -As such, the presentation we provide in this section will be for binary trees.

    Construction​

    In this section, we show how Merkle trees are constructed to compress a digest DD. -Suppose that the digest DD consists of 2n2^n entries; -we assume that the digest DD has this many entries since a Merkle tree is a binary tree. -Additionally, each digest can be padded to ensure that DD has the desired number of entries.

    Each leaf of the Merkle tree contains the hash of a digest entry. +As such, the presentation we provide in this section will be for binary trees.

    +

    Construction​

    +

    In this section, we show how Merkle trees are constructed to compress a digest DD. +Suppose that the digest DD consists of 2n2^n entries; +we assume that the digest DD has this many entries since a Merkle tree is a binary tree. +Additionally, each digest can be padded to ensure that DD has the desired number of entries.

    +

    Each leaf of the Merkle tree contains the hash of a digest entry. Each parent node contains the hash of the concatenation of their child nodes. Through this iterative construction, we reach the root of the tree. The value contained in the root node is called the root hash. -The root hash is a compressed representation of the digest DD.

    Image 3

    Each node in the Merkle tree is computed by taking a hash. -Since a binary tree with 2n2^n leaves has 2n+1−12^{n+1}-1 nodes, -then we need to evaluate 2n+1−12^{n+1}-1 hashes to construct the Merkle tree.

    Merkle tree intregrity​

    A large quantity of data can be compressed to a single hash value. +The root hash is a compressed representation of the digest DD.

    +

    Image 3

    +

    Each node in the Merkle tree is computed by taking a hash. +Since a binary tree with 2n2^n leaves has 2n+1−12^{n+1}-1 nodes, +then we need to evaluate 2n+1−12^{n+1}-1 hashes to construct the Merkle tree.

    +

    Merkle tree intregrity​

    +

    A large quantity of data can be compressed to a single hash value. A natural question to ask is: could a clever party find another digest that yields a Merkle tree with the same root hash? If possible, this would compromise the ledger since the blockchain's history could be altered. Fortunately, Merkle trees are quite secure. -In fact, Merkle trees can be used to both bind and hide a digest.

    The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). -A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value.

    This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on.

    In certain applications, +In fact, Merkle trees can be used to both bind and hide a digest.

    +

    The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). +A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value.

    +

    This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on.

    +

    In certain applications, it may be desirable for the digest of a Merkle tree to be kept confidential. This is achieved with the preimage resistant property of hash functions. A hash function is preimage resistant provided that it is difficult to reverse the hashing operation. -It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest.

    Now, we see that Merkle trees are secured structures that are tamper resistant.

    Proof of membership​

    An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. -This can be done with logarithmic storage and logarithmic computation time.

    Suppose that we want to show that data ℓ\ell is part of the Merkle tree's digest. -Additionally, suppose that hash\mathsf{hash} is the hash function used to construct the tree. -We assume that the hash function hash\mathsf{hash} can be computed in constant-time for any input.

    Suppose that a prover provides data ℓ\ell to a verifier, -and tells the verifier that ℓ\ell corresponds to the iith leaf of the Merkle tree. -For the verifier to be convinced that ℓ\ell is part of the digest, he needs to be able to construct the tree's root hash using hash\mathsf{hash}, ii, ℓ\ell and some additional information from the prover. +It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest.

    +

    Now, we see that Merkle trees are secured structures that are tamper resistant.

    +

    Proof of membership​

    +

    An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. +This can be done with logarithmic storage and logarithmic computation time.

    +

    Suppose that we want to show that data ℓ\ell is part of the Merkle tree's digest. +Additionally, suppose that hash\mathsf{hash} is the hash function used to construct the tree. +We assume that the hash function hash\mathsf{hash} can be computed in constant-time for any input.

    +

    Suppose that a prover provides data ℓ\ell to a verifier, +and tells the verifier that ℓ\ell corresponds to the iith leaf of the Merkle tree. +For the verifier to be convinced that ℓ\ell is part of the digest, he needs to be able to construct the tree's root hash using hash\mathsf{hash}, ii, ℓ\ell and some additional information from the prover. Specifically, the prover must provide the sibling hashes for each value that the verifier can compute. This enables the verifier to compute the parents of the siblings that the prover provides and the values that he was able to produce himself. -The last of the computed parents is the root.

    The leaf index ii indicates whether a hash value provided by the prover is a left or right sibling. -This is done by looking at the binary expansion of ii.

    The verifier can compute the leaf h0=hash(ℓ)h_0 = \mathsf{hash}(\ell). -Next, using h0h_0's sibling, h0′h'_0, provided by the prover, -the verifier can compute h1=hash(h0∥h0′)h_1 = \mathsf{hash}(h_0 \|h'_0) or h1=hash(h0′∥h0)h_1 = \mathsf{hash}(h'_0 \| h_0) -depending on whether h0′h'_0 is a left or right sibling. -This pathing continues until the verifier either successfully computes the root hash (in n+1n+1 hashes) or fails to do so.

    The prover has to provide nn sibling nodes for the proof of membership.

    There is a key detail that is essential for the proof of membership to be secure. -The root hash has to be provided to the verifier prior to the selection of data ℓ\ell. -Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership.

    Capped proof of membership​

    Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. +The last of the computed parents is the root.

    +

    The leaf index ii indicates whether a hash value provided by the prover is a left or right sibling. +This is done by looking at the binary expansion of ii.

    +

    The verifier can compute the leaf h0=hash(ℓ)h_0 = \mathsf{hash}(\ell). +Next, using h0h_0's sibling, h0′h'_0, provided by the prover, +the verifier can compute h1=hash(h0∥h0′)h_1 = \mathsf{hash}(h_0 \|h'_0) or h1=hash(h0′∥h0)h_1 = \mathsf{hash}(h'_0 \| h_0) +depending on whether h0′h'_0 is a left or right sibling. +This pathing continues until the verifier either successfully computes the root hash (in n+1n+1 hashes) or fails to do so.

    +

    The prover has to provide nn sibling nodes for the proof of membership.

    +

    There is a key detail that is essential for the proof of membership to be secure. +The root hash has to be provided to the verifier prior to the selection of data ℓ\ell. +Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership.

    +

    Capped proof of membership​

    +

    Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. A specific layer of the Merkle tree is published instead of just the root hash. -By doing this, a capped proof of membership is just the path from leaf to the published layer.

    Extensions of Merkle trees​

    Merkle trees can be extended in multiple ways. -In this section, we explore a select few of these extensions.

    Sparse Merkle trees​

    A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. +By doing this, a capped proof of membership is just the path from leaf to the published layer.

    +

    Extensions of Merkle trees​

    +

    Merkle trees can be extended in multiple ways. +In this section, we explore a select few of these extensions.

    +

    Sparse Merkle trees​

    +

    A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. Specifically, each digest entry has a particular leaf index. For simplicity, we assume that the index value is computed by taking the hash of the entry. -We note that this is a sorted SMT.

    Let nn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n leaves.

    An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\mathsf{null} marker inserted in. +We note that this is a sorted SMT.

    +

    Let nn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n leaves.

    +

    An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\mathsf{null} marker inserted in. This means that we can prove membership in the way described. -However, we can also prove nonmembership of an element by showing that null\mathsf{null} is located in the element's hash location. -The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership.

    We can take advantage of the sparse nature of SMTs to provide shortened proofs. +However, we can also prove nonmembership of an element by showing that null\mathsf{null} is located in the element's hash location. +The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership.

    +

    We can take advantage of the sparse nature of SMTs to provide shortened proofs. Specifically, it is unlikely for entries to cluster together. -Thus, it is efficient to maintain a list of values:

    Null values
    d0:=Hash(null),d_0 := \mathsf{Hash(null)},
    d1:=Hash(d0∥∥d0),d_1 := \mathsf{Hash(d_0 \|\| d_0)},
    d2:=Hash(d1∥∥d1),d_2 := \mathsf{Hash(d_1 \|\| d_1)},
    ⋮\vdots
    dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \mathsf{Hash(d_{n-2} \|\| d_{n-2})}.

    Each of the did_i's represents the root hash of a Merkle tree with 2i2^i leaves containing null\mathsf{null}. -These values can be used to shorten the time needed to construct an SMT and the length of proofs.

    Proof of nonmembership​

    In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. +Thus, it is efficient to maintain a list of values:

    +
    Null values
    d0:=Hash(null),d_0 := \mathsf{Hash(null)},
    d1:=Hash(d0∥∥d0),d_1 := \mathsf{Hash(d_0 \|\| d_0)},
    d2:=Hash(d1∥∥d1),d_2 := \mathsf{Hash(d_1 \|\| d_1)},
    ⋮\vdots
    dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \mathsf{Hash(d_{n-2} \|\| d_{n-2})}.
    +

    Each of the did_i's represents the root hash of a Merkle tree with 2i2^i leaves containing null\mathsf{null}. +These values can be used to shorten the time needed to construct an SMT and the length of proofs.

    +

    Proof of nonmembership​

    +

    In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. However, the proof of membership may result in false positives due to collisions. This would affect nonmembership proofs as well. -Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest.

    Why is sorting essential? +Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest.

    +

    Why is sorting essential? The sorting mechanism of data can be arbitrarily chosen. However, it is essential that there are no gaps in the ordering. The maximum number of elements that could ever exist in the digest must be known. A simple method for this is to use a hash function to provide fingerprints to the data. Each hash using either SHA-256 or Keccak has 256-bits. -Our entire digest could consist of a maximum of 22562^{256} entries. -This assumes that our digest does not contain collisions.

    The fingerprint of a piece of data ℓ\ell indicates which leaf of the SMT it is contained in. -This means that a nonmembership of ℓ\ell in the SMT becomes a matter of proving that null\mathsf{null} is contained in ℓ\ell's location.

    It is crucial for the SMT to be sorted. -Otherwise, a malicious party can append the entry ℓ\ell to a random location. +Our entire digest could consist of a maximum of 22562^{256} entries. +This assumes that our digest does not contain collisions.

    +

    The fingerprint of a piece of data ℓ\ell indicates which leaf of the SMT it is contained in. +This means that a nonmembership of ℓ\ell in the SMT becomes a matter of proving that null\mathsf{null} is contained in ℓ\ell's location.

    +

    It is crucial for the SMT to be sorted. +Otherwise, a malicious party can append the entry ℓ\ell to a random location. This allows for the malicious party to provide contradictory proofs that prove both membership and nonmembership. We note that the requirement that an SMT is sorted may be too strong of an assumption in centralized cases. -However, sortedness is a necessary property of SMTs for decentralized systems.

    Verkle Trees​

    A proof of membership grows in length as the Merkle tree grows. +However, sortedness is a necessary property of SMTs for decentralized systems.

    +

    Verkle Trees​

    +

    A proof of membership grows in length as the Merkle tree grows. The most obvious approach to remedy this scalability issue is to use Merkle trees in which each node has more than two children. However, this does not fix the issue. -A proof of membership in a kk-nary Merkle tree [6] (each node has kk children) has a proof size log⁡k(n)(k−1)\log_k(n)(k-1). -The multiple k−1k-1 is the number of silbings that a node has on each layer. -Hence, the proof size grows faster than a logarithmic function of the digest size.

    An alternate approach is to use a different data structure: Verkle trees [6]. +A proof of membership in a kk-nary Merkle tree [6] (each node has kk children) has a proof size log⁡k(n)(k−1)\log_k(n)(k-1). +The multiple k−1k-1 is the number of silbings that a node has on each layer. +Hence, the proof size grows faster than a logarithmic function of the digest size.

    +

    An alternate approach is to use a different data structure: Verkle trees [6]. A Verkle tree replaces hash functions with polynomial commitments [7, 8]. -We will explore Verkle trees in a future Vac 101 edition.

    References​

    ]]>     +We will explore Verkle trees in a future Vac 101 edition.

    +

    References​

    +]]>     <![CDATA[Large Message Handling in GossipSub: Potential Improvements]]> @@ -471,9 +837,13 @@ We will explore Verkle trees in a future Vac 101 edition.

    Motivation​

    The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). + Large Message Handling in GossipSub: Potential Improvements

    + +

    Motivation​

    +

    The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). It is particularly relevant for applications that require on-time, network-wide dissemination of large messages, -such as Ethereum and Waku [1,2].

    This matter has been extensively discussed in the libp2p community [3, +such as Ethereum and Waku [1,2].

    +

    This matter has been extensively discussed in the libp2p community [3, 4, 5, 6, @@ -483,164 +853,350 @@ and numerous improvements have been considered (or even incorporated) for the Go [3, 7, 9, -10].

    Problem description​

    Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\lceil \log_D(N) \rceil rounds, -with approximately (D−1)X−1×D(D-1)^{X-1} \times D transmissions in each round, -where X,DX, D represent the round number and mesh size.

    Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \times (F+D), -where FF represents the number of peers included in floodpublish.

    This arrangement works fine for relatively small/moderate message sizes. -However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen.

    Interestingly, a higher DD or FF can also degrade performance in this situation.

    Several aspects contribute to this behavior:

    1. Ideally, a message transmission to a single peer concludes in τ1=LR+P\tau_1 = \frac {L}{R}+P (ignoring any message processing time), -where L,R,PL, R, P represent message size, data rate, and link latency. +10].

      +

      Problem description​

      +

      Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\lceil \log_D(N) \rceil rounds, +with approximately (D−1)X−1×D(D-1)^{X-1} \times D transmissions in each round, +where X,DX, D represent the round number and mesh size.

      +

      Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \times (F+D), +where FF represents the number of peers included in floodpublish.

      +

      This arrangement works fine for relatively small/moderate message sizes. +However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen.

      +

      Interestingly, a higher DD or FF can also degrade performance in this situation.

      +

      Several aspects contribute to this behavior:

      +
        +
      1. +

        Ideally, a message transmission to a single peer concludes in τ1=LR+P\tau_1 = \frac {L}{R}+P (ignoring any message processing time), +where L,R,PL, R, P represent message size, data rate, and link latency. Therefore, the time required for sending a message on a 100Mbps link with 100ms latency -jumps from τ110KB=100.8ms\tau_1^{10KB} = 100.8ms for a 10KB message to τ11MB=180ms\tau_1^{1MB} = 180ms for a 1MB message. -For DD peers, the transmission time increases to τD1MB=(80×D)+100ms\tau_D^{1MB} = (80 \times D) + 100ms, -triggering additional queuing delays (proportional to the transmission queue size) during each transmission round.

      2. In practice, τ11MB\tau_1^{1MB} sometimes rises to several hundred milliseconds, +jumps from τ110KB=100.8ms\tau_1^{10KB} = 100.8ms for a 10KB message to τ11MB=180ms\tau_1^{1MB} = 180ms for a 1MB message. +For DD peers, the transmission time increases to τD1MB=(80×D)+100ms\tau_D^{1MB} = (80 \times D) + 100ms, +triggering additional queuing delays (proportional to the transmission queue size) during each transmission round.

        +
        3. +
      3. +

        In practice, τ11MB\tau_1^{1MB} sometimes rises to several hundred milliseconds, further exaggerating the abovementioned queuing delays. This rise is because TCP congestion avoidance algorithms usually limit maximum in-flight bytes in a round trip time (RTT) -based on the congestion window (CwndC_{wnd}) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \times MSS}, -with CwndC_{wnd} rising with the data transfer for each flow. +based on the congestion window (CwndC_{wnd}) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \times MSS}, +with CwndC_{wnd} rising with the data transfer for each flow. Consequently, sending the same message through a newly established (cold) connection takes longer. -The message transfer time lowers as the CwndC_{wnd} grows. +The message transfer time lowers as the CwndC_{wnd} grows. Therefore, performance-improvement practices such as floodpublish, frequent mesh adjustment, and lazy sending typically result in longer than expected message dissemination times for large messages (due to cold connections). -It is also worth mentioning that some TCP variants reset their CwndC_{wnd} after different periods of inactivity.

      4. Theoretically, the message transmission time to D peers (τD)(\tau_D) remains the same +It is also worth mentioning that some TCP variants reset their CwndC_{wnd} after different periods of inactivity.

        +
        5. +
      5. +

        Theoretically, the message transmission time to D peers (τD)(\tau_D) remains the same even if the message is relayed sequentially to all peers or simultaneous transmissions are carried out, -i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i +i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i However, sequential transmissions finish early for individual peers, allowing them to relay early. -This may result in quicker network-wide message dissemination.

      1. A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). +This may result in quicker network-wide message dissemination.

        +
        2. +
      2. +

        A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). As the message disseminates, it's not uncommon for some peers to receive it much earlier than others. Early gossip (IHAVE announcements) may bring in many IWANT requests to the early receivers (even from peers already receiving the same message), -which adds to their workload.

      3. A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. +which adds to their workload.

        +
        4. +
      4. +

        A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. As a result, already scheduled messages are prioritized over those published by the peer itself, introducing a significant initial delay to the locally published messages. Enqueuing IWANT replies to the outgoing message queue can further exaggerate the problem. The lack of adaptiveness and standardization in outgoing message prioritization are key factors that can lead to noticeable inconsistency -in message dissemination latency at each hop, even in similar network conditions.

      5. Message size directly contributes to peers' workloads in terms of processing and transmission time. +in message dissemination latency at each hop, even in similar network conditions.

        +
        6. +
      6. +

        Message size directly contributes to peers' workloads in terms of processing and transmission time. It also raises the probability of simultaneous redundant transmissions to the same peer, resulting in bandwidth wastage, congestion, and slow message propagation to the network. -Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers.

      7. Most use cases necessitate validating received messages before forwarding them to the next-hop peers. -For a higher message transfer time (τ)(\tau ), this store-and-forward delay accumulates across the hops traveled by the message.

      Possible improvements​

      1. Minimizing transfer time for large messages​

      The impact of message size and achievable data rate on message transmit time τ\tau is crucial -as this time accumulates due to the store-and-forward delay introduced at intermediate hops.

      Some possible improvements to minimize overall message dissemination latency include:

      a. Message fragmentation​

      In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) -can be simplified to roughly δ≈δTx+Ph\delta \approx \delta_{Tx} + P_h, where δTx\delta_{Tx} represents accumulative message transmit time denoted as δTx=SR×h\delta_{Tx} = \frac{S}{R} \times h, -with S,RS, R being the data size and data rate, -and h,Phh, P_h being the number of hops in the longest path and message propagation time through the longest path.

      Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\frac{\delta_{Tx}}{n}. +Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers.

      +
      2. +
    2. +

      Most use cases necessitate validating received messages before forwarding them to the next-hop peers. +For a higher message transfer time (τ)(\tau ), this store-and-forward delay accumulates across the hops traveled by the message.

      +
      3. +
    +

    Possible improvements​

    +

    1. Minimizing transfer time for large messages​

    +

    The impact of message size and achievable data rate on message transmit time τ\tau is crucial +as this time accumulates due to the store-and-forward delay introduced at intermediate hops.

    +

    Some possible improvements to minimize overall message dissemination latency include:

    +

    a. Message fragmentation​

    +

    In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) +can be simplified to roughly δ≈δTx+Ph\delta \approx \delta_{Tx} + P_h, where δTx\delta_{Tx} represents accumulative message transmit time denoted as δTx=SR×h\delta_{Tx} = \frac{S}{R} \times h, +with S,RS, R being the data size and data rate, +and h,Phh, P_h being the number of hops in the longest path and message propagation time through the longest path.

    +

    Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\frac{\delta_{Tx}}{n}. As a received fragment can be immediately relayed by the receiver (while the sender is still transmitting the remaining fragments), -it reduces the transmit time to δTx=SR×2h−1n\delta_{Tx} = \frac{S}{R} \times \frac{2h-1}{n}.

    This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions.

    However, it is worth noting that many applications require each fragment to be individually verifiable. +it reduces the transmit time to δTx=SR×2h−1n\delta_{Tx} = \frac{S}{R} \times \frac{2h-1}{n}.

    +

    This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions.

    +

    However, it is worth noting that many applications require each fragment to be individually verifiable. At the same time, message fragmentation allows a malicious peer to never relay some fragments of a message, -which can lead to a significant rise in the application's receive buffer size.

    Therefore, message fragmentation requires a careful tradeoff analysis between time and risks.

    b. Message staggering​

    Considering the same bandwidth, the time τD\tau_D required for sending a message to D peers stays the same, -even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i.

    However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\tau_1 \approx \frac{\tau_D}{D}) due to bandwidth concentration for a particular peer. -So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers.

    As a result, after every τDD\frac{\tau_D}{D} milliseconds, -the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\ \forall\ X \in \{0, D-1\} and by ∑k=X−DX−1λk ∀ X≥D\sum_{k=X-D}^{X-1} \lambda_k\ \forall\ X \geq D. -Here, XX represents message transmission round X=i⋅τDD∣i∈N0X = i \cdot \frac{\tau_D}{D} \mid i \in \mathbb{N}_0, and λk\lambda_k represents the number of peers that received the message in round kk.

    It is worth noting that a realistic network imposes certain constraints on staggering for peers. +which can lead to a significant rise in the application's receive buffer size.

    +

    Therefore, message fragmentation requires a careful tradeoff analysis between time and risks.

    +

    b. Message staggering​

    +

    Considering the same bandwidth, the time τD\tau_D required for sending a message to D peers stays the same, +even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\tau_D = \sum_{i=1}^{D} \tau_i.

    +

    However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\tau_1 \approx \frac{\tau_D}{D}) due to bandwidth concentration for a particular peer. +So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers.

    +

    As a result, after every τDD\frac{\tau_D}{D} milliseconds, +the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\ \forall\ X \in \{0, D-1\} and by ∑k=X−DX−1λk ∀ X≥D\sum_{k=X-D}^{X-1} \lambda_k\ \forall\ X \geq D. +Here, XX represents message transmission round X=i⋅τDD∣i∈N0X = i \cdot \frac{\tau_D}{D} \mid i \in \mathbb{N}_0, and λk\lambda_k represents the number of peers that received the message in round kk.

    +

    It is worth noting that a realistic network imposes certain constraints on staggering for peers. For instance, in a network with dissimilar peer capabilities, placing a slow peer (also in cases where many senders simultaneously select the same peer) at the head of the transmission queue -may result in head-of-line blocking for the message queue.

    At the same time, early receivers get many IWANT requests, increasing their workload.

    c. Message prioritization for slow senders​

    A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. +may result in head-of-line blocking for the message queue.

    +

    At the same time, early receivers get many IWANT requests, increasing their workload.

    +

    c. Message prioritization for slow senders​

    +

    A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. Any new message transmission at this stage (especially the locally published messages) gets delayed. -Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers.

    For instance, any GossipSub peer will likely receive every message from multiple senders, +Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers.

    +

    For instance, any GossipSub peer will likely receive every message from multiple senders, leading to redundant transmissions [11]. Implementing efficient strategies (only for slow senders) like lazy sending and prioritizing locally published messages/IWANT replies over already queued messages -can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers.

    A peer can identify itself as a slow peer by using any bandwidth estimation approach -or simply setting an outgoing message queue threshold for all mesh members.

    Eliminating/deprioritizing some messages can lower a peer's score, +can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers.

    +

    A peer can identify itself as a slow peer by using any bandwidth estimation approach +or simply setting an outgoing message queue threshold for all mesh members.

    +

    Eliminating/deprioritizing some messages can lower a peer's score, but it also earns the peer an overall better score by achieving some early message transfers.
    For instance, sending many near-first messages can only save a peer from a deficit penalty. On the other hand, sending only one message (assuming MeshMessageDeliveriesThreshold defaults to 1) -as the first delivered message can add to the accumulative peer score.

    2. Mitigating transport issues​

    Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time -as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd}) size.

    Rapid adaptation of CwndC_{wnd} to the available network conditions can help lower message dissemination latency.

    Therefore, selecting a more suitable TCP variant like BBR, +as the first delivered message can add to the accumulative peer score.

    +

    2. Mitigating transport issues​

    +

    Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time +as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd}) size.

    +

    Rapid adaptation of CwndC_{wnd} to the available network conditions can help lower message dissemination latency.

    +

    Therefore, selecting a more suitable TCP variant like BBR, which is known for its ability to dynamically adjust the congestion window based on network conditions, -can significantly enhance GossipSub's performance.

    At the same time, parameters like receive window scaling and initial CwndC_{wnd} also impact message transfer time, -but these are usually OS-specific system-wide choices.

    One possible solution is to raise CwndC_{wnd} by exchanging data over the newly established connection. +can significantly enhance GossipSub's performance.

    +

    At the same time, parameters like receive window scaling and initial CwndC_{wnd} also impact message transfer time, +but these are usually OS-specific system-wide choices.

    +

    One possible solution is to raise CwndC_{wnd} by exchanging data over the newly established connection. This data may involve useful details like peer exchange information and gossip to build initial trust, -or GossipSub can use some dummy data to raise CwndC_{wnd} to a reasonable level.

    It's important to understand that some TCP variants reset CwndC_{wnd} after specific periods of inactivity [12]. +or GossipSub can use some dummy data to raise CwndC_{wnd} to a reasonable level.

    +

    It's important to understand that some TCP variants reset CwndC_{wnd} after specific periods of inactivity [12]. This can lead to a decline in TCP's performance for applications -that generate traffic after intervals long enough to trigger the resetting of the congestion window.

    Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13].

    The limitations faced with CwndC_{wnd} scaling also impact some performance optimizations in GossipSub. -For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency.

    However, a small CwndC_{wnd} value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time +that generate traffic after intervals long enough to trigger the resetting of the congestion window.

    +

    Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13].

    +

    The limitations faced with CwndC_{wnd} scaling also impact some performance optimizations in GossipSub. +For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency.

    +

    However, a small CwndC_{wnd} value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time [4]. -Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth.

    The same is the case with IWANT replies.

    Maintaining a bigger mesh (with warm TCP connections) and relaying to DD peers can be a better alternative to this problem.

    3. Eliminating redundant transmissions​

    For every received packet, a peer makes roughly DD transmissions to contribute its fair share to the spread of messages. +Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth.

    +

    The same is the case with IWANT replies.

    +

    Maintaining a bigger mesh (with warm TCP connections) and relaying to DD peers can be a better alternative to this problem.

    +

    3. Eliminating redundant transmissions​

    +

    For every received packet, a peer makes roughly DD transmissions to contribute its fair share to the spread of messages. However, the fact that many recipients had already received the message (from some other peer) -makes this message propagation inefficient.

    Although the DD-spread is attributed to quicker dissemination and resilience against non-conforming peers, +makes this message propagation inefficient.

    +

    Although the DD-spread is attributed to quicker dissemination and resilience against non-conforming peers, many potential solutions can still minimize redundant transmissions -while preserving the resilience of GossipSub.

    These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, +while preserving the resilience of GossipSub.

    +

    These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, not only address the issue of redundancy but also provide an opportunity for bandwidth optimization, -especially for resource-constrained peers.

    For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], -can significantly reduce redundant transmissions.

    It allows any node to notify its mesh members that it has already received a message, +especially for resource-constrained peers.

    +

    For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], +can significantly reduce redundant transmissions.

    +

    It allows any node to notify its mesh members that it has already received a message, thereby preventing them from resending the same message. -This functionality is useful when a node receives a message larger than a specified threshold.

    In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages.

    It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, -i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message.

    This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience.

    However, the use of IDONTWANT messages alone has an inherent limitation. -For instance, a peer can only send an IDONTWANT after receiving the complete message.

    A large message transmission consumes significant time. -For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd} and latency).

    As a result, other mesh members may also start transmitting the same message during this interval. -A few potential solutions include:

    a. Staggering with IDONTWANT messages​

    As previously discussed, staggering can significantly reduce network-wide message dissemination latency. -This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach.

    Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. +This functionality is useful when a node receives a message larger than a specified threshold.

    +

    In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages.

    +

    It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, +i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message.

    +

    This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience.

    +

    However, the use of IDONTWANT messages alone has an inherent limitation. +For instance, a peer can only send an IDONTWANT after receiving the complete message.

    +

    A large message transmission consumes significant time. +For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd} and latency).

    +

    As a result, other mesh members may also start transmitting the same message during this interval. +A few potential solutions include:

    +

    a. Staggering with IDONTWANT messages​

    +

    As previously discussed, staggering can significantly reduce network-wide message dissemination latency. +This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach.

    +

    Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. This is because a node only saturates its bandwidth for a small subset of mesh peers, -leading to early transmissions and prompt IDONTWANT message notifications to the mesh members.

    It is worth highlighting that staggering can be implemented in various ways.

    For example, it can be applied to peers (peer staggering) -where a node sequentially relays the same message to all peers one by one.

    Alternatively, a node can send a different message to every peer (message staggering or rotational sending), +leading to early transmissions and prompt IDONTWANT message notifications to the mesh members.

    +

    It is worth highlighting that staggering can be implemented in various ways.

    +

    For example, it can be applied to peers (peer staggering) +where a node sequentially relays the same message to all peers one by one.

    +

    Alternatively, a node can send a different message to every peer (message staggering or rotational sending), allowing IDONTWANTs for other messages to arrive during this time. -The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time.

    As the peers in staggered sending are sequentially covered -(with a faster speed due to bandwidth concentration), this leads to another problem.

    The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. +The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time.

    +

    As the peers in staggered sending are sequentially covered +(with a faster speed due to bandwidth concentration), this leads to another problem.

    +

    The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. IHAVE announcements for newly received large messages trigger IWANTs from nodes (including those already receiving the same message), -leading to an additional workload for early receivers [14].

    Potential solutions to mitigate these problems include:

    1) Defering IHAVE announcements for large messages.

    Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. +leading to an additional workload for early receivers [14].

    +

    Potential solutions to mitigate these problems include:

    +
      +
    1. Defering IHAVE announcements for large messages.
      2. +
    +

    Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. However, deciding on a suitable deferred interval is crucial for optimal performance. -One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers.

    2) Defering IWANT requests for messages that are currently being received.

    This requires prior knowledge of msgIDs for the messages under reception. +One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers.

    +
      +
    1. Defering IWANT requests for messages that are currently being received.
      2. +
    +

    This requires prior knowledge of msgIDs for the messages under reception. Knowing the message length is also essential in deciding a suitable defer interval -to handle situations where a sender starts sending a message and never completes the transmission.

    3) Not issuing IWANT for a message if at least KK peers have transmitted IDONTWANT for the same message -(as this indicates that these peers will eventually relay this message).

    However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. -A delayed IWANT, along with negative peer scoring, can remedy this problem.

    b. IMReceiving message​

    A peer can issue an IDONTWANT only after it has received the entire message. +to handle situations where a sender starts sending a message and never completes the transmission.

    +
      +
    1. Not issuing IWANT for a message if at least KK peers have transmitted IDONTWANT for the same message +(as this indicates that these peers will eventually relay this message).
      2. +
    +

    However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. +A delayed IWANT, along with negative peer scoring, can remedy this problem.

    +

    b. IMReceiving message​

    +

    A peer can issue an IDONTWANT only after it has received the entire message. However, a large message transmission may take several hundred milliseconds to complete. -During this time, many other mesh members may start relaying the same message.

    Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, -significantly compromising the effectiveness of IDONTWANT messages.

    Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. +During this time, many other mesh members may start relaying the same message.

    +

    Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, +significantly compromising the effectiveness of IDONTWANT messages.

    +

    Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. If a receiver is already receiving the same message from another sender, -the receiver can request to defer this transmission by sending a brief IMReceiving message.

    An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval.

    However, waiting for IMReceiving after sending the preamble can delay the message transmission. +the receiver can request to defer this transmission by sending a brief IMReceiving message.

    +

    An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval.

    +

    However, waiting for IMReceiving after sending the preamble can delay the message transmission. On the other hand, proceeding with message transfer (after sending the preamble) leads to another problem: -it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message.

    To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), +it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message.

    +

    To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), urging other mesh peers to defer sending the same message [15, -16].

    The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. -This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages.

    While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, -it's crucial to be aware of the potential risks.

    A malicious user can exploit this approach to disrupt message transmission -either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers.

    This could ultimately result in network-wide slow message propagation.

    However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks.

    c. IDONTWANT message with reduced forwarding​

    It is common for slow peers to pile up outgoing message queues, +16].

    +

    The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. +This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages.

    +

    While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, +it's crucial to be aware of the potential risks.

    +

    A malicious user can exploit this approach to disrupt message transmission +either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers.

    +

    This could ultimately result in network-wide slow message propagation.

    +

    However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks.

    +

    c. IDONTWANT message with reduced forwarding​

    +

    It is common for slow peers to pile up outgoing message queues, especially for large message transfers. This results in a significant queuing delay for outgoing messages. -Reduced message forwarding can help decrease the workload of slower peers.

    On receiving a message longer than the specified threshold, -a slow peer can relay it to only K∈DK \in D peers and send an IDONTWANT message to all the peers in DD.

    In this arrangement, the IDONTWANT message serves an additional purpose: -to promptly announce data availability, reinforcing redundancy in the presence of adversaries.

    When a peer receives an IDONTWANT for an unseen message, +Reduced message forwarding can help decrease the workload of slower peers.

    +

    On receiving a message longer than the specified threshold, +a slow peer can relay it to only K∈DK \in D peers and send an IDONTWANT message to all the peers in DD.

    +

    In this arrangement, the IDONTWANT message serves an additional purpose: +to promptly announce data availability, reinforcing redundancy in the presence of adversaries.

    +

    When a peer receives an IDONTWANT for an unseen message, it learns about the new message and can request it by sending an IWANT request without waiting for the heartbeat (gossip) interval. -As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network.

    This approach conserves peer bandwidth by minimizing redundant transmissions -while ensuring GossipSub resilience at the cost of one RTT (for missing peers).

    Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages).

    However, finding an appropriate value for KK is crucial for optimal performance. -A smaller KK saves peer bandwidth, while a larger KK achieves quicker spread until outgoing message queues pile up. -Setting K=DlowK = D_{low} can be one option.

    It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-K peers). +As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network.

    +

    This approach conserves peer bandwidth by minimizing redundant transmissions +while ensuring GossipSub resilience at the cost of one RTT (for missing peers).

    +

    Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages).

    +

    However, finding an appropriate value for KK is crucial for optimal performance. +A smaller KK saves peer bandwidth, while a larger KK achieves quicker spread until outgoing message queues pile up. +Setting K=DlowK = D_{low} can be one option.

    +

    It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-K peers). However, a minimized workload enables early message dissemination to the remaining peers. -These early transmissions and randomized KK set selection can help achieve an overall better peer score.

    4. Message prioritization​

    Despite the standardized specifications of the GossipSub protocol, -the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization.

    It is worth mentioning that every node is responsible for transmitting different types of packets, -including control messages, locally published messages, messages received from mesh members, IWANT replies, etc.

    As long as traffic volume is lower than the available data rate, -the message forwarding mechanisms yield similar results due to negligible queuing delays.

    However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), -the outgoing message queue(s) sizes rise, potentially impacting the network's performance.

    In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, +These early transmissions and randomized KK set selection can help achieve an overall better peer score.

    +

    4. Message prioritization​

    +

    Despite the standardized specifications of the GossipSub protocol, +the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization.

    +

    It is worth mentioning that every node is responsible for transmitting different types of packets, +including control messages, locally published messages, messages received from mesh members, IWANT replies, etc.

    +

    As long as traffic volume is lower than the available data rate, +the message forwarding mechanisms yield similar results due to negligible queuing delays.

    +

    However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), +the outgoing message queue(s) sizes rise, potentially impacting the network's performance.

    +

    In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, introducing a queuing delay proportional to the queue size. -The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc.

    On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. -One possible solution is to use weighted queues for a fair spread of messages.

    Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time -and allow for customizable message handling.

    For example, staggering between peers and messages can be better managed by using priority queues. +The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc.

    +

    On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. +One possible solution is to use weighted queues for a fair spread of messages.

    +

    Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time +and allow for customizable message handling.

    +

    For example, staggering between peers and messages can be better managed by using priority queues. However, it is important to note that message prioritization also introduces additional complexity to the system, -necessitating sophisticated algorithms for better message handling.

    5. Maximizing benefits from IWANT messages​

    During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. +necessitating sophisticated algorithms for better message handling.

    +

    5. Maximizing benefits from IWANT messages​

    +

    During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. These peers can use IWANT messages to request any missing messages. -A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval.

    The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. +A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval.

    +

    The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. They bridge the information gap between nearby and far-off peers, ensuring that information can be disseminated to peers outside the mesh. -This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks.

    However, it is essential to understand that high transmission times for large messages -require careful due diligence when using IWANT messages for reasons not limited to:

    1) A large message reception may take several hundred milliseconds to complete. -During this time, an IHAVE message announcing the same message ID will trigger an IWANT request.

    2) A peer can send IWANT requests for the same message to multiple nodes, -leading to simultaneous transmissions of the same message.

    3) Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, -resulting in lower peer scores and slower message propagation.

    A few possible solutions to mitigate this problem may include:

    1) Issuing IHAVE announcements only after the message is delivered to many mesh peers.

    2) Allocating a volume-based budget to service IWANT requests during each heartbeat interval.

    3) Deferring IWANT requests for messages that are currently being received.

    4) Deferring IWANT requests if at least KK IDONTWANTs are received for the same message.

    5) A large message transmission can yield high CwndC_{wnd}; preferring such peers during mesh maintenance can be helpful.

    Summary​

    This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages.

    We delve into multiple factors, +This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks.

    +

    However, it is essential to understand that high transmission times for large messages +require careful due diligence when using IWANT messages for reasons not limited to:

    +
      +
    1. +

      A large message reception may take several hundred milliseconds to complete. +During this time, an IHAVE message announcing the same message ID will trigger an IWANT request.

      +
      2. +
    2. +

      A peer can send IWANT requests for the same message to multiple nodes, +leading to simultaneous transmissions of the same message.

      +
      3. +
    3. +

      Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, +resulting in lower peer scores and slower message propagation.

      +
      4. +
    +

    A few possible solutions to mitigate this problem may include:

    +
      +
    1. +

      Issuing IHAVE announcements only after the message is delivered to many mesh peers.

      +
      2. +
    2. +

      Allocating a volume-based budget to service IWANT requests during each heartbeat interval.

      +
      3. +
    3. +

      Deferring IWANT requests for messages that are currently being received.

      +
      4. +
    4. +

      Deferring IWANT requests if at least KK IDONTWANTs are received for the same message.

      +
      5. +
    5. +

      A large message transmission can yield high CwndC_{wnd}; preferring such peers during mesh maintenance can be helpful.

      +
      6. +
    +

    Summary​

    +

    This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages.

    +

    We delve into multiple factors, such as increased message transmit times, store-and-forward delays, congestion avoidance mechanisms, and prioritization between messages, -to establish a comprehensive understanding of the problem.

    The study also explores the performance of optimization efforts -like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions.

    A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. +to establish a comprehensive understanding of the problem.

    +

    The study also explores the performance of optimization efforts +like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions.

    +

    A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. Coupling this constraint with increased message transmission times -results in notable store-and-forward delays accumulating at each hop.

    Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation -by utilizing a considerable share of available bandwidth on redundant transmissions.

    Therefore, approaches focused on eliminating redundant transmissions +results in notable store-and-forward delays accumulating at each hop.

    +

    Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation +by utilizing a considerable share of available bandwidth on redundant transmissions.

    +

    Therefore, approaches focused on eliminating redundant transmissions (IDONTWANT, IMReceiving, lazy sending, etc.) can prove helpful. At the same time, strategies aimed at reducing store-and-forward delays -(fragmentation, staggering, prioritization, etc.) can prove beneficial.

    It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. +(fragmentation, staggering, prioritization, etc.) can prove beneficial.

    +

    It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. Some of these have already been explored and discussed to some extent [5, 17, 18]. -We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings.

    Please feel free to join the discussion and leave feedback regarding this post in the -VAC forum.

    References​

    [1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844

    [2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/

    [3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666

    [4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383

    [5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw

    [6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850

    [7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911

    [8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859

    [9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh

    [10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52

    [11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921

    [12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1

    [13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558

    [14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101

    [15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281

    [16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283

    [17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366

    [18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/

    ]]> +We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings.

    +

    Please feel free to join the discussion and leave feedback regarding this post in the +VAC forum.

    +

    References​

    +

    [1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844

    +

    [2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/

    +

    [3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666

    +

    [4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383

    +

    [5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw

    +

    [6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850

    +

    [7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911

    +

    [8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859

    +

    [9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh

    +

    [10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52

    +

    [11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921

    +

    [12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1

    +

    [13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558

    +

    [14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101

    +

    [15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281

    +

    [16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283

    +

    [17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366

    +

    [18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/

    ]]>     <![CDATA[Libp2p GossipSub IDONTWANT Message Performance Impact]]> @@ -648,33 +1204,76 @@ We are nearing the completion of a comprehensive performance evaluation of these https://vac.dev/rlog/gsub-idontwant-perf-eval Mon, 28 Oct 2024 12:00:00 GMT - This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.

    Overview​

    IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. -Cutting down on duplicates can potentially render two significant advantages:

    1. Reducing bandwidth requirements

    2. Reducing message dissemination time (latency)

    For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message.

    Duplicates investigation reveals that -the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network.

    This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, + This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.

    + +

    Overview​

    +

    IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. +Cutting down on duplicates can potentially render two significant advantages:

    +
      +
    1. +

      Reducing bandwidth requirements

      +
      2. +
    2. +

      Reducing message dissemination time (latency)

      +
      3. +
    +

    For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message.

    +

    Duplicates investigation reveals that +the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network.

    +

    This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, showing a potential for a significant drop in bandwidth utilization. -On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.

    Experiments​

    We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) -to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time.

    The experiments are performed on nim-libp2p using the shadow simulator. -The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages.

    In all experiments, ten messages are transmitted to the network, i.e., +On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.

    +

    Experiments​

    +

    We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) +to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time.

    +

    The experiments are performed on nim-libp2p using the shadow simulator. +The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages.

    +

    In all experiments, ten messages are transmitted to the network, i.e., ten peers (publishers) are selected as the message transmitters. Every publisher transmits exactly one message, and inter-packet spacing (delay) is set to four seconds for each published message. -For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class.

    At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}. -These messages are not included in latency computations.

    The simulation details are presented in the table below.

    ParameterValueParameterValue
    Peers2000Publishers10
    Peer bandwidth50-150 MbpsLink latency40-130 ms
    Message size1KB, 50KB, 500KB, 1MBDD8
    Heartbeat interval700ms, 1000ms, 1500msDlowD_{low}6
    FloodPublishFalseDhighD_{high}12
    Gossip factor0.05Muxeryamux

    Findings​

    We use bandwidth utilization and latency as evaluation metrics. +For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class.

    +

    At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}. +These messages are not included in latency computations.

    +

    The simulation details are presented in the table below.

    +
    ParameterValueParameterValue
    Peers2000Publishers10
    Peer bandwidth50-150 MbpsLink latency40-130 ms
    Message size1KB, 50KB, 500KB, 1MBDD8
    Heartbeat interval700ms, 1000ms, 1500msDlowD_{low}6
    FloodPublishFalseDhighD_{high}12
    Gossip factor0.05Muxeryamux
    +

    Findings​

    +

    We use bandwidth utilization and latency as evaluation metrics. Bandwidth utilization represents total network-wide traffic (including gossip and other control messages). Latency refers to network-wide message dissemination time. -The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights.

    Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. +The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights.

    +

    Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. A higher drop is seen with a higher heartbeat interval. Interestingly, a relatively low bandwidth reduction (12-20%) is seen for 1MB messages, -compared to 500KB messages (18-21%).

    Image 1
    Image 2
    Image 3

    This is because downloading a large message may consume several hundred milliseconds. +compared to 500KB messages (18-21%).

    +
    Image 1
    Image 2
    Image 3
    +

    This is because downloading a large message may consume several hundred milliseconds. During this time, a receiver will likely generate multiple IWANT requests -for the same message, increasing bandwidth utilization.

    Moreover, a peer can generate +for the same message, increasing bandwidth utilization.

    +

    Moreover, a peer can generate IDONTWANTs only after it has finished downloading the message. -A longer download time will result in simultaneous reception of the same message from other mesh members.

    Image 1
    Image 2

    These IWANT requests mainly overwhelm early message receivers, +A longer download time will result in simultaneous reception of the same message from other mesh members.

    +
    Image 1
    Image 2
    +

    These IWANT requests mainly overwhelm early message receivers, which can negatively impact message dissemination time on some occasions. -Therefore, a similar message dissemination time is seen with and without IDONTWANT messages.

    Image 1
    Image 2
    Image 3

    Similar results are seen on our large-scale deployment runs -(running Waku nodes in Kubernetes).

    Please feel free to join the discussion and leave feedback regarding this post in the -VAC forum.

    References​

    ]]>     +Therefore, a similar message dissemination time is seen with and without IDONTWANT messages.

    +
    Image 1
    Image 2
    Image 3
    +

    Similar results are seen on our large-scale deployment runs +(running Waku nodes in Kubernetes).

    +

    Please feel free to join the discussion and leave feedback regarding this post in the +VAC forum.

    +

    References​

    +]]>     <![CDATA[Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument]]> @@ -682,14 +1281,18 @@ Therefore, a similar message dissemination time is seen with and without IDONTWA https://vac.dev/rlog/vac101-fiat-shamir Tue, 15 Oct 2024 12:00:00 GMT - In this post, we introduce a common technique used to convert interactive protocols to their noninteractive variant.

    Introduction​

    The set of interactive protocols form a class of protocols that consist of communication between two parties: the Prover and the Verifier. + In this post, we introduce a common technique used to convert interactive protocols to their noninteractive variant.

    + +

    Introduction​

    +

    The set of interactive protocols form a class of protocols that consist of communication between two parties: the Prover and the Verifier. The Prover tries to convince the Verifier of a given claim. For example, the Prover may want to convince the Verifier that she owns a specific Unspent Transaction Output (UTXO); that is, the Prover possesses the ability to spend the UTXO. In many instances, there is information that the Prover does not wish to reveal to the Verifier. In our example, it is critical that the Prover does not provide the Verifier with the spending key associated with her UTXO. In addition to the Prover's claim and secret data, there is additional data, public parameters, that the claimed statement is expressed in terms of. -The public parameters can be thought of as the basis for all similar claims.

    In an interactive protocol, the Prover and the Verifier are in active communication. +The public parameters can be thought of as the basis for all similar claims.

    +

    In an interactive protocol, the Prover and the Verifier are in active communication. Specifically, the Prover and the Verifier exchange messages so that the Verifier can validate the Prover's claim. However, this communication is not practical for many applications. It is necessary that any party can verify the Prover's claim in decentralized systems. @@ -698,203 +1301,799 @@ Instead, it is desirable for the Prover to generate a proof on their own that ca To achieve this, it is necessary for the Prover to generate the Verifier's messages in such a way that the Prover cannot manipulate the Verifier's messages for her benefit. The Fiat-Shamir heuristic 1 is used for this purpose. -Even though much of our discussion will focus on Σ\Sigma-protocols, -the Fiat-Shamir heuristic is not limited to Σ\Sigma-protocols. +Even though much of our discussion will focus on Σ\Sigma-protocols, +the Fiat-Shamir heuristic is not limited to Σ\Sigma-protocols. The Fiat-Shamir heuristic has been applied to zk-SNARKs, but the security in this setting has been the subject of discussion and research in recent years. -Block et al. 2 provide the first formal analysis of Fiat-Shamir heuristic in zk-SNARKs.

    Sigma Protocols​

    A Σ\Sigma-protocol is a family of interactive protocols that consists of three publicly transmitted messages between the Prover and the Verifier. -In particular, the protocol has the following framework:

    ProverVerifier
    ⟶commitment\stackrel{\mathsf{commitment}}{\longrightarrow}
    ⟵challenge\stackrel{\mathsf{challenge}}{\longleftarrow}
    ⟶response\stackrel{\mathsf{response}}{\longrightarrow}

    These three messages form the protocol's transcript: (commitment,challenge,response)(\mathsf{commitment}, \mathsf{challenge}, \mathsf{response}). +Block et al. 2 provide the first formal analysis of Fiat-Shamir heuristic in zk-SNARKs.

    +

    Sigma Protocols​

    +

    A Σ\Sigma-protocol is a family of interactive protocols that consists of three publicly transmitted messages between the Prover and the Verifier. +In particular, the protocol has the following framework:

    +
    ProverVerifier
    ⟶commitment\stackrel{\mathsf{commitment}}{\longrightarrow}
    ⟵challenge\stackrel{\mathsf{challenge}}{\longleftarrow}
    ⟶response\stackrel{\mathsf{response}}{\longrightarrow}
    +

    These three messages form the protocol's transcript: (commitment,challenge,response)(\mathsf{commitment}, \mathsf{challenge}, \mathsf{response}). The Verifier uses all three of these messages to validate the Prover's original claim. The Verifier's challenge should be selected uniform random from all possible challenges. -Based on this selection, a dishonest Prover can only convince the Verifier with a negligible probability.

    The Schnorr Protocol​

    The Schnorr protocol 3 is usually the first Σ\Sigma-protocol that one studies. +Based on this selection, a dishonest Prover can only convince the Verifier with a negligible probability.

    +

    The Schnorr Protocol​

    +

    The Schnorr protocol 3 is usually the first Σ\Sigma-protocol that one studies. Additionally, the Schnorr protocol can be used as an efficient signature scheme. -The Schnorr protocol provides a framework that enables the Prover to convince the Verifier that: for group elements gg and XX, -the Prover knows the power to raise gg to obtain XX. -Specifically, the Prover possesses some integer xx so that X=gxX = g^x. +The Schnorr protocol provides a framework that enables the Prover to convince the Verifier that: for group elements gg and XX, +the Prover knows the power to raise gg to obtain XX. +Specifically, the Prover possesses some integer xx so that X=gxX = g^x. Cryptographic resources may use either multiplicative or additive notation for groups; we will use multiplicative notation. -Briefly, the element gg being combined with itself in multiplicative notation is g⋅g=g2g \cdot g = g^2, -while in additive notation it is g+g=2gg + g = 2g. -We assume that our group is of prime order pp, and is sufficiently large to satisfy the discrete logarithm assumption.

    The Schnorr protocol proceeds as follows:

    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t⟶T\stackrel{T}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TXcg^z \stackrel{?}{=} T X^c

    Chaum-Pedersen protocol​

    A tuple of group elements (U,V,W)(U,V,W) is a DH-triple if and only if there exists some x∈Zpx \in \mathbb{Z}_p so that V=gxV = g^x and W=UxW = U^x. -The Chaum-Pedersen protocol provides a framework that enables a Prover to convince a Verifier that she possesses such a xx for a claimed DH-triple (U,V,W)(U,V,W). -The Chaum-Pedersen protocol proceeds as follows:

    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t, S:=UtS := U^t⟶T,S\stackrel{T,S}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TVcg^z \stackrel{?}{=} T V^c and Uz=?SWcU^z \stackrel{?}{=} SW^c

    Hash Functions​

    Cryptographic hash functions serve as the backbone to the Fiat-Shamir heuristic. -A hash function, Hash\mathsf{Hash}, is a special function that takes in an arbitrary binary string and outputs a binary string of a predetermined fixed length. +Briefly, the element gg being combined with itself in multiplicative notation is g⋅g=g2g \cdot g = g^2, +while in additive notation it is g+g=2gg + g = 2g. +We assume that our group is of prime order pp, and is sufficiently large to satisfy the discrete logarithm assumption.

    +

    The Schnorr protocol proceeds as follows:

    +
    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t⟶T\stackrel{T}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TXcg^z \stackrel{?}{=} T X^c
    +

    Chaum-Pedersen protocol​

    +

    A tuple of group elements (U,V,W)(U,V,W) is a DH-triple if and only if there exists some x∈Zpx \in \mathbb{Z}_p so that V=gxV = g^x and W=UxW = U^x. +The Chaum-Pedersen protocol provides a framework that enables a Prover to convince a Verifier that she possesses such a xx for a claimed DH-triple (U,V,W)(U,V,W). +The Chaum-Pedersen protocol proceeds as follows:

    +
    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t, S:=UtS := U^t⟶T,S\stackrel{T,S}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TVcg^z \stackrel{?}{=} T V^c and Uz=?SWcU^z \stackrel{?}{=} SW^c
    +

    Hash Functions​

    +

    Cryptographic hash functions serve as the backbone to the Fiat-Shamir heuristic. +A hash function, Hash\mathsf{Hash}, is a special function that takes in an arbitrary binary string and outputs a binary string of a predetermined fixed length. Specifically, -Hash:{0,1}∗→{0,1}n\mathsf{Hash} : \{0,1\}^* \rightarrow \{0,1\}^n.

    The security of cryptographic hash functions will rely on certain tasks being computationally infeasible. -A task is computationally infeasible provided that there is no deterministic algorithm that can conclude the task in polynomial-time.

    A cryptographic hash function satisfies the following properties:

    • Succinct: The hash function should be easy to compute; the hash Hash(b)\mathsf{Hash}({\bf{b}}) can be efficiently computed for any binary string b{\bf{b}}.
    • Preimage Resistance: It should be computationally infeasible to work backwards given the output of a hash function. Let y{\bf{y}} be a binary string of length nn. -It should be 'impossible' to find some binary string x{\bf{x}} so that y=Hash(x){\bf{y}} = \mathsf{Hash}({\bf{x}}).
    • Collision Resistance: It should be difficult to find two strings that hash to the same value. -It should be computationally infeasible to find two binary strings x1{\bf{x}_1} and x2{\bf{x}_2} so that Hash(x1)=Hash(x2).\mathsf{Hash}({\bf{x}_1}) = \mathsf{Hash}({\bf{x}_2}).

    A related class of functions is one-way functions. +Hash:{0,1}∗→{0,1}n\mathsf{Hash} : \{0,1\}^* \rightarrow \{0,1\}^n.

    +

    The security of cryptographic hash functions will rely on certain tasks being computationally infeasible. +A task is computationally infeasible provided that there is no deterministic algorithm that can conclude the task in polynomial-time.

    +

    A cryptographic hash function satisfies the following properties:

    +
      +
    • Succinct: The hash function should be easy to compute; the hash Hash(b)\mathsf{Hash}({\bf{b}}) can be efficiently computed for any binary string b{\bf{b}}.
      • +
    • Preimage Resistance: It should be computationally infeasible to work backwards given the output of a hash function. Let y{\bf{y}} be a binary string of length nn. +It should be 'impossible' to find some binary string x{\bf{x}} so that y=Hash(x){\bf{y}} = \mathsf{Hash}({\bf{x}}).
      • +
    • Collision Resistance: It should be difficult to find two strings that hash to the same value. +It should be computationally infeasible to find two binary strings x1{\bf{x}_1} and x2{\bf{x}_2} so that Hash(x1)=Hash(x2).\mathsf{Hash}({\bf{x}_1}) = \mathsf{Hash}({\bf{x}_2}).
      • +
    +

    A related class of functions is one-way functions. A one-way function satisfies the first two conditions of a cryptographic hash function (succinct and preimage resistance). All cryptographic hash functions are a one-way functions. However, one-way functions do not necessarily satisfy collision-resistance. We will simply refer to cryptographic hash functions as hash functions for the rest of this blog. Commonly used hash functions include SHA-256 5, -Keccak 6, and Poseidon 7.

    The Fiat-Shamir heuristic​

    The Fiat-Shamir heuristic is the technique used to convert an interactive protocol to a noninteractive protocol. +Keccak 6, and Poseidon 7.

    +

    The Fiat-Shamir heuristic​

    +

    The Fiat-Shamir heuristic is the technique used to convert an interactive protocol to a noninteractive protocol. This is done by replacing each of the Verifier's messages with a hashed value. -Specifically, the Prover generates the Verifier's message by evaluating the hash function Hash\mathsf{Hash} +Specifically, the Prover generates the Verifier's message by evaluating the hash function Hash\mathsf{Hash} with the concatenation of all public values that appear in the protocol thus far. -If m0,…,mtm_0, \dots, m_t denote the public values in the protocol thus far, -then the Verifier's message is computed as mt+1:=Hash(m0∣∣⋯∣∣mt)m_{t+1} := \mathsf{Hash}(m_0|| \cdots || m_t).

    Since Hash\mathsf{Hash} can be efficiently computed, and the messages m0,…,mtm_0, \dots, m_t are public, then any verifying party can compute mt+1m_{t+1}. -Critically, since Hash\mathsf{Hash} is preimage resistant and collision resistant, -the Prover cannot manipulate her choices of the messages m0,…,mtm_0,\dots, m_t to influence the message mt+1m_{t+1}. -Hence, verifying parties can trust that mt+1m_{t+1} is sufficiently random with respect to the preceding messages.

    There are two variants of the Fiat-Shamir heuristic: weak and strong. +If m0,…,mtm_0, \dots, m_t denote the public values in the protocol thus far, +then the Verifier's message is computed as mt+1:=Hash(m0∣∣⋯∣∣mt)m_{t+1} := \mathsf{Hash}(m_0|| \cdots || m_t).

    +

    Since Hash\mathsf{Hash} can be efficiently computed, and the messages m0,…,mtm_0, \dots, m_t are public, then any verifying party can compute mt+1m_{t+1}. +Critically, since Hash\mathsf{Hash} is preimage resistant and collision resistant, +the Prover cannot manipulate her choices of the messages m0,…,mtm_0,\dots, m_t to influence the message mt+1m_{t+1}. +Hence, verifying parties can trust that mt+1m_{t+1} is sufficiently random with respect to the preceding messages.

    +

    There are two variants of the Fiat-Shamir heuristic: weak and strong. The weak variant uses all of the publicly traded messages in computing the Verifier's messages but does not include the public parameters. However, in the strong variant all of the publicly traded messages and public parameters are used to compute the Verifier's messages. -We will provide a discussion on issues that can arise from using the weak Fiat-Shamir heuristic.

    Schnorr Protocol with the strong Fiat-Shamir​

    When the strong Fiat-Shamir heuristic is applied to the Schnorr protocol, the message c=Hash(g∣∣X∣∣T)c = \mathsf{Hash}(g||X||T). -This choice of cc provides security since it should be computationally infeasible to find collisions for the outputs of Hash\mathsf{Hash}. -Thus, cc fixes the group elements gg, XX and TT.

    The elements that would be omitted in the hash by applying weak Fiat-Shamir heuristic are gg and XX.

    Chaum-Pedersen Protocol with the strong Fiat-Shamir​

    The message c=Hash(g∣∣U∣∣V∣∣W∣∣T∣∣S)c = Hash(g||U||V||W||T||S) when the Prover applies the strong Fiat-Shamir heuristic to the Chaum-Pedersen protocol. -The properties of Hash\mathsf{Hash} fixes the generator gg and the Prover's statement (U,V,W)(U,V,W).

    Improper use of the Fiat-Shamir heuristic​

    The Fiat-Shamir heuristic appears to be a fairly straightforward technique to implement. +We will provide a discussion on issues that can arise from using the weak Fiat-Shamir heuristic.

    +

    Schnorr Protocol with the strong Fiat-Shamir​

    +

    When the strong Fiat-Shamir heuristic is applied to the Schnorr protocol, the message c=Hash(g∣∣X∣∣T)c = \mathsf{Hash}(g||X||T). +This choice of cc provides security since it should be computationally infeasible to find collisions for the outputs of Hash\mathsf{Hash}. +Thus, cc fixes the group elements gg, XX and TT.

    +

    The elements that would be omitted in the hash by applying weak Fiat-Shamir heuristic are gg and XX.

    +

    Chaum-Pedersen Protocol with the strong Fiat-Shamir​

    +

    The message c=Hash(g∣∣U∣∣V∣∣W∣∣T∣∣S)c = Hash(g||U||V||W||T||S) when the Prover applies the strong Fiat-Shamir heuristic to the Chaum-Pedersen protocol. +The properties of Hash\mathsf{Hash} fixes the generator gg and the Prover's statement (U,V,W)(U,V,W).

    +

    Improper use of the Fiat-Shamir heuristic​

    +

    The Fiat-Shamir heuristic appears to be a fairly straightforward technique to implement. However, a subtle but serious issue that can occur in the application of the Fiat-Shamir heuristic has been a point of discussion for the past few years. The issue concerns what messages are included in the hash. -In particular, are the public parameters used to compute the hash value?

    Bernhard et al. 8 provide a discussion of Fiat-Shamir heuristic restricted to Σ\Sigma-protocols. +In particular, are the public parameters used to compute the hash value?

    +

    Bernhard et al. 8 provide a discussion of Fiat-Shamir heuristic restricted to Σ\Sigma-protocols. In particular, Bernhard et al. discuss the pitfalls of the weak Fiat-Shamir heuristic. Recall that the strong Fiat-Shamir heuristic requires that the public parameters are included in the calculations of the Verifier's messages while the weak version does not. The inclusion of the public parameters in the hash evaluations fixes these public values for the entire protocol. -This means that the Prover cannot retroactively change them.

    The issues with the differences in the variants of the Fiat-Shamir heuristics has persisted since Bernhard et al.'s paper. +This means that the Prover cannot retroactively change them.

    +

    The issues with the differences in the variants of the Fiat-Shamir heuristics has persisted since Bernhard et al.'s paper. In recent years, auditors from Trail of Bits and OpenZeppelin have released blogs (9, 10, 11, 12, 13) and papers (14, 15) -describing specific vulnerabilities in zero-knowledge papers and repositories associated with the use of the weak Fiat-Shamir heuristic.

    Trail of Bits coined the term FROZEN Heart to describe the use of weak Fiat-Shamir heuristic. +describing specific vulnerabilities in zero-knowledge papers and repositories associated with the use of the weak Fiat-Shamir heuristic.

    +

    Trail of Bits coined the term FROZEN Heart to describe the use of weak Fiat-Shamir heuristic. Frozen comes from the phrase "FoRging Of ZEro kNowledge proofs", -and Fiat-Shamir is the "heart" of transforming an interactive protocol to noninteractive protocol.

    Now, we examine how weak Fiat-Shamir affects the Schnorr protocol and Chaum-Pedersen protocol.

    Schnorr protocol with the weak Fiat-Shamir heuristic​

    For Schnorr, we will examine two variants: -the first where we only include the Prover's claim XX but not the public parameter gg, and -the second where we include the public parameter gg but not the Prover's claim XX.

    Since we omit the generator g∈Gg \in \mathbb{G} from the computation for the message cc in our first approach, -then c=Hash(X∣∣T)c = \mathsf{Hash}(X||T).

    Now, a malicious Prover can complete the transcript for the Schnorr protocol by selecting any z∈RZpz \in_R \mathbb{Z}_p. -Since, gg is not fixed as it was not included in the computation of cc. -But, the malicious Prover needs the transcript (T,c,z)(T,c,z) to satisfy gz=TXcg^z = TX^c. -Hence, the malicious Prover can compute the generator g=(TXc)z−1.g = (TX^c)^{z^{-1}}.

    In our second approach, we omit the group element X∈GX \in \mathbb{G} from the computation for the challenge cc. -That is, c=Hash(g∣∣T)c = \mathsf{Hash}(g||T).

    As with the previous example, the malicious Prover takes a Schnorr transcript (T,c,z)(T,c,z) where z∈RZpz \in_R \mathbb{Z}_p. -It is necessary for the malicious Prover to find a value XX so that gz=TXcg^z = TX^c. -This can be acheived by computing X=(gzT−1)c−1X = (g^z T^{-1})^{c^{-1}}.

    Chaum-Pedersen protocol with the Fiat-Shamir heuristic​

    The Verifier's message c=Hash(T,S)c = Hash(T,S) when weak Fiat-Shamir heuristic is applied. -The Prover's triple (U,V,W)(U,V,W) and the generator gg are not fixed by cc. -As such, a malicious Prover can generate values for U,V,WU,V,W, and gg that satisfy the Verifier's identity checks. -In the case of a malicious Prover, TT and SS are randomly group elements instead of being computed using a value tt that the Prover selected. -This means a malicious Prover must randomly select the value zz as well.

    Given the values that have been fixed so far, each of the Verifier's identities consists of two unknowns. +and Fiat-Shamir is the "heart" of transforming an interactive protocol to noninteractive protocol.

    +

    Now, we examine how weak Fiat-Shamir affects the Schnorr protocol and Chaum-Pedersen protocol.

    +

    Schnorr protocol with the weak Fiat-Shamir heuristic​

    +

    For Schnorr, we will examine two variants: +the first where we only include the Prover's claim XX but not the public parameter gg, and +the second where we include the public parameter gg but not the Prover's claim XX.

    +

    Since we omit the generator g∈Gg \in \mathbb{G} from the computation for the message cc in our first approach, +then c=Hash(X∣∣T)c = \mathsf{Hash}(X||T).

    +

    Now, a malicious Prover can complete the transcript for the Schnorr protocol by selecting any z∈RZpz \in_R \mathbb{Z}_p. +Since, gg is not fixed as it was not included in the computation of cc. +But, the malicious Prover needs the transcript (T,c,z)(T,c,z) to satisfy gz=TXcg^z = TX^c. +Hence, the malicious Prover can compute the generator g=(TXc)z−1.g = (TX^c)^{z^{-1}}.

    +

    In our second approach, we omit the group element X∈GX \in \mathbb{G} from the computation for the challenge cc. +That is, c=Hash(g∣∣T)c = \mathsf{Hash}(g||T).

    +

    As with the previous example, the malicious Prover takes a Schnorr transcript (T,c,z)(T,c,z) where z∈RZpz \in_R \mathbb{Z}_p. +It is necessary for the malicious Prover to find a value XX so that gz=TXcg^z = TX^c. +This can be acheived by computing X=(gzT−1)c−1X = (g^z T^{-1})^{c^{-1}}.

    +

    Chaum-Pedersen protocol with the Fiat-Shamir heuristic​

    +

    The Verifier's message c=Hash(T,S)c = Hash(T,S) when weak Fiat-Shamir heuristic is applied. +The Prover's triple (U,V,W)(U,V,W) and the generator gg are not fixed by cc. +As such, a malicious Prover can generate values for U,V,WU,V,W, and gg that satisfy the Verifier's identity checks. +In the case of a malicious Prover, TT and SS are randomly group elements instead of being computed using a value tt that the Prover selected. +This means a malicious Prover must randomly select the value zz as well.

    +

    Given the values that have been fixed so far, each of the Verifier's identities consists of two unknowns. Hence, it is necessary to select one of these unknowns from each identity so that a malicious Prover can compute the last value. -For instances, suppose that the malicious Prover randomly selects VV and WW. -The malicious Prover can compute g=(TVc)1/zg = (T V^c)^{1/z} and V=(SWc)1/zV = (SW^c)^{1/z}. -Thus, the malicious Prover has a claimed statement (U,V,W)(U,V,W) for generator gg that passes the Verifier's identities using weak Fiat-Shamir heuristic.

    The omission of any of the values U,V,W,U,V,W, and gg in the computation of cc allows a malicious Prover to forge a proof.

    Conclusion​

    The Fiat-Shamir heuristic is an essential technique to convert an interactive protocol to a variant that does not require communication. -Additionally, careful application of this technique is necessary to maintain the integrity of the system.

    References​

    ]]>     +For instances, suppose that the malicious Prover randomly selects VV and WW. +The malicious Prover can compute g=(TVc)1/zg = (T V^c)^{1/z} and V=(SWc)1/zV = (SW^c)^{1/z}. +Thus, the malicious Prover has a claimed statement (U,V,W)(U,V,W) for generator gg that passes the Verifier's identities using weak Fiat-Shamir heuristic.

    +

    The omission of any of the values U,V,W,U,V,W, and gg in the computation of cc allows a malicious Prover to forge a proof.

    +

    Conclusion​

    +

    The Fiat-Shamir heuristic is an essential technique to convert an interactive protocol to a variant that does not require communication. +Additionally, careful application of this technique is necessary to maintain the integrity of the system.

    +

    References​

    +]]>     <![CDATA[zkVM Testing Report: Evaluating Zero-Knowledge Virtual Machines for Nescience]]> https://vac.dev/rlog/zkVM-testing https://vac.dev/rlog/zkVM-testing Thu, 26 Sep 2024 12:00:00 GMT - - Introduction

    Following our initial exploration of zkVMs in our previous blog post [1], + + Following our initial exploration of zkVMs in our previous blog post [1], we have conducted a series of tests to identify the most suitable zkVM for the Nescience architecture [2]. This post outlines the testing process, results, and conclusions. Additionally, the full test suite and scripts can be found on our GitHub page [3], allowing others to replicate the results or explore the candidates further. Please note that we chose not to use hardware acceleration in our benchmarks, as our project is aimed at a broad audience. -Particularly, we cannot assume AVX512 support by default, as it is typically available only in high-end CPUs.

    We've shortlisted the following zkVMs for testing:

    • SP1 [4]
    • RISC0 [5]
    • Nexus [6]
    • ZkMIPS [7]
    • ZkWASM [8]
    • Valida [9]

    Why these candidates?

    When narrowing down the zkVMs, we focused on key factors:

    • True zero-knowledge functionality: The zkVMs had to demonstrate or be close to demonstrating the ability to generate and verify zero-knowledge proofs.
    • Performance baselines: We sought zkVMs with solid benchmarks in performance, particularly in speed and efficiency.
    • Specific functionalities: For Nescience, functionalities like lookup tables, precompiles, and recursive capabilities are critical.

    We need a zkVM that supports these to enable robust project development.

    Preliminary information on the candidates

    1. SP1 is a performant, open-source zkVM that verifies the execution of arbitrary Rust (or any LLVM-compiled language) programs. +Particularly, we cannot assume AVX512 support by default, as it is typically available only in high-end CPUs.

      +

      We've shortlisted the following zkVMs for testing:

      +
        +
      • SP1 [4]
        • +
      • RISC0 [5]
        • +
      • Nexus [6]
        • +
      • ZkMIPS [7]
        • +
      • ZkWASM [8]
        • +
      • Valida [9]
        • +
      +

      Why these candidates?

      +

      When narrowing down the zkVMs, we focused on key factors:

      +
        +
      • True zero-knowledge functionality: The zkVMs had to demonstrate or be close to demonstrating the ability to generate and verify zero-knowledge proofs.
        • +
      • Performance baselines: We sought zkVMs with solid benchmarks in performance, particularly in speed and efficiency.
        • +
      • Specific functionalities: For Nescience, functionalities like lookup tables, precompiles, and recursive capabilities are critical.
        • +
      +

      We need a zkVM that supports these to enable robust project development.

      +

      Preliminary information on the candidates

      +
        +
      1. +

        SP1 is a performant, open-source zkVM that verifies the execution of arbitrary Rust (or any LLVM-compiled language) programs. SP1 utilizes Plonky3, enabling recursive proofs and supporting a wide range of cryptographic algorithms, including ECC-based ones like Groth16. -While it supports aggregation, it appears not to support zero knowledge in a conventional manner.

      2. RISC0 zkVM allows one to prove the correct execution of arbitrary Rust code. Built on a RISC-V architecture, it is inherently adaptable +While it supports aggregation, it appears not to support zero knowledge in a conventional manner.

        +
        3. +
      3. +

        RISC0 zkVM allows one to prove the correct execution of arbitrary Rust code. Built on a RISC-V architecture, it is inherently adaptable for implementing standard cryptographic hash functions such as SHA-256 and ECDSA. RISC0 employs STARKs, providing a security level of 98 bits. -It supports multiple programming languages, including C and Rust, thanks to its compatibility with LLVM and WASM.

      4. Nexus is a modular, extensible, open-source, highly parallelized, prover-optimized, and contributor-friendly zkVM written in Rust. +It supports multiple programming languages, including C and Rust, thanks to its compatibility with LLVM and WASM.

        +
        5. +
      5. +

        Nexus is a modular, extensible, open-source, highly parallelized, prover-optimized, and contributor-friendly zkVM written in Rust. It focuses on performance and security, using the Nova folding scheme, which is particularly effective for recursive proofs. -Nexus also supports precompiles and targeted compilation, and besides Rust, it offers C++ support.

      6. ZkMIPS is a general verifiable computing infrastructure based on Plonky2 and the MIPS microarchitecture, aiming to empower Ethereum -as a global settlement layer. It can run arbitrary Rust code as well. Notably, zkMIPS is the only zkVM in this list that utilizes the MIPS opcode set.

      7. ZkWASM adheres to and supports the unmodified standard WASM bytecode specification. Since Rust code can be compiled to WASM bytecode, -one could theoretically run any Rust code on a zkWASM machine, providing flexibility and broad language support.

      8. Valida is a STARK-based virtual machine aiming to improve upon the state of the art in several categories:

        • Code reuse: The VM has a RISC-inspired instruction set, simplifying the targeting of conventional programming languages. +Nexus also supports precompiles and targeted compilation, and besides Rust, it offers C++ support.

          +
          • +
        • +

          ZkMIPS is a general verifiable computing infrastructure based on Plonky2 and the MIPS microarchitecture, aiming to empower Ethereum +as a global settlement layer. It can run arbitrary Rust code as well. Notably, zkMIPS is the only zkVM in this list that utilizes the MIPS opcode set.

          +
          • +
        • +

          ZkWASM adheres to and supports the unmodified standard WASM bytecode specification. Since Rust code can be compiled to WASM bytecode, +one could theoretically run any Rust code on a zkWASM machine, providing flexibility and broad language support.

          +
          • +
        • +

          Valida is a STARK-based virtual machine aiming to improve upon the state of the art in several categories:

          +
            +
          • Code reuse: The VM has a RISC-inspired instruction set, simplifying the targeting of conventional programming languages. A backend compiler is being developed to compile LLVM IR to the Valida ISA, enabling the proving of programs written in Rust, -Go, C++, and others with minimal to no changes in source code.
          • Prover performance: Engineered to maximize prover performance, Valida is compatible with a 31-bit field, restricted to degree 3 constraints, +Go, C++, and others with minimal to no changes in source code.
            • +
          • Prover performance: Engineered to maximize prover performance, Valida is compatible with a 31-bit field, restricted to degree 3 constraints, and features minimal instruction decoding. It operates directly on memory without general-purpose registers or a dedicated stack, -utilizing newer lookup arguments to reduce trace overhead involved in cross-chip communication.
          • Extensibility: Designed to be customizable, Valida can easily be extended to include an arbitrary number of user-defined instructions. -Procedural macros are used to construct the desired machine at compile time, avoiding any runtime penalties.

      Valida appears to be in the early stages of development but already showcases respectable performance metrics.

      Testing plan

      To thoroughly evaluate each zkVM, we devised a two-stage testing process:

      • Stage 1: Arithmetic operations

        The first phase focused on evaluating the zkVMs’ ability to handle basic arithmetic operations: addition, subtraction, multiplication, +utilizing newer lookup arguments to reduce trace overhead involved in cross-chip communication.

        • +
      • Extensibility: Designed to be customizable, Valida can easily be extended to include an arbitrary number of user-defined instructions. +Procedural macros are used to construct the desired machine at compile time, avoiding any runtime penalties.
        • +
      +
      2. +
    +

    Valida appears to be in the early stages of development but already showcases respectable performance metrics.

    +

    Testing plan

    +

    To thoroughly evaluate each zkVM, we devised a two-stage testing process:

    +
      +
    • +

      Stage 1: Arithmetic operations

      +

      The first phase focused on evaluating the zkVMs’ ability to handle basic arithmetic operations: addition, subtraction, multiplication, division, modulus division, and square root calculations. We designed the test around heptagonal numbers, which required zkVMs to process multiple arithmetic operations simultaneously. By using this method, we could measure efficiency and speed in handling complex mathematical calculations – -a crucial element for zkVM performance.

    • Stage 2: Memory consumption

      For the second phase, we evaluated each zkVM’s ability to manage memory under heavy loads. We tested several data structures, including lists, -hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent tests for the following operations:

      • Insert: How quickly can the zkVM add data to structures?
      • Delete: Does the zkVM handle memory release effectively?
      • Append: Can the zkVM efficiently grow data structures?
      • Search: How fast and efficient is the zkVM when retrieving stored data?

    The purpose of this stage was to identify any memory bottlenecks and to determine whether a zkVM could manage high-intensity tasks efficiently, -something vital for the Nescience project’s complex, data-heavy processes.

    Machine specifications

    The tests were conducted on the following hardware configuration:

    • CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
    • RAM: 600GiB DDR4 3200MHz ECC RAM, distributed across 16 DIMMs
    • Host OS: Proxmox 8.3
    • Hypervisor: KVM
    • Network layer: Open vSwitch
    • Machine model: Supermicro AS-2024US-TRT

    Results

    1. SP1​

    SP1 does not provide zero-knowledge capability in its proofs but delivers respectable performance, though slightly behind its main competitor. +a crucial element for zkVM performance.

    + +
  • +

    Stage 2: Memory consumption

    +

    For the second phase, we evaluated each zkVM’s ability to manage memory under heavy loads. We tested several data structures, including lists, +hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent tests for the following operations:

    +
      +
    • Insert: How quickly can the zkVM add data to structures?
      • +
    • Delete: Does the zkVM handle memory release effectively?
      • +
    • Append: Can the zkVM efficiently grow data structures?
      • +
    • Search: How fast and efficient is the zkVM when retrieving stored data?
      • +
    +
    • + +

    The purpose of this stage was to identify any memory bottlenecks and to determine whether a zkVM could manage high-intensity tasks efficiently, +something vital for the Nescience project’s complex, data-heavy processes.

    +

    Machine specifications

    +

    The tests were conducted on the following hardware configuration:

    +
      +
    • CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
      • +
    • RAM: 600GiB DDR4 3200MHz ECC RAM, distributed across 16 DIMMs
      • +
    • Host OS: Proxmox 8.3
      • +
    • Hypervisor: KVM
      • +
    • Network layer: Open vSwitch
      • +
    • Machine model: Supermicro AS-2024US-TRT
      • +
    +

    Results

    +

    1. SP1​

    +

    SP1 does not provide zero-knowledge capability in its proofs but delivers respectable performance, though slightly behind its main competitor. Memory leaks were minimal, staying below the 700 KB threshold. Interestingly, SP1 consumed more RAM during the basic arithmetic test than in memory allocation tests, showcasing the team's effective handling of memory under load. In the basic test, -allocations were primarily in the 9-16 B, 33-64 B, and 65-128 B ranges. For memory allocations, most fell into the 129-256 B range.

    • Stage 1: Hept 100 test
      • Proof size: 3.108 MB
      • Proof time: 16.95 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 3.17 MB
      • Proof time: 20.85 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    2. RISC0​

    RISC0 stands out with exceptional performance in proof size and generation time, ranking among the best +allocations were primarily in the 9-16 B, 33-64 B, and 65-128 B ranges. For memory allocations, most fell into the 129-256 B range.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 3.108 MB
        • +
      • Proof time: 16.95 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 3.17 MB
        • +
      • Proof time: 20.85 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    2. RISC0​

    +

    RISC0 stands out with exceptional performance in proof size and generation time, ranking among the best (with the exception of Valida and zkWASM's basic test). It also handles memory well, with minor leaks under 0.5 MB -and controlled RAM consumption staying below 2.2 GB. RISC0's memory allocations were consistent, primarily in the 17-32 B and 33-64 B ranges.

    • Stage 1: Hept 100 test
      • Proof size: 217.4 KB
      • Proof time: 9.73 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 217.4 KB
      • Proof time: 16.63 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    Based on these results, RISC0 is a solid candidate for Nescience.

    3. Nexus​

    Nexus' performance offers interesting insights into folding scheme-based zkVMs. Surprisingly, proof sizes remained consistent +and controlled RAM consumption staying below 2.2 GB. RISC0's memory allocations were consistent, primarily in the 17-32 B and 33-64 B ranges.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 217.4 KB
        • +
      • Proof time: 9.73 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 217.4 KB
        • +
      • Proof time: 16.63 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +

    Based on these results, RISC0 is a solid candidate for Nescience.

    +
    +

    3. Nexus​

    +

    Nexus' performance offers interesting insights into folding scheme-based zkVMs. Surprisingly, proof sizes remained consistent regardless of workload, with no significant memory leaks (under 700 KB). However, while RAM consumption increased slightly with higher -workloads (up to 1.2 GB), Nexus performed poorly during memory allocation tests, making it unsuitable for our use case.

    • Allocation details:

      • Basic test: Most allocations concentrated in 65-128 B
      • Memory-heavy test: Allocations in the 129-256 B range

    • Stage 1: Hept 100 test

      • Proof size: 46 MB
      • Proof time: 12.06 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 46 MB
      • Proof time: 56 minutes
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    4. ZkMIPS​

    ZkMIPS presents an intriguing case. While it shows good results in terms of proof size and generation time during the basic test, +workloads (up to 1.2 GB), Nexus performed poorly during memory allocation tests, making it unsuitable for our use case.

    +
      +
    • +

      Allocation details:

      +
        +
      • Basic test: Most allocations concentrated in 65-128 B
        • +
      • Memory-heavy test: Allocations in the 129-256 B range
        • +
      +
      • +
    • +

      Stage 1: Hept 100 test

      +
        +
      • Proof size: 46 MB
        • +
      • Proof time: 12.06 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 46 MB
        • +
      • Proof time: 56 minutes
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    4. ZkMIPS​

    +

    ZkMIPS presents an intriguing case. While it shows good results in terms of proof size and generation time during the basic test, these come at the cost of significant RAM usage and memory leaks. The memory allocation test revealed a concerning 6.7 GB memory leak, with 0.5 GB leaked during the basic test. Despite this, RAM consumption (while high at 17+ GB) remains stable under higher workloads. -Allocation sizes are spread across several ranges, with notable concentrations in the 17-32 B, 65-128 B, and 257-512 B slots.

    • Stage 1: Hept 100 test
      • Proof size: 4.3 MB
      • Proof time: 9.32 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 4.898 MB
      • Proof time: 42.57 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    This zkVM provides mixed results with strong proof generation but concerning memory management issues.

    5. ZkWASM​

    ZkWASM, unfortunately, performed poorly in both stages regarding proof size and generation time. RAM consumption was particularly high, +Allocation sizes are spread across several ranges, with notable concentrations in the 17-32 B, 65-128 B, and 257-512 B slots.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 4.3 MB
        • +
      • Proof time: 9.32 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 4.898 MB
        • +
      • Proof time: 42.57 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +

    This zkVM provides mixed results with strong proof generation but concerning memory management issues.

    +
    +

    5. ZkWASM​

    +

    ZkWASM, unfortunately, performed poorly in both stages regarding proof size and generation time. RAM consumption was particularly high, exceeding 7 GB in the basic test, and an astounding 57 GB during memory allocation tests. Despite its impressive memory usage, the proof sizes were relatively large at 18 KB and 334 KB respectively. Allocation sizes were mainly concentrated in the 33-64 B range, -with neighboring slots contributing small but notable amounts.

    • Stage 1: Hept 100 test
      • Proof size: 18 KB
      • Proof time: 42.7 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 334 KB
      • Proof time: 323 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    6. Valida​

    Valida delivered impressive results in proof generation speed and size, with a proof size of 280 KB and a proof time of < 1 second. +with neighboring slots contributing small but notable amounts.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 18 KB
        • +
      • Proof time: 42.7 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 334 KB
        • +
      • Proof time: 323 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    6. Valida​

    +

    Valida delivered impressive results in proof generation speed and size, with a proof size of 280 KB and a proof time of < 1 second. However, profiling was not possible due to Valida's limited Rust support. Valida currently compiles Rust using the LLVM backend, transpiling LLVM IR to leverage its C/C++ implementation, which leads to errors when handling Rust-specific data structures or dependencies. As a result, complex memory interactions couldn't be tested, and using Valida with Rust code is currently not advisable. -A GitHub issue has been opened to address this.

    Summary table​

    Stage 1​

    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP116.95 s3.108 MB2.1 GB656.8 KB
    RISC09.73 s217.4 KB1.9 GB470.5 KB
    Nexus12.06 s46 MB9.7 MB646.5 KB
    ZkMIPS9.32 s4.3 MB17.3 GB453.8 MB
    ZkWASM42.7 s18 KB8.2 GB259.4 KB
    Valida< 1 s280 KBN/AN/A

    Stage 2​

    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP120.85 s3.17 MB1.9 GB616 KB
    RISC016.63 s217.4 KB2.3 GB485.3 KB
    Nexus56 m46 MB1.9 GB616 KB
    ZkMIPS42.57 s4.898 MB18.9 GB6.9 GB
    ZkWASM323 s334 KB58.8 GB259.4 KB
    ValidaN/AN/AN/AN/A

    Summary

    After an extensive evaluation of six zkVM candidates for the Nescience project, RISC0 emerged as the top choice. +A GitHub issue has been opened to address this.

    +
    +

    Summary table​

    +

    Stage 1​

    +
    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP116.95 s3.108 MB2.1 GB656.8 KB
    RISC09.73 s217.4 KB1.9 GB470.5 KB
    Nexus12.06 s46 MB9.7 MB646.5 KB
    ZkMIPS9.32 s4.3 MB17.3 GB453.8 MB
    ZkWASM42.7 s18 KB8.2 GB259.4 KB
    Valida< 1 s280 KBN/AN/A
    +
    +

    Stage 2​

    +
    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP120.85 s3.17 MB1.9 GB616 KB
    RISC016.63 s217.4 KB2.3 GB485.3 KB
    Nexus56 m46 MB1.9 GB616 KB
    ZkMIPS42.57 s4.898 MB18.9 GB6.9 GB
    ZkWASM323 s334 KB58.8 GB259.4 KB
    ValidaN/AN/AN/AN/A
    +
    +

    Summary

    +

    After an extensive evaluation of six zkVM candidates for the Nescience project, RISC0 emerged as the top choice. It excels in both proof generation time and size while maintaining a reasonable memory footprint. With strong zero-knowledge proof capabilities and support for multiple programming languages, it aligns well with our project's needs for privacy, -performance, and flexibility. Its overall balance between performance and efficiency makes it the most viable zkVM at this stage.

    Valida, while promising with its potential for high prover performance, is still in early development and suffers from Rust integration issues. +performance, and flexibility. Its overall balance between performance and efficiency makes it the most viable zkVM at this stage.

    +

    Valida, while promising with its potential for high prover performance, is still in early development and suffers from Rust integration issues. The current LLVM IR transpilation limitations mean it cannot handle complex memory interactions, disqualifying it for now. -However, once its development matures, Valida could become a strong alternative, and we plan to revisit it as it evolves.

    SP1, though initially interesting, failed to meet the zero-knowledge proof requirement. Its performance in arithmetic operations was -respectable but insufficient to justify further consideration given its lack of ZK functionality – critical for our privacy-first objectives.

    Nexus demonstrated consistent proof sizes and manageable memory usage, but its lackluster performance during memory-intensive tasks and +However, once its development matures, Valida could become a strong alternative, and we plan to revisit it as it evolves.

    +

    SP1, though initially interesting, failed to meet the zero-knowledge proof requirement. Its performance in arithmetic operations was +respectable but insufficient to justify further consideration given its lack of ZK functionality – critical for our privacy-first objectives.

    +

    Nexus demonstrated consistent proof sizes and manageable memory usage, but its lackluster performance during memory-intensive tasks and its proof size (especially for larger workloads) disqualified it from being a top contender. While zkMIPS delivered solid proof times, -the memory issues were too significant to ignore, making it unsuitable.

    Finally, zkWASM exhibited the poorest results, struggling both in proof size and generation time. Despite its potential for WASM bytecode support, -the excessive RAM consumption (up to 57 GB in the memory test) rendered it impractical for Nescience’s use case.

    In conclusion, RISC0 is the best fit for Nescience at this stage, but Valida remains a future candidate as its development progresses.

    In the future, we plan to compare RISC0 and SP1 with CUDA acceleration. Ideally, by that time, more zkVMs will include similar acceleration capabilities, -enabling a fairer and more comprehensive comparison across platforms.

    We’d love to hear your thoughts on our zkVM testing process and results! Do you agree with our conclusions, or do you think we missed a promising zkVM? -We’re always open to feedback, insights, and suggestions from the community.

    Join the discussion and share your perspectives on +the memory issues were too significant to ignore, making it unsuitable.

    +

    Finally, zkWASM exhibited the poorest results, struggling both in proof size and generation time. Despite its potential for WASM bytecode support, +the excessive RAM consumption (up to 57 GB in the memory test) rendered it impractical for Nescience’s use case.

    +

    In conclusion, RISC0 is the best fit for Nescience at this stage, but Valida remains a future candidate as its development progresses.

    +

    In the future, we plan to compare RISC0 and SP1 with CUDA acceleration. Ideally, by that time, more zkVMs will include similar acceleration capabilities, +enabling a fairer and more comprehensive comparison across platforms.

    +

    We’d love to hear your thoughts on our zkVM testing process and results! Do you agree with our conclusions, or do you think we missed a promising zkVM? +We’re always open to feedback, insights, and suggestions from the community.

    +

    Join the discussion and share your perspectives on our forum or try out the -tests yourself through our GitHub page!

    References

    [1] Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? Retrieved from https://vac.dev/rlog/zkVM-explorations/

    [2] Nescience: A User-Centric State-Separation Architecture. Retrieved from https://vac.dev/rlog/Nescience-state-separation-architecture

    [3] Our GitHub Page for zkVM Testing. Retrieved from https://github.com/vacp2p/nescience-zkvm-testing

    [4] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    [5] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    [6] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    [7] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    [9] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    ]]> +tests yourself through our GitHub page!

    +

    References

    +

    [1] Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? Retrieved from https://vac.dev/rlog/zkVM-explorations/

    +

    [2] Nescience: A User-Centric State-Separation Architecture. Retrieved from https://vac.dev/rlog/Nescience-state-separation-architecture

    +

    [3] Our GitHub Page for zkVM Testing. Retrieved from https://github.com/vacp2p/nescience-zkvm-testing

    +

    [4] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    +

    [5] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    +

    [6] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    +

    [7] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    +

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    +

    [9] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    ]]>     <![CDATA[Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines?]]> https://vac.dev/rlog/zkVM-explorations https://vac.dev/rlog/zkVM-explorations Tue, 27 Aug 2024 12:00:00 GMT - - Introduction

    The blockchain space is rapidly evolving, and with it, new technologies are emerging that promise enhanced privacy, scalability, and security. + + The blockchain space is rapidly evolving, and with it, new technologies are emerging that promise enhanced privacy, scalability, and security. As decentralized systems grow in complexity and usage, the need for secure and private computation has never been greater. Zero-knowledge virtual machines (zkVMs) are one such innovation, allowing for computations to be proven correct without revealing the underlying data. ZkVMs have enormous implications for privacy-preserving applications, decentralized finance (DeFi), and other blockchain-based use cases. -However, as the term "zkVM" becomes more widely adopted, it is critical to distinguish between projects that truly satisfy the stringent requirements of a zkVM and those that do not.

    What is a zkVM?

    A zkVM is a virtual machine that combines the principles of cryptographic proof generation and privacy preservation with the computational model +However, as the term "zkVM" becomes more widely adopted, it is critical to distinguish between projects that truly satisfy the stringent requirements of a zkVM and those that do not.

    +

    What is a zkVM?

    +

    A zkVM is a virtual machine that combines the principles of cryptographic proof generation and privacy preservation with the computational model of traditional virtual machines. Essentially, a zkVM enables the execution of arbitrary programs while generating cryptographic proofs—specifically, zero-knowledge proofs (ZKPs)—that can verify the correctness of these computations without revealing any sensitive information. This ensures that computations can be trusted while protecting the privacy of the data involved. -The key characteristics of a zkVM include:

    • Proof generation: The ability to produce ZKPs that verify the correct execution of programs. There are several types of cryptographic techniques used in zkVMs to +The key characteristics of a zkVM include:

      +
        +
      • Proof generation: The ability to produce ZKPs that verify the correct execution of programs. There are several types of cryptographic techniques used in zkVMs to generate these proofs, such as zk-SNARKs, zk-STARKs, and recursive proofs. A zkVM’s ability to generate these proofs determines how effectively it can ensure the integrity of computations -in a privacy-preserving manner.
      • Privacy preservation: The system must maintain privacy, ensuring that only the proof is revealed, not the underlying computation or data. Privacy-preserving zkVMs allow users to maintain +in a privacy-preserving manner.
        • +
      • Privacy preservation: The system must maintain privacy, ensuring that only the proof is revealed, not the underlying computation or data. Privacy-preserving zkVMs allow users to maintain confidentiality without compromising the security or verifiability of their operations. However, not all zkVMs achieve the same level of privacy. Some may focus more on proof generation -and scalability while deprioritizing privacy features, which can limit their use in certain privacy-sensitive applications.
      • Scalability and performance: zkVMs should offer scalable and efficient computation, leveraging advanced cryptographic techniques like zk-SNARKs, zk-STARKs, or recursive proofs. -A zkVM's performance must also be measured in terms of latency (time to generate and verify a proof) and throughput (number of computations processed within a certain time frame).
      • Verifiable computation: The zkVM should be able to prove the execution of arbitrary programs in a secure and verifiable manner. Verifiable computation ensures that zkVMs can be deployed -across a wide range of applications, from DeFi to private data-sharing platforms and more.

      Why zkVMs matter

      The rise of zkVMs is a crucial development for the future of blockchain and decentralized technologies. As more systems require the ability to scale while maintaining privacy and trust, -zkVMs provide a powerful solution. They offer the potential to reshape the way decentralized applications (dapps) handle sensitive information, enabling them to be both efficient and private.

      It is essential to distinguish between projects that fully realize the potential of zkVMs and those that do not. In the remainder of this post, we evaluate several zkVM projects, analyzing -whether they satisfy the criteria for being classified as zkVMs based on our research.

      Our methodology

      We analyzed each project’s documentation, source code, and available benchmarks to determine whether they meet the definition of a zkVM. -Our criteria focus on the key capabilities of zkVMs—proof generation, privacy, scalability, and integration with existing systems.

      ZkVM project analysis

      1. [SP1]​

      • Overview: SP1 [1] is a developer-friendly zkVM designed to enable ZKP execution for LLVM-based languages like C, C++, Rust, and others. It supports a RISC-V-like instruction set architecture (ISA), -which makes it compatible with various programming languages compiled through LLVM.
      • Main focus: The main focus of SP1 is scalability, open-source contributions, and accessibility for developers. It prioritizes performance over privacy, -making it a good fit for environments where privacy isn't the primary concern.
      • Privacy: Not explicitly mentioned, making it less suitable for privacy-preserving applications.
      • Performance: SP1 has demonstrated up to 5.4x better performance than similar zkVMs like RISC0 for specific computations such as Fibonacci sequence generation.
      • Integration: SP1 is highly adaptable for rollups, light client verifiers, oracles, and even web2 projects like verifying the originality of images.
      • Conclusion: Yes, SP1 is a zkVM, but it does not prioritize zero-knowledge privacy, focusing more on scalability and performance.

      2. [Nexus]​

      • Overview: Nexus [2] is a highly modular zkVM designed to process up to a trillion CPU cycles per second. It relies on RISC-V instructions for computation, making it extensible and scalable. -However, it currently lacks full ZKP capabilities due to its use of Spartan proofs.
      • Main focus: Nexus focuses on high performance and scalability, aiming to create an efficient execution environment for computationally intensive tasks.
      • Privacy: Although zero-knowledge privacy isn't the primary feature of Nexus, the project hints at potential privacy enhancements in the future.
      • Performance: Nexus has a high theoretical throughput, but it has yet to demonstrate benchmarks on zero-knowledge privacy.
      • Integration: Nexus is a good fit for high-performance environments that do not necessarily require full privacy.
      • Conclusion: Yes, Nexus qualifies as a zkVM in terms of scalability and proof generation, but it does not yet achieve full zero-knowledge privacy.

      3. [RISC0]​

      • Overview: Risc0 [3] is a general-purpose zkVM with strong developer support. It allows for the execution of Rust and C code on a RISC-V virtual machine -and generates zk-SNARK and zk-STARK proofs for these computations.
      • Main focus: Risc0 is focused on ease of use for developers by abstracting away the complexities of circuit generation, making it accessible for a wide range of use cases.
      • Privacy: Full zero-knowledge privacy is supported via zk-SNARK and zk-STARK proofs, with Groth16 used for constant-size proof generation.
      • Performance: Risc0 offers strong benchmarks across different hardware setups, making it one of the most versatile zkVMs in terms of performance and scalability.
      • Integration: Risc0 integrates with several ecosystems, including Ethereum, and supports verifiable execution of Rust-based programs.
      • Conclusion: Yes, Risc0 qualifies as a zkVM, offering a balance of developer usability, scalability, and privacy.

      4. [Powdr]​

      • Overview: Powdr [4] is a toolkit for creating custom zkVMs. It allows developers to select from various front-end and back-end components to create zkVMs tailored to specific needs.
      • Main focus: Powdr is focused on providing a modular architecture for zkVM creation. It enables flexibility by allowing the combination of different ZK-proof backends like Halo2 or Valida.
      • Privacy: Powdr itself does not generate ZKPs, but it facilitates the creation of zkVMs that do.
      • Performance: The performance depends on the components chosen by the developer, as Powdr itself is more of a framework.
      • Integration: Powdr is highly customizable and can integrate with existing zkVM frameworks to extend their capabilities.
      • Conclusion: No, Powdr is not a zkVM itself, but it is a powerful tool for building customized zkVMs with different privacy and performance needs.

      5. [ZkMIPS]​

      • Overview: ZkMIPS [5] uses zk-STARKs to ensure privacy during computation, ensuring that private inputs are preserved while still proving correctness.
      • Performance: ZkMIPS is built for scalability, though explicit benchmarks are not widely published.
      • Integration: ZkMIPS can be integrated into systems that rely on MIPS architecture, making it versatile for legacy codebases that require privacy.
      • Conclusion: Yes, zkMIPS is a zkVM focused on scalability and privacy for MIPS-based architectures.

      6. [Valida]​

      • Overview: Valida [6] is a performance-oriented zkVM that generates proofs for programs using a custom ISA designed to optimize zkVM implementation. -It uses Plonky3 for its proof system.
      • Main focus: Valida is centered around optimizing prover performance and extensibility, making it a valuable tool for generating proofs efficiently.
      • Privacy: While Valida is focused on performance, it does not prioritize zero-knowledge privacy as much as other zkVMs.
      • Performance: Valida has benchmarks indicating its performance advantages in proving computations quickly, particularly through parallel processing.
      • Integration: Valida is specialized and may not integrate as seamlessly into general-purpose systems, as it is optimized for performance over broad applicability.
      • Conclusion: Yes, Valida qualifies as a zkVM based on proof generation, but its lack of focus on privacy makes it less suitable for privacy-first use cases.

      7. [Jolt]​

      • Overview: Jolt [7] is a zkVM built to optimize prover performance using a modified Hyrax polynomial commitment system. It relies on RISC-V instructions for computation -but falls short of full zero-knowledge capabilities.
      • Main focus: Jolt's main goal is to optimize the speed of proving program execution, making it suitable for high-performance applications where privacy isn't the primary concern.
      • Privacy: Jolt does not fully achieve zero-knowledge privacy due to the choice of polynomial commitment schemes.
      • Performance: Jolt offers strong performance, with benchmarks highlighting its ability to process proofs efficiently.
      • Integration: Jolt can be integrated with systems that prioritize speed over privacy, particularly where rapid proof generation is essential.
      • Conclusion: Yes, Jolt qualifies as a zkVM based on proof generation, though it does not provide full zero-knowledge privacy.

      8. [ZkWASM]​

      • Overview: ZkWASM [8] is a zkVM designed to execute WebAssembly (WASM) code in a privacy-preserving and scalable manner. It uses zk-SNARKs to prove the correctness of WASM -program execution while ensuring privacy.
      • Main focus: ZkWASM focuses on scalability and privacy for WebAssembly, making it ideal for dapps that require verifiable computation without compromising privacy.
      • Privacy: Full zero-knowledge privacy is provided through zk-SNARKs, ensuring that the execution of WASM programs remains confidential.
      • Performance: ZkWASM is optimized for running WASM programs efficiently, with offchain computation and onchain verification to enhance performance.
      • Integration: ZkWASM is ideal for dapps, particularly those that use WebAssembly and require verifiable execution.
      • Conclusion: Yes, zkWASM qualifies as a zkVM, providing strong privacy, scalability, and verifiable execution for WebAssembly code.

      9. [Aleo]​

      • Overview: Aleo's [9] snarkVM converts code into Aleo instructions, which are then compiled into bytecode executable on its zkVM. Aleo emphasizes building private, scalable dapps.
      • Main focus: Aleo prioritizes privacy and scalability for dapps, providing a robust framework for developers building private dapps.
      • Privacy: Aleo offers full privacy through zk-SNARK proofs, making it suitable for building fully private applications.
      • Performance: Aleo focuses on scalability through efficient proof systems, though detailed performance benchmarks are not widely available.
      • Integration: Aleo is built for privacy-first dapps and integrates with other zkVM-based systems.
      • Conclusion: Yes, Aleo qualifies as a zkVM, offering a comprehensive solution for private and scalable dapps.

      10. [Ola]​

      • Overview: Ola [10] is a ZK-friendly, high-performance layer-2 (L2) rollup platform that is still under development. It is designed to execute computations offchain while generating -validity proofs for these computations, ensuring that they are correctly executed without compromising security.
      • Privacy: Ola does not specifically prioritize privacy in the same way that zkVMs do. While it leverages ZKPs for scalability, its focus is on proving the correctness of -transactions and computations rather than ensuring that the data remains private.
      • Performance: Ola is designed to achieve high performance, particularly in terms of transaction throughput.
      • Integration: Ola is designed to be interoperable with various layer-1 blockchains. The platform supports a hybrid ZK-rollup architecture and is expected to include bridges for cross-chain -interoperability, enabling assets and data to move seamlessly between the layer-1 blockchain and the Ola rollup.
      • Conclusion: No, Ola is not a zkVM. While it leverages ZKPs (in the form of ZK-rollups) to ensure the validity of offchain computations, its primary focus is on scalability and performance +and scalability while deprioritizing privacy features, which can limit their use in certain privacy-sensitive applications.
        • +
      • Scalability and performance: zkVMs should offer scalable and efficient computation, leveraging advanced cryptographic techniques like zk-SNARKs, zk-STARKs, or recursive proofs. +A zkVM's performance must also be measured in terms of latency (time to generate and verify a proof) and throughput (number of computations processed within a certain time frame).
        • +
      • Verifiable computation: The zkVM should be able to prove the execution of arbitrary programs in a secure and verifiable manner. Verifiable computation ensures that zkVMs can be deployed +across a wide range of applications, from DeFi to private data-sharing platforms and more.
        • +
      +

      Why zkVMs matter

      +

      The rise of zkVMs is a crucial development for the future of blockchain and decentralized technologies. As more systems require the ability to scale while maintaining privacy and trust, +zkVMs provide a powerful solution. They offer the potential to reshape the way decentralized applications (dapps) handle sensitive information, enabling them to be both efficient and private.

      +

      It is essential to distinguish between projects that fully realize the potential of zkVMs and those that do not. In the remainder of this post, we evaluate several zkVM projects, analyzing +whether they satisfy the criteria for being classified as zkVMs based on our research.

      +

      Our methodology

      +

      We analyzed each project’s documentation, source code, and available benchmarks to determine whether they meet the definition of a zkVM. +Our criteria focus on the key capabilities of zkVMs—proof generation, privacy, scalability, and integration with existing systems.

      +

      ZkVM project analysis

      +

      1. [SP1]​

      +
        +
      • Overview: SP1 [1] is a developer-friendly zkVM designed to enable ZKP execution for LLVM-based languages like C, C++, Rust, and others. It supports a RISC-V-like instruction set architecture (ISA), +which makes it compatible with various programming languages compiled through LLVM.
        • +
      • Main focus: The main focus of SP1 is scalability, open-source contributions, and accessibility for developers. It prioritizes performance over privacy, +making it a good fit for environments where privacy isn't the primary concern.
        • +
      • Privacy: Not explicitly mentioned, making it less suitable for privacy-preserving applications.
        • +
      • Performance: SP1 has demonstrated up to 5.4x better performance than similar zkVMs like RISC0 for specific computations such as Fibonacci sequence generation.
        • +
      • Integration: SP1 is highly adaptable for rollups, light client verifiers, oracles, and even web2 projects like verifying the originality of images.
        • +
      • Conclusion: Yes, SP1 is a zkVM, but it does not prioritize zero-knowledge privacy, focusing more on scalability and performance.
        • +
      +

      2. [Nexus]​

      +
        +
      • Overview: Nexus [2] is a highly modular zkVM designed to process up to a trillion CPU cycles per second. It relies on RISC-V instructions for computation, making it extensible and scalable. +However, it currently lacks full ZKP capabilities due to its use of Spartan proofs.
        • +
      • Main focus: Nexus focuses on high performance and scalability, aiming to create an efficient execution environment for computationally intensive tasks.
        • +
      • Privacy: Although zero-knowledge privacy isn't the primary feature of Nexus, the project hints at potential privacy enhancements in the future.
        • +
      • Performance: Nexus has a high theoretical throughput, but it has yet to demonstrate benchmarks on zero-knowledge privacy.
        • +
      • Integration: Nexus is a good fit for high-performance environments that do not necessarily require full privacy.
        • +
      • Conclusion: Yes, Nexus qualifies as a zkVM in terms of scalability and proof generation, but it does not yet achieve full zero-knowledge privacy.
        • +
      +

      3. [RISC0]​

      +
        +
      • Overview: Risc0 [3] is a general-purpose zkVM with strong developer support. It allows for the execution of Rust and C code on a RISC-V virtual machine +and generates zk-SNARK and zk-STARK proofs for these computations.
        • +
      • Main focus: Risc0 is focused on ease of use for developers by abstracting away the complexities of circuit generation, making it accessible for a wide range of use cases.
        • +
      • Privacy: Full zero-knowledge privacy is supported via zk-SNARK and zk-STARK proofs, with Groth16 used for constant-size proof generation.
        • +
      • Performance: Risc0 offers strong benchmarks across different hardware setups, making it one of the most versatile zkVMs in terms of performance and scalability.
        • +
      • Integration: Risc0 integrates with several ecosystems, including Ethereum, and supports verifiable execution of Rust-based programs.
        • +
      • Conclusion: Yes, Risc0 qualifies as a zkVM, offering a balance of developer usability, scalability, and privacy.
        • +
      +

      4. [Powdr]​

      +
        +
      • Overview: Powdr [4] is a toolkit for creating custom zkVMs. It allows developers to select from various front-end and back-end components to create zkVMs tailored to specific needs.
        • +
      • Main focus: Powdr is focused on providing a modular architecture for zkVM creation. It enables flexibility by allowing the combination of different ZK-proof backends like Halo2 or Valida.
        • +
      • Privacy: Powdr itself does not generate ZKPs, but it facilitates the creation of zkVMs that do.
        • +
      • Performance: The performance depends on the components chosen by the developer, as Powdr itself is more of a framework.
        • +
      • Integration: Powdr is highly customizable and can integrate with existing zkVM frameworks to extend their capabilities.
        • +
      • Conclusion: No, Powdr is not a zkVM itself, but it is a powerful tool for building customized zkVMs with different privacy and performance needs.
        • +
      +

      5. [ZkMIPS]​

      +
        +
      • Overview: ZkMIPS [5] uses zk-STARKs to ensure privacy during computation, ensuring that private inputs are preserved while still proving correctness.
        • +
      • Performance: ZkMIPS is built for scalability, though explicit benchmarks are not widely published.
        • +
      • Integration: ZkMIPS can be integrated into systems that rely on MIPS architecture, making it versatile for legacy codebases that require privacy.
        • +
      • Conclusion: Yes, zkMIPS is a zkVM focused on scalability and privacy for MIPS-based architectures.
        • +
      +

      6. [Valida]​

      +
        +
      • Overview: Valida [6] is a performance-oriented zkVM that generates proofs for programs using a custom ISA designed to optimize zkVM implementation. +It uses Plonky3 for its proof system.
        • +
      • Main focus: Valida is centered around optimizing prover performance and extensibility, making it a valuable tool for generating proofs efficiently.
        • +
      • Privacy: While Valida is focused on performance, it does not prioritize zero-knowledge privacy as much as other zkVMs.
        • +
      • Performance: Valida has benchmarks indicating its performance advantages in proving computations quickly, particularly through parallel processing.
        • +
      • Integration: Valida is specialized and may not integrate as seamlessly into general-purpose systems, as it is optimized for performance over broad applicability.
        • +
      • Conclusion: Yes, Valida qualifies as a zkVM based on proof generation, but its lack of focus on privacy makes it less suitable for privacy-first use cases.
        • +
      +

      7. [Jolt]​

      +
        +
      • Overview: Jolt [7] is a zkVM built to optimize prover performance using a modified Hyrax polynomial commitment system. It relies on RISC-V instructions for computation +but falls short of full zero-knowledge capabilities.
        • +
      • Main focus: Jolt's main goal is to optimize the speed of proving program execution, making it suitable for high-performance applications where privacy isn't the primary concern.
        • +
      • Privacy: Jolt does not fully achieve zero-knowledge privacy due to the choice of polynomial commitment schemes.
        • +
      • Performance: Jolt offers strong performance, with benchmarks highlighting its ability to process proofs efficiently.
        • +
      • Integration: Jolt can be integrated with systems that prioritize speed over privacy, particularly where rapid proof generation is essential.
        • +
      • Conclusion: Yes, Jolt qualifies as a zkVM based on proof generation, though it does not provide full zero-knowledge privacy.
        • +
      +

      8. [ZkWASM]​

      +
        +
      • Overview: ZkWASM [8] is a zkVM designed to execute WebAssembly (WASM) code in a privacy-preserving and scalable manner. It uses zk-SNARKs to prove the correctness of WASM +program execution while ensuring privacy.
        • +
      • Main focus: ZkWASM focuses on scalability and privacy for WebAssembly, making it ideal for dapps that require verifiable computation without compromising privacy.
        • +
      • Privacy: Full zero-knowledge privacy is provided through zk-SNARKs, ensuring that the execution of WASM programs remains confidential.
        • +
      • Performance: ZkWASM is optimized for running WASM programs efficiently, with offchain computation and onchain verification to enhance performance.
        • +
      • Integration: ZkWASM is ideal for dapps, particularly those that use WebAssembly and require verifiable execution.
        • +
      • Conclusion: Yes, zkWASM qualifies as a zkVM, providing strong privacy, scalability, and verifiable execution for WebAssembly code.
        • +
      +

      9. [Aleo]​

      +
        +
      • Overview: Aleo's [9] snarkVM converts code into Aleo instructions, which are then compiled into bytecode executable on its zkVM. Aleo emphasizes building private, scalable dapps.
        • +
      • Main focus: Aleo prioritizes privacy and scalability for dapps, providing a robust framework for developers building private dapps.
        • +
      • Privacy: Aleo offers full privacy through zk-SNARK proofs, making it suitable for building fully private applications.
        • +
      • Performance: Aleo focuses on scalability through efficient proof systems, though detailed performance benchmarks are not widely available.
        • +
      • Integration: Aleo is built for privacy-first dapps and integrates with other zkVM-based systems.
        • +
      • Conclusion: Yes, Aleo qualifies as a zkVM, offering a comprehensive solution for private and scalable dapps.
        • +
      +

      10. [Ola]​

      +
        +
      • Overview: Ola [10] is a ZK-friendly, high-performance layer-2 (L2) rollup platform that is still under development. It is designed to execute computations offchain while generating +validity proofs for these computations, ensuring that they are correctly executed without compromising security.
        • +
      • Privacy: Ola does not specifically prioritize privacy in the same way that zkVMs do. While it leverages ZKPs for scalability, its focus is on proving the correctness of +transactions and computations rather than ensuring that the data remains private.
        • +
      • Performance: Ola is designed to achieve high performance, particularly in terms of transaction throughput.
        • +
      • Integration: Ola is designed to be interoperable with various layer-1 blockchains. The platform supports a hybrid ZK-rollup architecture and is expected to include bridges for cross-chain +interoperability, enabling assets and data to move seamlessly between the layer-1 blockchain and the Ola rollup.
        • +
      • Conclusion: No, Ola is not a zkVM. While it leverages ZKPs (in the form of ZK-rollups) to ensure the validity of offchain computations, its primary focus is on scalability and performance rather than privacy or verifiable execution of arbitrary programs. Ola is more accurately described as a ZK-rollup platform aimed at improving transaction throughput and reducing transaction costs on -layer-1 blockchains.

      11. [Miden]​

      • Overview: Miden zkVM [11] is a zk-STARK-based virtual machine that converts code into Miden VM instructions and proves the execution of these instructions with zero-knowledge privacy.
      • Main focus: Miden focuses on scalability and privacy for ZK-rollups, offering efficient proof generation for dapps.
      • Privacy: Miden ensures privacy for transactions and programs via zk-STARK proofs, making it suitable for private dapps.
      • Performance: Miden is optimized for scalability, with benchmarks showing its ability to handle up to 1,000 transactions per second (TPS).
      • Integration: Miden integrates well with ZK-rollup solutions, making it ideal for L2 scaling solutions on blockchains like Ethereum.
      • Conclusion: Yes, Miden qualifies as a zkVM, providing strong privacy and scalability for dapps and ZK-rollups.

      12. [ZkOS]​

      • Overview: ZkOS [12] is a verifiable operating system focused on running zkApps in a decentralized manner. It is built on the RISC-V architecture and aims to create -a world computer where all untrusted executions can be verified.
      • Main focus: ZkOS is primarily designed to offer a proof-of-concept operating system where all executions can be verified in a trustless manner. -However, its focus is more on the infrastructure for verifiable applications rather than being a traditional zkVM.
      • Privacy: ZkOS does not focus on privacy guarantees such as those found in zkVMs that generate ZKPs.
      • Performance: ZkOS focuses on the efficient execution of dapps, but performance benchmarks specific to ZKP generation are not provided.
      • Integration: ZkOS supports the execution of zkApps, but it is more of a verifiable operating system rather than a zkVM, making it distinct in its functionality.
      • Conclusion: No, zkOS is not a zkVM. It is a verifiable operating system focused on the infrastructure to support zkApps but does not directly generate ZKPs or focus on privacy preservation.

      13. [Triton]​

      • Overview: Triton [13] is a domain-specific language (DSL) and compiler designed primarily for high-performance GPU kernels, particularly those used in deep learning applications.
      • Main focus: The primary goal of Triton is to optimize computation for machine learning and GPU workloads. It is focused on enhancing performance and efficiency in processing data -rather than on ZKPs or verifiable computation.
      • Privacy: Triton does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on high-performance computation rather than cryptographic verifiability.
      • Performance: Triton is highly optimized for GPU execution, offering significant improvements in performance for computationally intensive tasks such as those found in deep learning.
      • Integration: Triton is integrated with GPU-based computation environments and is highly specialized for optimizing low-level operations on hardware rather -than being a general-purpose virtual machine.
      • Conclusion: No, Triton is not a zkVM. It is a specialized tool for optimizing GPU workloads, focusing on performance rather than privacy or ZKPs.

      14. [Cairo]​

      • Overview: Cairo zkVM [14] uses a custom language that compiles to an optimized STARK-based proof system, ensuring verifiable computation. It is primarily used in systems like Starknet.
      • Main focus: Cairo focuses on scalability and performance, using zk-STARK proofs to ensure the verifiable and secure execution of programs.
      • Privacy: Cairo provides privacy through zk-STARKs, but it focuses more on scalability and performance than privacy-first use cases.
      • Performance: Cairo is highly optimized for performance, making it well-suited for scalable applications on Starknet.
      • Integration: Cairo integrates deeply with systems like Starknet, supporting verifiable computation in a highly scalable and efficient manner.
      • Conclusion: Yes, Cairo qualifies as a zkVM, focusing on performance and verifiable execution while being ZK-friendly.

      15. [SnarkOS]​

      • Overview: SnarkOS [15] is a decentralized operating system designed to power Aleo's network, enabling secure and private dapps. -It manages transactions and consensus, making it a critical infrastructure component for Aleo's zkVM-based ecosystem.
      • Main focus: SnarkOS primarily focuses on securing Aleo's network through consensus mechanisms and privacy-preserving transactions rather than acting as a -zkVM that directly proves program execution.
      • Privacy: SnarkOS supports zero-knowledge privacy through its integration with Aleo's zkVM, but the operating system itself does not generate ZKPs for arbitrary computations.
      • Performance: SnarkOS is optimized for managing dapps on the Aleo network and handling private transactions, but its focus is more on infrastructure -and consensus than on proof generation.
      • Integration: SnarkOS integrates seamlessly with Aleo's zkVM to support private dapps and transactions, but its primary role is as a consensus layer.
      • Conclusion: No, SnarkOS is not a zkVM. It serves as an operating system for Aleo's decentralized network, focusing on privacy and consensus rather than on generating ZKPs for computations.

      16. [Lurk]​

      • Overview: Lurk [16] is a Turing-complete programming language designed for recursive zk-SNARKs. It focuses on enabling developers to build complex, -recursive ZKPs efficiently through a custom language tailored for verifiable computation.
      • Main focus: Lurk is centered around recursive proof generation rather than serving as a traditional virtual machine. Its purpose is to facilitate the creation of complex zk-SNARK-based proofs, -making it a specialized tool for cryptographic proofs rather than general-purpose computation.
      • Privacy: Lurk is built for generating zk-SNARKs, which inherently provide privacy. However, Lurk itself is a language and not a zkVM that executes arbitrary programs and generates ZKPs for them.
      • Performance: Lurk is optimized for recursive zk-SNARK generation, but specific performance metrics are tied to its proof-generation capabilities rather than traditional execution environments.
      • Integration: Lurk is specialized for zk-SNARKs and may not easily integrate with other general-purpose systems, as it focuses on specific cryptographic tasks.
      • Conclusion: No, Lurk is not a zkVM. It is a programming language designed for recursive zk-SNARKs and focuses on proof generation rather than program execution in a virtual machine environment.

      17. [Piecrust]​

      • Overview: Piecrust [17] is a WASM-based zkVM designed to run on the Dusk Network. It supports concurrent execution and focuses on providing privacy and scalability for smart contracts.
      • Main focus: Piecrust is designed to provide private and efficient execution of smart contracts through the use of ZKPs.
      • Privacy: Piecrust supports ZK-friendly computations and enhances privacy through cryptographic primitives such as Merkle trees.
      • Performance: Piecrust is designed to be scalable and concurrent, allowing multiple sessions to run simultaneously, which improves overall performance.
      • Integration: Piecrust integrates with the Dusk Network and supports private smart contracts, making it ideal for dapps.
      • Conclusion: Yes, Piecrust qualifies as a zkVM, offering scalability, privacy, and support for succinct proof generation.

      18. [Ceno]​

      • Overview: Ceno [18] is a zkVM that provides a theoretical framework for reducing proving time by grouping common portions of code together. It uses recursive proofs to enhance prover efficiency.
      • Main focus: Ceno aims to optimize prover performance through recursive proofs, making it a powerful tool for handling complex computations efficiently.
      • Privacy: Ceno supports zero-knowledge privacy through recursive proofs and is designed to handle large-scale computations securely.
      • Performance: Ceno's recursive proof framework ensures that it can efficiently prove the execution of programs, reducing the time required for proof generation.
      • Integration: Ceno can be integrated into systems that require high efficiency and privacy, particularly those handling complex, repeated computations.
      • Conclusion: Yes, Ceno qualifies as a zkVM, providing efficient and private computation through the use of recursive proofs.

      19. [Stellar]​

      • Overview: Stellar [19] is a decentralized protocol designed to facilitate cross-border transactions between digital and fiat currencies.
      • Main focus: Stellar's primary goal is to improve financial transactions by enabling decentralized, low-cost currency transfers. It does not aim to provide ZKPs or run verifiable computations -like a zkVM.
      • Privacy: Stellar focuses on confidentiality and security for financial transactions, but it does not employ ZKPs in the way zkVMs do for verifying computation without revealing data.
      • Performance: Stellar prioritizes the performance of financial transactions, ensuring low latency and high throughput across its decentralized network. -However, this performance focus is specific to transactions rather than general-purpose program execution.
      • Integration: Stellar is designed for integration with financial systems, enabling currency conversions and transfers, but it is not built for executing smart contracts or verifiable computations.
      • Conclusion: No, Stellar is not a zkVM. It is a decentralized financial protocol focused on facilitating cross-border payments rather than verifiable or privacy-preserving computation.

      20. [NovaNet]​

      • Overview: NovaNet [20] is an open peer-to-peer network that aims to build upon concepts of non-uniform incremental verifiable computation.
      • Main focus: NovaNet's focus is on peer-to-peer networking and decentralized computing rather than on proving the execution of programs in a zero-knowledge manner.
      • Privacy: NovaNet does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on decentralized networking and computation.
      • Performance: NovaNet prioritizes efficient decentralized computation but does not focus on privacy or performance benchmarks related to ZKPs.
      • Integration: NovaNet is built for decentralized networks but is not designed to integrate with systems requiring verifiable computation or ZKP generation.
      • Conclusion: No, NovaNet is not a zkVM. It is a decentralized peer-to-peer network focused on distributed computing rather than zero-knowledge computation.

      21. [ZkLLVM]​

      • Overview: ZkLLVM [21] is a compiler that transforms C++ or Rust code into circuits for use in zk-SNARK or zk-STARK systems. Its primary purpose is to bridge high-level programming -languages with ZKP systems by compiling code into arithmetic circuits that can be used to generate and verify proofs.
      • Main focus: ZkLLVM focuses on making ZKPs accessible to developers by enabling them to write code in familiar languages (C++, Rust) and then compile that code into ZK circuits.
      • Privacy: ZkLLVM enables the generation of ZKPs by compiling high-level code into ZK-compatible circuits. It plays a crucial role in privacy-preserving applications but does not act -as a zkVM itself.
      • Performance: ZkLLVM allows for the performance of ZKPs to be closely tied to the complexity of the compiled circuits. The performance depends on the underlying -zk-SNARK or zk-STARK system used.
      • Integration: ZkLLVM integrates with zk-SNARK and zk-STARK proof systems, making it useful for a variety of privacy-focused applications, but it does not serve as a zkVM -for general-purpose computation.
      • Conclusion: No, zkLLVM is not a zkVM. It is a compiler that transforms high-level code into ZK circuits, enabling ZKPs but not acting as a virtual machine for executing and proving programs.

      22. [ZkMove]​

      • Overview: ZkMove [22] is a zkVM designed to execute smart contracts written in the Move language. It utilizes ZKPs to ensure that the execution of these contracts remains verifiable and secure.
      • Main focus: ZkMove focuses on privacy and verifiable execution for Move-based smart contracts, providing a framework for ZK-friendly computation.
      • Privacy: ZkMove ensures that smart contract execution remains private through ZKPs, making it suitable for privacy-preserving applications.
      • Performance: ZkMove is optimized for verifiable execution, ensuring that contracts can be proven correct while preserving privacy.
      • Integration: ZkMove integrates well with systems that use the Move language, particularly in environments that require private smart contract execution.
      • Conclusion: Yes, zkMove qualifies as a zkVM, offering ZK-friendly execution and privacy for smart contracts written in the Move language.

      23. [O1VM]​

      • Overview: O1VM [23] is a general-purpose zkVM developed by o1Labs. It is designed to prove the execution of MIPS programs efficiently through a combination of zk-SNARKs -and specialized techniques like folding schemes and RAMLookups.
      • Main focus: O1VM focuses on scalability and verifiable computation for MIPS-based programs, making it a strong contender for executing and proving complex programs efficiently.
      • Privacy: O1VM ensures privacy through zk-SNARK proofs, keeping the details of the computation private while proving its correctness.
      • Performance: O1VM is optimized for handling long execution traces and complex computations, making it highly scalable.
      • Integration: O1VM integrates well with MIPS-based architectures and systems that require privacy-preserving computation.
      • Conclusion: Yes, o1VM qualifies as a zkVM, providing privacy, scalability, and strong proof generation for MIPS programs.

      Summary of findings

      Project nameZkVM statusZero knowledgeReasoning/comments
      SP1YesNoProves execution of LLVM-based programs but lacks privacy features.
      NexusYesNoStrong proof generation but lacks zero-knowledge privacy due to Spartan.
      Risc0YesYesSupports full ZKP generation for Rust programs.
      PowdrNoYesToolkit for creating custom zkVMs, not a zkVM itself.
      ZkMIPSYesYesSupports MIPS-like architecture with full zero-knowledge and proof generation.
      ValidaYesNoPerformance-focused zkVM, lacks privacy guarantees.
      JoltYesNoPerformance-focused zkVM, does not achieve zero-knowledge privacy.
      ZkWASMYesYesFull zero-knowledge and verifiable execution of WebAssembly code.
      AleoYesYesFully private and scalable dapps.
      OlaNoNoPrimarily a ZK-rollup platform, not a zkVM, focusing on scalability and performance rather than privacy.
      MidenYesYesZk-STARK-based zkVM with strong privacy and scalability.
      ZkOSNoNoVerifiable operating system focused on zkApps, not a zkVM.
      TritonNoNoOptimizes GPU workloads but not designed for ZKPs.
      CairoYesZK-friendlyCustom Rust-based language with zk-STARK proof generation.
      SnarkOSNoYesDecentralized OS for Aleo's network, focuses on consensus rather than verifiable computation.
      LurkNoNoProgramming language for recursive zk-SNARKs, not a zkVM.
      PiecrustYesZK-friendlyZkVM with recursive SNARK capabilities, focused on succinct proof generation.
      CenoYesYesTheoretical zkVM improving prover efficiency through recursive proofs.
      StellarNoNoFocuses on cross-border transactions, not ZK-proof generation or verifiable computation.
      NovaNetNoNoPeer-to-peer network focused on distributed computing, not zero-knowledge computation.
      ZkLLVMNoYes, in some casesCompiler for generating ZK-circuits, not a zkVM.
      ZkMoveYesZK-friendlyZkVM supporting Move language with ZKP execution.
      O1VMYesYesMIPS-based zkVM with strong privacy, scalability, and proof generation.

      Insights and conclusions

      Our analysis reveals that many of the projects labeled as zkVMs do meet the core criteria for zkVMs, offering verifiable computation and proof generation +layer-1 blockchains.

      • +
    +

    11. [Miden]​

    +
      +
    • Overview: Miden zkVM [11] is a zk-STARK-based virtual machine that converts code into Miden VM instructions and proves the execution of these instructions with zero-knowledge privacy.
      • +
    • Main focus: Miden focuses on scalability and privacy for ZK-rollups, offering efficient proof generation for dapps.
      • +
    • Privacy: Miden ensures privacy for transactions and programs via zk-STARK proofs, making it suitable for private dapps.
      • +
    • Performance: Miden is optimized for scalability, with benchmarks showing its ability to handle up to 1,000 transactions per second (TPS).
      • +
    • Integration: Miden integrates well with ZK-rollup solutions, making it ideal for L2 scaling solutions on blockchains like Ethereum.
      • +
    • Conclusion: Yes, Miden qualifies as a zkVM, providing strong privacy and scalability for dapps and ZK-rollups.
      • +
    +

    12. [ZkOS]​

    +
      +
    • Overview: ZkOS [12] is a verifiable operating system focused on running zkApps in a decentralized manner. It is built on the RISC-V architecture and aims to create +a world computer where all untrusted executions can be verified.
      • +
    • Main focus: ZkOS is primarily designed to offer a proof-of-concept operating system where all executions can be verified in a trustless manner. +However, its focus is more on the infrastructure for verifiable applications rather than being a traditional zkVM.
      • +
    • Privacy: ZkOS does not focus on privacy guarantees such as those found in zkVMs that generate ZKPs.
      • +
    • Performance: ZkOS focuses on the efficient execution of dapps, but performance benchmarks specific to ZKP generation are not provided.
      • +
    • Integration: ZkOS supports the execution of zkApps, but it is more of a verifiable operating system rather than a zkVM, making it distinct in its functionality.
      • +
    • Conclusion: No, zkOS is not a zkVM. It is a verifiable operating system focused on the infrastructure to support zkApps but does not directly generate ZKPs or focus on privacy preservation.
      • +
    +

    13. [Triton]​

    +
      +
    • Overview: Triton [13] is a domain-specific language (DSL) and compiler designed primarily for high-performance GPU kernels, particularly those used in deep learning applications.
      • +
    • Main focus: The primary goal of Triton is to optimize computation for machine learning and GPU workloads. It is focused on enhancing performance and efficiency in processing data +rather than on ZKPs or verifiable computation.
      • +
    • Privacy: Triton does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on high-performance computation rather than cryptographic verifiability.
      • +
    • Performance: Triton is highly optimized for GPU execution, offering significant improvements in performance for computationally intensive tasks such as those found in deep learning.
      • +
    • Integration: Triton is integrated with GPU-based computation environments and is highly specialized for optimizing low-level operations on hardware rather +than being a general-purpose virtual machine.
      • +
    • Conclusion: No, Triton is not a zkVM. It is a specialized tool for optimizing GPU workloads, focusing on performance rather than privacy or ZKPs.
      • +
    +

    14. [Cairo]​

    +
      +
    • Overview: Cairo zkVM [14] uses a custom language that compiles to an optimized STARK-based proof system, ensuring verifiable computation. It is primarily used in systems like Starknet.
      • +
    • Main focus: Cairo focuses on scalability and performance, using zk-STARK proofs to ensure the verifiable and secure execution of programs.
      • +
    • Privacy: Cairo provides privacy through zk-STARKs, but it focuses more on scalability and performance than privacy-first use cases.
      • +
    • Performance: Cairo is highly optimized for performance, making it well-suited for scalable applications on Starknet.
      • +
    • Integration: Cairo integrates deeply with systems like Starknet, supporting verifiable computation in a highly scalable and efficient manner.
      • +
    • Conclusion: Yes, Cairo qualifies as a zkVM, focusing on performance and verifiable execution while being ZK-friendly.
      • +
    +

    15. [SnarkOS]​

    +
      +
    • Overview: SnarkOS [15] is a decentralized operating system designed to power Aleo's network, enabling secure and private dapps. +It manages transactions and consensus, making it a critical infrastructure component for Aleo's zkVM-based ecosystem.
      • +
    • Main focus: SnarkOS primarily focuses on securing Aleo's network through consensus mechanisms and privacy-preserving transactions rather than acting as a +zkVM that directly proves program execution.
      • +
    • Privacy: SnarkOS supports zero-knowledge privacy through its integration with Aleo's zkVM, but the operating system itself does not generate ZKPs for arbitrary computations.
      • +
    • Performance: SnarkOS is optimized for managing dapps on the Aleo network and handling private transactions, but its focus is more on infrastructure +and consensus than on proof generation.
      • +
    • Integration: SnarkOS integrates seamlessly with Aleo's zkVM to support private dapps and transactions, but its primary role is as a consensus layer.
      • +
    • Conclusion: No, SnarkOS is not a zkVM. It serves as an operating system for Aleo's decentralized network, focusing on privacy and consensus rather than on generating ZKPs for computations.
      • +
    +

    16. [Lurk]​

    +
      +
    • Overview: Lurk [16] is a Turing-complete programming language designed for recursive zk-SNARKs. It focuses on enabling developers to build complex, +recursive ZKPs efficiently through a custom language tailored for verifiable computation.
      • +
    • Main focus: Lurk is centered around recursive proof generation rather than serving as a traditional virtual machine. Its purpose is to facilitate the creation of complex zk-SNARK-based proofs, +making it a specialized tool for cryptographic proofs rather than general-purpose computation.
      • +
    • Privacy: Lurk is built for generating zk-SNARKs, which inherently provide privacy. However, Lurk itself is a language and not a zkVM that executes arbitrary programs and generates ZKPs for them.
      • +
    • Performance: Lurk is optimized for recursive zk-SNARK generation, but specific performance metrics are tied to its proof-generation capabilities rather than traditional execution environments.
      • +
    • Integration: Lurk is specialized for zk-SNARKs and may not easily integrate with other general-purpose systems, as it focuses on specific cryptographic tasks.
      • +
    • Conclusion: No, Lurk is not a zkVM. It is a programming language designed for recursive zk-SNARKs and focuses on proof generation rather than program execution in a virtual machine environment.
      • +
    +

    17. [Piecrust]​

    +
      +
    • Overview: Piecrust [17] is a WASM-based zkVM designed to run on the Dusk Network. It supports concurrent execution and focuses on providing privacy and scalability for smart contracts.
      • +
    • Main focus: Piecrust is designed to provide private and efficient execution of smart contracts through the use of ZKPs.
      • +
    • Privacy: Piecrust supports ZK-friendly computations and enhances privacy through cryptographic primitives such as Merkle trees.
      • +
    • Performance: Piecrust is designed to be scalable and concurrent, allowing multiple sessions to run simultaneously, which improves overall performance.
      • +
    • Integration: Piecrust integrates with the Dusk Network and supports private smart contracts, making it ideal for dapps.
      • +
    • Conclusion: Yes, Piecrust qualifies as a zkVM, offering scalability, privacy, and support for succinct proof generation.
      • +
    +

    18. [Ceno]​

    +
      +
    • Overview: Ceno [18] is a zkVM that provides a theoretical framework for reducing proving time by grouping common portions of code together. It uses recursive proofs to enhance prover efficiency.
      • +
    • Main focus: Ceno aims to optimize prover performance through recursive proofs, making it a powerful tool for handling complex computations efficiently.
      • +
    • Privacy: Ceno supports zero-knowledge privacy through recursive proofs and is designed to handle large-scale computations securely.
      • +
    • Performance: Ceno's recursive proof framework ensures that it can efficiently prove the execution of programs, reducing the time required for proof generation.
      • +
    • Integration: Ceno can be integrated into systems that require high efficiency and privacy, particularly those handling complex, repeated computations.
      • +
    • Conclusion: Yes, Ceno qualifies as a zkVM, providing efficient and private computation through the use of recursive proofs.
      • +
    +

    19. [Stellar]​

    +
      +
    • Overview: Stellar [19] is a decentralized protocol designed to facilitate cross-border transactions between digital and fiat currencies.
      • +
    • Main focus: Stellar's primary goal is to improve financial transactions by enabling decentralized, low-cost currency transfers. It does not aim to provide ZKPs or run verifiable computations +like a zkVM.
      • +
    • Privacy: Stellar focuses on confidentiality and security for financial transactions, but it does not employ ZKPs in the way zkVMs do for verifying computation without revealing data.
      • +
    • Performance: Stellar prioritizes the performance of financial transactions, ensuring low latency and high throughput across its decentralized network. +However, this performance focus is specific to transactions rather than general-purpose program execution.
      • +
    • Integration: Stellar is designed for integration with financial systems, enabling currency conversions and transfers, but it is not built for executing smart contracts or verifiable computations.
      • +
    • Conclusion: No, Stellar is not a zkVM. It is a decentralized financial protocol focused on facilitating cross-border payments rather than verifiable or privacy-preserving computation.
      • +
    +

    20. [NovaNet]​

    +
      +
    • Overview: NovaNet [20] is an open peer-to-peer network that aims to build upon concepts of non-uniform incremental verifiable computation.
      • +
    • Main focus: NovaNet's focus is on peer-to-peer networking and decentralized computing rather than on proving the execution of programs in a zero-knowledge manner.
      • +
    • Privacy: NovaNet does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on decentralized networking and computation.
      • +
    • Performance: NovaNet prioritizes efficient decentralized computation but does not focus on privacy or performance benchmarks related to ZKPs.
      • +
    • Integration: NovaNet is built for decentralized networks but is not designed to integrate with systems requiring verifiable computation or ZKP generation.
      • +
    • Conclusion: No, NovaNet is not a zkVM. It is a decentralized peer-to-peer network focused on distributed computing rather than zero-knowledge computation.
      • +
    +

    21. [ZkLLVM]​

    +
      +
    • Overview: ZkLLVM [21] is a compiler that transforms C++ or Rust code into circuits for use in zk-SNARK or zk-STARK systems. Its primary purpose is to bridge high-level programming +languages with ZKP systems by compiling code into arithmetic circuits that can be used to generate and verify proofs.
      • +
    • Main focus: ZkLLVM focuses on making ZKPs accessible to developers by enabling them to write code in familiar languages (C++, Rust) and then compile that code into ZK circuits.
      • +
    • Privacy: ZkLLVM enables the generation of ZKPs by compiling high-level code into ZK-compatible circuits. It plays a crucial role in privacy-preserving applications but does not act +as a zkVM itself.
      • +
    • Performance: ZkLLVM allows for the performance of ZKPs to be closely tied to the complexity of the compiled circuits. The performance depends on the underlying +zk-SNARK or zk-STARK system used.
      • +
    • Integration: ZkLLVM integrates with zk-SNARK and zk-STARK proof systems, making it useful for a variety of privacy-focused applications, but it does not serve as a zkVM +for general-purpose computation.
      • +
    • Conclusion: No, zkLLVM is not a zkVM. It is a compiler that transforms high-level code into ZK circuits, enabling ZKPs but not acting as a virtual machine for executing and proving programs.
      • +
    +

    22. [ZkMove]​

    +
      +
    • Overview: ZkMove [22] is a zkVM designed to execute smart contracts written in the Move language. It utilizes ZKPs to ensure that the execution of these contracts remains verifiable and secure.
      • +
    • Main focus: ZkMove focuses on privacy and verifiable execution for Move-based smart contracts, providing a framework for ZK-friendly computation.
      • +
    • Privacy: ZkMove ensures that smart contract execution remains private through ZKPs, making it suitable for privacy-preserving applications.
      • +
    • Performance: ZkMove is optimized for verifiable execution, ensuring that contracts can be proven correct while preserving privacy.
      • +
    • Integration: ZkMove integrates well with systems that use the Move language, particularly in environments that require private smart contract execution.
      • +
    • Conclusion: Yes, zkMove qualifies as a zkVM, offering ZK-friendly execution and privacy for smart contracts written in the Move language.
      • +
    +

    23. [O1VM]​

    +
      +
    • Overview: O1VM [23] is a general-purpose zkVM developed by o1Labs. It is designed to prove the execution of MIPS programs efficiently through a combination of zk-SNARKs +and specialized techniques like folding schemes and RAMLookups.
      • +
    • Main focus: O1VM focuses on scalability and verifiable computation for MIPS-based programs, making it a strong contender for executing and proving complex programs efficiently.
      • +
    • Privacy: O1VM ensures privacy through zk-SNARK proofs, keeping the details of the computation private while proving its correctness.
      • +
    • Performance: O1VM is optimized for handling long execution traces and complex computations, making it highly scalable.
      • +
    • Integration: O1VM integrates well with MIPS-based architectures and systems that require privacy-preserving computation.
      • +
    • Conclusion: Yes, o1VM qualifies as a zkVM, providing privacy, scalability, and strong proof generation for MIPS programs.
      • +
    +

    Summary of findings

    +
    Project nameZkVM statusZero knowledgeReasoning/comments
    SP1YesNoProves execution of LLVM-based programs but lacks privacy features.
    NexusYesNoStrong proof generation but lacks zero-knowledge privacy due to Spartan.
    Risc0YesYesSupports full ZKP generation for Rust programs.
    PowdrNoYesToolkit for creating custom zkVMs, not a zkVM itself.
    ZkMIPSYesYesSupports MIPS-like architecture with full zero-knowledge and proof generation.
    ValidaYesNoPerformance-focused zkVM, lacks privacy guarantees.
    JoltYesNoPerformance-focused zkVM, does not achieve zero-knowledge privacy.
    ZkWASMYesYesFull zero-knowledge and verifiable execution of WebAssembly code.
    AleoYesYesFully private and scalable dapps.
    OlaNoNoPrimarily a ZK-rollup platform, not a zkVM, focusing on scalability and performance rather than privacy.
    MidenYesYesZk-STARK-based zkVM with strong privacy and scalability.
    ZkOSNoNoVerifiable operating system focused on zkApps, not a zkVM.
    TritonNoNoOptimizes GPU workloads but not designed for ZKPs.
    CairoYesZK-friendlyCustom Rust-based language with zk-STARK proof generation.
    SnarkOSNoYesDecentralized OS for Aleo's network, focuses on consensus rather than verifiable computation.
    LurkNoNoProgramming language for recursive zk-SNARKs, not a zkVM.
    PiecrustYesZK-friendlyZkVM with recursive SNARK capabilities, focused on succinct proof generation.
    CenoYesYesTheoretical zkVM improving prover efficiency through recursive proofs.
    StellarNoNoFocuses on cross-border transactions, not ZK-proof generation or verifiable computation.
    NovaNetNoNoPeer-to-peer network focused on distributed computing, not zero-knowledge computation.
    ZkLLVMNoYes, in some casesCompiler for generating ZK-circuits, not a zkVM.
    ZkMoveYesZK-friendlyZkVM supporting Move language with ZKP execution.
    O1VMYesYesMIPS-based zkVM with strong privacy, scalability, and proof generation.
    +

    Insights and conclusions

    +

    Our analysis reveals that many of the projects labeled as zkVMs do meet the core criteria for zkVMs, offering verifiable computation and proof generation as foundational features. However, a number of these projects fall short of delivering full zero-knowledge privacy. Projects like Risc0, Aleo, and Miden stand out as leading zkVM frameworks -that balance proof generation, privacy, and scalability, offering strong platforms for developers seeking to build privacy-preserving applications.

    Conversely, projects like SP1 and Nexus excel in generating verifiable proofs but currently lack comprehensive zero-knowledge privacy mechanisms. These platforms are excellent for -scenarios where proof generation and scalability are paramount, but privacy is not a primary concern.

    As zkVM technology continues to evolve, we expect to see more projects integrating enhanced privacy-preserving mechanisms while simultaneously improving performance and scalability. +that balance proof generation, privacy, and scalability, offering strong platforms for developers seeking to build privacy-preserving applications.

    +

    Conversely, projects like SP1 and Nexus excel in generating verifiable proofs but currently lack comprehensive zero-knowledge privacy mechanisms. These platforms are excellent for +scenarios where proof generation and scalability are paramount, but privacy is not a primary concern.

    +

    As zkVM technology continues to evolve, we expect to see more projects integrating enhanced privacy-preserving mechanisms while simultaneously improving performance and scalability. This ongoing development will likely broaden the application of zkVMs across the blockchain ecosystem, particularly in privacy-sensitive sectors such as finance, data security, -and decentralized applications.

    What are your thoughts on our zkVM analysis? Do you agree with our findings, or do you know of other zkVM projects that should be on our radar? We would love to hear your insights, questions, -or suggestions! Feel free to join the discussion on our forum.

    References

    [1] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    [2] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    [3] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    [4] Powdr. Retrieved from https://docs.powdr.org/

    [5] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    [6] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    [7] Building Jolt: A fast, easy-to-use zkVM. Retrieved from https://a16zcrypto.com/posts/article/building-jolt/

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    [9] Aleo. Retrieved from https://aleo.org/blog/

    [10] OlaVM Whitepaper V2. Retrieved from https://github.com/Sin7Y/olavm-whitepaper-v2/tree/master

    [11] Polygon Miden VM. Retrieved from https://0xpolygonmiden.github.io/miden-vm/intro/main.html

    [12] The Adventures of OS: Making a RISC-V Operating System using Rust. Retrieved from https://osblog.stephenmarz.com/index.html

    [13] Triton VM. Retrieved from https://triton-vm.org/spec/

    [14] How does the original Cairo VM work?. Retrieved from https://github.com/lambdaclass/cairo-vm/blob/main/docs/python_vm/README.md

    [15] Aleo completes security audits of snarkOS & snarkVM. Retrieved from https://aleo.org/post/aleo-completes-security-audits-of-snarkos-and-snarkvm/

    [16] Lurk zkVM. Retrieved from https://github.com/lurk-lab

    [17] Piecrust VM. Retrieved from https://docs.rs/piecrust/latest/piecrust/

    [18] Ceno: Non-uniform, Segment and Parallel Zero-knowledge Virtual Machine. Retrieved from https://eprint.iacr.org/2024/387

    [19] ZkVM: a new design for fast, confidential smart contracts. Retrieved from https://stellar.org/blog/developers/zkvm-a-new-design-for-fast-confidential-smart-contracts

    [20] Novanet. Retrieved from https://www.novanet.xyz/blog

    [21] ZKLLVM. Retrieved from https://github.com/NilFoundation/zkLLVM

    [22] zkMove 0.2.0 - Achieving Full Bytecode Compatibility with Move. Retrieved from https://www.zkmove.net/2023-06-20-zkMove-0.2.0-Achieving-Full-Bytecode-Compatibility-with-Move/

    [23] O1VM. Retrieved from https://github.com/o1-labs/proof-systems/tree/master/o1vm

    ]]>     +and decentralized applications.

    +

    What are your thoughts on our zkVM analysis? Do you agree with our findings, or do you know of other zkVM projects that should be on our radar? We would love to hear your insights, questions, +or suggestions! Feel free to join the discussion on our forum.

    +

    References

    +

    [1] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    +

    [2] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    +

    [3] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    +

    [4] Powdr. Retrieved from https://docs.powdr.org/

    +

    [5] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    +

    [6] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    +

    [7] Building Jolt: A fast, easy-to-use zkVM. Retrieved from https://a16zcrypto.com/posts/article/building-jolt/

    +

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    +

    [9] Aleo. Retrieved from https://aleo.org/blog/

    +

    [10] OlaVM Whitepaper V2. Retrieved from https://github.com/Sin7Y/olavm-whitepaper-v2/tree/master

    +

    [11] Polygon Miden VM. Retrieved from https://0xpolygonmiden.github.io/miden-vm/intro/main.html

    +

    [12] The Adventures of OS: Making a RISC-V Operating System using Rust. Retrieved from https://osblog.stephenmarz.com/index.html

    +

    [13] Triton VM. Retrieved from https://triton-vm.org/spec/

    +

    [14] How does the original Cairo VM work?. Retrieved from https://github.com/lambdaclass/cairo-vm/blob/main/docs/python_vm/README.md

    +

    [15] Aleo completes security audits of snarkOS & snarkVM. Retrieved from https://aleo.org/post/aleo-completes-security-audits-of-snarkos-and-snarkvm/

    +

    [16] Lurk zkVM. Retrieved from https://github.com/lurk-lab

    +

    [17] Piecrust VM. Retrieved from https://docs.rs/piecrust/latest/piecrust/

    +

    [18] Ceno: Non-uniform, Segment and Parallel Zero-knowledge Virtual Machine. Retrieved from https://eprint.iacr.org/2024/387

    +

    [19] ZkVM: a new design for fast, confidential smart contracts. Retrieved from https://stellar.org/blog/developers/zkvm-a-new-design-for-fast-confidential-smart-contracts

    +

    [20] Novanet. Retrieved from https://www.novanet.xyz/blog

    +

    [21] ZKLLVM. Retrieved from https://github.com/NilFoundation/zkLLVM

    +

    [22] zkMove 0.2.0 - Achieving Full Bytecode Compatibility with Move. Retrieved from https://www.zkmove.net/2023-06-20-zkMove-0.2.0-Achieving-Full-Bytecode-Compatibility-with-Move/

    +

    [23] O1VM. Retrieved from https://github.com/o1-labs/proof-systems/tree/master/o1vm

    ]]>     <![CDATA[Nescience: A User-Centric State-Separation Architecture]]> @@ -902,459 +2101,983 @@ or suggestions! Feel free to join the Introducing NSSA: A user-centric approach​

    NSSA gives users control over their privacy settings by introducing shielded (which creates a layer of privacy for the outputs, and only the necessary details are shared) +including its different execution types, cryptographic foundations, and unique challenges.

    +

    Introducing NSSA: A user-centric approach​

    +

    NSSA gives users control over their privacy settings by introducing shielded (which creates a layer of privacy for the outputs, and only the necessary details are shared) and deshielded (which reveal private details, making them publicly visible) execution types in addition to the traditional public and private modes. This flexibility allows users to customize their privacy settings to match their unique needs, whether they require high levels of confidentiality or more transparency. In NSSA, the system is divided into two states: public and private. The public state uses an account-based model while the private state employs a UTXO-based (unspent transaction output) model. Private executions within NSSA utilize UTXO exchanges, ensuring that transaction details remain confidential. The sequencer verifies these exchanges without accessing specific details, enhancing privacy by unlinking sender and receiver identities. Zero-knowledge proofs (ZKPs) allow users to prove transaction validity without revealing data, maintaining the integrity and confidentiality of private transactions. UTXOs contain assets such as balances, NFTs, or private storage data, and are stored in plaintext within Sparse Merkle trees (SMTs) in the private state and -as hashes in the public state. This dual-storage approach keeps UTXO details confidential while allowing public verification through hashes, achieving a balance between privacy and transparency.

    Implementing NSSA introduces unique challenges, particularly in cryptographic implementation and maintaining the integrity of private executions. These challenges are addressed +as hashes in the public state. This dual-storage approach keeps UTXO details confidential while allowing public verification through hashes, achieving a balance between privacy and transparency.

    +

    Implementing NSSA introduces unique challenges, particularly in cryptographic implementation and maintaining the integrity of private executions. These challenges are addressed through various solutions such as ZKPs, which ensure transaction validity without compromising privacy, and the dual-storage approach, which maintains confidentiality while enabling public verification. By allowing users to customize their privacy settings, NSSA enhances user experience and promotes wider adoption of private execution platforms. As we move towards -a future where user-empowered privacy control is crucial, NSSA provides a flexible and user-centric solution that meets the diverse needs of blockchain users.

    Why NSSA differs from other hybrid execution platforms​

    In many existing hybrid execution platforms, privacy settings are predefined by developers, often applying a one-size-fits-all approach that does not accommodate the +a future where user-empowered privacy control is crucial, NSSA provides a flexible and user-centric solution that meets the diverse needs of blockchain users.

    +

    Why NSSA differs from other hybrid execution platforms​

    +

    In many existing hybrid execution platforms, privacy settings are predefined by developers, often applying a one-size-fits-all approach that does not accommodate the diverse privacy needs of users. These platforms blend public and private states, but control over privacy remains with the application developers. While this approach is straightforward for developers (who bear the responsibility for any potential privacy leaks), it leaves users with no control over their own privacy settings. This rigidity becomes problematic as user needs evolve over time, or as new regulations necessitate changes to privacy configurations. In such cases, -updates to decentralized applications are required to adjust privacy settings, which can disrupt the user experience and create friction.

    NSSA addresses these limitations by introducing a groundbreaking concept: selective privacy. Unlike traditional platforms where privacy +updates to decentralized applications are required to adjust privacy settings, which can disrupt the user experience and create friction.

    +

    NSSA addresses these limitations by introducing a groundbreaking concept: selective privacy. Unlike traditional platforms where privacy is static and determined by developers, selective privacy empowers users to dynamically choose their own privacy levels based on their unique needs and sensitivity. -This flexibility is critical in a decentralized ecosystem where the diversity of users and use cases demands a more adaptable privacy solution.

    In the NSSA model, users have the autonomy to select how they interact with decentralized applications (dapps) by choosing from four types of transaction executions: public, +This flexibility is critical in a decentralized ecosystem where the diversity of users and use cases demands a more adaptable privacy solution.

    +

    In the NSSA model, users have the autonomy to select how they interact with decentralized applications (dapps) by choosing from four types of transaction executions: public, private, shielded, and deshielded. This model allows users to tailor their privacy settings on a per-transaction basis, selecting the most appropriate execution type for each specific interaction. For instance, a user concerned about data confidentiality might opt for a fully private transaction while another user, wary of privacy but seeking transparency, -might choose a public execution.

    While selective privacy may appear complex, especially for users who are not technically inclined, Nescience mitigates this by allowing the community or developers to +might choose a public execution.

    +

    While selective privacy may appear complex, especially for users who are not technically inclined, Nescience mitigates this by allowing the community or developers to establish best practices and recommended approaches. These guidelines provide users with an informed starting point, and over time, users can adjust their privacy settings as their preferences and trust in the platform evolve. Importantly, selective privacy gives users the right to alter their privacy level at any point in the future, -ensuring that their privacy settings remain aligned with their needs as they change.

    This approach not only empowers users but also facilitates greater adoption of dapps. Users who are skeptical about privacy concerns can initially engage with transparent +ensuring that their privacy settings remain aligned with their needs as they change.

    +

    This approach not only empowers users but also facilitates greater adoption of dapps. Users who are skeptical about privacy concerns can initially engage with transparent transactions and gradually shift towards more private executions as they gain confidence in the system and vice versa for users who start with privacy but later find transparency beneficial for certain transactions. In this way, selective privacy bridges the gap between privacy and transparency, allowing for an optimal balance to emerge from the community’s -collective preferences.

    To liken this to open-source projects: in traditional systems, developers fix privacy rules much like immutable code—users must comply with these fixed settings. -In contrast, with selective privacy, the rules are malleable and shaped by the users’ preferences, enabling the community to find the ideal balance between privacy and efficiency over time.

    NSSA is distinct from traditional zero-knowledge (ZK) rollups in several key ways. One of the unique features of NSSA is its public execution type, which does not +collective preferences.

    +

    To liken this to open-source projects: in traditional systems, developers fix privacy rules much like immutable code—users must comply with these fixed settings. +In contrast, with selective privacy, the rules are malleable and shaped by the users’ preferences, enabling the community to find the ideal balance between privacy and efficiency over time.

    +

    NSSA is distinct from traditional zero-knowledge (ZK) rollups in several key ways. One of the unique features of NSSA is its public execution type, which does not require ZKPs or a zero-knowledge virtual machine (zkVM). This provides a significant advantage in terms of scalability and efficiency as users can choose public executions for -transactions that do not require enhanced privacy, avoiding the overhead associated with ZKP generation and verification.

    Moreover, NSSA introduces two additional execution types—shielded and deshielded—which further distinguish it from traditional privacy-preserving rollups. +transactions that do not require enhanced privacy, avoiding the overhead associated with ZKP generation and verification.

    +

    Moreover, NSSA introduces two additional execution types—shielded and deshielded—which further distinguish it from traditional privacy-preserving rollups. These execution types allow for more nuanced control over privacy, giving users the ability to shield certain aspects of a transaction while deshielding others. -This flexibility sets NSSA apart as a more adaptable and user-centric platform, catering to a wide range of privacy needs without imposing a one-size-fits-all solution.

    By combining selective privacy with a flexible execution model, NSSA offers a more robust and adaptable framework for decentralized applications, -ensuring that users maintain control over their privacy while benefiting from the security and efficiency of blockchain technology.

    How Nescience state-separation architecture can be used​

    NSSA offers a flexible, privacy-preserving add-on that can be applied to existing dapps. +This flexibility sets NSSA apart as a more adaptable and user-centric platform, catering to a wide range of privacy needs without imposing a one-size-fits-all solution.

    +

    By combining selective privacy with a flexible execution model, NSSA offers a more robust and adaptable framework for decentralized applications, +ensuring that users maintain control over their privacy while benefiting from the security and efficiency of blockchain technology.

    +

    How Nescience state-separation architecture can be used​

    +

    NSSA offers a flexible, privacy-preserving add-on that can be applied to existing dapps. One of the emerging trends in the blockchain space is that each dapp is expected to have its own rollup for efficiency, and it is estimated that Ethereum could see the deployment of different rollups in the near future. A key question arises: how many of these rollups will incorporate privacy? For dapp developers who want to offer flexible, -user-centric privacy features, NSSA provides a solution through selective privacy.

    Use case: Adding privacy to existing dapps​

    Consider a dapp running on a transparent network that offers no inherent privacy to its users. Converting this dapp to a privacy-preserving architecture from scratch would +user-centric privacy features, NSSA provides a solution through selective privacy.

    +

    Use case: Adding privacy to existing dapps​

    +

    Consider a dapp running on a transparent network that offers no inherent privacy to its users. Converting this dapp to a privacy-preserving architecture from scratch would require significant effort, restructuring, and a deep understanding of cryptographic frameworks. However, with NSSA, the dapp does not need to undergo extensive changes. -Instead, the Nescience state-separation model can be deployed as an add-on, offering selective privacy as an option for the dapp’s users.

    This allows the dapp to retain its existing functionality while providing users with a choice between the traditional, transparent version and a new version with selective privacy features. +Instead, the Nescience state-separation model can be deployed as an add-on, offering selective privacy as an option for the dapp’s users.

    +

    This allows the dapp to retain its existing functionality while providing users with a choice between the traditional, transparent version and a new version with selective privacy features. With NSSA, the privacy settings are flexible, meaning users can tailor their level of privacy according to their individual needs while the dapp operates on its current infrastructure. -This contrasts sharply with the typical approach, where dapps are either entirely transparent or fully private, with no flexibility for users to select their own privacy preferences.

    Key advantage: Decoupling from the host chain​

    A key feature of NSSA is that it operates independently of the privacy characteristics of the host blockchain. Whether the host chain is fully transparent or fully private, +This contrasts sharply with the typical approach, where dapps are either entirely transparent or fully private, with no flexibility for users to select their own privacy preferences.

    +

    Key advantage: Decoupling from the host chain​

    +

    A key feature of NSSA is that it operates independently of the privacy characteristics of the host blockchain. Whether the host chain is fully transparent or fully private, the Nescience state-separation architecture can be deployed on top of it, offering users the ability to choose their own privacy settings. -This decoupling from the host chain’s inherent privacy model is critical as it allows users to benefit from selective privacy even in environments that were not originally designed to offer it.

    In fully private chains, NSSA allows users to selectively reveal transaction details when compliance with regulations or other requirements is necessary. -In fully transparent chains, NSSA allows users to maintain privacy for specific transactions, offering flexibility that would not otherwise be possible.

    Conclusion​

    NSSA provides a powerful tool for dapp developers who want to offer selective privacy to their users without the need for a complete overhaul of their existing systems. +This decoupling from the host chain’s inherent privacy model is critical as it allows users to benefit from selective privacy even in environments that were not originally designed to offer it.

    +

    In fully private chains, NSSA allows users to selectively reveal transaction details when compliance with regulations or other requirements is necessary. +In fully transparent chains, NSSA allows users to maintain privacy for specific transactions, offering flexibility that would not otherwise be possible.

    +

    Conclusion​

    +

    NSSA provides a powerful tool for dapp developers who want to offer selective privacy to their users without the need for a complete overhaul of their existing systems. By deploying NSSA as an add-on, dapps can give users the ability to choose their own privacy settings whether they are operating on -transparent or private blockchains. This flexibility makes NSSA a valuable option for any dapp looking to provide enhanced privacy options while maintaining efficiency and ease of use.

    B. Design

    In this section, we will delve into the core design components of the Nescience state-separation architecture, covering its key structural elements and the mechanisms -that drive its functionality. We will explore the following topics:

    1. Architecture's components: An in-depth look at the foundational building blocks of NSSA, including the public and private states, UTXO structures, zkVM, and smart contracts. -These components work together to facilitate secure, flexible, and scalable transactions within the architecture.

    2. General execution overview: We will outline the overall flow of transaction execution within NSSA, describing how users interact with the system and how the architecture -supports various types of executions—public, private, shielded, and deshielded—while preserving privacy and efficiency.

    3. Execution processes and UTXO management: This section will focus on the lifecycle of UTXOs within the architecture, from their generation to consumption. -We will also cover the processes involved in managing UTXOs, including proof generation, state transitions, and ensuring transaction validity.

    These topics will provide a comprehensive understanding of how NSSA enables flexible and secure interactions within dapps.

    1. Architecture's components​

    NSSA introduces an advanced prototype execution framework designed to enhance privacy and security in blockchain applications. -This framework integrates several essential components: the public state, private state, zkVM, various execution types, Nescience users, and smart contracts.

    a) Public state​

    The public state in the NSSA is a fundamental component designed to hold all publicly accessible information within +transparent or private blockchains. This flexibility makes NSSA a valuable option for any dapp looking to provide enhanced privacy options while maintaining efficiency and ease of use.

    +

    B. Design

    +

    In this section, we will delve into the core design components of the Nescience state-separation architecture, covering its key structural elements and the mechanisms +that drive its functionality. We will explore the following topics:

    +
      +
    1. +

      Architecture's components: An in-depth look at the foundational building blocks of NSSA, including the public and private states, UTXO structures, zkVM, and smart contracts. +These components work together to facilitate secure, flexible, and scalable transactions within the architecture.

      +
      2. +
    2. +

      General execution overview: We will outline the overall flow of transaction execution within NSSA, describing how users interact with the system and how the architecture +supports various types of executions—public, private, shielded, and deshielded—while preserving privacy and efficiency.

      +
      3. +
    3. +

      Execution processes and UTXO management: This section will focus on the lifecycle of UTXOs within the architecture, from their generation to consumption. +We will also cover the processes involved in managing UTXOs, including proof generation, state transitions, and ensuring transaction validity.

      +
      4. +
    +

    These topics will provide a comprehensive understanding of how NSSA enables flexible and secure interactions within dapps.

    +

    1. Architecture's components​

    +
    +

    NSSA introduces an advanced prototype execution framework designed to enhance privacy and security in blockchain applications. +This framework integrates several essential components: the public state, private state, zkVM, various execution types, Nescience users, and smart contracts.

    +

    a) Public state​

    +
    +

    The public state in the NSSA is a fundamental component designed to hold all publicly accessible information within the blockchain network. This state is organized as a single Merkle tree structure, a sophisticated data structure that ensures efficient and secure data verification. -The public state includes critical information such as user balances and the public storage data of smart contracts.

    In an account-based model, the public state operates by storing each account or smart contract's public data as individual leaf nodes within the Merkle tree. +The public state includes critical information such as user balances and the public storage data of smart contracts.

    +

    In an account-based model, the public state operates by storing each account or smart contract's public data as individual leaf nodes within the Merkle tree. When transactions occur, they directly modify the state by updating these leaf nodes. This direct modification ensures that the most current state of the network -is always reflected accurately.

    The Merkle tree structure is essential for maintaining data integrity. Each leaf node contains a hash of a data block, and each non-leaf node contains the +is always reflected accurately.

    +

    The Merkle tree structure is essential for maintaining data integrity. Each leaf node contains a hash of a data block, and each non-leaf node contains the hash of its child nodes. This hierarchical arrangement means that any change in the data will result in a change in the corresponding hash, making it easy to detect any tampering. The root hash, or Merkle root, is stored on the blockchain, providing a cryptographic guarantee of the data's integrity. This root hash serves as a single, -concise representation of the entire state, enabling quick and reliable verification by any network participant.

    Transparency is a key feature of the public state. All data stored within this state is openly accessible and verifiable by any participant in the network. +concise representation of the entire state, enabling quick and reliable verification by any network participant.

    +

    Transparency is a key feature of the public state. All data stored within this state is openly accessible and verifiable by any participant in the network. This openness ensures that all transactions and state changes are visible and auditable, fostering trust and accountability. For example, user balances are publicly viewable, which helps ensure transparency and trust in the system. Similarly, public smart contract storage can be accessed and verified by anyone, -making it suitable for applications that require public scrutiny and auditability, such as public record updates and some financial transactions.

    The workflow of managing the public state involves several steps to ensure data integrity and transparency. When a user initiates a transaction involving public data, +making it suitable for applications that require public scrutiny and auditability, such as public record updates and some financial transactions.

    +

    The workflow of managing the public state involves several steps to ensure data integrity and transparency. When a user initiates a transaction involving public data, the relevant changes are proposed and applied to the public state tree. The transaction details, such as transferring funds between accounts or updating smart contract storage, update the corresponding leaf nodes in the Merkle tree. Following this, the hashes of the affected nodes are recalculated up to the root, ensuring that the entire tree accurately reflects the new state of the network. The updated Merkle root is then recorded on the blockchain, allowing all network participants to verify the integrity -of the public state. Any discrepancy in the data will result in a mismatched root hash, signaling potential tampering or errors.

    In summary, the public state in NSSA leverages the robustness of the Merkle tree structure to provide a secure, transparent, and verifiable environment for publicly +of the public state. Any discrepancy in the data will result in a mismatched root hash, signaling potential tampering or errors.

    +

    In summary, the public state in NSSA leverages the robustness of the Merkle tree structure to provide a secure, transparent, and verifiable environment for publicly accessible information. By operating on an account-based model and maintaining rigorous data integrity checks, the public state ensures that all transactions are -transparent and trustworthy, laying a strong foundation for a reliable blockchain network.

    b) Private state​

    The private state in the NSSA is a sophisticated system designed to maintain user privacy while ensuring transaction integrity. +transparent and trustworthy, laying a strong foundation for a reliable blockchain network.

    +

    b) Private state​

    +
    +

    The private state in the NSSA is a sophisticated system designed to maintain user privacy while ensuring transaction integrity. Each user has their own individual Merkle tree, which holds their private information such as balances and storage data. This structure is distinct from the public state, which uses an account-based model. Instead, the private state employs a UTXO-based model. In this model, each transaction output is a discrete -unit that can be independently spent in future transactions. This design provides users with granular control over their transaction outputs.

    A key aspect of maintaining privacy within the private state is the use of ZKPs. ZKPs allow transactions to be validated without revealing any +unit that can be independently spent in future transactions. This design provides users with granular control over their transaction outputs.

    +

    A key aspect of maintaining privacy within the private state is the use of ZKPs. ZKPs allow transactions to be validated without revealing any underlying private data. This means that while the system can verify that a transaction is legitimate, the details of the transaction remain confidential. Only parties -with the appropriate viewing key can access and reconstruct the user’s list of UTXOs, ensuring that sensitive information is protected.

    The private state also employs a dual-storage approach to balance privacy and transparency. UTXOs are stored in plaintext within SMTs in the private state, +with the appropriate viewing key can access and reconstruct the user’s list of UTXOs, ensuring that sensitive information is protected.

    +

    The private state also employs a dual-storage approach to balance privacy and transparency. UTXOs are stored in plaintext within SMTs in the private state, providing detailed and accessible records for the user. In contrast, the public state only holds hashes of these UTXOs. This method ensures that while the public can verify -the existence and integrity of private transactions through these hashes, they cannot access the specific details.

    The workflow for a transaction in the private state begins with the user initiating a transaction involving their private data, such as transferring a private balance or +the existence and integrity of private transactions through these hashes, they cannot access the specific details.

    +

    The workflow for a transaction in the private state begins with the user initiating a transaction involving their private data, such as transferring a private balance or updating private smart contract storage. The transaction involves spending existing UTXOs, represented as leaves in the Merkle tree, and creating new UTXOs, which are then appended to the user’s private list. The zkVM generates a ZKP to validate the transaction without revealing -any private data, ensuring the transaction adheres to the system's rules.

    Once the proof is generated, it is submitted to the sequencer, which verifies the transaction’s validity. Upon successful verification, the nullifier is added to the nullifier set, -preventing double spending of the same UTXO. The use of ZKPs and nullifiers ensures that the private state maintains both security and privacy.

    In summary, the private state in NSSA is meticulously designed to provide users with control over their private information while ensuring the security and integrity of transactions. +any private data, ensuring the transaction adheres to the system's rules.

    +

    Once the proof is generated, it is submitted to the sequencer, which verifies the transaction’s validity. Upon successful verification, the nullifier is added to the nullifier set, +preventing double spending of the same UTXO. The use of ZKPs and nullifiers ensures that the private state maintains both security and privacy.

    +

    In summary, the private state in NSSA is meticulously designed to provide users with control over their private information while ensuring the security and integrity of transactions. By utilizing a UTXO-based model, individual Merkle trees, ZKPs, and a dual-storage system, NSSA achieves a balance between confidentiality and verifiability, -making it a robust solution for managing private blockchain transactions.

    c) ZkVM (zero-knowledge virtual machine)​

    The zkVM is a pivotal component in NSSA, designed to uphold the highest standards +making it a robust solution for managing private blockchain transactions.

    +

    c) ZkVM (zero-knowledge virtual machine)​

    +
    +

    The zkVM is a pivotal component in NSSA, designed to uphold the highest standards of privacy and security in blockchain transactions. Its primary function is to generate and aggregate ZKPs, enabling users to validate the correctness of their transactions without disclosing any underlying details. This capability is crucial for maintaining the confidentiality and integrity of sensitive -data within the blockchain network.

    ZKPs are sophisticated cryptographic protocols that allow one party, the prover, to convince another party, the verifier, that a certain statement is true, +data within the blockchain network.

    +

    ZKPs are sophisticated cryptographic protocols that allow one party, the prover, to convince another party, the verifier, that a certain statement is true, without revealing any information beyond the validity of the statement itself. In the context of the zkVM, this means users can prove their transactions are valid without -exposing transaction specifics, such as amounts or parties involved. This process is essential for transactions within the private state, where maintaining confidentiality is paramount.

    The generation of ZKPs involves intricate cryptographic computations. When a user initiates a transaction, the zkVM processes the transaction inputs and produces a proof +exposing transaction specifics, such as amounts or parties involved. This process is essential for transactions within the private state, where maintaining confidentiality is paramount.

    +

    The generation of ZKPs involves intricate cryptographic computations. When a user initiates a transaction, the zkVM processes the transaction inputs and produces a proof that the transaction adheres to the protocol's rules. This proof must be robust enough to convince the verifier of the transaction's validity while preserving the privacy -of the transaction details.

    Performance optimization is another critical function of the zkVM. In a typical blockchain scenario, verifying multiple individual proofs can be computationally intensive +of the transaction details.

    +

    Performance optimization is another critical function of the zkVM. In a typical blockchain scenario, verifying multiple individual proofs can be computationally intensive and time consuming, potentially leading to network congestion and delays. To address this, the zkVM can aggregate multiple ZKPs into a single, consolidated proof. This aggregation significantly reduces the verification overhead as the verifier needs to check only one comprehensive proof rather than multiple individual ones. -This efficiency is vital for maintaining high throughput and low latency in the blockchain network, ensuring that the system can handle a large volume of transactions swiftly and securely.

    Furthermore, the zkVM's role extends beyond mere proof generation and aggregation. It also ensures that all transactions meet the required privacy and security standards +This efficiency is vital for maintaining high throughput and low latency in the blockchain network, ensuring that the system can handle a large volume of transactions swiftly and securely.

    +

    Furthermore, the zkVM's role extends beyond mere proof generation and aggregation. It also ensures that all transactions meet the required privacy and security standards before they are recorded on the blockchain. By interacting seamlessly with other components such as the public and private states, the zkVM ensures that any transaction, -whether it involves public data, private data, or a mix of both, is thoroughly validated and secured.

    In summary, the zkVM is essential for the NSSA, providing the cryptographic backbone necessary to support secure and private transactions. Its ability to generate and +whether it involves public data, private data, or a mix of both, is thoroughly validated and secured.

    +

    In summary, the zkVM is essential for the NSSA, providing the cryptographic backbone necessary to support secure and private transactions. Its ability to generate and aggregate ZKPs not only preserves the confidentiality of user data but also enhances the overall efficiency and scalability of the blockchain network. -By ensuring that all transactions are validated without revealing sensitive information, the zkVM upholds the integrity and trustworthiness of the Nescience blockchain system.

    d) Execution types in NSSA​

    NSSA incorporates multiple execution types to cater to varying levels of privacy and security requirements. -These execution types—public, private, shielded, and deshielded—are designed to provide users with flexible options for managing their transactions based on their specific privacy needs.

    1. Public executions​

    Public executions are straightforward transactions that involve reading from and writing to the public state. In this model, data is openly accessible and verifiable +By ensuring that all transactions are validated without revealing sensitive information, the zkVM upholds the integrity and trustworthiness of the Nescience blockchain system.

    +

    d) Execution types in NSSA​

    +
    +

    NSSA incorporates multiple execution types to cater to varying levels of privacy and security requirements. +These execution types—public, private, shielded, and deshielded—are designed to provide users with flexible options for managing their transactions based on their specific privacy needs.

    +

    1. Public executions​

    +

    Public executions are straightforward transactions that involve reading from and writing to the public state. In this model, data is openly accessible and verifiable by all participants in the network. Public executions do not require ZKPs since transparency is the primary goal. This execution type is ideal -for non-sensitive transactions where public visibility is beneficial, such as updating public records, performing open financial transactions, or interacting with public smart contracts.

    The workflow for a public execution starts with a user initiating a transaction that modifies public data. The transaction details are then used to update the relevant +for non-sensitive transactions where public visibility is beneficial, such as updating public records, performing open financial transactions, or interacting with public smart contracts.

    +

    The workflow for a public execution starts with a user initiating a transaction that modifies public data. The transaction details are then used to update the relevant leaf nodes in the Merkle tree. As changes are made, the hashes of affected nodes are recalculated up to the root, ensuring that the entire tree reflects the most recent state. -Finally, the updated Merkle root is recorded on the blockchain, making the new state publicly verifiable.

    2. Private executions​

    Private executions are designed for confidential transactions, reading from and writing to the private state. These transactions require ZKPs to ensure that while the +Finally, the updated Merkle root is recorded on the blockchain, making the new state publicly verifiable.

    +

    2. Private executions​

    +

    Private executions are designed for confidential transactions, reading from and writing to the private state. These transactions require ZKPs to ensure that while the transaction details are validated, the actual data remains private. This execution type is suitable for scenarios where privacy is crucial, such as private financial -transactions or sensitive data management within smart contracts.

    In a private execution, the user initiates a transaction involving private data. The transaction spends existing UTXOs and creates new ones, all of which are represented as +transactions or sensitive data management within smart contracts.

    +

    In a private execution, the user initiates a transaction involving private data. The transaction spends existing UTXOs and creates new ones, all of which are represented as leaves in the Merkle tree. The zkVM generates a ZKP to validate the transaction without revealing private data. This proof is submitted to the sequencer, which verifies the proof to ensure the transaction's validity. Upon successful verification, the nullifier is added to the nullifier set, and the private state is updated -with the new Merkle root.

    3. Shielded executions​

    Shielded executions create a layer of privacy for the outputs by allowing interactions between the public and private states. When a transaction occurs in a shielded execution, +with the new Merkle root.

    +

    3. Shielded executions​

    +

    Shielded executions create a layer of privacy for the outputs by allowing interactions between the public and private states. When a transaction occurs in a shielded execution, details of the transaction are processed within the private state, ensuring that sensitive information remains confidential. Only the necessary details are shared with the public state, -often in a masked or encrypted form. This approach allows for the validation of the transaction without revealing critical data, thus preserving the privacy of the involved parties.

    The workflow for shielded executions begins with the user initiating a transaction that reads from the public state and prepares to write to the private state. Public data is accessed, +often in a masked or encrypted form. This approach allows for the validation of the transaction without revealing critical data, thus preserving the privacy of the involved parties.

    +

    The workflow for shielded executions begins with the user initiating a transaction that reads from the public state and prepares to write to the private state. Public data is accessed, and the private state is prepared to receive new data. The zkVM generates a ZKP to hide the receiver’s identity. This proof is submitted to the sequencer, which verifies the proof to ensure the transaction's validity. If valid, the private state is updated with the new data while the public state reflects the change without revealing private details. -This type of execution is particularly useful for scenarios where the receiver’s identity needs to be hidden, such as in anonymous donation systems or confidential data storage.

    4. Deshielded executions​

    Deshielded executions operate in the opposite manner of shielded executions, where data is read from the private state and written to the public state. This execution type is useful -in situations where the sender's identity needs to be kept confidential while making the transaction results publicly visible.

    In a deshielded execution, the user initiates a transaction that reads from the private state and prepares to write to the public state. Private data is accessed, +This type of execution is particularly useful for scenarios where the receiver’s identity needs to be hidden, such as in anonymous donation systems or confidential data storage.

    +

    4. Deshielded executions​

    +

    Deshielded executions operate in the opposite manner of shielded executions, where data is read from the private state and written to the public state. This execution type is useful +in situations where the sender's identity needs to be kept confidential while making the transaction results publicly visible.

    +

    In a deshielded execution, the user initiates a transaction that reads from the private state and prepares to write to the public state. Private data is accessed, and the transaction details are prepared. The zkVM generates a ZKP to hide the sender’s identity. This proof is then submitted to the sequencer, which verifies the proof to ensure the transaction's validity. Once verified, the public state is updated with the new data, reflecting the change while keeping the sender’s details confidential. This can be useful when transparency is needed, such as when auditing or proving certain aspects of a transaction to a wider audience. By selectively deshielding certain transactions, users can control what information is shared publicly, thus maintaining a balance between privacy and transparency -as required by their specific use case.

    Table of execution types​

    TypeRead fromWrite toZKP requiredUse caseDescription
    PublicPublic statePublic stateNoNon-sensitive transactions requiring transparency.Ideal for transactions that do not require privacy, ensuring full transparency.
    PrivatePrivate statePrivate stateYesConfidential transactions needing privacy.Suitable for transactions that require confidentiality. Ensures that transaction details remain private through the use of ZKPs.
    ShieldedPublic statePrivate stateYesTransactions where the receiver’s identity needs to be hidden.Hides the identity of the receiver while keeping the transaction details private. Suitable for anonymous donations or confidential data storage.
    DeshieldedPrivate statePublic stateYesTransactions where the sender’s identity needs to be hidden.Ensures the sender’s identity remains confidential while making the transaction results public. Suitable for confidential disbursements or anonymized data publication.

    By supporting a range of execution types, NSSA provides a flexible and robust framework for managing privacy and security in blockchain transactions. +as required by their specific use case.

    +

    Table of execution types​

    +
    TypeRead fromWrite toZKP requiredUse caseDescription
    PublicPublic statePublic stateNoNon-sensitive transactions requiring transparency.Ideal for transactions that do not require privacy, ensuring full transparency.
    PrivatePrivate statePrivate stateYesConfidential transactions needing privacy.Suitable for transactions that require confidentiality. Ensures that transaction details remain private through the use of ZKPs.
    ShieldedPublic statePrivate stateYesTransactions where the receiver’s identity needs to be hidden.Hides the identity of the receiver while keeping the transaction details private. Suitable for anonymous donations or confidential data storage.
    DeshieldedPrivate statePublic stateYesTransactions where the sender’s identity needs to be hidden.Ensures the sender’s identity remains confidential while making the transaction results public. Suitable for confidential disbursements or anonymized data publication.
    +
    +

    By supporting a range of execution types, NSSA provides a flexible and robust framework for managing privacy and security in blockchain transactions. Whether the need is for complete transparency, total privacy, or a balanced approach, NSSA's execution types allow users to select the level of confidentiality -that best fits their requirements. This flexibility enhances the overall utility of the blockchain, making it suitable for a wide array of applications and use cases.

    e) Nescience users​

    Nescience users are integral to the architecture, managing balances and assets within the blockchain network and invoking smart contracts with various privacy options. -They can choose the appropriate execution type—public, private, shielded, or deshielded—based on their specific privacy and security needs.

    Users handle both public and private balances. Public balances are visible to all network participants and suitable for non-sensitive transactions, +that best fits their requirements. This flexibility enhances the overall utility of the blockchain, making it suitable for a wide array of applications and use cases.

    +

    e) Nescience users​

    +
    +

    Nescience users are integral to the architecture, managing balances and assets within the blockchain network and invoking smart contracts with various privacy options. +They can choose the appropriate execution type—public, private, shielded, or deshielded—based on their specific privacy and security needs.

    +

    Users handle both public and private balances. Public balances are visible to all network participants and suitable for non-sensitive transactions, while private balances are confidential and used for transactions requiring privacy. Digital wallets provide a user-friendly interface for managing -these balances, assets, and transactions, allowing users to select the desired execution type seamlessly.

    Security is ensured through the use of cryptographic keys, which authenticate and verify transactions. ZKPs maintain privacy -by validating transaction correctness without revealing underlying data, ensuring sensitive information remains confidential even during verification.

    The workflow for users involves initiating a transaction, preparing inputs, interacting with smart contracts, generating proofs if needed, +these balances, assets, and transactions, allowing users to select the desired execution type seamlessly.

    +

    Security is ensured through the use of cryptographic keys, which authenticate and verify transactions. ZKPs maintain privacy +by validating transaction correctness without revealing underlying data, ensuring sensitive information remains confidential even during verification.

    +

    The workflow for users involves initiating a transaction, preparing inputs, interacting with smart contracts, generating proofs if needed, and submitting the transaction to the sequencer for verification and state update. This flexible approach supports various use cases, -from financial transactions and decentralized applications to data privacy management, allowing users to maintain control over their privacy settings.

    By offering this high degree of flexibility and security, Nescience enables users to tailor their privacy settings to their specific needs, +from financial transactions and decentralized applications to data privacy management, allowing users to maintain control over their privacy settings.

    +

    By offering this high degree of flexibility and security, Nescience enables users to tailor their privacy settings to their specific needs, ensuring sensitive transactions remain confidential while non-sensitive ones are transparent. This integration of cryptographic keys and ZKPs -provides a robust framework for a wide range of blockchain applications, enhancing both utility and trust within the network.

    f) Smart contracts in NSSA​

    Smart contracts are a core feature of NSSA, providing a way to automate and execute predefined actions based on coded rules. +provides a robust framework for a wide range of blockchain applications, enhancing both utility and trust within the network.

    +

    f) Smart contracts in NSSA​

    +
    +

    Smart contracts are a core feature of NSSA, providing a way to automate and execute predefined actions based on coded rules. Once deployed on the blockchain, these contracts become immutable, meaning their behavior cannot be altered. This ensures that they perform exactly as intended without the risk of tampering. Because the state and data of the contract are stored permanently on the blockchain, all interactions are fully -transparent and auditable, creating a reliable and trustworthy environment.

    One of the key strengths of smart contracts is their ability to automate processes. They are designed to automatically execute when specific conditions are met, +transparent and auditable, creating a reliable and trustworthy environment.

    +

    One of the key strengths of smart contracts is their ability to automate processes. They are designed to automatically execute when specific conditions are met, reducing the need for manual oversight or intermediaries. For example, a smart contract might transfer funds when a certain deadline is reached or update a record -once a task is completed. This self-executing nature makes them efficient and minimizes human error.

    Smart contracts operate deterministically, meaning they will always produce the same result given the same inputs. This predictability is crucial for ensuring reliability, -especially in complex systems. Additionally, they run in isolated environments on the blockchain, which enhances security by preventing unintended interactions with other processes.

    Security is another critical feature of smart contracts. They leverage the underlying cryptographic protections of the blockchain, ensuring that every interaction +once a task is completed. This self-executing nature makes them efficient and minimizes human error.

    +

    Smart contracts operate deterministically, meaning they will always produce the same result given the same inputs. This predictability is crucial for ensuring reliability, +especially in complex systems. Additionally, they run in isolated environments on the blockchain, which enhances security by preventing unintended interactions with other processes.

    +

    Security is another critical feature of smart contracts. They leverage the underlying cryptographic protections of the blockchain, ensuring that every interaction is secure and authenticated. Before deployment, the contract code can be audited and verified to ensure it functions correctly. Once on the blockchain, -the immutable nature of the code prevents unauthorized modifications, further ensuring the integrity of the system.

    Running smart contracts requires computational resources, which are compensated through gas fees. These fees vary depending on the complexity of the operations within the contract. -More resource-intensive contracts incur higher fees, which helps manage the computational load on the blockchain network.

    The workflow of a smart contract begins with its development, where developers code the contract using languages like Rust. Once the code is compiled and deployed to the blockchain, +the immutable nature of the code prevents unauthorized modifications, further ensuring the integrity of the system.

    +

    Running smart contracts requires computational resources, which are compensated through gas fees. These fees vary depending on the complexity of the operations within the contract. +More resource-intensive contracts incur higher fees, which helps manage the computational load on the blockchain network.

    +

    The workflow of a smart contract begins with its development, where developers code the contract using languages like Rust. Once the code is compiled and deployed to the blockchain, it becomes a permanent part of the network. Users can then interact with the contract by sending transactions that invoke specific functions. The contract checks whether the -required conditions are met, and if so, it automatically executes the specified actions, such as transferring tokens or updating data on the blockchain.

    The benefits of smart contracts are numerous. They eliminate the need for intermediaries by providing a system where trust is built into the code itself. +required conditions are met, and if so, it automatically executes the specified actions, such as transferring tokens or updating data on the blockchain.

    +

    The benefits of smart contracts are numerous. They eliminate the need for intermediaries by providing a system where trust is built into the code itself. This not only reduces costs but also increases efficiency by automating repetitive processes. The inherent security of smart contracts, combined with their transparency—where every action is recorded and visible on the blockchain—makes them a powerful tool for ensuring accountability and trust in decentralized systems. -They can be ideal for managing decentralized autonomous organizations (DAOs), where governance decisions are automated through coded rules.

    By integrating smart contracts, NSSA offers a highly versatile, secure, and transparent framework that can support a wide range of applications -across various industries, from finance to governance, supply chains, and more.

    2. General execution overview​

    This section explains the execution process within NSSA, providing an overview of how it works from start to finish. -It outlines the steps involved in each execution type, guiding the reader through the entire process from user interaction to completion.

    The process begins when a user initiates a transaction by invoking a smart contract. This invocation involves selecting at least one of +They can be ideal for managing decentralized autonomous organizations (DAOs), where governance decisions are automated through coded rules.

    +

    By integrating smart contracts, NSSA offers a highly versatile, secure, and transparent framework that can support a wide range of applications +across various industries, from finance to governance, supply chains, and more.

    +

    2. General execution overview​

    +
    +

    This section explains the execution process within NSSA, providing an overview of how it works from start to finish. +It outlines the steps involved in each execution type, guiding the reader through the entire process from user interaction to completion.

    +

    The process begins when a user initiates a transaction by invoking a smart contract. This invocation involves selecting at least one of the four execution types: public, private, shielded, or deshielded. The choice of execution type determines how data will be read from and written to the blockchain, affecting the transaction's privacy and security levels. Each execution type caters to different privacy needs, allowing the user to tailor the transaction according -to their specific requirements, whether it be full transparency or complete confidentiality.

    general

    User actions​

    Step 1: Smart contract selection and input creation

    • Smart contract selection: The user selects a smart contract they wish to invoke.
    • Input creation: The user creates a set of inputs required for the invocation by reading the necessary data from both the public and private states. This includes:
      • Public data such as current account balances, public keys, and smart contract states.
      • Private data such as private account balances and UTXOs.

    Step 2: Choosing execution type

    • Execution type selection: The user selects the type of execution based on their privacy needs. The options include:
      • Public execution: Suitable for transactions where transparency is desired.
      • Private execution: Used when transaction details need to be confidential.
      • Shielded execution: Hides the receiver's identity.
      • Deshielded execution: Hides the sender's identity.
    • ZkVM requirement: If the execution involves private, shielded, or deshielded types, the user must call the zkVM to handle these confidential transactions. -For purely public executions, the zkVM is not needed, and the user can directly transmit the transaction code to the sequencer.

    Step 3: Calling zkVM for proof generation

    • ZkVM compilation: The user calls the zkVM to compile the smart contract with both public and private inputs.
      • Kernel circuit proofs: The zkVM generates individual proofs for each execution type through kernel circuits.
      • Proof aggregation: The zkVM aggregates these individual proofs into a single comprehensive proof, combining both private and public inputs.

    Step 4: Transmitting public inputs and retaining private inputs

    • Retaining private inputs: The user keeps the private inputs secure and does not transmit them.
    • Revealing public inputs: The user transmits the following public inputs to the sequencer:
      • Public inputs of the recursive proof
      • Hashes of UTXOs
      • Updates to the public state
      • Transaction signature
      • Nullifiers (to prevent double spending)

    After completing these steps, the user's part of the execution is done, and the sequencer takes over the process.

    Sequencer actions​

    Step 5: Proof verification

    • Proof and data reception: The sequencer receives the proof and public inputs from the user.
    • Verification process:
      • For private, shielded, and deshielded executions, the sequencer verifies the proof using the provided public data.
      • For public executions, the sequencer reruns the smart contract code with the provided inputs to check the results.
    • Validation: If both the zkVM proofs and public execution results are verified successfully, the sequencer collects the proof and public data to proceed. -If verification fails, the process is aborted, and the transaction is rejected.

    Step 6: Aggregating proofs and finalizing the block

    • Proof aggregation: The sequencer calls the zkVM again to aggregate all received proofs into one comprehensive proof to finalize the block.
    • Finalizing the block:
      • Public state update: The sequencer updates the public state with the new transaction data.
      • Nullifier tree update: Updates the nullifier tree to reflect the new state and prevent double spending.
      • Synchronization mechanism: Runs synchronization mechanisms to ensure fairness and consistency across the network.
      • UTXO validation: Validates the exchanged UTXOs to complete the transaction process.

    This comprehensive process ensures that transactions are executed securely, with the appropriate level of privacy and state updates synchronized across the network.

    Below, we outline the execution process of the four different execution types within NSSA:

    • Public execution:

    public

    • Private execution:

    private

    • Shielded execution:

    shielded

    • Deshielded execution:

    deshielded

    3. Execution processes and UTXO management​

    In Nescience state-separation architecture, UTXOs are key components for managing private data and assets. They serve as private entities that hold both storage and assets, +to their specific requirements, whether it be full transparency or complete confidentiality.

    +

    general

    +

    User actions​

    +
    +

    Step 1: Smart contract selection and input creation

    +
      +
    • Smart contract selection: The user selects a smart contract they wish to invoke.
      • +
    • Input creation: The user creates a set of inputs required for the invocation by reading the necessary data from both the public and private states. This includes: +
        +
      • Public data such as current account balances, public keys, and smart contract states.
        • +
      • Private data such as private account balances and UTXOs.
        • +
      +
      • +
    +

    Step 2: Choosing execution type

    +
      +
    • Execution type selection: The user selects the type of execution based on their privacy needs. The options include: +
        +
      • Public execution: Suitable for transactions where transparency is desired.
        • +
      • Private execution: Used when transaction details need to be confidential.
        • +
      • Shielded execution: Hides the receiver's identity.
        • +
      • Deshielded execution: Hides the sender's identity.
        • +
      +
      • +
    • ZkVM requirement: If the execution involves private, shielded, or deshielded types, the user must call the zkVM to handle these confidential transactions. +For purely public executions, the zkVM is not needed, and the user can directly transmit the transaction code to the sequencer.
      • +
    +

    Step 3: Calling zkVM for proof generation

    +
      +
    • ZkVM compilation: The user calls the zkVM to compile the smart contract with both public and private inputs. +
        +
      • Kernel circuit proofs: The zkVM generates individual proofs for each execution type through kernel circuits.
        • +
      • Proof aggregation: The zkVM aggregates these individual proofs into a single comprehensive proof, combining both private and public inputs.
        • +
      +
      • +
    +

    Step 4: Transmitting public inputs and retaining private inputs

    +
      +
    • Retaining private inputs: The user keeps the private inputs secure and does not transmit them.
      • +
    • Revealing public inputs: The user transmits the following public inputs to the sequencer: +
        +
      • Public inputs of the recursive proof
        • +
      • Hashes of UTXOs
        • +
      • Updates to the public state
        • +
      • Transaction signature
        • +
      • Nullifiers (to prevent double spending)
        • +
      +
      • +
    +

    After completing these steps, the user's part of the execution is done, and the sequencer takes over the process.

    +

    Sequencer actions​

    +
    +

    Step 5: Proof verification

    +
      +
    • Proof and data reception: The sequencer receives the proof and public inputs from the user.
      • +
    • Verification process: +
        +
      • For private, shielded, and deshielded executions, the sequencer verifies the proof using the provided public data.
        • +
      • For public executions, the sequencer reruns the smart contract code with the provided inputs to check the results.
        • +
      +
      • +
    • Validation: If both the zkVM proofs and public execution results are verified successfully, the sequencer collects the proof and public data to proceed. +If verification fails, the process is aborted, and the transaction is rejected.
      • +
    +

    Step 6: Aggregating proofs and finalizing the block

    +
      +
    • Proof aggregation: The sequencer calls the zkVM again to aggregate all received proofs into one comprehensive proof to finalize the block.
      • +
    • Finalizing the block: +
        +
      • Public state update: The sequencer updates the public state with the new transaction data.
        • +
      • Nullifier tree update: Updates the nullifier tree to reflect the new state and prevent double spending.
        • +
      • Synchronization mechanism: Runs synchronization mechanisms to ensure fairness and consistency across the network.
        • +
      • UTXO validation: Validates the exchanged UTXOs to complete the transaction process.
        • +
      +
      • +
    +

    This comprehensive process ensures that transactions are executed securely, with the appropriate level of privacy and state updates synchronized across the network.

    +

    Below, we outline the execution process of the four different execution types within NSSA:

    +
      +
    • Public execution:
      • +
    +

    public

    +
      +
    • Private execution:
      • +
    +

    private

    +
      +
    • Shielded execution:
      • +
    +

    shielded

    +
      +
    • Deshielded execution:
      • +
    +

    deshielded

    +

    3. Execution processes and UTXO management​

    +
    +

    In Nescience state-separation architecture, UTXOs are key components for managing private data and assets. They serve as private entities that hold both storage and assets, facilitating secure and confidential transactions. UTXOs are utilized in three of the four execution types within NSSA: private, shielded, -and deshielded executions. This section explores the lifecycle of UTXOs, detailing their generation, transfer, encryption, and eventual consumption within the private execution framework.

    a) Components of a Nescience UTXO​

    A Nescience UTXO is a critical and versatile component of the private state in the Nescience state-separation architecture. +and deshielded executions. This section explores the lifecycle of UTXOs, detailing their generation, transfer, encryption, and eventual consumption within the private execution framework.

    +

    a) Components of a Nescience UTXO​

    +
    +

    A Nescience UTXO is a critical and versatile component of the private state in the Nescience state-separation architecture. It carries essential information that ensures its proper functionality within private execution, such as the owner, value, private storage slot, non-fungibles, -and other cryptographic components. Below is a detailed breakdown of each component and its role in maintaining the integrity, security, and privacy of the system:

    • Owner: +and other cryptographic components. Below is a detailed breakdown of each component and its role in maintaining the integrity, security, and privacy of the system:

      +
        +
      • +

        Owner: The owner component represents the public key of the entity that controls the UTXO. Only the owner can spend this UTXO, ensuring its security and privacy through public key cryptography. This means that the UTXO remains secure as only the rightful owner, using their private key, can generate valid signatures to authorize the transaction. For example, if Alice owns a UTXO linked to her public key, she must sign any transaction to spend it using her private key. This cryptographic protection ensures that only Alice can authorize -spending the UTXO and transfer it to someone else, such as Bob.

      • Value: +spending the UTXO and transfer it to someone else, such as Bob.

        +
        • +
      • +

        Value: The value in a UTXO represents the balance or asset contained within it. This could be cryptocurrency, tokens, or other digital assets. The value ensures accurate accounting, preventing double spending and maintaining the overall integrity of the system. For instance, if Alice's UTXO has a value of 10 tokens, this represents her ownership of that amount -within the network, and when spent, this value will be deducted from her UTXO and transferred accordingly.

      • Private storage slot: +within the network, and when spent, this value will be deducted from her UTXO and transferred accordingly.

        +
        • +
      • +

        Private storage slot: The private storage slot is an arbitrary and flexible storage space within the UTXO for Nescience applications. It allows users and smart contracts to store additional private data that is only accessible by the owner. This could be used to hold metadata, smart contract states, or user-specific information. For example, if a smart contract is holding private user data, -this information is securely stored in the private storage slot and can only be accessed or modified by the owner, ensuring privacy and security.

      • Non-fungibles: +this information is securely stored in the private storage slot and can only be accessed or modified by the owner, ensuring privacy and security.

        +
        • +
      • +

        Non-fungibles: Non-fungibles within the UTXO represent unique assets, such as NFTs (Non-Fungible Tokens). Each non-fungible asset is assigned a unique serial number or identifier within the UTXO, ensuring its distinctiveness and traceability. For example, if Alice owns a digital artwork represented as an NFT, the non-fungible component of the UTXO will store the unique identifier -for this NFT, preventing duplication or forgery of the digital asset.

      • Random commitment key: +for this NFT, preventing duplication or forgery of the digital asset.

        +
        • +
      • +

        Random commitment key: The random commitment key (RCK) is a randomly generated number used to create a cryptographic commitment to the contents of the UTXO. This commitment ensures the integrity of the data without revealing any private information. By generating a random key for the commitment, the system ensures that even if someone observes the commitment, they cannot infer any details -about the underlying UTXO. For example, RCK helps maintain confidentiality in the system while still allowing the verification of transactions.

      • Nullifier key: +about the underlying UTXO. For example, RCK helps maintain confidentiality in the system while still allowing the verification of transactions.

        +
        • +
      • +

        Nullifier key: The Nullifier key is another randomly generated number, used to ensure that a UTXO is only spent once. When a UTXO is spent, its nullifier key is recorded in a nullifier set to prevent double spending. This key guarantees that once a UTXO is spent, it cannot be reused in another transaction, effectively nullifying it from future use. This mechanism is crucial for -maintaining the security and integrity of the system, as it ensures that no UTXO can be spent more than once.

      b) UTXO lifecycle: From generation to consumption​

      UTXOs in NSSA are created when a transaction outputs a specific value, asset, or data intended for future use. Once generated, these UTXOs become private entities -owned by specific users, containing sensitive information such as balances, private data, or unique assets like NFTs.

      To maintain the required level of confidentiality, UTXOs are encrypted and transferred anonymously across the network. This encryption process ensures that the data within each UTXO +maintaining the security and integrity of the system, as it ensures that no UTXO can be spent more than once.

      +
      • +
    +

    b) UTXO lifecycle: From generation to consumption​

    +
    +

    UTXOs in NSSA are created when a transaction outputs a specific value, asset, or data intended for future use. Once generated, these UTXOs become private entities +owned by specific users, containing sensitive information such as balances, private data, or unique assets like NFTs.

    +

    To maintain the required level of confidentiality, UTXOs are encrypted and transferred anonymously across the network. This encryption process ensures that the data within each UTXO remains hidden from network participants, including the sequencer, while still allowing for verification and validation through ZKPs. These proofs enable the network -to ensure that UTXOs are valid, prevent double spending, and maintain security, all without revealing any sensitive information.

    When a user wishes to spend or transfer a UTXO, the lifecycle progresses towards its consumption. The user must prove ownership and validity of the UTXO through a ZKP, +to ensure that UTXOs are valid, prevent double spending, and maintain security, all without revealing any sensitive information.

    +

    When a user wishes to spend or transfer a UTXO, the lifecycle progresses towards its consumption. The user must prove ownership and validity of the UTXO through a ZKP, which is then verified by the sequencer. This process occurs in private, shielded, and deshielded executions, where confidentiality is a priority. Once the proof is validated, -the UTXO is consumed, meaning it is marked as spent and cannot be reused, ensuring the integrity of the transaction and preventing double spending.

    UTXOs are central to the private, shielded, and deshielded execution types in Nescience. In private executions, UTXOs are transferred securely between parties without revealing any +the UTXO is consumed, meaning it is marked as spent and cannot be reused, ensuring the integrity of the transaction and preventing double spending.

    +

    UTXOs are central to the private, shielded, and deshielded execution types in Nescience. In private executions, UTXOs are transferred securely between parties without revealing any details to the public state. In shielded executions, UTXOs are used to receive assets from the public state while keeping the recipient's identity confidential. Finally, -in deshielded executions, UTXOs are used to send assets from the private state to the public state, while preserving the sender's anonymity.

    Since UTXOs are not exchanged in public executions, this lifecycle analysis is focused solely on private, shielded, and deshielded executions, where privacy and confidentiality are essential. +in deshielded executions, UTXOs are used to send assets from the private state to the public state, while preserving the sender's anonymity.

    +

    Since UTXOs are not exchanged in public executions, this lifecycle analysis is focused solely on private, shielded, and deshielded executions, where privacy and confidentiality are essential. In these contexts, the careful management and transfer of UTXOs ensure that the users' private data and assets remain secure, while still allowing for seamless and confidential transactions -within the network.

    At this point, it's crucial to introduce two key components that will play a significant role in the next section: the ephemeral key and the nillifier.

    • Ephemeral key: The ephemeral key is embedded in the transaction message and plays a crucial role in maintaining privacy. It is used by the sender, alongside the receiver's public key, +within the network.

      +

      At this point, it's crucial to introduce two key components that will play a significant role in the next section: the ephemeral key and the nillifier.

      +
        +
      • +

        Ephemeral key: The ephemeral key is embedded in the transaction message and plays a crucial role in maintaining privacy. It is used by the sender, alongside the receiver's public key, in a key agreement protocol to derive a shared secret. This shared secret is then employed to encrypt the transaction details, ensuring that only those with the receiver's viewing key can decrypt the transaction. By using the ephemeral key, the receiver can regenerate the shared secret, granting access to the transaction's contents. The sender generates the ephemeral key -using their spending key and the UTXO's nullifier, reinforcing the security of the transaction. (more details in key management and addresses section)

      • Nullifier: A nullifier is a unique value tied to a specific UTXO, ensuring that it has not been spent before. Its uniqueness is essential, as a nullifier must never correspond to more +using their spending key and the UTXO's nullifier, reinforcing the security of the transaction. (more details in key management and addresses section)

        +
        • +
      • +

        Nullifier: A nullifier is a unique value tied to a specific UTXO, ensuring that it has not been spent before. Its uniqueness is essential, as a nullifier must never correspond to more than one UTXO—otherwise, even if both UTXOs are valid, only one could be spent. This would undermine the integrity of the system. To spend a UTXO, a proof must be provided showing that the nullifier does not already exist in the Nullifier Tree. Once the transaction is confirmed and included in the blockchain, the nullifier is added to the Nullifier Tree, preventing any future reuse of the same UTXO. A UTXO's nullifier is generated by combining the receiver's nullifier key with the transaction note's commitment, further ensuring its distinctiveness -and security. (More details in nullifier tree section.)

      I) UTXOs in private executions​

      In private executions within NSSA, transactions are handled ensuring maximum privacy by concealing all transaction details from the public state. +and security. (More details in nullifier tree section.)

      +
      • +
    +

    I) UTXOs in private executions​

    +
    +

    In private executions within NSSA, transactions are handled ensuring maximum privacy by concealing all transaction details from the public state. This approach is particularly useful for confidential payments, where the identities of the sender and receiver, as well as the transaction amounts, must remain hidden. -The process is powered by ZKPs, ensuring that only the involved parties have access to the transaction details while maintaining the integrity of the network.

    • Stages of private execution: Private executions operate in two key stages: UTXO consumption and UTXO creation. In the first stage, UTXOs from the private state are used +The process is powered by ZKPs, ensuring that only the involved parties have access to the transaction details while maintaining the integrity of the network.

      +
        +
      • +

        Stages of private execution: Private executions operate in two key stages: UTXO consumption and UTXO creation. In the first stage, UTXOs from the private state are used as inputs for the transaction. In the second stage, new UTXOs are generated as outputs and stored back in the private state. Throughout this process, the details of the -transaction are kept confidential and only shared between the sender and receiver.

      • Private transaction workflow (transaction initialization): The user initiates a private transaction by selecting the input UTXOs that will be spent and determining the +transaction are kept confidential and only shared between the sender and receiver.

        +
        • +
      • +

        Private transaction workflow (transaction initialization): The user initiates a private transaction by selecting the input UTXOs that will be spent and determining the output UTXOs to be created. This involves specifying the amounts to be transferred and the recipient’s private address (a divestified address that hides the recipient's public address from the network). The nullifier key and random number for commitments (RCK) are also generated at this stage to define how these UTXOs can be spent or nullified in the -future by the receiver.

      • Proof generation and verification: Next, the zkVM generates a ZKP to validate the transaction. This proof includes both a membership proof for the input UTXOs, +future by the receiver.

        +
        • +
      • +

        Proof generation and verification: Next, the zkVM generates a ZKP to validate the transaction. This proof includes both a membership proof for the input UTXOs, confirming their presence in the hashed UTXO tree, and a non-membership proof to ensure that the input UTXOs have not already been spent (i.e., they are not in the nullifier tree). -The proof also confirms that the total input value matches the total output value, ensuring no discrepancies. The user then submits the proof, along with the necessary metadata, to the sequencer.

      • Shared secret and encryption: To maintain confidentiality, the sender uses the receiver’s divestified address to generate an ephemeral public key. +The proof also confirms that the total input value matches the total output value, ensuring no discrepancies. The user then submits the proof, along with the necessary metadata, to the sequencer.

        +
        • +
      • +

        Shared secret and encryption: To maintain confidentiality, the sender uses the receiver’s divestified address to generate an ephemeral public key. This allows the creation of a shared secret between the sender and receiver. Using a key derivation function, a symmetric encryption key is generated from the shared secret. -The input and output UTXOs are then encrypted using this symmetric key, ensuring that only the intended recipient can decrypt the data.

      • Broadcasting the transaction: The user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. -These committed UTXOs are sent to the sequencer, which updates the hashed UTXO tree without knowing the transaction details.

      • Decryption by the receiver: After the broadcast, the receiver attempts to decrypt the broadcast UTXOs using their symmetric key, derived from the ephemeral public key. +The input and output UTXOs are then encrypted using this symmetric key, ensuring that only the intended recipient can decrypt the data.

        +
        • +
      • +

        Broadcasting the transaction: The user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. +These committed UTXOs are sent to the sequencer, which updates the hashed UTXO tree without knowing the transaction details.

        +
        • +
      • +

        Decryption by the receiver: After the broadcast, the receiver attempts to decrypt the broadcast UTXOs using their symmetric key, derived from the ephemeral public key. If the receiver successfully decrypts a UTXO, it confirms ownership of that UTXO. The receiver then computes the nullifier for the UTXO and verifies its presence in the hashed -UTXO tree and its absence from the nullifier tree, ensuring it has not been spent. Finally, the new UTXO is added to the receiver’s locally stored UTXO tree for future transactions.

      Throughout the private execution process, the identities of both the sender and receiver, as well as all transaction details, remain hidden from the public. +UTXO tree and its absence from the nullifier tree, ensuring it has not been spent. Finally, the new UTXO is added to the receiver’s locally stored UTXO tree for future transactions.

      +
      • +
    +

    Throughout the private execution process, the identities of both the sender and receiver, as well as all transaction details, remain hidden from the public. The use of ZKPs ensures that the integrity of the transaction is verified without revealing any sensitive information. At the end of the process, -the network guarantees that no participant, aside from the sender and receiver, can deduce any details about the transaction or the involved parties.

    II) UTXOs in shielded executions​

    In shielded executions, the interaction between public and private states provides a hybrid privacy model that balances transparency and confidentiality. +the network guarantees that no participant, aside from the sender and receiver, can deduce any details about the transaction or the involved parties.

    +

    II) UTXOs in shielded executions​

    +
    +

    In shielded executions, the interaction between public and private states provides a hybrid privacy model that balances transparency and confidentiality. This model is suitable for scenarios where the initial step, such as a public transaction, requires visibility, while subsequent actions, such as private asset management, need to remain confidential. One common use case is asset conversion—where a public token is converted into a private token. The conversion is visible on the public ledger, -but subsequent transactions remain private.

    a) How shielded executions work​

    Shielded executions operate in two distinct stages: first, there is a modification of the public state, and then new UTXOs are created and stored in the private state. +but subsequent transactions remain private.

    +
    a) How shielded executions work​
    +

    Shielded executions operate in two distinct stages: first, there is a modification of the public state, and then new UTXOs are created and stored in the private state. Importantly, shielded executions do not consume UTXOs but instead mint them, as new UTXOs are created to reflect the changes in the private state. This structure demands ZKPs to ensure that the newly minted UTXOs are consistent with the modifications in the public state. Here’s a step-by-step breakdown of how the shielded -execution process unfolds:

    1. Transaction initiation: The user initiates a transaction that modifies the public state, such as converting a public token to a private token. -The transaction alters the public state (e.g., balances or smart contract storage) while simultaneously preparing to mint new UTXOs in the private state.

    2. Generating UTXOs: After modifying the public state, the system mints new UTXOs in the private state. These UTXOs must be securely created, ensuring their integrity -and consistency with the initial public state modification. A ZKP is generated by the user to prove that these new UTXOs align with the changes made in the public state.

    3. Key setup for privacy: The sender retrieves the receiver's address and uses it to create a shared secret through an ephemeral public key. This shared secret is then used -to derive a symmetric key, which encrypts the output UTXOs. This encryption ensures that only the intended receiver can decrypt and access the UTXOs.

    4. Broadcasting and verifying UTXOs: After encrypting the UTXOs, the sender broadcasts them to the network. The new hashed UTXOs are sent to the sequencer, +execution process unfolds:

      +
        +
      1. +

        Transaction initiation: The user initiates a transaction that modifies the public state, such as converting a public token to a private token. +The transaction alters the public state (e.g., balances or smart contract storage) while simultaneously preparing to mint new UTXOs in the private state.

        +
        2. +
      2. +

        Generating UTXOs: After modifying the public state, the system mints new UTXOs in the private state. These UTXOs must be securely created, ensuring their integrity +and consistency with the initial public state modification. A ZKP is generated by the user to prove that these new UTXOs align with the changes made in the public state.

        +
        3. +
      3. +

        Key setup for privacy: The sender retrieves the receiver's address and uses it to create a shared secret through an ephemeral public key. This shared secret is then used +to derive a symmetric key, which encrypts the output UTXOs. This encryption ensures that only the intended receiver can decrypt and access the UTXOs.

        +
        4. +
      4. +

        Broadcasting and verifying UTXOs: After encrypting the UTXOs, the sender broadcasts them to the network. The new hashed UTXOs are sent to the sequencer, which verifies the validity of the UTXOs and attaches them to the hashed UTXO tree within the private state. The public inputs for the ZKP circuits consist of the -Pedersen-hashed UTXOs and the modifications in the public state.

      5. Receiver's role: Once the UTXOs are broadcast, the receiver attempts to decrypt each UTXO using the symmetric key derived from the shared secret. If the decryption is successful, -the UTXO belongs to the receiver. The receiver then verifies the UTXO’s validity by checking its inclusion in the hashed UTXO tree and ensuring that its nullifier has not yet been used.

      6. Nullifier check and integration: To prevent double spending, the receiver computes the nullifier for the received UTXO and verifies that it is not already present in the nullifier tree. -Once verified, the receiver adds the UTXO to their locally stored UTXO tree for future use in private transactions.

      While shielded executions offer privacy, certain information is still exposed to the public state, such as the sender's identity. To further enhance privacy, +Pedersen-hashed UTXOs and the modifications in the public state.

      +
      5. +
    5. +

      Receiver's role: Once the UTXOs are broadcast, the receiver attempts to decrypt each UTXO using the symmetric key derived from the shared secret. If the decryption is successful, +the UTXO belongs to the receiver. The receiver then verifies the UTXO’s validity by checking its inclusion in the hashed UTXO tree and ensuring that its nullifier has not yet been used.

      +
      6. +
    6. +

      Nullifier check and integration: To prevent double spending, the receiver computes the nullifier for the received UTXO and verifies that it is not already present in the nullifier tree. +Once verified, the receiver adds the UTXO to their locally stored UTXO tree for future use in private transactions.

      +
      7. +
    +

    While shielded executions offer privacy, certain information is still exposed to the public state, such as the sender's identity. To further enhance privacy, the sender can create empty UTXOs—UTXOs that don’t belong to anyone but are included in the transaction to obfuscate the true details of the transaction. -Though this approach increases the size of the data, it adds a layer of privacy by complicating the identification of meaningful transactions.

    b) Summary of shielded execution flow​
    • Stage 1 (public modification): The user modifies public state data, such as converting tokens from public to private. This stage is visible to the public.
    • Stage 2 (UTXO minting and privacy): New UTXOs are minted in the private state, encrypted, and broadcast to the network. The transaction remains private from this point forward, -secured by ZKPs and cryptographic keys.
    • Receiver’s role: The receiver decrypts the UTXOs and verifies their validity, ensuring the UTXOs are not double spent and are ready for future transactions.

    In summary, shielded executions enable a hybrid privacy model in Nescience, balancing public transparency and private confidentiality. They are well-suited for -transactions requiring initial public visibility, such as asset conversions, while ensuring that subsequent actions remain secure and private within the network.

    III) UTXOs in deshielded executions​

    In NSSA, deshielded executions offer a unique way to move data and assets from the private state to the public state, revealing previously private +Though this approach increases the size of the data, it adds a layer of privacy by complicating the identification of meaningful transactions.

    +
    b) Summary of shielded execution flow​
    +
      +
    • Stage 1 (public modification): The user modifies public state data, such as converting tokens from public to private. This stage is visible to the public.
      • +
    • Stage 2 (UTXO minting and privacy): New UTXOs are minted in the private state, encrypted, and broadcast to the network. The transaction remains private from this point forward, +secured by ZKPs and cryptographic keys.
      • +
    • Receiver’s role: The receiver decrypts the UTXOs and verifies their validity, ensuring the UTXOs are not double spent and are ready for future transactions.
      • +
    +

    In summary, shielded executions enable a hybrid privacy model in Nescience, balancing public transparency and private confidentiality. They are well-suited for +transactions requiring initial public visibility, such as asset conversions, while ensuring that subsequent actions remain secure and private within the network.

    +

    III) UTXOs in deshielded executions​

    +
    +

    In NSSA, deshielded executions offer a unique way to move data and assets from the private state to the public state, revealing previously private information in a controlled and verifiable manner. This type of execution allows for selective disclosure, ensuring transparency when needed while still maintaining the security and privacy of critical details through cryptographic techniques like ZKPs. Deshielded executions are particularly valuable for use cases -such as regulatory compliance reporting, where specific transaction details must be revealed to meet legal requirements, while other sensitive transactions remain private.

    a) Stages of deshielded executions​

    • Stage 1 (UTXO consumption): The process begins in the private state, where UTXOs are consumed as inputs for the transaction. This involves gathering all necessary -UTXOs that contain the assets or balances to be made public, as well as any associated private data stored in memory slots.

    • Stage 2 (public state modification): After the UTXOs are consumed, the transaction details are made public by modifying the public state. This update includes changes +such as regulatory compliance reporting, where specific transaction details must be revealed to meet legal requirements, while other sensitive transactions remain private.

      +
      a) Stages of deshielded executions​
      +
        +
      • +

        Stage 1 (UTXO consumption): The process begins in the private state, where UTXOs are consumed as inputs for the transaction. This involves gathering all necessary +UTXOs that contain the assets or balances to be made public, as well as any associated private data stored in memory slots.

        +
        • +
      • +

        Stage 2 (public state modification): After the UTXOs are consumed, the transaction details are made public by modifying the public state. This update includes changes to the public balances, storage data, and any necessary public records. While the public state is updated, the sender’s identity and other sensitive information remain hidden, -thanks to the privacy-preserving properties of ZKPs.

      This model ensures that private data can be selectively revealed when needed, offering both flexibility and transparency. It is particularly useful for scenarios requiring -auditing or compliance reporting, where specific details must be made publicly verifiable without exposing the entire history or contents of private transactions.

      b) How deshielded executions work​

      The deshielded execution process starts when a user initiates a transaction using private UTXOs. The Nescience zkVM is called to generate a ZKP, -which validates the transaction without revealing sensitive details such as the sender's identity or the specifics of the Nescience application being executed.

      During the transaction, the UTXOs from the private state are consumed, meaning they are used up as inputs and will no longer be available for future transactions. +thanks to the privacy-preserving properties of ZKPs.

      +
      • +
    +

    This model ensures that private data can be selectively revealed when needed, offering both flexibility and transparency. It is particularly useful for scenarios requiring +auditing or compliance reporting, where specific details must be made publicly verifiable without exposing the entire history or contents of private transactions.

    +
    b) How deshielded executions work​
    +

    The deshielded execution process starts when a user initiates a transaction using private UTXOs. The Nescience zkVM is called to generate a ZKP, +which validates the transaction without revealing sensitive details such as the sender's identity or the specifics of the Nescience application being executed.

    +

    During the transaction, the UTXOs from the private state are consumed, meaning they are used up as inputs and will no longer be available for future transactions. Instead of generating new UTXOs, the transaction modifies the public state, updating the necessary balances or memory slots related to the transaction. -Here’s a step-by-step breakdown of how the deshielded execution process unfolds:

    1. Get receiver's public address: The sender first identifies the public address of the receiver, to which the information or assets will be made public.

    2. Determine input UTXOs and public state modifications: The sender gathers all the input UTXOs needed for the transaction and determines the public state modifications -necessary for the Nescience applications and token transfers involved.

    3. Calculate nullifiers: Nullifiers are generated for each input UTXO, ensuring that these UTXOs cannot be reused or double spent. The nullifiers are derived from the -corresponding UTXO commitments.

    4. Call zkVM with deshielded circuits: The sender invokes the zkVM with deshielded kernel circuits, which generates the proof. The proof ensures that all input UTXOs -are valid by verifying their membership in the UTXO tree and their non-membership in the nullifier tree, ensuring they haven’t been spent.

    5. Generate and submit proof: The zkVM generates a ZKP that verifies the correctness of the transaction without revealing private details. -The proof includes the nullifiers and the planned modifications to the public state.

    6. Send proof to sequencer: The sender then sends the proof and any relevant public information to the sequencer. The sequencer is responsible for verifying the proof, -updating the public state accordingly, and adding the nullifiers to the nullifier tree.

    Once the proof and public information have been broadcast to the network, the receiver does not need to take any further action. +Here’s a step-by-step breakdown of how the deshielded execution process unfolds:

    +
      +
    1. +

      Get receiver's public address: The sender first identifies the public address of the receiver, to which the information or assets will be made public.

      +
      2. +
    2. +

      Determine input UTXOs and public state modifications: The sender gathers all the input UTXOs needed for the transaction and determines the public state modifications +necessary for the Nescience applications and token transfers involved.

      +
      3. +
    3. +

      Calculate nullifiers: Nullifiers are generated for each input UTXO, ensuring that these UTXOs cannot be reused or double spent. The nullifiers are derived from the +corresponding UTXO commitments.

      +
      4. +
    4. +

      Call zkVM with deshielded circuits: The sender invokes the zkVM with deshielded kernel circuits, which generates the proof. The proof ensures that all input UTXOs +are valid by verifying their membership in the UTXO tree and their non-membership in the nullifier tree, ensuring they haven’t been spent.

      +
      5. +
    5. +

      Generate and submit proof: The zkVM generates a ZKP that verifies the correctness of the transaction without revealing private details. +The proof includes the nullifiers and the planned modifications to the public state.

      +
      6. +
    6. +

      Send proof to sequencer: The sender then sends the proof and any relevant public information to the sequencer. The sequencer is responsible for verifying the proof, +updating the public state accordingly, and adding the nullifiers to the nullifier tree.

      +
      7. +
    +

    Once the proof and public information have been broadcast to the network, the receiver does not need to take any further action. The sequencer manages the public state updates and ensures that the transaction is properly executed. By the end of the deshielded execution, specific transaction details become publicly visible, such as the identity of the receiver and the outcome of the transaction. This allows participants in the public state to extract information about the transaction, including the receiver's identity and some details about the execution. While the receiver's identity is revealed, the sender's identity and sensitive transaction details remain hidden, thanks to the use of ZKPs. -This makes deshielded executions ideal for cases where transparency is needed, but complete privacy is still a priority for certain elements of the transaction.

    Summary of UTXO consumption in NSSA​

    In NSSA, consuming UTXOs is a critical step in maintaining the security and integrity of the blockchain by preventing double spending. -When a UTXO is consumed, it is used as an input in a transaction, effectively marking it as spent. This ensures that the UTXO cannot be reused, preserving the integrity of the blockchain.

    1. The process of consuming UTXOs: The process of consuming a UTXO begins when a user selects a UTXO from their private state. The user verifies the UTXO’s existence and -ownership using their viewing key, ensuring that they are the legitimate owner of the UTXO. Once verified, the user generates two key cryptographic proofs:
      * **Membership proof:** This proof confirms that the UTXO exists within the hashed UTXO tree, ensuring its validity within the system.
      * **Non-membership proof:** This proof ensures that the UTXO has not been previously consumed by checking its absence in the nullifier tree, which tracks spent UTXOs.

    To mark the UTXO as spent, a nullifier is generated. This nullifier is a unique cryptographic hash derived from the UTXO, which is then added to the nullifier tree in the public state. -Adding the nullifier to the tree prevents the UTXO from being reused in future transactions, thus preventing double spending.

    After generating the membership and non-membership proofs, the user compiles the transaction using the zkVM. The zkVM is responsible for generating the necessary ZKPs, -which validate the transaction without revealing sensitive details. The compiled transaction, along with the proofs, is then submitted to the sequencer for verification.

    1. The role of the sequencer: Once the transaction is submitted, the sequencer verifies the ZKPs to confirm that the transaction is valid. If the proofs are verified +This makes deshielded executions ideal for cases where transparency is needed, but complete privacy is still a priority for certain elements of the transaction.

      +

      Summary of UTXO consumption in NSSA​

      +
      +

      In NSSA, consuming UTXOs is a critical step in maintaining the security and integrity of the blockchain by preventing double spending. +When a UTXO is consumed, it is used as an input in a transaction, effectively marking it as spent. This ensures that the UTXO cannot be reused, preserving the integrity of the blockchain.

      +
        +
      1. The process of consuming UTXOs: The process of consuming a UTXO begins when a user selects a UTXO from their private state. The user verifies the UTXO’s existence and +ownership using their viewing key, ensuring that they are the legitimate owner of the UTXO. Once verified, the user generates two key cryptographic proofs: +
          +
        • Membership proof: This proof confirms that the UTXO exists within the hashed UTXO tree, ensuring its validity within the system.
          • +
        • Non-membership proof: This proof ensures that the UTXO has not been previously consumed by checking its absence in the nullifier tree, which tracks spent UTXOs.
          • +
        +
        2. +
      +

      To mark the UTXO as spent, a nullifier is generated. This nullifier is a unique cryptographic hash derived from the UTXO, which is then added to the nullifier tree in the public state. +Adding the nullifier to the tree prevents the UTXO from being reused in future transactions, thus preventing double spending.

      +

      After generating the membership and non-membership proofs, the user compiles the transaction using the zkVM. The zkVM is responsible for generating the necessary ZKPs, +which validate the transaction without revealing sensitive details. The compiled transaction, along with the proofs, is then submitted to the sequencer for verification.

      +
        +
      1. The role of the sequencer: Once the transaction is submitted, the sequencer verifies the ZKPs to confirm that the transaction is valid. If the proofs are verified successfully, the sequencer updates both the private and public states to reflect the transaction. This includes updating the nullifier tree with the newly generated nullifier, -ensuring that the UTXO is marked as spent and cannot be reused.

      Example: Alice sending tokens to Bob​

      Consider an example where Alice wants to send 5 Nescience tokens to Bob using a private execution. Alice selects a UTXO from her private state that contains 5 Nescience tokens. +ensuring that the UTXO is marked as spent and cannot be reused.

      2. +
    +

    Example: Alice sending tokens to Bob​

    +
    +

    Consider an example where Alice wants to send 5 Nescience tokens to Bob using a private execution. Alice selects a UTXO from her private state that contains 5 Nescience tokens. She generates the necessary membership and non-membership proofs, ensuring that her UTXO exists in the system and has not been previously spent. Alice then creates a nullifier by -hashing the UTXO and compiles the transaction with the zkVM.

    Once Alice submits the transaction, the sequencer verifies the proofs and updates the blockchain by adding the nullifier to the nullifier tree and recording the transaction details. -This ensures that Alice’s UTXO is marked as spent and cannot be used again, while Bob receives the 5 tokens.

    The importance of nullifiers​

    Nullifiers are a key mechanism in preventing double spending. By marking consumed UTXOs as spent and tracking them in the nullifier tree, NSSA ensures that +hashing the UTXO and compiles the transaction with the zkVM.

    +

    Once Alice submits the transaction, the sequencer verifies the proofs and updates the blockchain by adding the nullifier to the nullifier tree and recording the transaction details. +This ensures that Alice’s UTXO is marked as spent and cannot be used again, while Bob receives the 5 tokens.

    +

    The importance of nullifiers​

    +
    +

    Nullifiers are a key mechanism in preventing double spending. By marking consumed UTXOs as spent and tracking them in the nullifier tree, NSSA ensures that once a UTXO is used in a transaction, it cannot be reused in any future transactions. This process is fundamental to maintaining the integrity and security of the blockchain, -as it guarantees that assets are only spent once and prevents potential attacks on the system.

    In conclusion, the process of consuming UTXOs in NSSA combines cryptographic proofs, nullifiers, and ZKPs to ensure that transactions -are secure, confidential, and free from the risks of double spending.

    C. Cryptographic primitives in NSSA

    In the NSSA, cryptographic primitives are the foundational elements that ensure the security, privacy, and efficiency of the state separation model. +as it guarantees that assets are only spent once and prevents potential attacks on the system.

    +

    In conclusion, the process of consuming UTXOs in NSSA combines cryptographic proofs, nullifiers, and ZKPs to ensure that transactions +are secure, confidential, and free from the risks of double spending.

    +

    C. Cryptographic primitives in NSSA

    +

    In the NSSA, cryptographic primitives are the foundational elements that ensure the security, privacy, and efficiency of the state separation model. These cryptographic tools enable private transactions, secure data management, and robust verification processes across both public and private states. The architecture leverages a wide range of cryptographic mechanisms, including advanced hash functions, key management systems, tree structures, and ZKPs, -to safeguard user data and maintain the integrity of transactions.

    Cryptographic hash functions play a pivotal role in concealing UTXO details, generating nullifiers, and constructing sparse Merkle trees, which organize and verify +to safeguard user data and maintain the integrity of transactions.

    +

    Cryptographic hash functions play a pivotal role in concealing UTXO details, generating nullifiers, and constructing sparse Merkle trees, which organize and verify data efficiently within the network. Key management and address generation further enhance the security of user assets and identity, ensuring that only authorized -users can access and control their holdings.

    The architecture also relies on specialized tree structures for organizing data, verifying the existence of UTXOs, and tracking nullifiers, which prevent double spending. +users can access and control their holdings.

    +

    The architecture also relies on specialized tree structures for organizing data, verifying the existence of UTXOs, and tracking nullifiers, which prevent double spending. Additionally, Nescience features a privacy-preserving zero-knowledge virtual machine (zk-zkVM), which allows users to prove the correctness of an execution without -disclosing sensitive information. This enables private transactions and maintains confidentiality across the network.

    As Nescience evolves, optional cryptographic mechanisms such as multi-party computation (MPC) may be integrated to enhance synchronization across privacy levels. +disclosing sensitive information. This enables private transactions and maintains confidentiality across the network.

    +

    As Nescience evolves, optional cryptographic mechanisms such as multi-party computation (MPC) may be integrated to enhance synchronization across privacy levels. This MPC-based synchronization mechanism is still under development and under review for potential inclusion in the system. Together, these cryptographic primitives -form the backbone of Nescience’s security architecture, ensuring that users can transact and interact privately, securely, and efficiently.

    In the following sections, we will explore each of these cryptographic components in detail, beginning with the role of hash functions.

    a) Hash functions in Nescience​

    Hash functions are a foundational element of Nescience’s cryptographic framework, serving multiple critical roles that ensure the security, privacy, and efficiency of the system. +form the backbone of Nescience’s security architecture, ensuring that users can transact and interact privately, securely, and efficiently.

    +

    In the following sections, we will explore each of these cryptographic components in detail, beginning with the role of hash functions.

    +

    a) Hash functions in Nescience​

    +
    +

    Hash functions are a foundational element of Nescience’s cryptographic framework, serving multiple critical roles that ensure the security, privacy, and efficiency of the system. One of the primary uses of hash functions in Nescience is to conceal sensitive details of UTXOs by converting them into fixed-size hashes. This process allows UTXO details to remain private, ensuring that sensitive information is not directly exposed on the blockchain, while still enabling their existence and integrity to be verified. Hashing -the UTXO details allows the actual data to remain confidential, with the hashes stored in a global tree structure for efficient management and retrieval.

    Additionally, hash functions are essential for generating nullifiers, which play a crucial role in preventing double spending. Nullifiers are created by hashing UTXOs +the UTXO details allows the actual data to remain confidential, with the hashes stored in a global tree structure for efficient management and retrieval.

    +

    Additionally, hash functions are essential for generating nullifiers, which play a crucial role in preventing double spending. Nullifiers are created by hashing UTXOs and are used to mark them as spent, ensuring that they cannot be reused in subsequent transactions. These nullifiers are stored in a nullifier tree, and each transaction must prove that its UTXO’s nullifier is not already present in the tree before it can be processed. This ensures that the UTXO has not been spent before, maintaining the -integrity of the transaction process.

    Hash functions are also vital in the construction of sparse Merkle trees, which provide an efficient and secure method for verifying data within the blockchain. +integrity of the transaction process.

    +

    Hash functions are also vital in the construction of sparse Merkle trees, which provide an efficient and secure method for verifying data within the blockchain. Sparse Merkle trees enable quick and reliable proofs of membership and non-membership, making them essential for verifying both UTXOs and nullifiers. By using hash functions -to build these trees, Nescience can ensure the integrity of the data, as any tampering with the data would result in a change in the hash, making the manipulation detectable.

    Another critical consideration in Nescience is the compatibility of hash functions with ZKPs. ZK-friendly hash functions are optimized for efficient +to build these trees, Nescience can ensure the integrity of the data, as any tampering with the data would result in a change in the hash, making the manipulation detectable.

    +

    Another critical consideration in Nescience is the compatibility of hash functions with ZKPs. ZK-friendly hash functions are optimized for efficient computation within the constraints of ZK circuits, ensuring that they do not become a bottleneck in the proof generation or verification process. These hash functions maintain strong cryptographic security properties while enabling efficient computations in ZKP systems, which is essential for maintaining privacy and -integrity within the ZK framework.

    The primary advantage of using hash functions in Nescience is their ability to ensure that transaction details remain private while still allowing for verification +integrity within the ZK framework.

    +

    The primary advantage of using hash functions in Nescience is their ability to ensure that transaction details remain private while still allowing for verification of their validity. Furthermore, by integrating hash functions into Merkle trees, the blockchain data becomes tamper-proof, enabling quick and efficient verification -processes that uphold the system’s security and privacy standards.

    Use case: How to use the Pedersen hash to create the UTXO commitment​

    As mentioned in the UTXOs in private executions section, the user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs +processes that uphold the system’s security and privacy standards.

    +

    Use case: How to use the Pedersen hash to create the UTXO commitment​

    +
    +

    As mentioned in the UTXOs in private executions section, the user broadcasts the encrypted UTXOs to the network, along with a commitment to the output UTXOs using Pedersen hashes. The Pedersen hash is used to create the UTXO commitment. The Pedersen hash is a homomorphic commitment scheme that allows secure commitments -while maintaining privacy and enabling proofs of correctness in transactions. The commitment formula is as follows:

    Commitment=C(UTXO,RCK)=gUTXO⋅hRCKCommitment = C(UTXO,RCK) =g^{UTXO}⋅h^{RCK}

    In this formula, gg and hh are two generators of a cryptographic group where no known relationship exists between them. This ensures that the commitment is secure -and computationally infeasible to reverse or manipulate without knowing the original UTXO components. The random number RCKRCK adds an additional layer of security -by blinding the UTXO's contents, ensuring that the commitment doesn't leak any information about the underlying data.

    Importance of homomorphic commitments

    It is essential to use a homomorphic commitment like the Pedersen commitment for UTXOs because it allows for the verification of important properties in transactions, +while maintaining privacy and enabling proofs of correctness in transactions. The commitment formula is as follows:

    +

    Commitment=C(UTXO,RCK)=gUTXO⋅hRCKCommitment = C(UTXO,RCK) =g^{UTXO}⋅h^{RCK}

    +

    In this formula, gg and hh are two generators of a cryptographic group where no known relationship exists between them. This ensures that the commitment is secure +and computationally infeasible to reverse or manipulate without knowing the original UTXO components. The random number RCKRCK adds an additional layer of security +by blinding the UTXO's contents, ensuring that the commitment doesn't leak any information about the underlying data.

    +

    Importance of homomorphic commitments

    +

    It is essential to use a homomorphic commitment like the Pedersen commitment for UTXOs because it allows for the verification of important properties in transactions, such as ensuring that the total input value of a transaction equals the total output value. This balance is crucial for preventing the unauthorized creation of funds or d discrepancies in transactions. A homomorphic commitment enables these proofs because of its additive properties. Specifically, the exponents in the commitment formula are additive, -meaning that commitments can be combined and verified without revealing the individual components. For instance, if you have two UTXOs with commitments C(UTXO1,RCK1)C(UTXO_1,RCK_1) -and C(UTXO2,RCK2)C(UTXO_2,RCK_2), you can combine them and verify that the resulting commitment is valid without exposing the actual amounts.

    This capability is leveraged through a modified version of the Schnorr protocol, which is used in conjunction with the Pedersen hash to verify the correctness of transactions. -The Schnorr protocol allows users to prove, without revealing the actual values, that the sum of inputs equals the sum of outputs, ensuring that no funds are created or lost in the transaction.

    Limitations of standard cryptographic hashes

    Standard cryptographic hash functions, such as SHA-256, are not suitable for this purpose because they lack the algebraic structure needed for homomorphic properties. +meaning that commitments can be combined and verified without revealing the individual components. For instance, if you have two UTXOs with commitments C(UTXO1,RCK1)C(UTXO_1,RCK_1) +and C(UTXO2,RCK2)C(UTXO_2,RCK_2), you can combine them and verify that the resulting commitment is valid without exposing the actual amounts.

    +

    This capability is leveraged through a modified version of the Schnorr protocol, which is used in conjunction with the Pedersen hash to verify the correctness of transactions. +The Schnorr protocol allows users to prove, without revealing the actual values, that the sum of inputs equals the sum of outputs, ensuring that no funds are created or lost in the transaction.

    +

    Limitations of standard cryptographic hashes

    +

    Standard cryptographic hash functions, such as SHA-256, are not suitable for this purpose because they lack the algebraic structure needed for homomorphic properties. In particular, while SHA-256 provides strong security for general hashing purposes, it does not allow the additive properties that are required to perform the type of ZKPs used in Nescience for UTXO commitments. This is why the Pedersen hash is preferred, as it enables the secure and private execution of transactions -while allowing for balance verification and other critical proofs.

    Conclusion

    By using homomorphic commitments like the Pedersen hash, NSSA ensures that UTXOs can be securely committed and validated without exposing sensitive information. +while allowing for balance verification and other critical proofs.

    +

    Conclusion

    +

    By using homomorphic commitments like the Pedersen hash, NSSA ensures that UTXOs can be securely committed and validated without exposing sensitive information. The random component (RCK) adds an additional layer of security, and the additive properties of the Pedersen commitment enable powerful ZKPs that maintain the -integrity of the system.

    b) Key management and addresses in Nescience​

    NSSA utilizes different cryptographic schemes, such as public key encryption and digital signatures, to ensure secure private executions through +integrity of the system.

    +

    b) Key management and addresses in Nescience​

    +
    +

    NSSA utilizes different cryptographic schemes, such as public key encryption and digital signatures, to ensure secure private executions through the exchange of UTXOs. These schemes rely on a structured set of cryptographic keys, each serving a specific purpose in maintaining privacy, security, and control over assets. -Here's a breakdown of the keys used in Nescience:

    I. Spending key​

    The spending key is the fundamental secret key in NSSA, acting as the primary control mechanism for a user’s UTXOs and other digital assets. -It plays a critical role in the cryptographic security of the system, ensuring that only the rightful owner can authorize and spend their assets.

    • Role of the spending key: The spending key is responsible for generating the user’s private keys, which are used in various cryptographic operations such as +Here's a breakdown of the keys used in Nescience:

      +

      I. Spending key​

      +

      The spending key is the fundamental secret key in NSSA, acting as the primary control mechanism for a user’s UTXOs and other digital assets. +It plays a critical role in the cryptographic security of the system, ensuring that only the rightful owner can authorize and spend their assets.

      +
        +
      • +

        Role of the spending key: The spending key is responsible for generating the user’s private keys, which are used in various cryptographic operations such as signing transactions and creating commitments. This hierarchical relationship means that the spending key sits at the root of a user’s key structure, safeguarding access to all associated private keys and, consequently, to the user’s assets. In Nescience’s privacy-focused model, the spending key is never exposed or shared outside the user’s control. Unlike other keys, it does not interact with the public state, kernel circuits, or even the ZKP system. This isolation ensures that the spending key remains completely private and inaccessible to external entities. By keeping the spending key separate from the operational aspects of the network, -Nescience minimizes the risk of key leakage or compromise.

      • Generation and security of the spending key: The spending key is generated randomly from the scalar field, a large mathematical space that ensures uniqueness +Nescience minimizes the risk of key leakage or compromise.

        +
        • +
      • +

        Generation and security of the spending key: The spending key is generated randomly from the scalar field, a large mathematical space that ensures uniqueness and cryptographic strength. This randomness is crucial because it prevents attackers from predicting or replicating the key, thereby safeguarding the user’s assets from unauthorized access: it is computationally infeasible for an attacker to guess or brute-force the key. Once the spending key is generated, it is securely stored -by the user, typically in a hardware wallet or another secure storage mechanism that prevents unauthorized access.

      • Spending UTXOs with the spending key: The spending key’s primary function is to authorize the spending of UTXOs in private transactions. When a user initiates +by the user, typically in a hardware wallet or another secure storage mechanism that prevents unauthorized access.

        +
        • +
      • +

        Spending UTXOs with the spending key: The spending key’s primary function is to authorize the spending of UTXOs in private transactions. When a user initiates a transaction, the spending key is used to generate the necessary cryptographic proofs and signatures, ensuring that the transaction is valid and originates from the rightful owner. However, even though the spending key generates these proofs, it is never directly exposed during the transaction process. Instead, derived private keys handle the operational aspects while the spending key remains secure in the background. For example, when Alice decides to spend a UTXO in a private execution, her spending key generates the required private keys that will sign the transaction and ensure its validity. However, the spending key itself -never appears in any public state or transaction data, preserving its confidentiality.

      • Ensuring security through isolation: One of the key security principles of the spending key is its isolation from the network. Since it never interacts with +never appears in any public state or transaction data, preserving its confidentiality.

        +
        • +
      • +

        Ensuring security through isolation: One of the key security principles of the spending key is its isolation from the network. Since it never interacts with public-facing elements, such as the public state or kernel circuits, the risk of exposure is significantly reduced. This isolation ensures that even if other parts -of the cryptographic infrastructure are compromised, the spending key remains protected, preventing unauthorized spending of UTXOs.

      In summary, the spending key in Nescience is a powerful and carefully guarded element of the cryptographic system. It is the root key from which other private keys +of the cryptographic infrastructure are compromised, the spending key remains protected, preventing unauthorized spending of UTXOs.

      +
      • +
    +

    In summary, the spending key in Nescience is a powerful and carefully guarded element of the cryptographic system. It is the root key from which other private keys are derived, allowing users to spend their UTXOs securely and privately. Its isolation from the public state and its random generation from a secure scalar field ensures -that the spending key remains protected, making it a cornerstone of security in NSSA.

    II. Private keys​

    In Nescience, the private key is an essential cryptographic element responsible for facilitating various secure operations, such as generating commitments and signing +that the spending key remains protected, making it a cornerstone of security in NSSA.

    +

    II. Private keys​

    +

    In Nescience, the private key is an essential cryptographic element responsible for facilitating various secure operations, such as generating commitments and signing transactions. While the spending key plays a foundational role in safeguarding access to UTXOs and assets, the private keys handle the operational aspects of transactions -and cryptographic proofs. The private key consists of three critical components: privatekey.rsd{private}_{key}.rsd, privatekey.rcm{private}_{key}.rcm, and privatekey.sig{private}_{key}.sig, each serving a -distinct purpose within the Nescience cryptographic framework.

    1. privatekey.rsd{private}_{key}.rsd (random seed): The random seed (privatekey.rsd{private}_{key}.rsd) is the first and foundational component of the private key. It is a value randomly chosen from the scalar field, which ensures +and cryptographic proofs. The private key consists of three critical components: privatekey.rsd{private}_{key}.rsd, privatekey.rcm{private}_{key}.rcm, and privatekey.sig{private}_{key}.sig, each serving a +distinct purpose within the Nescience cryptographic framework.

      +
        +
      1. +

        privatekey.rsd{private}_{key}.rsd (random seed): The random seed (privatekey.rsd{private}_{key}.rsd) is the first and foundational component of the private key. It is a value randomly chosen from the scalar field, which ensures its cryptographic security and unpredictability. This seed is generated using a random number generator, making it virtually impossible to predict or replicate. The random seed is essential because it is used to derive the other two components of the private key. By leveraging a secure random seed, Nescience ensures that the entire private key structure is rooted in randomness, preventing external entities from guessing or deriving the key through brute-force attacks. -The strength of the random seed ensures the overall security of the private key and, consequently, the integrity of the user's transactions and commitments.

      2. privatekey.rcm{private}_{key}.rcm (random commitment): The random commitment component (privatekey.rcm{private}_{key}.rcm) is a crucial part of the private key used specifically in the commitment scheme. It acts as a blinding factor, -adding a layer of security to commitments made by the user. The privatekey.rcm{private}_{key}.rcm value is also drawn from the scalar field and is used to ensure that the commitment -to any UTXO or other sensitive data remains confidential. The commitment scheme in Nescience requires the use of privatekey.rcm{private}_{key}.rcm to create cryptographic commitments -that bind the user to specific data (such as UTXO details) without revealing the actual data. The role of privatekey.rcm{private}_{key}.rcm is to ensure that these commitments are -non-malleable and secure, preventing anyone from modifying the committed data without detection. For instance, when Alice commits to a UTXO, privatekey.rcm{private}_{key}.rcm is used +The strength of the random seed ensures the overall security of the private key and, consequently, the integrity of the user's transactions and commitments.

        +
        3. +
      3. +

        privatekey.rcm{private}_{key}.rcm (random commitment): The random commitment component (privatekey.rcm{private}_{key}.rcm) is a crucial part of the private key used specifically in the commitment scheme. It acts as a blinding factor, +adding a layer of security to commitments made by the user. The privatekey.rcm{private}_{key}.rcm value is also drawn from the scalar field and is used to ensure that the commitment +to any UTXO or other sensitive data remains confidential. The commitment scheme in Nescience requires the use of privatekey.rcm{private}_{key}.rcm to create cryptographic commitments +that bind the user to specific data (such as UTXO details) without revealing the actual data. The role of privatekey.rcm{private}_{key}.rcm is to ensure that these commitments are +non-malleable and secure, preventing anyone from modifying the committed data without detection. For instance, when Alice commits to a UTXO, privatekey.rcm{private}_{key}.rcm is used to generate a Pedersen commitment that ensures the UTXO details are hidden but can still be verified cryptographically. This means that even though the actual UTXO details -are concealed, their existence and integrity can be proven.

      4. privatekey.sig{private}_{key}.sig (signing key for transactions): The signing key (privatekey.sig{private}_{key}.sig) is the third and final component of the private key, used primarily for signing transactions. One possible approach is that -Nescience employs Schnorr signatures, a cryptographic protocol known for its efficiency and security. In this case, the privatekey.sig{private}_{key}.sig component would generate +are concealed, their existence and integrity can be proven.

        +
        5. +
      5. +

        privatekey.sig{private}_{key}.sig (signing key for transactions): The signing key (privatekey.sig{private}_{key}.sig) is the third and final component of the private key, used primarily for signing transactions. One possible approach is that +Nescience employs Schnorr signatures, a cryptographic protocol known for its efficiency and security. In this case, the privatekey.sig{private}_{key}.sig component would generate Schnorr signatures that are used to authenticate transactions, ensuring that only the rightful owner of the private key can authorize the spending of UTXOs. Schnorr signatures are important as they provide a secure and non-repudiable method of verifying that a transaction was initiated by the legitimate owner of the assets. -When Alice signs a transaction using her privatekey.sig{private}_{key}.sig, the corresponding public key allows others to verify that the transaction was indeed signed by Alice, +When Alice signs a transaction using her privatekey.sig{private}_{key}.sig, the corresponding public key allows others to verify that the transaction was indeed signed by Alice, without revealing her private key. This verification process ensures that all transactions are legitimate and prevents unauthorized entities from forging transactions -or spending assets they do not control. Even if an attacker gains access to the signed transaction, they cannot reverse engineer the privatekey.sig{private}_{key}.sig, ensuring -the security of Alice's future transactions.

      Robustness of private keys in Nescience

      Despite the critical role of the private key in the operation of NSSA, the system is designed to maintain security even in the event that the +or spending assets they do not control. Even if an attacker gains access to the signed transaction, they cannot reverse engineer the privatekey.sig{private}_{key}.sig, ensuring +the security of Alice's future transactions.

      +
      2. +
    +

    Robustness of private keys in Nescience

    +

    Despite the critical role of the private key in the operation of NSSA, the system is designed to maintain security even in the event that the private key is compromised. This resilience is achieved through the integrity of the spending key, which is never exposed in the process of signing or committing. The spending key acts as the ultimate safeguard, ensuring that even if a private key component is compromised, the attacker cannot access or spend the user's assets -without control over the spending key.

    The architecture’s design, where private keys handle operational tasks but rely on the spending key for ultimate control, ensures a layered approach to security. -This way, the system can mitigate the damage of a compromised private key by maintaining the inviolability of the user's assets.

    Conclusion

    In summary, the private key in Nescience consists of three interrelated components that together ensure secure transaction signing, commitment creation, and the -protection of user data. The privatekey.rsd{private}_{key}.rsd serves as the root from which the other key components are derived, ensuring randomness and security. -The privatekey.rcm{private}_{key}.rcm plays a crucial role in generating commitments, while privatekey.sig{private}_{key}.sig provides the signing capability needed for transaction authentication. -Together, these components enable users to engage in private, secure transactions while preserving the integrity of their assets, even in the face of potential key compromise.

    III. Public keys​

    Public keys in Nescience serve as the user's interface with the network, allowing for secure interaction and verification without exposing the user's private keys. +without control over the spending key.

    +

    The architecture’s design, where private keys handle operational tasks but rely on the spending key for ultimate control, ensures a layered approach to security. +This way, the system can mitigate the damage of a compromised private key by maintaining the inviolability of the user's assets.

    +

    Conclusion

    +

    In summary, the private key in Nescience consists of three interrelated components that together ensure secure transaction signing, commitment creation, and the +protection of user data. The privatekey.rsd{private}_{key}.rsd serves as the root from which the other key components are derived, ensuring randomness and security. +The privatekey.rcm{private}_{key}.rcm plays a crucial role in generating commitments, while privatekey.sig{private}_{key}.sig provides the signing capability needed for transaction authentication. +Together, these components enable users to engage in private, secure transactions while preserving the integrity of their assets, even in the face of potential key compromise.

    +

    III. Public keys​

    +

    Public keys in Nescience serve as the user's interface with the network, allowing for secure interaction and verification without exposing the user's private keys. Derived directly from the user's private keys, public keys play a crucial role in enabling cryptographic operations such as transaction verification, commitment schemes, and deterministic computations. The public key components correspond to their private key counterparts and ensure that transactions and commitments are securely processed -and validated across the network.

    1. publickey.sig{public}_{key}.sig (verifying Schnorr signatures):

    The publickey.sig{public}_{key}.sig is derived from the signing component of the private key (privatekey.sig{private}_{key}.sig) and is used for verifying Schnorr signatures. +and validated across the network.

    +
      +
    1. publickey.sig{public}_{key}.sig (verifying Schnorr signatures):
      2. +
    +

    The publickey.sig{public}_{key}.sig is derived from the signing component of the private key (privatekey.sig{private}_{key}.sig) and is used for verifying Schnorr signatures. Schnorr signatures are used to authenticate transactions, ensuring that they have been signed by the legitimate owner of the private key. This public key is essentially a verification key, allowing others in the network to confirm that a specific transaction was indeed authorized by the user. When a transaction is -broadcast to the network, publickey.sig{public}_{key}.sig enables any participant to verify that the transaction’s signature matches the user’s private key without +broadcast to the network, publickey.sig{public}_{key}.sig enables any participant to verify that the transaction’s signature matches the user’s private key without needing access to the private key itself. This mechanism prevents forgeries as only the legitimate owner with access to the private key can generate a valid Schnorr signature. -For example, if Alice sends a transaction, she signs it with her private key (privatekey.sig{private}_{key}.sig). Bob, or any other network participant, can use Alice’s publickey.sig{public}_{key}.sig -to verify the signature. If the signature is valid, Bob can be confident that the transaction was authorized by Alice and not by an imposter.

    1. publickey.rcm{public}_{key}.rcm (commitment schemes)

    The publickey.rcm{public}_{key}.rcm is derived from the commitment component of the private key (privatekey.rcm{private}_{key}.rcm). It is used in the commitment schemes +For example, if Alice sends a transaction, she signs it with her private key (privatekey.sig{private}_{key}.sig). Bob, or any other network participant, can use Alice’s publickey.sig{public}_{key}.sig +to verify the signature. If the signature is valid, Bob can be confident that the transaction was authorized by Alice and not by an imposter.

    +
      +
    1. publickey.rcm{public}_{key}.rcm (commitment schemes)
      2. +
    +

    The publickey.rcm{public}_{key}.rcm is derived from the commitment component of the private key (privatekey.rcm{private}_{key}.rcm). It is used in the commitment schemes that underpin Nescience’s privacy-preserving architecture. Commitments are a crucial cryptographic technique that allows users to commit to a piece of data (such as a UTXO) -without revealing the actual data, while still enabling proof of its integrity and existence. In Nescience, the publickey.rcm{public}_{key}.rcm is used as part of the Pedersen commitment scheme, +without revealing the actual data, while still enabling proof of its integrity and existence. In Nescience, the publickey.rcm{public}_{key}.rcm is used as part of the Pedersen commitment scheme, where it functions as a public commitment to certain transaction details. Even though the actual values are hidden (thanks to the private key component), the commitment can -still be verified by other network participants using publickey.rcm{public}_{key}.rcm. This enables secure and private transactions while maintaining the ability to verify that commitments -are consistent with the original data. For instance, when Alice commits to a UTXO, she uses her private key to generate the commitment, and the publickey.rcm{public}_{key}.rcm is available -to others to verify the commitment’s validity without revealing the underlying details.

    1. publickey.sk(prf){public}_{key}.sk(prf) (pseudorandom function)

    The publickey.sk(prf){public}_{key}.sk(prf) is derived from a random field element within the private key and is used to generate the pseudorandom function (PRF) associated with the user's account. +still be verified by other network participants using publickey.rcm{public}_{key}.rcm. This enables secure and private transactions while maintaining the ability to verify that commitments +are consistent with the original data. For instance, when Alice commits to a UTXO, she uses her private key to generate the commitment, and the publickey.rcm{public}_{key}.rcm is available +to others to verify the commitment’s validity without revealing the underlying details.

    +
      +
    1. publickey.sk(prf){public}_{key}.sk(prf) (pseudorandom function)
      2. +
    +

    The publickey.sk(prf){public}_{key}.sk(prf) is derived from a random field element within the private key and is used to generate the pseudorandom function (PRF) associated with the user's account. This PRF is essential for producing deterministic outputs based on the user’s keys and transaction data while ensuring that these outputs are unique to the user and cannot be predicted or replicated by others. The PRF is crucial in scenarios where the user needs to derive unique identifiers or values that are tied to their specific account, -ensuring that these values remain consistent across different transactions or interactions without revealing sensitive information. For example, publickey.sk(prf){public}_{key}.sk(prf) may be -used in generating deterministic yet secure addresses or transaction references, which can be linked to the user’s activity in a controlled manner. By using publickey.sk(prf){public}_{key}.sk(prf), +ensuring that these values remain consistent across different transactions or interactions without revealing sensitive information. For example, publickey.sk(prf){public}_{key}.sk(prf) may be +used in generating deterministic yet secure addresses or transaction references, which can be linked to the user’s activity in a controlled manner. By using publickey.sk(prf){public}_{key}.sk(prf), Nescience ensures that certain operations, like generating addresses or computing deterministic transaction outcomes, remain both private and cryptographically secure. The public key’s -role in this process is to maintain consistency in these outputs while preventing unauthorized parties from reverse engineering the associated private keys or transaction data.

    Summary

    Public keys in Nescience are essential for secure interactions within the network. "publickey.sig{public}_{key}.sig" allows others to verify that transactions were signed by the legitimate owner, -ensuring the authenticity of every operation. "publickey.rcm{public}_{key}.rcm" enables secure and private commitment schemes, allowing participants to commit to transaction details without -revealing sensitive information. Finally, "publickey.sk(prf){public}_{key}.sk(prf)" powers deterministic outputs through a pseudorandom function, ensuring that user-specific data remains consistent -and secure throughout various transactions. Together, these public key components facilitate privacy, security, and trust within NSSA, enabling seamless interactions while safeguarding user data.

    IV. Viewing key​

    The viewing key in NSSA is a specialized cryptographic key that allows a user to decrypt both incoming and outgoing transactions associated with their account. +role in this process is to maintain consistency in these outputs while preventing unauthorized parties from reverse engineering the associated private keys or transaction data.

    +

    Summary

    +

    Public keys in Nescience are essential for secure interactions within the network. "publickey.sig{public}_{key}.sig" allows others to verify that transactions were signed by the legitimate owner, +ensuring the authenticity of every operation. "publickey.rcm{public}_{key}.rcm" enables secure and private commitment schemes, allowing participants to commit to transaction details without +revealing sensitive information. Finally, "publickey.sk(prf){public}_{key}.sk(prf)" powers deterministic outputs through a pseudorandom function, ensuring that user-specific data remains consistent +and secure throughout various transactions. Together, these public key components facilitate privacy, security, and trust within NSSA, enabling seamless interactions while safeguarding user data.

    +

    IV. Viewing key​

    +

    The viewing key in NSSA is a specialized cryptographic key that allows a user to decrypt both incoming and outgoing transactions associated with their account. This key is designed to offer a degree of transparency to the user, enabling them to view the details of their transactions without compromising the security of their assets or granting -control over those assets.

    • Role of the viewing key: The primary function of the viewing key is to provide visibility into transaction details while maintaining the integrity of private, shielded, +control over those assets.

      +
        +
      • +

        Role of the viewing key: The primary function of the viewing key is to provide visibility into transaction details while maintaining the integrity of private, shielded, or deshielded transactions. It enables the user to see the specifics of the transactions they are involved in—such as amounts transferred, asset types, and metadata—without exposing the sensitive transaction data to the broader network. For instance, if Alice has executed a private transaction with Bob, her viewing key allows her to decrypt and review the details of the transaction, ensuring that everything was processed correctly. This ability to audit her own transactions helps Alice maintain confidence in the integrity -of her private interactions on the blockchain.

      • Security considerations: Despite its utility, the viewing key must be handled with care as its exposure could potentially compromise the user’s privacy. +of her private interactions on the blockchain.

        +
        • +
      • +

        Security considerations: Despite its utility, the viewing key must be handled with care as its exposure could potentially compromise the user’s privacy. Although possessing the viewing key does not provide the ability to spend or sign transactions (that authority remains strictly with the spending key and private keys), it does allow anyone with access to the viewing key to decrypt the details of the user’s private transactions. This means that if the viewing key is leaked or stolen, the privacy guarantees of Nescience’s private, shielded, and deshielded executions could be undermined. Specifically, the viewing key could be used to link various transactions, breaking the unlinkability of private transactions. For example, an attacker with access to the viewing key could decrypt past and future transactions, exposing the relationships between different parties and transaction flows. To mitigate this risk, Nescience recommends that users treat their viewing key with the same level of protection as their private keys. -It should be stored securely in encrypted hardware wallets or other secure storage solutions to prevent unauthorized access.

      • Balancing privacy and transparency: The viewing key provides an essential balance between privacy and transparency in NSSA. While it ensures that users +It should be stored securely in encrypted hardware wallets or other secure storage solutions to prevent unauthorized access.

        +
        • +
      • +

        Balancing privacy and transparency: The viewing key provides an essential balance between privacy and transparency in NSSA. While it ensures that users can monitor their transaction history and verify the details of their private transactions, it does so without compromising the control of their funds. This allows users to maintain a transparent view of their interactions while keeping their assets secure. For example, if Alice is using shielded execution to transfer assets, her viewing key enables her to audit the transaction without allowing anyone else, including Bob or external observers, to see the specific details unless they also have access to the viewing key. Moreover, since the viewing key does not grant signing or spending authority, even if it were exposed, an attacker would still not be able to manipulate the user’s assets. However, -to maintain the unlinkability and confidentiality of private transactions, the viewing key must be kept secure at all times.

      • Protecting transaction unlinkability: In private transactions, unlinkability is one of the core privacy guarantees. This property ensures that individual +to maintain the unlinkability and confidentiality of private transactions, the viewing key must be kept secure at all times.

        +
        • +
      • +

        Protecting transaction unlinkability: In private transactions, unlinkability is one of the core privacy guarantees. This property ensures that individual transactions cannot be correlated with each other or linked to the same user unless that user chooses to reveal the connection. The viewing key must be carefully protected to preserve this unlinkability, as its compromise could allow someone to map out a user’s private transaction history. For instance, in deshielded transactions, the viewing key allows the user to see which private UTXOs were consumed and how the public state was modified. If the viewing key is compromised, an attacker could potentially -link private UTXOs across multiple transactions, unraveling the user’s privacy.

      Conclusion

      The viewing key in Nescience is a powerful tool for providing insight into both incoming and outgoing transactions without granting control over assets. It allows users +link private UTXOs across multiple transactions, unraveling the user’s privacy.

      +
      • +
    +

    Conclusion

    +

    The viewing key in Nescience is a powerful tool for providing insight into both incoming and outgoing transactions without granting control over assets. It allows users to decrypt and verify their transaction details, maintaining transparency in their interactions. However, due to its potential to compromise privacy if exposed, the viewing key must be handled with great care. Proper security measures are necessary to protect the viewing key, ensuring that the unlinkability of private, shielded, and deshielded -transactions remains intact. In this way, the viewing key offers a crucial balance between privacy and transparency within the Nescience ecosystem.

    V. Ephemeral key​

    The ephemeral key is generated using a combination of the sender’s spending key and the UTXO's nullifier, ensuring that the key is unique to each transaction. -The process can be informally described as follows:

    1. Ephemeral key generation
      -Let ρ\rho denote the nullifier of the UTXO being consumed in the transaction. The sender uses the receiver’s public key component publickey.sk(prf){public}_{key}.sk(prf), -which is derived from the receiver’s private key, to compute an ephemeral secret key (eskesk). The computation is based on the nullifier ρ\rho and a base value:

      esk=publickey.sk(prf((0,0,0,0)∣∣ρ)esk = {public}_{key}.sk(prf((0,0,0,0) || \rho) -This formula binds the secret key to the specific transaction, leveraging the receiver’s cryptographic identity and the unique properties of the UTXO being spent.

    2. Deriving the ephemeral public key
      -After computing the ephemeral secret key (eskesk), the next step is to derive the corresponding ephemeral public key (epk). This is done using the Key Agreement -Protocol's DerivePublic algorithm, which generates the public key associated with the shared secret key. The ephemeral public key is computed as:

      epk=KA.DerivePublic(esk,gd)epk = KA.DerivePublic(esk, gd)

    Here, (gdgd) is the diversifier address associated with the receiver’s account. The diversifier address is computed from the receiver’s -account using the DiversifierHash function:

    gd=receiver.DiversifierHash(d)gd = receiver.DiversifierHash(d)

    The diversifier (dd) is a random value selected by the sender to add randomness to the process. This diversifier ensures that even if a single receiver is involved -in multiple transactions, the derived keys remain distinct for each transaction. The value (dd) is included in the transaction note for transparency and reproducibility.

    1. Establishing the shared secret
      The shared secret, used to encrypt the transaction details, is derived from the key agreement between the sender’s ephemeral key and the receiver’s viewing key. +transactions remains intact. In this way, the viewing key offers a crucial balance between privacy and transparency within the Nescience ecosystem.

      +

      V. Ephemeral key​

      +

      The ephemeral key is generated using a combination of the sender’s spending key and the UTXO's nullifier, ensuring that the key is unique to each transaction. +The process can be informally described as follows:

      +
        +
      1. Ephemeral key generation
        +Let ρ\rho denote the nullifier of the UTXO being consumed in the transaction. The sender uses the receiver’s public key component publickey.sk(prf){public}_{key}.sk(prf), +which is derived from the receiver’s private key, to compute an ephemeral secret key (eskesk). The computation is based on the nullifier ρ\rho and a base value:
        2. +
      +

      esk=publickey.sk(prf((0,0,0,0)∣∣ρ)esk = {public}_{key}.sk(prf((0,0,0,0) || \rho) +This formula binds the secret key to the specific transaction, leveraging the receiver’s cryptographic identity and the unique properties of the UTXO being spent.

      +
        +
      1. Deriving the ephemeral public key
        +After computing the ephemeral secret key (eskesk), the next step is to derive the corresponding ephemeral public key (epk). This is done using the Key Agreement +Protocol's DerivePublic algorithm, which generates the public key associated with the shared secret key. The ephemeral public key is computed as:
        2. +
      +

      epk=KA.DerivePublic(esk,gd)epk = KA.DerivePublic(esk, gd)

      +

      Here, (gdgd) is the diversifier address associated with the receiver’s account. The diversifier address is computed from the receiver’s +account using the DiversifierHash function:

      +

      gd=receiver.DiversifierHash(d)gd = receiver.DiversifierHash(d)

      +

      The diversifier (dd) is a random value selected by the sender to add randomness to the process. This diversifier ensures that even if a single receiver is involved +in multiple transactions, the derived keys remain distinct for each transaction. The value (dd) is included in the transaction note for transparency and reproducibility.

      +
        +
      1. Establishing the shared secret
        +The shared secret, used to encrypt the transaction details, is derived from the key agreement between the sender’s ephemeral key and the receiver’s viewing key. Any party possessing the receiver’s viewing key can use it in conjunction with the ephemeral key to compute the shared secret, which is then used to decrypt the transaction. -This ensures that only the intended recipient (or anyone with their viewing key) can access the transaction details.

      Key components and protocol

      The formal protocol for generating ephemeral keys closely follows this informal description but involves additional intermediate steps for converting values to +This ensures that only the intended recipient (or anyone with their viewing key) can access the transaction details.

      2. +
    +

    Key components and protocol

    +

    The formal protocol for generating ephemeral keys closely follows this informal description but involves additional intermediate steps for converting values to binary sequences to fit implementation requirements. These steps are essential for ensuring compatibility with cryptographic algorithms used in NSSA. -The protocol uses the following key components:

    • Nullifier (ρ\rho): Ensures that the ephemeral key is tied to the specific UTXO being consumed, preventing reuse of the key in future transactions.
    • Receiver’s public key (publickey.sk(prf){public}_{key}.sk(prf): Establishes the receiver's identity in the key generation process, ensuring that the shared secret can -only be derived by the intended party.
    • Diversifier (dd): Adds randomness to the transaction, ensuring that keys remain unique across different transactions involving the same receiver.

    The end result is an ephemeral key system that provides strong cryptographic guarantees for transaction privacy, leveraging key agreement protocols and secure -cryptographic primitives to prevent unauthorized access to sensitive transaction data.

    Conclusion

    The ephemeral key in Nescience is a critical element for maintaining transaction confidentiality. It facilitates a secure key agreement between the sender and the receiver, +The protocol uses the following key components:

    +
      +
    • Nullifier (ρ\rho): Ensures that the ephemeral key is tied to the specific UTXO being consumed, preventing reuse of the key in future transactions.
      • +
    • Receiver’s public key (publickey.sk(prf){public}_{key}.sk(prf): Establishes the receiver's identity in the key generation process, ensuring that the shared secret can +only be derived by the intended party.
      • +
    • Diversifier (dd): Adds randomness to the transaction, ensuring that keys remain unique across different transactions involving the same receiver.
      • +
    +

    The end result is an ephemeral key system that provides strong cryptographic guarantees for transaction privacy, leveraging key agreement protocols and secure +cryptographic primitives to prevent unauthorized access to sensitive transaction data.

    +

    Conclusion

    +

    The ephemeral key in Nescience is a critical element for maintaining transaction confidentiality. It facilitates a secure key agreement between the sender and the receiver, allowing for the encryption of transaction details with a shared secret that can only be derived by the intended recipient. By incorporating the nullifier, receiver's public key, and diversifier address, the ephemeral key ensures that transaction privacy is preserved while preventing unauthorized access to transaction information, even in a complex, -multi-party blockchain environment.

    VI. Nescience addresses​

    Nescience’s dual address system is a core component of its privacy-focused architecture, designed to balance transparency and confidentiality across different types of transactions. -The architecture provides each user or smart contract with both public addresses and private addresses, allowing them to participate in both open and confidential activities on the blockchain.

    a) Public addresses​

    Public addresses in Nescience are visible to all participants on the network and reside within the public state. These addresses are essential for engaging in +multi-party blockchain environment.

    +

    VI. Nescience addresses​

    +

    Nescience’s dual address system is a core component of its privacy-focused architecture, designed to balance transparency and confidentiality across different types of transactions. +The architecture provides each user or smart contract with both public addresses and private addresses, allowing them to participate in both open and confidential activities on the blockchain.

    +

    a) Public addresses​

    +

    Public addresses in Nescience are visible to all participants on the network and reside within the public state. These addresses are essential for engaging in transparent and verifiable interactions, such as sending tokens or invoking smart contracts that are meant to be publicly auditable. Public addresses serve as -the interface for users who need to engage with the transparent elements of the system, including public transactions or smart contracts that require public access.

    They are analogous to traditional blockchain addresses seen in systems like Ethereum or Bitcoin, where every participant can see the address and the transactions associated with it. +the interface for users who need to engage with the transparent elements of the system, including public transactions or smart contracts that require public access.

    +

    They are analogous to traditional blockchain addresses seen in systems like Ethereum or Bitcoin, where every participant can see the address and the transactions associated with it. For example, when Alice wants to receive tokens from Bob in a public transaction, she can provide her public address, allowing Bob to send the tokens transparently. -Anyone on the network can verify the transaction, providing accountability and trust in the public state.

    Because public addresses are visible and auditable, they are typically used for interactions where privacy is not a concern or where transparency is desirable. +Anyone on the network can verify the transaction, providing accountability and trust in the public state.

    +

    Because public addresses are visible and auditable, they are typically used for interactions where privacy is not a concern or where transparency is desirable. This could include simple token transfers, public contract calls, or interactions with dapps that require public accountability, -such as voting or governance systems.

    b) Private addresses​

    In contrast, private addresses are designed for confidentiality and are not visible onchain. These addresses are used exclusively for private transactions and executions, +such as voting or governance systems.

    +

    b) Private addresses​

    +

    In contrast, private addresses are designed for confidentiality and are not visible onchain. These addresses are used exclusively for private transactions and executions, ensuring that sensitive details—such as the sender, receiver, or amount transferred—remain hidden from the public state. Private addresses are a key feature of -Nescience’s private, shielded, and deshielded execution models, where preserving the confidentiality of participants is crucial.

    Users can generate an unlimited number of private addresses using their private keys. This flexibility allows users to compartmentalize their interactions, +Nescience’s private, shielded, and deshielded execution models, where preserving the confidentiality of participants is crucial.

    +

    Users can generate an unlimited number of private addresses using their private keys. This flexibility allows users to compartmentalize their interactions, giving them the ability to provide different private addresses to different parties. For instance, Alice could create a unique private address for each entity she interacts with, thereby ensuring that her transactions remain isolated and difficult to trace. This feature enhances privacy by preventing any direct linkage -between different transactions or activities associated with a single user.

    Private addresses are not tied to the public state and are only accessible through the user’s private key infrastructure. Transactions involving private addresses +between different transactions or activities associated with a single user.

    +

    Private addresses are not tied to the public state and are only accessible through the user’s private key infrastructure. Transactions involving private addresses are conducted within the confines of the private state and are only decrypted by the intended participants. For example, when Alice sends tokens to Bob using -a private address, the details of that transaction remain confidential, accessible only to Alice and Bob, unless they choose to reveal it.

    Role of the viewing key in private addresses: A key feature of Nescience’s private address system is the viewing key, which allows users to decrypt any transaction +a private address, the details of that transaction remain confidential, accessible only to Alice and Bob, unless they choose to reveal it.

    +

    Role of the viewing key in private addresses: A key feature of Nescience’s private address system is the viewing key, which allows users to decrypt any transaction involving their private addresses. This capability provides oversight and transparency into the user’s private transactions, ensuring that they can monitor their own activity without exposing the details to the public. The viewing key does not compromise the security of the user's assets as it does not grant spending or signing authority. However, it does allow the user to audit and verify the accuracy of their private transactions, ensuring that everything proceeds as expected. For instance, Alice can use her viewing key to review the details of a private transaction she conducted with Bob, ensuring that the correct amount was transferred and that the transaction was properly processed. This functionality is critical for users who want to maintain control over their private interactions while still benefiting from transparency into their transaction history. -The ability to generate multiple private addresses and decrypt them with the viewing key ensures that users can maintain compartmentalized privacy without sacrificing oversight.

    Summary

    Nescience’s dual address system—comprising public and private addresses—provides users with the flexibility to engage in both transparent and confidential transactions. +The ability to generate multiple private addresses and decrypt them with the viewing key ensures that users can maintain compartmentalized privacy without sacrificing oversight.

    +

    Summary

    +

    Nescience’s dual address system—comprising public and private addresses—provides users with the flexibility to engage in both transparent and confidential transactions. Public addresses are visible onchain and are used for open, public interactions that require accountability and auditability. In contrast, private addresses are -invisible onchain and are used for confidential transactions, enhancing privacy and security.

    By allowing users to generate multiple private addresses, Nescience gives individuals control over the visibility of their transactions. Combined with the viewing +invisible onchain and are used for confidential transactions, enhancing privacy and security.

    +

    By allowing users to generate multiple private addresses, Nescience gives individuals control over the visibility of their transactions. Combined with the viewing key’s ability to decrypt transactions involving private addresses, the system ensures that users can maintain transparency over their private transactions without exposing sensitive information to the public state. This dual-address approach enables users to seamlessly switch between public and private interactions depending on their needs, -providing a robust framework for both privacy and transparency in NSSA.

    VII. Conclusion​

    Key management in NSSA is a carefully designed system that strikes an optimal balance between security, privacy, and flexibility. +providing a robust framework for both privacy and transparency in NSSA.

    +

    VII. Conclusion​

    +

    Key management in NSSA is a carefully designed system that strikes an optimal balance between security, privacy, and flexibility. The architecture’s hierarchical structure, with distinct roles for the spending key, private keys, and public keys, ensures that users retain full control over their assets while maintaining the integrity of their transactions. The spending key, as the root of security, provides unassailable control over the user's UTXOs and assets, ensuring that only the rightful owner can authorize spending. Private keys, derived from the spending key, enable users to engage -in cryptographic operations such as signing transactions and generating commitments without exposing sensitive information to the network.

    The viewing key adds another layer of transparency, allowing users to decrypt and review their transactions without compromising their authority over their assets. -While it provides a window into transaction history, the viewing key does not grant spending power, preserving the critical separation between visibility and control.

    The dual system of public and private addresses gives users the flexibility to navigate between open, transparent transactions and confidential, privacy-protected activities. +in cryptographic operations such as signing transactions and generating commitments without exposing sensitive information to the network.

    +

    The viewing key adds another layer of transparency, allowing users to decrypt and review their transactions without compromising their authority over their assets. +While it provides a window into transaction history, the viewing key does not grant spending power, preserving the critical separation between visibility and control.

    +

    The dual system of public and private addresses gives users the flexibility to navigate between open, transparent transactions and confidential, privacy-protected activities. Public addresses allow users to engage in verifiable, public interactions while private addresses enable compartmentalized, secure transactions that remain hidden from the public eye. This dual-address framework ensures that users can seamlessly adapt to different privacy requirements, whether they are participating in public -dapps or conducting sensitive financial operations.

    Overall, Nescience’s cryptographic infrastructure is designed to empower users to engage confidently in both transparent and confidential activities. +dapps or conducting sensitive financial operations.

    +

    Overall, Nescience’s cryptographic infrastructure is designed to empower users to engage confidently in both transparent and confidential activities. By providing flexible, secure key management and address systems, Nescience ensures that users can fully participate in the blockchain ecosystem without compromising their privacy or control. The architecture supports the nuanced needs of modern blockchain users, who require both the transparency of public -interactions and the security of private transactions, all while maintaining the integrity and confidentiality of their assets.

    c) Trees in NSSA​

    Trees in NSSA serve as verifiable databases, essential for maintaining privacy and security. Different types of trees are used for various purposes:

    1. Global state tree: The global state tree is a single, public tree that holds all public assets and storage information. It acts as a central repository for all -publicly accessible data on the blockchain. By organizing this data in a Merkle tree structure, the Global State Tree allows for efficient and secure verification of public information.

    2. Hashed UTXO tree: The hashed UTXO tree is a public tree that contains hashes of all created UTXOs. When users wish to consume a UTXO, they provide a membership +interactions and the security of private transactions, all while maintaining the integrity and confidentiality of their assets.

      +

      c) Trees in NSSA​

      +

      Trees in NSSA serve as verifiable databases, essential for maintaining privacy and security. Different types of trees are used for various purposes:

      +
        +
      1. +

        Global state tree: The global state tree is a single, public tree that holds all public assets and storage information. It acts as a central repository for all +publicly accessible data on the blockchain. By organizing this data in a Merkle tree structure, the Global State Tree allows for efficient and secure verification of public information.

        +
        2. +
      2. +

        Hashed UTXO tree: The hashed UTXO tree is a public tree that contains hashes of all created UTXOs. When users wish to consume a UTXO, they provide a membership proof to demonstrate that the UTXO exists within this tree. This process ensures that only valid and existing UTXOs can be spent, maintaining the integrity of transactions. In fact, users generate membership proofs that verify the presence of specific UTXOs in the tree without revealing their actual data. The benefit here is that the Merkle -tree structure allows for quick and efficient verification of UTXO existence.

      3. UTXO trees (private states): Each user or smart contract has its private state stored in UTXO trees. These trees are kept as plaintext on the client’s +tree structure allows for quick and efficient verification of UTXO existence.

        +
        4. +
      4. +

        UTXO trees (private states): Each user or smart contract has its private state stored in UTXO trees. These trees are kept as plaintext on the client’s local system (off-chain), ensuring privacy as sensitive information remains confidential. The private state includes all UTXOs owned by the user or the smart contract, and these -are not directly exposed to the public blockchain. For instance, users have full control over their private state, which is not visible to other participants in the network.

      In conclusion, the tree structures enable efficient verification of transaction validity without compromising privacy. By using Merkle trees, +are not directly exposed to the public blockchain. For instance, users have full control over their private state, which is not visible to other participants in the network.

      +
      3. +
    +

    In conclusion, the tree structures enable efficient verification of transaction validity without compromising privacy. By using Merkle trees, Nescience ensures that any tampering with the data can be easily detected. The efficient structure of these trees supports the scalability of the architecture, allowing it to handle a large number of transactions and data entries. By leveraging different types of trees, Nescience ensures efficient and secure management -of both public and private states.

    ## d) Nullifier tree in Nescience

    The nullifier tree is a fundamental component of NSSA, designed to prevent double spending by securely tracking all consumed UTXOs. -This tree acts as a public ledger of spent UTXOs, ensuring that once a UTXO is consumed in a transaction, it cannot be reused in future transactions.

    The primary function of the nullifier Tree is to store the nullifiers of all consumed UTXOs. By recording the nullifiers in a public tree, -the system ensures that each UTXO is spent only once, thereby safeguarding the integrity of the entire network.

    • Ensuring non-membership and preventing double spending +of both public and private states.

      +

      ## d) Nullifier tree in Nescience

      +

      The nullifier tree is a fundamental component of NSSA, designed to prevent double spending by securely tracking all consumed UTXOs. +This tree acts as a public ledger of spent UTXOs, ensuring that once a UTXO is consumed in a transaction, it cannot be reused in future transactions.

      +

      The primary function of the nullifier Tree is to store the nullifiers of all consumed UTXOs. By recording the nullifiers in a public tree, +the system ensures that each UTXO is spent only once, thereby safeguarding the integrity of the entire network.

      +
        +
      • +

        Ensuring non-membership and preventing double spending Before a user can consume a UTXO in a transaction, they must provide a non-membership proof. This proof demonstrates that the UTXO’s nullifier does not already exist in the Nullifier Tree, proving that the UTXO has not been spent before. If the UTXO’s nullifier is found in the tree, the system will reject the transaction, preventing double spending. The non-membership proof ensures that users cannot attempt to spend the same UTXO in multiple transactions. This mechanism is critical for maintaining the security and reliability of NSSA. The tree structure, which is typically built using a cryptographic tree like a Merkle tree, allows for efficient verification of nullifiers. -Verifiers can quickly check whether a nullifier is present or absent in the tree, ensuring that each UTXO is only spent once.

      • Nullifier tree structure and operation +Verifiers can quickly check whether a nullifier is present or absent in the tree, ensuring that each UTXO is only spent once.

        +
        • +
      • +

        Nullifier tree structure and operation The nullifier tree is likely structured as a Merkle tree, which is a cryptographic binary tree where each node represents the hash of its child nodes. This structure allows for efficient storage and verification of large sets of nullifiers as only the root hash of the tree needs to be stored on the blockchain. When a new nullifier is added to the tree, the tree is recalculated, and the root hash is updated. This process ensures that all consumed UTXOs are securely recorded. @@ -1362,56 +3085,135 @@ Each time a transaction consumes a UTXO, the nullifier is added to the Nullifier UTXO has not been double spent, verifiers can use the tree’s root hash and a proof of inclusion or exclusion (membership or non-membership proof) to check whether the nullifier is present in the tree. For example, if Alice wants to spend a UTXO, she must prove that the nullifier associated with that UTXO is not already in the Nullifier Tree. She generates a non-membership proof that shows her nullifier is not recorded in the tree, and the transaction is allowed to proceed. Once the transaction is completed, -the nullifier is added to the tree, ensuring that the UTXO cannot be used again.

      Conclusion +the nullifier is added to the tree, ensuring that the UTXO cannot be used again.

      +
      • +
    +

    Conclusion The Nullifier Tree is a crucial element of Nescience's security. By recording all consumed UTXOs and ensuring that nullifiers are unique, the tree prevents double spending and maintains the integrity of the blockchain. The non-membership proof mechanism guarantees that every transaction is validated against the tree. This structure supports -the scalability and security of NSSA, providing a reliable method for verifying the validity of transactions while preventing malicious behavior.

    e) Recursive-friendly privacy-preserving zk-zkVM​

    The development of the zk-zkVM in Nescience is a work in progress, as the architecture continues to evolve to support privacy-preserving transactions +the scalability and security of NSSA, providing a reliable method for verifying the validity of transactions while preventing malicious behavior.

    +

    e) Recursive-friendly privacy-preserving zk-zkVM​

    +

    The development of the zk-zkVM in Nescience is a work in progress, as the architecture continues to evolve to support privacy-preserving transactions and efficient ZKP generation. The goal of the zk-zkVM is to seamlessly integrate with the Nescience state-separation architecture, -ensuring that private transactions remain confidential while allowing the network to verify their validity without compromising privacy.

    Currently, we are exploring and testing several existing zkVMs to identify the most suitable platform for our needs. Our focus is on finding a zkVM +ensuring that private transactions remain confidential while allowing the network to verify their validity without compromising privacy.

    +

    Currently, we are exploring and testing several existing zkVMs to identify the most suitable platform for our needs. Our focus is on finding a zkVM that not only supports the core features of Nescience, such as state separation and privacy, but also provides the efficiency and scalability required for a decentralized system. Once a suitable zkVM is chosen, we will begin implementing advanced privacy features on top of it, including support for -confidential transactions, selective disclosure, and recursive proof aggregation.

    The integration of these privacy-preserving features with an existing zkVM will enable Nescience to fully employ its state-separation architecture, +confidential transactions, selective disclosure, and recursive proof aggregation.

    +

    The integration of these privacy-preserving features with an existing zkVM will enable Nescience to fully employ its state-separation architecture, ensuring that users can conduct private transactions with robust security and scalability. This approach will allow us to leverage the strengths of -proven zkVM technologies while enhancing them with the unique privacy and state-separation capabilities that Nescience requires.

    • Privacy-preserving features: At its core, the zk-zkVM is designed with privacy in mind. One of the zk-zkVM’s standout privacy features is selective disclosure, +proven zkVM technologies while enhancing them with the unique privacy and state-separation capabilities that Nescience requires.

      +
        +
      • +

        Privacy-preserving features: At its core, the zk-zkVM is designed with privacy in mind. One of the zk-zkVM’s standout privacy features is selective disclosure, which allows users to reveal only specific details of a transaction as needed. For example, a user could disclose the transaction amount while concealing the identities of the participants. The zk-zkVM employs advanced encryption techniques to protect this sensitive data. All transaction data is encrypted before being stored on the blockchain, so even if the data is intercepted, it cannot be deciphered without the appropriate decryption keys. Another of the crucial privacy-preserving features is the support for confidential transactions. Only the parties involved in the transaction can access the encrypted data. Furthermore, the zk-zkVM supports verifiable encryption, a powerful capability that allows encrypted data to be included in ZKPs without needing to decrypt it. This ensures that transaction details remain private -while their correctness can still be proven.

      • Lightweight design for accessibility: The zk-zkVM is being designed to be lightweight and efficient, enabling it to run on standard consumer-grade hardware. -This makes it accessible to a wide range of users without requiring specialized equipment or significant computational resources.

      • Faster proving time: To maintain a seamless user experience, especially during high transaction volumes, the zk-zkVM is being optimized for fast proving times. -Fast proof generation is particularly important for ensuring that the system remains usable during periods of peak activity, preventing bottlenecks and maintaining the fluidity of the network.

      • Recursive-friendly operations: One of the most advanced features of the zk-zkVM will be its support for recursive operations. Recursion enables the aggregation -of multiple proofs into a single proof, improving efficiency on both the client and sequencer sides of the network.

      • Client-side recursion (batch processing): When a single transaction involves multiple executions, each requiring its own ZKP, these individual +while their correctness can still be proven.

        +
        • +
      • +

        Lightweight design for accessibility: The zk-zkVM is being designed to be lightweight and efficient, enabling it to run on standard consumer-grade hardware. +This makes it accessible to a wide range of users without requiring specialized equipment or significant computational resources.

        +
        • +
      • +

        Faster proving time: To maintain a seamless user experience, especially during high transaction volumes, the zk-zkVM is being optimized for fast proving times. +Fast proof generation is particularly important for ensuring that the system remains usable during periods of peak activity, preventing bottlenecks and maintaining the fluidity of the network.

        +
        • +
      • +

        Recursive-friendly operations: One of the most advanced features of the zk-zkVM will be its support for recursive operations. Recursion enables the aggregation +of multiple proofs into a single proof, improving efficiency on both the client and sequencer sides of the network.

        +
        • +
      • +

        Client-side recursion (batch processing): When a single transaction involves multiple executions, each requiring its own ZKP, these individual proofs can be recursively aggregated before being sent to the sequencer. This reduces the overall data transmitted, enhancing the efficiency of the transaction process -by compressing multiple proofs into a single package. -

      • Sequencer-side recursion (reduced redundancy): The sequencer, which is responsible for processing transactions and creating verifiable blocks, collects transactions +by compressing multiple proofs into a single package.

        +
        • +
      • +

        Sequencer-side recursion (reduced redundancy): The sequencer, which is responsible for processing transactions and creating verifiable blocks, collects transactions containing aggregated proofs. These proofs are further merged into a single comprehensive proof, ensuring that all transactions within a block are validated collectively. -This process reduces redundancy and optimizes the blockchain’s efficiency by minimizing the size and complexity of the proofs required for verification.

      • Developer-friendly language: To foster widespread adoption and innovation within the Nescience ecosystem, the zk-zkVM would include a developer-friendly language. +This process reduces redundancy and optimizes the blockchain’s efficiency by minimizing the size and complexity of the proofs required for verification.

        +
        • +
      • +

        Developer-friendly language: To foster widespread adoption and innovation within the Nescience ecosystem, the zk-zkVM would include a developer-friendly language. This high-level language simplifies the process of building applications that leverage state separation and privacy-preserving transactions. The language should offer extensive support for modular design, APIs, and SDKs, enabling developers to integrate their applications with the zk-zkVM more easily. By lowering the barrier to entry, Nescience encourages -innovation and helps expand the range of privacy-preserving applications that can be built on its platform.

      Conclusion

      The zk-zkVM in Nescience is a powerful and versatile virtual machine that embodies the principles of privacy, efficiency, and scalability. By supporting ZKPs +innovation and helps expand the range of privacy-preserving applications that can be built on its platform.

      +
      • +
    +

    Conclusion

    +

    The zk-zkVM in Nescience is a powerful and versatile virtual machine that embodies the principles of privacy, efficiency, and scalability. By supporting ZKPs and integrating with advanced privacy technologies like homomorphic encryption. Its lightweight design allows it to run efficiently on standard hardware, promoting decentralization, and its recursive operations further enhance the system's scalability. With its developer-friendly language and fast proving times, the zk-zkVM is positioned as a key component in -fostering the growth and adoption of privacy-preserving blockchain applications.

    f) MPC-based synchronization mechanism (under review)​

    Nescience is developing an MPC-based synchronization mechanism to balance privacy and fairness between public and private execution types. +fostering the growth and adoption of privacy-preserving blockchain applications.

    +

    f) MPC-based synchronization mechanism (under review)​

    +

    Nescience is developing an MPC-based synchronization mechanism to balance privacy and fairness between public and private execution types. This mechanism extracts common information from encrypted UTXOs without revealing private details, ensuring privacy and preventing UTXO linkage to users or specific transactions. -It guarantees that public and private executions remain equitable, with the total input equaling the public output.

    The mechanism employs MPC protocols to perform computations privately, ZKPs to verify correctness, and cryptographic protocols +It guarantees that public and private executions remain equitable, with the total input equaling the public output.

    +

    The mechanism employs MPC protocols to perform computations privately, ZKPs to verify correctness, and cryptographic protocols to secure data during synchronization. This ensures a consistent and fair environment for all users, regardless of their chosen privacy level. Currently, -this feature is under development and review for potential inclusion depending on the research output and compatibility.

    D. Future plans for Nescience

    Nescience is committed to continuously evolving its architecture to ensure scalability, privacy, and security in a growing blockchain landscape. +this feature is under development and review for potential inclusion depending on the research output and compatibility.

    +

    D. Future plans for Nescience

    +

    Nescience is committed to continuously evolving its architecture to ensure scalability, privacy, and security in a growing blockchain landscape. One of the primary goals is to integrate the zk-zkVM and the Nescience state-separation architecture into a fully functioning node, -enabling efficient private transactions while maintaining network integrity.

    • Addressing scalability challenges: A key challenge facing Nescience is the increasing size of nullifier and hashed UTXO trees, which could impact -network performance and scalability over time. To mitigate this, Nescience plans to adopt state-of-the-art scalable privacy techniques such as:
      • Mutator sets: Dynamically adjusting data structures to manage the growth of the nullifier set efficiently.
      • SNARK-based accumulators: Compressing data in a verifiable way to ensure that only relevant information is stored while maintaining cryptographic security.
      • Pruning techniques: Periodically trimming unnecessary data from trees to maintain optimal size and performance, ensuring that the network scales logarithmically -rather than exponentially as more transactions occur.

    By implementing these approaches, Nescience aims to keep the size of its data structures manageable, ensuring that scalability does not come at the cost of performance or privacy.

    • Enhanced key management: Another critical focus for Nescience is improving key management to streamline operations and enhance security. +enabling efficient private transactions while maintaining network integrity.

      +
        +
      • Addressing scalability challenges: A key challenge facing Nescience is the increasing size of nullifier and hashed UTXO trees, which could impact +network performance and scalability over time. To mitigate this, Nescience plans to adopt state-of-the-art scalable privacy techniques such as: +
          +
        • Mutator sets: Dynamically adjusting data structures to manage the growth of the nullifier set efficiently.
          • +
        • SNARK-based accumulators: Compressing data in a verifiable way to ensure that only relevant information is stored while maintaining cryptographic security.
          • +
        • Pruning techniques: Periodically trimming unnecessary data from trees to maintain optimal size and performance, ensuring that the network scales logarithmically +rather than exponentially as more transactions occur.
          • +
        +
        • +
      +

      By implementing these approaches, Nescience aims to keep the size of its data structures manageable, ensuring that scalability does not come at the cost of performance or privacy.

      +
        +
      • +

        Enhanced key management: Another critical focus for Nescience is improving key management to streamline operations and enhance security. The plan is to integrate the different keys used for signatures, addresses, UTXO encryption, and SNARK verification into a unified system. This integration will simplify key management for users while reducing the risk of security breaches caused by complex, disparate key systems. Nescience also plans to implement Hierarchical Deterministic (HD) keys, which allow users to derive multiple keys from a single seed, enhancing both security and usability. This approach reduces the complexity of managing multiple keys across various functions and provides an additional layer of protection for private transactions. Additionally, multi-signature schemes will be introduced, requiring multiple parties to authorize transactions. -This feature increases security by reducing the likelihood of unauthorized access, ensuring that a single compromised key cannot lead to malicious transactions.

      • Integrating advanced cryptographic techniques: Nescience will integrate advanced cryptographic techniques, enhancing both privacy and scalability. Among these are:

        • Homomorphic encryption: Allowing computations to be performed on encrypted data without the need to decrypt it, preserving privacy while enabling secure, complex data processing.
        • Zero-knowledge rollups: Bundling multiple transactions into a single proof to reduce the amount of data processed and stored on the blockchain, -significantly improving scalability without sacrificing security.

      These cryptographic enhancements will ensure that Nescience can support a growing network while continuing to protect user privacy and maintaining high transaction throughput.

      • Long-term vision

      The ultimate goal for Nescience is to deploy a fully operational node powered by zk-zkVM and the Nescience state-separation architecture. +This feature increases security by reducing the likelihood of unauthorized access, ensuring that a single compromised key cannot lead to malicious transactions.

      +
      • +
    • +

      Integrating advanced cryptographic techniques: Nescience will integrate advanced cryptographic techniques, enhancing both privacy and scalability. Among these are:

      +
        +
      • Homomorphic encryption: Allowing computations to be performed on encrypted data without the need to decrypt it, preserving privacy while enabling secure, complex data processing.
        • +
      • Zero-knowledge rollups: Bundling multiple transactions into a single proof to reduce the amount of data processed and stored on the blockchain, +significantly improving scalability without sacrificing security.
        • +
      +
      • +
    +

    These cryptographic enhancements will ensure that Nescience can support a growing network while continuing to protect user privacy and maintaining high transaction throughput.

    +
      +
    • Long-term vision
      • +
    +

    The ultimate goal for Nescience is to deploy a fully operational node powered by zk-zkVM and the Nescience state-separation architecture. This node will handle complex, private transactions at scale while integrating all of the advanced cryptographic techniques outlined in the roadmap. -Nescience aims to provide users with an infrastructure that balances privacy, security, and efficiency, ensuring the network remains resilient and capable of handling future demands.

    By pursuing these future plans, Nescience is poised to not only address current challenges around scalability and key management but also lead the way in +Nescience aims to provide users with an infrastructure that balances privacy, security, and efficiency, ensuring the network remains resilient and capable of handling future demands.

    +

    By pursuing these future plans, Nescience is poised to not only address current challenges around scalability and key management but also lead the way in applying advanced cryptography to decentralized systems. This vision will help secure the long-term integrity and performance of the Nescience state-separation -model as the blockchain grows and evolves.

    References

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrieved from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    [4] Wuille, P. (2012). Bitcoin Improvement Proposal (BIP) 32. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    [5] Sin7y Tech Review (29): Design Principles of Private Transactions in Aleo & Zcash. Retrieved from https://hackmd.io/@sin7y/rkxFXLkgs

    [6] Sin7y Tech Review (33): Principles of private transactions and regulatory compliance issues. Retrieved from https://hackmd.io/@sin7y/S16RyFzZn

    [7] Zcash Protocol Specification. Retrieved from https://zips.z.cash/protocol/protocol.pdf

    [8] Anatomy of a Zcash Transaction. Retrieved from https://electriccoin.co/blog/anatomy-of-zcash

    [9] The Penumbra Protocol: Notes, Nullifiers, and Trees. Retrieved from https://protocol.penumbra.zone/main/concepts/notes_nullifiers_trees.html

    [10] Zero-knowledge Virtual Machine (ZKVM). Retrieved from https://medium.com/@abhilashkrish/zero-knowledge-virtual-machine-zkvm-95adc2082cfd

    [11] What's a Sparse Merkle tree?. Retrieved from https://medium.com/@kelvinfichter/whats-a-sparse-merkle-tree-acda70aeb837

    [12] Lecture 10: Accounts Model and Merkle Trees. Retrieved from https://web.stanford.edu/class/ee374/lec_notes/lec10.pdf

    [13] The UTXO vs Account Model. Retrieved from https://www.horizen.io/academy/utxo-vs-account-model/

    [14] Addresses and Value Pools in Zcash. Retrieved from https://zcash.readthedocs.io/en/latest/rtd_pages/addresses.html

    ]]> +model as the blockchain grows and evolves.

    +

    References

    +

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    +

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrieved from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    +

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    +

    [4] Wuille, P. (2012). Bitcoin Improvement Proposal (BIP) 32. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    +

    [5] Sin7y Tech Review (29): Design Principles of Private Transactions in Aleo & Zcash. Retrieved from https://hackmd.io/@sin7y/rkxFXLkgs

    +

    [6] Sin7y Tech Review (33): Principles of private transactions and regulatory compliance issues. Retrieved from https://hackmd.io/@sin7y/S16RyFzZn

    +

    [7] Zcash Protocol Specification. Retrieved from https://zips.z.cash/protocol/protocol.pdf

    +

    [8] Anatomy of a Zcash Transaction. Retrieved from https://electriccoin.co/blog/anatomy-of-zcash

    +

    [9] The Penumbra Protocol: Notes, Nullifiers, and Trees. Retrieved from https://protocol.penumbra.zone/main/concepts/notes_nullifiers_trees.html

    +

    [10] Zero-knowledge Virtual Machine (ZKVM). Retrieved from https://medium.com/@abhilashkrish/zero-knowledge-virtual-machine-zkvm-95adc2082cfd

    +

    [11] What's a Sparse Merkle tree?. Retrieved from https://medium.com/@kelvinfichter/whats-a-sparse-merkle-tree-acda70aeb837

    +

    [12] Lecture 10: Accounts Model and Merkle Trees. Retrieved from https://web.stanford.edu/class/ee374/lec_notes/lec10.pdf

    +

    [13] The UTXO vs Account Model. Retrieved from https://www.horizen.io/academy/utxo-vs-account-model/

    +

    [14] Addresses and Value Pools in Zcash. Retrieved from https://zcash.readthedocs.io/en/latest/rtd_pages/addresses.html

    ]]>     <![CDATA[Vac 101: Membership with Bloom Filters and Cuckoo Filters]]> @@ -1419,84 +3221,193 @@ model as the blockchain grows and evolves.

    References

    [1] https://vac.dev/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters Fri, 19 Jul 2024 12:00:00 GMT - We examine two data structures: Bloom filters and Cuckoo filters.

    Membership with Bloom Filters and Cuckoo Filters​

    The ability to efficiently query the membership of an element in a given data set is crucial. + We examine two data structures: Bloom filters and Cuckoo filters.

    + +

    Membership with Bloom Filters and Cuckoo Filters​

    +

    The ability to efficiently query the membership of an element in a given data set is crucial. In certain applications, it is more important to output a result quickly than to have a 'perfect' result. In particular, false positives may be an acceptable tradeoff for speed. In this blog, we examine Bloom and Cuckoo data filters. -Both of these filters are data structures that can be used for membership proofs.

    Everyone is familiar with the process of creating a new account for various websites, whether it is an e-mail account or a social media account. +Both of these filters are data structures that can be used for membership proofs.

    +

    Everyone is familiar with the process of creating a new account for various websites, whether it is an e-mail account or a social media account. Consider when you enter your desired username. Many sites provide real-time feedback, as you type, on the availability of a given string. In this scenario, it is necessary that the result is seemingly instant, regardless of the number of existing accounts. However, it is not important that the usernames that are flagged as unavailable are, in fact, in use. -That is, it is sufficient to have a probabilistic check for membership.

    Bloom filters and Cuckoo filters are data structures that can be used to accumulate data with a fixed amount of space. -The associated filter FF for a digest of data DD can be queried to determine whether an element is (possibly) a member of DD:

    • 0: The queried element is definitely not a member of digest DD.
    • 1: The entry is possibly a member of the digest DD.

    The algorithms associated with Bloom filters and Cuckoo filters, which we will discuss shortly, are deterministic. -The possibility of false positives arises from the query algorithm.

    Bloom filters​

    A Bloom filter is a data structure that can be used to accumulate an arbitrary amount of data with a fixed amount of space. +That is, it is sufficient to have a probabilistic check for membership.

    +

    Bloom filters and Cuckoo filters are data structures that can be used to accumulate data with a fixed amount of space. +The associated filter FF for a digest of data DD can be queried to determine whether an element is (possibly) a member of DD:

    +
      +
    • 0: The queried element is definitely not a member of digest DD.
      • +
    • 1: The entry is possibly a member of the digest DD.
      • +
    +

    The algorithms associated with Bloom filters and Cuckoo filters, which we will discuss shortly, are deterministic. +The possibility of false positives arises from the query algorithm.

    +

    Bloom filters​

    +

    A Bloom filter is a data structure that can be used to accumulate an arbitrary amount of data with a fixed amount of space. Bloom filters have been a popular data structure for proof of non-membership due to their small storage size. -Specifically, a Bloom filter consists of a binary string v∈{0,1}n{\bf{v}} \in \{0,1\}^n and kk hash functions {hi:{0,1}∗→{0,…,n−1}}i=0k−1\{h_i: \{0,1\}^* \rightarrow \{0,\dots,n-1\}\}_{i=0}^{k-1}. -We note that each hash function hih_i is used to determine an index of our binary string v{\bf{v}} to flip the associated bit to 1. -The binary string v{\bf{v}} is initialized with every entry as 0. -The hash functions do not need to be cryptographic hash functions.

    • Append: Suppose that we wish to add the element xx to the Bloom filter.
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=hi(x){\bf{b}}[i] := h_i(x) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
      • Update the binary string v[b[i]]←1{\bf{v}}[{\bf{b}}[i]] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
    • Query: Suppose that we wish to query the Bloom filter for element yy.
      • Return 1 provided v[hi(y)]=1{\bf{v}}[h_i(y)] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.

    The algorithm Query will output 1 for every element yy that has been added to the Bloom filter. +Specifically, a Bloom filter consists of a binary string v∈{0,1}n{\bf{v}} \in \{0,1\}^n and kk hash functions {hi:{0,1}∗→{0,…,n−1}}i=0k−1\{h_i: \{0,1\}^* \rightarrow \{0,\dots,n-1\}\}_{i=0}^{k-1}. +We note that each hash function hih_i is used to determine an index of our binary string v{\bf{v}} to flip the associated bit to 1. +The binary string v{\bf{v}} is initialized with every entry as 0. +The hash functions do not need to be cryptographic hash functions.

    +
      +
    • +

      Append: Suppose that we wish to add the element xx to the Bloom filter.

      +
        +
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=hi(x){\bf{b}}[i] := h_i(x) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      • Update the binary string v[b[i]]←1{\bf{v}}[{\bf{b}}[i]] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Bloom filter for element yy.

      +
        +
      • Return 1 provided v[hi(y)]=1{\bf{v}}[h_i(y)] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.
        • +
      +
      • +
    +

    The algorithm Query will output 1 for every element yy that has been added to the Bloom filter. This is a consequence of the Append algorithm. However, due to potential collisions over a set of hash functions, it is possible for false positives to occur. -Moreover, the possibility of collisions makes it impossible to remove elements from the Bloom filter.

    Complexity​

    The storage of a Bloom filter requires constant space. -Specifically, the Bloom filter uses nn bits regardless of the size of the digest. -So, regardless of the number of elements that we append, the Bloom filter will use nn bits. -Further, if we assume that each of the kk hash functions runs in constant time, then we can append/query an entry in O(k)O(k).

    Example​

    Suppose that k=3k = 3 and n=10n = 10. -Our Bloom filter is initialized as v=(0000000000).\bf{v} = \begin{pmatrix}0&0&0&0&0&0&0&0&0&0\end{pmatrix}. -Now, we will append the words addadd, sumsum, and equalequal. -Suppose that

    h0(add)=1h1(add)=4h2(add)=7h0(sum)=9h1(sum)=2h2(sum)=1h0(equal)=5h1(equal)=8h2(equal)=0.\begin{matrix} h_0(add) = 1 & h_1(add) = 4 & h_2(add) = 7\\ h_0(sum) = 9 & h_1(sum) = 2 & h_2(sum) = 1\\ h_0(equal) = 5 & h_1(equal) = 8 & h_2(equal) = 0. \end{matrix}

    After appending these words, the Bloom filter is v=(1110110111).\bf{v} = \begin{pmatrix}1&1&1&0&1&1&0&1&1&1\end{pmatrix}.

    Now, suppose that we query the words subtractsubtract and multiplemultiple so that

    h0(subtract)=3h1(subtract)=5h2(subtract)=1h0(multiple)=7h1(multiple)=1h2(multiple)=4\begin{matrix} h_0(subtract) = 3 & h_1(subtract) = 5 & h_2(subtract) = 1\\ h_0(multiple) = 7 & h_1(multiple) = 1 & h_2(multiple) = 4\\ \end{matrix}.

    The query for subtractsubtract returns 0 since v[3]=0{\bf{v}}[3]=0. -On the other hand, the query for multiplemultiple returns 1 since v[1]=1,v[4]=1{\bf{v}}[1]=1, {\bf{v}}[4] = 1, and v[7]=1{\bf{v}}[7]=1. -Even though multiplemultiple was not used to generate the Bloom filter v{\bf{v}}, our query returns the false positive.

    Probability of false positives​

    For our analysis, we will assume that the probabilities that arise in our analysis are independent. -However, this assumption can be removed to gain the same approximation.

    We note that for a single hash function, the probability that a specific bit is flipped to 1 is 1/n1/n. -So, the probability that the specific bit is not flipped by the hash function is 1−1/n1-1/n. -Applying our assumption that the kk hash functions are 'independent,' +Moreover, the possibility of collisions makes it impossible to remove elements from the Bloom filter.

    +

    Complexity​

    +

    The storage of a Bloom filter requires constant space. +Specifically, the Bloom filter uses nn bits regardless of the size of the digest. +So, regardless of the number of elements that we append, the Bloom filter will use nn bits. +Further, if we assume that each of the kk hash functions runs in constant time, then we can append/query an entry in O(k)O(k).

    +

    Example​

    +

    Suppose that k=3k = 3 and n=10n = 10. +Our Bloom filter is initialized as v=(0000000000).\bf{v} = \begin{pmatrix}0&0&0&0&0&0&0&0&0&0\end{pmatrix}. +Now, we will append the words addadd, sumsum, and equalequal. +Suppose that

    +

    h0(add)=1h1(add)=4h2(add)=7h0(sum)=9h1(sum)=2h2(sum)=1h0(equal)=5h1(equal)=8h2(equal)=0.\begin{matrix} +h_0(add) = 1 & h_1(add) = 4 & h_2(add) = 7\\ +h_0(sum) = 9 & h_1(sum) = 2 & h_2(sum) = 1\\ +h_0(equal) = 5 & h_1(equal) = 8 & h_2(equal) = 0. +\end{matrix}

    +

    After appending these words, the Bloom filter is v=(1110110111).\bf{v} = \begin{pmatrix}1&1&1&0&1&1&0&1&1&1\end{pmatrix}.

    +

    Now, suppose that we query the words subtractsubtract and multiplemultiple so that

    +

    h0(subtract)=3h1(subtract)=5h2(subtract)=1h0(multiple)=7h1(multiple)=1h2(multiple)=4\begin{matrix} h_0(subtract) = 3 & h_1(subtract) = 5 & h_2(subtract) = 1\\ h_0(multiple) = 7 & h_1(multiple) = 1 & h_2(multiple) = 4\\ +\end{matrix}.

    +

    The query for subtractsubtract returns 0 since v[3]=0{\bf{v}}[3]=0. +On the other hand, the query for multiplemultiple returns 1 since v[1]=1,v[4]=1{\bf{v}}[1]=1, {\bf{v}}[4] = 1, and v[7]=1{\bf{v}}[7]=1. +Even though multiplemultiple was not used to generate the Bloom filter v{\bf{v}}, our query returns the false positive.

    +

    Probability of false positives​

    +

    For our analysis, we will assume that the probabilities that arise in our analysis are independent. +However, this assumption can be removed to gain the same approximation.

    +

    We note that for a single hash function, the probability that a specific bit is flipped to 1 is 1/n1/n. +So, the probability that the specific bit is not flipped by the hash function is 1−1/n1-1/n. +Applying our assumption that the kk hash functions are 'independent,' the probability that the specific bit is not flipped by any of the hash functions is -(1−1/n)k(1-1/n)^k.

    Recall the calculus fact lim⁡∞(1−1/n)n=e−1\lim_{\infty} (1-1/n)^n = e^{-1}. -That is, as we increase the number of bits that our Bloom filter uses, the approximate probability that a given bit is not flipped by any of the kk hash functions is e−k/ne^{-k/n}.

    Suppose that ℓ\ell entries have been added to the Bloom filter. -The probability that a specific bit is still 0 after the ℓ\ell entries have been added is approximately e−ℓk/ne^{-\ell k/n}. +(1−1/n)k(1-1/n)^k.

    +

    Recall the calculus fact lim⁡∞(1−1/n)n=e−1\lim_{\infty} (1-1/n)^n = e^{-1}. +That is, as we increase the number of bits that our Bloom filter uses, the approximate probability that a given bit is not flipped by any of the kk hash functions is e−k/ne^{-k/n}.

    +

    Suppose that ℓ\ell entries have been added to the Bloom filter. +The probability that a specific bit is still 0 after the ℓ\ell entries have been added is approximately e−ℓk/ne^{-\ell k/n}. The probability that a queried element is erroneously claimed as a member of the digest is approximately -(1−e−ℓk/n)k(1-e^{-\ell k/n})^k.

    The following table provides concrete values for these approximations.

    nnkkℓ\ell(1−e−ℓk/n)k(1-e^{-\ell k/n})^k
    32330.01474
    32370.11143
    323120.30802
    323170.50595
    323280.79804

    Notice that the probability of false positives increases as the number of elements (ℓ\ell) that have been added to the digest increases.

    Sliding-Window Bloom filter​

    Our toy example and table illustrated an issue concerning Bloom filters. -The number of entries that can be added to a Bloom filter is restricted by our choice of kk and nn. +(1−e−ℓk/n)k(1-e^{-\ell k/n})^k.

    +

    The following table provides concrete values for these approximations.

    +
    nnkkℓ\ell(1−e−ℓk/n)k(1-e^{-\ell k/n})^k
    32330.01474
    32370.11143
    323120.30802
    323170.50595
    323280.79804
    +

    Notice that the probability of false positives increases as the number of elements (ℓ\ell) that have been added to the digest increases.

    +

    Sliding-Window Bloom filter​

    +

    Our toy example and table illustrated an issue concerning Bloom filters. +The number of entries that can be added to a Bloom filter is restricted by our choice of kk and nn. Not only does the probability that false positives will occur increase, -but it is possible that our vector v{\bf{v}} can be a string of all 1s. -Szepieniec and Værge proposed a modification to Bloom filters to handle this.

    Instead of having a fixed number of bits for our Bloom filter, we dynamically allot memory based on the number of entries that have been added to the filter. -Given a predetermined threshold (bb) for the number of entries, we shift our 'window' of flipping bits by ss bits. +but it is possible that our vector v{\bf{v}} can be a string of all 1s. +Szepieniec and Værge proposed a modification to Bloom filters to handle this.

    +

    Instead of having a fixed number of bits for our Bloom filter, we dynamically allot memory based on the number of entries that have been added to the filter. +Given a predetermined threshold (bb) for the number of entries, we shift our 'window' of flipping bits by ss bits. Note that this means that it is necessary to keep track of when a given entry is added to the digest. -This means that querying the Sliding-Window Bloom filter will yield different results when different timestamps are used.

    This can be done with kk hash functions as we used earlier. -Alternatively, Szepieniec and Værge proposed using the same hash function but to produce kk entries in the current window. -Specifically, we obtain the bits we wish to flip to 1s by computing h(X∣∣i)h(X || i) for each i∈{0,…,k−1}i \in \{0,\dots, k-1\} and XX as we will define next. -For Sliding-Window Bloom filters, XX is more than just the element we wish to append to the filter. -Instead, XX consists of the element xx and a timestamp tt. -The timestamp tt is used to locate the correct window for bits, as we see below:

    • Append: Suppose that we wish to add the element xx with timestamp tt to the Sliding-Window Bloom filter.
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=h(x∣∣t∣∣i){\bf{b}}[i] := h(x||t||i) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
      • Update the binary string v[b[i]+⌊t/b⌋s]←1{\bf{v}}[{\bf{b}}[i]+\lfloor t/b \rfloor s] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
    • Query: Suppose that we wish to query the Bloom filter for element yy with timestamp tt.
      • Return 1 provided v[h(y∣∣t∣∣i)+⌊t/b⌋s]=1{\bf{v}}[h(y||t||i) + \lfloor t/b \rfloor s] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.

    By incorporating a shifting window, we maintain efficient querying and appending at the cost of constant space. -However, by losing constant space, we gain 'infinite' scalability.

    Cuckoo filters​

    A Cuckoo filter is a data structure for probabilistic membership proofs based on Cuckoo hash tables. +This means that querying the Sliding-Window Bloom filter will yield different results when different timestamps are used.

    +

    This can be done with kk hash functions as we used earlier. +Alternatively, Szepieniec and Værge proposed using the same hash function but to produce kk entries in the current window. +Specifically, we obtain the bits we wish to flip to 1s by computing h(X∣∣i)h(X || i) for each i∈{0,…,k−1}i \in \{0,\dots, k-1\} and XX as we will define next. +For Sliding-Window Bloom filters, XX is more than just the element we wish to append to the filter. +Instead, XX consists of the element xx and a timestamp tt. +The timestamp tt is used to locate the correct window for bits, as we see below:

    +
      +
    • +

      Append: Suppose that we wish to add the element xx with timestamp tt to the Sliding-Window Bloom filter.

      +
        +
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=h(x∣∣t∣∣i){\bf{b}}[i] := h(x||t||i) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      • Update the binary string v[b[i]+⌊t/b⌋s]←1{\bf{v}}[{\bf{b}}[i]+\lfloor t/b \rfloor s] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Bloom filter for element yy with timestamp tt.

      +
        +
      • Return 1 provided v[h(y∣∣t∣∣i)+⌊t/b⌋s]=1{\bf{v}}[h(y||t||i) + \lfloor t/b \rfloor s] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.
        • +
      +
      • +
    +

    By incorporating a shifting window, we maintain efficient querying and appending at the cost of constant space. +However, by losing constant space, we gain 'infinite' scalability.

    +

    Cuckoo filters​

    +

    A Cuckoo filter is a data structure for probabilistic membership proofs based on Cuckoo hash tables. The specific design goal for Cuckoo filters is to address the inability to remove elements from a Bloom Filter. This is done by replacing a list of bits with a list of 'fingerprints.' A fingerprint can be thought of as the hash value for an entry in the digest. A Cuckoo filter is a fixed-length list of 'fingerprints.' -If the maximum number of entries that a Cuckoo filter can hold is nn and a fingerprint occupies ff bits, -then the Cuckoo filter occupies nfnf bits.

    Now, we describe the algorithms associated with the Cuckoo filter CC with hash function hash(X)hash(X) and fingerprint function fingerprint(X)fingerprint(X).

    • Append: Suppose that we wish to add the element xx to the Cuckoo filter.

      • If either position ix:=hash(x)i_x := hash(x) or jx:=i⊗hash(fingerprint(x))j_x := i \otimes hash(fingerprint(x)) of CC is empty, -then fingerprint(x)fingerprint(x) is inserted into an empty position.
      • If both ixi_x and jxj_x are occupied with a fingerprint that is distinct from fingerprint(x)fingerprint(x), -then we select either ixi_x or jxj_x to insert fingerprint(x)fingerprint(x). +If the maximum number of entries that a Cuckoo filter can hold is nn and a fingerprint occupies ff bits, +then the Cuckoo filter occupies nfnf bits.

        +

        Now, we describe the algorithms associated with the Cuckoo filter CC with hash function hash(X)hash(X) and fingerprint function fingerprint(X)fingerprint(X).

        +
          +
        • +

          Append: Suppose that we wish to add the element xx to the Cuckoo filter.

          +
            +
          • If either position ix:=hash(x)i_x := hash(x) or jx:=i⊗hash(fingerprint(x))j_x := i \otimes hash(fingerprint(x)) of CC is empty, +then fingerprint(x)fingerprint(x) is inserted into an empty position.
            • +
          • If both ixi_x and jxj_x are occupied with a fingerprint that is distinct from fingerprint(x)fingerprint(x), +then we select either ixi_x or jxj_x to insert fingerprint(x)fingerprint(x). The fingerprint that had previously occupied this position cannot be discarried. Instead, we insert this fingerprint into its alternate location. This reshuffling process either ends with fingerprints all having their own bucket or one that cannot be inserted. -In the case that we have a fingerprint that cannot be inserted, then the Cuckoo filter is overfilled.

        • Query: Suppose that we wish to query the Cuckoo filter for element yy.

          • Return 1 provided fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y.

        • Delete: Suppose that we wish to delete the element yy from the Cuckoo filter.

          • If yy has been added to the Cuckoo filter, then fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y. -We remove fingerprint(y)fingerprint(y) from the appropriate position.

        We note that false positives in Cuckoo filters only occur when an element shares a fingerprint and hash with a value that has already been added to the Cuckoo filter.

        Example​

        In this example, we will append the words addadd, sumsum, and equalequal to a Cuckoo filter with 8 slots.

        For each word xx, we compute two indices: -ix:=hash(x) and jx:=hash(x)⊗hash(fingerprint(x)).i_x := hash(x) \text{ and } j_x := hash(x) \otimes hash(fingerprint(x)). +In the case that we have a fingerprint that cannot be inserted, then the Cuckoo filter is overfilled.

        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Cuckoo filter for element yy.

      +
        +
      • Return 1 provided fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y.
        • +
      +
      • +
    • +

      Delete: Suppose that we wish to delete the element yy from the Cuckoo filter.

      +
        +
      • If yy has been added to the Cuckoo filter, then fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y. +We remove fingerprint(y)fingerprint(y) from the appropriate position.
        • +
      +
      • +
    +

    We note that false positives in Cuckoo filters only occur when an element shares a fingerprint and hash with a value that has already been added to the Cuckoo filter.

    +

    Example​

    +

    In this example, we will append the words addadd, sumsum, and equalequal to a Cuckoo filter with 8 slots.

    +

    For each word xx, we compute two indices: +ix:=hash(x) and jx:=hash(x)⊗hash(fingerprint(x)).i_x := hash(x) \text{ and } j_x := hash(x) \otimes hash(fingerprint(x)). Suppose that we have the following values for -our words:

    wordixi_xjxj_x
    addadd(0,1,0)(0,1,0)(1,0,0)(1,0,0)
    sumsum(1,0,1)(1,0,1)(1,1,0)(1,1,0)
    equalequal(0,1,0)(0,1,0)(1,0,1)(1,0,1)

    For clarity of the example, we append the words directly to the buckets instead of fingerprints of our data.

    01234567
    append addaddaddadd
    append sumsumaddaddsumsum

    Notice that both of the buckets (2 and 5) that equalequal can map to are occupied. -So, we select one of these buckets (say 2) to insert equalequal into. -Then, we have to insert addadd to its possible bucket (1). -This leaves us with the Cuckoo filter:

    01234567
    addaddequalequalsumsum

    Complexity​

    Notice that deletions and queries to Cuckoo filters are done in constant time. -Specifically, only two locations need to be checked for any data xx. +our words:

    +
    wordixi_xjxj_x
    addadd(0,1,0)(0,1,0)(1,0,0)(1,0,0)
    sumsum(1,0,1)(1,0,1)(1,1,0)(1,1,0)
    equalequal(0,1,0)(0,1,0)(1,0,1)(1,0,1)
    +

    For clarity of the example, we append the words directly to the buckets instead of fingerprints of our data.

    +
    01234567
    append addaddaddadd
    append sumsumaddaddsumsum
    +

    Notice that both of the buckets (2 and 5) that equalequal can map to are occupied. +So, we select one of these buckets (say 2) to insert equalequal into. +Then, we have to insert addadd to its possible bucket (1). +This leaves us with the Cuckoo filter:

    +
    01234567
    addaddequalequalsumsum
    +

    Complexity​

    +

    Notice that deletions and queries to Cuckoo filters are done in constant time. +Specifically, only two locations need to be checked for any data xx. Appends may require shuffling previously added elements to their alternate locations. -As such, the append does not run in constant time.

    Bloom filters vs Cuckoo filters​

    The design of Bloom filters is focused on space efficiency and quick query time. +As such, the append does not run in constant time.

    +

    Bloom filters vs Cuckoo filters​

    +

    The design of Bloom filters is focused on space efficiency and quick query time. Even though they occupy constant space, -Cuckoo filters require significantly more space for nn items than Bloom filters. +Cuckoo filters require significantly more space for nn items than Bloom filters. The worst-case append in a Cuckoo filter is slower than the append in a Bloom filter. However, an append that does not require any shuffling in a Cuckoo filter can be quicker than appends in Bloom filters. Cuckoo filters make up for these disadvantages with quicker query time and the ability to delete entries. -Further, the probability of false positives in Cuckoo filters is lower than the probability of false positives in Bloom filters.

    Combining Filters with RLN​

    In a series of posts (1,2,3), +Further, the probability of false positives in Cuckoo filters is lower than the probability of false positives in Bloom filters.

    +

    Combining Filters with RLN​

    +

    In a series of posts (1,2,3), various versons of rate limiting nullifiers (RLN) that are used by Waku has been discussed. RLN uses a sparse Merkle tree for the membership set. The computational power required to construct the Merkle tree prevent light clients from participating in verifying membership proofs. @@ -1505,17 +3416,31 @@ it was proposed to move the membership set on-chain so that it would not be nece Unfortunately, the naive approach is not practical as the gas limit for a single call is too restrictive for an appropriately sized tree. Instead, it was proposed to make utilize of subtrees. In this section, we provide a discussion of an alternate solution for light clients by using filters for the membership set. -The two parts of RLN that we will focus on are user registration and deletion.

    Both Bloom and Cuckoo filters support user registration as this is can be done as an append. +The two parts of RLN that we will focus on are user registration and deletion.

    +

    Both Bloom and Cuckoo filters support user registration as this is can be done as an append. The fixed size of these filters would restrict the total number of users that can register. This can be migitated by using Sliding-Window Bloom filter as this supports system growth. The Sliding-Window can be adapted to Cuckoo filters as well. In the case of a Sliding-Window filter, an user would maintain the epoch of when they registered. The registration of new users to Bloom filters can be done in constant time which is a significant improvement over appending to subtrees. -Unfortunately, the complexity of registration to Cuckoo filters cannot be as easily computed.

    A user could be slashed from the RLN by sending too many messages in a given epoch. +Unfortunately, the complexity of registration to Cuckoo filters cannot be as easily computed.

    +

    A user could be slashed from the RLN by sending too many messages in a given epoch. Unfortunately, Bloom filters do not support the deletion of members. -Luckily, Cuckoo filters allow for deletions that can performed in constant time.

    Cuckoo filter that use Sliding-Window could be used so that light clients are able to verify proofs of membership in the RLN. +Luckily, Cuckoo filters allow for deletions that can performed in constant time.

    +

    Cuckoo filter that use Sliding-Window could be used so that light clients are able to verify proofs of membership in the RLN. These proofs are not a substitute to the usual proofs that a heavy client can verify due to the allowance of false positives. -However, with the allowance of false positives, a light client can participate in verification RLN proofs in an efficient manner.

    References​

    ]]> +However, with the allowance of false positives, a light client can participate in verification RLN proofs in an efficient manner.

    +

    References​

    +]]>     <![CDATA[RLN-v3: Towards a Flexible and Cost-Efficient Implementation]]> @@ -1523,54 +3448,171 @@ However, with the allowance of false positives, a light client can participate i https://vac.dev/rlog/rln-v3 Mon, 13 May 2024 12:00:00 GMT - Improving on the previous version of RLN by allowing dynamic epoch sizes.

    Introduction​

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    The premise of RLN-v3 is to have a variable message rate per variable epoch, -which can be explained in the following way:

    • RLN-v1: “Alice can send 1 message per global epoch”

      Practically, this is 1 msg/second

    • RLN-v2: “Alice can send x messages per global epoch”

      Practically, this is x msg/second

    • RLN-v3: “Alice can send x messages within a time interval y chosen by herself. + Improving on the previous version of RLN by allowing dynamic epoch sizes.

      + +

      Introduction​

      +

      Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

      +

      The premise of RLN-v3 is to have a variable message rate per variable epoch, +which can be explained in the following way:

      +
        +
      • +

        RLN-v1: “Alice can send 1 message per global epoch”

        +

        Practically, this is 1 msg/second

        +
        • +
      • +

        RLN-v2: “Alice can send x messages per global epoch”

        +

        Practically, this is x msg/second

        +
        • +
      • +

        RLN-v3: “Alice can send x messages within a time interval y chosen by herself. The funds she has to pay are affected by both the number of messages and the chosen time interval. -Other participants can choose different time intervals fitting their specific needs.

        Practically, this is x msg/y seconds

      RLN-v3 allows higher flexibility and ease of payment/stake for users who have more predictable usage patterns and therefore, -more predictable bandwidth usage on a p2p network (Waku, etc.).

      For example:

      • An AMM that broadcasts bids, asks, and fills over Waku may require a lot of throughput in the smallest epoch possible and hence may register an RLN-v3 membership of 10000 msg/1 second. -They could do this with RLN-v2, too.
      • Alice, a casual user of a messaging app built on Waku, who messages maybe 3-4 people infrequently during the day, may register an RLN-v3 membership of 100 msg/hour, +Other participants can choose different time intervals fitting their specific needs.

        +

        Practically, this is x msg/y seconds

        +
        • +
      +

      RLN-v3 allows higher flexibility and ease of payment/stake for users who have more predictable usage patterns and therefore, +more predictable bandwidth usage on a p2p network (Waku, etc.).

      +

      For example:

      +
        +
      • An AMM that broadcasts bids, asks, and fills over Waku may require a lot of throughput in the smallest epoch possible and hence may register an RLN-v3 membership of 10000 msg/1 second. +They could do this with RLN-v2, too.
        • +
      • Alice, a casual user of a messaging app built on Waku, who messages maybe 3-4 people infrequently during the day, may register an RLN-v3 membership of 100 msg/hour, which would not be possible in RLN-v2 considering the global epoch was set to 1 second. With RLN-v2, Alice would have to register with a membership of 1 msg/sec, which would translate to 3600 msg/hour. This is much higher than her usage and would -result in her overpaying to stake into the membership set.
      • A sync service built over Waku, +result in her overpaying to stake into the membership set.
        • +
      • A sync service built over Waku, whose spec defines that it MUST broadcast a set of public keys every hour, may register an RLN-v3 membership of 1 msg/hour, -cutting down the costs to enter the membership set earlier.

      Theory​

      Modification to leaves set in the membership Merkle tree​

      To ensure that a user’s epoch size (user_epoch_limit) is included within their membership we must modify the user’s commitment/leaf in the tree to contain it. +cutting down the costs to enter the membership set earlier.

      • +
    +

    Theory​

    +

    Modification to leaves set in the membership Merkle tree​

    +

    To ensure that a user’s epoch size (user_epoch_limit) is included within their membership we must modify the user’s commitment/leaf in the tree to contain it. A user’s commitment/leaf in the tree is referred to as a rate_commitment, which was previously derived from their public key (identity_commitment) -and their variable message rate (user_message_limit).

    In RLN-v2:

    rate_commitment=poseidon([identity_commitment,user_message_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit])

    In RLN-v3:

    rate_commitment=poseidon([identity_commitment,user_message_limit,user_epoch_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit, user\_epoch\_limit])

    Modification to circuit inputs​

    To detect double signaling, +and their variable message rate (user_message_limit).

    +

    In RLN-v2:

    +rate_commitment=poseidon([identity_commitment,user_message_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit]) +

    In RLN-v3:

    +rate_commitment=poseidon([identity_commitment,user_message_limit,user_epoch_limit])rate\_commitment = poseidon([identity\_commitment, user\_message\_limit, user\_epoch\_limit]) +

    Modification to circuit inputs​

    +

    To detect double signaling, we make use of a circuit output nullifier, which remains the same if a user generates a proof with the same message_id and external_nullifier, -where the external_nullifier and nullifier are defined as:

    external_nullifier=poseidon([epoch,rln_identifier])nullifier=poseidon([identity_secret,external_nullifier,message_id])external\_nullifier = poseidon([epoch, rln\_identifier]) \\ nullifier = poseidon([identity\_secret, external\_nullifier, message\_id])

    Where:

    • epoch is defined as the Unix epoch timestamp with seconds precision.
    • rln_identifier uniquely identifies an application for which a user submits a proof.
    • identity_secret is the private key of the user.
    • message_id is the sequence number of the user’s message within user_message_limit in an epoch.

    In RLN-v2, the global epoch was 1 second, +where the external_nullifier and nullifier are defined as:

    +external_nullifier=poseidon([epoch,rln_identifier])nullifier=poseidon([identity_secret,external_nullifier,message_id])external\_nullifier = poseidon([epoch, rln\_identifier]) \\ +nullifier = poseidon([identity\_secret, external\_nullifier, message\_id]) +

    Where:

    +
      +
    • epoch is defined as the Unix epoch timestamp with seconds precision.
      • +
    • rln_identifier uniquely identifies an application for which a user submits a proof.
      • +
    • identity_secret is the private key of the user.
      • +
    • message_id is the sequence number of the user’s message within user_message_limit in an epoch.
      • +
    +

    In RLN-v2, the global epoch was 1 second, hence we did not need to perform any assertions to the epoch’s value inside the circuit, -and the validation of the epoch was handled off-circuit (i.e., too old, too large, bad values, etc.).

    In RLN-v3, we propose that the epoch that is passed into the circuit +and the validation of the epoch was handled off-circuit (i.e., too old, too large, bad values, etc.).

    +

    In RLN-v3, we propose that the epoch that is passed into the circuit must be a valid multiple of user_epoch_limit -since the user may pass in values of the epoch which do not directly correlate with the user_epoch_limit.

    For example:

    • A user with user_epoch_limit of 120 +since the user may pass in values of the epoch which do not directly correlate with the user_epoch_limit.

      +

      For example:

      +
        +
      • A user with user_epoch_limit of 120 passes in an epoch of 237 generates user_message_limit proofs with it, can increment the epoch by 1, and generate user_message_limit proofs with it, -thereby allowing them to bypass the message per epoch restriction.

      One could say that we could perform this validation outside of the circuit, +thereby allowing them to bypass the message per epoch restriction.

      • +
    +

    One could say that we could perform this validation outside of the circuit, but we maintain the user_epoch_limit as a private input to the circuit so that the user is not deanonymized by the anonymity set connected to that user_epoch_limit. Since user_epoch_limit is kept private, -the verifier does not have access to that value and cannot perform validation on it.

    If we ensure that the epoch is a multiple of user_epoch_limit, -we have the following scenarios:

    • A user with user_epoch_limit of 120 +the verifier does not have access to that value and cannot perform validation on it.

      +

      If we ensure that the epoch is a multiple of user_epoch_limit, +we have the following scenarios:

      +
        +
      • A user with user_epoch_limit of 120 passes in an epoch of 237. -Proof generation fails since the epoch is not a multiple of user_epoch_limit.
      • A user with user_epoch_limit of 120 +Proof generation fails since the epoch is not a multiple of user_epoch_limit.
        • +
      • A user with user_epoch_limit of 120 passes in an epoch of 240 and -can generate user_message_limit proofs without being slashed.

      Since we perform operations on the epoch, we must include it as a circuit input (previously, it was removed from the circuit inputs to RLN-v2).

      Therefore, the new circuit inputs are as follows:

      // unchanged
      private identity_secret
      private user_message_limit
      private message_id
      private pathElements[]
      private pathIndices[]
      public x // messageHash

      // new/changed
      private user_epoch_limit
      private user_epoch_quotient // epoch/user_epoch_limit to assert within circuit
      public epoch
      public rln_identifier

      The circuit outputs remain the same.

      Additional circuit constraints​

      1. Since we accept the epoch, user_epoch_quotient, and user_epoch_limit, -we must ensure that the relation between these 3 values is preserved. I.e.:

        epoch==user_epoch_limit∗user_epoch_quotientepoch == user\_epoch\_limit * user\_epoch\_quotient

      2. To ensure no overflows/underflows occur in the above multiplication, +can generate user_message_limit proofs without being slashed.

        3. +
    +

    Since we perform operations on the epoch, we must include it as a circuit input (previously, it was removed from the circuit inputs to RLN-v2).

    +

    Therefore, the new circuit inputs are as follows:

    +
    // unchanged
    private identity_secret
    private user_message_limit
    private message_id
    private pathElements[]
    private pathIndices[]
    public x // messageHash

    // new/changed
    private user_epoch_limit
    private user_epoch_quotient // epoch/user_epoch_limit to assert within circuit
    public epoch
    public rln_identifier
    +

    The circuit outputs remain the same.

    +

    Additional circuit constraints​

    +
      +
    1. +

      Since we accept the epoch, user_epoch_quotient, and user_epoch_limit, +we must ensure that the relation between these 3 values is preserved. I.e.:

      +epoch==user_epoch_limit∗user_epoch_quotientepoch == user\_epoch\_limit * user\_epoch\_quotient +
      2. +
    2. +

      To ensure no overflows/underflows occur in the above multiplication, we must constrain the inputs of epoch, user_epoch_quotient, and user_epoch_limit. -We have assumed 3600 to be the maximum valid size of the user_epoch_quotient.

    size(epoch)≤64 bitssize(user_epoch_limit)≤12 bitsuser_epoch_limit≤3600user_epoch_limit≤epochuser_epoch_quotient<user_epoch_limitsize(epoch) \leq 64\ bits \\ size(user\_epoch\_limit) \leq 12\ bits \\ user\_epoch\_limit \leq 3600 \\ user\_epoch\_limit \leq epoch \\ user\_epoch\_quotient < user\_epoch\_limit

    Modifications to external epoch validation (Waku, etc.)​

    For receivers of an RLN-v3 proof +We have assumed 3600 to be the maximum valid size of the user_epoch_quotient.

    + + +size(epoch)≤64 bitssize(user_epoch_limit)≤12 bitsuser_epoch_limit≤3600user_epoch_limit≤epochuser_epoch_quotient<user_epoch_limitsize(epoch) \leq 64\ bits \\ +size(user\_epoch\_limit) \leq 12\ bits \\ +user\_epoch\_limit \leq 3600 \\ +user\_epoch\_limit \leq epoch \\ +user\_epoch\_quotient < user\_epoch\_limit +

    Modifications to external epoch validation (Waku, etc.)​

    +

    For receivers of an RLN-v3 proof to detect if a message is too old, we must use the higher bound of the user_epoch_limit, which has been set to 3600. -The trade-off here is that we allow hour-old messages to propagate within the network.

    Modifications to double signaling detection scheme (Waku, etc.)​

    For verifiers of RLN-v1/v2 proofs, +The trade-off here is that we allow hour-old messages to propagate within the network.

    +

    Modifications to double signaling detection scheme (Waku, etc.)​

    +

    For verifiers of RLN-v1/v2 proofs, a log of nullifiers seen in the last epoch is maintained, and if there is a match with a pre-existing nullifier, -double signaling has been detected and the verifier MAY proceed to slash the spamming user.

    With the RLN-v3 scheme, +double signaling has been detected and the verifier MAY proceed to slash the spamming user.

    +

    With the RLN-v3 scheme, we need to increase the size of the nullifier log used, which previously cleared itself every second to the higher bound of the user_epoch_limit, which is 3600. -Now, the RLN proof verifier must clear the nullifier log every 3600 seconds to satisfactorily detect double signaling.

    The implementation​

    An implementation of the RLN-v3 scheme in gnark can be found here.

    Comments on performance​

    • Hardware: Macbook Air M2, 16GB RAM
    • Circuit: RLN-v3
    • Proving system: Groth16
    • Framework: gnark
    • Elliptic curve: bn254 (aka bn128) (not to be confused with the 254-bit Weierstrass curve)
    • Finite field: Prime-order subgroup of the group of points on the bn254 curve
    • Default Merkle tree height: 20
    • Hashing algorithm: Poseidon
    • Merkle tree: Sparse Indexed Merkle Tree

    Proving​

    The proving time for the RLN-v3 circuit is 90ms for a single proof.

    Verification​

    The verification time for the RLN-v3 circuit is 1.7ms for a single proof.

    Conclusion​

    The RLN-v3 scheme introduces a new epoch-based message rate-limiting scheme to the RLN protocol. -It enhances the user's flexibility in setting their message limits and cost-optimizes their stake.

    Future work​

    • Implementing the RLN-v3 scheme in Zerokit
    • Implementing the RLN-v3 scheme in Waku
    • Formal security analysis of the RLN-v3 scheme

    References​

    ]]>     +Now, the RLN proof verifier must clear the nullifier log every 3600 seconds to satisfactorily detect double signaling.

    +

    The implementation​

    +

    An implementation of the RLN-v3 scheme in gnark can be found here.

    +

    Comments on performance​

    +
      +
    • Hardware: Macbook Air M2, 16GB RAM
      • +
    • Circuit: RLN-v3
      • +
    • Proving system: Groth16
      • +
    • Framework: gnark
      • +
    • Elliptic curve: bn254 (aka bn128) (not to be confused with the 254-bit Weierstrass curve)
      • +
    • Finite field: Prime-order subgroup of the group of points on the bn254 curve
      • +
    • Default Merkle tree height: 20
      • +
    • Hashing algorithm: Poseidon
      • +
    • Merkle tree: Sparse Indexed Merkle Tree
      • +
    +

    Proving​

    +

    The proving time for the RLN-v3 circuit is 90ms for a single proof.

    +

    Verification​

    +

    The verification time for the RLN-v3 circuit is 1.7ms for a single proof.

    +

    Conclusion​

    +

    The RLN-v3 scheme introduces a new epoch-based message rate-limiting scheme to the RLN protocol. +It enhances the user's flexibility in setting their message limits and cost-optimizes their stake.

    +

    Future work​

    +
      +
    • Implementing the RLN-v3 scheme in Zerokit
      • +
    • Implementing the RLN-v3 scheme in Waku
      • +
    • Formal security analysis of the RLN-v3 scheme
      • +
    +

    References​

    +]]>     <![CDATA[Verifying RLN Proofs in Light Clients with Subtrees]]> @@ -1578,39 +3620,115 @@ It enhances the user's flexibility in setting their message limits and cost-opti https://vac.dev/rlog/rln-light-verifiers Fri, 03 May 2024 12:00:00 GMT - How resource-restricted devices can verify RLN proofs fast and efficiently.

    Introduction​

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    This post expands upon ideas described in the previous post, -focusing on how resource-restricted devices can verify RLN proofs fast and efficiently.

    Previously, it was required to fetch all the memberships from the smart contract, + How resource-restricted devices can verify RLN proofs fast and efficiently.

    + +

    Introduction​

    +

    Recommended previous reading: Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku.

    +

    This post expands upon ideas described in the previous post, +focusing on how resource-restricted devices can verify RLN proofs fast and efficiently.

    +

    Previously, it was required to fetch all the memberships from the smart contract, construct the merkle tree locally, and derive the merkle root, -which is subsequently used to verify RLN proofs.

    This process is not feasible for resource-restricted devices since it involves a lot of RPC calls, computation and fault tolerance. -One cannot expect a mobile phone to fetch all the memberships from the smart contract and construct the merkle tree locally.

    Constraints and requirements​

    An alternative solution to the one proposed in this post is to construct the merkle tree on-chain, +which is subsequently used to verify RLN proofs.

    +

    This process is not feasible for resource-restricted devices since it involves a lot of RPC calls, computation and fault tolerance. +One cannot expect a mobile phone to fetch all the memberships from the smart contract and construct the merkle tree locally.

    +

    Constraints and requirements​

    +

    An alternative solution to the one proposed in this post is to construct the merkle tree on-chain, and have the root accessible with a single RPC call. -However, this approach increases gas costs for inserting new memberships and may not be feasible until it is optimized further with batching mechanisms, etc.

    The other methods have been explored in more depth here.

    Following are the requirements and constraints for the solution proposed in this post:

    1. Cheap membership insertions.
    2. As few RPC calls as possible to reduce startup time.
    3. Merkle root of the tree is available on-chain.
    4. No centralized services to sequence membership insertions.
    5. Map inserted commitments to the block in which they were inserted.

    Metrics on sync time for a tree with 2,653 leaves​

    The following metrics are based on the current implementation of RLN in the Waku gen0 network.

    Test bench​

    • Hardware: Macbook Air M2, 16GB RAM
    • Network: 120 Megabits/sec
    • Nwaku commit: e61e4ff
    • RLN membership set contract: 0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4
    • Deployed block number: 4,230,716
    • RLN Membership set depth: 20
    • Hash function: PoseidonT3 (which is a gas guzzler)
    • Max size of the membership set: 2^20 = 1,048,576 leaves

    Metrics​

    • Time to sync the whole tree: 4 minutes
    • RPC calls: 702
    • Number of leaves: 2,653

    One can argue that the time to sync the tree at the current state is not that bad. +However, this approach increases gas costs for inserting new memberships and may not be feasible until it is optimized further with batching mechanisms, etc.

    +

    The other methods have been explored in more depth here.

    +

    Following are the requirements and constraints for the solution proposed in this post:

    +
      +
    1. Cheap membership insertions.
      2. +
    2. As few RPC calls as possible to reduce startup time.
      3. +
    3. Merkle root of the tree is available on-chain.
      4. +
    4. No centralized services to sequence membership insertions.
      5. +
    5. Map inserted commitments to the block in which they were inserted.
      6. +
    +

    Metrics on sync time for a tree with 2,653 leaves​

    +

    The following metrics are based on the current implementation of RLN in the Waku gen0 network.

    +

    Test bench​

    +
      +
    • Hardware: Macbook Air M2, 16GB RAM
      • +
    • Network: 120 Megabits/sec
      • +
    • Nwaku commit: e61e4ff
      • +
    • RLN membership set contract: 0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4
      • +
    • Deployed block number: 4,230,716
      • +
    • RLN Membership set depth: 20
      • +
    • Hash function: PoseidonT3 (which is a gas guzzler)
      • +
    • Max size of the membership set: 2^20 = 1,048,576 leaves
      • +
    +

    Metrics​

    +
      +
    • Time to sync the whole tree: 4 minutes
      • +
    • RPC calls: 702
      • +
    • Number of leaves: 2,653
      • +
    +

    One can argue that the time to sync the tree at the current state is not that bad. However, the number of RPC calls is a concern, which scales linearly with the number of blocks since the contract was deployed This is because the implementation fetches all events from the contract, chunking 2,000 blocks at a time. This is done to avoid hitting the block limit of 10,000 events per call, -which is a limitation of popular RPC providers.

    Proposed solution​

    From a theoretical perspective, +which is a limitation of popular RPC providers.

    +

    Proposed solution​

    +

    From a theoretical perspective, one could construct the merkle tree on-chain, in a view call, in-memory. -However, this is not feasible due to the gas costs associated with it.

    To compute the root of a Merkle tree with 2202^{20} leaves it costs approximately 2 billion gas. +However, this is not feasible due to the gas costs associated with it.

    +

    To compute the root of a Merkle tree with 2202^{20} leaves it costs approximately 2 billion gas. With Infura and Alchemy capping the gas limit to 350M and 550M gas respectively, -it is not possible to compute the root of the tree in a single call.

    Acknowledging that Polygon Miden and Penumbra both make use of a tiered commitment tree, -we propose a similar approach for RLN.

    A tiered commitment tree is a tree which is sharded into multiple smaller subtrees, +it is not possible to compute the root of the tree in a single call.

    +

    Acknowledging that Polygon Miden and Penumbra both make use of a tiered commitment tree, +we propose a similar approach for RLN.

    +

    A tiered commitment tree is a tree which is sharded into multiple smaller subtrees, each of which is a tree in itself. This allows scaling in terms of the number of leaves, -as well as reducing state bloat by just storing the root of a subtree when it is full instead of all its leaves.

    Here, the question arises: -What is the maximum number of leaves in a subtree with which the root can be computed in a single call?

    It costs approximately 217M gas to compute the root of a Merkle tree with 2102^{10} leaves.

    This is a feasible number for a single call, -and hence we propose a tiered commitment tree with a maximum of 2102^{10} leaves in a subtree and the number of subtrees is 2102^{10}. -Therefore, the maximum number of leaves in the tree is 2202^{20} (the same as the current implementation).

    img

    Insertion​

    When a commitment is inserted into the tree it is first inserted into the first subtree. -When the first subtree is full the next insertions go into the second subtree and so on.

    Syncing​

    When syncing the tree, +as well as reducing state bloat by just storing the root of a subtree when it is full instead of all its leaves.

    +

    Here, the question arises: +What is the maximum number of leaves in a subtree with which the root can be computed in a single call?

    +

    It costs approximately 217M gas to compute the root of a Merkle tree with 2102^{10} leaves.

    +

    This is a feasible number for a single call, +and hence we propose a tiered commitment tree with a maximum of 2102^{10} leaves in a subtree and the number of subtrees is 2102^{10}. +Therefore, the maximum number of leaves in the tree is 2202^{20} (the same as the current implementation).

    +

    img

    +

    Insertion​

    +

    When a commitment is inserted into the tree it is first inserted into the first subtree. +When the first subtree is full the next insertions go into the second subtree and so on.

    +

    Syncing​

    +

    When syncing the tree, one only needs to fetch the roots of the subtrees. -The root of the full tree can be computed in-memory or on-chain.

    This allows us to derive the following relation:

    number_of_rpc_calls=number_of_filled_subtrees+1number\_of\_rpc\_calls = number\_of\_filled\_subtrees + 1

    This is a significant improvement over the current implementation, -which requires fetching all the memberships from the smart contract.

    Gas costs​

    The gas costs for inserting a commitment into the tree are the same as the current implementation except it consists of an extra SSTORE operation to store the shardIndex of the commitment.

    Events​

    The events emitted by the contract are the same as the current implementation, -appending the shardIndex of the commitment.

    Proof of concept​

    A proof of concept implementation of the tiered commitment tree is available here, -and is deployed on Sepolia at 0xE7987c70B54Ff32f0D5CBbAA8c8Fc1cAf632b9A5.

    It is compatible with the current implementation of the RLN verifier.

    Future work​

    1. Optimize the gas costs of the tiered commitment tree.
    2. Explore using different number of leaves under a given node in the tree (currently set to 2).

    Conclusion​

    The tiered commitment tree is a promising approach to reduce the number of RPC calls required to sync the tree and reduce the gas costs associated with computing the root of the tree. -Consequently, it allows for a more scalable and efficient RLN verifier.

    References​

    ]]>     +The root of the full tree can be computed in-memory or on-chain.

    +

    This allows us to derive the following relation:

    +number_of_rpc_calls=number_of_filled_subtrees+1number\_of\_rpc\_calls = number\_of\_filled\_subtrees + 1 +

    This is a significant improvement over the current implementation, +which requires fetching all the memberships from the smart contract.

    +

    Gas costs​

    +

    The gas costs for inserting a commitment into the tree are the same as the current implementation except it consists of an extra SSTORE operation to store the shardIndex of the commitment.

    +

    Events​

    +

    The events emitted by the contract are the same as the current implementation, +appending the shardIndex of the commitment.

    +

    Proof of concept​

    +

    A proof of concept implementation of the tiered commitment tree is available here, +and is deployed on Sepolia at 0xE7987c70B54Ff32f0D5CBbAA8c8Fc1cAf632b9A5.

    +

    It is compatible with the current implementation of the RLN verifier.

    +

    Future work​

    +
      +
    1. Optimize the gas costs of the tiered commitment tree.
      2. +
    2. Explore using different number of leaves under a given node in the tree (currently set to 2).
      3. +
    +

    Conclusion​

    +

    The tiered commitment tree is a promising approach to reduce the number of RPC calls required to sync the tree and reduce the gas costs associated with computing the root of the tree. +Consequently, it allows for a more scalable and efficient RLN verifier.

    +

    References​

    +]]>     <![CDATA[Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku]]> @@ -1618,51 +3736,158 @@ Consequently, it allows for a more scalable and efficient RLN verifier.

    https://vac.dev/rlog/rln-anonymous-dos-prevention Tue, 07 Nov 2023 12:00:00 GMT - Rate Limiting Nullifiers in practice, applied to an anonymous p2p network, like Waku.

    Introduction​

    Rate Limiting Nullifier (RLN) is a zero-knowledge gadget that allows users to prove 2 pieces of information,

    1. They belong to a permissioned membership set
    2. Their rate of signaling abides by a fixed number that has been previously declared

    The "membership set" introduced above, is in the form of a sparse, indexed merkle tree. + Rate Limiting Nullifiers in practice, applied to an anonymous p2p network, like Waku.

    + +

    Introduction​

    +

    Rate Limiting Nullifier (RLN) is a zero-knowledge gadget that allows users to prove 2 pieces of information,

    +
      +
    1. They belong to a permissioned membership set
      2. +
    2. Their rate of signaling abides by a fixed number that has been previously declared
      3. +
    +

    The "membership set" introduced above, is in the form of a sparse, indexed merkle tree. This membership set can be maintained on-chain, off-chain or as a hybrid depending on the network's storage costs. Waku makes use of a hybrid membership set, where insertions are tracked in a smart contract. In addition, each Waku node maintains a local copy of the tree, -which is updated upon each insertion.

    Users register themselves with a hash of a locally generated secret, +which is updated upon each insertion.

    +

    Users register themselves with a hash of a locally generated secret, which is then inserted into the tree at the next available index. After having registered, users can prove their membership by proving their knowledge of the pre-image of the respective leaf in the tree. The leaf hashes are also referred to as commitments of the respective users. -The actual proof is done by a Merkle Inclusion Proof, which is a type of ZK proof.

    The circuit ensures that the user's secret does indeed hash to a leaf in the tree, -and that the provided Merkle proof is valid.

    After a User generates this Merkle proof, +The actual proof is done by a Merkle Inclusion Proof, which is a type of ZK proof.

    +

    The circuit ensures that the user's secret does indeed hash to a leaf in the tree, +and that the provided Merkle proof is valid.

    +

    After a User generates this Merkle proof, they can transmit it to other users, who can verify the proof. Including a message's hash within the proof generation, -additionally guarantees integrity of that message.

    A malicious user could generate multiple proofs per epoch. +additionally guarantees integrity of that message.

    +

    A malicious user could generate multiple proofs per epoch. they generate multiple proofs per epoch. However, when multiple proofs are generated per epoch, the malicious user's secret is exposed, which strongly disincentivizes this attack. -This mechanism is further described in malicious User secret interpolation mechanism

    Note: This blog post describes rln-v1, which excludes the range check in favor of a global rate limit for all users, -which is once per time window. This version is currently in use in waku-rln-relay.

    RLN Protocol parameters​

    Given below is the set of cryptographic primitives, -and constants that are used in the RLN protocol.

    1. Proving System: groth16
    2. Elliptic Curve: bn254 (aka bn128) (not to be confused with the 254 bit Weierstrass curve)
    3. Finite Field: Prime-order subgroup of the group of points on the bn254 curve
    4. Default Merkle Tree Height: 20
    5. Hashing algorithm: Poseidon
    6. Merkle Tree: Sparse Indexed Merkle Tree
    7. Messages per epoch: 1
    8. Epoch duration: 10 seconds

    Malicious User secret interpolation mechanism​

    note: all the parameters mentioned below are elements in the finite field mentioned above.

    The private inputs to the circuit are as follows: -

    identitySecret: the randomly generated secret of the user
    identityPathIndex: the index of the commitment derived from the secret
    pathElements: elements included in the path to the index of the commitment

    Following are the public inputs to the circuit -

    x: hash of the signal to the finite field
    rlnIdentifier: application-specific identifier which this proof is being generated for
    epoch: the timestamp which this proof is being generated for

    The outputs of the circuit are as follows: -

    y: result of Shamir's secret sharing calculation
    root: root of the Merkle tree obtained after applying the inclusion proof
    nullifier: uniquely identifies a message, derived from rlnIdentifier, epoch, and the user's secret

    With the above data in mind, following is the circuit pseudocode -

    identityCommitment = Poseidon([identitySecret])
    root = MerkleInclusionProof(identityCommitment, identityPathIndex, pathElements)
    externalNullifier = Poseidon([epoch, rlnIdentifier])
    a1 = Poseidon([identitySecret, externalNullifier])
    y = identitySecret + a1 * x
    nullifier = Poseidon([a1])

    To interpolate the secret of a user who has sent multiple signals during the same epoch to the same rln-based application, we may make use of the following formula -

    a1=(y1−y2)(x1−x2)a_1 = {(y_1 - y_2) \over (x_1 - x_2)}

    where x1x_1, y1y_1 and x2x_2, y2y_2 are shares from different messages

    subsequently, we may use one pair of the shares, x1x_1 and y1y_1 to obtain the identitySecret

    identitySecret=y1−a1∗xidentitySecret = y_1 - a_1 * x

    This enables RLN to be used for rate limiting with a global limit. For arbitrary limits, -please refer to an article written by @curryrasul, rln-v2.

    Waku's problem with DoS​

    In a decentralized, privacy focused messaging system like Waku, -Denial of Service (DoS) vulnerabilities are very common, and must be addressed to promote network scale and optimal bandwidth utilization.

    DoS prevention with user metadata​

    There are a couple of ways a user can be rate-limited, either -

    1. IP Logging
    2. KYC Logging

    Both IP and KYC logging prevent systems from being truly anonymous, and hence, cannot be used as a valid DoS prevention mechanism for Waku.

    RLN can be used as an alternative, which provides the best of both worlds, i.e a permissioned membership set, as well as anonymous signaling. -However, we are bound by k-anonymity rules of the membership set.

    Waku-RLN-Relay is a libp2p pubsub validator that verifies if a proof attached to a given message is valid. -In case the proof is valid, the message is relayed.

    Performance analysis​

    Test bench specs: AMD EPYC 7502P 32-Core, 4x32GB DDR4 Reg.ECC Memory

    This simulation was conducted by @alrevuelta, and is described in more detail here.

    The simulation included 100 waku nodes running in parallel.

    Proof generation times - -

    img

    Proof verification times - -

    img

    A spammer node publishes 3000 msg/epoch, which is detected by all connected nodes, and subsequently disconnect to prevent further spam - -

    img

    Security analysis​

    Barbulescu and Duquesne +This mechanism is further described in malicious User secret interpolation mechanism

    +

    Note: This blog post describes rln-v1, which excludes the range check in favor of a global rate limit for all users, +which is once per time window. This version is currently in use in waku-rln-relay.

    +

    RLN Protocol parameters​

    +

    Given below is the set of cryptographic primitives, +and constants that are used in the RLN protocol.

    +
      +
    1. Proving System: groth16
      2. +
    2. Elliptic Curve: bn254 (aka bn128) (not to be confused with the 254 bit Weierstrass curve)
      3. +
    3. Finite Field: Prime-order subgroup of the group of points on the bn254 curve
      4. +
    4. Default Merkle Tree Height: 20
      5. +
    5. Hashing algorithm: Poseidon
      6. +
    6. Merkle Tree: Sparse Indexed Merkle Tree
      7. +
    7. Messages per epoch: 1
      8. +
    8. Epoch duration: 10 seconds
      9. +
    +

    Malicious User secret interpolation mechanism​

    +
    +

    note: all the parameters mentioned below are elements in the finite field mentioned above.

    +
    +

    The private inputs to the circuit are as follows: -

    +
    identitySecret: the randomly generated secret of the user
    identityPathIndex: the index of the commitment derived from the secret
    pathElements: elements included in the path to the index of the commitment
    +

    Following are the public inputs to the circuit -

    +
    x: hash of the signal to the finite field
    rlnIdentifier: application-specific identifier which this proof is being generated for
    epoch: the timestamp which this proof is being generated for
    +

    The outputs of the circuit are as follows: -

    +
    y: result of Shamir's secret sharing calculation
    root: root of the Merkle tree obtained after applying the inclusion proof
    nullifier: uniquely identifies a message, derived from rlnIdentifier, epoch, and the user's secret
    +

    With the above data in mind, following is the circuit pseudocode -

    +
    identityCommitment = Poseidon([identitySecret])
    root = MerkleInclusionProof(identityCommitment, identityPathIndex, pathElements)
    externalNullifier = Poseidon([epoch, rlnIdentifier])
    a1 = Poseidon([identitySecret, externalNullifier])
    y = identitySecret + a1 * x
    nullifier = Poseidon([a1])
    +

    To interpolate the secret of a user who has sent multiple signals during the same epoch to the same rln-based application, we may make use of the following formula -

    +

    a1=(y1−y2)(x1−x2)a_1 = {(y_1 - y_2) \over (x_1 - x_2)}

    +

    where x1x_1, y1y_1 and x2x_2, y2y_2 are shares from different messages

    +

    subsequently, we may use one pair of the shares, x1x_1 and y1y_1 to obtain the identitySecret

    +

    identitySecret=y1−a1∗xidentitySecret = y_1 - a_1 * x

    +

    This enables RLN to be used for rate limiting with a global limit. For arbitrary limits, +please refer to an article written by @curryrasul, rln-v2.

    +

    Waku's problem with DoS​

    +

    In a decentralized, privacy focused messaging system like Waku, +Denial of Service (DoS) vulnerabilities are very common, and must be addressed to promote network scale and optimal bandwidth utilization.

    +

    DoS prevention with user metadata​

    +

    There are a couple of ways a user can be rate-limited, either -

    +
      +
    1. IP Logging
      2. +
    2. KYC Logging
      3. +
    +

    Both IP and KYC logging prevent systems from being truly anonymous, and hence, cannot be used as a valid DoS prevention mechanism for Waku.

    +

    RLN can be used as an alternative, which provides the best of both worlds, i.e a permissioned membership set, as well as anonymous signaling. +However, we are bound by k-anonymity rules of the membership set.

    +

    Waku-RLN-Relay is a libp2p pubsub validator that verifies if a proof attached to a given message is valid. +In case the proof is valid, the message is relayed.

    +

    Performance analysis​

    +
    +

    Test bench specs: AMD EPYC 7502P 32-Core, 4x32GB DDR4 Reg.ECC Memory

    +
    +

    This simulation was conducted by @alrevuelta, and is described in more detail here.

    +

    The simulation included 100 waku nodes running in parallel.

    +

    Proof generation times - +

    img

    +

    Proof verification times - +

    img

    +

    A spammer node publishes 3000 msg/epoch, which is detected by all connected nodes, and subsequently disconnect to prevent further spam - +

    img

    +

    Security analysis​

    +

    Barbulescu and Duquesne conclude that that the bn254 curve has only 100 bits of security. Since the bn254 curve has a small embedding degree, it is vulnerable to the MOV attack. However, the MOV attack is only applicable to pairings, and not to the elliptic curve itself. It is acceptable to use the bn254 curve for RLN, -since the circuit does not make use of pairings.

    An analysis on the number of rounds in the Poseidon hash function was done, -which concluded that the hashing rounds should not be reduced,

    The smart contracts have not been audited, and are not recommended for real world deployments yet.

    Storage analysis​

    commitment_size=32 bytestree_height=20total_leaves=220max_tree_size=total_leaves∗commitment_sizemax_tree_size=220∗32=33,554,432∴max_tree_size=33.55 megabytescommitment\_size = 32\ bytes \\ tree\_height =20 \\ total\_leaves = 2^{20} \\ max\_tree\_size = total\_leaves * commitment\_size \\ max\_tree\_size = 2^{20} * 32 = 33,554,432 \\ ∴max\_tree\_size = 33.55\ megabytes

    The storage overhead introduced by RLN is minimal. +since the circuit does not make use of pairings.

    +

    An analysis on the number of rounds in the Poseidon hash function was done, +which concluded that the hashing rounds should not be reduced,

    +

    The smart contracts have not been audited, and are not recommended for real world deployments yet.

    +

    Storage analysis​

    +commitment_size=32 bytestree_height=20total_leaves=220max_tree_size=total_leaves∗commitment_sizemax_tree_size=220∗32=33,554,432∴max_tree_size=33.55 megabytescommitment\_size = 32\ bytes \\ +tree\_height =20 \\ +total\_leaves = 2^{20} \\ +max\_tree\_size = total\_leaves * commitment\_size \\ +max\_tree\_size = 2^{20} * 32 = 33,554,432 \\ +∴max\_tree\_size = 33.55\ megabytes +

    The storage overhead introduced by RLN is minimal. RLN only requires 34 megabytes of storage, which poses no problem on most end-user hardware, with the exception of IoT/microcontrollers. -Still, we are working on further optimizations allowing proof generation without having to store the full tree.

    The bare minimum requirements to run RLN​

    With proof generation time in sub-second latency, along with low storage overhead for the tree, -it is possible for end users to generate and verify RLN proofs on a modern smartphone.

    Following is a demo provided by @rramos that demonstrates -waku-rln-relay used in react native.

    Warning: The react native sdk will be deprecated soon, and the above demo should serve as a PoC for RLN on mobiles

    RLN usage guide​

    Zerokit implements api's that allow users to handle operations to the tree, -as well as generate/verify RLN proofs.

    Our main implementation of RLN can be accessed via this Rust crate, +Still, we are working on further optimizations allowing proof generation without having to store the full tree.

    +

    The bare minimum requirements to run RLN​

    +

    With proof generation time in sub-second latency, along with low storage overhead for the tree, +it is possible for end users to generate and verify RLN proofs on a modern smartphone.

    +

    Following is a demo provided by @rramos that demonstrates +waku-rln-relay used in react native.

    +
    +

    Warning: The react native sdk will be deprecated soon, and the above demo should serve as a PoC for RLN on mobiles

    +
    +

    RLN usage guide​

    +

    Zerokit implements api's that allow users to handle operations to the tree, +as well as generate/verify RLN proofs.

    +

    Our main implementation of RLN can be accessed via this Rust crate, which is documented here. It can used in other langugages via the FFI API, which is documented here. The usage of RLN in Waku is detailed in our RLN Implementers guide, -which provides step-by-step instructions on how to run Waku-RLN-Relay.

    Following is a diagram that will help understand the dependency tree -

    rln-dep-tree

    Future work​

    • Optimizations to zerokit for proof generation time.
    • Incrementing tree depth from 20 to 32, to allow more memberships.
    • Optimizations to the smart contract.
    • An ability to signal validity of a message in different time windows.
    • Usage of proving systems other than Groth16.

    References​

    ]]>     +which provides step-by-step instructions on how to run Waku-RLN-Relay.

    +

    Following is a diagram that will help understand the dependency tree -

    +

    rln-dep-tree

    +

    Future work​

    +
      +
    • Optimizations to zerokit for proof generation time.
      • +
    • Incrementing tree depth from 20 to 32, to allow more memberships.
      • +
    • Optimizations to the smart contract.
      • +
    • An ability to signal validity of a message in different time windows.
      • +
    • Usage of proving systems other than Groth16.
      • +
    +

    References​

    +]]>     <![CDATA[GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks]]> @@ -1670,90 +3895,194 @@ which provides step-by-step instructions on how to run Waku-RLN-Relay.

    Fol https://vac.dev/rlog/GossipSub Improvements Mon, 06 Nov 2023 12:00:00 GMT - GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

    Motivitation​

    We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, + GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks

    + +

    Motivitation​

    +

    We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, as in the case of Ethereum Improvement Proposal EIP-4844. This work led to a comprehensive study of unstructured P2P networks. -The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.

    Introduction​

    Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, -query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh.

    Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. -At the same time, the underlying IP layer is unreliable.

    Therefore, tree-like overlays are not best suited for reliable information propagation. -Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches.

    Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. -Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links.

    An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DD random peers. -Publishing through a D-regular overlay triggers approximately N×DN \times D transmissions. -Reducing DD reduces the redundant transmissions but compromises reachability and latency. -Sharing metadata through a K-regular overlay (where K>DK > D) allows nodes to pull missing messages.

    GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. -Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency.

    Striking the right balance between parameters like D,KD, K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem.

    At the same time, many other aspects can significantly contribute to the network's performance and scalability. -One option is to realize peers' suitability and continuously changing capabilities while forming overlays.

    For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. -Reshuffling of peering links according to the changing network conditions can lead to superior performance.

    Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. -Grouping multiple super nodes to form virtual node(s) can solve this problem.

    Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks.

    Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. -Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn.

    This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. +The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.

    +

    Introduction​

    +

    Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, +query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh.

    +

    Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. +At the same time, the underlying IP layer is unreliable.

    +

    Therefore, tree-like overlays are not best suited for reliable information propagation. +Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches.

    +

    Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. +Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links.

    +

    An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DD random peers. +Publishing through a D-regular overlay triggers approximately N×DN \times D transmissions. +Reducing DD reduces the redundant transmissions but compromises reachability and latency. +Sharing metadata through a K-regular overlay (where K>DK > D) allows nodes to pull missing messages.

    +

    GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. +Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency.

    +

    Striking the right balance between parameters like D,KD, K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem.

    +

    At the same time, many other aspects can significantly contribute to the network's performance and scalability. +One option is to realize peers' suitability and continuously changing capabilities while forming overlays.

    +

    For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. +Reshuffling of peering links according to the changing network conditions can lead to superior performance.

    +

    Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. +Grouping multiple super nodes to form virtual node(s) can solve this problem.

    +

    Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks.

    +

    Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. +Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn.

    +

    This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. We look into different constraints imposed by application-specific needs (performance goals) and investigate various choices that can augment the network's performance. We explore overlay designs/freshness, peer selection approaches, message-relaying mechanisms, and resilience against adversaries/peer churn. We consider GossipSub a baseline protocol to explore various possibilities and decisively commit to the ones demonstrating superior performance. -We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.

    GOAL1: Low Latency Operation​

    Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. +We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.

    +

    GOAL1: Low Latency Operation​

    +

    Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. A message delivered after the imposed time bounds is considered as dropped. -An early message delivery in applications like live streaming can further enhance the viewing quality.

    The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. -Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance.

    Interestingly, shortest-path message delivery trees have many limitations:

    1) Changing network dynamics requires a quicker and continuous readjustment of the multicast tree. -2) The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion. -3) Node failure can result in partitions, making many segments unreachable. -4) Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology.

    Solutions involve creating multiple random trees to add redundancy [2]. -Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push).

    Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. -New nodes are added from the overlay on node failure, but it requires non-faulty node selection.

    GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages).

    The mesh degree DLow≤D≤DHighD_{Low} \leq D \leq D_{High} is crucial in deciding message dissemination latency. -A smaller value for DD results in higher latency due to increased rounds, whereas a higher DD reduces latency on the cost of increased bandwidth. -At the same time, keeping DD independent of the growing network size (NN) may increase network-wide message dissemination latency. -Adjusting DD with NN maintains similar latency on the cost of increased workload for peers. -Authors in [3] suggest only a logarithmic increase in DD to maintain a manageable workload for peers. -In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \ln(N) + C for an optimal operation, -where CC is a small constant.

    Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].

    GOAL2: Considering Heterogeneity In Overlay Design​

    Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. +An early message delivery in applications like live streaming can further enhance the viewing quality.

    +

    The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. +Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance.

    +

    Interestingly, shortest-path message delivery trees have many limitations:

    +
      +
    1. Changing network dynamics requires a quicker and continuous readjustment of the multicast tree.
      2. +
    2. The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion.
      3. +
    3. Node failure can result in partitions, making many segments unreachable.
      4. +
    4. Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology.
      5. +
    +

    Solutions involve creating multiple random trees to add redundancy [2]. +Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push).

    +

    Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. +New nodes are added from the overlay on node failure, but it requires non-faulty node selection.

    +

    GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages).

    +

    The mesh degree DLow≤D≤DHighD_{Low} \leq D \leq D_{High} is crucial in deciding message dissemination latency. +A smaller value for DD results in higher latency due to increased rounds, whereas a higher DD reduces latency on the cost of increased bandwidth. +At the same time, keeping DD independent of the growing network size (NN) may increase network-wide message dissemination latency. +Adjusting DD with NN maintains similar latency on the cost of increased workload for peers. +Authors in [3] suggest only a logarithmic increase in DD to maintain a manageable workload for peers. +In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \ln(N) + C for an optimal operation, +where CC is a small constant.

    +

    Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].

    +

    GOAL2: Considering Heterogeneity In Overlay Design​

    +

    Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. Most of the research on P2P networks provides simulation results assuming nodes with similar capabilities. -The aspect of dissimilar capabilities and resource-constrained nodes is less explored.

    It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg} does not exceed ln⁡(N)+C\ln(N) + C. -Enforcing all the nodes to have approximately ln⁡(N)+C\ln(N) + C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. +The aspect of dissimilar capabilities and resource-constrained nodes is less explored.

    +

    It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg} does not exceed ln⁡(N)+C\ln(N) + C. +Enforcing all the nodes to have approximately ln⁡(N)+C\ln(N) + C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. At the same time, connecting high-bandwidth nodes through a low-bandwidth node undermines the network's performance. Ideally, the workload on any node should not exceed its available resources. -A better solution involves a two-phased operation:

    1. Every node computes its available bandwidth and selects a node degree DD proportional to its available bandwidth [4]. +A better solution involves a two-phased operation:

      +
        +
      1. +

        Every node computes its available bandwidth and selects a node degree DD proportional to its available bandwidth [4]. Different bandwidth estimation approaches are suggested in literature [5,6]. Simple bandwidth estimation approaches like variable packet size probing [6] yield similar results with less complexity. It is also worth mentioning that many nodes may want to allocate only a capped share of their bandwidth to the network. -Lowering DD according to the available bandwidth can still prove helpful. +Lowering DD according to the available bandwidth can still prove helpful. Additionally, bandwidth preservation at the transport layer through approaches like µTP can be useful. -To further conform to the suggested mesh-degree average DavgD_{avg}, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}.

      2. From the available local view, every node tries connecting peers with the lowest latency until DD connections are made. -We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning.

      The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \leq D \leq D_{High} connections.

      GOAL3: Bandwidth Optimization​

      Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. +To further conform to the suggested mesh-degree average DavgD_{avg}, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}.

      +
      2. +
    2. +

      From the available local view, every node tries connecting peers with the lowest latency until DD connections are made. +We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning.

      +
      3. +
    +

    The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \leq D \leq D_{High} connections.

    +

    GOAL3: Bandwidth Optimization​

    +

    Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. This not only adds to the network-wide message dissemination latency but a significant share of the network's bandwidth is wasted on (usually) unnecessary transmissions. -It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures.

    Many efforts have been made to minimize the impact of redundant transmissions. +It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures.

    +

    Many efforts have been made to minimize the impact of redundant transmissions. These solutions include multicast delivery trees, metadata sharing to enable pull-based operation, in-network information caching, etc. [7,8]. -GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages).

    A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. +GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages).

    +

    A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. As a result, redundant message transmissions can serve as a critical advantage of the GossipSub protocol. -Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3. Assuming an XOR coding function, -we know two trivial properties: x1⊕x2⊕x2=x1x1 \oplus x2 \oplus x2 = x1 and ∣x1∣=∣x1⊕x2⊕x2∣\vert x1 \vert = \vert x1 \oplus x2 \oplus x2 \vert.

    This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. +Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3. Assuming an XOR coding function, +we know two trivial properties: x1⊕x2⊕x2=x1x1 \oplus x2 \oplus x2 = x1 and ∣x1∣=∣x1⊕x2⊕x2∣\vert x1 \vert = \vert x1 \oplus x2 \oplus x2 \vert.

    +

    This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. The receiver can reconstruct the original message from encoded segments. -As a result, fewer transmissions are sufficient for sending more messages to the network.

    However, sharing linear combinations of messages requires organizing messages in intervals, +As a result, fewer transmissions are sufficient for sending more messages to the network.

    +

    However, sharing linear combinations of messages requires organizing messages in intervals, and devising techniques to identify all messages belonging to each interval. In addition, combining messages from different publishers requires more complex arrangements, involving embedding publisher/message IDs, delayed forwarding (to accommodate more messages), and mechanisms to ensure the decoding of messages at all peers. -Careful application-specific need analysis can help decide the benefits against the added complexity.

    GOAL4: Handling Large Messages​

    Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. -This introduces two challenges:

    1) Redundant large message transmissions result in severe network congestion. -2) Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages.

    The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. +Careful application-specific need analysis can help decide the benefits against the added complexity.

    +

    GOAL4: Handling Large Messages​

    +

    Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. +This introduces two challenges:

    +
      +
    1. Redundant large message transmissions result in severe network congestion.
      2. +
    2. Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages.
      3. +
    +

    The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. Most of the work done for handling large messages involves curtailing redundant transmissions using multicast delivery trees, -reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc.

    Approaches like message aggregation also prove helpful in minimizing bandwidth wastage.

    Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions:

    1. Using IDontWant message proposal [13] and staggered sending.

      IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. +reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc.

      +

      Approaches like message aggregation also prove helpful in minimizing bandwidth wastage.

      +

      Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions:

      +
        +
      1. +

        Using IDontWant message proposal [13] and staggered sending.

        +

        IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. Staggered sending enables relaying the message to a short subset of peers in each round. We argue that simultaneously relaying a message to all peers hampers the effectiveness of the IDontWant message. -Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages.

      2. Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. -We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.

      GOAL5: Scalability​

      P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. -In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \times D bandwidth, -where RR is average data arrival rate. -It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\lceil \log_D (N) \rceil hops. -Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size.

      Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. +Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages.

      +
      2. +
    2. +

      Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. +We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.

      +
      3. +
    +

    GOAL5: Scalability​

    +

    P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. +In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \times D bandwidth, +where RR is average data arrival rate. +It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\lceil \log_D (N) \rceil hops. +Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size.

    +

    Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. Most efforts for bringing scalability to the P2P systems have focused on curtailing redundant transmissions and flat overlay adjustments. -Hierarchical overlay designs, on the other hand, are less explored.

    Placing a logical structure in unstructured P2P systems can help scale P2P networks.

    One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. -An abstract operation of such overlay design is provided below:

    1. Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. -For this purpose, every node tries connecting peers with the lowest latency until DD connections are made or the cluster limit is reached.

    2. A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. +Hierarchical overlay designs, on the other hand, are less explored.

      +

      Placing a logical structure in unstructured P2P systems can help scale P2P networks.

      +

      One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. +An abstract operation of such overlay design is provided below:

      +
        +
      1. +

        Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. +For this purpose, every node tries connecting peers with the lowest latency until DD connections are made or the cluster limit is reached.

        +
        2. +
      2. +

        A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. These super nodes form a fully connected mesh and jointly act as a virtual node, -mitigating the problem of peer churn among super nodes.

      3. Virtual nodes form a fully connected mesh to construct a hierarchical overlay. +mitigating the problem of peer churn among super nodes.

        +
        4. +
      4. +

        Virtual nodes form a fully connected mesh to construct a hierarchical overlay. Each virtual node is essentially a collection of super nodes; -a link to any of the constituent super nodes represents a link to the virtual node.

      5. One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.

      Summary​

      Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. +a link to any of the constituent super nodes represents a link to the virtual node.

      +
      3. +
    3. +

      One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.

      +
      4. +
    +

    Summary​

    +

    Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. On the other hand, a hierarchical overlay can bring scalability at the cost of increased complexity. Regardless of the overlay design, a continuous readjustment to appropriate peering links is essential for superior performance. At the same time, bandwidth preservation (through message aggregation, caching at strategic locations, metadata sharing, pull-based operation, etc.) can help minimize latency. -However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.

    References

    • [1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf
    • [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905
    • [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf
    • [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document
    • [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454
    • [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658
    • [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473
    • [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12
    • [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576
    • [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676
    • [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446
    • [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844
    • [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413
    • [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5
    • [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16
    • [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458
    ]]> +However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.

    +

    References

    +
      +
    • [1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf
      • +
    • [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905
      • +
    • [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf
      • +
    • [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document
      • +
    • [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454
      • +
    • [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658
      • +
    • [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473
      • +
    • [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12
      • +
    • [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576
      • +
    • [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676
      • +
    • [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446
      • +
    • [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844
      • +
    • [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413
      • +
    • [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5
      • +
    • [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16
      • +
    • [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458
      • +
    ]]>     <![CDATA[Nescience - A zkVM leveraging hiding properties]]> @@ -1761,148 +4090,714 @@ However, problems like peer churn and in-network adversaries can be best allevia https://vac.dev/rlog/Nescience-A-zkVM-leveraging-hiding-properties Mon, 28 Aug 2023 12:00:00 GMT - Nescience, a privacy-first blockchain zkVM.

    Introduction​

    Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. + Nescience, a privacy-first blockchain zkVM.

    + +

    Introduction​

    +

    Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. The goals include creating a state separation architecture for public/private computation, designing a versatile virtual machine based on mainstream instruction sets, creating proofs for private state updates, implementing a kernel-based architecture for correct execution of private functions, -and implementing core DeFi protocols such as AMMs and staking from a privacy perspective.

    It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. +and implementing core DeFi protocols such as AMMs and staking from a privacy perspective.

    +

    It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. To achieve this goal, Nescience will implement a versatile virtual machine that can be used to implement existing blockchain applications, -while also enabling the development of privacy-centric protocols such as private staking and private DEXs.

    To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, -while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover.

    It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. +while also enabling the development of privacy-centric protocols such as private staking and private DEXs.

    +

    To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, +while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover.

    +

    It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. Finally, Nescience intends to implement permissive licensing, which means that the source code will be open-source, -and developers will be able to use and modify the code without any restriction.

    Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, +and developers will be able to use and modify the code without any restriction.

    +

    Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, potential solutions, and alternatives that lay ahead. Each step is a testament to our commitment to thoroughness; we systematically test various possibilities and decisively commit to the one that demonstrates paramount performance and utility. For instance, as we progress towards achieving Goal 2, we are undertaking a rigorous benchmarking of the Nova proof system against its contemporaries. Should Nova showcase superior performance metrics, we stand ready to integrate it as our proof system of choice. Through such meticulous approaches, -we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.

    Goal 1: Create a State Separation Architecture​

    The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, -employing an account-based framework for the public state and a UTXO-based structure for the private state.

    The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, +we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.

    +

    Goal 1: Create a State Separation Architecture​

    +

    The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, +employing an account-based framework for the public state and a UTXO-based structure for the private state.

    +

    The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, while the account-based paradigm assigns balances to accounts that transactions can modify. Although the UTXO model bolsters privacy by concealing comprehensive balances, the pursuit of a dual architecture mandates a meticulous synchronization of these state models, -ensuring that private transactions remain inconspicuous in the wider public network state.

    This task is further complicated by the divergent transaction processing methods intrinsic to each model, +ensuring that private transactions remain inconspicuous in the wider public network state.

    +

    This task is further complicated by the divergent transaction processing methods intrinsic to each model, necessitating a thoughtful and innovative approach to harmonize their functionality. To seamlessly bring together the dual architecture, harmonizing the account-based model for public state with the UTXO-based model for private state, -a comprehensive strategy is essential.

    The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. -It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model.

    Here's a breakdown and a potential strategy for harmonizing these models:

    Rationale Behind the Dual Architecture: ​

    • Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, -and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications.

    • UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. -By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.

    Harmonizing the Two Systems: ​

    1. Translation Layer

      • Role: Interface between UTXO and account-based states.

      • UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. +a comprehensive strategy is essential.

        +

        The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. +It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model.

        +

        Here's a breakdown and a potential strategy for harmonizing these models:

        +

        Rationale Behind the Dual Architecture: ​

        +
          +
        • +

          Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, +and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications.

          +
          • +
        • +

          UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. +By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.

          +
          • +
        +

        Harmonizing the Two Systems: ​

        +
          +
        1. +

          Translation Layer

          +
            +
          • +

            Role: Interface between UTXO and account-based states.

            +
            • +
          • +

            UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. This could involve creating a temporary 'pseudo-account' that mirrors the -UTXO's attributes.

          • Account-to-UTXO Adapter: When an account wishes to make a private transaction, -it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction.

        2. Unified Identity Management

          • Role: Maintain a unified identity (or address) system that works across both state models, -allowing users to easily manage their public and private states without requiring separate identities.

          • Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. -This ensures privacy while keeping management centralized for the user.

        1. State Commitments

          • Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions.

          • Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. -Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies.

          • Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy.

        1. Batch Processing & Anonymity Sets

          • Role: Group several UTXO-based private transactions into a single public account-based transaction. -This can provide a level of obfuscation and can make synchronization between the two models more efficient.

          • CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy.

          • Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, -providing zk-SNARKs-based private transactions.

        2. Event Hooks & Smart Contracts

          • Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. -For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model.

          • Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, -a smart contract might release funds (account-based) once a specific UTXO is spent.

          • Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, -allowing for private logic execution.

        Challenges and Solutions ​

        1. Synchronization Overhead

          • Challenge: Combining two distinct transaction models creates an inherent synchronization challenge.

          • State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. -Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required.

          • Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. -This structure helps reduce the immediate load on the primary system.

          • Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. -This can ensure consistency without constant synchronization.

        2. Double Spending

          • Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks.

          • Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements.

          • Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. -If discrepancies arise, the transaction can be halted.

          • Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending.

        3. Complexity in User Experience

          • Challenge: The dual model, while powerful, is inherently complex.

          • Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, -allowing users to make transactions without needing to understand the nuances of the dual model.

          • Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, -especially emphasizing when and why they might choose one transaction type over the other.

          • Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. -This real-time feedback can be invaluable for iterative design improvements.

        4. Security

          • Challenge: Merging two systems can introduce unforeseen vulnerabilities.

          • Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, -especially those that might exploit the interaction between the two systems.

          • Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. -Each layer can act as a fail-safe if a potential threat bypasses another.

          • Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. -If any suspicious activity is detected, they can take corrective measures or raise alerts.

        5. Gas & Fee Management:

          • Challenge: A dual model can lead to convoluted fee structures.

          • Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. -This can ensure fairness and prevent network abuse.

          • Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. -This helps users understand potential costs upfront.

          • Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, -simplifying the gas acquisition process.

        By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, -combining the strengths of both UTXO and account-based models without their standalone limitations.

        AspectDetails
        Harmony- Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures.
        Challenges- Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs.
        Proposed Solutions- Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.

        Goal 2: Virtual Machine Creation​

        The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. -Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications.

        This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. -Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures.

        WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, -and boasts seamless integration across diverse blockchain platforms.

        Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, +UTXO's attributes.

        +
        • +
      • +

        Account-to-UTXO Adapter: When an account wishes to make a private transaction, +it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction.

        +
        • +
      +
      2. +
    2. +

      Unified Identity Management

      +
        +
      • +

        Role: Maintain a unified identity (or address) system that works across both state models, +allowing users to easily manage their public and private states without requiring separate identities.

        +
        • +
      • +

        Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. +This ensures privacy while keeping management centralized for the user.

        +
        • +
      +
      3. +
    3. +

      State Commitments

      +
        +
      • +

        Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions.

        +
        • +
      • +

        Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. +Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies.

        +
        • +
      • +

        Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy.

        +
        • +
      +
      4. +
    4. +

      Batch Processing & Anonymity Sets

      +
        +
      • +

        Role: Group several UTXO-based private transactions into a single public account-based transaction. +This can provide a level of obfuscation and can make synchronization between the two models more efficient.

        +
        • +
      • +

        CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy.

        +
        • +
      • +

        Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, +providing zk-SNARKs-based private transactions.

        +
        • +
      +
      5. +
    5. +

      Event Hooks & Smart Contracts

      +
        +
      • +

        Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. +For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model.

        +
        • +
      • +

        Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, +a smart contract might release funds (account-based) once a specific UTXO is spent.

        +
        • +
      • +

        Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, +allowing for private logic execution.

        +
        • +
      +
      6. +
    +

    Challenges and Solutions ​

    +
      +
    1. +

      Synchronization Overhead

      +
        +
      • +

        Challenge: Combining two distinct transaction models creates an inherent synchronization challenge.

        +
        • +
      • +

        State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. +Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required.

        +
        • +
      • +

        Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. +This structure helps reduce the immediate load on the primary system.

        +
        • +
      • +

        Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. +This can ensure consistency without constant synchronization.

        +
        • +
      +
      2. +
    2. +

      Double Spending

      +
        +
      • +

        Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks.

        +
        • +
      • +

        Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements.

        +
        • +
      • +

        Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. +If discrepancies arise, the transaction can be halted.

        +
        • +
      • +

        Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending.

        +
        • +
      +
      3. +
    3. +

      Complexity in User Experience

      +
        +
      • +

        Challenge: The dual model, while powerful, is inherently complex.

        +
        • +
      • +

        Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, +allowing users to make transactions without needing to understand the nuances of the dual model.

        +
        • +
      • +

        Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, +especially emphasizing when and why they might choose one transaction type over the other.

        +
        • +
      • +

        Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. +This real-time feedback can be invaluable for iterative design improvements.

        +
        • +
      +
      4. +
    4. +

      Security

      +
        +
      • +

        Challenge: Merging two systems can introduce unforeseen vulnerabilities.

        +
        • +
      • +

        Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, +especially those that might exploit the interaction between the two systems.

        +
        • +
      • +

        Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. +Each layer can act as a fail-safe if a potential threat bypasses another.

        +
        • +
      • +

        Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. +If any suspicious activity is detected, they can take corrective measures or raise alerts.

        +
        • +
      +
      5. +
    5. +

      Gas & Fee Management:

      +
        +
      • +

        Challenge: A dual model can lead to convoluted fee structures.

        +
        • +
      • +

        Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. +This can ensure fairness and prevent network abuse.

        +
        • +
      • +

        Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. +This helps users understand potential costs upfront.

        +
        • +
      • +

        Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, +simplifying the gas acquisition process.

        +
        • +
      +
      6. +
    +

    By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, +combining the strengths of both UTXO and account-based models without their standalone limitations.

    +
    AspectDetails
    Harmony- Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures.
    Challenges- Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs.
    Proposed Solutions- Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.
    +

    Goal 2: Virtual Machine Creation​

    +

    The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. +Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications.

    +

    This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. +Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures.

    +

    WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, +and boasts seamless integration across diverse blockchain platforms.

    +

    Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, elevating the levels of security and privacy. However, it is crucial to acknowledge that employing WASM or RISC-V might present challenges, -given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs).

    Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, +given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs).

    +

    Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, while potentially addressing optimization concerns, necessitate extensive research and testing due to their relatively nascent status within the field. -This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance.

    The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. +This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance.

    +

    The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. The combination of two renowned instruction sets, WASM and RISC-V, in tandem with ZK, is an innovation that could redefine privacy standards in blockchain. -Let's dissect the challenges and possibilities inherent in this goal:

    1. Established Mainstream Instruction Sets - WASM and RISC-V

      • Strengths:

        • WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. -Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape.

        • RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. +Let's dissect the challenges and possibilities inherent in this goal:

          +
            +
          1. +

            Established Mainstream Instruction Sets - WASM and RISC-V

            +
              +
            • +

              Strengths:

              +
                +
              • +

                WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. +Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape.

                +
                • +
              • +

                RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. Its adaptability to both UTXO and account-based structures coupled with its compatibility with trusted execution environments like SGX and TEE augments its appeal, -especially in domains that prioritize security and privacy.

            • Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, -they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts.

          1. Pioneering a New, Specialized Instruction Set

            • Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, -potentially offering unmatched performance and optimizations tailored to the project's requirements.

            • Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. -It would also need to garner community trust and support over time.

          1. Contemporary Proof Systems - (Super)Nova, STARKs, Sangria

            • Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, -they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization.

            • Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. +especially in domains that prioritize security and privacy.

              +
              • +
            +
            2. +
          2. +

            Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, +they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts.

            +
            3. +
        +
        • +
      • +

        Pioneering a New, Specialized Instruction Set

        +
          +
        • +

          Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, +potentially offering unmatched performance and optimizations tailored to the project's requirements.

          +
          • +
        • +

          Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. +It would also need to garner community trust and support over time.

          +
          • +
        +
        • +
      • +

        Contemporary Proof Systems - (Super)Nova, STARKs, Sangria

        +
          +
        • +

          Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, +they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization.

          +
          • +
        • +

          Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. The unknowns associated with these systems could introduce unforeseen vulnerabilities or complexities. -While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing.

    Mainstream (WASM, RISC-V)ZK-optimized (New Instruction Set)
    Existing ToolingYESNO
    Blockchain-focusedNOYES
    PerformantDEPENDSYES

    Optimization Concerns for WASM and RISC-V: ​

    • Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. -Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed.

    • Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. -Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently.

    • Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. -Fine-tuned memory management mechanisms are essential to prevent bottlenecks.

    Emerging ZKP Optimized Systems Considerations: ​

    • Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. -The trade-offs between proof size, computational efficiency, and security need to be balanced.

    • Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. -A universal system can be more versatile for diverse applications on the blockchain.

    • Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. -Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.

    Strategies for Integration: ​

    • Iterative Development: Given the complexities, an iterative development approach can be beneficial. -Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities.

    • Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization.

    • External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.

    Goal 3: Proofs Creation and Verification​

    The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. +While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing.

    + + + + +
    Mainstream (WASM, RISC-V)ZK-optimized (New Instruction Set)
    Existing ToolingYESNO
    Blockchain-focusedNOYES
    PerformantDEPENDSYES
    +

    Optimization Concerns for WASM and RISC-V: ​

    +
      +
    • +

      Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. +Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed.

      +
      • +
    • +

      Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. +Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently.

      +
      • +
    • +

      Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. +Fine-tuned memory management mechanisms are essential to prevent bottlenecks.

      +
      • +
    +

    Emerging ZKP Optimized Systems Considerations: ​

    +
      +
    • +

      Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. +The trade-offs between proof size, computational efficiency, and security need to be balanced.

      +
      • +
    • +

      Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. +A universal system can be more versatile for diverse applications on the blockchain.

      +
      • +
    • +

      Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. +Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.

      +
      • +
    +

    Strategies for Integration: ​

    +
      +
    • +

      Iterative Development: Given the complexities, an iterative development approach can be beneficial. +Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities.

      +
      • +
    • +

      Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization.

      +
      • +
    • +

      External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.

      +
      • +
    +

    Goal 3: Proofs Creation and Verification​

    +

    The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. Concurrently, the responsibility of verifying these proofs and executing public functions within the virtual machine can be effectively delegated to an external prover, a role that is incentivized to operate with utmost honesty and integrity. This intricate balance seeks to safeguard against information leakage, -preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework.

    To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. +preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework.

    +

    To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. This two-pronged approach serves as a compelling deterrent against dishonest behavior and fosters an environment of accountability. In addition to incentivization, a crucial consideration is the economic aspect of verification and execution. -The verification process has been intentionally designed to be more cost-effective than execution.

    This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. +The verification process has been intentionally designed to be more cost-effective than execution.

    +

    This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. By maintaining a cost balance that favors verification, we bolster the system’s resilience against fraudulent activities while ensuring its efficiency. In sum, our multifaceted approach endeavors to strike an intricate equilibrium between user-initiated proof creation, external verification, and incentivization. -This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability.

    As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, +This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability.

    +

    As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, and the overall integrity of the system is fortified against potential adversarial actions. To achieve the goals of user-initiated proof creation, -external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed:

    1. User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies.

      • Challenges:

        • Maintaining the quality and integrity of the proofs generated by users.

        • Ensuring that users have the tools and knowledge to produce valid proofs.

      • Solutions:

        • Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process.

        • Implement checks at the verifier's end to ensure the quality of proofs.

    1. External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine.

      • Challenges:

        • Ensuring that the external prover acts honestly.

        • Avoiding centralized points of failure.

      • Solutions:

        • Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work.

        • Use reputation systems to rank provers based on their past performances, creating a trust hierarchy.

    2. Incentivization Framework: A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task.

      • Challenges:

        • Determining the right balance of rewards and penalties.

        • Ensuring that the system cannot be gamed for undue advantage.

      • Solutions1:

        • Implement a dynamic reward system that adjusts based on network metrics and provers' performance.

        • Use a staking mechanism where provers need to lock up a certain amount of assets. -Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets.

    3. Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks.

      • Challenges:

        • Setting the right cost metrics for both verification and execution.

        • Ensuring that genuine users aren’t priced out of the system.

      • Solutions:

        • Use a dynamic pricing model, adjusting costs in real-time based on network demand.

        • Implement gas-like mechanisms to differentiate operation costs and ensure fairness.

    4. Maintaining Trustworthiness: Create a system that's transparent, holds all actors accountable, and is economically sound.

      • Challenges:

        • Keeping the balance where users feel their privacy is intact, while provers feel incentivized.

        • Ensuring the system remains resilient against adversarial attacks.

      • Solutions:

        • Implement layered checks and balances.

        • Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO).

    Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, +external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed:

    +
      +
    1. +

      User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Maintaining the quality and integrity of the proofs generated by users.

          +
          • +
        • +

          Ensuring that users have the tools and knowledge to produce valid proofs.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process.

          +
          • +
        • +

          Implement checks at the verifier's end to ensure the quality of proofs.

          +
          • +
        +
        • +
      +
      2. +
    2. +

      External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Ensuring that the external prover acts honestly.

          +
          • +
        • +

          Avoiding centralized points of failure.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work.

          +
          • +
        • +

          Use reputation systems to rank provers based on their past performances, creating a trust hierarchy.

          +
          • +
        +
        • +
      +
      3. +
    3. +

      ** Incentivization Framework:** A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Determining the right balance of rewards and penalties.

          +
          • +
        • +

          Ensuring that the system cannot be gamed for undue advantage.

          +
          • +
        +
        • +
      • +

        Solutions1:

        +
          +
        • +

          Implement a dynamic reward system that adjusts based on network metrics and provers' performance.

          +
          • +
        • +

          Use a staking mechanism where provers need to lock up a certain amount of assets. +Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets.

          +
          • +
        +
        • +
      +
      4. +
    4. +

      Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Setting the right cost metrics for both verification and execution.

          +
          • +
        • +

          Ensuring that genuine users aren’t priced out of the system.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Use a dynamic pricing model, adjusting costs in real-time based on network demand.

          +
          • +
        • +

          Implement gas-like mechanisms to differentiate operation costs and ensure fairness.

          +
          • +
        +
        • +
      +
      5. +
    5. +

      ** Maintaining Trustworthiness:** Create a system that's transparent, holds all actors accountable, and is economically sound.

      +
        +
      • +

        Challenges:

        +
          +
        • +

          Keeping the balance where users feel their privacy is intact, while provers feel incentivized.

          +
          • +
        • +

          Ensuring the system remains resilient against adversarial attacks.

          +
          • +
        +
        • +
      • +

        Solutions:

        +
          +
        • +

          Implement layered checks and balances.

          +
          • +
        • +

          Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO).

          +
          • +
        +
        • +
      +
      6. +
    +

    Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, cost dynamics, and verification integrity. A thoughtful combination of these mechanisms ensures that the system remains robust, resilient, -and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation.

    AspectDetails
    Design Principle- User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions.
    Trust & Privacy- Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage.
    Incentivization Framework- Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior.
    Economic Considerations- Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency.
    OutcomeAn ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.

    Goal 4: Kernel-based Architecture Implementation​

    This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. +and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation.

    +
    AspectDetails
    Design Principle- User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions.
    Trust & Privacy- Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage.
    Incentivization Framework- Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior.
    Economic Considerations- Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency.
    OutcomeAn ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.
    +

    Goal 4: Kernel-based Architecture Implementation​

    +

    This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. This innovative approach employs recursion to construct a call stack, which is then validated through iterative recursive computations. -At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack.

    The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. -While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies.

    The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. +At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack.

    +

    The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. +While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies.

    +

    The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. Moreover, the iterative recursive computations could potentially exhibit computational expansion as the depth of recursion increases. -This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail.

    In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. +This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail.

    +

    In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. By weaving recursion and iterative recursive computations into the fabric of our system, we aim to establish a mechanism that accentuates the trustworthiness of private function executions, -while conscientiously navigating the computational demands that ensue.

    To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, -several strategic steps and considerations can be undertaken: recursion handling and depth management.

    Recursion Handling

    • Call Stack Management:

      • Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state.

    • Proof Accumulation:

      • Design a mechanism to accumulate proof data for each function call within the call stack. -This includes cryptographic commitments, intermediate results, and cryptographic challenges.

      • Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process.

    • Intermediary SNARK Proofs:

      • Develop an intermediary SNARK proof for each function call’s correctness within the call stack. -This proof should demonstrate that the function executed correctly and produced expected outputs.

      • Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack.

    Depth management

    • Depth Limitation:

      • Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations.

      • Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth.

    • Graceful Degradation:

      • Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. -This may involve transitioning to alternative execution modes or optimization techniques.

      • Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed.

    • Resource Monitoring:

      • Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. -Provide real-time feedback to users about the cost and impact of recursive execution.

    • Dynamic Depth Adjustment:

      • Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources.

      • Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints.

    • Fallback Mechanisms:

      • Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. -These mechanisms could involve alternative verification methods or delayed execution.

    • User Notifications:

      • Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage.

    Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. +while conscientiously navigating the computational demands that ensue.

    +

    To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, +several strategic steps and considerations can be undertaken: recursion handling and depth management.

    + Recursion Handling +
      +
    • +

      Call Stack Management:

      +
        +
      • Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state.
        • +
      +
      • +
    • +

      _Proof Accumulation: _

      +
        +
      • +

        Design a mechanism to accumulate proof data for each function call within the call stack. +This includes cryptographic commitments, intermediate results, and cryptographic challenges.

        +
        • +
      • +

        Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process.

        +
        • +
      +
      • +
    • +

      Intermediary SNARK Proofs:

      +
        +
      • +

        Develop an intermediary SNARK proof for each function call’s correctness within the call stack. +This proof should demonstrate that the function executed correctly and produced expected outputs.

        +
        • +
      • +

        Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack.

        +
        • +
      +
      • +
    + Depth management +
      +
    • +

      Depth Limitation:

      +
        +
      • +

        Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations.

        +
        • +
      • +

        Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth.

        +
        • +
      +
      • +
    • +

      Graceful Degradation:

      +
        +
      • +

        Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. +This may involve transitioning to alternative execution modes or optimization techniques.

        +
        • +
      • +

        Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed.

        +
        • +
      +
      • +
    • +

      Resource Monitoring:

      +
        +
      • Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. +Provide real-time feedback to users about the cost and impact of recursive execution.
        • +
      +
      • +
    • +

      Dynamic Depth Adjustment:

      +
        +
      • +

        Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources.

        +
        • +
      • +

        Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints.

        +
        • +
      +
      • +
    • +

      Fallback Mechanisms:

      +
        +
      • Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. +These mechanisms could involve alternative verification methods or delayed execution.
        • +
      +
      • +
    • +

      User Notifications:

      +
        +
      • Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage.
        • +
      +
      • +
    +

    Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. While the approach promises robust outcomes, it's pivotal to maneuver through its intricacies with astute strategies, ensuring computational efficiency and economic viability. -By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.

    Goal 5: Seamless Interaction Design​

    Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. -This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions.

    A notable challenge in this endeavor lies in the intricate interplay between public and private states, -wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage.

    The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. -This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality.

    A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. +By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.

    +

    Goal 5: Seamless Interaction Design​

    +

    Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. +This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions.

    +

    A notable challenge in this endeavor lies in the intricate interplay between public and private states, +wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage.

    +

    The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. +This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality.

    +

    A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. The solution involves devising strategies to mitigate the risk of revealing private transaction details when connected to corresponding public actions. -By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability.

    Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, -without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design.

    To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. -In the table below, a comprehensive list of solutions that address these objectives:

    Solution CategoryDescription
    Layer 2 SolutionsEmploy zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs.
    Intermediary Smart ContractsCraft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially.
    Decentralized Identity & PseudonymityImplement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs.
    Confidential Sidechains & Cross-ChainSet up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains.
    Temporal Data StructuresCreate chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy.
    Homomorphic Encryption & MPCApply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers.
    Commit-Reveal SchemesIntroduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions.
    Auditability & VerifiabilityUse on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation.
    Data Fragmentation & ShardingFragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography.
    Ring Signatures & CoinJoinIncorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.

    Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework​

    The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. +By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability.

    +

    Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, +without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design.

    +

    To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. +In the table below, a comprehensive list of solutions that address these objectives:

    +
    Solution CategoryDescription
    Layer 2 SolutionsEmploy zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs.
    Intermediary Smart ContractsCraft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially.
    Decentralized Identity & PseudonymityImplement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs.
    Confidential Sidechains & Cross-ChainSet up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains.
    Temporal Data StructuresCreate chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy.
    Homomorphic Encryption & MPCApply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers.
    Commit-Reveal SchemesIntroduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions.
    Auditability & VerifiabilityUse on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation.
    Data Fragmentation & ShardingFragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography.
    Ring Signatures & CoinJoinIncorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.
    +

    Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework​

    +

    The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. This endeavor comes with inherent challenges, especially considering the heterogeneity of existing DeFi protocols, predominantly built on Ethereum. These variations in programming languages and VMs exacerbate the quest for interoperability. Furthermore, the success and functionality of DeFi protocols is closely tied to liquidity, -which in turn is influenced by user engagement and the amount of funds locked into the system.

    Strategic Roadmap for Goal 6 ​

    1. Pioneering Privacy-Centric DeFi Models: Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity.

    2. Specialized Smart Contracts with Privacy: Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape.

    3. Optimized User Interfaces: Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security.

    4. Tackling Interoperability:

      • Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments.

      • Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly.

    1. Enhancing and Sustaining Liquidity:

      • Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space.

      • Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity.

      • Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies.

    1. Amplifying Community Engagement: Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, -thereby nurturing a vibrant, privacy-advocating DeFi community.

    Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, -all while effectively overcoming the obstacles related to interoperability and liquidity concerns.

    Summary of the Architecture​

    In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). +which in turn is influenced by user engagement and the amount of funds locked into the system.

    +

    Strategic Roadmap for Goal 6 ​

    +
      +
    1. +

      ** Pioneering Privacy-Centric DeFi Models: ** Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity.

      +
      2. +
    2. +

      ** Specialized Smart Contracts with Privacy: ** Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape.

      +
      3. +
    3. +

      ** Optimized User Interfaces: ** Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security.

      +
      4. +
    4. +

      ** Tackling Interoperability: **

      +
        +
      • +

        Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments.

        +
        • +
      • +

        Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly.

        +
        • +
      +
      5. +
    5. +

      ** Enhancing and Sustaining Liquidity: **

      +
        +
      • +

        Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space.

        +
        • +
      • +

        Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity.

        +
        • +
      • +

        Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies.

        +
        • +
      +
      6. +
    6. +

      ** Amplifying Community Engagement:** Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, +thereby nurturing a vibrant, privacy-advocating DeFi community.

      +
      7. +
    +

    Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, +all while effectively overcoming the obstacles related to interoperability and liquidity concerns.

    +

    Summary of the Architecture​

    +

    In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). These proofs ensure that while private state data remains undisclosed, public state transitions can still be carried out and subsequently verified by third parties. This blend of public and private state is envisaged to be achieved through a state tree representing the public state, while the encrypted state leaves stand for the private state. Each user's private state indicates validity through the absence of a corresponding nullifier. A nullifier is a unique cryptographic value generated in privacy-preserving blockchain transactions to prevent double-spending, -ensuring that each private transaction is spent only once without revealing its details.

    Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. +ensuring that each private transaction is spent only once without revealing its details.

    +

    Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. For validating a singular private function call, we're leaning into the kernel-based model inspired by the ZEXE protocol. Defined as kernel circuits, these functions validate the correct execution of each private function call. Due to their recursive circuit structure, a succession of private function calls can be executed by calculating proofs in an iterative manner. -Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs.

    Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. -The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities.

    We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. +Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs.

    +

    Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. +The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities.

    +

    We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. Instead of the ZEXE protocol's structure, our approach pivots to a method anchored in the security of a secret combined with the uniqueness from hashing with the contract address. The underlying rationale is straightforward: the sender, equipped with a unique nonce and salt for the transaction, hashes the secret, payload, nonce, and salt. This result is then hashed with the contract address for the final value. The hash function's unidirectional nature ensures that the input cannot be deduced easily from its output. A specific concern, however, is the potential repetition of secret and payload values across transactions, which could jeopardize privacy. -Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation.

    Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. +Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation.

    +

    Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. Fully Homomorphic Encryption (FHE) offers another pathway, enabling function execution on encrypted data, or Multi-Party Computation (MPC) which guarantees non-disclosure of function or inputs. Yet, integrating ZKPs with either FHE or MPC presents a challenge. Combining cryptographic functions like SHA-3 and BLAKE2 can also bolster security and uniqueness. -It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.

    Current State​

    Our aim is to revolutionize the privacy and security paradigms through Nescience. +It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.

    +

    Current State​

    +

    Our aim is to revolutionize the privacy and security paradigms through Nescience. As we strive to set milestones and achieve groundbreaking advancements, -our current focus narrows onto the realization of Goal 2 and Goal 3.

    Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. +our current focus narrows onto the realization of Goal 2 and Goal 3.

    +

    Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. We believe that integrating the right proof system is pivotal to our project's success, which brings us to Nova [8]. In our project journey, we have opted to integrate the Nova proof system, recognizing its potential alignment with our overarching goals. However, as part of our meticulous approach to innovation and optimization, we acknowledge the need to thoroughly examine Nova’s performance capabilities, -particularly due to its status as a pioneering and relatively unexplored proof system.

    This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], +particularly due to its status as a pioneering and relatively unexplored proof system.

    +

    This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], pitting Nova against other prominent proof systems in the field, including Halo2 [10], Plonky2 [11], and Starky [12]. This ongoing and methodical initiative is designed to ensure a fair and impartial assessment, enabling us to draw meaningful conclusions about Nova’s strengths and limitations in relation to its counterparts. @@ -1910,17 +4805,52 @@ By leveraging the Poseidon recursion technique, we are poised to conduct an exha Through this testing framework, we aim to discern whether Nova possesses the potential to outshine its contemporaries in terms of efficiency, scalability, and overall performance. The outcome of this rigorous evaluation will be pivotal in shaping our strategic decisions moving forward. Armed with a comprehensive understanding of Nova’s performance metrics vis-à-vis other proof systems, -we can confidently chart a course that maximizes the benefits of our project’s optimization efforts.

    Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, +we can confidently chart a course that maximizes the benefits of our project’s optimization efforts.

    +

    Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, incentivizing honest behaviour, and ensuring the cost-effective verification of transactions. At the heart of this endeavor is our drive to empower users by allowing them the autonomy of generating proofs for private state updates, thereby reducing dependencies and enhancing privacy. We would like to actively work on providing comprehensive documentation, user-friendly tools, -and tutorials to aid users in this intricate process.

    Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. +and tutorials to aid users in this intricate process.

    +

    Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. Our commitment is further cemented by our efforts to introduce a dynamic reward system that adjusts based on network metrics and prover performance. -This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.

    References

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    [4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    [5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md

    [6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin

    [7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui

    [8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370

    [9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations

    [10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/

    [11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2

    [12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582

    1. Incentive Mechanisms:

      • Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. -This incentivizes participation and encourages integrity.

      • Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. -Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded.

      • Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, -aligning incentives with the blockchain’s broader consensus mechanism.

      ↩

    ]]>     +This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.

    +

    References

    +

    [1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf

    +

    [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/

    +

    [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14

    +

    [4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

    +

    [5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md

    +

    [6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin

    +

    [7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui

    +

    [8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370

    +

    [9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations

    +

    [10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/

    +

    [11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2

    +

    [12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582

    + +

    Footnotes​

    +
      +
    1. +

      Incentive Mechanisms:

      +
        +
      • +

        Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. +This incentivizes participation and encourages integrity.

        +
        • +
      • +

        Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. +Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded.

        +
        • +
      • +

        Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, +aligning incentives with the blockchain’s broader consensus mechanism.

        +
        • +
      +↩ +
      2. +
    +
    ]]>     <![CDATA[Device Pairing in Js-waku and Go-waku]]> @@ -1928,7 +4858,43 @@ aligning incentives with the blockchain’s broader consensus mechanism.

    https://vac.dev/rlog/device-pairing-in-js-waku-and-go-waku Mon, 24 Apr 2023 12:00:00 GMT - Device pairing and secure message exchange using Waku and noise protocol.

    As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku.

    Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security.

    This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography.

    One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information

    Device Pairing​

    In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access.

    To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake.

    The process works as follows:

    Actors:

    • Alice the initiator
    • Bob the responder
    1. The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated
    2. Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process.
    3. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully
    4. Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker.

    The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.

    Conclusion​

    With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out!

    Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better.

    Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!

    References​

    ]]>     + Device pairing and secure message exchange using Waku and noise protocol.

    + +

    As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku.

    +

    Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security.

    +

    This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography.

    +

    One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information

    +

    Device Pairing​

    +

    In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access.

    +

    To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake.

    +

    The process works as follows:

    +

    Actors:

    +
      +
    • Alice the initiator
      • +
    • Bob the responder
      • +
    +
      +
    1. The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated
      2. +
    2. Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process.
      3. +
    3. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully
      4. +
    4. Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker.
      5. +
    +

    The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.

    +

    Conclusion​

    +

    With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out!

    +

    Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better.

    +

    Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!

    +

    References​

    +]]>         <![CDATA[The Future of Waku Network: Scaling, Incentivization, and Heterogeneity]]> @@ -1936,44 +4902,96 @@ aligning incentives with the blockchain’s broader consensus mechanism.

    https://vac.dev/rlog/future-of-waku-network Mon, 03 Apr 2023 00:00:00 GMT - Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like.

    Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an + Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like.

    + +

    Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an overview of recent discussions and research outputs, aiming to give you a better understanding of how the Waku network -may look like in terms of scaling and incentivization.

    DOS Mitigation for Status Communities​

    Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN +may look like in terms of scaling and incentivization.

    +

    DOS Mitigation for Status Communities​

    +

    Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN (Rate Limiting Nullifiers) remains the go-to DOS protection solution due to its privacy-preserving and censorship-resistant properties, there is still more work to be done. We are excited to collaborate with PSE -(Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.

    A Heterogeneous Waku Network​

    As we noted in a previous forum post, Waku's protocol +(Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.

    +

    A Heterogeneous Waku Network​

    +

    As we noted in a previous forum post, Waku's protocol incentivization model needs to be flexible to accommodate various business models. Flexibility ensures that projects -can choose how they want to use Waku based on their specific needs.

    Reversing the Incentivization Question​

    Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to -reframe the question and instead ask, "How do we pay for the infrastructure?"

    Waku does not intend to offer a free lunch. +can choose how they want to use Waku based on their specific needs.

    +

    Reversing the Incentivization Question​

    +

    Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to +reframe the question and instead ask, "How do we pay for the infrastructure?"

    +

    Waku does not intend to offer a free lunch. Ethereum's infrastructure is supported by transaction fees and inflation, with validators receiving rewards from both sources. However, this model does not suit a communication network like Waku. Users and platforms would not want to pay for every single message they send. Additionally, Waku aims to support instant -ephemeral messages that do not require consensus or long-term storage.

    Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or +ephemeral messages that do not require consensus or long-term storage.

    +

    Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or inter-wallet communication, may have different value extraction models. Some users might provide services for the project and expect to receive value by running nodes, while others may pay for the product or run infrastructure to contribute back. Waku aims to support each of these use cases, which means there will be various ways to "pay for the -infrastructure."

    In his talk, Oskar addressed two strategies: RLN and service credentials.

    RLN and Service Credentials​

    RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, -and you can send messages.

    Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in -stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community.

    Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, +infrastructure."

    +

    In his talk, Oskar addressed two strategies: RLN and service credentials.

    +

    RLN and Service Credentials​

    +

    RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, +and you can send messages.

    +

    Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in +stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community.

    +

    Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, a community owner could pay for more storage or bandwidth, similar to Slack's model. Alternatively, individual users could contribute financially, akin to Discord's Server Boost, or by sharing their own -resources with their community.

    We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.

    Waku Network: Ethereum or Cosmos?​

    Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos.

    For those not familiar with the difference between both, in a very concise manner:

    • Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure
    • Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties

    We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. +resources with their community.

    +

    We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.

    +

    Waku Network: Ethereum or Cosmos?​

    +

    Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos.

    +

    For those not familiar with the difference between both, in a very concise manner:

    +
      +
    • Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure
      • +
    • Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties
      • +
    +

    We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. If each application has to deploy its own network, we will not achieve this goal. Therefore, we aim Waku to be not only an open source set of protocols, but also a shared infrastructure that anyone can leverage to build applications on top, with some guarantees in terms of decentralization and anonymity. This approach is closer in spirit to Ethereum than Cosmos. -Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network.

    Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. +Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network.

    +

    Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. We currently assume that there will be separate gossipsub networks with different funding models. -Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property.

    Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.

    Scalability and Discovery Protocols​

    To enable scalability, the flow of messages in the Waku network will be divided in shards, +Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property.

    +

    Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.

    +

    Scalability and Discovery Protocols​

    +

    To enable scalability, the flow of messages in the Waku network will be divided in shards, so that not every node has to forward every message of the whole network. -Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in.

    Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between).

    Marketplace protocols may also be developed to help operators understand how they can best support the network and where -their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.

    Open Problems​

    Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. -This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues.

    However, it does not cap traffic. -There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection.

    We have several ongoing initiatives:

    • Analyzing the Status Community protocol to confirm efficient usage of Waku [4]
    • Simulating the Waku Network to measure actual bandwidth usage [5]
    • Segregating chat messages from control and media messages [6]

    The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.

    The Evolving Waku Network​

    The definition of the "Waku Network" will likely change over time. In the near future, it will transition from a single +Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in.

    +

    Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between).

    +

    Marketplace protocols may also be developed to help operators understand how they can best support the network and where +their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.

    +

    Open Problems​

    +

    Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. +This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues.

    +

    However, it does not cap traffic. +There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection.

    +

    We have several ongoing initiatives:

    +
      +
    • Analyzing the Status Community protocol to confirm efficient usage of Waku [4]
      • +
    • Simulating the Waku Network to measure actual bandwidth usage [5]
      • +
    • Segregating chat messages from control and media messages [6]
      • +
    +

    The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.

    +

    The Evolving Waku Network​

    +

    The definition of the "Waku Network" will likely change over time. In the near future, it will transition from a single gossipsub network to a sharded set of networks unified by a common discovery layer. This change will promote scalability -and allow various payment models to coexist within the Waku ecosystem.

    In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly +and allow various payment models to coexist within the Waku ecosystem.

    +

    In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly maintaining its core principles. As Waku continues to evolve, we expect it to accommodate a diverse range of use cases and business models, all while preserving privacy, resisting censorship, avoiding surveillance, and remaining accessible -to devices with limited resources.

    References​

    1. WAKU2-RELAY-SHARDING
    2. 57/STATUS-Simple-Scaling
    3. RLN-V2
    4. Scaling Status Communities: Potential Problems
    5. Waku Network Testing
    6. WAKU2-RELAY-SHARDING: Control Message Shards
    ]]>     +to devices with limited resources.

    +

    References​

    +
      +
    1. WAKU2-RELAY-SHARDING
      2. +
    2. 57/STATUS-Simple-Scaling
      3. +
    3. RLN-V2
      4. +
    4. Scaling Status Communities: Potential Problems
      5. +
    5. Waku Network Testing
      6. +
    6. WAKU2-RELAY-SHARDING: Control Message Shards
      7. +
    ]]>     <![CDATA[Waku for All Decentralized Applications and Infrastructures]]> @@ -1985,49 +5003,147 @@ to devices with limited resources.

    Background​

    We have built Waku to be the communication layer for Web3. Waku is a collection of protocols to chose from for your +and infrastructure projects. We have prioritized our effort to make Waku usable on various platforms and environments.

    + +

    Background​

    +

    We have built Waku to be the communication layer for Web3. Waku is a collection of protocols to chose from for your messaging needs. It enables secure, censorship-resistant, privacy-preserving, spam-protected communication for its user. -It is designed to run on any device, from mobile to the cloud.

    Waku is available on many systems and environments and used by several applications and SDKs for decentralized communications.

    This involved research efforts in various domains: conversational security, protocol incentivization, zero-knowledge, -etc.

    Waku uses novel technologies. Hence, we knew that early dogfooding of Waku was necessary. Even if research -was still in progress [1]. Thus, as soon as Waku protocols and software were usable, we started to push -for the adoption of Waku. This started back in 2021.

    Waku is the communication component of the Web3 trifecta. This trifecta was Ethereum (contracts), Swarm +It is designed to run on any device, from mobile to the cloud.

    +

    Waku is available on many systems and environments and used by several applications and SDKs for decentralized communications.

    +

    This involved research efforts in various domains: conversational security, protocol incentivization, zero-knowledge, +etc.

    +

    Waku uses novel technologies. Hence, we knew that early dogfooding of Waku was necessary. Even if research +was still in progress [1]. Thus, as soon as Waku protocols and software were usable, we started to push +for the adoption of Waku. This started back in 2021.

    +

    Waku is the communication component of the Web3 trifecta. This trifecta was Ethereum (contracts), Swarm (storage) and Whisper (communication). Hence, it made sense to first target dApps which already uses one of the pillars: -Ethereum.

    As most dApps are web apps, we started the development of js-waku for the browser.

    Once ready, we reached out to dApps to integrate Waku, added prizes to hackathons -and gave talks.

    We also assumed we would see patterns in the usage of Waku, that we would facilitate with the help of -SDKs.

    Finally, we created several web apps: +Ethereum.

    +

    As most dApps are web apps, we started the development of js-waku for the browser.

    +

    Once ready, we reached out to dApps to integrate Waku, added prizes to hackathons +and gave talks.

    +

    We also assumed we would see patterns in the usage of Waku, that we would facilitate with the help of +SDKs.

    +

    Finally, we created several web apps: examples -and PoCs.

    By discussing with Waku users and watching it being used, we learned a few facts:

    1. The potential use cases for Waku are varied and many:
    • Wallet <> dApp communication: WalletConnect, XMTP
    • Off-chain (and private) marketplace: +and PoCs.

      +

      By discussing with Waku users and watching it being used, we learned a few facts:

      +
        +
      1. The potential use cases for Waku are varied and many:
        2. +
      +
      1. Many projects are interested in having an embedded chat in their dApp,
      2. There are complex applications that need Waku as a solution. Taking RAILGUN as an example:
      • Web wallet
      • + React Native mobile wallet
      • + NodeJS node/backend.

      (1) means that it is not that easy to create SDKs for common use cases.

      (2) was a clear candidate for an SDK. Yet, building a chat app is a complex task. Hence, the Status app team tackled -this in the form of Status Web.

      Finally, (3) was the most important lesson. We learned that multi-tier applications need Waku for decentralized and +Decentralized Uber

      • +
    • Signature exchange for a multi-sign wallet: Gnosis Safe x Waku
      • +
    • Off-chain Game moves/actions: Super Card Game (EthOnline 2021)
      • +
    • Decentralized Pastebin: Debin
      • +
    +
      +
    1. Many projects are interested in having an embedded chat in their dApp,
      2. +
    2. There are complex applications that need Waku as a solution. Taking RAILGUN as an example:
      3. +
    +
      +
    • Web wallet
      • +
    • + React Native mobile wallet
      • +
    • + NodeJS node/backend.
      • +
    +

    (1) means that it is not that easy to create SDKs for common use cases.

    +

    (2) was a clear candidate for an SDK. Yet, building a chat app is a complex task. Hence, the Status app team tackled +this in the form of Status Web.

    +

    Finally, (3) was the most important lesson. We learned that multi-tier applications need Waku for decentralized and censorship-resistant communications. For these projects, js-waku is simply not enough. They need Waku to work in their -Golang backend, Unity desktop game and React Native mobile app.

    We understood that we should see the whole Waku software suite +Golang backend, Unity desktop game and React Native mobile app.

    +

    We understood that we should see the whole Waku software suite (js-waku, nwaku, go-waku, waku-react-native, etc) as an asset for its success. -That we should not limit outreach, marketing, documentation efforts to the web, but target all platforms.

    From a market perspective, we identified several actors:

    • platforms: Projects that uses Waku to handle communication,
    • operators: Operators run Waku nodes and are incentivized to do so,
    • developers: Developers are usually part of a platforms or solo hackers learning Web3,
    • contributors: Developers and researchers with interests in decentralization, privacy, censorship-resistance, -zero-knowledge, etc.

    Waku for All Decentralized Applications and Infrastructures​

    In 2022, we shifted our focus to make the various Waku implementations usable and used.

    We made Waku multi-plaform.

    We shifted Waku positioning to leverage all Waku implementations and better serve the user's needs:

    We are consolidating the documentation for all implementations on a single website (work in progress) -to improve developer experience.

    This year, we also started the operator outreach effort to push for users to run their own Waku nodes. We have +That we should not limit outreach, marketing, documentation efforts to the web, but target all platforms.

    +

    From a market perspective, we identified several actors:

    +
      +
    • platforms: Projects that uses Waku to handle communication,
      • +
    • operators: Operators run Waku nodes and are incentivized to do so,
      • +
    • developers: Developers are usually part of a platforms or solo hackers learning Web3,
      • +
    • contributors: Developers and researchers with interests in decentralization, privacy, censorship-resistance, +zero-knowledge, etc.
      • +
    +

    Waku for All Decentralized Applications and Infrastructures​

    +

    In 2022, we shifted our focus to make the various Waku implementations usable and used.

    +

    We made Waku multi-plaform.

    +

    We shifted Waku positioning to leverage all Waku implementations and better serve the user's needs:

    + +

    We are consolidating the documentation for all implementations on a single website (work in progress) +to improve developer experience.

    +

    This year, we also started the operator outreach effort to push for users to run their own Waku nodes. We have recently concluded our first operator trial run. Nwaku's documentation, stability and performance has improved. It is now easier to -run your own Waku node.

    Today, operator wannabes most likely run their own nodes to support or use the Waku network. +run your own Waku node.

    +

    Today, operator wannabes most likely run their own nodes to support or use the Waku network. We are dogfooding Waku RLN, our novel economic spam protection protocol, and looking at incentivizing the Waku Store protocol. -This way, we are adding reasons to run your own Waku node.

    For those who were following us in 2021, know that we are retiring the Waku Connect branding in favour of the Waku -branding.

    Waku for Your Project​

    As discussed, Waku is now available on various platforms. The question remains: How can Waku benefit your project?

    Here are a couple of use cases we recently investigated:

    Layer-2 Decentralization​

    Most ([2] [3] roll-ups use a centralized sequencer or equivalent. Running several sequencers is not as straightforward as running several execution nodes. -Waku can help:

    • Provide a neutral marketplace for a mempool: If sequencers compete for L2 tx fees, they may not be incentivized to +This way, we are adding reasons to run your own Waku node.

      +

      For those who were following us in 2021, know that we are retiring the Waku Connect branding in favour of the Waku +branding.

      +

      Waku for Your Project​

      +

      As discussed, Waku is now available on various platforms. The question remains: How can Waku benefit your project?

      +

      Here are a couple of use cases we recently investigated:

      +

      Layer-2 Decentralization​

      +

      Most ([2] [3] roll-ups use a centralized sequencer or equivalent. Running several sequencers is not as straightforward as running several execution nodes. +Waku can help:

      +
        +
      • Provide a neutral marketplace for a mempool: If sequencers compete for L2 tx fees, they may not be incentivized to share transactions with other sequencers. Waku nodes can act as a neutral network to enable all sequences to access -transactions.
      • Enable censorship-resistant wallet<>L2 communication,
      • Provide rate limiting mechanism for spam protection: Using RLN to prevent DDOS.

      Device pairing and communication​

      With Waku Device Pairing, a user can setup a secure encrypted communication channel +transactions.

      • +
    • Enable censorship-resistant wallet<>L2 communication,
      • +
    • Provide rate limiting mechanism for spam protection: Using RLN to prevent DDOS.
      • +
    +

    Device pairing and communication​

    +

    With Waku Device Pairing, a user can setup a secure encrypted communication channel between their devices. As this channel would operate over Waku, it would be censorship-resistant and privacy preserving. -These two devices could be:

    • Ethereum node and mobile phone to access a remote admin panel,
    • Alice's phone and Bob's phone for any kind of secure communication,
    • Mobile wallet and desktop/browser dApp for transaction and signature exchange.

    Check js-waku#950 for the latest update on this.

    Get Involved​

    Developer? Grab any of the Waku implementations and integrate it in your app: https://waku.org/platform.

    Researcher? See https://vac.dev/contribute to participate in Waku research.

    Tech-savvy? Try to run your own node: https://waku.org/operator.

    Otherwise, play around with the various web examples.

    If you want to help, we are hiring!

    Moving Forward​

    What you can expect next:

    • Scalability and performance studies and improvement across Waku software,
    • New websites to easily find documentation about Waku and its implementations,
    • New Waku protocols implemented in all code bases and cross client PoCs +These two devices could be:

      +
        +
      • Ethereum node and mobile phone to access a remote admin panel,
        • +
      • Alice's phone and Bob's phone for any kind of secure communication,
        • +
      • Mobile wallet and desktop/browser dApp for transaction and signature exchange.
        • +
      +

      Check js-waku#950 for the latest update on this.

      +

      Get Involved​

      +

      Developer? Grab any of the Waku implementations and integrate it in your app: https://waku.org/platform.

      +

      Researcher? See https://vac.dev/contribute to participate in Waku research.

      +

      Tech-savvy? Try to run your own node: https://waku.org/operator.

      +

      Otherwise, play around with the various web examples.

      +

      If you want to help, we are hiring!

      +

      Moving Forward​

      +

      What you can expect next:

      +

      References​

      • [1] Waku is modular; it is a suite of protocols; hence some Waku protocols may be mature, while +waku-rln-relay, etc),
        • +
      • Easier to run your own waku node, more operator trials,
        • +
      • Dogfooding and Improvement of existing protocols (e.g. Waku Filter),
        • +
      • Continue our focus Waku portability: Browser, +Raspberry Pi Zero and other restricted-resource environments,
        • +
      • More communication & marketing effort around Waku and the Waku developer community.
        • +
      +
      +

      References​

      +
        +
      • [1] Waku is modular; it is a suite of protocols; hence some Waku protocols may be mature, while new protocols are still being designed. Which means that research continues to be ongoing while -Waku is already used in production.
      • [2] The Optimism Foundation runs the only block produce on the Optimism network.
      • [3] Top 10 L2s are documented has having a centralized operator.
      ]]> +Waku is already used in production.
      • +
    • [2] The Optimism Foundation runs the only block produce on the Optimism network.
      • +
    • [3] Top 10 L2s are documented has having a centralized operator.
      • +
    ]]>     <![CDATA[Building Privacy-Protecting Infrastructure]]> @@ -2035,202 +5151,298 @@ Waku is already used in production.
  • here

    Intro​

    In this write-up, we are going to talk about building privacy-protecting -infrastructure. What is it, why do we need it and how can we build it?

    We'll look at Waku, the communication layer for Web3. We'll look at how we are + What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it.

    + +

    This write-up is based on a talk given at DevCon 6 in Bogota, a video can be found here

    +

    Intro​

    +

    In this write-up, we are going to talk about building privacy-protecting +infrastructure. What is it, why do we need it and how can we build it?

    +

    We'll look at Waku, the communication layer for Web3. We'll look at how we are using Zero Knowledge (ZK) technology to incentivize and protect the Waku network. We'll also look at Zerokit, a library we are writing to make ZKP easier -to use in different environments.

    At the end of this write-up, I hope you'll come away with an understanding of -the importance of privacy-protecting infrastructure and how we can build it.

    About​

    First, briefly about Vac. We build public good protocols for the decentralized +to use in different environments.

    +

    At the end of this write-up, I hope you'll come away with an understanding of +the importance of privacy-protecting infrastructure and how we can build it.

    +

    About​

    +

    First, briefly about Vac. We build public good protocols for the decentralized web, with a focus on privacy and communication. We do applied research based on which we build protocols, libraries and publications. We are also the custodians -of protocols that reflect a set of principles.

    Principles

    It has its origins in the Status app and trying to improve +of protocols that reflect a set of principles.

    +

    Principles

    +

    It has its origins in the Status app and trying to improve the underlying protocols and infrastructure. We build Waku, -among other things.

    Why build privacy-protecting infrastructure?​

    Privacy is the power to selectively reveal yourself. It is a requirement for -freedom and self-determination.

    Just like you need decentralization in order to get censorship-resistance, you -need privacy to enable freedom of expression.

    To build applications that are decentralized and privacy-protecting, you need -the base layer, the infrastructure itself, to have those properties.

    We see this a lot. It is easier to make trade-offs at the application layer than +among other things.

    +

    Why build privacy-protecting infrastructure?​

    +

    Privacy is the power to selectively reveal yourself. It is a requirement for +freedom and self-determination.

    +

    Just like you need decentralization in order to get censorship-resistance, you +need privacy to enable freedom of expression.

    +

    To build applications that are decentralized and privacy-protecting, you need +the base layer, the infrastructure itself, to have those properties.

    +

    We see this a lot. It is easier to make trade-offs at the application layer than doing them at the base layer. You can build custodial solutions on top of a decentralized and non-custodial network where participants control their own -keys, but you can't do the opposite.

    If you think about it, buildings can be seen as a form of privacy-protecting +keys, but you can't do the opposite.

    +

    If you think about it, buildings can be seen as a form of privacy-protecting infrastructure. It is completely normal and obvious in many ways, but when it comes to the digital realm our mental models and way of speaking about it hasn't -caught up yet for most people.

    I'm not going too much more into the need for privacy or what happens when you +caught up yet for most people.

    +

    I'm not going too much more into the need for privacy or what happens when you don't have it, but suffice to say it is an important property for any open -society.

    When we have conversations, true peer-to-peer offline conversations, we can talk -privately. If we use cash to buy things we can do commerce privately.

    On the Internet, great as it is, there are a lot of forces that makes this +society.

    +

    When we have conversations, true peer-to-peer offline conversations, we can talk +privately. If we use cash to buy things we can do commerce privately.

    +

    On the Internet, great as it is, there are a lot of forces that makes this natural state of things not the default. Big Tech has turned users into a commodity, a product, and monetized user's attention for advertising. To optimize for your attention they need to surveil your habits and activities, and hence breach your privacy. As opposed to more old-fashioned models, where someone is buying a useful service from a company and the incentives are more -aligned.

    We need to build credibly neutral infrastructure that protects your privacy at +aligned.

    +

    We need to build credibly neutral infrastructure that protects your privacy at the base layer, in order to truly enable applications that are -censorship-resistant and encourage meaningful freedom of expression.

    Web3 infrastructure​

    Infrastructure is what lies underneath. Many ways of looking at this but I'll +censorship-resistant and encourage meaningful freedom of expression.

    +

    Web3 infrastructure​

    +

    Infrastructure is what lies underneath. Many ways of looking at this but I'll keep it simple as per the original Web3 vision. You had Ethereum for compute/consensus, Swarm for storage, and Whisper for messaging. Waku has taken over the mantle from Whisper and is a lot more usable today than Whisper ever was, -for many reasons.

    Web3 Infrastructure

    On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, +for many reasons.

    +

    Web3 Infrastructure

    +

    On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, especially when you try to add privacy back "on top". It takes a lot of effort and it is easier to censor. We see this with recent action around Tornado Cash. Compare this with something like Zcash or Monero, where privacy is there by -default.

    There are also problems when it comes to the p2p networking side of things, for +default.

    +

    There are also problems when it comes to the p2p networking side of things, for example with Ethereum validator privacy and hostile actors and jurisdictions. If someone can easily find out where a certain validator is physically located, that's a problem in many parts of the world. Being able to have stronger -privacy-protection guarantees would be very useful for high-value targets.

    This doesn't begin to touch on the so called "dapps" that make a lot of +privacy-protection guarantees would be very useful for high-value targets.

    +

    This doesn't begin to touch on the so called "dapps" that make a lot of sacrifices in how they function, from the way domains work, to how websites are hosted and the reliance on centralized services for communication. We see this time and time again, where centralized, single points of failure systems work -for a while, but then eventually fail.

    In many cases an individual user might not care enough though, and for platforms +for a while, but then eventually fail.

    +

    In many cases an individual user might not care enough though, and for platforms the lure to take shortcuts is strong. That is why it is important to be principled, but also pragmatic in terms of the trade-offs that you allow on top. -We'll touch more on this in the design goals around modularity that Waku has.

    ZK for privacy-protecting infrastructure​

    ZKPs are a wonderful new tool. Just like smart contracts enables programmable +We'll touch more on this in the design goals around modularity that Waku has.

    +

    ZK for privacy-protecting infrastructure​

    +

    ZKPs are a wonderful new tool. Just like smart contracts enables programmable money, ZKPs allow us to express fundamentally new things. In line with the great tradition of trust-minimization, we can prove statement while revealing the absolute minimum information necessary. This fits the definition of privacy, the power to selectively reveal yourself, perfectly. I'm sure I don't need to tell anyone reading this but this is truly revolutionary. The technology is advancing -extremely fast and often it is our imagination that is the limit.

    Zero knowledge

    Waku​

    What is Waku? It is a set of modular protocols for p2p communication. It has a +extremely fast and often it is our imagination that is the limit.

    +

    Zero knowledge

    +

    Waku​

    +

    What is Waku? It is a set of modular protocols for p2p communication. It has a focus on privacy, security and being able to run anywhere. It is the spiritual -success to Whisper.

    By modular we mean that you can pick and choose protocols and how you use them +success to Whisper.

    +

    By modular we mean that you can pick and choose protocols and how you use them depending on constraints and trade-offs. For example, bandwidth usage vs -privacy.

    It is designed to work in resource restricted environments, such as mobile +privacy.

    +

    It is designed to work in resource restricted environments, such as mobile phones and in web browsers. It is important that infrastructure meets users where they are and supports their real-world use cases. Just like you don't need your own army and a castle to have your own private bathroom, you shouldn't need to have a powerful always-on node to get reasonable privacy and -censorship-resistance. We might call this self-sovereignty.

    Waku - adaptive nodes​

    One way of looking at Waku is as an open service network. There are nodes with +censorship-resistance. We might call this self-sovereignty.

    +

    Waku - adaptive nodes​

    +

    One way of looking at Waku is as an open service network. There are nodes with varying degrees of capabilities and requirements. For example when it comes to bandwidth usage, storage, uptime, privacy requirements, latency requirements, -and connectivity restrictions.

    We have a concept of adaptive nodes that can run a variety of protocols. A node +and connectivity restrictions.

    +

    We have a concept of adaptive nodes that can run a variety of protocols. A node operator can choose which protocols they want to run. Naturally, there'll be some nodes that do more consumption and other nodes that do more provisioning. This gives rise to the idea of a service network, where services are provided -for and consumed.

    Adaptive Nodes

    Waku - protocol interactions​

    There are many protocols that interact. Waku Relay protocol is based on libp2p +for and consumed.

    +

    Adaptive Nodes

    +

    Waku - protocol interactions​

    +

    There are many protocols that interact. Waku Relay protocol is based on libp2p GossipSub for p2p messaging. We have filter for bandwidth-restricted nodes to only receive subset of messages. Lightpush for nodes with short connection windows to push messages into network. Store for nodes that want to retrieve -historical messages.

    On the payload layer, we provide support for Noise handshakes/key-exchanges. +historical messages.

    +

    On the payload layer, we provide support for Noise handshakes/key-exchanges. This means that as a developers, you can get end-to-end encryption and expected guarantees out of the box. We have support for setting up a secure channel from scratch, and all of this paves the way for providing Signal's Double Ratchet at the protocol level much easier. We also have experimental support for multi-device usage. Similar features have existed in for example the Status app for a while, but with this we make it easier for any platform using Waku to use -it.

    There are other protocols too, related to peer discovery, topic usage, etc. See -specs for more details.

    Protocol Interactions

    Waku - Network​

    For the Waku network, there are a few problems. For example, when it comes to +it.

    +

    There are other protocols too, related to peer discovery, topic usage, etc. See +specs for more details.

    +Protocol Interactions +

    Waku - Network​

    +

    For the Waku network, there are a few problems. For example, when it comes to network spam and incentivizing service nodes. We want to address these while keeping privacy-guarantees of the base layer. I'm going to go into both of -these.

    The spam problem arises on the gossip layer when anyone can overwhelm the +these.

    +

    The spam problem arises on the gossip layer when anyone can overwhelm the network with messages. The service incentivization is a problem when nodes don't directly benefit from the provisioning of a certain service. This can happen if they are not using the protocol directly themselves as part of normal operation, or if they aren't socially inclined to provide a certain service. This depends a -lot on how an individual platform decides to use the network.

    Waku Network

    Dealing with network spam and RLN Relay​

    Since the p2p relay network is open to anyone, there is a problem with spam. If +lot on how an individual platform decides to use the network.

    +

    Waku Network

    +

    Dealing with network spam and RLN Relay​

    +

    Since the p2p relay network is open to anyone, there is a problem with spam. If we look at existing solutions for dealing with spam in traditional messaging systems, a lot of entities like Google, Facebook, Twitter, Telegram, Discord use phone number verification. While this is largely sybil-resistant, it is -centralized and not private at all.

    Historically, Whisper used PoW which isn't good for heterogenerous networks. +centralized and not private at all.

    +

    Historically, Whisper used PoW which isn't good for heterogenerous networks. Peer scoring is open to sybil attacks and doesn't directly address spam -protection in an anonymous p2p network.

    The key idea here is to use RLN for private economic spam protection using -zkSNARKs.

    I'm not going to go into too much detail of RLN here. If you are interested, I +protection in an anonymous p2p network.

    +

    The key idea here is to use RLN for private economic spam protection using +zkSNARKs.

    +

    I'm not going to go into too much detail of RLN here. If you are interested, I gave a talk in Amsterdam at Devconnect about this. We have some write-ups on RLN here by Sanaz who has been pushing a lot of this from our side. There's also another talk at Devcon by Tyler going into RLN in -more detail. Finally, here's the RLN spec.

    I'll briefly go over what it is, the interface and circuit and then talk about -how it is used in Waku.

    RLN - Overview and Flow​

    RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting +more detail. Finally, here's the RLN spec.

    +

    I'll briefly go over what it is, the interface and circuit and then talk about +how it is used in Waku.

    +

    RLN - Overview and Flow​

    +

    RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting mechanism based on zkSNARKs. By rate limiting we mean you can only send N messages in a given period. By anonymity we mean that you can't link message to a publisher. We can think of it as a voting booth, where you are only allowed to -vote once every election.

    Voting Booth

    It can be used for spam protection in p2p messaging systems, and also rate -limiting in general, such as for a decentralized captcha.

    There are three parts to it. You register somewhere, then you can signal and +vote once every election.

    +

    Voting Booth

    +

    It can be used for spam protection in p2p messaging systems, and also rate +limiting in general, such as for a decentralized captcha.

    +

    There are three parts to it. You register somewhere, then you can signal and finally there's a verification/slashing phase. You put some capital at risk, -either economic or social, and if you double signal you get slashed.

    RLN - Circuit​

    Here's what the private and public inputs to the circuit look like. The identity +either economic or social, and if you double signal you get slashed.

    +

    RLN - Circuit​

    +

    Here's what the private and public inputs to the circuit look like. The identity secret is generated locally, and we create an identity commitment that is inserted into a Merkle tree. We then use Merkle proofs to prove membership. Registered member can only signal once for a given epoch or external nullifier, for example every ten seconds in Unix time. RLN identifer is for a specific RLN -app.

    We also see what the circuit output looks like. This is calculated locally. y +app.

    +

    We also see what the circuit output looks like. This is calculated locally. y is a share of the secret equation, and the (internal) nullifier acts as a unique fingerprint for a given app/user/epoch combination. How do we calculate y and -the internal nullifier?

    // Private input
    signal input identity_secret;
    signal input path_elements[n_levels][1];
    signal input identity_path_index[n_levels];

    // Public input
    signal input x; // signal_hash
    signal input epoch; // external_nullifier
    signal input rln_identifier;

    // Circuit output
    signal output y;
    signal output root;
    signal output nullifier;

    RLN - Shamir's secret sharing​

    This is done using Shamir's secret +the internal nullifier?

    +
    // Private input
    signal input identity_secret;
    signal input path_elements[n_levels][1];
    signal input identity_path_index[n_levels];

    // Public input
    signal input x; // signal_hash
    signal input epoch; // external_nullifier
    signal input rln_identifier;

    // Circuit output
    signal output y;
    signal output root;
    signal output nullifier;
    +

    RLN - Shamir's secret sharing​

    +

    This is done using Shamir's secret sharing. Shamir’s secret sharing is based on idea of splitting a secret into shares. This is how -we enable slashing of funds.

    In this case, we have two shares. If a given identity a0 signals twice in +we enable slashing of funds.

    +

    In this case, we have two shares. If a given identity a0 signals twice in epoch/external nullifier, a1 is the same. For a given RLN app, internal_nullifier then stays the same. x is signal hash which is different, and y is public, so we can reconstruct identity_secret. With the identity -secret revealed, this gives access to e.g. financial stake.

    a_0 = identity_secret // secret S
    a_1 = poseidonHash([a0, external_nullifier])

    y = a_0 + x * a_1

    internal_nullifier = poseidonHash([a_1, rln_identifier])

    Shamir&#39;s secret sharing

    RLN Relay​

    This is how RLN is used with Relay/GossipSub protocol. A node registers and +secret revealed, this gives access to e.g. financial stake.

    +
    a_0 = identity_secret // secret S
    a_1 = poseidonHash([a0, external_nullifier])

    y = a_0 + x * a_1

    internal_nullifier = poseidonHash([a_1, rln_identifier])
    +

    Shamir&#39;s secret sharing

    +

    RLN Relay​

    +

    This is how RLN is used with Relay/GossipSub protocol. A node registers and locks up funds, and after that it can send messages. It publishes a message -containing the Zero Knowledge proof and some other details.

    Each relayer node listens to the membership contract for new members, and it +containing the Zero Knowledge proof and some other details.

    +

    Each relayer node listens to the membership contract for new members, and it also keeps track of relevant metadata and merkle tree. Metadata is needed to be -able to detect double signaling and perform slashing.

    Before forwarding a message, it does some verification checks to ensure there +able to detect double signaling and perform slashing.

    +

    Before forwarding a message, it does some verification checks to ensure there are no duplicate messages, ZKP is valid and no double signaling has occured. It is worth noting that this can be combined with peer scoring, for example for -duplicate messages or invalid ZK proofs.

    In line of Waku's goals of modularity, RLN Relay is applied on a specific subset -of pubsub and content topics. You can think of it as an extra secure channel.

    RLN Relay

    RLN Relay cross-client testnet​

    Where are we with RLN Relay deployment? We've recently launched our second +duplicate messages or invalid ZK proofs.

    +

    In line of Waku's goals of modularity, RLN Relay is applied on a specific subset +of pubsub and content topics. You can think of it as an extra secure channel.

    +

    RLN Relay

    +

    RLN Relay cross-client testnet​

    +

    Where are we with RLN Relay deployment? We've recently launched our second testnet. This is using RLN Relay with a smart contract on Goerli. It integrates with our example p2p chat application, and it does so through three different clients, nwaku, go-waku and js-waku for browsers. This is our first p2p -cross-client testnet for RLN Relay.

    Here's a video that shows a user +cross-client testnet for RLN Relay.

    +

    Here's a video that shows a user registering in a browser, signaling through JS-Waku. It then gets relayed to a nwaku node, that verifies the proof. The second video shows what happens in the spam case. when more than one message is sent in a given epoch, it detects it as spam and discards it. Slashing hasn't been implemented fully yet in the client -and is a work in progress.

    If you are curious and want to participate, you can join the effort on our Vac +and is a work in progress.

    +

    If you are curious and want to participate, you can join the effort on our Vac Discord. We also have tutorials -setup for all clients so you can play around with it.

    As part of this, and to make it work in multiple different environments, we've +setup for all clients so you can play around with it.

    +

    As part of this, and to make it work in multiple different environments, we've also been developing a new library called Zerokit. I'll talk about this a bit -later.

    Private settlement / Service credentials​

    Going back to the service network idea, let's talk about service credentials. +later.

    +

    Private settlement / Service credentials​

    +

    Going back to the service network idea, let's talk about service credentials. The idea behind service credentials and private settlement is to enable two actors to pay for and provide services without compromising their privacy. We do not want the payment to create a direct public link between the service provider -and requester.

    Recall the Waku service network illustration with adaptive nodes that choose +and requester.

    +

    Recall the Waku service network illustration with adaptive nodes that choose which protocols they want to run. Many of these protocols aren't very heavy and just work by default. For example the relay protocol is enabled by default. Other protocols are much heavier to provide, such as storing historical -messages.

    It is desirable to have additional incentives for this, especially for platforms +messages.

    +

    It is desirable to have additional incentives for this, especially for platforms that aren't community-based where some level of altruism can be assumed (e.g. -Status Communities, or WalletConnect cloud infrastructure).

    You have a node Alice that is often offline and wants to consume historical +Status Communities, or WalletConnect cloud infrastructure).

    +

    You have a node Alice that is often offline and wants to consume historical messages on some specific content topics. You have another node Bob that runs a server at home where they store historical messages for the last several weeks. Bob is happy to provide this service for free because he's excited about running privacy-preserving infrastructure and he's using it himself, but his node is getting overwhelmed by freeloaders and he feels like he should be paid something -for continuing to provide this service.

    Alice deposits some funds in a smart contract which registers it in a tree, +for continuing to provide this service.

    +

    Alice deposits some funds in a smart contract which registers it in a tree, similar to certain other private settlement mechanisms. A fee is taken or burned. In exchange, she gets a set of tokens or service credentials. When she wants to do a query with some criteria, she sends this to Bob. Bob responds with size of response, cost, and receiver address. Alice then sends a proof of delegation of a service token as a payment. Bob verifies the proof and resolves -the query.

    The end result is that Alice has consumed some service from Bob, and Bob has +the query.

    +

    The end result is that Alice has consumed some service from Bob, and Bob has received payment for this. There's no direct transaction link between Alice and Bob, and gas fees can be minimized by extending the period before settling on -chain.

    This can be complemented with altruistic service provisioning, for example by +chain.

    +

    This can be complemented with altruistic service provisioning, for example by splitting the peer pool into two slots, or only providing a few cheap queries -for free.

    The service provisioning is general, and can be generalized for any kind of -request/response service provisoning that we want to keep private.

    This isn't a perfect solution, but it is an incremental improvement on top of +for free.

    +

    The service provisioning is general, and can be generalized for any kind of +request/response service provisoning that we want to keep private.

    +

    This isn't a perfect solution, but it is an incremental improvement on top of the status quo. It can be augmented with more advanced techniques such as better -non-repudiable node reputation, proof of correct service provisioning, etc.

    We are currently in the raw spec / proof of concept stage of this. We expect to -launch a testnet of this later this year or early next year.

    Service credentials flow

    Zerokit​

    Zerokit is a set of Zero Knowledge modules, +non-repudiable node reputation, proof of correct service provisioning, etc.

    +

    We are currently in the raw spec / proof of concept stage of this. We expect to +launch a testnet of this later this year or early next year.

    +

    Service credentials flow

    +

    Zerokit​

    +

    Zerokit is a set of Zero Knowledge modules, written in Rust and designed to be used in many different environments. The initial goal is to get the best of both worlds with Circom/Solidity/JS and Rust/ZK ecosystem. This enables people to leverage Circom-based constructs from -non-JS environments.

    For the RLN module, it is using Circom circuits via ark-circom and Rust for +non-JS environments.

    +

    For the RLN module, it is using Circom circuits via ark-circom and Rust for scaffolding. It exposes a C FFI API that can be used through other system programming environments, like Nim and Go. It also exposes an experimental WASM -API that can be used through web browsers.

    Waku is p2p infrastructure running in many different environments, such as -Nim/JS/Go/Rust, so this a requirement for us.

    Circom and JS strengths are access to Dapp developers, tooling, generating +API that can be used through web browsers.

    +

    Waku is p2p infrastructure running in many different environments, such as +Nim/JS/Go/Rust, so this a requirement for us.

    +

    Circom and JS strengths are access to Dapp developers, tooling, generating verification code, circuits etc. Rust strengths is that it is systems-based and easy to interface with other language runtime such as Nim, Go, Rust, C. It also gives access to other Rust ZK ecosystems such as arkworks. This opens door for using other constructs, such as Halo2. This becomes especially relevant for constructs where you don't want to do a trusted setup or where circuits are more -complex/custom and performance requirements are higher.

    In general with Zerokit, we want to make it easy to build and use ZKP in a +complex/custom and performance requirements are higher.

    +

    In general with Zerokit, we want to make it easy to build and use ZKP in a multitude of environments, such as mobile phones and web browsers. Currently it is too complex to write privacy-protecting infrastructure with ZKPs considering all the languages and tools you have to learn, from JS, Solidity and Circom to @@ -2238,23 +5450,29 @@ Rust, WASM and FFI. And that isn't even touching on things like secure key storage or mobile dev. Luckily more and more projects are working on this, including writing DSLs etc. It'd also be exciting if we can make a useful toolstack for JS-less ZK dev to reduce cognitive overhead, similar to what we -have with something like Foundry.

    Other research​

    I also want to mention a few other things we are doing. One thing is +have with something like Foundry.

    +

    Other research​

    +

    I also want to mention a few other things we are doing. One thing is protocol specifications. We think this is very important for p2p infra, and we see a lot of other projects that claim to do it p2p infrastructure but they aren't clear about guarantees or how stable something is. That makes it hard to have multiple implementations, to collaborate across -different projects, and to analyze things objectively.

    Related to that is publishing papers. We've put +different projects, and to analyze things objectively.

    +

    Related to that is publishing papers. We've put out three so far, related to Waku and RLN-Relay. This makes it easier to interface with academia. There's a lot of good researchers out there and we want -to build a better bridge between academia and industry.

    Another thing is network +to build a better bridge between academia and industry.

    +

    Another thing is network privacy. Waku is modular with respect to privacy guarantees, and there are a lot of knobs to turn here depending on specific deployments. For example, if you are running the full relay protocol you currently have much stronger receiver anonymity than if you -are running filter protocol from a bandwidth or connectivity-restricted node.

    We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym +are running filter protocol from a bandwidth or connectivity-restricted node.

    +

    We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym come with some trade-offs but are a useful tool in the arsenal. A good mental model to keep in mind is the anonymity trilemma, where you can only pick 2/3 out -of low latency, low bandwidth usage and strong anonymity.

    We are currently exploring Dandelion-like +of low latency, low bandwidth usage and strong anonymity.

    +

    We are currently exploring Dandelion-like additions to the relay/gossip protocol, which would provide for stronger sender anonymity, especially in a multi-node/botnet attacker model. As part of this we are looking into different @@ -2262,18 +5480,24 @@ parameters choices and general possibilities for lower latency usage. This could make it more amenable for latency sensitive environments, such as validator privacy, for specific threat models. The general theme here is we want to be rigorous with the guarantees we provide, under what conditions and for what -threat models.

    Another thing mentioned earlier is Noise payload +threat models.

    +

    Another thing mentioned earlier is Noise payload encryption, and specifically things like allowing for pairing different devices with e.g. QR codes. This makes it easier for developers to provide secure messaging in many realistic scenarios in a -multi-device world.

    Other research

    Summary​

    We've gone over what privacy-protecting infrastructure is, why we want it and +multi-device world.

    +

    Other research

    +

    Summary​

    +

    We've gone over what privacy-protecting infrastructure is, why we want it and how we can build it. We've seen how ZK is a fundamental building block for this. We've looked at Waku, the communication layer for Web3, and how it uses Zero Knowledge proofs to stay private and function better. We've also looked at -Zerokit and how we can make it easier to do ZKP in different environments.

    Finally we also looked at some other research we've been doing. All of the +Zerokit and how we can make it easier to do ZKP in different environments.

    +

    Finally we also looked at some other research we've been doing. All of the things mentioned in this article, and more, is available as write-ups, specs, or -discussions on our forum or Github.

    If you find any of this exciting to work on, feel free to reach out on our +discussions on our forum or Github.

    +

    If you find any of this exciting to work on, feel free to reach out on our Discord. We are also hiring, and we have started expanding into other privacy infrastructure tech like private and provable computation with ZK-WASM.

    ]]> @@ -2284,1006 +5508,281 @@ computation with ZK-WASM.

    ]]> https://vac.dev/rlog/wakuv2-relay-anon Fri, 22 Jul 2022 10:00:00 GMT - Introducing a basic threat model and privacy/anonymity analysis for the Waku v2 relay protocol.

    Waku v2 enables secure, privacy preserving communication using a set of modular P2P protocols. + Introducing a basic threat model and privacy/anonymity analysis for the Waku v2 relay protocol.

    + +

    Waku v2 enables secure, privacy preserving communication using a set of modular P2P protocols. Waku v2 also aims at protecting the user's anonymity. This post is the first in a series about Waku v2 security, privacy, and anonymity. The goal is to eventually have a full privacy and anonymity analysis for each of the Waku v2 protocols, as well as covering the interactions of various Waku v2 protocols. -This provides transparency with respect to Waku's current privacy and anonymity guarantees, and also identifies weak points that we have to address.

    In this post, we first give an informal description of security, privacy and anonymity in the context of Waku v2. +This provides transparency with respect to Waku's current privacy and anonymity guarantees, and also identifies weak points that we have to address.

    +

    In this post, we first give an informal description of security, privacy and anonymity in the context of Waku v2. For each definition, we summarize Waku's current guarantees regarding the respective property. -We also provide attacker models, an attack-based threat model, and a first anonymity analysis of Waku v2 relay within the respective models.

    Waku comprises many protocols that can be combined in a modular way. +We also provide attacker models, an attack-based threat model, and a first anonymity analysis of Waku v2 relay within the respective models.

    +

    Waku comprises many protocols that can be combined in a modular way. For our privacy and anonymity analysis, we start with the relay protocol because it is at the core of Waku v2 enabling Waku's publish subscribe approach to P2P messaging. -In its current form, Waku relay is a minor extension of libp2p GossipSub.

    Figure 1: The Waku v2 relay mesh is based on the [GossipSub mesh](https://docs.libp2p.io/concepts/publish-subscribe#types-of-peering)

    Informal Definitions: Security, Privacy, and Anonymity​

    The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.

    Security​

    Of the three, Security has the clearest agreed upon definition, -at least regarding its key concepts: confidentiality, integrity, and availability.

    • confidentiality: data is not disclosed to unauthorized entities.
    • integrity: data is not modified by unauthorized entities.
    • availability: data is available, i.e. accessible by authorized entities.

    While these are the key concepts, the definition of information security has been extended over time including further concepts, +In its current form, Waku relay is a minor extension of libp2p GossipSub.

    +

    Figure 1: The Waku v2 relay mesh is based on the GossipSub mesh

    +

    Informal Definitions: Security, Privacy, and Anonymity​

    +

    The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.

    +

    Security​

    +

    Of the three, Security has the clearest agreed upon definition, +at least regarding its key concepts: confidentiality, integrity, and availability.

    +
      +
    • confidentiality: data is not disclosed to unauthorized entities.
      • +
    • integrity: data is not modified by unauthorized entities.
      • +
    • availability: data is available, i.e. accessible by authorized entities.
      • +
    +

    While these are the key concepts, the definition of information security has been extended over time including further concepts, e.g. authentication and non-repudiation. -We might cover these in future posts.

    Privacy​

    Privacy allows users to choose which data and information

    • they want to share
    • and with whom they want to share it.

    This includes data and information that is associated with and/or generated by users. +We might cover these in future posts.

    +

    Privacy​

    +

    Privacy allows users to choose which data and information

    +
      +
    • they want to share
      • +
    • and with whom they want to share it.
      • +
    +

    This includes data and information that is associated with and/or generated by users. Protected data also comprises metadata that might be generated without users being aware of it. This means, no further information about the sender or the message is leaked. Metadata that is protected as part of the privacy-preserving property does not cover protecting the identities of sender and receiver. -Identities are protected by the anonymity property.

    Often privacy is realized by the confidentiality property of security. +Identities are protected by the anonymity property.

    +

    Often privacy is realized by the confidentiality property of security. This neither makes privacy and security the same, nor the one a sub category of the other. While security is abstract itself (its properties can be realized in various ways), privacy lives on a more abstract level using security properties. Privacy typically does not use integrity and availability. -An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.

    Waku offers confidentiality via secure channels set up with the help of the Noise Protocol Framework. +An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.

    +

    Waku offers confidentiality via secure channels set up with the help of the Noise Protocol Framework. Using these secure channels, message content is only disclosed to the intended receivers. They also provide good metadata protection properties. However, we do not have a metadata protection analysis as of yet, -which is part of our privacy/anonymity roadmap.

    Anonymity​

    Privacy and anonymity are closely linked. +which is part of our privacy/anonymity roadmap.

    +

    Anonymity​

    +

    Privacy and anonymity are closely linked. Both the identity of a user and data that allows inferring a user's identity should be part of the privacy policy. -For the purpose of analysis, we want to have a clearer separation between these concepts.

    We define anonymity as unlinkablity of users' identities and their shared data and/or actions.

    We subdivide anonymity into receiver anonymity and sender anonymity.

    Receiver Anonymity​

    We define receiver anonymity as unlinkability of users' identities and the data they receive and/or related actions. +For the purpose of analysis, we want to have a clearer separation between these concepts.

    +

    We define anonymity as unlinkablity of users' identities and their shared data and/or actions.

    +

    We subdivide anonymity into receiver anonymity and sender anonymity.

    +

    Receiver Anonymity​

    +

    We define receiver anonymity as unlinkability of users' identities and the data they receive and/or related actions. The data transmitted via Waku relay must be a Waku message, which contains a content topic field. Because each message is associated with a content topic, and each receiver is interested in messages with specific content topics, receiver anonymity in the context of Waku corresponds to subscriber-topic unlinkability. -An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.

    The Waku message's content topic is not related to the libp2p pubsub topic. +An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.

    +

    The Waku message's content topic is not related to the libp2p pubsub topic. For now, Waku uses a single libp2p pubsub topic, which means messages are propagated via a single mesh of peers. With this, the receiver discloses its participation in Waku on the gossipsub layer. -We will leave the analysis of libp2p gossipsub to a future article within this series, and only provide a few hints and pointers here.

    Waku offers k-anonymity regarding content topic interest in the global adversary model. -K-anonymity in the context of Waku means an attacker can link receivers to content topics with a maximum certainty of 1/k1/k. -The larger kk, the less certainty the attacker gains. -Receivers basically hide in a pool of kk content topics, any subset of which could be topics they subscribed to. +We will leave the analysis of libp2p gossipsub to a future article within this series, and only provide a few hints and pointers here.

    +

    Waku offers k-anonymity regarding content topic interest in the global adversary model. +K-anonymity in the context of Waku means an attacker can link receivers to content topics with a maximum certainty of 1/k1/k. +The larger kk, the less certainty the attacker gains. +Receivers basically hide in a pool of kk content topics, any subset of which could be topics they subscribed to. The attacker does not know which of those the receiver actually subscribed to, and the receiver enjoys plausible deniability regarding content topic subscription. -Assuming there are nn Waku content topics, a receiver has nn-anonymity with respect to association to a specific content topic.

    Technically, Waku allows distributing messages over several libp2p pubsub topics. -This yields kk-anonymity, assuming kk content topics share the same pubsub topic. +Assuming there are nn Waku content topics, a receiver has nn-anonymity with respect to association to a specific content topic.

    +

    Technically, Waku allows distributing messages over several libp2p pubsub topics. +This yields kk-anonymity, assuming kk content topics share the same pubsub topic. However, if done wrongly, such sharding of pubsub topics can breach anonymity. -A formal specification of anonymity-preserving topic sharding building on the concepts of partitioned topics is part of our roadmap.

    Also, Waku is not directly concerned with 1:1 communication, so for this post, 1:1 communication is out of scope. +A formal specification of anonymity-preserving topic sharding building on the concepts of partitioned topics is part of our roadmap.

    +

    Also, Waku is not directly concerned with 1:1 communication, so for this post, 1:1 communication is out of scope. Channels for 1:1 communication can be implemented on top of Waku relay. In the future, a 1:1 communication protocol might be added to Waku. -Similar to topic sharding, it would maintain receiver anonymity leveraging partitioned topics.

    Sender Anonymity​

    We define sender anonymity as unlinkability of users' identities and the data they send and/or related actions. -Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.

    In summary, Waku offers weak sender anonymity because of Waku's strict no sign policy, +Similar to topic sharding, it would maintain receiver anonymity leveraging partitioned topics.

    +

    Sender Anonymity​

    +

    We define sender anonymity as unlinkability of users' identities and the data they send and/or related actions. +Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.

    +

    In summary, Waku offers weak sender anonymity because of Waku's strict no sign policy, which has its origins in the Ethereum consensus specs. -17/WAKU-RLN-RELAY and 18/WAKU2-SWAP mitigate replay and injection attacks.

    Waku currently does not offer sender anonymity in stronger attacker models, as well as cannot protect against targeted attacks in weaker attacker models like the single or multi node attacker. -We will cover this in more detail in later sections.

    Anonymity Trilemma​

    The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global on-net attacker model. +17/WAKU-RLN-RELAY and 18/WAKU2-SWAP mitigate replay and injection attacks.

    +

    Waku currently does not offer sender anonymity in stronger attacker models, as well as cannot protect against targeted attacks in weaker attacker models like the single or multi node attacker. +We will cover this in more detail in later sections.

    +

    Anonymity Trilemma​

    +

    The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global on-net attacker model. Waku's goal, being a modular set of protocols, is to offer any combination of two out of these three properties, as well as blends. -An example for blending is an adjustable number of pubsub topics and peers in the respective pubsub topic mesh; this allows tuning the trade-off between anonymity and bandwidth.

    Figure 2: Anonymity Trilemma: pick two.

    A fourth factor that influences the anonymity trilemma is frequency and patterns of messages. +An example for blending is an adjustable number of pubsub topics and peers in the respective pubsub topic mesh; this allows tuning the trade-off between anonymity and bandwidth.

    +

    Figure 2: Anonymity Trilemma: pick two.

    +

    A fourth factor that influences the anonymity trilemma is frequency and patterns of messages. The more messages there are, and the more randomly distributed they are, the better the anonymity protection offered by a given anonymous communication protocol. So, incentivising users to use the protocol, for instance by lowering entry barriers, helps protecting the anonymity of all users. -The frequency/patterns factor is also related to the above described k-anonymity.

    Censorship Resistance​

    Another security related property that Waku aims to offer is censorship resistance. +The frequency/patterns factor is also related to the above described k-anonymity.

    +

    Censorship Resistance​

    +

    Another security related property that Waku aims to offer is censorship resistance. Censorship resistance guarantees that users can participate even if an attacker tries to deny them access. So, censorship resistance ties into the availability aspect of security. In the context of Waku that means users should be able to send messages as well as receive all messages they are interested in, -even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.

    Currently, Waku only guarantees censorship resistance in the weak single node attacker model. +even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.

    +

    Currently, Waku only guarantees censorship resistance in the weak single node attacker model. While currently employed secure channels mitigate targeted censorship, e.g. blocking specific content topics, general censorship resistance in strong attacker models is part of our roadmap. -Among other options, we will investigate Pluggable Transports in future articles.

    Attacker Types​

    The following lists various attacker types with varying degrees of power. -The more power an attacker has, the more difficult it is to gain the respective attacker position.

    Each attacker type comes in a passive and an active variant. +Among other options, we will investigate Pluggable Transports in future articles.

    +

    Attacker Types​

    +

    The following lists various attacker types with varying degrees of power. +The more power an attacker has, the more difficult it is to gain the respective attacker position.

    +

    Each attacker type comes in a passive and an active variant. While a passive attacker can stay hidden and is not suspicious, -the respective active attacker has more (or at least the same) deanonymization power.

    We also distinguish between internal and external attackers.

    Internal​

    With respect to Waku relay, an internal attacker participates in the same pubsub topic as its victims. -Without additional measures on higher layer protocols, access to an internal position is easy to get.

    Single Node​

    This attacker controls a single node. -Because this position corresponds to normal usage of Waku relay, it is trivial to obtain.

    Multi Node​

    This attacker controls several nodes. We assume a smaller static number of controlled nodes. +the respective active attacker has more (or at least the same) deanonymization power.

    +

    We also distinguish between internal and external attackers.

    +

    Internal​

    +

    With respect to Waku relay, an internal attacker participates in the same pubsub topic as its victims. +Without additional measures on higher layer protocols, access to an internal position is easy to get.

    +

    Single Node​

    +

    This attacker controls a single node. +Because this position corresponds to normal usage of Waku relay, it is trivial to obtain.

    +

    Multi Node​

    +

    This attacker controls several nodes. We assume a smaller static number of controlled nodes. The multi node position can be achieved relatively easily by setting up multiple nodes. Botnets might be leveraged to increase the number of available hosts. Multi node attackers could use Sybil attacks to increase the number of controlled nodes. -A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.

    Linearly Scaling Nodes​

    This attacker controls a number of nodes that scales linearly with the number of nodes in the network. +A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.

    +

    Linearly Scaling Nodes​

    +

    This attacker controls a number of nodes that scales linearly with the number of nodes in the network. This attacker is especially interesting to investigate in the context of DHT security, -which Waku uses for ambient peer discovery.

    External​

    An external attacker can only see encrypted traffic (protected by a secure channel set up with WAKU2-NOISE). +which Waku uses for ambient peer discovery.

    +

    External​

    +

    An external attacker can only see encrypted traffic (protected by a secure channel set up with WAKU2-NOISE). Because an internal position can be easily obtained, in practice external attackers would mount combined attacks that leverage both internal an external attacks. -We cover this more below when describing attacks.

    Local​

    A local attacker has access to communication links in a local network segment. -This could be a rogue access point (with routing capability).

    AS​

    An AS attacker controls a single AS (autonomous system). +We cover this more below when describing attacks.

    +

    Local​

    +

    A local attacker has access to communication links in a local network segment. +This could be a rogue access point (with routing capability).

    +

    AS​

    +

    An AS attacker controls a single AS (autonomous system). A passive AS attacker can listen to traffic on arbitrary links within the AS. -An active AS attacker can drop, inject, and alter traffic on arbitrary links within the AS.

    In practice, a malicious ISP would be considered as an AS attacker. +An active AS attacker can drop, inject, and alter traffic on arbitrary links within the AS.

    +

    In practice, a malicious ISP would be considered as an AS attacker. A malicious ISP could also easily setup a set of nodes at specific points in the network, -gaining internal attack power similar to a strong multi node attacker.

    Global On-Net​

    A global on-net attacker has complete overview over the whole network. +gaining internal attack power similar to a strong multi node attacker.

    +

    Global On-Net​

    +

    A global on-net attacker has complete overview over the whole network. A passive global attacker can listen to traffic on all links, while the active global attacker basically carries the traffic: it can freely drop, inject, and alter traffic at all positions in the network. -This basically corresponds to the Dolev-Yao model.

    An entity with this power would, in practice, also have the power of the internal linearly scaling nodes attacker.

    Attack-based Threat Analysis​

    The following lists various attacks including the weakest attacker model in which the attack can be successfully performed. -The respective attack can be performed in all stronger attacker models as well.

    An attack is considered more powerful if it can be successfully performed in a weaker attacker model.

    If not stated otherwise, we look at these attacks with respect to their capability to deanonymize the message sender.

    Scope​

    In this post, we introduce a simple tightly scoped threat model for Waku v2 Relay, which will be extended in the course of this article series.

    In this first post, we will look at the relay protocol in isolation. +This basically corresponds to the Dolev-Yao model.

    +

    An entity with this power would, in practice, also have the power of the internal linearly scaling nodes attacker.

    +

    Attack-based Threat Analysis​

    +

    The following lists various attacks including the weakest attacker model in which the attack can be successfully performed. +The respective attack can be performed in all stronger attacker models as well.

    +

    An attack is considered more powerful if it can be successfully performed in a weaker attacker model.

    +

    If not stated otherwise, we look at these attacks with respect to their capability to deanonymize the message sender.

    +

    Scope​

    +

    In this post, we introduce a simple tightly scoped threat model for Waku v2 Relay, which will be extended in the course of this article series.

    +

    In this first post, we will look at the relay protocol in isolation. Even though many threats arise from layers Waku relay is based on, and layers that in turn live on top of relay, we want to first look at relay in isolation because it is at the core of Waku v2. -Addressing and trying to solve all security issues of a complex system at once is an overwhelming task, which is why we focus on the soundness of relay first.

    This also goes well with the modular design philosophy of Waku v2, as layers of varying levels of security guarantees can be built on top of relay, all of which can relay on the guarantees that Waku provides. -Instead of looking at a multiplicative explosion of possible interactions, we look at the core in this article, and cover the most relevant combinations in future posts.

    Further restricting the scope, we will look at the data field of a relay message as a black box. +Addressing and trying to solve all security issues of a complex system at once is an overwhelming task, which is why we focus on the soundness of relay first.

    +

    This also goes well with the modular design philosophy of Waku v2, as layers of varying levels of security guarantees can be built on top of relay, all of which can relay on the guarantees that Waku provides. +Instead of looking at a multiplicative explosion of possible interactions, we look at the core in this article, and cover the most relevant combinations in future posts.

    +

    Further restricting the scope, we will look at the data field of a relay message as a black box. In a second article on Waku v2 relay, we will look into the data field, which according to the specification of Waku v2 relay must be a Waku v2 message. -We only consider messages with version field 2, which indicates that the payload has to be encoded using WAKU2-NOISE.

    Prerequisite: Get a Specific Position in the Network​

    Some attacks require the attacker node(s) to be in a specific position in the network. -In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.

    In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. +We only consider messages with version field 2, which indicates that the payload has to be encoded using WAKU2-NOISE.

    +

    Prerequisite: Get a Specific Position in the Network​

    +

    Some attacks require the attacker node(s) to be in a specific position in the network. +In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.

    +

    In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. If the victim node still has open slots, the attacker gets the desired position. -This only requires the attacker to know the gossipsub multiaddress of the victim node.

    A linearly scaling nodes attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. -Waku v2 discv5 will employ countermeasures that mitigate the amplifying effect this attacker type can achieve.

    Replay Attack​

    In the scope we defined above, Waku v2 is resilient against replay attacks. +This only requires the attacker to know the gossipsub multiaddress of the victim node.

    +

    A linearly scaling nodes attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. +Waku v2 discv5 will employ countermeasures that mitigate the amplifying effect this attacker type can achieve.

    +

    Replay Attack​

    +

    In the scope we defined above, Waku v2 is resilient against replay attacks. GossipSub nodes, and by extension Waku relay nodes, feature a seen cache, and only relay messages they have not seen before. -Further, replay attacks will be punished by RLN and SWAP.

    Neighbourhood Surveillance​

    This attack can be performed by a single node attacker that is connected to all peers of the victim node vv with respect to a specific topic mesh. -The attacker also has to be connected to vv. -In this position, the attacker will receive messages mvm_v sent by vv both on the direct path from vv, and on indirect paths relayed by peers of vv. -It will also receive messages mxm_x that are not sent by vv. These messages mxm_x are relayed by both vv and the peers of vv. -Messages that are received (significantly) faster from vv than from any other of vv's peers are very likely messages that vv sent, -because for these messages the attacker is one hop closer to the source.

    The attacker can (periodically) measure latency between itself and vv, and between itself and the peers of vv to get more accurate estimates for the expected timings. -An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between vv and its well-behaving peers. -An active AS attacker could also increase the latency between vv and its peers to make the timing differences more prominent. -This, however, might lead to vv switching to other peers.

    This attack cannot (reliably) distinguish messages mvm_v sent by vv from messages mym_y relayed by peers of vv the attacker is not connected to. +Further, replay attacks will be punished by RLN and SWAP.

    +

    Neighbourhood Surveillance​

    +

    This attack can be performed by a single node attacker that is connected to all peers of the victim node vv with respect to a specific topic mesh. +The attacker also has to be connected to vv. +In this position, the attacker will receive messages mvm_v sent by vv both on the direct path from vv, and on indirect paths relayed by peers of vv. +It will also receive messages mxm_x that are not sent by vv. These messages mxm_x are relayed by both vv and the peers of vv. +Messages that are received (significantly) faster from vv than from any other of vv's peers are very likely messages that vv sent, +because for these messages the attacker is one hop closer to the source.

    +

    The attacker can (periodically) measure latency between itself and vv, and between itself and the peers of vv to get more accurate estimates for the expected timings. +An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between vv and its well-behaving peers. +An active AS attacker could also increase the latency between vv and its peers to make the timing differences more prominent. +This, however, might lead to vv switching to other peers.

    +

    This attack cannot (reliably) distinguish messages mvm_v sent by vv from messages mym_y relayed by peers of vv the attacker is not connected to. Still, there are hop-count variations that might be leveraged. -Messages mvm_v always have a hop-count of 1 on the path from vv to the attacker, while all other paths are longer. -Messages mym_y might have the same hop-count on the path from vv as well as on other paths.

    Controlled Neighbourhood​

    If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from vv.

    Observing Messages​

    If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving vv which have not been relayed to vv. -These are the messages sent by vv. -Waku relay protects against this attack by employing secure channels setup using Noise.

    Correlation​

    Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from vv. -This (alone) does not allow an external attacker to learn which message vv sent, but it allows identifying the respective traffic propagating through the network. -The more traffic in the network, the lower the success rate of this attack.

    Combined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.

    DoS​

    An active single node attacker could run a disruption attack by

    • (1) dropping messages that should be relayed
    • (2) flooding neighbours with bogus messages

    While (1) has a negative effect on availability, the impact is not significant. +Messages mvm_v always have a hop-count of 1 on the path from vv to the attacker, while all other paths are longer. +Messages mym_y might have the same hop-count on the path from vv as well as on other paths.

    +

    Controlled Neighbourhood​

    +

    If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from vv.

    +

    Observing Messages​

    +

    If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving vv which have not been relayed to vv. +These are the messages sent by vv. +Waku relay protects against this attack by employing secure channels setup using Noise.

    +

    Correlation​

    +

    Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from vv. +This (alone) does not allow an external attacker to learn which message vv sent, but it allows identifying the respective traffic propagating through the network. +The more traffic in the network, the lower the success rate of this attack.

    +

    Combined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.

    +

    DoS​

    +

    An active single node attacker could run a disruption attack by

    +
      +
    • (1) dropping messages that should be relayed
      • +
    • (2) flooding neighbours with bogus messages
      • +
    +

    While (1) has a negative effect on availability, the impact is not significant. A linearly scaling botnet attacker, however, could significantly disrupt the network with such an attack. (2) is thwarted by RLN. -Also SWAP helps mitigating DoS attacks.

    A local attacker can DoS Waku by dropping all Waku traffic within its controlled network segment. +Also SWAP helps mitigating DoS attacks.

    +

    A local attacker can DoS Waku by dropping all Waku traffic within its controlled network segment. An AS attacker can DoS Waku within its authority, while a global attacker can DoS the whole network. -A countermeasure are censorship resistance techniques like Pluggable Transports.

    Summary and Future Work​

    Currently, Waku v2 relay offers k-anonymity with respect to receiver anonymity. -This also includes k-anonymity towards legitimate members of the same topic.

    Waku v2 relay offers sender anonymity in the single node attacker model with its strict no sign policy. +A countermeasure are censorship resistance techniques like Pluggable Transports.

    +

    Summary and Future Work​

    +

    Currently, Waku v2 relay offers k-anonymity with respect to receiver anonymity. +This also includes k-anonymity towards legitimate members of the same topic.

    +

    Waku v2 relay offers sender anonymity in the single node attacker model with its strict no sign policy. Currently, Waku v2 does not guarantee sender anonymity in the multi node and stronger attacker models. However, we are working on modular anonymity-preserving protocols and building blocks as part of our privacy/anonymity roadmap. The goal is to allow tunable anonymity with respect to trade offs between strong anonymity, low bandwidth, and low latency. All of these cannot be fully guaranteed as the the anonymity trilemma states. Some applications have specific requirements, e.g. low latency, which require a compromise on anonymity. -Anonymity-preserving mechanisms we plan to investigate and eventually specify as pluggable anonymity protocols for Waku comprise

    • Dandelion++ for lightweight anonymity;
    • onion routing as a building block adding a low latency anonymization layer;
    • a mix network for providing strong anonymity (on top of onion routing) even in the strongest attacker model at the cost of higher latency.

    These pluggable anonymity-preserving protocols will form a sub-set of the Waku v2 protocol set. -As an intermediate step, we might directly employ Tor for onion-routing, and Nym as a mix-net layer.

    In future research log posts, we will cover further Waku v2 protocols and identify anonymity problems that will be added to our roadmap. -These protocols comprise

    • 13/WAKU2-STORE, which can violate receiver anonymity as it allows filtering by content topic. -A countermeasure is using the content topic exclusively for local filters.
    • 12/WAKU2-FILTER, which discloses nodes' interest in topics;
    • 19/WAKU2-LIGHTPUSH, which also discloses nodes' interest in topics and links the lightpush client as the sender of a message to the lightpush service node;
    • 21/WAKU2-FTSTORE, which discloses nodes' interest in specific time ranges allowing to infer information like online times.

    While these protocols are not necessary for the operation of Waku v2, and can be seen as pluggable features, -we aim to provide alternatives without the cost of lowering the anonymity level.

    References​

    ]]>     - - - <![CDATA[Noise handshakes as key-exchange mechanism for Waku]]> - https://vac.dev/rlog/wakuv2-noise - https://vac.dev/rlog/wakuv2-noise - Tue, 17 May 2022 10:00:00 GMT - - We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2.

    Introduction​

    In this post we will provide an overview of how Waku v2 users can adopt Noise handshakes to agree on cryptographic keys used to securely encrypt messages.

    This process belongs to the class of key-exchange mechanisms, consisting of all those protocols that, with different levels of complexity and security guarantees, allow two parties to publicly agree on a secret without letting anyone else know what this secret is.

    But why do we need key-exchange mechanisms in the first place?

    With the advent of public-key cryptography, it become possible to decouple encryption from decryption through use of two distinct cryptographic keys: one public, used to encrypt information and that can be made available to anyone, and one private (kept secret), which enables decryption of messages encrypted with its corresponding public key. The same does not happen in the case of symmetric encryption schemes where, instead, the same key is used for both encryption and decryption operations and hence cannot be publicly revealed as for public keys.

    In order to address specific application needs, many different public, symmetric and hybrid cryptographic schemes were designed: Waku v1 and Waku v2, which inherits part of their design from the Ethereum messaging protocol Whisper, provide support to both public-key primitives (ECIES, ECDSA) and symmetric primitives (AES-256-GCM, KECCAK-256), used to sign, hash, encrypt and decrypt exchanged messages.

    In principle, when communications employ public-key based encryption schemes (ECIES, in the case of Waku), there is no need for a key-agreement among parties: messages can be directly encrypted using the recipient's public-key before being sent over the network. However, public-key encryption and decryption primitives are usually very inefficient in processing large amount of data, and this may constitute a bottleneck for many of today's applications. Symmetric encryption schemes such as AES-256-GCM, on the other hand, are much more efficient, but the encryption/decryption key needs to be shared among users beforehand any encrypted messages is exchanged.

    To counter the downsides given by each of these two approaches while taking advantage of their strengths, hybrid constructions were designed. In these, public-key primitives are employed to securely agree on a secret key which, in turn, is used with a symmetric cipher for encrypting messages. In other words, such constructions specify a (public-key based) key-agreement mechanism!

    Waku, up to payload version 1, does not implement nor recommend any protocol for exchanging symmetric ciphers' keys, leaving such task to the application layer. It is important to note that the kind of key-agreement employed has a direct impact on the security properties that can be granted on later encrypted messages, while security requirements usually depend on the specific application for which encryption is needed in the first place.

    In this regard, Status, which builds on top of Waku, implements a custom version of the X3DH key-agreement protocol, in order to allow users to instantiate end-to-end encrypted communication channels. However, although such a solution is optimal when applied to (distributed) E2E encrypted chats, it is not flexible enough to fit or simplify the variety of applications Waku aims to address. -Hence, proposing and implementing one or few key-agreements which provide certain (presumably strong) security guarantees, would inevitably degrade performances of all those applications for which, given their security requirements, more tailored and efficient key-exchange mechanisms can be employed.

    Guided by different examples, in the following sections we will overview Noise, a protocol framework we are currently integrating in Waku, for building secure key-agreements between two parties. One of the great advantage of using Noise is that it is possible to add support to new key-exchanges by just specifying users' actions from a predefined list, requiring none to minimal modifications to existing implementations. Furthermore, Noise provides a framework to systematically analyze protocols' security properties and the corresponding attacker threat models. This allows not only to easily design new key-agreements eventually optimized for specific applications we want to address, but also to easily analyze or even formally verify any of such custom protocol!

    We believe that with its enormous flexibility and features, Noise represents a perfect candidate for bringing key-exchange mechanisms in Waku.

    The Diffie-Hellman Key-exchange​

    The formalization of modern public-key cryptography started with the pioneering work of Whitefield Diffie and Martin Hellman, who detailed one of the earliest known key-agreement protocols: the famous Diffie-Hellman Key-Exchange.

    Diffie-Hellman (DH) key-exchange is largely used today and represents the main cryptographic building block on which Noise handshakes' security is based.

    In turn, the security of DH is based on a mathematical problem called discrete logarithm which is believed to be hard when the agreement is practically instantiated using certain elliptic curves EE defined over finite fields Fp\mathbb{F}_p.

    Informally, a DH exchange between Alice and Bob proceeds as follows:

    • Alice picks a secret scalar sA∈Fps_A\in\mathbb{F}_p and computes, using the underlying curve's arithmetic, the point PA=sA⋅P∈E(Fp)P_A = s_A\cdot P\in E(\mathbb{F}_p) for a certain pre-agreed public generator PP of the elliptic curve E(Fp)E(\mathbb{F}_p). She then sends PAP_A to Bob.
    • Similarly, Bob picks a secret scalar sB∈Fps_B\in\mathbb{F}_p, computes PB=sB⋅P∈E(Fp)P_B = s_B\cdot P\in E(\mathbb{F}_p) and sends PBP_B to Alice.
    • By commutativity of scalar multiplication, both Alice and Bob can now compute the point PAB=sAsB⋅PP_{AB} = s_As_B\cdot P, using the elliptic curve point received from the other party and their secret scalar.

    The assumed hardness of computing discrete logarithms in the elliptic curve, ensures that it is not possible to compute sAs_A or sBs_B from PAP_A and PBP_B, respectively. Another security assumption (named Computational Diffie-Hellman assumption) ensures that it is not possible to compute PABP_{AB} from PP, PAP_A and PBP_B. Hence the point PABP_{AB} shared by Alice and Bob at the end of the above protocol cannot be efficiently computed by an attacker intercepting PAP_A and PBP_B, and can then be used to generate a secret to be later employed, for example, as a symmetric encryption key.

    On a side note, this protocol shows the interplay between two components typical to public-key based schemes: the scalars sAs_A and sBs_B can be seen as private keys associated to the public keys PAP_A and PBP_B, respectively, which allow Alice and Bob only to compute the shared secret point PABP_{AB}.

    Ephemeral and Static Public Keys​

    Although we assumed that it is practically impossible for an attacker to compute the randomly picked secret scalar from the corresponding public elliptic curve point, it may happen that such scalar gets compromised or can be guessed due to a faulty employed random number generator. In such cases, an attacker will be able to recover the final shared secret and all encryption keys eventually derived from that, with clear catastrophic consequences for the privacy of exchanged messages.

    To mitigate such issues, multiple DH operations can be combined using two different types of exchanged elliptic curve points or, better, public keys: ephemeral keys, that is random keys used only once in a DH operation, and long-term static keys, used mainly for authentication purposes since employed multiple times.

    Just to provide an example, let us suppose Alice and Bob perform the following custom DH-based key-exchange protocol:

    • Alice generates an ephemeral key EA=eA⋅PE_A=e_A\cdot P by picking a random scalar eAe_A and sends EAE_A to Bob;
    • Similarly, Bob generates an ephemeral key EB=eB⋅PE_B=e_B\cdot P and sends EBE_B to Alice;
    • Alice and Bob computes EAB=eAeB⋅PE_{AB} = e_Ae_B \cdot P and from it derive a secret encryption key kk.
    • Bob sends to Alice his static key SB=sB⋅PS_B = s_B\cdot P encrypted with kk.
    • Alice encrypts with kk her static key SA=sA⋅PS_A = s_A\cdot P and sends it to Bob.
    • Alice and Bob decrypt the received static keys, compute the secret SAB=sAsB⋅PS_{AB} = s_As_B \cdot P and use it together with EABE_{AB} to derive a new encryption key k~\tilde{k} to be later used with a symmetric cipher.

    In this protocol, if Alice's and/or Bob's static keys get compromised, it would not possible to derive the final secret key k~\tilde{k}, since at least one ephemeral key among EAE_A and EBE_B has to be compromised too in order to recover the secret EABE_{AB}. Furthermore, since Alice's and Bob's long-term static keys are encrypted, an attacker intercepting exchanged (encrypted) public keys will not be able to link such communication to Alice or Bob, unless one of the ephemeral key is compromised (and, even in such case, none of the messages encrypted under the key k~\tilde{k} can be decrypted).

    The Noise Protocol Framework​

    In previous section we gave a small intuition on how multiple DH operations over ephemeral and static users' public keys can be combined to create different key-exchange protocols.

    The Noise Protocol Framework, defines various rules for building custom key-exchange protocols while allowing easy analysis of the security properties and threat models provided given the type and order of the DH operations employed.

    In Noise terminology, a key-agreement or Noise protocol consists of one or more Noise handshakes. During a Noise handshake, Alice and Bob exchange multiple (handshake) messages containing their ephemeral keys and/or static keys. These public keys are then used to perform a handshake-dependent sequence of Diffie-Hellman operations, whose results are all hashed into a shared secret key. Similarly as we have seen above, after a handshake is complete, each party will use the derived secret key to send and receive authenticated encrypted data by employing a symmetric cipher.

    Depending on the handshake pattern adopted, different security guarantees can be provided on messages encrypted using a handshake-derived key.

    The Noise handshakes we support in Waku all provide the following security properties:

    • Confidentiality: the adversary should not be able to learn what data is being sent between Alice and Bob.
    • Strong forward secrecy: an active adversary cannot decrypt messages nor infer any information on the employed encryption key, even in the case he has access to Alice's and Bob's long-term private keys (during or after their communication).
    • Authenticity: the adversary should not be able to cause either Alice or Bob to accept messages coming from a party different than their original senders.
    • Integrity: the adversary should not be able to cause Alice or Bob to accept data that has been tampered with.
    • Identity-hiding: once a secure communication channel is established, a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient by knowing their long-term static keys.

    We refer to Noise specification for more formal security definitions and precise threat models relative to Waku supported Noise Handshake patterns.

    Message patterns​

    Noise handshakes involving DH operations over ephemeral and static keys can be succinctly sketched using the following set of handshake message tokens: e,s,ee,se,es,ss.

    Tokens employing single letters denote (the type of) users' public keys: e refers to randomly generated ephemeral key(s), while s indicates the users' long-term static key(s).

    Two letters tokens, instead, denotes DH operations over the two users' public keys the token refers to, given that the left token letter refers to the handshake initiator's public key, while the right token letter indicates the used responder's public key. Thus, if Alice started a handshake with Bob, the es token will shortly represent a DH operation among Alice's ephemeral key e and Bob's static key s.

    Since, in order to perform any DH operations users need to share (or pre-share) the corresponding public keys, Noise compactly represents messages' exchanges using the two direction -> and <-, where the -> denotes a message (arbitrary and/or DH public key) from the initiator to the responder, while <- the opposite.

    Hence a message pattern consisting of a direction and one or multiple tokens such as <- e, s, es has to be interpreted one token at a time: in this example, the responder is sending his ephemeral and static key to the initiator and is then executing a DH operation over the initiator's ephemeral key e (shared in a previously exchanged message pattern) and his static key s. On the other hand, such message indicates also that the initiator received the responder's ephemeral and static keys e and s, respectively, and performed a DH operation over his ephemeral key and the responder's just received static key s. In this way, both parties will be able to derive at the end of each message pattern processed the same shared secret, which is eventually used to update any derived symmetric encryption keys computed so far.

    In some cases, DH public keys employed in a handshake are pre-shared before the handshake itself starts. In order to chronologically separate exchanged keys and DH operations performed before and during a handshake, Noise employs the ... delimiter.

    For example, the following message patterns

    <- e
    ...
    -> e, ee

    indicates that the initiator knew the responder's ephemeral key before he sends his own ephemeral key and executes a DH operation between both parties ephemeral keys (similarly, the responder receives the initiator's ephemeral key and does a ee DH operation).

    At this point it should be clear how such notation is able to compactly represent a large variety of DH based key-agreements. Nevertheless, we can easily define additional tokens and processing rules in order to address specific applications and security requirements, such as the psk token used to process arbitrary pre-shared key material.

    As an example of Noise flexibility, the custom protocol we detailed above can be shortly represented as (Alice is on the left):

    -> e
    <- e, ee, s
    -> s, ss

    where after each DH operation an encryption key is derived (along with the secrets computed by all previously executed DH operations) in order to encrypt/decrypt any subsequent sent/received message.

    Another example is given by the possibility to replicate within Noise the well established Signal's X3DH key-agreement protocols, thus making the latter a general framework to design and study security of many practical and widespread DH-based key-exchange protocols.

    The Noise State Objects​

    We mentioned multiple times that parties derive an encryption key each time they perform a DH operation, but how does this work in more details?

    Noise defines three state object: a Handshake State, a Symmetric State and a Cipher State, each encapsulated into each other and instantiated during the execution of a handshake.

    The Handshake State object stores the user's and other party's received ephemeral and static keys (if any) and embeds a Symmetric State object.

    The Symmetric State, instead, stores a handshake hash value h, iteratively updated with any message read/received and DH secret computed, and a chaining key ck, updated using a key derivation function every time a DH secret is computed. This object further embeds a Cipher State.

    Lastly, the Cipher State stores a symmetric encryption k key and a counter n used to encrypt and decrypt messages exchanged during the handshake (not only static keys, but also arbitrary payloads). These key and counter are refreshed every time the chaining key is updated.

    While processing each handshake's message pattern token, all these objects are updated according to some specific processing rules which employ a combination of public-key primitives, hash and key-derivation functions and symmetric ciphers. It is important to note, however, that at the end of each processed message pattern, the two users will share the same Symmetric and Cipher State embedded in their respective Handshake States.

    Once a handshake is complete, users derive two new Cipher States and can then discard the Handshake State object (and, thus, the embedded Symmetric State and Cipher State objects) -employed during the handshake.

    These two Cipher states are used to encrypt and decrypt all outbound and inbound after-handshake messages, respectively, and only to these will be granted the confidentiality, authenticity, integrity and identity-hiding properties we detailed above.

    For more details on processing rules, we refer to Noise specifications.

    Supported Noise Handshakes in Waku​

    The Noise handshakes we provided support to in Waku address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:

    • Alice and Bob know each others' static key.
    • Alice knows Bob's static key;
    • Alice and Bob share no key material and they don't know each others' static key.
    • Alice and Bob share some key material, but they don't know each others' static key.

    The possibility to have handshakes based on the reciprocal knowledge parties have of each other, allows designing Noise handshakes that can quickly reach the desired level of security on exchanged encrypted messages while keeping the number of interactions between Alice and Bob minimum.

    Nonetheless, due to the pure token-based nature of handshake processing rules, implementations can easily add support to any custom handshake pattern with minor modifications, in case more specific application use-cases need to be addressed.

    On a side note, we already mentioned that identity-hiding properties can be guaranteed against a passive attacker that only reads the communication occurring between Alice and Bob. However, an active attacker who compromised one party's static key and actively interferes with the parties' exchanged messages, may lower the identity-hiding security guarantees provided by some handshake patterns. In our security model we exclude such adversary, but, for completeness, in the following we report a summary of possible de-anonymization attacks that can be performed by such an active attacker.

    For more details on supported handshakes and on how these are implemented in Waku, we refer to WAKU2-NOISE RFC.

    The K1K1 Handshake​

    If Alice and Bob know each others' static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a K1K1 handshake. In Noise notation (Alice is on the left) this can be sketched as:

     K1K1:
        ->  s
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  se

    We note that here only ephemeral keys are exchanged. This handshake is useful in case Alice needs to instantiate a new separate encrypted communication channel with Bob, e.g. opening multiple parallel connections, file transfers, etc.

    Security considerations on identity-hiding (active attacker): no static key is transmitted, but an active attacker impersonating Alice can check candidates for Bob's static key.

    The XK1 Handshake​

    Here, Alice knows how to initiate a communication with Bob and she knows his public static key: such discovery can be achieved, for example, through a publicly accessible register of users' static keys, smart contracts, or through a previous public/private advertisement of Bob's static key.

    A Noise handshake pattern that suits this scenario is XK1:

     XK1:
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  s, se

    Within this handshake, Alice and Bob reciprocally authenticate their static keys s using ephemeral keys e. We note that while Bob's static key is assumed to be known to Alice (and hence is not transmitted), Alice's static key is sent to Bob encrypted with a key derived from both parties ephemeral keys and Bob's static key.

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob's static key against recorded/accepted exchanged handshake messages.

    The XX and XXpsk0 Handshakes​

    If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an XX handshake, where each party tranXmits to the other its own static key.

    The handshake goes as follows:

     XX:
        ->  e
        <-  e, ee, s, es
        ->  s, se

    We note that the main difference with XK1 is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange.

    This handshake can be slightly changed in case both Alice and Bob pre-shares some secret psk which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called XXpsk0, goes as follow:

     XXpsk0:
        ->  psk, e
        <-  e, ee, s, es
        ->  s, se

    The main difference with XX is that Alice's and Bob's static keys, when transmitted, would be encrypted with a key derived from psk as well.

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party for both XX and XXpsk0 handshakes. In XX, Bob's static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In XXpsk0, instead, Bob's secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret psk but not through any static key), provided that psk was not previously compromised (in such case identity-hiding properties provided by the XX handshake applies).

    Session Management and Multi-Device Support​

    When two users complete a Noise handshake, an encryption/decryption session - or Noise session - consisting of two Cipher States is instantiated.

    By identifying Noise session with a session-id derived from the handshake's cryptographic material, we can take advantage of the PubSub/GossipSub protocols used by Waku for relaying messages in order to manage instantiated Noise sessions.

    The core idea is to exchange after-handshake messages (encrypted with a Cipher State specific to the Noise session), over a content topic derived from the (secret) session-id the corresponding session refers to.

    This allows to decouple the handshaking phase from the actual encrypted communication, thus improving users' identity-hiding capabilities.

    Furthermore, by publicly revealing a value derived from session-id on the corresponding session content topic, a Noise session can be marked as stale, enabling peers to save resources by discarding any eventually stored message sent to such content topic.

    One relevant aspect in today's applications is the possibility for users to employ different devices in their communications. In some cases, this is non-trivial to achieve since, for example, encrypted messages might be required to be synced on different devices which do not necessarily share the necessary key material for decryption and may be temporarily offline.

    We address this by requiring each user's device to instantiate multiple Noise sessions either with all user's other devices which, in turn, all together share a Noise session with the other party, or by directly instantiating a Noise session with all other party's devices.

    We named these two approaches N11MN11M and NMNM, respectively, which are in turn loosely based on the paper “Multi-Device for Signal” and Signal’s Sesame Algorithm.

    Informally, in the N11MN11M session management scheme, once the first Noise session between any of Alice’s and Bob’s device is instantiated, its session information is securely propagated to all other devices using previously instantiated Noise sessions. Hence, all devices are able to send and receive new messages on the content topic associated to such session.

    In the NMNM session management scheme, instead, all pairs of Alice's and Bob's devices have a distinct Noise session: a message is then sent from the currently-in-use sender’s device to all recipient’s devices, by properly encrypting and sending it to the content topics of each corresponding Noise session. If sent messages should be available on all sender’s devices as well, we require each pair of sender’s devices to instantiate a Noise session used for syncing purposes.

    For more technical details on how Noise sessions are instantiated and managed within these two mechanisms and the different trade-offs provided by the latter, we refer to WAKU2-NOISE-SESSIONS.

    Conclusions​

    In this post we provided an overview of Noise, a protocol framework for designing Diffie-Hellman based key-exchange mechanisms allowing systematic security and threat model analysis.

    The flexibility provided by Noise components allows not only to fully replicate with same security guarantees well established key-exchange primitives such as X3DH, currently employed by Status TRANSPORT-SECURITY, but enables also optimizations based on the reciprocal knowledge parties have of each other while allowing easier protocols' security analysis and (formal) verification.

    Furthermore, different handshakes can be combined and executed one after each other, a particularly useful feature to authenticate multiple static keys employed by different applications but also to ease keys revocation.

    The possibility to manage Noise sessions over multiple devices and the fact that handshakes can be concretely instantiated using modern, fast and secure cryptographic primitives such as ChaChaPoly and BLAKE2b, make Noise one of the best candidates for efficiently and securely address the many different needs of applications built on top of Waku requiring key-agreement.

    Future steps​

    The available implementation of Noise in nwaku, although mostly complete, is still in its testing phase. As future steps we would like to:

    • have an extensively tested and robust Noise implementation;
    • formalize, implement and test performances of the two proposed N11MN11M and NMNM session management mechanisms and their suitability for common use-case scenarios;
    • provide Waku network nodes a native protocol to readily support key-exchanges, strongly-encrypted communication and multi-device session management mechanisms with none-to-little interaction besides applications' connection requests.

    References​

    ]]>     -     - - <![CDATA[Waku v2 Ambient Peer Discovery]]> - https://vac.dev/rlog/wakuv2-apd - https://vac.dev/rlog/wakuv2-apd - Mon, 09 May 2022 10:00:00 GMT - - Introducing and discussing ambient peer discovery methods currently used by Waku v2, as well as future plans in this area.

    Waku v2 comprises a set of modular protocols for secure, privacy preserving communication. -Avoiding centralization, these protocols exchange messages over a P2P network layer. -In order to build a P2P network, participating nodes first have to discover peers within this network. -This is where ambient peer discovery comes into play: -it allows nodes to find peers, making it an integral part of any decentralized application.

    In this post the term node to refers to our endpoint or the endpoint that takes action, -while the term peer refers to other endpoints in the P2P network. -These endpoints can be any device connected to the Internet: e.g. servers, PCs, notebooks, mobile devices, or applications like a browser. -As such, nodes and peers are the same. We use these terms for the ease of explanation without loss of generality.

    In Waku's modular design, ambient peer discovery is an umbrella term for mechanisms that allow nodes to find peers. -Various ambient peer discovery mechanisms are supported, and each is specified as a separate protocol. -Where do these protocols fit into Waku's protocol stack? -The P2P layer of Waku v2 builds on libp2p gossipsub. -Nodes participating in a gossipsub protocol manage a mesh network that is used for routing messages. -This mesh network is an unstructured P2P network -offering high robustness and resilience against attacks. -Gossipsub implements many improvements overcoming the shortcomings typically associated with unstructured P2P networks, e.g. inefficient flooding based routing. -The gossipsub mesh network is managed in a decentralized way, which requires each node to know other participating peers. -Waku v2 may use any combination of its ambient discovery protocols to find appropriate peers.

    Summarizing, Waku v2 comprises a peer management layer based on libp2p gossipsub, -which manages the peers of nodes, and an ambient peer discovery layer, -which provides information about peers to the peer management layer.

    We focus on ambient peer discovery methods that are in line with our goal of building a fully decentralized, generalized, privacy-preserving and censorship-resistant messaging protocol. -Some of these protocols still need adjustments to adhere to our privacy and anonymity requirements. For now, we focus on operational stability and feasibility. -However, when choosing techniques, we pay attention to selecting mechanisms that can feasibly be tweaked for privacy in future research efforts. -Because of the modular design and the fact that Waku v2 has several discovery methods at its disposal, we could even remove a protocol in case future evaluation deems it not fitting our standards.

    This post covers the current state and future considerations of ambient peer discovery for Waku v2, -and gives reason for changes and modifications we made or plan to make. -The ambient peer discovery protocols currently supported by Waku v2 are a modified version of Ethereum's Discovery v5 -and DNS-based discovery. -Waku v2 further supports gossipsub's peer exchange protocol. -In addition, we plan to introduce protocols for general peer exchange and capability discovery, respectively. -The former allows resource restricted nodes to outsource querying for peers to stronger peers, -the latter allows querying peers for their supported capabilities. -Besides these new protocols, we are working on integrating capability discovery in our existing ambient peer discovery protocols.

    Static Node Lists​

    The simplest method of learning about peers in a P2P network is via static node lists. -These can be given to nodes as start-up parameters or listed in a config-file. -They can also be provided in a script-parseable format, e.g. in JSON. -While this method of providing bootstrap nodes is very easy to implement, it requires static peers, which introduce centralized elements. -Also, updating static peer information introduces significant administrative overhead: -code and/or config files have to be updated and released. -Typically, static node lists only hold a small number of bootstrap nodes, which may lead to high load on these nodes.

    DNS-based Discovery​

    Compared to static node lists, -DNS-based discovery (specified in EIP-1459) -provides a more dynamic way of discovering bootstrap nodes. -It is very efficient, can easily be handled by resource restricted devices and provides very good availability. -In addition to a naive DNS approach, Ethereum's DNS-based discovery introduces efficient authentication leveraging Merkle trees.

    A further advantage over static node lists is the separation of code/release management and bootstrap node management. -However, changing and updating the list of bootstrap nodes still requires administrative privileges because DNS records have to be added or updated.

    While this method of discovery still requires centralized elements, -node list management can be delegated to various DNS zones managed by other entities mitigating centralization.

    Discovery V5​

    A much more dynamic method of ambient peer discovery is Discovery v5, which is Ethereum's peer discovery protocol. -It is based on the Kademlia distributed hashtable (DHT). -An introduction to discv5 and its history, and a discv5 Waku v2 feasibility study -can be found in previous posts on this research log.

    We use Discovery v5 as an ambient peer discovery method for Waku v2 because it is decentralized, efficient, actively researched, and has web3 as its main application area. -Discv5 also offers mitigation techniques for various attacks, which we cover later in this post.

    Using a DHT (structured P2P network) as a means for ambient peer discovery, while using the gossipsub mesh network (unstructured P2P network) for transmitting actual messages, -Waku v2 leverages advantages from both worlds. -One of the main benefits of DHTs is offering a global view over participating nodes. -This, in turn, allows sampling random sets of nodes which is important for equally distributing load. -Gossipsub, on the other hand, offers great robustness and resilience against attacks. -Even if discv5 discovery should not work in advent of a DoS attack, Waku v2 can still operate switching to different discovery methods.

    Discovery methods that use separate P2P networks still depend on bootstrapping, -which Waku v2 does via parameters on start-up or via DNS-based discovery. -This might raise the question of why such discovery methods are beneficial. -The answer lies in the aforementioned global view of DHTs. Without discv5 and similar methods, the bootstrap nodes are used as part of the gossipsub mesh. -This might put heavy load on these nodes and further, might open pathways to inference attacks. -Discv5, on the other hand, uses the bootstrap nodes merely as an entry to the discovery network and can provide random sets of nodes (sampled from a global view) -for bootstrapping or expanding the mesh.

    DHT Background​

    Distributed Hash Tables are a class of structured P2P overlay networks. -A DHT can be seen as a distributed node set of which each node is responsible for a part of the hash space. -In contrast to unstructured P2P networks, e.g. the mesh network maintained by gossipsub, -DHTs have a global view over the node set and the hash space (assuming the participating nodes behave well).

    DHTs are susceptible to various kinds of attacks, especially Sybil attacks -and eclipse attacks. -While security aspects have been addressed in various research papers, general practical solutions are not available. -However, discv5 introduced various practical mitigation techniques.

    Random Walk Discovery​

    While discv5 is based on the Kademlia DHT, it only uses the distributed node set aspect of DHTs. -It does not map values (items) into the distributed hash space. -This makes sense, because the main purpose of discv5 is discovering other nodes that support discv5, which are expected to be Ethereum nodes. -Ethereum nodes that want to discover other Ethereum nodes simply query the discv5 network for a random set of peers. -If Waku v2 would do the same, only a small subset of the retrieved nodes would support Waku v2.

    A first naive solution for Waku v2 discv5 discovery is

    • retrieve a random node set, which is achieved by querying for a set of randomly chosen node IDs
    • filter the returned nodes on the query path based on Waku v2 capability via the Waku v2 ENR
    • repeat until enough Waku v2 capable nodes are found

    This query process boils down to random walk discovery, which is very resilient against attacks, but also very inefficient if the number of nodes supporting the desired capability is small. -We refer to this as the needle-in-the-haystack problem.

    Random Walk Performance Estimation​

    This subsection provides a rough estimation of the overhead introduced by random walk discovery.

    Given the following parameters:

    • nn number of total nodes participating in discv5
    • pp percentage of nodes supporting Waku
    • WW the event of having at least one Waku node in a random sample
    • kk the size of a random sample (default = 16)
    • α\alpha the number of parallel queries started
    • bb bits per hop
    • qq the number of queries

    A query takes log2bnlog_{2^b}n hops to retrieve a random sample of nodes.

    P(W)=1−(1−p/100)kP(W) = 1 - (1-p/100)^k is the probability of having at least one Waku node in the sample.

    P(Wq)=1−(1−p/100)kqP(W^q) = 1 - (1-p/100)^{kq} is the probability of having at least one Waku node in the union of qq samples.

    Expressing this in terms of qq, we can write: -P(Wq)=1−(1−p/100)kq  ⟺  q=log(1−p/100)k(1−P(Wq))P(W^q) = 1 - (1-p/100)^{kq} \iff q = log_{(1-p/100)^k}(1-P(W^q))

    Figure 1 shows a log-log plot for P(Wq)=90%P(W^q) = 90\%.

    Figure 1: log-log plot showing the number of queries necessary to retrieve a Waku v2 node with a probability of 90% in relation to the Waku v2 node concentration in the network.

    Assuming p=0.1p=0.1, we would need

    0.9=1−(1−0.1/100)16q=>q≈1440.9 = 1 - (1-0.1/100)^{16q} => q \approx 144

    queries to get a Waku node with 90% probability, which leads to ≈144∗18=2592\approx 144 * 18 = 2592 overlay hops. -Choosing b=3b=3 would reduce the number to ≈144∗6=864\approx 144 * 6 = 864. -Even when choosing α=10\alpha = 10 we would have to wait at least 80 RTTs. -This effort is just for retrieving a single Waku node. Ideally, we want at least 3 Waku nodes for bootstrapping a Waku relay.

    The discv5 doc roughly estimates p=1p=1% to be the threshold for acceptably efficient random walk discovery. -This is in line with our estimation:

    0.9=1−(1−1/100)16q=>q≈140.9 = 1 - (1-1/100)^{16q} => q \approx 14

    The number of necessary queries is linearly dependent on the percentage pp of Waku nodes. -The number of hops per query is logarithmically dependent on nn. -Thus, random walk searching is inefficient for small percentages pp. -Still, random walks are more resilient against attacks.

    We can conclude that a Waku node concentration below 1% renders vanilla discv5 unfit for our needs. -Our current solution and future plans for solving this issue are covered in the next subsections.

    Simple Solution: Separate Discovery Network​

    The simple solution we currently use for Waku v2 discv5 is a separate discv5 network. -All (well behaving) nodes in this network support Waku v2, resulting in a very high query efficiency. -However, this solution reduces resilience because the difficulty of attacking a DHT scales with the number of participating nodes.

    Discv5 Topic Discovery​

    We did not base our solution on the current version of discv5 topic discovery, -because, similar to random walk discovery, it suffers from poor performance for relatively rare capabilities/topics.

    However, there is ongoing research in discv5 topic discovery which is close to ideas we explored when pondering efficient and resilient Waku discv5 solutions. -We keep a close eye on this research, give feedback, and make suggestions, as we plan to switch to this version of topic discovery in the future.

    In a nutshell, topic discovery will manage separate routing tables for each topic. -These topic specific tables are initialized with nodes from the discv5 routing table. -While the buckets of the discv5 routing table represent distance intervals from the node's node ID, the topic table buckets represent distance intervals from topic IDs.

    Nodes that want to register a topic try to register that topic at one random peer per bucket. -This leads to registering the topic at peers in closer and closer neighbourhoods around the topic ID, which -yields a very efficient and resilient compromise between random walk discovery and DHT discovery. -Peers in larger neighbourhoods around the topic ID are less efficient to discover, however more resilient against eclipse attacks and vice versa.

    Further, this works well with the overload and DoS protection discv5 employs. -Discv5 limits the amount of nodes registered per topic on a single peer. Further, discv5 enforces a waiting time before nodes can register topics at peers. -So, for popular topics, a node might fail to register the topic in a close neighbourhood. -However, because the topic is popular (has a high occurrence percentage pp), it can still be efficiently discovered.

    In the future, we also plan to integrate Waku v2 capability discovery, which will not only allow asking for nodes that support Waku v2, -but asking for Waku v2 nodes supporting specific Waku v2 protocols like filter or store. -For the store protocol we envision sub-capabilities reflecting message topics and time frames of messages. -We will also investigate related security implications.

    Attacks on DHTs​

    In this post, we only briefly describe common attacks on DHTs. -These attacks are mainly used for denial of service (DoS), -but can also used as parts of more sophisticated attacks, e.g. deanonymization attacks. -A future post on this research log will cover security aspects of ambient peer discovery with a focus on privacy and anonymity.

    Sybil Attack

    The power of an attacker in a DHT is proportional to the number of controlled nodes. -Controlling nodes comes at a high resource cost and/or requires controlling a botnet via a preliminary attack.

    In a Sybil attack, an attacker generates lots of virtual node identities. -This allows the attacker to control a large portion of the ID space in a DHT at a relatively low cost. -Sybil attacks are especially powerful when the attacker can freely choose the IDs of generated nodes, -because this allows positioning at chosen points in the DHT.

    Because Sybil attacks amplify the power of many attacks against DHTs, -making Sybil attacks as difficult as possible is the basis for resilient DHT operation. -The typical abstract mitigation approach is binding node identities to physical network interfaces. -To some extend, this can be achieved by introducing IP address based limits. -Further, generating node IDs can be bound by proof of work (PoW), -which, however, comes with a set of shortcomings, e.g. relatively high costs on resource restricted devices. -The discv5 doc -describes both Sybil and eclipse attacks, as well as concrete mitigation techniques employed by discv5.

    Eclipse Attack

    In an eclipse attack, nodes controlled by the attacker poison the routing tables of other nodes in a way that parts of the DHT become eclipsed, i.e. invisible. -When a controlled node is asked for the next step in a path, -it provides another controlled node as the next step, -effectively navigating the querying node around or away from certain areas of the DHT. -While several mitigation techniques have been researched, there is no definitive protection against eclipse attacks available as of yet. -One mitigation technique is increasing α\alpha, the number of parallel queries, and following each concurrent path independently for the lookup.

    The eclipse attack becomes very powerful in combination with a successful Sybil attack; -especially when the attacker can freely choose the position of the Sybil nodes.

    The aforementioned new topic discovery of discv5 provides a good balance between protection against eclipse attacks and query performance.

    Peer Exchange Protocol​

    While discv5 based ambient peer discovery has many desirable properties, resource restricted nodes and nodes behind restrictive NAT setups cannot run discv5 satisfactory. -With these nodes in mind, we started working on a simple peer exchange protocol based on ideas proposed here. -The peer exchange protocol will allow nodes to ask peers for additional peers. -Similar to discv5, the peer exchange protocol will also support capability discovery.

    The new peer exchange protocol can be seen as a simple replacement for the Rendezvous protocol, which Waku v2 does not support. -While the rendezvous protocol involves nodes registering at rendezvous peers, the peer exchange protocol simply allows nodes to ask any peer for a list of peers (with a certain set of capabilities). -Rendezvous tends to introduce centralized elements as rendezvous peers have a super-peer role.

    In the future, we will investigate resource usage of Waku v2 discv5 and provide suggestions for minimal resources nodes should have to run discv5 satisfactory.

    Waku v2 comprises further protocols related to ambient peer discovery. We shortly mention them for context, even though they are not strictly ambient peer discovery protocols.

    Gossipsub Peer Exchange Protocol​

    Gossipsub provides an integrated peer exchange mechanism which is also supported by Waku v2. -Gossipsub peer exchange works in a push manner. Nodes send peer lists to peers they prune from the active mesh. -This pruning is part of the gossipsub peer management, blurring the boundaries of peer management and ambient peer discovery.

    We will investigate anonymity implications of this protocol and might disable it in favour of more anonymity-preserving protocols. -Sending a list of peers discloses information about the sending node. -We consider restricting these peer lists to cached peers that are currently not used in the active gossipsub mesh.

    Capability Negotiation​

    Some of the ambient peer discovery methods used by Waku2 will support capability discovery. -This allows to narrow down the set of retrieved peers to peers that support specific capabilities. -This is efficient because it avoids establishing connections to nodes that we are not interested in.

    However, the ambient discovery interface does not require capability discovery, which will lead to nodes having peers with unknown capabilities in their peer lists. -We work on a capability negotiation protocol which allows nodes to ask peers

    • for their complete list of capabilities, and
    • whether they support a specific capability

    We will investigate security implications, especially when sending full capability lists.

    NAT traversal​

    For NAT traversal, Waku v2 currently supports the port mapping protocols UPnP and NAT-PMP / PCP.

    In the future, we plan to add support for parts of ICE, e.g. STUN. -We do not plan to support TURN because TURN relays would introduce a centralized element. -A modified decentralized version of TURN featuring incentivization might be an option in the future; -strong peers could offer a relay service similar to TURN.

    There are plans to integrate more NAT traversal into discv5, in which we might participate. -So far, the only traversal technique supported by discv5 is nodes receiving their external IP address in pong messages.

    While NAT traversal is very important, adding more NAT traversal techniques is not a priority at the moment. -Nodes behind restrictive symmetric NAT setups cannot be discovered, but they can still discover peers in less restrictive setups. -While we wish to have as many nodes as possible to be discoverable via ambient peer discovery, two nodes behind a restrictive symmetric NAT can still exchange Waku v2 messages if they discovered a shared peer. -This is one of the nice resilience related properties of flooding based routing algorithms.

    For mobile nodes, which suffer from changing IP addresses and double NAT setups, we plan using the peer exchange protocol to ask peers for more peers. -Besides saving resources on resource restricted devices, this approach works as long as peers are in less restrictive environments.

    Conclusion and Future Prospects​

    Ambient peer discovery is an integral part of decentralized applications. It allows nodes to learn about peers in the network. -As of yet, Waku v2 supports DNS-based discovery and a slightly modified version of discv5. -We are working on further protocols, including a peer exchange protocol that allows resource restricted nodes to ask stronger peers for peer lists. -Further, we are working on adding capability discovery to our ambient discovery protocols, allowing nodes to find peers with desired properties.

    These protocols can be combined in a modular way and allow Waku v2 nodes to build a strong and resilient mesh network, -even if some discovery methods are not available in a given situation.

    We will investigate security properties of these discovery mechanisms with a focus on privacy and anonymity in a future post on this research log. -As an outlook we can already state that DHT approaches typically allow inferring information about the querying node. -Further, sending peer lists allows inferring the position of a node within the mesh, and by extension information about the node. -Waku v2 already provides some mitigation, because the mesh for transmitting actual messages, and the peer discovery network are separate. -To mitigate information leakage by transmitting peer lists, we plan to only reply with lists of peers that nodes do not use in their active meshes.

    References​

    ]]>     -     - - <![CDATA[Introducing nwaku]]> - https://vac.dev/rlog/introducing-nwaku - https://vac.dev/rlog/introducing-nwaku - Tue, 12 Apr 2022 10:00:00 GMT - - Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.

    Background​

    If you've been following our research log, -you'll know that many things have happened in the world of Waku v2 since our last general update. -In line with our long term goals, -we've introduced new protocols, -tweaked our existing protocols -and expanded our team. -We've also shown in a series of practical experiments that Waku v2 does indeed deliver on some of the theoretical advantages it was designed to have over its predecessor, Waku v1. -A sustainability and business workshop led to the formulation of a clearer vision for Vac as a team.

    From the beginning, our protocol development has been complemented by various client implementations of these protocols, -first in Nim, -but later also in JavaScript -and Go. -A follow-up post will clarify the purposes, similarities and differences between these three clients. -The Nim client, is our reference implementation, -developed by the research team in parallel with the specs -and building on a home-grown implementation of libp2p. -The Nim client is suitable to run as a standalone adaptive node, -managed by individual operators -or as an encapsulated service node in other applications. -This post looks at some recent developments within the Nim client.

    1. nim-waku is now known as nwaku​

    Pronounced NWHA-koo. -You may already have seen us refer to "nwaku" on Vac communication channels, -but it is now official: -The nim-waku Waku v2 client has been named nwaku. -Why? Well, we needed a recognizable name for our client that could easily be referred to in everyday conversations -and nim-waku just didn't roll off the tongue. -We've followed the example of the closely related nimbus project to find a punchier name -that explicitly links the client to both the Waku set of protocols and the Nim language.

    2. Improvements in stability and performance​

    The initial implementation of Waku v2 demonstrated how the suite of protocols can be applied -to form a generalized, peer-to-peer messaging network, -while addressing a wide range of adaptive requirements. -This allowed us to lift several protocol specifications from raw to draft status, -indicating that a reference implementation exists for each. -However, as internal dogfooding increased and more external applications started using nwaku, -we stepped up our focus on the client's stability and performance. -This is especially true where we want nwaku to run unsupervised in a production environment -without any degradation in the services it provides.

    Some of the more significant productionization efforts over the last couple of months included:

    1. Reworking the store implementation to maintain stable memory usage -while storing historical messages -and serving multiple clients querying history simultaneously. -Previously, a store node would see gradual service degradation -due to inefficient memory usage when responding to history queries. -Queries that often took longer than 8 mins now complete in under 100 ms.

    2. Improved peer management. -For example, filter nodes will now remove unreachable clients after a number of connection failures, -whereas they would previously keep accumulating dead peers.

    3. Improved disk usage. -nwaku nodes that persist historical messages on disk now manage their own storage size based on the --store-capacity. -This can significantly improve node start-up times.

    More stability issues may be addressed in future as nwaku matures, -but we've noticed a marked improvement in the reliability of running nwaku nodes. -These include environments where nwaku nodes are expected to run with a long uptime. -Vac currently operates two long-running fleets of nwaku nodes, wakuv2.prod and wakuv2.test, -for internal dogfooding and -to serve as experimental bootstrapping nodes. -Status has also recently deployed similar fleets for production and testing based on nwaku. -Our goal is to have nwaku be stable, performant and flexible enough -to be an attractive option for operators to run and maintain their own Waku v2 nodes. -See also the future work section below for more on our general goal of nwaku for operators.

    3. Improvements in interoperability​

    We've implemented several features that improve nwaku's usability in different environments -and its interoperability with other Waku v2 clients. -One major step forward here was adding support for both secure and unsecured WebSocket connections as libp2p transports. -This allows direct connectivity with js-waku -and paves the way for native browser usage. -We've also added support for parsing and resolving DNS-type multiaddrs, -i.e. multiaddress protocol schemes dns, dns4, dns6 and dnsaddr. -A nwaku node can now also be configured with its own IPv4 DNS domain name -allowing dynamic IP address allocation without impacting a node's reachability by its peers.

    4. Peer discovery​

    Peer discovery is the method by which nodes become aware of each other’s existence. -The question of peer discovery in a Waku v2 network has been a focus area since the protocol was first conceptualized. -Since then several different approaches to discovery have been proposed and investigated. -We've implemented three discovery mechanisms in nwaku so far:

    DNS-based discovery​

    nwaku nodes can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. -Our implementation is based on EIP-1459.

    GossipSub peer exchange​

    GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism -whereby a pruning peer may provide a pruned peer with a set of alternative peers -where it can connect to reform its mesh. -This is a very suitable mechanism to gradually discover more peers -from an initial connection to a small set of bootstrap peers. -It is enabled in a nwaku node by default.

    Waku Node Discovery Protocol v5​

    This is a DHT-based discovery mechanism adapted to store and relay node records. -Our implementation is based on Ethereum's Discovery v5 protocol -with some minor modifications to isolate our discovery network from that of Ethereum. -The decision to separate the Waku Discovery v5 network from Ethereum's was made on considerations of lookup efficiency. -This comes at a possible tradeoff in network resilience. -We are considering merging with the Ethereum Discovery v5 network in future, -or even implement a hybrid solution. -This post explains the decision and future steps.

    5. Spam protection using RLN​

    An early addition to our suite of protocols was an extension of 11/WAKU-RELAY -that provided spam protection using Rate Limiting Nullifiers (RLN). -The nwaku client now contains a working demonstration and integration of RLN relay. -Check out this tutorial to see the protocol in action using a toy chat application built on nwaku. -We'd love for people to join us in dogfooding RLN spam protection as part of our operator incentive testnet. -Feel free to join our Vac Discord server -and head to the #rln channel for more information.

    Future work​

    As we continue working towards our goal of a fully decentralized, generalized and censorship-resistant messaging protocol, -these are some of the current and future focus areas for nwaku:

    Reaching out to operators:​

    We are starting to push for operators to run and maintain their own Waku v2 nodes, -preferably contributing to the default Waku v2 network as described by the default pubsub topic (/waku/2/default-waku/proto). -Amongst other things, a large fleet of stable operator-run Waku v2 nodes will help secure the network, -provide valuable services to a variety of applications -and ensure the future sustainability of both Vac as a research organization and the Waku suite of protocols.

    We are targeting nwaku as the main option for operator-run nodes.
    -Specifically, we aim to provide through nwaku:

    1. a lightweight and robust Waku v2 client. -This client must be first in line to support innovative and new Waku v2 protocols, -but configurable enough to serve the adaptive needs of various operators.
    2. an easy-to-follow guide for operators to configure, -set up and maintain their own nodes
    3. a set of operator-focused tools to monitor and maintain a running node

    Better conversational security layer guarantees​

    Conversational security guarantees in Waku v2 are currently designed around the Status application. -Developers building their own applications on top of Waku would therefore -either have to reimplement a set of tools similar to Status -or build their own security solutions on the application layer above Waku. -We are working on a set of features built into Waku -that will provide the general security properties Waku users may desire -and do so in a modern and simple way. -This is useful for applications outside of Status that want similar security guarantees. -As a first step, we've already made good progress toward integrating noise handshakes as a key exchange mechanism in Waku v2.

    Protocol incentivization​

    We want to design incentivization around our protocols to encourage desired behaviors in the Waku network, -rewarding nodes providing costly services -and punishing adversarial actions. -This will increase the overall security of the network -and encourage operators to run their own Waku nodes. -In turn, the sustainability of Vac as an organization will be better guaranteed. -As such, protocol incentivization was a major focus in our recent Vac Sustainability and Business Workshop. -Our first step here is to finish integrating RLN relay into Waku -with blockchain interaction to manage members, -punish spammers -and reward spam detectors. -After this, we want to design monetary incentivization for providers of store, lightpush and filter services. -This may also tie into a reputation mechanism for service nodes based on a network-wide consensus on service quality. -A big challenge for protocol incentivization is doing it in a private fashion, -so we can keep similar metadata protection guarantees as the Waku base layer. -This ties into our focus on Zero Knowledge tech.

    Improved store capacity​

    The nwaku store currently serves as an efficient in-memory store for historical messages, -dimensioned by the maximum number of messages the store node is willing to keep. -This makes the nwaku store appropriate for keeping history over a short term -without any time-based guarantees, -but with the advantage of providing fast responses to history queries. -Some applications, such as Status, require longer-term historical message storage -with time-based dimensioning -to guarantee that messages will be stored for a specified minimum period. -Because of the relatively high cost of memory compared to disk space, -a higher capacity store, with time guarantees, should operate as a disk-only database of historical messages. -This is an ongoing effort.

    Multipurpose discovery​

    In addition to the three discovery methods already implemented in nwaku, -we are working on improving discovery on at least three fronts:

    Capability discovery:​

    Waku v2 nodes may be interested in peers with specific capabilities, for example:

    1. peers within a specific pubsub topic mesh,
    2. peers with store capability,
    3. store peers with x days of history for a specific content topic, etc.

    Capability discovery entails mechanisms by which such capabilities can be advertised and discovered/negotiated. -One major hurdle to overcome is the increased complexity of finding a node with specific capabilities within the larger network (a needle in a haystack). -See the original problem statement for more.

    Improvements in Discovery v5​

    Of the implemented discovery methods, -Discovery v5 best addresses our need for a decentralized and scalable discovery mechanism. -With the basic implementation done, -there are some improvements planned for Discovery v5, -including methods to increase security such as merging with the Ethereum Discovery v5 network, -introducing explicit NAT traversal -and utilizing topic advertisement. -The Waku v2 Discovery v5 Roadmap contains more details.

    Generalized peer exchange​

    nwaku already implements GossipSub peer exchange. -We now need a general request-response mechanism outside of GossipSub -by which a node may learn about other Waku v2 nodes -by requesting and receiving a list of peers from a neighbor. -This could, for example, be a suitable way for resource-restricted devices to request a stronger peer -to perform a random Discovery v5 lookup on their behalf -or simply to be informed of a subset of the peers known to that neighbor. -See this issue for more.

    This concludes a general outline of some of the main recent developments in the nwaku client -and a summary of the current and future focus areas. -Much more is happening behind the scenes, of course, -so for more information, or to join the conversation, -feel free to join our Vac Discord server -or to check out the nwaku repo on Github. -You can also view the changelog for past releases here.

    References​

    ]]>     -     - - <![CDATA[Opinion: Pseudo-ethics in the Surveillance Tech Industry]]> - https://vac.dev/rlog/ethics-surveillance-tech - https://vac.dev/rlog/ethics-surveillance-tech - Fri, 03 Dec 2021 10:00:00 GMT - - A look at typical ethical shortfalls in the global surveillance tech industry.

    This is an opinion piece by pseudonymous contributor, circe.

    Preface​

    The Vac team aims to provide a public good in the form of freely available, open source tools and protocols for decentralized communication. -As such, we value our independence and the usefulness of our protocols for a wide range of applications. -At the same time, we realize that all technical development, including ours, has a moral component. -As a diverse team we are guided by a shared devotion to the principles of human rights and liberty. -This explains why we place such a high premium on security, censorship-resistance and privacy - -a stance we share with the wider Status Network. -The post below takes a different approach from our usual more technical analyses, -by starting to peel back the curtain on the ethical shortfalls of the global surveillance tech industry.

    Spotlight on an industry​

    Apple's announcement of their lawsuit against Israel's NSO Group -marks the latest in a series of recent setbacks for the surveillance tech company. -In early November, the United States blacklisted the firm, -citing concerns about the use of their spyware by foreign governments targeting civilians such as "journalists, businesspeople, activists" and more. -The company is already embroiled in a lawsuit with Whatsapp -over their exploit of the chat app's video calling service to install malware on target devices. -NSO Group's most infamous product, Pegasus, operates as a hidden exploit installed on victims' mobile phones, -sometimes without even requiring as much as an unguarded click on a malicious link. -It has the potential to lay bare, and report to its owners, everything within the reach of the infected device. -For most people this amounts to a significant portion of their private lives and thoughts. -Pegasus can read your private messages (even encrypted), collect your passwords, record calls, track your location and access your device's microphone and camera. -No activity or application on an infected phone would be hidden.

    The latest controversies are perhaps less because of the novelty of the revelations - -the existence of Pegasus has been known to civil activists since at least 2016. -Rather, the public was reminded again of the potential scope of surveillance tech -in the indiscriminate use of Pegasus on private citizens. -This has far-reaching implications for human freedoms worldwide. -Earlier this year, a leaked list of over 50,000 targets, or possible targets, of Pegasus included -the phone numbers of human rights advocates, independent journalists, lawyers and political activists. -This should have come as no surprise. -The type of autocratically inclined agents, and governments, who would venture to buy and use such invasive cyber-arms often target those they find politically inconvenient. -Pegasus, and similar technologies, simply extend the reach and capacity of such individuals and governments - -no border or distance, no political rank or social advantage, no sanctity of profession or regard for dignity, -provide any indemnity from becoming a victim. -Your best hope is to remain uninteresting enough to escape consideration.

    The NSO Group has, of course, denied allegations of culpability and questions the authenticity of the list. -At this stage, the latter is almost beside the point: -Amnesty International's cybersecurity team, Security Lab, did find forensic evidence of Pegasus on the phones of several volunteers whose numbers appeared on the original list, -including those of journalists and human rights activists. -(Security Lab has since opened up their infection finding tool to the public.) -French intelligence has similarly inspected and confirmed infection of at least three devices belonging to journalists. -The phones of several people who were close to the Saudi-American journalist, Jamal Khashoggi, were confirmed hacked -both before and after Khashoggi's brutal murder at the Saudi embassy in Istanbul in 2018. -More reports of confirmed Pegasus hacks are still published with some regularity. -It is now an open secret that many authoritarian governments have bought Pegasus. -It's not difficult to extrapolate from existing reports and such clients' track records -what the potential injuries to human freedoms are that they can inflict with access to such a powerful cyberweapon.

    A typical response​

    NSO's response to the allegations follows a textbook approach -of avoiding earnest ethical introspection on the manufacturing, and selling, of cyber-arms. -Firstly, shift ethical responsibility to a predetermined process, a list of checkboxes of your own making. -The Group, for example, claims to sell only to "vetted governments", following a classification process -of which they have now published some procedural details but no tangible criteria. -The next step is to reaffirm continuously, and repetitively, your dedication to the legal combat against crime, -"legitimate law enforcement agencies" (note the almost tautological phrasing), -adherence to international arms trade laws, -compliance clauses in customer contracts, etc. -Thirdly, having been absolved of any moral suspicions that might exist about product and process, -from conception to engineering to trade, -distance yourself from the consequences of its use in the world. -"NSO does not operate its technology, does not collect, nor possesses, nor has any access to any kind of data of its customers." -It is interesting that directly after this statement they claim with contradictory confidence that -their "technology was not associated in any way with the heinous murder of Jamal Khashoggi". -The unapologetic tone seems hardly appropriate when the same document confirms that the Group had to -shut down customers' systems due to "confirmed misuse" and have had to do so "multiple times" in the past. -Given all this, the response manages to evade any serious interrogation of the "vetting" process itself, -which forced the company to reject "approximately 15% of potential new opportunities for Pegasus" in one year. -Courageous.

    We have heard this all before. -There exists a multi-billion dollar industry of private companies and engineering firms thriving on proceeds from -selling surveillance tools and cyber-arms to dubious agencies and foreign governments. -In turn, the most power-hungry and oppressive regimes often rely on such technological innovations - -for which they lack the in-country engineering expertise - -to maintain control, suppress uprisings, intimidate opposing journalists, and track their citizens. -It's a lucrative business opportunity, and resourceful companies have sprung up everywhere to supply this demand, -often in countries where citizens, including employees of the company, would be horrified if they were similarly subject to the oppressions of their own products. -When, in 2014, Italy's HackingTeam were pulsed by the United Nations about their (then alleged) selling of spyware to Sudan, -which would have been a contravention of the UN's weapon export ban, -they simply replied that their product was not controlled as a weapon and therefore not subject to such scrutiny. -They remained within their legal bounds, technically. -Furthermore, they similarly shifted ethical responsibility to external standards of legitimacy, -claiming their "software is not sold to governments that are blacklisted by the EU, the US, NATO, and similar international organizations". -When the company themselves were hacked in 2015, -revelations (confirmations, that is) of widespread misuse by repressive governments were damaging enough to force them to disappear and rebrand as Memento Labs. -Their website boasts an impressive list of statutes, regulations, procedures, export controls and legal frameworks, -all of which the rebranded hackers proudly comply with. -Surely no further ethical scrutiny is necessary?

    Ethics != the law​

    The law is trailing behind​

    Such recourse to the legality of your action as ethical justification is moot for several reasons. -The first is glaringly obvious - -our laws are ill-equipped to address the implications of modern technology. -Legal systems are a cumbersome inheritance built over generations. -This is especially true of the statutes and regulations governing international trade, behind which these companies so often hide. -Our best legal systems are trailing miles behind the technology for which we seek guidelines. -Legislators are still struggling to make sense of technologies like face recognition, -the repercussions of smart devices acting "on their own" and biases in algorithms. -To claim you are performing ethical due diligence by resorting to an outdated and incomplete system of legal codes is disingenuous.

    The law depends on ethics​

    The second reason is more central to my argument, -and an important flaw in these sleight of hand justifications appearing from time to time in the media. -Ethics can in no way be confused as synonymous with legality or legitimacy. -These are incommensurable concepts. -In an ideal world, of course, the law is meant to track the minimum standards of ethical conduct in a society. -Laws are often drafted exactly from some ethical, and practical, impulse to minimize harmful conduct -and provide for corrective and punitive measures where transgressions do occur. -The law, however, has a much narrower scope than ethics. -It can be just or unjust. -In fact, it is in need of ethics to constantly reform. -Ethics and values are born out of collective self-reflection. -It develops in our conversation with ourselves and others about the type of society we strive for. -As such, an ethical worldview summarizes our deepest intuitions about how we should live and measure our impact on the world. -For this reason, ethics is primarily enforced by social and internal pressures, not legal boundaries - -our desire to do what ought to be done, however we define that. -Ethics is therefore a much grander scheme than global legal systems -and the diplomatic frameworks that grants legitimacy to governments. -These are but one limited outflow of the human aspiration to form societies in accordance with our ideologies and ethics.

    International law is vague and exploitable​

    Of course, the cyber-arms trade has a favorite recourse, international law, which is even more limited. -Since such products are seldomly sold to governments and agencies within the country of production, -it enables a further distancing from consequences. -Many private surveillance companies are based in fairly liberal societies with (seemingly) strict emphases on human rights in their domestic laws. -International laws are much more complicated - for opportunists a synonym for "more grey areas in which to hide". -Company conduct can now be governed, and excused, by a system that follows -the whims of autocrats with exploitative intent and vastly different ethical conceptions from the company's purported aims. -International law, and the ways it is most often enforced by way of, say, UN-backed sanctions, -have long been shaped by the compromises of international diplomacy. -To be blunt: these laws are weak and subject to exactly the sort of narrow interests behind which mercenaries have always hidden. -The surveillance tech industry is no exception.

    Conclusion​

    My point is simple: -selling cyber-arms with the potential to become vast tools of oppression to governments and bodies with blatant histories of human rights violations, -and all but the publicly announced intention to continue operating in this way, -is categorically unconscionable. -This seems obvious no matter what ethics system you argue from, -provided it harbors any consideration for human dignity and freedom. -It is a sign of poor moral discourse that such recourses to law and legitimacy are often considered synonymous with ethical justification. -"I have acted within the bounds of law", "We supply only to legitimate law enforcement agencies", etc. are no substitutes. -Ethical conduct requires an honest evaluation of an action against some conception of "the good", -however you define that. -Too often the surveillance tech industry precisely sidesteps this question, -both in internal processes and external rationalisations to a concerned public.

    John Locke, he of the life-liberty-and-property, articulated the idea that government exists solely through the consent of the governed. -Towards the end of the 17th century, he wrote in his Second Treatise on Civil Government, -"[w]henever legislators endeavor to take away, -and destroy the property of the people, or to reduce them to slavery under arbitrary power, -they put themselves in a state of war with the people, who are thereupon absolved from any further obedience". -The inference is straightforward and humanist in essence: -legitimacy is not something that is conferred by governments and institutions. -Rather, they derive their legitimacy from us, their citizens, holding them to standards of ethics and societal ideals. -This legitimacy only remains in tact as long as this mandate is honored and continuously extended by a well-informed public. -This is the principle of informed consent on which all reciprocal ethics is based.

    The surveillance tech industry may well have nothing more or less noble in mind than profit-making within legal bounds -when developing and selling their products. -However, when such companies are revealed again and again to have supplied tools of gross human rights violations to known human rights violators, -they will do well to remember that ethics always precedes requirements of legality and legitimacy. -It is a fallacy to take normative guidance from the concept of "legitimacy" -if the concept itself depends on such normative guidelines for definition. -Without examining the ethical standards by which institutions, governments, and laws, were created, -no value-judgements about their legitimacy can be made. -Hiding behind legal compliance as substitute for moral justification is not enough. -Targets of increasingly invasive governmental snooping are too often chosen precisely to suppress the mechanisms from which the legitimacy of such governments flow - -the consent of ordinary civilians. -Free and fair elections, free speech, free media, freedom of thought are all at risk.

    References​

    ]]>     -     - - <![CDATA[Waku v1 vs Waku v2: Bandwidth Comparison]]> - https://vac.dev/rlog/waku-v1-v2-bandwidth-comparison - https://vac.dev/rlog/waku-v1-v2-bandwidth-comparison - Wed, 03 Nov 2021 10:00:00 GMT - - A local comparison of bandwidth profiles showing significantly improved scalability in Waku v2 over Waku v1.

    Background​

    The original plan for Waku v2 suggested theoretical improvements in resource usage over Waku v1, -mainly as a result of the improved amplification factors provided by GossipSub. -In its turn, Waku v1 proposed improvements over its predecessor, Whisper.

    Given that Waku v2 is aimed at resource restricted environments, -we are specifically interested in its scalability and resource usage characteristics. -However, the theoretical performance improvements of Waku v2 over Waku v1, -has never been properly benchmarked and tested.

    Although we're working towards a full performance evaluation of Waku v2, -this would require significant planning and resources, -if it were to simulate "real world" conditions faithfully and measure bandwidth and resource usage across different network connections, -robustness against attacks/losses, message latencies, etc. -(There already exists a fairly comprehensive evaluation of GossipSub v1.1, -on which 11/WAKU2-RELAY is based.)

    As a starting point, -this post contains a limited and local comparison of the bandwidth profile (only) between Waku v1 and Waku v2. -It reuses and adapts existing network simulations for Waku v1 and Waku v2 -and compares bandwidth usage for similar message propagation scenarios.

    Theoretical improvements in Waku v2​

    Messages are propagated in Waku v1 using flood routing. -This means that every peer will forward every new incoming message to all its connected peers (except the one it received the message from). -This necessarily leads to unnecessary duplication (termed amplification factor), -wasting bandwidth and resources. -What's more, we expect this effect to worsen the larger the network becomes, -as each connection will receive a copy of each message, -rather than a single copy per peer.

    Message routing in Waku v2 follows the libp2p GossipSub protocol, -which lowers amplification factors by only sending full message contents to a subset of connected peers. -As a Waku v2 network grows, each peer will limit its number of full-message ("mesh") peerings - -libp2p suggests a maximum of 12 such connections per peer. -This allows much better scalability than a flood-routed network. -From time to time, a Waku v2 peer will send metadata about the messages it has seen to other peers ("gossip" peers).

    See this explainer for a more detailed discussion.

    Methodology​

    The results below contain only some scenarios that provide an interesting contrast between Waku v1 and Waku v2. -For example, star network topologies do not show a substantial difference between Waku v1 and Waku v2. -This is because each peer relies on a single connection to the central node for every message, -which barely requires any routing: -each connection receives a copy of every message for both Waku v1 and Waku v2. -Hybrid topologies similarly show only a difference between Waku v1 and Waku v2 for network segments with mesh-like connections, -where routing decisions need to be made.

    For this reason, the following approach applies to all iterations:

    1. Simulations are run locally. -This limits the size of possible scenarios due to local resource constraints, -but is a way to quickly get an approximate comparison.
    2. Nodes are treated as a blackbox for which we only measure bandwidth, -using an external bandwidth monitoring tool. -In other words, we do not consider differences in the size of the envelope (for v1) or the message (for v2).
    3. Messages are published at a rate of 50 new messages per second to each network, -except where explicitly stated otherwise.
    4. Each message propagated in the network carries 8 bytes of random payload, which is encrypted. -The same symmetric key cryptographic algorithm (with the same keys) are used in both Waku v1 and v2.
    5. Traffic in each network is generated from 10 nodes (randomly-selected) and published in a round-robin fashion to 10 topics (content topics for Waku v2). -In practice, we found no significant difference in average bandwidth usage when tweaking these two parameters (the number of traffic generating nodes and the number of topics).
    6. Peers are connected in a decentralized full mesh topology, -i.e. each peer is connected to every other peer in the network. -Waku v1 is expected to flood all messages across all existing connections. -Waku v2 gossipsub will GRAFT some of these connections for full-message peerings, -with the rest being gossip-only peerings.
    7. After running each iteration, we verify that messages propagated to all peers (comparing the number of published messages to the metrics logged by each peer).

    For Waku v1, nodes are configured as "full" nodes (i.e. with full bloom filter), -while Waku v2 nodes are relay nodes, all subscribing and publishing to the same PubSub topic.

    Network size comparison​

    Iteration 1: 10 nodes​

    Let's start with a small network of 10 nodes only and see how Waku v1 bandwidth usage compares to that of Waku v2. -At this small scale we don't expect to see improved bandwidth usage in Waku v2 over Waku v1, -since all connections, for both Waku v1 and Waku v2, will be full-message connections. -The number of connections is low enough that Waku v2 nodes will likely GRAFT all connections to full-message peerings, -essentially flooding every message on every connection in a similar fashion to Waku v1. -If our expectations are confirmed, it helps validate our methodology, -showing that it gives more or less equivalent results between Waku v1 and Waku v2 networks.

    Sure enough, the figure shows that in this small-scale setup, -Waku v1 actually has a lower per-peer bandwidth usage than Waku v2. -One reason for this may be the larger overall proportion of control messages in a gossipsub-routed network such as Waku v2. -These play a larger role when the total network traffic is comparatively low, as in this iteration. -Also note that the average bandwidth remains more or less constant as long as the rate of published messages remains stable.

    Iteration 2: 30 nodes​

    Now, let's run the same scenario for a larger network of highly-connected nodes, this time consisting of 30 nodes. -At this point, the Waku v2 nodes will start pruning some connections to limit the number of full-message peerings (to a maximum of 12), -while the Waku v1 nodes will continue flooding messages to all connected peers. -We therefore expect to see a somewhat improved bandwidth usage in Waku v2 over Waku v1.

    Bandwidth usage in Waku v2 has increased only slightly from the smaller network of 10 nodes (hovering between 2000 and 3000 kbps). -This is because there are only a few more full-message peerings than before. -Compare this to the much higher increase in bandwidth usage for Waku v1, which now requires more than 4000 kbps on average.

    Iteration 3: 50 nodes​

    For an even larger network of 50 highly connected nodes, -the divergence between Waku v1 and Waku v2 is even larger. -The following figure shows comparative average bandwidth usage for a throughput of 50 messages per second.

    Average bandwidth usage (for the same message rate) has remained roughly the same for Waku v2 as it was for 30 nodes, -indicating that the number of full-message peerings per node has not increased.

    Iteration 4: 85 nodes​

    We already see a clear trend in the bandwidth comparisons above, -so let's confirm by running the test once more for a network of 85 nodes. -Due to local resource constraints, the effective throughput for Waku v1 falls to below 50 messages per second, -so the v1 results below have been normalized and are therefore approximate. -The local Waku v2 simulation maintains the message throughput rate without any problems.

    Iteration 5: 150 nodes​

    Finally, we simulate message propagation in a network of 150 nodes. -Due to local resource constraints, we run this simulation at a lower rate - -35 messages per second - -and for a shorter amount of time.

    Notice how the Waku v1 bandwidth usage is now more than 10 times worse than that of Waku v2. -This is to be expected, as each Waku v1 node will try to flood each new message to 149 other peers, -while the Waku v2 nodes limit their full-message peerings to no more than 12.

    Discussion​

    Let's summarize average bandwidth growth against network growth for a constant message propagation rate. -Since we are particularly interested in how Waku v1 compares to Waku v2 in terms of bandwidth usage, -the results are normalised to the Waku v2 average bandwidth usage for each network size.

    Extrapolation is a dangerous game, -but it's safe to deduce that the divergence will only grow for even larger network topologies. -Although control signalling contributes more towards overall bandwidth for Waku v2 networks, -this effect becomes less noticeable for larger networks. -For network segments with more than ~18 densely connected nodes, -the advantage of using Waku v2 above Waku v1 becomes clear.

    Network traffic comparison​

    The analysis above controls the average message rate while network size grows. -In reality, however, active users (and therefore message rates) are likely to grow in conjunction with the network. -This will have an effect on bandwidth for both Waku v1 and Waku v2, though not in equal measure. -Consider the impact of an increasing rate of messages in a network of constant size:

    The rate of increase in bandwidth for Waku v2 is slower than that for Waku v1 for a corresponding increase in message propagation rate. -In fact, for a network of 30 densely-connected nodes, -if the message propagation rate increases by 1 per second, -Waku v1 requires an increased average bandwidth of almost 70kbps at each node. -A similar traffic increase in Waku v2 requires on average 40kbps more bandwidth per peer, just over half that of Waku v1.

    Conclusions​

    • Waku v2 scales significantly better than Waku v1 in terms of average bandwidth usage, -especially for densely connected networks.
    • E.g. for a network consisting of 150 or more densely connected nodes, -Waku v2 provides more than 10x better average bandwidth usage rates than Waku v1.
    • As the network continues to scale, both in absolute terms (number of nodes) and in network traffic (message rates) the disparity between Waku v2 and Waku v1 becomes even larger.

    Future work​

    Now that we've confirmed that Waku v2's bandwidth improvements over its predecessor matches theory, -we can proceed to a more in-depth characterisation of Waku v2's resource usage. -Some questions that we want to answer include:

    • What proportion of Waku v2's bandwidth usage is used to propagate payload versus bandwidth spent on control messaging to maintain the mesh?
    • To what extent is message latency (time until a message is delivered to its destination) affected by network size and message rate?
    • How reliable is message delivery in Waku v2 for different network sizes and message rates?
    • What are the resource usage profiles of other Waku v2 protocols (e.g.12/WAKU2-FILTER and 19/WAKU2-LIGHTPUSH)?

    Our aim is to get ever closer to a "real world" understanding of Waku v2's performance characteristics, -identify and fix vulnerabilities -and continually improve the efficiency of our suite of protocols.

    References​

    ]]>     -     - - <![CDATA[[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging]]> - https://vac.dev/rlog/waku-v2-ethereum-coscup - https://vac.dev/rlog/waku-v2-ethereum-coscup - Fri, 06 Aug 2021 12:00:00 GMT - - Learn more about Waku v2, its origins, goals, protocols, implementation and ongoing research. Understand how it is used and how it can be useful for messaging in Ethereum.

    This is the English version of a talk originally given in Chinese at COSCUP in Taipei.

    video recording with Chinese and English subtitles.

    Introduction​

    Hi everyone!

    Today I'll talk to you about Waku v2. What it is, what problems it is solving, -and how it can be useful for things such as messaging in Ethereum. First, let me -start with some brief background.

    Brief history and background​

    Back when Ethereum got started, there used to be this concept of the "holy -trinity". You had Ethereum for compute/consensus, Swarm for storage, and Whisper -for messaging. This is partly where the term Web3 comes from.

    Status started out as an app with the goal of being a window onto Ethereum and -a secure messenger. As one of the few, if not the only, apps using Whisper in -production, not to mention on a mobile phone, we quickly realized there were -problems with the underlying protocols and infrastructure. Protocols such as -Whisper weren't quite ready for prime time yet when it came to things such as -scalability and working in the real world.

    As we started addressing some of these challenges, and moved from app -developement to focusing on protocols, research and infrastructure, we created -Vac. Vac is an r&d unit doing protocol research focused on creating modular p2p -messaging protocols for private, secure, censorship resistant communication.

    I won't go into too much detail on the issues with Whisper, if you are -interested in this check out this talk -here or this -article.

    In a nutshell, we forked Whisper to address immediate shortcomings and this -became Waku v1. Waku v2 is complete re-thought implementation from scratch on top -of libp2p. This will be the subject of today's talk.

    Waku v2​

    Overview​

    Waku v2 is a privacy-preserving peer-to-peer messaging protocol for resource -restricted devices. We can look at Waku v2 as several things:

    • Set of protocols
    • Set of implementations
    • Network of nodes

    Let's first look at what the goals are.

    Goals​

    Waku v2 provides a PubSub based messaging protocol with the following -characteristics:

    1. Generalized messaging. Applications that require a messaging protocol to -communicate human to human, machine to machine, or a mix.
    2. Peer-to-peer. For applications that require a p2p solution.
    3. Resource restricted. For example, running with limited bandwidth, being -mostly-offline, or in a browser.
    4. Privacy. Applications that have privacy requirements, such as pseudonymity, -metadata protection, etc.

    And to provide these properties in a modular fashion, where applications can -choose their desired trade-offs.

    Protocols​

    Waku v2 consists of several protocols. Here we highlight a few of the most -important ones:

    • 10/WAKU2 - main specification, details how all the pieces fit together
    • 11/RELAY - thin layer on top of GossipSub for message dissemination
    • 13/STORE - fetching of historical messages
    • 14/MESSAGE - message payload

    This is the recommended subset for a minimal Waku v2 client.

    In addition to this there are many other types of specifications at various -stages of maturity, such as: content based filtering, bridge mode to Waku v1, -JSON RPC API, zkSNARKS based spam protection with RLN, accounting and -settlements with SWAP, fault-tolerant store nodes, recommendations around topic -usage, and more.

    See https://rfc.vac.dev/ for a full overview.

    Implementations​

    Waku v2 consists of multiple implementations. This allows for client diversity, -makes it easier to strengthen the protocols, and allow people to use Waku v2 in -different contexts.

    • nim-waku - the reference client written in Nim, most full-featured.
    • js-waku - allow usage of Waku v2 from browsers, focus on interacting with dapps.
    • go-waku - subset of Waku v2 to ease integration into the Status app.

    Testnet Huilong and dogfooding​

    In order to test the protocol we have setup a testnet across all implementations -called Huilong. Yes, that's the Taipei subway station!

    Among us core devs we have disabled the main #waku Discord channel used for -development, and people run their own node connected to this toy chat application.

    Feel free to join and say hi! Instructions can be found here:

    Research​

    While Waku v2 is being used today, we are actively researching improvements. -Since the design is modular, we can gracefully introduce new capabilities. Some -of these research areas are:

    • Privacy-preserving spam protection using zkSNARKs and RLN
    • Accounting and settlement of resource usage to incentivize nodes to provide services with SWAP
    • State synchronization for store protocol to make it easier to run a store node without perfect uptime
    • Better node discovery
    • More rigorous privacy analysis
    • Improving interaction with wallets and dapp

    Use cases​

    Let's look at where Waku v2 is and can be used.

    Prelude: Topics in Waku v2​

    To give some context, there are two different types of topics in Waku v2. One is -a PubSub topic, for routing. The other is a content topic, which is used for -content based filtering. Here's an example of the default PubSub topic:

    /waku/2/default-waku/proto

    This is recommended as it increases privacy for participants and it is stored by -default, however this is up to the application.

    The second type of topic is a content topic, which is application specific. For -example, here's the content topic used in our testnet:

    /toychat/2/huilong/proto

    For more on topics, see https://rfc.vac.dev/waku/informational/23/topics

    Status app​

    In the Status protocol, content topics - topics in Whisper/Waku v1 - are used for several things:

    • Contact code topic to discover X3DH bundles for perfect forward secrecy
      • Partitioned into N (currently 5000) content topics to balance privacy with efficiency
    • Public chats correspond to hash of the plaintext name
    • Negotiated topic for 1:1 chat with DHKE derived content topic

    See more here http://rfc.vac.dev/status/deprecated/waku-usage

    Currently, Status app is in the process of migrating to and testing Waku v2.

    DappConnect: Ethereum messaging​

    It is easy to think of Waku as being for human messaging, since that's how it is -primarily used in the Status app, but the goal is to be useful for generalized -messaging, which includes Machine-To-Machine (M2M) messaging.

    Recall the concept of the holy trinity with Ethereum/Swarm/Whisper and Web3 that -we mentioned in the beginning. Messaging can be used as a building block for -dapps, wallets, and users to communicate with each other. It can be used for -things such as:

    • Multisig and DAO vote transactions only needing one on-chain operation
    • Giving dapps ability to send push notifications to users
    • Giving users ability to directly respond to requests from dapps
    • Decentralized WalletConnect
    • Etc

    Basically anything that requires communication and doesn't have to be on-chain.

    WalletConnect v2​

    WalletConnect is an open protocol for connecting dapps to wallets with a QR -code. Version 2 is using Waku v2 as a communication channel to do so in a -decentralized and private fashion.

    See for more: https://docs.walletconnect.org/v/2.0/tech-spec

    WalletConnect v2 is currently in late alpha using Waku v2.

    More examples​

    • Gasless voting and vote aggregation off-chain
    • Dapp games using Waku as player discovery mechanism
    • Send encrypted message to someone with an Ethereum key
    • <Your dapp here>

    These are all things that are in progress / proof of concept stage.

    Contribute​

    We'd love to see contributions of any form!

    Conclusion​

    In this talk we've gone over the original vision for Web3 and how Waku came to -be. We've also looked at what Waku v2 aims to do. We looked at its protocols, -implementations, the current testnet as well as briefly on some ongoing -research for Vac.

    We've also looked at some specific use cases for Waku. First we looked at how -Status uses it with different topics. Then we looked at how it can be useful for -messaging in Ethereum, including for things like WalletConnect.

    I hope this talk gives you a better idea of what Waku is, why it exists, and -that it inspires you to contribute, either to Waku itself or by using it in your -own project!

    ]]>     -     - - <![CDATA[Presenting JS-Waku: Waku v2 in the Browser]]> - https://vac.dev/rlog/presenting-js-waku - https://vac.dev/rlog/presenting-js-waku - Fri, 04 Jun 2021 12:00:00 GMT - - JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!

    For the past 3 months, we have been working on bringing Waku v2 to the browser. -Our aim is to empower dApps with Waku v2, and it led to the creation of a new library. -We believe now is good time to introduce it!

    Waku v2​

    First, let's review what Waku v2 is and what problem it is trying to solve.

    Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised -communications on resource restricted devices (i.e., mobile phones).

    The Status chat feature was initially built over Whisper. -However, Whisper has a number of caveats which makes it inefficient for mobile phones. -For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.

    To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced. -To further enable web and restricted resource environments, Waku v2 was created based on libp2p. -The migration of the Status chat feature to Waku v2 is currently in progress.

    We see the need of such solution in the broader Ethereum ecosystem, beyond Status. -This is why we are building Waku v2 as a decentralised communication platform for all to use and build on. -If you want to read more about Waku v2 and what it aims to achieve, -checkout What's the Plan for Waku v2?.

    Since last year, we have been busy defining and implementing Waku v2 protocols in nim-waku, -from which you can build wakunode2. -Wakunode2 is an adaptive and modular Waku v2 node, -it allows users to run their own node and use the Waku v2 protocols they need. -The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.

    Waku v2 in the browser​

    We believe that dApps and wallets can benefit from the Waku network in several ways. -For some dApps, it makes sense to enable peer-to-peer communications. -For others, machine-to-machine communications would be a great asset. -For example, in the case of a DAO, -Waku could be used for gas-less voting. -Enabling the DAO to notify their users of a new vote, -and users to vote without interacting with the blockchain and spending gas.

    Murmur was the first attempt to bring Whisper to the browser, -acting as a bridge between devp2p and libp2p. -Once Waku v2 was started and there was a native implementation on top of libp2p, -a chat POC was created to demonstrate the potential of Waku v2 -in web environment. -It showed how using js-libp2p with few modifications enabled access to the Waku v2 network. -There was still some unresolved challenges. -For example, nim-waku only support TCP connections which are not supported by browser applications. -Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets, -which in turn could connect to wakunode2 via TCP.

    However, to enable dApp and Wallet developers to easily integrate Waku in their product, -we need to give them a library that is easy to use and works out of the box: -introducing JS-Waku.

    JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network. -It is available right now on npm:

    npm install js-waku.

    As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.

    Key Waku v2 protocols are already available: -message, store, relay and light push, -enabling your dApp to:

    • Send and receive near-instant messages on the Waku network (relay),
    • Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
    • Send messages with confirmations (light push).

    JS-Waku needs to operate in the same context from which Waku v2 was born: -a restricted environment were connectivity or uptime are not guaranteed; -JS-Waku brings Waku v2 to the browser.

    Achievements so far​

    We focused the past month on developing a ReactJS Chat App. -The aim was to create enough building blocks in JS-Waku to enable this showcase web app that -we now use for dogfooding purposes.

    Most of the effort was on getting familiar with the js-libp2p library -that we heavily rely on. -JS-Waku is the second implementation of Waku v2 protocol, -so a lot of effort on interoperability was needed. -For example, to ensure compatibility with the nim-waku reference implementation, -we run our tests against wakunode2 as part of the CI.

    This interoperability effort helped solidify the current Waku v2 specifications: -By clarifying the usage of topics -(#327, #383), -fix discrepancies between specs and nim-waku -(#418, #419) -and fix small nim-waku & nim-libp2p bugs -(#411, #439).

    To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes. -A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser. -Unfortunately websocket support is not yet available in nim-libp2p so -we ended up deploying websockify alongside wakunode2 instances.

    As we built the web chat app, -we were able to fine tune the API to provide a simple and succinct interface. -You can start a node, connect to other nodes and send a message in less than ten lines of code:

    import { Waku } from 'js-waku'

    const waku = await Waku.create({})

    const nodes = await getStatusFleetNodes()
    await Promise.all(nodes.map((addr) => waku.dial(addr)))

    const msg = WakuMessage.fromUtf8String(
      'Here is a message!',
      '/my-cool-app/1/my-use-case/proto',
    )
    await waku.relay.send(msg)

    We have also put a bounty at 0xHack for using JS-Waku -and running a workshop. -We were thrilled to have a couple of hackers create new software using our libraries. -One of the projects aimed to create a decentralised, end-to-end encrypted messenger app, -similar to what the ETH-DM protocol aims to achieve. -Another project was a decentralised Twitter platform. -Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.

    As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network -while preserving this out of the box experience.

    What's next?​

    Next, we are directing our attention towards Developer Experience. -We already have documentation available but we want to provide more: -Tutorials, various examples -and showing how JS-Waku can be used with Web3.

    By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets. -We think JS-Waku builds a strong case for machine-to-machine (M2M) communications. -The first use cases we are looking into are dApp notifications: -Enabling dApp to notify their user directly in their wallets! -Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified -of events such as DAO voting.

    We already have some POC in the pipeline to enable voting and polling on the Waku network, -allowing users to save gas by not broadcasting each individual vote on the blockchain.

    To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples -of signing, validating, encrypting and decrypting messages using Web3. -Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.

    As you can read, we have grand plans for JS-Waku and Waku v2. -There is a lot to do, and we would love some help so feel free to -check out the new role in our team: -js-waku: Wallet & Dapp Integration Developer. -We also have a number of positions open to work on Waku protocol and nim-waku.

    If you are as excited as us by JS-Waku, why not build a dApp with it? -You can find documentation on the npmjs page.

    Whether you are a developer, you can come chat with us using WakuJS Web Chat -or chat2. -You can get support in #dappconnect-support on Vac Discord or Telegram. -If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.

    ]]>     -     - - <![CDATA[Privacy-preserving p2p economic spam protection in Waku v2]]> - https://vac.dev/rlog/rln-relay - https://vac.dev/rlog/rln-relay - Fri, 05 Mar 2021 12:00:00 GMT - - This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.

    Introduction​

    This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol2 through Rate-Limiting Nullifiers3 4 or RLN for short.

    Let me give a little background about Waku(v2)1. Waku is a privacy-preserving peer-to-peer (p2p) messaging protocol for resource-restricted devices. Being p2p means that Waku relies on No central server. Instead, peers collaboratively deliver messages in the network. Waku uses GossipSub16 as the underlying routing protocol (as of the writeup of this post). At a high level, GossipSub is based on publisher-subscriber architecture. That is, peers, congregate around topics they are interested in and can send messages to topics. Each message gets delivered to all peers subscribed to the topic. In GossipSub, a peer has a constant number of direct connections/neighbors. In order to publish a message, the author forwards its message to a subset of neighbors. The neighbors proceed similarly till the message gets propagated in the network of the subscribed peers. The message publishing and routing procedures are part of the Waku Relay17 protocol. -

    Figure 1: An overview of privacy-preserving p2p economic spam protection in Waku v2 RLN-Relay protocol.

    What do we mean by spamming?​

    In centralized messaging systems, a spammer usually indicates an entity that uses the messaging system to send an unsolicited message (spam) to large numbers of recipients. However, in Waku with a p2p architecture, spam messages not only affect the recipients but also all the other peers involved in the routing process as they have to spend their computational power/bandwidth/storage capacity on processing spam messages. As such, we define a spammer as an entity that uses the messaging system to publish a large number of messages in a short amount of time. The messages issued in this way are called spam. In this definition, we disregard the intention of the spammer as well as the content of the message and the number of recipients.

    Possible Solutions​

    Has the spamming issue been addressed before? Of course yes! Here is an overview of the spam protection techniques with their trade-offs and use-cases. In this overview, we distinguish between protection techniques that are targeted for centralized messaging systems and those for p2p architectures.

    Centralized Messaging Systems​

    In traditional centralized messaging systems, spam usually signifies unsolicited messages sent in bulk or messages with malicious content like malware. Protection mechanisms include

    • authentication through some piece of personally identifiable information e.g., phone number
    • checksum-based filtering to protect against messages sent in bulk
    • challenge-response systems
    • content filtering on the server or via a proxy application

    These methods exploit the fact that the messaging system is centralized and a global view of the users' activities is available based on which spamming patterns can be extracted and defeated accordingly. Moreover, users are associated with an identifier e.g., a username which enables the server to profile each user e.g., to detect suspicious behavior like spamming. Such profiling possibility is against the user's anonymity and privacy.

    Among the techniques enumerated above, authentication through phone numbers is a some-what economic-incentive measure as providing multiple valid phone numbers will be expensive for the attacker. Notice that while using an expensive authentication method can reduce the number of accounts owned by a single spammer, cannot address the spam issue entirely. This is because the spammer can still send bulk messages through one single account. For this approach to be effective, a centralized mediator is essential. That is why such a solution would not fit the p2p environments where no centralized control exists.

    P2P Systems​

    What about spam prevention in p2p messaging platforms? There are two techniques, namely Proof of Work8 deployed by Whisper9 and Peer scoring6 method (namely reputation-based approach) adopted by LibP2P. However, each of these solutions has its own shortcomings for real-life use-cases as explained below.

    Proof of work​

    The idea behind the Proof Of Work i.e., POW8 is to make messaging a computationally costly operation hence lowering the messaging rate of all the peers including the spammers. In specific, the message publisher has to solve a puzzle and the puzzle is to find a nonce such that the hash of the message concatenated with the nonce has at least z leading zeros. z is known as the difficulty of the puzzle. Since the hash function is one-way, peers have to brute-force to find a nonce. Hashing is a computationally-heavy operation so is the brute-force. While solving the puzzle is computationally expensive, it is comparatively cheap to verify the solution.

    POW is also used as the underlying mining algorithm in Ethereum and Bitcoin blockchain. There, the goal is to contain the mining speed and allow the decentralized network to come to a consensus, or agree on things like account balances and the order of transactions.

    While the use of POW makes perfect sense in Ethereum / Bitcoin blockchain, it shows practical issues in heterogeneous p2p messaging systems with resource-restricted peers. Some peers won't be able to carry the designated computation and will be effectively excluded. Such exclusion showed to be practically an issue in applications like Status, which used to rely on POW for spam-protection, to the extent that the difficulty level had to be set close to zero.

    Peer Scoring​

    The peer scoring method6 that is utilized by libp2p is to limit the number of messages issued by a peer in connection to another peer. That is each peer monitors all the peers to which it is directly connected and adjusts their messaging quota i.e., to route or not route their messages depending on their past activities. For example, if a peer detects its neighbor is sending more than x messages per month, can drop its quota to z.x where z is less than one. The shortcoming of this solution is that scoring is based on peers' local observations and the concept of the score is defined in relation to one single peer. This leaves room for an attack where a spammer can make connections to k peers in the system and publishes k.(x-1) messages by exploiting all of its k connections. Another attack scenario is through botnets consisting of a large number of e.g., a million bots. The attacker rents a botnet and inserts each of them as a legitimate peer to the network and each can publish x-1 messages per month7.

    Economic-Incentive Spam protection​

    Is this the end of our spam-protection journey? Shall we simply give up and leave spammers be? Certainly not! -Waku RLN-Relay gives us a p2p spam-protection method which:

    • suits p2p systems and does not rely on any central entity.
    • is efficient i.e., with no unreasonable computational, storage, memory, and bandwidth requirement! as such, it fits the network of heterogeneous peers.
    • respects users privacy unlike reputation-based and centralized methods.
    • deploys economic-incentives to contain spammers' activity. Namely, there is a financial sacrifice for those who want to spam the system. How? follow along ...

    We devise a general rule to save everyone's life and that is

    No one can publish more than M messages per epoch without being financially charged!

    We set M to 1 for now, but this can be any arbitrary value. You may be thinking "This is too restrictive! Only one per epoch?". Don't worry, we set the epoch to a reasonable value so that it does not slow down the communication of innocent users but will make the life of spammers harder! Epoch here can be every second, as defined by UTC date-time +-20s.

    The remainder of this post is all about the story of how to enforce this limit on each user's messaging rate as well as how to impose the financial cost when the limit gets violated. This brings us to the Rate Limiting Nullifiers and how we integrate this technique into Waku v2 (in specific the Waku Relay protocol) to protect our valuable users against spammers.

    Technical Terms​

    Zero-knowledge proof: Zero-knowledge proof (ZKP)14 allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do the trust-minimized computation that is also privacy-preserving. As a basic example, instead of showing your ID when going to a bar you simply give them proof that you are over 18, without showing the doorman your id. In this write-up, by ZKP we essentially mean zkSNARK15 which is one of the many types of ZKPs.

    Threshold Secret Sharing Scheme: (m,n) Threshold secret-sharing is a method by which you can split a secret value s into n pieces in a way that the secret s can be reconstructed by having m pieces (m <= n). The economic-incentive spam protection utilizes a (2,n) secret sharing realized by Shamir Secret Sharing Scheme13.

    Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers​

    Context: We started the idea of economic-incentive spam protection more than a year ago and conducted a feasibility study to identify blockers and unknowns. The results are published in our prior post. Since then major progress has been made and the prior identified blockers that are listed below are now addressed. Kudos to Barry WhiteHat, Onur Kilic, Koh Wei Jie for all of their hard work, research, and development which made this progress possible.

    • the proof time22 which was initially in the order of minutes ~10 mins and now is almost 0.5 seconds
    • the prover key size21 which was initially ~110MB and now is ~3.9MB
    • the lack of Shamir logic19 which is now implemented and part of the RLN repository4
    • the concern regarding the potential multi-party computation for the trusted setup of zkSNARKs which got resolved20
    • the lack of end-to-end integration that now we made it possible, have it implemented, and are going to present it in this post. New blockers are also sorted out during the e2e integration which we will discuss in the Feasibility and Open Issues section.

    Now that you have more context, let's see how the final solution works. The fundamental point is to make it economically costly to send more than your share of messages and to do so in a privacy-preserving and e2e fashion. To do that we have the following components:

    • 1- Group: We manage all the peers inside a large group (later we can split peers into smaller groups, but for now consider only one). The group management is done via a smart contract which is devised for this purpose and is deployed on the Ethereum blockchain.
    • 2- Membership: To be able to send messages and in specific for the published messages to get routed by all the peers, publishing peers have to register to the group. Membership involves setting up public and private key pairs (think of it as the username and password). The private key remains at the user side but the public key becomes a part of the group information on the contract (publicly available) and everyone has access to it. Public keys are not human-generated (like usernames) and instead they are random numbers, as such, they do not reveal any information about the owner (think of public keys as pseudonyms). Registration is mandatory for the users who want to publish a message, however, users who only want to listen to the messages are more than welcome and do not have to register in the group.
    • Membership fee: Membership is not for free! each peer has to lock a certain amount of funds during the registration (this means peers have to have an Ethereum account with sufficient balance for this sake). This fund is safely stored on the contract and remains intact unless the peer attempts to break the rules and publish more than one message per epoch.
    • Zero-knowledge Proof of membership: Do you want your message to get routed to its destination, fine, but you have to prove that you are a member of the group (sorry, no one can escape the registration phase!). Now, you may be thinking that should I attach my public key to my message to prove my membership? Absolutely Not! we said that our solution respects privacy! membership proofs are done in a zero-knowledge manner that is each message will carry cryptographic proof asserting that "the message is generated by one of the current members of the group", so your identity remains private and your anonymity is preserved!
    • Slashing through secret sharing: Till now it does not seem like we can catch spammers, right? yes, you are right! now comes the exciting part, detecting spammers and slashing them. The core idea behind the slashing is that each publishing peer (not routing peers!) has to integrate a secret share of its private key inside the message. The secret share is deterministically computed over the private key and the current epoch. The content of this share is harmless for the peer's privacy (it looks random) unless the peer attempts to publish more than one message in the same epoch hence disclosing more than one secret share of its private key. Indeed two distinct shares of the private key under the same epoch are enough to reconstruct the entire private key. Then what should you do with the recovered private key? hurry up! go to the contract and withdraw the private key and claim its fund and get rich!! Are you thinking what if spammers attach junk values instead of valid secret shares? Of course, that wouldn't be cool! so, there is a zero-knowledge proof for this sake as well where the publishing peer has to prove that the secret shares are generated correctly.

    A high-level overview of the economic spam protection is shown in Figure 1.

    Flow​

    In this section, we describe the flow of the economic-incentive spam detection mechanism from the viewpoint of a single peer. An overview of this flow is provided in Figure 3.

    Setup and Registration​

    A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members' public keys. An overview of registration is illustrated in Figure 2.

    For the registration, a peer creates a transaction that sends x amount of Ether to the contract. The peer who has the "private key" sk associated with that deposit would be able to withdraw x Ether by providing valid proof. Note that sk is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch. -The following relation holds between the sk and pk i.e., pk = H(sk) where H denotes a hash function. -

    Figure 2: Registration

    Maintaining the membership Merkle Tree​

    The ZKP of membership that we mentioned before relies on the representation of the entire group as a Merkle Tree. The tree construction and maintenance is delegated to the peers (the initial idea was to keep the tree on the chain as part of the contract, however, the cost associated with member deletion and insertion was high and unreasonable, please see Feasibility and Open Issues for more details). As such, each peer needs to build the tree locally and sync itself with the contract updates (peer insertion and deletion) to mirror them on its tree. -Two pieces of information of the tree are important as they enable peers to generate zero-knowledge proofs. One is the root of the tree and the other is the membership proof (or the authentication path). The tree root is public information whereas the membership proof is private data (or more precisely the index of the peer in the tree).

    Publishing​

    In order to publish at a given epoch, each message must carry a proof i.e., a zero-knowledge proof signifying that the publishing peer is a registered member, and has not exceeded the messaging rate at the given epoch.

    Recall that the enforcement of the messaging rate was through associating a secret shared version of the peer's sk into the message together with a ZKP that the secret shares are constructed correctly. As for the secret sharing part, the peer generates the following data:

    1. shareX
    2. shareY
    3. nullifier

    The pair (shareX, shareY) is the secret shared version of sk that are generated using Shamir secret sharing scheme. Having two such pairs for an identical nullifier results in full disclosure of peer's sk and hence burning the associated deposit. Note that the nullifier is a deterministic value derived from sk and epoch therefore any two messages issued by the same peer (i.e., using the same sk) for the same epoch are guaranteed to have identical nullifiers.

    Finally, the peer generates a zero-knowledge proof zkProof asserting the membership of the peer in the group and the correctness of the attached secret share (shareX, shareY) and the nullifier. In order to generate a valid proof, the peer needs to have two private inputs i.e., its sk and its authentication path. Other inputs are the tree root, epoch, and the content of the message.

    Privacy Hint: Note that the authentication path of each peer depends on the recent list of members (hence changes when new peers register or leave). As such, it is recommended (and necessary for privacy/anonymity) that the publisher updates her authentication path based on the latest status of the group and attempts the proof using the updated version.

    An overview of the publishing procedure is provided in Figure 3.

    Routing​

    Upon the receipt of a message, the routing peer needs to decide whether to route it or not. This decision relies on the following factors:

    1. If the epoch value attached to the message has a non-reasonable gap with the routing peer's current epoch then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs).
    2. The message MUST contain valid proof that gets verified by the routing peer. -If the preceding checks are passed successfully, then the message is relayed. In case of an invalid proof, the message is dropped. If spamming is detected, the publishing peer gets slashed (see Spam Detection and Slashing).

    An overview of the routing procedure is provided in Figure 3.

    Spam Detection and Slashing​

    In order to enable local spam detection and slashing, routing peers MUST record the nullifier, shareX, and shareY of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.

    1. The routing peer first verifies the zkProof and drops the message if not verified.
    2. Otherwise, it checks whether a message with an identical nullifier has already been relayed.
      • a) If such message exists and its shareX and shareY components are different from the incoming message, then slashing takes place (if the shareX and shareY fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).
      • b) If none found, then the message gets relayed.

    An overview of the slashing procedure is provided in Figure 3. -

    Figure 3: Publishing, Routing and Slashing workflow.

    Feasibility and Open Issues​

    We've come a long way since a year ago, blockers resolved, now we have implemented it end-to-end. We learned lot and could identify further issues and unknowns some of which are blocking getting to production. The summary of the identified issues are presented below.

    Storage overhead per peer​

    Currently, peers are supposed to maintain the entire tree locally and it imposes storage overhead which is linear in the size of the group (see this issue11 for more details). One way to cope with this is to use the light-node and full-node paradigm in which only a subset of peers who are more resourceful retain the tree whereas the light nodes obtain the necessary information by interacting with the full nodes. Another way to approach this problem is through a more storage efficient method (as described in this research issue12) where peers store a partial view of the tree instead of the entire tree. Keeping the partial view lowers the storage complexity to O(log(N)) where N is the size of the group. There are still unknown unknowns to this solution, as such, it must be studied further to become fully functional.

    Cost-effective way of member insertion and deletion​

    Currently, the cost associated with RLN-Relay membership is around 30 USD10. We aim at finding a more cost-effective approach. Please feel free to share with us your solution ideas in this regard in this issue.

    Exceeding the messaging rate via multiple registrations​

    While the economic-incentive solution has an economic incentive to discourage spamming, we should note that there is still expensive attack(s)23 that a spammer can launch to break the messaging rate limit. That is, the attacker can pay for multiple legit registrations e.g., k, hence being able to publish k messages per epoch. We believe that the higher the membership fee is, the less probable would be such an attack, hence a stronger level of spam-protection can be achieved. Following this argument, the high fee associated with the membership (which we listed above as an open problem) can indeed be contributing to a better protection level.

    Conclusion and Future Steps​

    As discussed in this post, Waku RLN Relay can achieve a privacy-preserving economic spam protection through rate-limiting nullifiers. The idea is to financially discourage peers from publishing more than one message per epoch. In specific, exceeding the messaging rate results in a financial charge. Those who violate this rule are called spammers and their messages are spam. The identification of spammers does not rely on any central entity. Also, the financial punishment of spammers is cryptographically guaranteed. -In this solution, privacy is guaranteed since: 1) Peers do not have to disclose any piece of personally identifiable information in any phase i.e., neither in the registration nor in the messaging phase 2) Peers can prove that they have not exceeded the messaging rate in a zero-knowledge manner and without leaving any trace to their membership accounts. -Furthermore, all the computations are light hence this solution fits the heterogenous p2p messaging system. Note that the zero-knowledge proof parts are handled through zkSNARKs and the benchmarking result can be found in the RLN benchmark report5.

    Future steps:

    We are still at the PoC level, and the development is in progress. As our future steps,

    • we would like to evaluate the running time associated with the Merkle tree operations. Indeed, the need to locally store Merkle tree on each peer was one of the unknowns discovered during this PoC and yet the concrete benchmarking result in this regard is not available.
    • We would also like to pursue our storage-efficient Merkle Tree maintenance solution in order to lower the storage overhead of peers.
    • In line with the storage optimization, the full-node light-node structure is another path to follow.
    • Another possible improvement is to replace the membership contract with a distributed group management scheme e.g., through distributed hash tables. This is to address possible performance issues that the interaction with the Ethereum blockchain may cause. For example, the registration transactions are subject to delay as they have to be mined before being visible in the state of the membership contract. This means peers have to wait for some time before being able to publish any message.

    Acknowledgement​

    Thanks to Onur Kılıç for his explanation and pointers and for assisting with development and runtime issues. Also thanks to Barry Whitehat for his time and insightful comments. Special thanks to Oskar Thoren for his constructive comments and his guides during the development of this PoC and the writeup of this post.

    References​

    1. RLN-Relay specification: https://rfc.vac.dev/waku/standards/core/17/rln-relay↩
    2. RLN documentation: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both↩
    3. RLN repositories: https://github.com/kilic/RLN and https://github.com/kilic/rlnapp↩
    4. Waku v2: https://rfc.vac.dev/waku/standards/core/10/waku2↩
    5. GossipSub: https://docs.libp2p.io/concepts/publish-subscribe/↩
    6. Waku Relay: https://rfc.vac.dev/waku/standards/core/11/relay↩
    7. Proof of work: http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf and https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf↩
    8. EIP-627 Whisper: https://eips.ethereum.org/EIPS/eip-627↩
    9. Peer Scoring: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring↩
    10. Peer scoring security issues: https://github.com/vacp2p/research/issues/44↩
    11. Zero Knowledge Proof: https://dl.acm.org/doi/abs/10.1145/3335741.3335750 and https://en.wikipedia.org/wiki/Zero-knowledge_proof↩
    12. zkSNARKs: https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11 and https://coinpare.io/whitepaper/zcash.pdf↩
    13. Shamir Secret Sharing Scheme: https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing↩
    14. zkSNARKs proof time: https://github.com/vacp2p/research/issues/7↩
    15. Prover key size: https://github.com/vacp2p/research/issues/8↩
    16. The lack of Shamir secret sharing in zkSNARKs: https://github.com/vacp2p/research/issues/10↩
    17. The MPC required for zkSNARKs trusted setup: https://github.com/vacp2p/research/issues/9↩
    18. Storage overhead per peer: https://github.com/vacp2p/research/issues/57↩
    19. Storage-efficient Merkle Tree maintenance: https://github.com/vacp2p/research/pull/54↩
    20. Cost-effective way of member insertion and deletion: https://github.com/vacp2p/research/issues/56↩
    21. Attack on the messaging rate: https://github.com/vacp2p/specs/issues/251↩
    22. RLN Benchmark: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks↩
    ]]>     -     - - <![CDATA[[Talk] Vac, Waku v2 and Ethereum Messaging]]> - https://vac.dev/rlog/waku-v2-ethereum-messaging - https://vac.dev/rlog/waku-v2-ethereum-messaging - Tue, 10 Nov 2020 12:00:00 GMT - - Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging.

    The following post is a transcript of the talk given at the Taipei Ethereum meetup, November 5. There is also a video recording.

    0. Introduction​

    Hi! My name is Oskar and I'm the protocol research lead at Vac. This talk will be divided into two parts. First I'll talk about the journey from Whisper, to Waku v1 and now to Waku v2. Then I'll talk about messaging in Ethereum. After this talk, you should have an idea of what Waku v2 is, the problems it is trying to solve, as well as where it can be useful for messaging in Ethereum.

    PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2​

    1. Vac intro​

    First, what is Vac? Vac grew out of our efforts Status to create a window on to Ethereum and secure messenger. Vac is modular protocol stack for p2p secure messaging, paying special attention to resource restricted devices, privacy and censorship resistance.

    Today we are going to talk mainly about Waku v2, which is the transport privacy / routing aspect of the Vac protocol stack. It sits "above" the p2p overlay, such as libp2p dealing with transports etc, and below a conversational security layer dealing with messaging encryption, such as using Double Ratchet etc.

    2. Whisper to Waku v1​

    In the beginning, there was Whisper. Whisper was part of the holy trinity of Ethereum. You had Ethereum for consensus/computation, Whisper for messaging, and Swarm for storage.

    However, for various reasons, Whisper didn't get the attention it deserved. Development dwindled, it promised too much and it suffered from many issues, such as being extremely inefficient and not being suitable for running on e.g. mobile phone. Despite this, Status used it in its app from around 2017 to 2019. As far as I know, it was one of very few, if not the only, production uses of Whisper.

    In an effort to solve some of its immediate problems, we forked Whisper into Waku and formalized it with a proper specification. This solved immediate bandwidth issues for light nodes, introduced rate limiting for better spam protection, improved historical message support, etc.

    If you are interested in this journey, checkout the EthCC talk Dean and I gave in Paris earlier this year.

    Status upgraded to Waku v1 early 2020. What next?

    3. Waku v1 to v2​

    We were far from done. The changes we had made were quite incremental and done in order to get tangible improvements as quickly as possible. This meant we couldn't address more fundamental issues related to full node routing scalability, running with libp2p for more transports, better security, better spam protection and incentivization.

    This kickstarted Waku v2 efforts, which is what we've been working on since July. This work was and is initally centered around a few pieces:

    (a) Moving to libp2p

    (b) Better routing

    (c) Accounting and user-run nodes

    The general theme was: making the Waku network more scalable and robust.

    We also did a scalability study to show at what point the network would run into issues, due to the inherent lack of routing that Whisper and Waku v1 provided.

    You can read more about this here.

    3.5 Waku v2 - Design goals​

    Taking a step back, what problem does Waku v2 attempt to solve compared to all the other solutions that exists out there? What type of applications should use it and why? We have the following design goals:

    1. Generalized messaging. Many applications requires some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix.

    2. Peer-to-peer. These applications sometimes have requirements that make them suitable for peer-to-peer solutions.

    3. Resource restricted. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:

      • limited bandwidth, CPU, memory, disk, battery, etc
      • not being publicly connectable
      • only being intermittently connected; mostly-offline

    4. Privacy. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.

    As well as to do so in a modular fashion. Meaning you can find a reasonable trade-off depending on your exact requirements. For example, you usually have to trade off some bandwidth to get metadata protection, and vice versa.

    The concept of designing for resource restricted devices also leads to the concept of adaptive nodes, where you have more of a continuum between full nodes and light nodes. For example, if you switch your phone from mobile data to WiFi you might be able to handle more bandwidth, and so on.

    4. Waku v2 - Breakdown​

    Where is Waku v2 at now, and how is it structured?

    It is running over libp2p and we had our second internal testnet last week or so. As a side note, we name our testnets after subway stations in Taipei, the first one being Nangang, and the most recent one being Dingpu.

    The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:

    • Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.
    • Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.
    • Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.
    • Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.
    • Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.
    • Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.

    Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.

    You can read more about the breakdown in this update though some progress has been made since then, as well was in the main Waku v2 spec.

    5. Waku v2 - Upcoming​

    What's coming up next? There are a few things.

    For Status to use it in production, it needs to be integrated into the main app using the Nim Node API. The bridge also needs to be implemented and tested.

    For other users, we are currently overhauling the API to allow usage from a browser, e.g. To make this experience great, there are also a few underlying infrastructure things that we need in nim-libp2p, such as a more secure HTTP server in Nim, Websockets and WebRTC support.

    There are also some changes we made to at what level content encryption happens, and this needs to be made easier to use in the API. This means you can use a node without giving your keys to it, which is useful in some environments.

    More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:

    • Better scaling, by using topic sharding.
    • Accounting and user-run nodes, to account for and incentives full nodes.
    • Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.
    • Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.

    As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next.

    PART 2 - ETHEREUM MESSAGING​

    A lot of what follows is inspired by exploratory work that John Lea has done at Status, previously Head of UX Architecture at Ubuntu.

    6. Ethereum Messaging - Why?​

    It is easy to think that Waku v2 is only for human to human messaging, since that's how Waku is currently primarily used in the Status app. However, the goal is to be useful for generalized messaging, which includes other type of information as well as machine to machine messaging.

    What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:

    • Reducing on-chain transactions
    • Reduce latency for operations
    • Decentralize centrally coordinated services (like WalletConnect)
    • Improve UX of dapps
    • Broadcast live information
    • A message transport layer for state channels

    And so on.

    7. Ethereum Messaging - Why? (Cont)​

    What are some examples of practical things Waku as used for Ethereum Messaging could solve?

    • Multisig transfers only needing one on chain transaction
    • DAO votes only needing one one chain transaction
    • Giving dapps ability to direct push notifications to users
    • Giving users ability to directly respond to requests from daps
    • Decentralized Wallet Connect

    Etc.

    8. What's needed to deliver this?​

    We can break it down into our actors:

    • Decentralized M2M messaging system (Waku)
    • Native wallets (Argent, Metamask, Status, etc)
    • Dapps that benefit from M2M messaging
    • Users whose problems are being solved

    Each of these has a bunch of requirements in turn. The messaging system needs to be decentralized, scalable, robust, etc. Wallets need support for messaging layer, dapps need to integrate this, etc.

    This is a lot! Growing adoption is a challenge. There is a catch 22 in terms of justifying development efforts for wallets, when no dapps need it, and likewise for dapps when no wallets support Waku. In addition to this, there must be proven usage of Waku before it can be relied on, etc. How can we break this up into smaller pieces of work?

    9. Breaking up the problem and a high level roadmap​

    We can start small. It doesn't and need to be used for critical features first. A more hybrid approach can be taken where it acts more as nice-to-haves.

    1. Forking Whisper and solving scalablity, spam etc issues with it. -This is a work in progress. What we talked about in part 1.
    2. Expose messaging API for Dapp developers.
    3. Implement decentralized version of WalletConnect. -Currently wallets connect ot dapps with centralized service. Great UX.
    4. Solve DAO/Multi-Sig coordination problem. -E.g. send message to wallet-derived key when it is time to sign a transaction.
    5. Extend dapp-to-user and user-to-dapp communication to more dapps. -Use lessons learned and examples to drive adoptation for wallets/dapps.

    And then build up from there.

    10. We are hiring!​

    A lot of this will happen in Javascript and browsers, since that's the primarily environment for a lot of wallets and dapps. We are currently hiring for a Waku JS Wallet integration lead to help push this effort further.

    Come talk to me after or apply here.

    That's it! You can find us on Status, Telegram, vac.dev. I'm on twitter here.

    Questions?

    ]]>     -     - - <![CDATA[Waku v2 Update]]> - https://vac.dev/rlog/waku-v2-update - https://vac.dev/rlog/waku-v2-update - Mon, 28 Sep 2020 12:00:00 GMT - - A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?

    It has been a while since the last post. It is time for an update on Waku v2. Aside from getting more familiar with libp2p (specifically nim-libp2p) and some vacation, what have we been up to? In this post we'll talk about what we've gotten done since last time, and briefly talk about immediate next steps and future. But first, a recap.

    Recap​

    In the last post (Waku v2 plan) we explained the rationale of Waku v2 - the current Waku network is fragile and doesn't scale. To solve this, Waku v2 aims to reduce amplification factors and get more user run nodes. We broke the work down into three separate tracks.

    1. Track 1 - Move to libp2p
    2. Track 2 - Better routing
    3. Track 3 - Accounting and user-run nodes

    As well as various rough components for each track. The primary initial focus is track 1. This means things like: moving to FloodSub, simplify the protocol, core integration, topic interest behavior, historical message caching, and Waku v1<>v2 bridge.

    Current state​

    Let's talk about the state of specs and our main implementation nim-waku. Then we'll go over our recent testnet, Nangang, and finish off with a Web PoC.

    Specs​

    After some back and forth on how to best structure things, we ended up breaking down the specs into a few pieces. While Waku v2 is best thought of as a cohesive whole in terms of its capabilities, it is made up of several protocols. Here's a list of the current specs and their status:

    Raw means there is not yet an implementation that corresponds fully to the spec, and draft means there is an implementation that corresponds to the spec. In the interest of space, we won't go into too much detail on the specs here except to note a few things:

    • The relay spec is essentially a thin wrapper on top of PubSub/FloodSub/GossipSub
    • The filter protocol corresponds to previous light client mode in Waku v1
    • The store protocol corresponds to the previous mailserver construct in Waku v1

    The filter and store protocol allow for adaptive nodes, i.e. nodes that have various capabilities. For example, a node being mostly offline, or having limited bandwidth capacity. The bridge spec outlines how to bridge the Waku v1 and v2 networks.

    Implementation​

    The main implementation we are working on is nim-waku. This builds on top of libraries such as nim-libp2p and others that the Nimbus team have been working on as part of their Ethereum 2.0 client.

    Currently nim-waku implements the relay protocol, and is close to implementing filter and store protocol. It also exposes a Nim Node API that allows libraries such as nim-status to use it. Additionally, there is also a rudimentary JSON RPC API for command line scripting.

    Nangang testnet​

    Last week we launched a very rudimentary internal testnet called Nangang. The goal was to test basic connectivity and make sure things work end to end. It didn't have things like: client integration, encryption, bridging, multiple clients, store/filter protocol, or even a real interface. What it did do is allow Waku developers to "chat" via RPC calls and looking in the log output. Doing this meant we exposed and fixed a few blockers, such as connection issues, deployment, topic subscription management, protocol and node integration, and basic scripting/API usage. After this, we felt confident enough to upgrade the main and relay spec to "draft" status.

    Waku Web PoC​

    As a bonus, we wanted to see what it'd take to get Waku running in a browser. This is a very powerful capability that enables a lot of use cases, and something that libp2p enables with its multiple transport support.

    Using the current stack, with nim-waku, would require quite a lot of ground work with WASM, WebRTC, Websockets support etc. Instead, we decided to take a shortcut and hack together a JS implementation called Waku Web Chat. This quick hack wouldn't be possible without the people behind js-libp2p-examples and js-libp2p and all its libraries. These are people like Jacob Heun, Vasco Santos, and Cayman Nava. Thanks!

    It consists of a brower implementation, a NodeJS implementation and a bootstrap server that acts as a signaling server for WebRTC. It is largely a bastardized version of GossipSub, and while it isn't completely to spec, it does allow messages originating from a browser to eventually end up at a nim-waku node, and vice versa. Which is pretty cool.

    Coming up​

    Now that we know what the current state is, what is still missing? what are the next steps?

    Things that are missing​

    While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope:

    1. Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level.

    2. Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status (status-nim has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings -us too...

    3. Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 (1, 2). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc).

    4. Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented.

    5. General tightening up of specs and implementation.

    While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope.

    Upcoming​

    Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details.

    1. Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors.

    2. Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner.

    3. Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet -format can improve things like unlinkability.

    4. zkSnarks RLN for spam protection and incentivization. We studied this last year and recent developments have made this relevant to study again. Create an experimental spec/PoC as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this!

    5. Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we'd like to deliver on that.

    As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates.

    If you are feeling adventurous and want to use early stage alpha software, check out the docs. If you want to read the specs, head over to Waku spec. If you want to talk with us, join us on Status or on Telegram (they are bridged).

    ]]>     -     - - <![CDATA[What's the Plan for Waku v2?]]> - https://vac.dev/rlog/waku-v2-plan - https://vac.dev/rlog/waku-v2-plan - Wed, 01 Jul 2020 12:00:00 GMT - - Read about our plans for Waku v2, moving to libp2p, better routing, adaptive nodes and accounting!

    tldr: The Waku network is fragile and doesn't scale. Here's how to solve it.

    NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.

    Problem​

    The Waku network is fragile and doesn't scale.

    As Status is moving into a user-acquisition phase and is improving retention rates for users they need the infrastructure to keep up, specifically when it comes to messaging.

    Based on user acquisition models, the initial goal is to support 100k DAU in September, with demand growing from there.

    With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.

    **1. Connection limits**. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.

    **2. Bandwidth as a bottleneck**. We notice that memory usage appears to not be -the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.

    **3. Amplification factors**. Reducing amplification factors with better routing, would have a high impact, but it is likely we'd need additional measures as well, such as topic sharding or similar. See Fig 8-13.

    Figure 1-5:

    -
    -
    -
    -

    See https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0 for the full report.

    What we need to do is:

    1. Reduce amplification factors
    2. Get more user-run full nodes

    Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:

    • keep increasing requirements for Internet speed for full nodes
    • are vulnerable to censorship and attacks
    • have to control the topology in an artifical manner to keep up with load
    • basically re-invent a traditional centralized client-server app with extra steps
    • deliberately ignore most of our principles
    • risk the network being shut down when we run out of cash

    Appetite​

    Our initial risk appetite for this is 6 weeks for a small team.

    The idea is that we want to make tangible progress towards the goal in a limited period of time, as opposed to getting bogged down in trying to find a theoretically perfect generalized solution. Fixed time, variable scope.

    It is likely some elements of a complete solution will be done separately. See later sections for that.

    Solution​

    There are two main parts of the solution. One is to reduce amplification factors, and the other is incentivization to get more user run full nodes with desktop, etc.

    What does a full node provide? It provides connectivity to the network, can act as a bandwidth "barrier" and be high or reasonably high availability. What this means right now is essentially topic interest and storing historical messages.

    The goal is here to improve the status quo, not get a perfect solution from the get go. All of this can be iterated on further, for stronger guarantees, as well as replaced by other new modules.

    Let's first look at the baseline, and then go into some of the tracks and their phases. Track 1 is best done first, after which track 2 and 3 can be executed in parallel. Track 1 gives us more options for track 2 and 3. The work in track 1 is currently more well-defined, so it is likely the specifics of track 2 and 3 will get refined at a later stage.

    Baseline​

    Here's where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can't easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.

    Track 1 - Move to libp2p​

    Moving to PubSub over libp2p wouldn't improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.

    See more on libp2p PubSub here: https://docs.libp2p.io/concepts/publish-subscribe/

    As part of this move, there are a few individual pieces that are needed.

    1. FloodSub​

    This is essentially what Waku over libp2p would look like in its most basic form.

    One difference that is worth noting is that the app topics would not be the same as Waku topics. Why? In Waku we currently don't use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.

    Why can't we use Waku topics for routing directly? PubSub over libp2p isn't built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.

    Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.

    Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: https://github.com/status-im/nim-waku

    2. Simplify the protocol​

    Due to Waku's origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (https://rfc.vac.dev/waku/standards/legacy/6/waku1/). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?

    Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.

    The essential characteristic that has to be maintained is that we don't need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.

    3. Core integration​

    As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.

    4. Topic interest behavior​

    While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (https://vac.dev/fixing-whisper-with-waku). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.

    5. Historical message caching​

    Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there's no reason to reimplement a poor API if we don't have to.

    Also see section below on adaptive nodes and capabilities.

    6. Waku v1 <> Libp2p bridge​

    To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.

    Track 2 - Better routing​

    This is where we improve the amplification factors.

    1. GossipSub​

    This is a subprotocol of FloodSub in the libp2p world. Moving to GossipSub would allow traffic between full nodes to go from an amplification factor of ~25 to ~6. This basically creates a mesh of stable bidirectional connections, together with some gossiping capabilities outside of this view.

    Explaining how GossipSub works is out of scope of this document. It is implemented in nim-libp2p and used by Nimbus as part of Eth2. You can read the specs here in more detail if you are interested: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md and https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md

    -
    -
    -

    While we technically could implement this over existing Waku, we'd have to re-implement it, and we'd lose out on all the other benefits libp2p would provide, as well as the ecosystem of people and projects working on improving the scalability and security of these protocols.

    2. Topic sharding​

    This one is slightly more speculative in terms of its ultimate impact. The basic idea is to split the application topic into N shards, say 10, and then each full node can choose which shards to listen to. This can reduce amplification factors by another factor of 10.

    -

    Note that this means a light node that listens to several topics would have to be connected to more full nodes to get connectivity. For a more exotic version of this, see https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47

    This is orthogonal from the choice of FloodSub or GossipSub, but due to GossipSub's more dynamic nature it is likely best combined with it.

    3. Other factors​

    Not a primary focus, but worth a look. Looking at the scaling model, there might be other easy wins to improve overall bandwidth consumption between full nodes. For example, can we reduce envelope size by a significant factor?

    Track 3 - Accounting and user-run nodes​

    This is where we make sure the network isn't fragile, become a true p2p app, get our users excited and engaged, and allow us to scale the network without creating an even bigger cluster.

    To work in practice, this has a soft dependency on node discovery such as DNS based discovery (https://eips.ethereum.org/EIPS/eip-1459) or Discovery v5 (https://vac.dev/feasibility-discv5).

    1. Adaptive nodes and capabilities​

    We want to make the gradation between light nodes, full nodes, storing (partial set of) historical messages, only acting for a specific shard, etc more flexible and explicit. This is required to identify and discover the nodes you want. See https://github.com/vacp2p/specs/issues/87

    Depending on how the other tracks come together, this design should allow for a desktop node to identify as a full relaying node for some some app topic shard, but also express waku topic interest and retrieve historical messages itself.

    E.g. Disc v5 can be used to supply node properties through ENR.

    2. Accounting​

    This is based on a few principles:

    1. Some nodes contribute a lot more than other nodes in the network
    2. We can account for the difference in contribution in some fashion
    3. We want to incentivize nodes to tell the true, and be incentivized not to lie

    Accounting here is a stepping stone, where accounting is the raw data upon which some settlement later occurs. It can have various forms of granularity. See https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31 for discussion.

    We also note that in GossipSub, the mesh is bidrectional. Additionally, it doesn't appears to be a high priority issue in terms of nodes misreporting. What is an issue is having people run full nodes in the first place. There are a few points to that. It has to be possible in the end-user UX, nodes have to be discovered, and it has to be profitable/visible that you are contributing. UX and discovery are out of scope for this work, whereas visibility/accounting is part of this scope. Settlement is a stretch goal here.

    The general shape of the solution is inspired by the Swarm model, where we do accounting separate from settlement. It doesn't require any specific proofs, but nodes are incentivized to tell the truth in the following way:

    1. Both full node and light node do accounting in a pairwise, local fashion
    2. If a light node doesn't ultimately pay or lie about reporting, they get disconnected (e.g.)
    3. If a full node doesn't provide its service the light node may pick another full node (e.g.)

    While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:

    • end to end delivery
    • online time
    • completeness of storage

    This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See https://discuss.status.im/t/network-incentivisation-first-draft/1037 for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51 for using validity proofs and removing liveness requirement for settlement.

    All of this is optional at this stage, because our goal here is to improve the status quo for user run nodes. Accounting at this stage should be visible and correspond to the net benefit a node provides to another.

    As a concrete example: a light node has some topic interest and cares about historical messages on some topic. A full node communicates envelopes as they come in, communicates their high availability (online time) and stores/forward stored messages. Both nodes have this information, and if they agree settlement (initially just a mock message) can be sending a payment to an address at some time interval / over some defined volume. See future sections for how this can be improved upon.

    Also see below in section 4, using constructs such as eigentrust as a local reputation mechanism.

    3. Relax high availability requirement​

    If we want desktop nodes to participate in the storing of historical messages, high availability is a problem. It is a problem for any node, especially if they lie about it, but assuming they are honest it is still an issue.

    By being connected to multiple nodes, we can get an overlapping online window. Then these can be combined together to get consistency. This is obviously experimental and would need to be tested before being deployed, but if it works it'd be very useful.

    Additionally or alternatively, instead of putting a high requirement on message availability, focus on detection of missing information. This likely requires re-thinking how we do data sync / replication.

    4. Incentivize light and full nodes to tell the truth (policy, etc)​

    In accounting phase it is largely assumed nodes are honest. What happens when they lie, and how do we incentivize them to be honest? In the case of Bittorrent this is done with tit-for-tat, however this is a different kind of relationship. What follows are some examples of how this can be done.

    For light nodes:

    • if they don't, they get disconnected
    • prepayment (especially to "high value" nodes)

    For full nodes:

    • multiple nodes reporting to agree, where truth becomes a shelling point
    • use eigentrust
    • staking for discovery visibility with slashing

    5. Settlement PoC​

    Can be done after phase 2 if so desired. Basically integrate payments based on accounting and policy.

    Out of scope​

    1. We assume the Status Base model requirements are accurate.
    2. We assume Core will improve retention rates.
    3. We assume the Stimbus production team will enable integration of nim-waku.
    4. We assume Discovery mechanisms such as DNS and Discovery v5 will be worked on separately.
    5. We assume Core will, at some point, provide an UX for integrating payment of services.
    6. We assume the desktop client is sufficiently usable.
    7. We assume Core and Infra will investigate ways of improving MaxPeers.
    ]]>     -     - - <![CDATA[Feasibility Study: Discv5]]> - https://vac.dev/rlog/feasibility-discv5 - https://vac.dev/rlog/feasibility-discv5 - Mon, 27 Apr 2020 12:00:00 GMT - - Looking at discv5 and the theoretical numbers behind finding peers.

    Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers.

    This post serves as a more authoritative overview of the discv5 study, for a discussionary post providing more context make sure to check out the corresponding discuss post. Additionally, if you are unfamiliar with discv5, check out my previous write-up: "From Kademlia to Discv5".

    Motivating Problem​

    The discovery method currently used by Status, is made up of various components and grew over time to solve a mix of problems. We want to simplify this while maintaining some of the properties we currently have.

    Namely, we want to ensure censorship resistance to state-level adversaries. One of the issues Status had which caused us them add to their discovery method was the fact that addresses from providers like AWS and GCP were blocked both in Russia and China. Additionally, one of the main factors required is the ability to function on resource restricted devices.

    Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider:

    • Battery consumption - constant connections like websockets consume a lot of battery life.
    • CPU usage - certain discovery methods may be CPU incentive, slowing an app down and making it unusable.
    • Bandwidth consumption - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans.
    • Short connection windows - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time.
    • Not publicly connectable - There is a good chance that most resource restricted devices are not publicly connectable.

    For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node.

    With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation.

    In order to answer our questions, we formulated the problem with a simple method for testing. The "needle in a haystack" problem was formulated to figure out how easily a specific node can be found within a given network. This issue was fully formulated in vacp2p/research#15.

    Overview​

    The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer:

    • We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.
    • We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through topic tables.

    Feasbility​

    To be able to investigate the feasibility of discv5, we used various methods including rough calculations which can be found in the notebook, and a simulation isolated in vacp2p/research#19.

    CPU & Memory Usage​

    The experimental discv5 has already been used within Status, however what was noticed was that the CPU and memory usage was rather high. It therefore should be investiaged if this is still the case, and if it is, it should be isolated where this stems from. Additionally it is worth looking at whether or not this is the case with both the go and nim implementation.

    See details: vacp2p/research#31

    NAT on Cellular Data​

    If a peer is not publically connectable it can not participate in the DHT both ways. A lot of mobile phones are behind symmetric NATs which UDP hole-punching close to impossible. It should be investigated whether or not mobile phones will be able to participate both ways and if there are good methods for doing hole-punching.

    See details: vacp2p/research#29

    Topic Tables​

    Topic Tables allow us the ability to efficiently find nodes given a specific topic. However, they are not implemented in the status-im/nim-eth implementation nor are they fully finalized in the spec. These are important if the network grows past a size where the concentration of specific nodes is relatively low making them hard to find.

    See details: vacp2p/research#26

    Finding a node​

    It is important to note, that given a network is relatively small sized, eg 100-500 nodes, then finding a node given a specific address is relatively managable. Additionally, if the concentration of a specific capability in a network is reasonable, then finding a node advertising its capabilities using an ENR rather than the topic table is also managable. A reasonable concentration for example would be 10%, which would give us an 80% chance of getting a node with that capability in the first lookup request. This can be explored more using our discv5 notebook.

    Results​

    Research has shown that finding a node in the DHT has a relatively low effect on bandwidth, both inbound and outbound. For example when trying to find a node in a network of 100 nodes, it would take roughly 5668 bytes total. Additionally if we assume 100ms latency per request it would range at ≈ 300ms latency, translating to 3 requests to find a specific node.

    General Thoughts​

    One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more "features" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.

    Acknowledgements​

    • Oskar Thoren
    • Dmitry Shmatko
    • Kim De Mey
    • Corey Petty
    ]]>     -     - - <![CDATA[What Would a WeChat Replacement Need?]]> - https://vac.dev/rlog/wechat-replacement-need - https://vac.dev/rlog/wechat-replacement-need - Thu, 16 Apr 2020 12:00:00 GMT - - What would a self-sovereign, private, censorship-resistant and open alternative to WeChat look like?

    What would it take to replace WeChat? More specifically, what would a self-sovereign, private, censorship-resistant and open alternative look like? One that allows people to communicate, coordinate and transact freely.

    Background​

    What WeChat provides to the end-user​

    Let's first look at some of the things that WeChat providers. It is a lot:

    • Messaging: 1:1 and group chat. Text, as well as voice and video. Post gifs. Share location.
    • Group chat: Limited to 500 people; above 100 people people need to verify with a bank account. Also has group video chat and QR code to join a group.
    • Timeline/Moments: Post comments with attachments and have people like/comment on it.
    • Location Discovery: See WeChat users that are nearby.
    • Profile: Nickname and profile picture; can alias people.
    • "Broadcast" messages: Send one message to many contacts, up to 200 people (spam limited).
    • Contacts: Max 5000 contacts (people get around it with multiple accounts and sim cards).
    • App reach: Many diferent web apps, extensions, native apps, etc. Scan QR code to access web app from phone.
    • Selective posting: Decide who can view your posts and who can view your comments on other people's post.
    • Transact: Send money gifts through red envelopes.
    • Transact: Use WeChat pay to transfer money to friends and businesses; linked account with Alipay that is connected to your bank account.
    • Services: Find taxis and get notifications; book flights, train tickets, hotels etc.
    • Mini apps: API for all kinds of apps that allow you to provide services etc.
    • Picture in picture: allowing you to have a video call while using the app.

    And much more. Not going to through it all in detail, and there are probably many things I don't know about WeChat since I'm not a heavy user living in mainland China.

    How WeChat works - a toy model​

    This is an overly simplistic model of how WeChat works, but it is sufficient for our purposes. This general design applies to most traditional client-server apps today.

    To sign up for account you need a phone number or equivalent. To get access to some features you need to verify your identity further, for example with official ID and/or bank account.

    When you signup this creates an entry in the WeChat server, from now on treated as a black box. You authenticate with that box, and thats where you get your messages from. If you go online the app asks that box for messages you have received while you were offline. If you login from a different app your contacts and conversations are synced from that box.

    The box gives you an account, it deals with routing to your contacts, it stores messages and attachments and gives access to mini apps that people have uploaded. For transacting money, there is a partnership with a different company that has a different box which talks to your bank account.

    This is done in a such a way that they can support a billion users with the features above, no sweat.

    Whoever controls that box can sees who you are talking with and what the content of those messages are. There is no end to end encryption. If WeChat/Tencent disagrees with you for some reason they can ban you. This means you can't interact with the box under that name anymore.

    What do we want?​

    We want something that is self-sovereign, private, censorship-resistant and open that allows individuals and groups of people to communicate and transact freely. To explore what this means in more detail, without getting lost in the weeds, we provide the following list of properties. A lot of these are tied together, and some fall out of the other requirements. Some of them stand in slight opposition to each other.

    Self-sovereignity identity. Exercises authority within your own sphere. If you aren't harming anyone, you should be able to have an account and communicate with other people.

    Pseudonymity, and ideally total anonymity. Not having your identity tied to your real name (e.g. through phone number, bank account, ID, etc). This allows people to act more freely without being overly worried about censorship and coercion in the real world. While total anonymity is even more desirable - especially to break multiple hops to a true-name action - real-world constraints sometimes makes this more challenging.

    Private and secure communication. Your communication and who you transact with should be for your eyes only. This includes transactions (transfer of value) as a form of communication.

    Censorship-resistance. Not being able to easily censor individuals on the platform. Both at an individual, group and collective level. Not having single points of failure that allow service to be disrupted.

    Decentralization. Partly falls out of censorship-resistance and other properties. If infrastructure isn't decentralized it means there's a single point of failure that can be disrupted. This is more of a tool than a goal on its own, but it is an important tool.

    Built for mass adoption. Includes scalabiltiy, UX (latency, reliability, bandwidth consumption, UI etc), and allowing for people to stick around. One way of doing this is to allow users to discover people they want to talk to.

    Scalability. Infrastructure needs to support a lot of users to be a viabile alternative. Like, a billion of them (eventually).

    Fundamentals in place to support great user experience. To be a viable alternative, aside from good UI and distribution, fundamentals such as latency, bandwidth usage, consistency etc must support great UX to be a viable alternative.

    Works for resource restricted devices, including smartphones. Most people will use a smartphone to use this. This means it has to work well on them and similar devices, without becoming a second-class citizen where we ignore properties such as censorship-resistance and privacy. Some concession to reality will be necessary due to additional constraints, which leads us to...

    Adaptive nodes. Nodes will have different capabilities, and perhaps at different times. To maintain a lot of the properties described here it is desirable if as many participants as possible are first-class citizens. If a phone is switching from a limited data plan to a WiFi network or from battery to AC power it can do more useful work, and so on. Likewise for a laptop with a lot of free disk space and spare compute power, etc.

    Sustainable. If there's no centralized, top down ad-driven model, this means all the infrastructure has to be sustainable somehow. Since these are individual entitites, this means it has to be paid for. While altruistic modes and similar can be used, this likely requires some form of incentivization scheme for useful services provided in the network. Related: free rider problem.

    Spam resistant. Relates to sustainability, scalability and built for mass adoption. Made more difficult by pseudonymous identity due to whitewashing attacks.

    Trust-minimized. To know that properties are provided for and aren't compromised, various ways of minimizing trust requirements are useful. This also related to mass adoption and social cohesion. Examples include: open and audited protocols, open source, reproducible builds, etc. This also relates to how mini apps are provided for, since we may not know their source but want to be able to use them anyway.

    Open source. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn't been compromised, e.g. by uploading private data to a third party.

    Some of these are graded and a bit subtle, i.e.:

    • Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible.
    • Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience.
    • Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals.
    • "Discover people" and striving for "total anonymity" might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy.
    • Users often want some form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement.

    Counterpoint 1​

    We could refute the above by saying that the design goals are undesirable. We want a system where people can censor others, and where everyone is tied to their real identity. Or we could say something like, freedom of speech is a general concept, and it doesn't apply to Internet companies, even if they provide a vital service. You can survive without it and you should've read the terms of service. This roughly charactericizes the mainstream view.

    Additional factor here is the idea that a group of people know more about what's good for you then you do, so they are protecting you.

    Counterpoint 2​

    We could agree with all these design goals, but think they are too extreme in terms of their requirements. For example, we could operate as a non profit, take donations and volunteers, and then host the whole infrastructure ourselves. We could say we are in a friendly legislation, so we won't be a single point of failure. Since we are working on this and maybe even our designs are open, you can trust us and we'll provide service and infrastructure that gives you what you want without having to pay for it or solve all these complex decentralized computation and so on problems. If you don't trust us for some reason, you shouldn't use us regardless. Also, this is better than status quo. And we are more likely to survive by doing this, either by taking shortcuts or by being less ambituous in terms of scope.

    Principal components​

    There are many ways to skin a cat, but this is one way of breaking down the problem. We have a general direction with the properties listed above, together with some understanding of how WeChat works for the everday user. Now the question is, what infrastructure do we need to support this? How do we achieve the above properties, or at least get closer to them? We want to figure out the necessary building blocks, and one of doing this is to map out likely necessary components.

    Background: Ethereum and Web3 stack​

    It is worth noting that a lot of the required infrastructure has been developed, at least as concepts, in the original Ethereum / Web3 vision. In it there is Ethereum for consensus/compute/transact, storage through Swarm, and communication through Whisper. That said, the main focus has been on the Ethereum blockchain itself, and a lot of things have happened in the last 5y+ with respect to technology around privacy and scalabilty. It is worth revisiting things from a fresh point of view, with the WeChat alternative in mind as a clear use case.

    Account - self-sovereign identity and the perils of phone numbers​

    Starting from the most basic: what is an account and how do you get one? With most internet services today, WeChat and almost all popular messaging apps included, you need to signup with some centralized authority. Usually you also have to verify this with some data that ties this account to you as an individual. E.g. by requiring a phone number, which in most jurisdictions 1 means giving out your real ID. This also means you can be banned from using the service by a somewhat arbitrary process, with no due process.

    Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this.

    Justifications for requiring phone numbers are usually centered around three claims:

    1. Avoiding spam
    2. Tying your account to your real name, for various reasons
    3. Using as a commonly shared identifier as a social network discovery mechanism

    Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on.

    In contrast, a self-sovereign system would allow you to "create an account" completely on your own. This can easily be done with public key cryptograpy, and it also paves the way for end-to-end encryption to make your messages private.

    The main issue with this that you need to get more creative about avoiding spam (e.g. through white washing attacks), and ideally there is some other form of social discovery mechanism.

    Just having a public key as an account isn't enough though. If it goes through a central server, then nothing is stopping that server from arbitrarly blocking requests related to that public key. Of course, this also depends on how transparent such requests are. Fundamentally, lest we rely completely on goodwill, there needs to be multiple actors by which you can use the service. This naturally points to decentralization as a requirement. See counterpoint.

    Even so, if the system is closed source we don't know what it is doing. Perhaps the app communicating is also uploading data to another place, or somehow making it possible to see who is who and act accordingly.

    You might notice that just one simple property, self-sovereign identity, leads to a slew of other requirements and properties. You might also notice that WeChat is far from alone in this, even if their identity requirements might be a bit stringent than, say, Telegram. Their control aspects are also a bit more extreme, at least for someone with western sensibilities 2.

    Most user facing applications have similar issues, Google Apps/FB/Twitter etc. For popular tools that have this built in, we can look at git - which is truly decentralized and have keypair at the bottom. It is for a very specific technical domain, and even then people rely on Github. Key management is fairly difficult even for technical people, and for normal people even more so. Banks are generally far behind on this tech, relying on arcane procedures and special purpose hardware for 2FA. That's another big issue.

    Let's shift gears a bit and talk about some other functional requirements.

    Routing - packets from A to B​

    In order to get a lot of the features WeChat provides, we need the ability to do three things: communicate, store data, and transact with people. We need a bit more than that, but let's focus on this for now.

    To communicate with people, in the base case, we need to go from one phone to another phone that is separated by a large distance. This requires some form of routing. The most natural platform to build this on is the existing Internet, though not the only one. Most phones are resource restricted, and are only "on" for brief periods of time. This is needed to preserve battery and bandwidth. Additionally, Internet uses IPs as endpoints, which change as a phones move through space. NAT punching etc isn't always perfect either. This means we need a way to get a message from one public key to another, and through some intermediate nodes. We can think of these nodes as a form of service network. Similar to how a power grid works, or phone lines, or collection of ISPs.

    One important property here is to ensure we don't end up in a situation like the centralized capture scenario above, something we've seen with centralized ISPs 3 4 where they can choose which traffic is good and which is bad. We want to allow the use of different service nodes, just like if a restaurant gives you food poisioning you can go to the one next door and then the first one goes out of business after a while. And the circle of life continues.

    We shouldn't be naive though, and think that this is something nodes are likely to do for free. They need to be adequately compensated for their services, in some of incentivization scheme. That can either be monetary, or as in the case of Bittorrent, more of a barter situation where you use game theory to coordinate with strangers 5, and some form of reputation attached to it (for private trackers).

    There are many ways of doing routing, and we won't go into too much technical detail here. Suffice to say is that you likely want both a structured and unstructured alternative, and that these comes with several trade-offs when it comes to efficiency, metadata protection, ability to incentivize, compatibility with existing topologies, and suitability for mobilephones (mostly offline, bandwidth restricted, not directly connectable). Expect more on this in a future article.

    Some of these considerations naturally leads us into the storage and transaction components.

    Storage - available and persistant for later​

    If mobile phones are mostly offline, we need some way to store these messages so they can be retrieved when online again. The same goes for various kinds attachments as well, and for when people are switching devices. A user might control their timeline, but in the WeChat case that timeline is stored on Tencent's servers, and queried from there as well. This naturally needs to happen by some other service nodes. In the WeChat case, and for most IMs, the way these servers are paid for is through some indirect ad mechanism. The entity controlling these ads and so on is the same one as the one operating the servers for storage. A more direct model with different entities would see these services being compensated for their work.

    We also need storage for attachments, mini-apps, as well as a way of understanding the current state of consensus when it comes to the compute/transact module. In the WeChat case, this state is completely handled by the bank institution or one of their partners, such as Alibaba. When it comes to bearer instruments like cash, no state needs to be kept as that's a direct exchange in the physical world. This isn't directly compatible with transfering value over a distance.

    All of this state requires availability and persistance. It should be done in a trust minimized fashion and decentralized, which requires some form of incentivization for keeping data around. If it isn't, you are relying on social cohesion which breaks down at very large scales.

    Since data will be spread out across multiple nodes, you need a way to sync data and transfer it in the network. As well as being able to add and query data from it. All of this requires a routing component.

    To make it more censorship resistant it might be better to keep it as a general-purpose store, i.e. individuals don't need to know what they storing. Otherwise, you naturally end up in a situation where individual nodes can be pressured to not store certain content.

    Messaging - from me to you to all of us (not them)​

    This builds on top of routing, but it has a slightly different focus. The goal is to allow for individuals and groups to communicate in a private, secure and censorship-resistant manner.

    It also needs to provide a decent interface to the end user, in terms of dealing seamlessly with offline messages, providing reliable and timely messaging.

    In order to get closer to the ideal of total anonymity, it is useful to be able to hide metadata of who is talking to whom. This applies to both normal communication as well as for transactions. Ideally, no one but the parties involved can see who is taking part in a conversation. This can be achieved through various techniques such as mixnets, anonymous credentials, private information retrieval, and so on. Many of these techniques have a fundamental trade-off with latency and bandwidth, something that is a big concern for mobilephones. Being able to do some form of tuning, in an adaptive node manner, depending on your threat model and current capabilities is useful here.

    The baseline here is pseudonymity, and having tools to allow individuals to "cut off" ties to their real world identity and transactions. People act different in different circles in the real world, and this should be mimicked online as well. Your company, family or government shouldn't be able to know what exactly you use your paycheck for, and who you are talking to.

    Compute - transact, contract and settle​

    The most immediate need here is transaction from A to B. Direct exchange. There is also a more indirect need for private lawmaking and contracting.

    We talked about routing and storage and how they likely need to be incentivized to work properly. How are they going to be compensated? While this could in theory work via existing banking system and so on, this would be rather heavy. It'd also very likely require tying your identifier to your legal name, something that goes against what we want to achieve. What we want is something that acts more as right-to-access, similar to the way cash functions in a society 6. I pay for a fruit with something that is valuable to you and then I'm on my way.

    While there might be other candidates, such as pre-paid debit cards and so on, this transaction mode pretty much requires a cryptocurrency component. The alternative is to do it on a reputation basis, which might work for small communities, due to social cohesion, but quickly detoriates for large ones 7. Ad hoc models like private Bittorrent trackers are centralized and easy to censor.

    Now, none of the existing cryptocurrency models are ideal. They also all suffer from lack of widespread use, and it is difficult to get onboarded to them in the first place. Transactions in Bitcoin are slow. Ethereum is faster and has more capabilities, but it still suffers from linking payments over time, which makes the privacy part of this more difficult. Zcash, Monero and similar are interesting, but also require more use. For Zcash, shielded transactions appear to only account for less than 2% of all transactions in 2019 8 9.

    Another dimension is what sets general purpose cryptocurrencies like Ethereum apart. Aside from just paying from A to B, you can encode rules about when something should be paid out and not. This is very useful for doing a form of private lawmaking, contracting, for setting up service agreements with these nodes. If there's no trivial recourse as in the meatspace world, where you know someone's name and you can sue them, you need a different kind of model.

    What makes something like Zcash interesting is that it works more like digital cash. Instead of leaving a public trail for everyone, where someone can see where you got the initial money from and then trace you across various usage, for Zcash every hop is privacy preserving.

    To fulfill the general goals of being censorship resistance and secure, it is also vital that the system being used stays online and can't be easily disrupted. That points to disintermediation, as opposed to using gateways and exchanges. This is a case where something like cash, or gold, is more direct, since no one can censor this transaction without being physically present where this direct exchange is taking place. However, like before, this doesn't work over distance.

    Secure chat - just our business​

    Similar to the messaging module above. The distinction here is that we assume the network part has already taken place. Here we are interested in keeping the contents of messages private, so that means confidentiality/end-to-end encryption, integrity, authentication, as well as forward secrecy and plausible deniability. This means that even if there's some actor that gets some private key material, or confiscated your phone, there is some level of...ephemerality to your conversations. Another issue here in terms of scalable private group chat.

    Extensible mini apps​

    This relates to the compute and storage module above. Essentially we want to provide mini apps as in WeChat, but to do so in a way that is compatible with what we want to achieve more generally. This allows individuals and small businesses to create small tools for various purposes, and coordinate with strangers. E.g. booking a cab or getting an insurance, and so on.

    This has a higher dependency on the contracting/general computation aspect. I.e. often it isn't only a transaction, but you might want to encode some specific rules here that strangers can abide by without having too high trust requirements. As a simple example: escrows.

    This also needs an open API that anyone can use. It should be properly secured, so using one doesn't compromise the rest of the system it is operating in. To be censorship resistant it requires the routing and storage component to work properly.

    Where are we now?​

    Let's look back at some of desirable properties we set out in the beginning and see how close we are to building out the necessary components. Is it realistic at all or just a pipe dream? We'll see that there are many building blocks in place, and there's reason for hope.

    Self-sovereignity identity. Public key crypto and web of trust like constructs makes this possible.

    Pseudonymity, and ideally total anonymity. Pseudonymity can largely be achieved with public key crypto and open systems that allow for permissionless participation. For transactions, pseudonymity exists in most cryptocurrencies. The challenge is linkage across time, especially when interfacing with other "legacy" system. There are stronger constructs that are actively being worked on and are promising here, such as mixnets (Nym), mixers (Wasabi Wallet, Tornado.Cash) and zero knowledge proofs (Zcash, Ethereum, Starkware). This area of applied research has exploded over the last few years.

    Private and secure communication. Signal has pioneered a lot of this, following OTR. Double Ratchet, X3DH. E2EE is minimum these days, and properties like PFS and PD are getting better. For metadata protection, you have Tor, with its faults, and more active research on mixnets and private information retrieval, etc.

    Censorship-resistance. This covers a lot of ground across the spectrum. You have technologies like Bittorrent, Bitcoin/Ethereum, Tor obfuscated transports, E2EE by default, partial mesh networks in production, abilit to move/replicate host machines more quickly have all made this more of a reality than it used to be. this easier. Of course, techniques such as deep packet inspection and internet shutdowns have increased.

    Decentralization. Cryptocurrencies, projects like libp2p and IPFS. Need to be mindful here of many projects that claim decentralization but are still vulnerable to single points of failures, such as relying on gateways.

    Built for mass adoption. This one is more subjective. There's definitely a lot of work to be done here, both when it comes to fundamental performance, key management and things like social discoverability. Directionally these things are improving and becoming easier for the average person but there is a lot ot be done here.

    Scalability. With projects like Ethereum 2.0 and IPFS more and more resources are a being put into this, both at the consensus/compute layer as well as networking (gossip, scalable Kademlia) layer. Also various layer 2 solutions for transactions.

    Fundamentals in place to support great user experience. Similar to built for mass adoption. As scalability becomes more important, more applied research is being done in the p2p area to improve things like latency, bandwidth.

    Works for resource restricted devices, including smartphones. Work in progress and not enough focus here, generally an after thought. Also have stateless clients etc.

    Adaptive nodes. See above. With subprotocols and capabilities in Ethereum and libp2p, this is getting easier.

    Sustainable. Token economics is a thing. While a lot of it won't stay around, there are many more projects working on making themselves dispensable. Being open source, having an engaged community and enabling users run their own infrastructure. Users as stakeholders.

    Spam resistant. Tricky problem if you want to be pseudonymous, but some signs of hope with incentivization mechanisms, zero knowledge based signaling, etc. Together with various forms of rate limiting and better controlling of topology and network amplification. And just generally being battle-tested by real world attacks, such as historical Ethereum DDoS attacks.

    Trust minimized. Bitcoin. Zero knowledge provable computation. Open source. Reproducible builds. Signed binaries. Incentive compatible structures. Independent audits. Still a lot of work, but getting better.

    Open source. Big and only getting bigger. Including mainstream companies.

    What's next?​

    We've look at what WeChat provides and what we'd like an alternative to look like. We've also seen a few principal modules that are necessary to achieve those goals. To achieve all of this is a daunting task, and one might call it overly ambitiuous. We've also seen how far we've come with some of the goals, and how a lot of the pieces are there, in one form or another. Then it is a question of putting them all together in the right mix.

    The good news is that a lot of people are working all these building blocks and thinking about these problems. Compared to a few years ago we've come quite far when it comes to p2p infrastructure, privacy, security, scalability, and general developer mass and mindshare. If you want to join us in building some of these building blocks, and assembling them, check out our forum.

    PS. We are hiring protocol engineers. DS

    Acknowledgements​

    Corey, Dean, Jacek.

    References​

    1. Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws↩
    2. On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/↩
    3. Net Neutrality: https://www.eff.org/issues/net-neutrality↩
    4. ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/↩
    5. Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf↩
    6. The Case for Electronic Cash: https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf↩
    7. Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html↩
    8. Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions↩
    9. Shielded transactions usage (stats page 404s): https://z.cash/support/faq/↩
    ]]>     -     - - <![CDATA[From Kademlia to Discv5]]> - https://vac.dev/rlog/kademlia-to-discv5 - https://vac.dev/rlog/kademlia-to-discv5 - Thu, 09 Apr 2020 16:00:00 GMT - - A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.

    If you've been working on Ethereum or adjacent technologies you've probably heard of discv4 or discv5. But what are they actually? How do they work and what makes them different? To answer these questions, we need to start at the beginning, so this post will assume that there is little knowledge on the subject so the post should be accessible for anyone.

    The Beginning​

    Let's start right at the beginning: the problem of discovery and organization of nodes in peer-to-peer networks.

    Early P2P file sharing technologies, such as Napster, would share information about who holds what file using a single server. A node would connect to the central server and give it a list of the files it owns. Another node would then connect to that central server, find a node that has the file it is looking for and contact that node. This however was a flawed system -- it was vulnerable to attacks and left a single party open to lawsuits.

    It became clear that another solution was needed, and after years of research and experimentation, we were given the distributed hash table or DHT.

    Distributed Hash Tables​

    In 2001 4 new protocols for such DHTs were conceived, Tapestry, Chord, CAN and Pastry, all of which made various trade-offs and changes in their core functionality, giving them unique characteristics.

    But as said, they're all DHTs. So what is a DHT?

    A distributed hash table (DHT) is essentially a distributed key-value list. Nodes participating in the DHT can easily retrieve the value for a key.

    If we have a network with 9 key-value pairs and 3 nodes, ideally each node would store 3 (optimally 6 for redundancy) of those key-value pairs, meaning that if a key-value pair were to be updated, only part of the network would responsible for ensuring that it is. The idea is that any node in the network would know where to find the specific key-value pair it is looking for based on how things are distributed amongst the nodes.

    Kademlia​

    So now that we know what DHTs are, let's get to Kademlia, the predecessor of discv4. Kademlia was created by Petar Maymounkov and David Mazières in 2002. I will naively say that this is probably one of the most popular and most used DHT protocols. It's quite simple in how it works, so let's look at it.

    In Kademlia, nodes and values are arranged by distance (in a very mathematical definition). This distance is not a geographical one, but rather based on identifiers. It is calculated how far 2 identifiers are from eachother using some distance function.

    Kademlia uses an XOR as its distance function. An XOR is a function that outputs true only when inputs differ. Here is an example with some binary identifiers:

    XOR 10011001
        00110010
        --------
        10101011

    The top in decimal numbers means that the distance between 153 and 50 is 171.

    There are several reasons why XOR was taken:

    1. The distance from one ID to itself will be 0.
    2. Distance is symmetric, A to B is the same as B to A.
    3. Follows triangle inequality, if A, B and C are points on a triangle then the distance A to B is closer or equal to that of A to C plus the one from B to C.

    In summary, this distance function allows a node to decide what is "close" to it and make decisions based on that "closeness".

    Kademlia nodes store a routing table. This table contains multiple lists. Each subsequent list contains nodes which are a little further distanced than the ones included in the previous list. Nodes maintain detailed knowledge about nodes closest to them, and the further away a node is, the less knowledge the node maintains about it.

    So let's say I want to find a specific node. What I would do is go to any node which I already know and ask them for all their neighbours closest to my target. I repeat this process for the returned neighbours until I find my target.

    The same thing happens for values. Values have a certain distance from nodes and their IDs are structured the same way so we can calculate this distance. If I want to find a value, I simply look for the neighbours closest to that value's key until I find the one storing said value.

    For Kademlia nodes to support these functions, there are several messages with which the protocol communicates.

    • PING - Used to check whether a node is still running.
    • STORE - Stores a value with a given key on a node.
    • FINDNODE - Returns the closest nodes requested to a given ID.
    • FINDVALUE - The same as FINDNODE, except if a node stores the specific value it will return it directly.

    This is a very simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the paper or a more in-depth design specification

    Discv4​

    Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT.

    Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the FINDVALUE and STORE commands described by Kademlia.

    The lookup method previously described by Kademlia describes how a node gets its peers. A node contacts some node and asks it for the nodes closest to itself. It does so until it can no longer find any new nodes.

    Additionally, discv4 adds mutual endpoint verification. This is meant to ensure that a peer calling FINDNODE also participates in the discovery protocol.

    Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called ENRRequest.

    If you want some more details on ENRs, check out one of my posts "Network Addresses in Ethereum"

    Discv4 comes with its own range of problems however. Let's look at a few of them.

    Firstly, the way discv4 works right now, there is no way to differentiate between node sub-protocols. This means for example that an Ethereum node could add an Ethereum Classic Node, Swarm or Whisper node to its DHT without realizing that it is invalid until more communication has happened. This inability to differentiate sub-protocols makes it harder to find specific nodes, such as Ethereum nodes with light-client support.

    Next, in order to prevent replay attacks, discv4 uses timestamps. This however can lead to various issues when a host's clock is wrong. For more details, see the "Known Issues" section of the discv4 specification.

    Finally, we have an issue with the way mutual endpoint verification works. Messages can get dropped and there is no way to tell if both peers have verified eachother. This means that we could consider our peer verified while it does not consider us so making them drop the FINDNODE packet.

    Discv5​

    Finally, let's look at discv5. The next iteration of discv4 and the discovery protocol which will be used by Eth 2.0. It aims at fixing various issues present in discv4.

    The first change is the way FINDNODE works. In traditional Kademlia as well as in discv5, we pass an identifier. However, in discv5 we instead pass the logarithmic distance, meaning that a FINDNODE request gets a response containing all nodes at the specified logarithmic distance from the called node.

    Logarithmic distance means we first calculate the distance and then run it through our log base 2 function. See:

    log2(A xor B)

    And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are first in first out lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering ads on their peers.

    As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place ads on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network.

    Additionally, it is unclear how many peers a node should place these ads on and exactly which peers to place them on. For more details, check out the issue devp2p#136.

    There are a bunch more smaller changes to the protocol, but they are less important hence they were ommitted from this summary.

    Nevertheless, discv5 still does not resolve a couple issues present in discv4, such as unreliable endpoint verification. As of writing this post, there is currently no new method in discv5 to improve the endpoint verification process.

    As you can see discv5 is still a work in progress and has a few large challenges to overcome. However if it does, it will most likely be a large improvement to a more naive Kademlia implementations.

    Hopefully this article helped explain what these discovery protocols are and how they work. If you're interested in their full specifications you can find them on github.

    ]]>     -     - - <![CDATA[Waku Update]]> - https://vac.dev/rlog/waku-update - https://vac.dev/rlog/waku-update - Fri, 14 Feb 2020 12:00:00 GMT - - A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?

    Waku is our fork of Whisper where we address the shortcomings of Whisper in an iterative manner. We've seen a in previous post that Whisper doesn't scale, and why. In this post we'll talk about what the current state of Waku is, how many users it can support, and future plans.

    Current state​

    Specs:

    We released Waku spec v0.3 this week! You can see the full changelog here.

    The main change from 0.2 is making the handshake more flexible. This enables us to communicate topic interest immediately without ambiguity. We also did the following:

    • added recommendation for DNS based discovery
    • added an upgradability and compatibility policy
    • cut the spec up into several components

    We cut the spec up in several components to make Vac as modular as possible. The components right now are:

    We can probably factor these out further as the main spec is getting quite big, but this is good enough for now.

    Clients:

    There are currently two clients that implement Waku v0.3, these are Nimbus (Update: now nim-waku) in Nim and status-go in Go.

    For more details on what each client support and don't, you can follow the work in progress checklist.

    Work is currently in progress to integrate it into the Status core app. Waku is expected to be part of their upcoming 1.1 release (see Status app roadmap (link deprecated)).

    Simulation:

    We have a simulation that verifies - or rather, fails to falsify - our scalability model. More on the simulation and what it shows below.

    How many users does Waku support?​

    This is our current understanding of how many users a network running Waku can support. Specifically in the context of the Status chat app, since that's the most immediate consumer of Waku. It should generalize fairly well to most deployments.

    tl;dr (for Status app):

    • beta: 100 DAU
    • v1: 1k DAU
    • v1.1 (waku only): 10k DAU (up to x10 with deployment hotfixes)
    • v1.2 (waku+dns): 100k DAU (can optionally be folded into v1.1)

    Assuming 10 concurrent users = 100 DAU. Estimate uncertainty increases for each order of magnitude until real-world data is observed.

    As far as we know right now, these are the bottlenecks we have:

    • Immediate bottleneck - Receive bandwidth for end user clients (aka ‘Fixing Whisper with Waku’)
    • Very likely bottleneck - Nodes and cluster capacity (aka ‘DNS based node discovery’)
    • Conjecture but not unlikely to appear- Full node traffic (aka ‘the routing / partition problem’)

    We've already seen the first bottleneck being discussed in the initial post. Dean wrote a post on DNS based discovery which explains how we will address the likely second bottleneck. More on the third one in future posts.

    For more details on these bottlenecks, see Scalability estimate: How many users can Waku and the Status app support?.

    Simulation​

    The ultimate test is real-world usage. Until then, we have a simulation thanks to Kim De Mey from the Nimbus team!

    We have two network topologies, Star and full mesh. Both networks have 6 full nodes, one traditional light node with bloom filter, and one Waku light node.

    One of the full nodes sends 1 envelope over 1 of the 100 topics that the two light nodes subscribe to. After that, it sends 10000 envelopes over random topics.

    For light node, bloom filter is set to almost 10% false positive (bloom filter: n=100, k=3, m=512). It shows the number of valid and invalid envelopes received for the different nodes.

    Star network:

    DescriptionPeersValidInvalid
    Master node7100010
    Full node 13100010
    Full node 21100010
    Full node 31100010
    Full node 41100010
    Full node 51100010
    Light node28150
    Waku light node210

    Full mesh:

    DescriptionPeersValidInvalid
    Full node 071000120676
    Full node 17100019554
    Full node 251000123304
    Full node 351000111983
    Full node 451000124425
    Full node 551000123472
    Light node2803803
    Waku light node211

    Things to note:

    • Whisper light node with ~10% false positive gets ~10% of total traffic
    • Waku light node gets ~1000x less envelopes than Whisper light node
    • Full mesh results in a lot more duplicate messages, expect for Waku light node

    Run the simulation yourself here. The parameters are configurable, and it is integrated with Prometheus and Grafana.

    Difference between Waku and Whisper​

    Summary of main differences between Waku v0 spec and Whisper v6, as described in EIP-627:

    • Handshake/Status message not compatible with shh/6 nodes; specifying options as association list
    • Include topic-interest in Status handshake
    • Upgradability policy
    • topic-interest packet code
    • RLPx subprotocol is changed from shh/6 to waku/0.
    • Light node capability is added.
    • Optional rate limiting is added.
    • Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits
    • Mail Server and Mail Client functionality is now part of the specification.
    • P2P Message packet contains a list of envelopes instead of a single envelope.

    Next steps and future plans​

    Several challenges remain to make Waku a robust and suitable base -communication protocol. Here we outline a few challenges that we are addressing and will continue to work on:

    • scalability of the network
    • incentived infrastructure and spam-resistance
    • build with resource restricted devices in mind, including nodes being mostly offline

    For the third bottleneck, a likely candidate for fixing this is Kademlia routing. This is similar to what is done in Swarm's PSS. We are in the early stages of experimenting with this over libp2p in nim-libp2p. More on this in a future post!

    Acknowledgements​

    Image from "caged sky" by mh.xbhd.org is licensed under CC BY 2.0 (https://ccsearch.creativecommons.org/photos/a9168311-78de-4cb7-a6ad-f92be8361d0e)

    ]]>     -     - - <![CDATA[DNS Based Discovery]]> - https://vac.dev/rlog/dns-based-discovery - https://vac.dev/rlog/dns-based-discovery - Fri, 07 Feb 2020 12:00:00 GMT - - A look at EIP-1459 and the benefits of DNS based discovery.

    Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as Kademlia which utilizes a distributed hash table or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data.

    Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a static list of nodes that a client can connect to.

    In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable.

    How do we do this?

    EIP 1459: Node Discovery via DNS, which is one of the strategies we are using for discovering waku nodes. EIP-1459 is a DNS-based discovery protocol that stores merkle trees in DNS records which contain connection information for nodes.

    Waku is our fork of Whisper. Oskar recently wrote an entire post explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification here

    DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be.

    > dig dnsseed.bluematt.me +short
    129.226.73.12
    107.180.78.111
    169.255.56.123
    91.216.149.28
    85.209.240.91
    66.232.124.232
    207.55.53.96
    86.149.241.168
    193.219.38.57
    190.198.210.139
    74.213.232.234
    158.181.226.33
    176.99.2.207
    202.55.87.45
    37.205.10.3
    90.133.4.73
    176.191.182.3
    109.207.166.232
    45.5.117.59
    178.211.170.2
    160.16.0.30

    The above displays the result of querying on of these DNS seeds. All the nodes are stored as A records for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the IRC bootstrapping method in v0.8.2.

    What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices.

    Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name.

    As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so.

    One thing you notice when looking at EIP-1459 is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method?

    There are multiple reasons, but the main one is security. In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in EIP-1459 where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records.

    Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party.

    So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using merkle trees as follows:

    • The root of the tree is stored in a TXT record, this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above.

      Additionally, there is also a root hash to a second tree called a link tree, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names.

      The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version.

    • Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a TXT record. The branch record contains all the hashes of the branch's leafs.

    • Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at EIP-778, or read a short blog post I wrote explaining them here.

    Below is the zone file taken from the EIP-1459, displaying how this looks in practice.

    ; name                        ttl     class type  content
    @                             60      IN    TXT   enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA
    C7HRFPF3BLGF3YR4DY5KX3SMBE    86900   IN    TXT   enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org
    JWXYDBPXYWG6FX3GMDIBFA6CJ4    86900   IN    TXT   enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24
    2XS2367YHAXJFGLZHVAWLQD4ZY    86900   IN    TXT   enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA
    H4FHT4B454P6UXFD7JCYQ5PWDY    86900   IN    TXT   enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI
    MHTDO6TMUBRIA2XWG5LUDACK24    86900   IN    TXT   enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o

    All of this has already been introduced into go-ethereum with the pull request #20094, created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on github.

    I hope this was a helpful explainer into DNS based discovery, and shows EIP-1459's benefits over more traditional DNS-based discovery schemes.

    ]]>     -     - - <![CDATA[Fixing Whisper with Waku]]> - https://vac.dev/rlog/fixing-whisper-with-waku - https://vac.dev/rlog/fixing-whisper-with-waku - Tue, 03 Dec 2019 12:00:00 GMT - - A research log. Why Whisper doesn't scale and how to fix it.

    This post will introduce Waku. Waku is a fork of Whisper that attempts to -addresses some of Whisper's shortcomings in an iterative fashion. We will also -introduce a theoretical scaling model for Whisper that shows why it doesn't -scale, and what can be done about it.

    Introduction​

    Whisper is a gossip-based communication protocol or an ephemeral key-value store -depending on which way you look at it. Historically speaking, it is the -messaging pilllar of Web3, together with -Ethereum for consensus and Swarm for storage.

    Whisper, being a somewhat esoteric protocol and with some fundamental issues, -hasn't seen a lot of usage. However, applications such as Status are using it, -and have been making minor ad hoc modifications to it to make it run on mobile -devices.

    What are these fundamental issues? In short:

    1. scalability, most immediately when it comes to bandwidth usage
    2. spam-resistance, proof of work is a poor mechanism for heterogeneous nodes
    3. no incentivized infrastructure, leading to centralized choke points
    4. lack of formal and unambiguous specification makes it hard to analyze and implement
    5. running over devp2p, which limits where it can run and how

    In this post, we'll focus on the first problem, which is scalability through bandwidth usage.

    Whisper theoretical scalability model​

    (Feel free to skip this section if you want to get right to the results).

    There's widespread implicit knowledge that Whisper "doesn't scale", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see Status Whisper usage -spec).

    Caveats​

    First, some caveats: this model likely contains bugs, has wrong assumptions, or completely misses certain dimensions. However, it acts as a form of existence proof for unscalability, with clear reasons.

    If certain assumptions are wrong, then we can challenge them and reason about them in isolation. It doesn’t mean things will definitely work as the model predicts, and that there aren’t unknown unknowns.

    The model also only deals with receiving bandwidth for end nodes, uses mostly static assumptions of averages, and doesn’t deal with spam resistance, privacy guarantees, accounting, intermediate node or network wide failures.

    Goals​

    1. Ensure network scales by being user or usage bound, as opposed to bandwidth growing in proportion to network size.
    2. Staying with in a reasonable bandwidth limit for limited data plans.
    3. Do the above without materially impacting existing nodes.

    It proceeds through various case with clear assumptions behind them, starting from the most naive assumptions. It shows results for 100 users, 10k users and 1m users.

    Model​

    Case 1. Only receiving messages meant for you [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A4. Only receiving messages meant for you.

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is 1000.0KB/day
    For  1m users, receiving bandwidth is 1000.0KB/day

    ------------------------------------------------------------

    Case 2. Receiving messages for everyone [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A5. Received messages for everyone.

    For 100 users, receiving bandwidth is   97.7MB/day
    For 10k users, receiving bandwidth is    9.5GB/day
    For  1m users, receiving bandwidth is  953.7GB/day

    ------------------------------------------------------------

    Case 3. All private messages go over one discovery topic [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A8. All private messages are received by everyone (same topic) (static).

    For 100 users, receiving bandwidth is   49.3MB/day
    For 10k users, receiving bandwidth is    4.8GB/day
    For  1m users, receiving bandwidth is  476.8GB/day

    ------------------------------------------------------------

    Case 4. All private messages are partitioned into shards [naive case]

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    Case 5. 4 + Bloom filter with false positive rate

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1

    For 100 users, receiving bandwidth is   10.7MB/day
    For 10k users, receiving bandwidth is  978.0MB/day
    For  1m users, receiving bandwidth is   95.5GB/day

    NOTE: Traffic extremely sensitive to bloom false positives
    This completely dominates network traffic at scale.
    With p=1% we get 10k users ~100MB/day and 1m users ~10gb/day)

    ------------------------------------------------------------

    Case 6. Case 5 + Benign duplicate receives

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).

    For 100 users, receiving bandwidth is   21.5MB/day
    For 10k users, receiving bandwidth is    1.9GB/day
    For  1m users, receiving bandwidth is  190.9GB/day

    ------------------------------------------------------------

    Case 7. 6 + Mailserver under good conditions; small bloom fp; mostly offline

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000
    - A10. Bloom filter size (m) (static): 512
    - A11. Bloom filter hash functions (k) (static): 3
    - A12. Bloom filter elements, i.e. topics, (n) (static): 100
    - A13. Bloom filter assuming optimal k choice (sensitive to m, n).
    - A14. Bloom filter false positive proportion of full traffic, p=0.1
    - A15. Benign duplicate receives factor (static): 2
    - A16. No bad envelopes, bad PoW, expired, etc (static).
    - A17. User is offline p% of the time (static) p=0.9
    - A18. No bad request, dup messages for mailservers; overlap perfect (static).
    - A19. Mailserver requests can change false positive rate to be p=0.01

    For 100 users, receiving bandwidth is    3.9MB/day
    For 10k users, receiving bandwidth is  284.8MB/day
    For  1m users, receiving bandwidth is   27.8GB/day

    ------------------------------------------------------------

    Case 8. No metadata protection w bloom filter; 1 node connected; static shard

    Aka waku mode.

    Next step up is to either only use contact code, or shard more aggressively.
    Note that this requires change of other nodes behavior, not just local node.

    Assumptions:
    - A1. Envelope size (static): 1024kb
    - A2. Envelopes / message (static): 10
    - A3. Received messages / day (static): 100
    - A6. Proportion of private messages (static): 0.5
    - A7. Public messages only received by relevant recipients (static).
    - A9. Private messages partitioned across partition shards (static), n=5000

    For 100 users, receiving bandwidth is 1000.0KB/day
    For 10k users, receiving bandwidth is    1.5MB/day
    For  1m users, receiving bandwidth is   98.1MB/day

    ------------------------------------------------------------

    See source -for more detail on the model and its assumptions.

    Takeaways​

    1. Whisper as it currently works doesn’t scale, and we quickly run into unacceptable bandwidth usage.
    2. There are a few factors of this, but largely it boils down to noisy topics usage and use of bloom filters. Duplicate (e.g. see Whisper vs PSS) and bad envelopes are also factors, but this depends a bit more on specific deployment configurations.
    3. Waku mode (case 8) is an additional capability that doesn’t require other nodes to change, for nodes that put a premium on performance.
    4. The next bottleneck after this is the partitioned topics (app/network specific), which either needs to gracefully (and potentially quickly) grow, or an alternative way of consuming those messages needs to be deviced.

    The results are summarized in the graph above. Notice the log-log scale. The -colored backgrounds correspond to the following bandwidth usage:

    • Blue: <10mb/d (<~300mb/month)
    • Green: <30mb/d (<~1gb/month)
    • Yellow: <100mb/d (<~3gb/month)
    • Red: >100mb/d (>3gb/month)

    These ranges are somewhat arbitrary, but are based on user -requirements for users -on a limited data plan, with comparable usage for other messaging apps.

    Introducing Waku​

    Motivation for a new protocol​

    Apps such as Status will likely use something like Whisper for the forseeable -future, and we want to enable them to use it with more users on mobile devices -without bandwidth exploding with minimal changes.

    Additionally, there's not a clear cut alternative that maps cleanly to the -desired use cases (p2p, multicast, privacy-preserving, open, etc).

    We are actively researching, developing and collaborating with more greenfield -approaches. It is likely that Waku will either converge to those, or Waku will -lay the groundwork (clear specs, common issues/components) necessary to make -switching to another protocol easier. In this project we want to emphasize -iterative work with results on the order of weeks.

    Briefly on Waku mode​

    • Doesn’t impact existing clients, it’s just a separate node and capability.
    • Other nodes can still use Whisper as is, like a full node.
    • Sacrifices metadata protection and incurs higher connectivity/availability requirements for scalbility

    Requirements:

    • Exposes API to get messages from a set of list of topics (no bloom filter)
    • Way of being identified as a Waku node (e.g. through version string)
    • Option to statically encode this node in app, e.g. similar to custom bootnodes/mailserver
    • Only node that needs to be connected to, possibly as Whisper relay / mailserver hybrid

    Provides:

    • likely provides scalability of up to 10k users and beyond
    • with some enhancements to partition topic logic, can possibly scale up to 1m users (app/network specific)

    Caveats:

    • hasn’t been tested in a large-scale simulation
    • other network and intermediate node bottlenecks might become apparent (e.g. full bloom filter and private cluster capacity; can likely be dealt with in isolation using known techniques, e.g. load balancing) (deployment specific)

    Progress so far​

    In short, we have a Waku version 0 spec up as well as a PoC for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for Waku mode. See rough roadmap, project board [link deprecated] and progress thread on the Vac forum.

    The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously ad hoc implemented features, such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the geth (Go), status/whisper (Go) and nim-eth (Nim) versions, specifically around light node usage and the handshake.

    If you are interested in this effort, please check out our forum for questions, comments and proposals. We already have some discussion for better spam protection (see previous post for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement.

    ]]>     -     - - <![CDATA[Feasibility Study: Semaphore rate limiting through zkSNARKs]]> - https://vac.dev/rlog/feasibility-semaphore-rate-limiting-zksnarks - https://vac.dev/rlog/feasibility-semaphore-rate-limiting-zksnarks - Fri, 08 Nov 2019 12:00:00 GMT - - A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks.

    tldr: Moon math promising for solving spam in Whisper, but to get there we need to invest more in performance work and technical upskilling.

    Motivating problem​

    In open p2p networks for messaging, one big problem is spam-resistance. Existing solutions, such as Whisper's proof of work, are insufficient, especially for heterogeneous nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.

    One possible solution is to use a right-to-access staking-based method, where a node is only able to send a message, signal, at a certain rate, and otherwise they can be slashed. One problem with this is in terms of privacy-preservation, where we specifically don't want a user to be tied to a specific payment or unique fingerprint.

    In addition to above, there are a lot of related problems that share similarities in terms of their structure and proposed solution.

    • Private transactions (Zcash, AZTEC)
    • Private voting (Semaphore)
    • Private group membership (Semaphore)
    • Layer 2 scaling, poss layer 1 (ZK Rollup; StarkWare/Eth2-3)

    Overview​

    Basic terminology​

    A zero-knowledge proof allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id.

    zkSNARKs is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more.

    Semaphore is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See current Github home for more.

    Circom is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See Github for more.

    Basic flow​

    We start with a private voting example, and then extend it to the slashable rate limiting example.

    1. A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are.

    2. When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some public outputs, and can be verified by anyone (including a smart contract).

    3. Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as "2020-01-01 vote on Foo Bar for president".

    4. Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once.

    5. As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified.

    Rate limiting example​

    In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat.

    NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same.

    1. Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s.

    2. Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key).

    3. This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them.

    4. As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it.

    Briefly on scope of 'approved users'​

    In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration.

    Technical details​

    For a fairly self-contained set of examples above, see exploration in Vac research repo. Note that the Shamir secret sharing is not inside the SNARK, but out-of-band for now.

    The current version of Semaphore is using NodeJS and Circom from Iden3 for Snarks.

    For more on rate limiting idea, see ethresearch post.

    Feasibility​

    The above repo was used to exercise the basic paths and to gain intution of feasibility. Based on it and related reading we outline a few blockers and things that require further study.

    Technical feasibility​

    Proof time​

    Prove time for Semaphore (https://github.com/kobigurk/semaphore) zKSNARKs using circom, groth and snarkjs is currently way too long. It takes on the order of ~10m to generate a proof. With Websnark, it is likely to take 30s, which might still be too long. We should experiment with native code on mobile here.

    See details.

    Proving key size​

    Prover key size is ~110mb for Semaphore. Assuming this is embedded on mobile device, it bloats the APK a lot. Current APK size is ~30mb and even that might be high for people with limited bandwidth.

    See details.

    Trusted setup​

    Using zkSNARKs a trusted setup is required to generate prover and verifier keys. As part of this setup, a toxic parameter lambda is generated. If a party gets access to this lambda, they can prove anything. This means people using zKSNARKs usually have an elaborate MPC ceremony to ensure this parameter doesn't get discovered.

    See details.

    Shamir logic in SNARK​

    For Semaphore RLN we need to embed the Shamir logic inside the SNARK in order to do slashing for spam. Currently the implementation is trusted and very hacky.

    See details.

    End to end integation​

    Currently is standalone and doesn't touch multiple users, deployed contract with merkle tree and verification, actual transactions, a mocked network, add/remove members, etc. There are bound to be edge cases and unknown unknowns here.

    See details.

    Licensing issues​

    Currently Circom uses a GPL license, which can get tricky when it comes to the App Store etc.

    See details.

    Alternative ZKPs?​

    Some of the isolated blockers for zKSNARKs (#7, #8, #9) might be mitigated by the use of other ZKP technology. However, they likely have their own issues.

    See details.

    Social feasibility​

    Technical skill​

    zkSNARKs and related technologies are quite new. To learn how they work and get an intuition for them requires individuals to dedicate a lot of time to studying them. This means we must make getting competence in these technologies if we wish to use them to our advantage.

    Time and resources​

    In order for this and related projects (such as private transaction) to get anywhere, it must be made an explicit area of focus for an extend period of time.

    General thoughts​

    Similar to Whisper, and in line with moving towards protocol and infrastructure, we need to upskill and invest resources into this. This doesn't mean developing all of the technologies ourselves, but gaining enough competence to leverage and extend existing solutions by the growing ZKP community.

    For example, this might also include leveraging largely ready made solutions such as AZTEC for private transaction; more fundamental research into ZK rollup and similar; using Semaphore for private group membership and private voting; Nim based wrapper aronud Bellman, etc.

    Acknowledgement​

    Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues.

    Peacock header image from [Tonos](<https://en.wikipedia.org/wiki/File:Flickr-lo.tangelini-Tonos(1).jpg>)._

    ]]>     -     - - <![CDATA[P2P Data Sync with a Remote Log]]> - https://vac.dev/rlog/remote-log - https://vac.dev/rlog/remote-log - Fri, 04 Oct 2019 12:00:00 GMT - - A research log. Asynchronous P2P messaging? Remote logs to the rescue!

    A big problem when doing end-to-end data sync between mobile nodes is that most devices are offline most of the time. With a naive approach, you quickly run into issues of 'ping-pong' behavior, where messages have to be constantly retransmitted. We saw some basic calculations of what this bandwidth multiplier looks like in a previous post.

    While you could do some background processing, this is really battery-draining, and on iOS these capabilities are limited. A better approach instead is to loosen the constraint that two nodes need to be online at the same time. How do we do this? There are two main approaches, one is the store and forward model, and the other is a remote log.

    In the store and forward model, we use an intermediate node that forward messages on behalf of the recipient. In the remote log model, you instead replicate the data onto some decentralized storage, and have a mutable reference to the latest state, similar to DNS. While both work, the latter is somewhat more elegant and "pure", as it has less strict requirements of an individual node's uptime. Both act as a highly-available cache to smoothen over non-overlapping connection windows between endpoints.

    In this post we are going to describe how such a remote log schema could work. Specifically, how it enhances p2p data sync and takes care of the following requirements:

    1. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices -that are resource restricted, mostly-offline and often changing network.
    1. MAY use helper services in order to be more mobile-friendly. Examples of -helper services are decentralized file storage solutions such as IPFS and -Swarm. These help with availability and latency of data for mostly-offline -devices.

    Remote log​

    A remote log is a replication of a local log. This means a node can read data from a node that is offline.

    The spec is in an early draft stage and can be found here. A very basic spike / proof-of-concept can be found here.

    Definitions​

    TermDefinition
    CASContent-addressed storage. Stores data that can be addressed by its hash.
    NSName system. Associates mutable data to a name.
    Remote logReplication of a local log at a different location.

    Roles​

    There are four fundamental roles:

    1. Alice
    2. Bob
    3. Name system (NS)
    4. Content-addressed storage (CAS)

    The remote log is the data format of what is stored in the name system.

    "Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.

    Flow​

    Figure 1: Remote log data synchronization.

    Data format​

    The remote log lets receiving nodes know what data they are missing. Depending on the specific requirements and capabilities of the nodes and name system, the information can be referred to differently. We distinguish between three rough modes:

    1. Fully replicated log
    2. Normal sized page with CAS mapping
    3. "Linked list" mode - minimally sized page with CAS mapping

    A remote log is simply a mapping from message identifiers to their corresponding address in a CAS:

    Message Identifier (H1)CAS Hash (H2)
    H1_3H2_3
    H1_2H2_2
    H1_1H2_1
    address to next page

    The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.

    Message Identifier (H1)Content
    H1_3C3
    H1_2C2
    H1_1C1
    address to next page

    Both patterns can be used in parallel, e,g. by storing the last k messages directly and use CAS pointers for the rest. Together with the next_page page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.

    Interaction with MVDS​

    vac.mvds.Message payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.

    Future work​

    The spec is still in an early draft stage, so it is expected to change. Same with the proof of concept. More work is needed on getting a fully featured proof of concept with specific CAS and NAS instances. E.g. Swarm and Swarm Feeds, or IPFS and IPNS, or something else.

    For data sync in general:

    • Make consistency guarantees more explicit for app developers with support for sequence numbers and DAGs, as well as the ability to send non-synced messages. E.g. ephemeral typing notifications, linear/sequential history and casual consistency/DAG history
    • Better semantics and scalability for multi-user sync contexts, e.g. CRDTs and joining multiple logs together
    • Better usability in terms of application layer usage (data sync clients) and supporting more transports

    PS1. Thanks everyone who submitted great logo proposals for Vac!

    PPS2. Next week on October 10th decanus and I will be presenting Vac at Devcon, come say hi :)

    ]]>     -     - - <![CDATA[Vac - A Rough Overview]]> - https://vac.dev/rlog/vac-overview - https://vac.dev/rlog/vac-overview - Fri, 02 Aug 2019 12:00:00 GMT - - Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. What does that mean? Let's unpack it a bit.

    Basic terms​

    messaging stack. While the initial focus is on data sync, we are concerned with all layers in the stack. That means all the way from underlying transports, p2p overlays and routing, to initial trust establishment and semantics for things like group chat. The ultimate goal is to give application developers the tools they need to provide secure messaging for their users, so they can focus on their domain expertise.

    modular. Unlike many other secure messaging applications, our goal is not to have a tightly coupled set of protocols, nor is it to reinvent the wheel. Instead, we aim to provide options at each layer in the stack, and build on the shoulders of giants, putting a premimum on interoperability. It's similar in philosophy to projects such as libp2p or Substrate in that regard. Each choice comes with different trade-offs, and these look different for different applications.

    peer-to-peer. The protocols we work on are pure p2p, and aim to minimize centralization. This too is in opposition to many initiatives in the secure messaging space.

    messaging. By messaging we mean messaging in a generalized sense. This includes both human to human communication, as well machine to machine communication. By messaging we also mean something more fundamental than text messages, we also include things like transactions (state channels, etc) under this moniker.

    secure messaging. Outside of traditional notions of secure messaging, such as ensuring end to end encryption, forward secrecy, avoiding MITM-attacks, etc, we are also concerned with two other forms of secure messaging. We call these private messaging and censorship-resistance. Private messaging means viewing privacy as a security property, with all that entails. Censorship resistance ties into being p2p, but also in terms of allowing for transports and overlays that can't easily be censored by port blocking, traffic analysis, and similar.

    Vāc. Is a Vedic goddess of speech. It also hints at being a vaccine.

    Protocol stack​

    What does this stack look like? We take inspiration from core internet architecture, existing survey work and other efforts that have been done to decompose the problem into orthogonal pieces. Each layer provides their own set of properties and only interact with the layers it is adjacent to. Note that this is a rough sketch.

    Layer / ProtocolPurposeExamples
    Application layerEnd user semantics1:1 chat, group chat
    Data SyncData consistencyMVDS, BSP
    Secure TransportConfidentiality, PFS, etcDouble Ratchet, MLS
    Transport PrivacyTransport and metadata protectionWhisper, Tor, Mixnet
    P2P OverlayOverlay routing, NAT traversaldevp2p, libp2p
    Trust EstablishmentEstablishing end-to-end trustTOFU, web of trust

    As an example, end user semantics such as group chat or moderation capabilities can largely work regardless of specific choices further down the stack. Similarly, using a mesh network or Tor doesn't impact the use of Double Ratchet at the Secure Transport layer.

    Data Sync plays a similar role to what TCP does at the transport layer in a traditional Internet architecture, and for some applications something more like UDP is likely to be desirable.

    In terms of specific properties and trade-offs at each layer, we'll go deeper down into them as we study them. For now, this is best treated as a rough sketch or mental map.

    Problems and rough priorities​

    With all the pieces involved, this is quite an undertaking. Luckily, a lot of pieces are already in place and can be either incorporated as-is or iterated on. In terms of medium and long term, here's a rough sketch of priorities and open problems.

    1. Better data sync. While the current MVDS works, it is lacking in a few areas:
    • Lack of remote log for mostly-offline offline devices
    • Better scalability for multi-user chat contexts
    • Better usability in terms of application layer usage and supporting more transports
    1. Better transport layer support. Currently MVDS runs primarily over Whisper, which has a few issues:
    • scalability, being able to run with many nodes
    • spam-resistance, proof of work is a poor mechanism for heterogeneous devices
    • no incentivized infrastructure, leading to centralized choke points

    In addition to these most immediate concerns, there are other open problems. Some of these are overlapping with the above.

    1. Adaptive nodes. Better support for resource restricted devices and nodes of varying capabilities. Light connection strategy for resources and guarantees. Security games to outsource processing with guarantees.

    2. Incentivized and spam-resistant messaging. Reasons to run infrastructure and not relying on altruistic nodes. For spam resistance, in p2p multicast spam is a big attack vector due to amplification. There are a few interesting directions here, such as EigenTrust, proof of burn with micropayments, and leveraging zero-knowledge proofs.

    3. Strong privacy guarantees at transport privacy layer. More rigorous privacy guarantees and explicit trade-offs for metadata protection. Includes Mixnet.

    4. Censorship-resistant and robust P2P overlay. NAT traversal; running in the browser; mesh networks; pluggable transports for traffic obfuscation.

    5. Scalable and decentralized secure conversational security. Strong security guarantees such as forward secrecy, post compromise security, for large group chats. Includes projects such MLS and extending Double Ratchet.

    6. Better trust establishment and key handling. Avoiding MITM attacks while still enabling a good user experience. Protecting against ghost users in group chat and providing better ways to do key handling.

    There is also a set of more general problems, that touch multiple layers:

    1. Ensuring modularity and interoperability. Providing interfaces that allow for existing and new protocols to be at each layer of the stack.

    2. Better specifications. Machine-readable and formally verified specifications. More rigorous analysis of exact guarantees and behaviors. Exposing work in such a way that it can be analyzed by academics.

    3. Better simulations. Providing infrastructure and tooling to be able to test protocols in adverse environments and at scale.

    4. Enabling excellent user experience. A big reason for the lack of widespread adoption of secure messaging is the fact that more centralized, insecure methods provide a better user experience. Given that incentives can align better for users interested in secure messaging, providing an even better user experience should be doable.

    We got some work to do. Come help us if you want. See you in the next update!

    ]]>     -     - - <![CDATA[P2P Data Sync for Mobile]]> - https://vac.dev/rlog/p2p-data-sync-for-mobile - https://vac.dev/rlog/p2p-data-sync-for-mobile - Fri, 19 Jul 2019 12:00:00 GMT - - A research log. Reliable and decentralized, pick two.

    Together with decanus, I've been working on the problem of data sync lately.

    In building p2p messaging systems, one problem you quickly come across is the problem of reliably transmitting data. If there's no central server with high availability guarantees, you can't meaningfully guarantee that data has been transmitted. One way of solving this problem is through a synchronization protocol.

    There are many synchronization protocols out there and I won't go into detail of how they differ with our approach here. Some common examples are Git and Bittorrent, but there are also projects like IPFS, Swarm, Dispersy, Matrix, Briar, SSB, etc.

    Problem motivation​

    Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices.

    Why p2p?​

    For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from.

    Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat.

    Why reliably sync data?​

    At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on.

    Why mobilephones?​

    Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence.

    More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc.

    Minimal Requirements​

    For requirements or design goals for a solution, here's what we came up with.

    1. MUST sync data reliably between devices. By reliably we mean having the ability to deal with messages being out of order, dropped, duplicated, or delayed.

    2. MUST NOT rely on any centralized services for reliability. By centralized services we mean any single point of failure that isn’t one of the endpoint devices.

    3. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices that are resource restricted, mostly-offline and often changing network.

    4. MAY use helper services in order to be more mobile-friendly. Examples of helper services are decentralized file storage solutions such as IPFS and Swarm. These help with availability and latency of data for mostly-offline devices.

    5. MUST have the ability to provide casual consistency. By casual consistency we mean the commonly accepted definition in distributed systems literature. This means messages that are casually related can achieve a partial ordering.

    6. MUST support ephemeral messages that don’t need replication. That is, allow for messages that don’t need to be reliabily transmitted but still needs to be transmitted between devices.

    7. MUST allow for privacy-preserving messages and extreme data loss. By privacy-preserving we mean things such as exploding messages (self-destructing messages). By extreme data loss we mean the ability for two trusted devices to recover from a, deliberate or accidental, removal of data.

    8. MUST be agnostic to whatever transport it is running on. It should not rely on specific semantics of the transport it is running on, nor be tightly coupled with it. This means a transport can be swapped out without loss of reliability between devices.

    MVDS - a minimium viable version​

    The first minimum viable version is in an alpha stage, and it has a specification, implementation and we have deployed it in a console client for end to end functionality. It's heavily inspired by Bramble Sync Protocol.

    The spec is fairly minimal. You have nodes that exchange records over some secure transport. These records are of different types, such as OFFER, MESSAGE, REQUEST, and ACK. A peer keep tracks of the state of message for each node it is interacting with. There's also logic for message retransmission with exponential delay. The positive ACK and retransmission model is quite similar to how TCP is designed.

    There are two different modes of syncing, interactive and batch mode. See sequence diagrams below.

    Interactive mode: -

    Interactive mode

    Batch mode: -

    Batch mode

    Which mode should you choose? It's a tradeoff of latency and bandwidth. If you want to minimize latency, batch mode is better. If you care about preserving bandwidth interactive mode is better. The choice is up to each node.

    Basic simulation​

    Initial ad hoc bandwidth and latency testing shows some issues with a naive approach. Running with the default simulation settings:

    • communicating nodes: 2
    • nodes using interactive mode: 2
    • interval between messages: 5s
    • time node is offine: 90%
    • nodes each node is sharing with: 2

    we notice a huge overhead. More specifically, we see a ~5 minute latency overhead and a bandwidth multiplier of x100-1000, i.e. 2-3 orders of magnitude just for receiving a message with interactive mode, without acks.

    Now, that seems terrible. A moment of reflection will reveal why that is. If each node is offline uniformly 90% of the time, that means that each record will be lost 90% of the time. Since interactive mode requires offer, request, payload (and then ack), that's three links just for Bob to receive the actual message.

    Each failed attempt implies another retransmission. That means we have (1/0.1)^3 = 1000 expected overhead to receive a message in interactive mode. The latency follows naturally from that, with the retransmission logic.

    Mostly-offline devices​

    The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost.

    There are a few ways of getting around this issue. One is having a store and forward model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status.

    Another, arguably more pure and robust, way is having a remote log, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS.

    What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are required to get reliable data transmission.

    Basic calculations for bandwidth multiplier​

    While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions:

    • two nodes exchanging a single message in batch mode
    • 10% uniformly random uptime for each node
    • in HA cache case, 100% uptime of a piece of infrastructure C
    • retransmission every epoch (with constant or exponential backoff)
    • only looking at average (p50) case

    First case, no helper services​

    A sends a message to B, and B acks it.

    A message -> B (10% chance of arrival)
    A   <- ack   B (10% chance of arrival)

    With a constant backoff, A will send messages at epoch 1, 2, 3, .... With exponential backoff and a multiplier of 2, this would be 1, 2, 4, 8, .... Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier.

    There's a difference between time to receive and time to stop sending. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore:

    1. A will send messages until it receives an ACK.
    2. B will send ACK if it receives a message.

    To get an average of one ack through, A needs to send 100 messages, and B send on average 10 acks. That's a multiplier of roughly a 100. That's roughly what we saw with the simulation above for receiving a message in interactive mode.

    Second case, high-availability caching layer​

    Let's introduce a helper node or piece of infrastructure, C. Whenever A or B sends a message, it also sends it to C. Whenever A or B comes online, it queries for messages with C.

    A message    -> B (10% chance of arrival)
    A message    -> C (100% chance of arrival)
    B <- req/res -> C (100% chance of arrival)
    A   <- ack      B (10% chance of arrival)
    C   <- ack      B (100% chance of arrival)
    A <- req/res -> C (100% chance of arrival)

    What's the probability that A's messages will arrive at B? Directly, it's still 10%. But we can assume it's 100% that C picks up the message. (Giving C a 90% chance success rate doesn't materially change the numbers).

    B will pick up A's message from C after an average of 10 epochs. Then B will send ack to A, which will also be picked up by C 100% of the time. Once A comes online again, it'll query C and receive B's ack.

    Assuming we use exponential backoff with a multiplier of 2, A will send a message directly to B at epoch 1, 2, 4, 8 (assuming it is online). At this point, epoch 10, B will be online in the average case. These direct sends will likely fail, but B will pick the message up from C and send one ack, both directly to A and to be picked up by C. Once A comes online, it'll query C and receive the ack from B, which means it won't do any more retransmits.

    How many messages have been sent? Not counting interactions with C, A sends 4 (at most) and B 1. Depending on if the interaction with C is direct or indirect (i.e. multicast), the factor for interaction with C will be ~2. This means the total bandwidth multiplier is likely to be <10, which is a lot more acceptable.

    Since the syncing semantics are end-to-end, this is without relying on the reliablity of C.

    Caveat​

    Note that both of these are probabilistic argument. They are also based on heuristics. More formal analysis would be desirable, as well as better simulations to experimentally verify them. In fact, the calculations could very well be wrong!

    Future work​

    There are many enhancements that can be made and are desirable. Let's outline a few.

    1. Data sync clients. Examples of actual usage of data sync, with more interesting domain semantics. This also includes usage of sequence numbers and DAGs to know what content is missing and ought to be synced.

    2. Remote log. As alluded to above, this is necessary. It needs a more clear specification and solid proof of concepts.

    3. More efficient ways of syncing with large number of nodes. When the number of nodes goes up, the algorithmic complexity doesn't look great. This also touches on things such as ambient content discovery.

    4. More robust simulations and real-world deployments. Exisiting simulation is ad hoc, and there are many improvements that can be made to gain more confidence and identify issues. Additionally, better formal analysis.

    5. Example usage over multiple transports. Including things like sneakernet and meshnets. The described protocol is designed to work over unstructured, structured and private p2p networks. In some cases it can leverage differences in topology, such as multicast, or direct connections.

    ]]>     +Anonymity-preserving mechanisms we plan to investigate and eventually specify as pluggable anonymity protocols for Waku comprise

    +
      +
    • Dandelion++ for lightweight anonymity;
      • +
    • onion routing as a building block adding a low latency anonymization layer;
      • +
    • a mix network for providing strong anonymity (on top of onion routing) even in the strongest attacker model at the cost of higher latency.
      • +
    +

    These pluggable anonymity-preserving protocols will form a sub-set of the Waku v2 protocol set. +As an intermediate step, we might directly employ Tor for onion-routing, and Nym as a mix-net layer.

    +

    In future research log posts, we will cover further Waku v2 protocols and identify anonymity problems that will be added to our roadmap. +These protocols comprise

    +
      +
    • 13/WAKU2-STORE, which can violate receiver anonymity as it allows filtering by content topic. +A countermeasure is using the content topic exclusively for local filters.
      • +
    • 12/WAKU2-FILTER, which discloses nodes' interest in topics;
      • +
    • 19/WAKU2-LIGHTPUSH, which also discloses nodes' interest in topics and links the lightpush client as the sender of a message to the lightpush service node;
      • +
    • 21/WAKU2-FTSTORE, which discloses nodes' interest in specific time ranges allowing to infer information like online times.
      • +
    +

    While these protocols are not necessary for the operation of Waku v2, and can be seen as pluggable features, +we aim to provide alternatives without the cost of lowering the anonymity level.

    +

    References​

    +]]>     \ No newline at end of file diff --git a/rlog/vac-overview/index.html b/rlog/vac-overview/index.html index c4d28eba..909234df 100644 --- a/rlog/vac-overview/index.html +++ b/rlog/vac-overview/index.html @@ -1,26 +1,94 @@ - + - -Vac - A Rough Overview | Vac Research + +Vac - A Rough Overview | Vac Research - - - - + + + + -
    -

    Vac - A Rough Overview

    by
    6 min read

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. What does that mean? Let's unpack it a bit.

    Basic terms​

    messaging stack. While the initial focus is on data sync, we are concerned with all layers in the stack. That means all the way from underlying transports, p2p overlays and routing, to initial trust establishment and semantics for things like group chat. The ultimate goal is to give application developers the tools they need to provide secure messaging for their users, so they can focus on their domain expertise.

    modular. Unlike many other secure messaging applications, our goal is not to have a tightly coupled set of protocols, nor is it to reinvent the wheel. Instead, we aim to provide options at each layer in the stack, and build on the shoulders of giants, putting a premimum on interoperability. It's similar in philosophy to projects such as libp2p or Substrate in that regard. Each choice comes with different trade-offs, and these look different for different applications.

    peer-to-peer. The protocols we work on are pure p2p, and aim to minimize centralization. This too is in opposition to many initiatives in the secure messaging space.

    messaging. By messaging we mean messaging in a generalized sense. This includes both human to human communication, as well machine to machine communication. By messaging we also mean something more fundamental than text messages, we also include things like transactions (state channels, etc) under this moniker.

    secure messaging. Outside of traditional notions of secure messaging, such as ensuring end to end encryption, forward secrecy, avoiding MITM-attacks, etc, we are also concerned with two other forms of secure messaging. We call these private messaging and censorship-resistance. Private messaging means viewing privacy as a security property, with all that entails. Censorship resistance ties into being p2p, but also in terms of allowing for transports and overlays that can't easily be censored by port blocking, traffic analysis, and similar.

    Vāc. Is a Vedic goddess of speech. It also hints at being a vaccine.

    Protocol stack​

    What does this stack look like? We take inspiration from core internet architecture, existing survey work and other efforts that have been done to decompose the problem into orthogonal pieces. Each layer provides their own set of properties and only interact with the layers it is adjacent to. Note that this is a rough sketch.

    Layer / ProtocolPurposeExamples
    Application layerEnd user semantics1:1 chat, group chat
    Data SyncData consistencyMVDS, BSP
    Secure TransportConfidentiality, PFS, etcDouble Ratchet, MLS
    Transport PrivacyTransport and metadata protectionWhisper, Tor, Mixnet
    P2P OverlayOverlay routing, NAT traversaldevp2p, libp2p
    Trust EstablishmentEstablishing end-to-end trustTOFU, web of trust

    As an example, end user semantics such as group chat or moderation capabilities can largely work regardless of specific choices further down the stack. Similarly, using a mesh network or Tor doesn't impact the use of Double Ratchet at the Secure Transport layer.

    Data Sync plays a similar role to what TCP does at the transport layer in a traditional Internet architecture, and for some applications something more like UDP is likely to be desirable.

    In terms of specific properties and trade-offs at each layer, we'll go deeper down into them as we study them. For now, this is best treated as a rough sketch or mental map.

    Problems and rough priorities​

    With all the pieces involved, this is quite an undertaking. Luckily, a lot of pieces are already in place and can be either incorporated as-is or iterated on. In terms of medium and long term, here's a rough sketch of priorities and open problems.

    1. Better data sync. While the current MVDS works, it is lacking in a few areas:
    • Lack of remote log for mostly-offline offline devices
    • Better scalability for multi-user chat contexts
    • Better usability in terms of application layer usage and supporting more transports
    1. Better transport layer support. Currently MVDS runs primarily over Whisper, which has a few issues:
    • scalability, being able to run with many nodes
    • spam-resistance, proof of work is a poor mechanism for heterogeneous devices
    • no incentivized infrastructure, leading to centralized choke points

    In addition to these most immediate concerns, there are other open problems. Some of these are overlapping with the above.

    1. Adaptive nodes. Better support for resource restricted devices and nodes of varying capabilities. Light connection strategy for resources and guarantees. Security games to outsource processing with guarantees.

    2. Incentivized and spam-resistant messaging. Reasons to run infrastructure and not relying on altruistic nodes. For spam resistance, in p2p multicast spam is a big attack vector due to amplification. There are a few interesting directions here, such as EigenTrust, proof of burn with micropayments, and leveraging zero-knowledge proofs.

    3. Strong privacy guarantees at transport privacy layer. More rigorous privacy guarantees and explicit trade-offs for metadata protection. Includes Mixnet.

    4. Censorship-resistant and robust P2P overlay. NAT traversal; running in the browser; mesh networks; pluggable transports for traffic obfuscation.

    5. Scalable and decentralized secure conversational security. Strong security guarantees such as forward secrecy, post compromise security, for large group chats. Includes projects such MLS and extending Double Ratchet.

    6. Better trust establishment and key handling. Avoiding MITM attacks while still enabling a good user experience. Protecting against ghost users in group chat and providing better ways to do key handling.

    There is also a set of more general problems, that touch multiple layers:

    1. Ensuring modularity and interoperability. Providing interfaces that allow for existing and new protocols to be at each layer of the stack.

    2. Better specifications. Machine-readable and formally verified specifications. More rigorous analysis of exact guarantees and behaviors. Exposing work in such a way that it can be analyzed by academics.

    3. Better simulations. Providing infrastructure and tooling to be able to test protocols in adverse environments and at scale.

    4. Enabling excellent user experience. A big reason for the lack of widespread adoption of secure messaging is the fact that more centralized, insecure methods provide a better user experience. Given that incentives can align better for users interested in secure messaging, providing an even better user experience should be doable.

    We got some work to do. Come help us if you want. See you in the next update!

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Vac - A Rough Overview

    by
    6 min read

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.

    + +

    Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. What does that mean? Let's unpack it a bit.

    +

    Basic terms​

    +

    messaging stack. While the initial focus is on data sync, we are concerned with all layers in the stack. That means all the way from underlying transports, p2p overlays and routing, to initial trust establishment and semantics for things like group chat. The ultimate goal is to give application developers the tools they need to provide secure messaging for their users, so they can focus on their domain expertise.

    +

    modular. Unlike many other secure messaging applications, our goal is not to have a tightly coupled set of protocols, nor is it to reinvent the wheel. Instead, we aim to provide options at each layer in the stack, and build on the shoulders of giants, putting a premimum on interoperability. It's similar in philosophy to projects such as libp2p or Substrate in that regard. Each choice comes with different trade-offs, and these look different for different applications.

    +

    peer-to-peer. The protocols we work on are pure p2p, and aim to minimize centralization. This too is in opposition to many initiatives in the secure messaging space.

    +

    messaging. By messaging we mean messaging in a generalized sense. This includes both human to human communication, as well machine to machine communication. By messaging we also mean something more fundamental than text messages, we also include things like transactions (state channels, etc) under this moniker.

    +

    secure messaging. Outside of traditional notions of secure messaging, such as ensuring end to end encryption, forward secrecy, avoiding MITM-attacks, etc, we are also concerned with two other forms of secure messaging. We call these private messaging and censorship-resistance. Private messaging means viewing privacy as a security property, with all that entails. Censorship resistance ties into being p2p, but also in terms of allowing for transports and overlays that can't easily be censored by port blocking, traffic analysis, and similar.

    +

    Vāc. Is a Vedic goddess of speech. It also hints at being a vaccine.

    +

    Protocol stack​

    +

    What does this stack look like? We take inspiration from core internet architecture, existing survey work and other efforts that have been done to decompose the problem into orthogonal pieces. Each layer provides their own set of properties and only interact with the layers it is adjacent to. Note that this is a rough sketch.

    +
    Layer / ProtocolPurposeExamples
    Application layerEnd user semantics1:1 chat, group chat
    Data SyncData consistencyMVDS, BSP
    Secure TransportConfidentiality, PFS, etcDouble Ratchet, MLS
    Transport PrivacyTransport and metadata protectionWhisper, Tor, Mixnet
    P2P OverlayOverlay routing, NAT traversaldevp2p, libp2p
    Trust EstablishmentEstablishing end-to-end trustTOFU, web of trust
    +

    As an example, end user semantics such as group chat or moderation capabilities can largely work regardless of specific choices further down the stack. Similarly, using a mesh network or Tor doesn't impact the use of Double Ratchet at the Secure Transport layer.

    +

    Data Sync plays a similar role to what TCP does at the transport layer in a traditional Internet architecture, and for some applications something more like UDP is likely to be desirable.

    +

    In terms of specific properties and trade-offs at each layer, we'll go deeper down into them as we study them. For now, this is best treated as a rough sketch or mental map.

    +

    Problems and rough priorities​

    +

    With all the pieces involved, this is quite an undertaking. Luckily, a lot of pieces are already in place and can be either incorporated as-is or iterated on. In terms of medium and long term, here's a rough sketch of priorities and open problems.

    +
      +
    1. Better data sync. While the current MVDS works, it is lacking in a few areas:
      2. +
    +
      +
    • Lack of remote log for mostly-offline offline devices
      • +
    • Better scalability for multi-user chat contexts
      • +
    • Better usability in terms of application layer usage and supporting more transports
      • +
    +
      +
    1. Better transport layer support. Currently MVDS runs primarily over Whisper, which has a few issues:
      2. +
    +
      +
    • scalability, being able to run with many nodes
      • +
    • spam-resistance, proof of work is a poor mechanism for heterogeneous devices
      • +
    • no incentivized infrastructure, leading to centralized choke points
      • +
    +

    In addition to these most immediate concerns, there are other open problems. Some of these are overlapping with the above.

    +
      +
    1. +

      Adaptive nodes. Better support for resource restricted devices and nodes of varying capabilities. Light connection strategy for resources and guarantees. Security games to outsource processing with guarantees.

      +
      2. +
    2. +

      Incentivized and spam-resistant messaging. Reasons to run infrastructure and not relying on altruistic nodes. For spam resistance, in p2p multicast spam is a big attack vector due to amplification. There are a few interesting directions here, such as EigenTrust, proof of burn with micropayments, and leveraging zero-knowledge proofs.

      +
      3. +
    3. +

      Strong privacy guarantees at transport privacy layer. More rigorous privacy guarantees and explicit trade-offs for metadata protection. Includes Mixnet.

      +
      4. +
    4. +

      Censorship-resistant and robust P2P overlay. NAT traversal; running in the browser; mesh networks; pluggable transports for traffic obfuscation.

      +
      5. +
    5. +

      Scalable and decentralized secure conversational security. Strong security guarantees such as forward secrecy, post compromise security, for large group chats. Includes projects such MLS and extending Double Ratchet.

      +
      6. +
    6. +

      Better trust establishment and key handling. Avoiding MITM attacks while still enabling a good user experience. Protecting against ghost users in group chat and providing better ways to do key handling.

      +
      7. +
    +

    There is also a set of more general problems, that touch multiple layers:

    +
      +
    1. +

      Ensuring modularity and interoperability. Providing interfaces that allow for existing and new protocols to be at each layer of the stack.

      +
      2. +
    2. +

      Better specifications. Machine-readable and formally verified specifications. More rigorous analysis of exact guarantees and behaviors. Exposing work in such a way that it can be analyzed by academics.

      +
      3. +
    3. +

      Better simulations. Providing infrastructure and tooling to be able to test protocols in adverse environments and at scale.

      +
      4. +
    4. +

      Enabling excellent user experience. A big reason for the lack of widespread adoption of secure messaging is the fact that more centralized, insecure methods provide a better user experience. Given that incentives can align better for users interested in secure messaging, providing an even better user experience should be doable.

      +
      5. +
    +
    +

    We got some work to do. Come help us if you want. See you in the next update!

    \ No newline at end of file diff --git a/rlog/vac101-fiat-shamir/index.html b/rlog/vac101-fiat-shamir/index.html index 618c0d0c..27d60121 100644 --- a/rlog/vac101-fiat-shamir/index.html +++ b/rlog/vac101-fiat-shamir/index.html @@ -1,32 +1,35 @@ - + - -Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument | Vac Research + +Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument | Vac Research - - - - + + + + -
    -

    Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument

    by
    11 min read

    In this post, we introduce a common technique used to convert interactive protocols to their noninteractive variant.

    Introduction​

    The set of interactive protocols form a class of protocols that consist of communication between two parties: the Prover and the Verifier. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument

    by
    11 min read

    In this post, we introduce a common technique used to convert interactive protocols to their noninteractive variant.

    + +

    Introduction​

    +

    The set of interactive protocols form a class of protocols that consist of communication between two parties: the Prover and the Verifier. The Prover tries to convince the Verifier of a given claim. For example, the Prover may want to convince the Verifier that she owns a specific Unspent Transaction Output (UTXO); that is, the Prover possesses the ability to spend the UTXO. In many instances, there is information that the Prover does not wish to reveal to the Verifier. In our example, it is critical that the Prover does not provide the Verifier with the spending key associated with her UTXO. In addition to the Prover's claim and secret data, there is additional data, public parameters, that the claimed statement is expressed in terms of. -The public parameters can be thought of as the basis for all similar claims.

    In an interactive protocol, the Prover and the Verifier are in active communication. +The public parameters can be thought of as the basis for all similar claims.

    +

    In an interactive protocol, the Prover and the Verifier are in active communication. Specifically, the Prover and the Verifier exchange messages so that the Verifier can validate the Prover's claim. However, this communication is not practical for many applications. It is necessary that any party can verify the Prover's claim in decentralized systems. @@ -35,85 +38,209 @@ Instead, it is desirable for the Prover to generate a proof on their own that ca To achieve this, it is necessary for the Prover to generate the Verifier's messages in such a way that the Prover cannot manipulate the Verifier's messages for her benefit. The Fiat-Shamir heuristic 1 is used for this purpose. -Even though much of our discussion will focus on Σ\Sigma-protocols, -the Fiat-Shamir heuristic is not limited to Σ\Sigma-protocols. +Even though much of our discussion will focus on Σ\Sigma-protocols, +the Fiat-Shamir heuristic is not limited to Σ\Sigma-protocols. The Fiat-Shamir heuristic has been applied to zk-SNARKs, but the security in this setting has been the subject of discussion and research in recent years. -Block et al. 2 provide the first formal analysis of Fiat-Shamir heuristic in zk-SNARKs.

    Sigma Protocols​

    A Σ\Sigma-protocol is a family of interactive protocols that consists of three publicly transmitted messages between the Prover and the Verifier. -In particular, the protocol has the following framework:

    ProverVerifier
    ⟶commitment\stackrel{\mathsf{commitment}}{\longrightarrow}
    ⟵challenge\stackrel{\mathsf{challenge}}{\longleftarrow}
    ⟶response\stackrel{\mathsf{response}}{\longrightarrow}

    These three messages form the protocol's transcript: (commitment,challenge,response)(\mathsf{commitment}, \mathsf{challenge}, \mathsf{response}). +Block et al. 2 provide the first formal analysis of Fiat-Shamir heuristic in zk-SNARKs.

    +

    Sigma Protocols​

    +

    A Σ\Sigma-protocol is a family of interactive protocols that consists of three publicly transmitted messages between the Prover and the Verifier. +In particular, the protocol has the following framework:

    +
    ProverVerifier
    ⟶commitment\stackrel{\mathsf{commitment}}{\longrightarrow}
    ⟵challenge\stackrel{\mathsf{challenge}}{\longleftarrow}
    ⟶response\stackrel{\mathsf{response}}{\longrightarrow}
    +

    These three messages form the protocol's transcript: (commitment,challenge,response)(\mathsf{commitment}, \mathsf{challenge}, \mathsf{response}). The Verifier uses all three of these messages to validate the Prover's original claim. The Verifier's challenge should be selected uniform random from all possible challenges. -Based on this selection, a dishonest Prover can only convince the Verifier with a negligible probability.

    The Schnorr Protocol​

    The Schnorr protocol 3 is usually the first Σ\Sigma-protocol that one studies. +Based on this selection, a dishonest Prover can only convince the Verifier with a negligible probability.

    +

    The Schnorr Protocol​

    +

    The Schnorr protocol 3 is usually the first Σ\Sigma-protocol that one studies. Additionally, the Schnorr protocol can be used as an efficient signature scheme. -The Schnorr protocol provides a framework that enables the Prover to convince the Verifier that: for group elements gg and XX, -the Prover knows the power to raise gg to obtain XX. -Specifically, the Prover possesses some integer xx so that X=gxX = g^x. +The Schnorr protocol provides a framework that enables the Prover to convince the Verifier that: for group elements gg and XX, +the Prover knows the power to raise gg to obtain XX. +Specifically, the Prover possesses some integer xx so that X=gxX = g^x. Cryptographic resources may use either multiplicative or additive notation for groups; we will use multiplicative notation. -Briefly, the element gg being combined with itself in multiplicative notation is g⋅g=g2g \cdot g = g^2, -while in additive notation it is g+g=2gg + g = 2g. -We assume that our group is of prime order pp, and is sufficiently large to satisfy the discrete logarithm assumption.

    The Schnorr protocol proceeds as follows:

    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t⟶T\stackrel{T}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TXcg^z \stackrel{?}{=} T X^c

    Chaum-Pedersen protocol​

    A tuple of group elements (U,V,W)(U,V,W) is a DH-triple if and only if there exists some x∈Zpx \in \mathbb{Z}_p so that V=gxV = g^x and W=UxW = U^x. -The Chaum-Pedersen protocol provides a framework that enables a Prover to convince a Verifier that she possesses such a xx for a claimed DH-triple (U,V,W)(U,V,W). -The Chaum-Pedersen protocol proceeds as follows:

    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t, S:=UtS := U^t⟶T,S\stackrel{T,S}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TVcg^z \stackrel{?}{=} T V^c and Uz=?SWcU^z \stackrel{?}{=} SW^c

    Hash Functions​

    Cryptographic hash functions serve as the backbone to the Fiat-Shamir heuristic. -A hash function, Hash\mathsf{Hash}, is a special function that takes in an arbitrary binary string and outputs a binary string of a predetermined fixed length. +Briefly, the element gg being combined with itself in multiplicative notation is g⋅g=g2g \cdot g = g^2, +while in additive notation it is g+g=2gg + g = 2g. +We assume that our group is of prime order pp, and is sufficiently large to satisfy the discrete logarithm assumption.

    +

    The Schnorr protocol proceeds as follows:

    +
    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t⟶T\stackrel{T}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TXcg^z \stackrel{?}{=} T X^c
    +

    Chaum-Pedersen protocol​

    +

    A tuple of group elements (U,V,W)(U,V,W) is a DH-triple if and only if there exists some x∈Zpx \in \mathbb{Z}_p so that V=gxV = g^x and W=UxW = U^x. +The Chaum-Pedersen protocol provides a framework that enables a Prover to convince a Verifier that she possesses such a xx for a claimed DH-triple (U,V,W)(U,V,W). +The Chaum-Pedersen protocol proceeds as follows:

    +
    ProverVerifier
    t∈RZpt \in_R \mathbb{Z}_p, T:=gtT := g^t, S:=UtS := U^t⟶T,S\stackrel{T,S}{\longrightarrow}
    ⟵c\stackrel{c}{\longleftarrow}c∈RZpc \in_R \mathbb{Z}_p
    z:=t+xcz := t + xc⟶z\stackrel{z}{\longrightarrow}
    output 1 provided gz=?TVcg^z \stackrel{?}{=} T V^c and Uz=?SWcU^z \stackrel{?}{=} SW^c
    +

    Hash Functions​

    +

    Cryptographic hash functions serve as the backbone to the Fiat-Shamir heuristic. +A hash function, Hash\mathsf{Hash}, is a special function that takes in an arbitrary binary string and outputs a binary string of a predetermined fixed length. Specifically, -Hash:{0,1}∗→{0,1}n\mathsf{Hash} : \{0,1\}^* \rightarrow \{0,1\}^n.

    The security of cryptographic hash functions will rely on certain tasks being computationally infeasible. -A task is computationally infeasible provided that there is no deterministic algorithm that can conclude the task in polynomial-time.

    A cryptographic hash function satisfies the following properties:

    • Succinct: The hash function should be easy to compute; the hash Hash(b)\mathsf{Hash}({\bf{b}}) can be efficiently computed for any binary string b{\bf{b}}.
    • Preimage Resistance: It should be computationally infeasible to work backwards given the output of a hash function. Let y{\bf{y}} be a binary string of length nn. -It should be 'impossible' to find some binary string x{\bf{x}} so that y=Hash(x){\bf{y}} = \mathsf{Hash}({\bf{x}}).
    • Collision Resistance: It should be difficult to find two strings that hash to the same value. -It should be computationally infeasible to find two binary strings x1{\bf{x}_1} and x2{\bf{x}_2} so that Hash(x1)=Hash(x2).\mathsf{Hash}({\bf{x}_1}) = \mathsf{Hash}({\bf{x}_2}).

    A related class of functions is one-way functions. +Hash:{0,1}∗→{0,1}n\mathsf{Hash} : \{0,1\}^* \rightarrow \{0,1\}^n.

    +

    The security of cryptographic hash functions will rely on certain tasks being computationally infeasible. +A task is computationally infeasible provided that there is no deterministic algorithm that can conclude the task in polynomial-time.

    +

    A cryptographic hash function satisfies the following properties:

    +
      +
    • Succinct: The hash function should be easy to compute; the hash Hash(b)\mathsf{Hash}({\bf{b}}) can be efficiently computed for any binary string b{\bf{b}}.
      • +
    • Preimage Resistance: It should be computationally infeasible to work backwards given the output of a hash function. Let y{\bf{y}} be a binary string of length nn. +It should be 'impossible' to find some binary string x{\bf{x}} so that y=Hash(x){\bf{y}} = \mathsf{Hash}({\bf{x}}).
      • +
    • Collision Resistance: It should be difficult to find two strings that hash to the same value. +It should be computationally infeasible to find two binary strings x1{\bf{x}_1} and x2{\bf{x}_2} so that Hash(x1)=Hash(x2).\mathsf{Hash}({\bf{x}_1}) = \mathsf{Hash}({\bf{x}_2}).
      • +
    +

    A related class of functions is one-way functions. A one-way function satisfies the first two conditions of a cryptographic hash function (succinct and preimage resistance). All cryptographic hash functions are a one-way functions. However, one-way functions do not necessarily satisfy collision-resistance. We will simply refer to cryptographic hash functions as hash functions for the rest of this blog. Commonly used hash functions include SHA-256 5, -Keccak 6, and Poseidon 7.

    The Fiat-Shamir heuristic​

    The Fiat-Shamir heuristic is the technique used to convert an interactive protocol to a noninteractive protocol. +Keccak 6, and Poseidon 7.

    +

    The Fiat-Shamir heuristic​

    +

    The Fiat-Shamir heuristic is the technique used to convert an interactive protocol to a noninteractive protocol. This is done by replacing each of the Verifier's messages with a hashed value. -Specifically, the Prover generates the Verifier's message by evaluating the hash function Hash\mathsf{Hash} +Specifically, the Prover generates the Verifier's message by evaluating the hash function Hash\mathsf{Hash} with the concatenation of all public values that appear in the protocol thus far. -If m0,…,mtm_0, \dots, m_t denote the public values in the protocol thus far, -then the Verifier's message is computed as mt+1:=Hash(m0∣∣⋯∣∣mt)m_{t+1} := \mathsf{Hash}(m_0|| \cdots || m_t).

    Since Hash\mathsf{Hash} can be efficiently computed, and the messages m0,…,mtm_0, \dots, m_t are public, then any verifying party can compute mt+1m_{t+1}. -Critically, since Hash\mathsf{Hash} is preimage resistant and collision resistant, -the Prover cannot manipulate her choices of the messages m0,…,mtm_0,\dots, m_t to influence the message mt+1m_{t+1}. -Hence, verifying parties can trust that mt+1m_{t+1} is sufficiently random with respect to the preceding messages.

    There are two variants of the Fiat-Shamir heuristic: weak and strong. +If m0,…,mtm_0, \dots, m_t denote the public values in the protocol thus far, +then the Verifier's message is computed as mt+1:=Hash(m0∣∣⋯∣∣mt)m_{t+1} := \mathsf{Hash}(m_0|| \cdots || m_t).

    +

    Since Hash\mathsf{Hash} can be efficiently computed, and the messages m0,…,mtm_0, \dots, m_t are public, then any verifying party can compute mt+1m_{t+1}. +Critically, since Hash\mathsf{Hash} is preimage resistant and collision resistant, +the Prover cannot manipulate her choices of the messages m0,…,mtm_0,\dots, m_t to influence the message mt+1m_{t+1}. +Hence, verifying parties can trust that mt+1m_{t+1} is sufficiently random with respect to the preceding messages.

    +

    There are two variants of the Fiat-Shamir heuristic: weak and strong. The weak variant uses all of the publicly traded messages in computing the Verifier's messages but does not include the public parameters. However, in the strong variant all of the publicly traded messages and public parameters are used to compute the Verifier's messages. -We will provide a discussion on issues that can arise from using the weak Fiat-Shamir heuristic.

    Schnorr Protocol with the strong Fiat-Shamir​

    When the strong Fiat-Shamir heuristic is applied to the Schnorr protocol, the message c=Hash(g∣∣X∣∣T)c = \mathsf{Hash}(g||X||T). -This choice of cc provides security since it should be computationally infeasible to find collisions for the outputs of Hash\mathsf{Hash}. -Thus, cc fixes the group elements gg, XX and TT.

    The elements that would be omitted in the hash by applying weak Fiat-Shamir heuristic are gg and XX.

    Chaum-Pedersen Protocol with the strong Fiat-Shamir​

    The message c=Hash(g∣∣U∣∣V∣∣W∣∣T∣∣S)c = Hash(g||U||V||W||T||S) when the Prover applies the strong Fiat-Shamir heuristic to the Chaum-Pedersen protocol. -The properties of Hash\mathsf{Hash} fixes the generator gg and the Prover's statement (U,V,W)(U,V,W).

    Improper use of the Fiat-Shamir heuristic​

    The Fiat-Shamir heuristic appears to be a fairly straightforward technique to implement. +We will provide a discussion on issues that can arise from using the weak Fiat-Shamir heuristic.

    +

    Schnorr Protocol with the strong Fiat-Shamir​

    +

    When the strong Fiat-Shamir heuristic is applied to the Schnorr protocol, the message c=Hash(g∣∣X∣∣T)c = \mathsf{Hash}(g||X||T). +This choice of cc provides security since it should be computationally infeasible to find collisions for the outputs of Hash\mathsf{Hash}. +Thus, cc fixes the group elements gg, XX and TT.

    +

    The elements that would be omitted in the hash by applying weak Fiat-Shamir heuristic are gg and XX.

    +

    Chaum-Pedersen Protocol with the strong Fiat-Shamir​

    +

    The message c=Hash(g∣∣U∣∣V∣∣W∣∣T∣∣S)c = Hash(g||U||V||W||T||S) when the Prover applies the strong Fiat-Shamir heuristic to the Chaum-Pedersen protocol. +The properties of Hash\mathsf{Hash} fixes the generator gg and the Prover's statement (U,V,W)(U,V,W).

    +

    Improper use of the Fiat-Shamir heuristic​

    +

    The Fiat-Shamir heuristic appears to be a fairly straightforward technique to implement. However, a subtle but serious issue that can occur in the application of the Fiat-Shamir heuristic has been a point of discussion for the past few years. The issue concerns what messages are included in the hash. -In particular, are the public parameters used to compute the hash value?

    Bernhard et al. 8 provide a discussion of Fiat-Shamir heuristic restricted to Σ\Sigma-protocols. +In particular, are the public parameters used to compute the hash value?

    +

    Bernhard et al. 8 provide a discussion of Fiat-Shamir heuristic restricted to Σ\Sigma-protocols. In particular, Bernhard et al. discuss the pitfalls of the weak Fiat-Shamir heuristic. Recall that the strong Fiat-Shamir heuristic requires that the public parameters are included in the calculations of the Verifier's messages while the weak version does not. The inclusion of the public parameters in the hash evaluations fixes these public values for the entire protocol. -This means that the Prover cannot retroactively change them.

    The issues with the differences in the variants of the Fiat-Shamir heuristics has persisted since Bernhard et al.'s paper. +This means that the Prover cannot retroactively change them.

    +

    The issues with the differences in the variants of the Fiat-Shamir heuristics has persisted since Bernhard et al.'s paper. In recent years, auditors from Trail of Bits and OpenZeppelin have released blogs (9, 10, 11, 12, 13) and papers (14, 15) -describing specific vulnerabilities in zero-knowledge papers and repositories associated with the use of the weak Fiat-Shamir heuristic.

    Trail of Bits coined the term FROZEN Heart to describe the use of weak Fiat-Shamir heuristic. +describing specific vulnerabilities in zero-knowledge papers and repositories associated with the use of the weak Fiat-Shamir heuristic.

    +

    Trail of Bits coined the term FROZEN Heart to describe the use of weak Fiat-Shamir heuristic. Frozen comes from the phrase "FoRging Of ZEro kNowledge proofs", -and Fiat-Shamir is the "heart" of transforming an interactive protocol to noninteractive protocol.

    Now, we examine how weak Fiat-Shamir affects the Schnorr protocol and Chaum-Pedersen protocol.

    Schnorr protocol with the weak Fiat-Shamir heuristic​

    For Schnorr, we will examine two variants: -the first where we only include the Prover's claim XX but not the public parameter gg, and -the second where we include the public parameter gg but not the Prover's claim XX.

    Since we omit the generator g∈Gg \in \mathbb{G} from the computation for the message cc in our first approach, -then c=Hash(X∣∣T)c = \mathsf{Hash}(X||T).

    Now, a malicious Prover can complete the transcript for the Schnorr protocol by selecting any z∈RZpz \in_R \mathbb{Z}_p. -Since, gg is not fixed as it was not included in the computation of cc. -But, the malicious Prover needs the transcript (T,c,z)(T,c,z) to satisfy gz=TXcg^z = TX^c. -Hence, the malicious Prover can compute the generator g=(TXc)z−1.g = (TX^c)^{z^{-1}}.

    In our second approach, we omit the group element X∈GX \in \mathbb{G} from the computation for the challenge cc. -That is, c=Hash(g∣∣T)c = \mathsf{Hash}(g||T).

    As with the previous example, the malicious Prover takes a Schnorr transcript (T,c,z)(T,c,z) where z∈RZpz \in_R \mathbb{Z}_p. -It is necessary for the malicious Prover to find a value XX so that gz=TXcg^z = TX^c. -This can be acheived by computing X=(gzT−1)c−1X = (g^z T^{-1})^{c^{-1}}.

    Chaum-Pedersen protocol with the Fiat-Shamir heuristic​

    The Verifier's message c=Hash(T,S)c = Hash(T,S) when weak Fiat-Shamir heuristic is applied. -The Prover's triple (U,V,W)(U,V,W) and the generator gg are not fixed by cc. -As such, a malicious Prover can generate values for U,V,WU,V,W, and gg that satisfy the Verifier's identity checks. -In the case of a malicious Prover, TT and SS are randomly group elements instead of being computed using a value tt that the Prover selected. -This means a malicious Prover must randomly select the value zz as well.

    Given the values that have been fixed so far, each of the Verifier's identities consists of two unknowns. +and Fiat-Shamir is the "heart" of transforming an interactive protocol to noninteractive protocol.

    +

    Now, we examine how weak Fiat-Shamir affects the Schnorr protocol and Chaum-Pedersen protocol.

    +

    Schnorr protocol with the weak Fiat-Shamir heuristic​

    +

    For Schnorr, we will examine two variants: +the first where we only include the Prover's claim XX but not the public parameter gg, and +the second where we include the public parameter gg but not the Prover's claim XX.

    +

    Since we omit the generator g∈Gg \in \mathbb{G} from the computation for the message cc in our first approach, +then c=Hash(X∣∣T)c = \mathsf{Hash}(X||T).

    +

    Now, a malicious Prover can complete the transcript for the Schnorr protocol by selecting any z∈RZpz \in_R \mathbb{Z}_p. +Since, gg is not fixed as it was not included in the computation of cc. +But, the malicious Prover needs the transcript (T,c,z)(T,c,z) to satisfy gz=TXcg^z = TX^c. +Hence, the malicious Prover can compute the generator g=(TXc)z−1.g = (TX^c)^{z^{-1}}.

    +

    In our second approach, we omit the group element X∈GX \in \mathbb{G} from the computation for the challenge cc. +That is, c=Hash(g∣∣T)c = \mathsf{Hash}(g||T).

    +

    As with the previous example, the malicious Prover takes a Schnorr transcript (T,c,z)(T,c,z) where z∈RZpz \in_R \mathbb{Z}_p. +It is necessary for the malicious Prover to find a value XX so that gz=TXcg^z = TX^c. +This can be acheived by computing X=(gzT−1)c−1X = (g^z T^{-1})^{c^{-1}}.

    +

    Chaum-Pedersen protocol with the Fiat-Shamir heuristic​

    +

    The Verifier's message c=Hash(T,S)c = Hash(T,S) when weak Fiat-Shamir heuristic is applied. +The Prover's triple (U,V,W)(U,V,W) and the generator gg are not fixed by cc. +As such, a malicious Prover can generate values for U,V,WU,V,W, and gg that satisfy the Verifier's identity checks. +In the case of a malicious Prover, TT and SS are randomly group elements instead of being computed using a value tt that the Prover selected. +This means a malicious Prover must randomly select the value zz as well.

    +

    Given the values that have been fixed so far, each of the Verifier's identities consists of two unknowns. Hence, it is necessary to select one of these unknowns from each identity so that a malicious Prover can compute the last value. -For instances, suppose that the malicious Prover randomly selects VV and WW. -The malicious Prover can compute g=(TVc)1/zg = (T V^c)^{1/z} and V=(SWc)1/zV = (SW^c)^{1/z}. -Thus, the malicious Prover has a claimed statement (U,V,W)(U,V,W) for generator gg that passes the Verifier's identities using weak Fiat-Shamir heuristic.

    The omission of any of the values U,V,W,U,V,W, and gg in the computation of cc allows a malicious Prover to forge a proof.

    Conclusion​

    The Fiat-Shamir heuristic is an essential technique to convert an interactive protocol to a variant that does not require communication. -Additionally, careful application of this technique is necessary to maintain the integrity of the system.

    References​

    - - +For instances, suppose that the malicious Prover randomly selects VV and WW. +The malicious Prover can compute g=(TVc)1/zg = (T V^c)^{1/z} and V=(SWc)1/zV = (SW^c)^{1/z}. +Thus, the malicious Prover has a claimed statement (U,V,W)(U,V,W) for generator gg that passes the Verifier's identities using weak Fiat-Shamir heuristic.

    +

    The omission of any of the values U,V,W,U,V,W, and gg in the computation of cc allows a malicious Prover to forge a proof.

    +

    Conclusion​

    +

    The Fiat-Shamir heuristic is an essential technique to convert an interactive protocol to a variant that does not require communication. +Additionally, careful application of this technique is necessary to maintain the integrity of the system.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters/index.html b/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters/index.html index b79570b4..adc07a6f 100644 --- a/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters/index.html +++ b/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters/index.html @@ -1,102 +1,210 @@ - + - -Vac 101: Membership with Bloom Filters and Cuckoo Filters | Vac Research + +Vac 101: Membership with Bloom Filters and Cuckoo Filters | Vac Research - - - - + + + + -
    -

    Vac 101: Membership with Bloom Filters and Cuckoo Filters

    by
    13 min read

    We examine two data structures: Bloom filters and Cuckoo filters.

    Membership with Bloom Filters and Cuckoo Filters​

    The ability to efficiently query the membership of an element in a given data set is crucial. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Vac 101: Membership with Bloom Filters and Cuckoo Filters

    by
    13 min read

    We examine two data structures: Bloom filters and Cuckoo filters.

    + +

    Membership with Bloom Filters and Cuckoo Filters​

    +

    The ability to efficiently query the membership of an element in a given data set is crucial. In certain applications, it is more important to output a result quickly than to have a 'perfect' result. In particular, false positives may be an acceptable tradeoff for speed. In this blog, we examine Bloom and Cuckoo data filters. -Both of these filters are data structures that can be used for membership proofs.

    Everyone is familiar with the process of creating a new account for various websites, whether it is an e-mail account or a social media account. +Both of these filters are data structures that can be used for membership proofs.

    +

    Everyone is familiar with the process of creating a new account for various websites, whether it is an e-mail account or a social media account. Consider when you enter your desired username. Many sites provide real-time feedback, as you type, on the availability of a given string. In this scenario, it is necessary that the result is seemingly instant, regardless of the number of existing accounts. However, it is not important that the usernames that are flagged as unavailable are, in fact, in use. -That is, it is sufficient to have a probabilistic check for membership.

    Bloom filters and Cuckoo filters are data structures that can be used to accumulate data with a fixed amount of space. -The associated filter FF for a digest of data DD can be queried to determine whether an element is (possibly) a member of DD:

    • 0: The queried element is definitely not a member of digest DD.
    • 1: The entry is possibly a member of the digest DD.

    The algorithms associated with Bloom filters and Cuckoo filters, which we will discuss shortly, are deterministic. -The possibility of false positives arises from the query algorithm.

    Bloom filters​

    A Bloom filter is a data structure that can be used to accumulate an arbitrary amount of data with a fixed amount of space. +That is, it is sufficient to have a probabilistic check for membership.

    +

    Bloom filters and Cuckoo filters are data structures that can be used to accumulate data with a fixed amount of space. +The associated filter FF for a digest of data DD can be queried to determine whether an element is (possibly) a member of DD:

    +
      +
    • 0: The queried element is definitely not a member of digest DD.
      • +
    • 1: The entry is possibly a member of the digest DD.
      • +
    +

    The algorithms associated with Bloom filters and Cuckoo filters, which we will discuss shortly, are deterministic. +The possibility of false positives arises from the query algorithm.

    +

    Bloom filters​

    +

    A Bloom filter is a data structure that can be used to accumulate an arbitrary amount of data with a fixed amount of space. Bloom filters have been a popular data structure for proof of non-membership due to their small storage size. -Specifically, a Bloom filter consists of a binary string v∈{0,1}n{\bf{v}} \in \{0,1\}^n and kk hash functions {hi:{0,1}∗→{0,…,n−1}}i=0k−1\{h_i: \{0,1\}^* \rightarrow \{0,\dots,n-1\}\}_{i=0}^{k-1}. -We note that each hash function hih_i is used to determine an index of our binary string v{\bf{v}} to flip the associated bit to 1. -The binary string v{\bf{v}} is initialized with every entry as 0. -The hash functions do not need to be cryptographic hash functions.

    • Append: Suppose that we wish to add the element xx to the Bloom filter.
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=hi(x){\bf{b}}[i] := h_i(x) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
      • Update the binary string v[b[i]]←1{\bf{v}}[{\bf{b}}[i]] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
    • Query: Suppose that we wish to query the Bloom filter for element yy.
      • Return 1 provided v[hi(y)]=1{\bf{v}}[h_i(y)] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.

    The algorithm Query will output 1 for every element yy that has been added to the Bloom filter. +Specifically, a Bloom filter consists of a binary string v∈{0,1}n{\bf{v}} \in \{0,1\}^n and kk hash functions {hi:{0,1}∗→{0,…,n−1}}i=0k−1\{h_i: \{0,1\}^* \rightarrow \{0,\dots,n-1\}\}_{i=0}^{k-1}. +We note that each hash function hih_i is used to determine an index of our binary string v{\bf{v}} to flip the associated bit to 1. +The binary string v{\bf{v}} is initialized with every entry as 0. +The hash functions do not need to be cryptographic hash functions.

    +
      +
    • +

      Append: Suppose that we wish to add the element xx to the Bloom filter.

      +
        +
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=hi(x){\bf{b}}[i] := h_i(x) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      • Update the binary string v[b[i]]←1{\bf{v}}[{\bf{b}}[i]] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Bloom filter for element yy.

      +
        +
      • Return 1 provided v[hi(y)]=1{\bf{v}}[h_i(y)] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.
        • +
      +
      • +
    +

    The algorithm Query will output 1 for every element yy that has been added to the Bloom filter. This is a consequence of the Append algorithm. However, due to potential collisions over a set of hash functions, it is possible for false positives to occur. -Moreover, the possibility of collisions makes it impossible to remove elements from the Bloom filter.

    Complexity​

    The storage of a Bloom filter requires constant space. -Specifically, the Bloom filter uses nn bits regardless of the size of the digest. -So, regardless of the number of elements that we append, the Bloom filter will use nn bits. -Further, if we assume that each of the kk hash functions runs in constant time, then we can append/query an entry in O(k)O(k).

    Example​

    Suppose that k=3k = 3 and n=10n = 10. -Our Bloom filter is initialized as v=(0000000000).\bf{v} = \begin{pmatrix}0&0&0&0&0&0&0&0&0&0\end{pmatrix}. -Now, we will append the words addadd, sumsum, and equalequal. -Suppose that

    h0(add)=1h1(add)=4h2(add)=7h0(sum)=9h1(sum)=2h2(sum)=1h0(equal)=5h1(equal)=8h2(equal)=0.\begin{matrix} h_0(add) = 1 & h_1(add) = 4 & h_2(add) = 7\\ h_0(sum) = 9 & h_1(sum) = 2 & h_2(sum) = 1\\ h_0(equal) = 5 & h_1(equal) = 8 & h_2(equal) = 0. \end{matrix}

    After appending these words, the Bloom filter is v=(1110110111).\bf{v} = \begin{pmatrix}1&1&1&0&1&1&0&1&1&1\end{pmatrix}.

    Now, suppose that we query the words subtractsubtract and multiplemultiple so that

    h0(subtract)=3h1(subtract)=5h2(subtract)=1h0(multiple)=7h1(multiple)=1h2(multiple)=4\begin{matrix} h_0(subtract) = 3 & h_1(subtract) = 5 & h_2(subtract) = 1\\ h_0(multiple) = 7 & h_1(multiple) = 1 & h_2(multiple) = 4\\ \end{matrix}.

    The query for subtractsubtract returns 0 since v[3]=0{\bf{v}}[3]=0. -On the other hand, the query for multiplemultiple returns 1 since v[1]=1,v[4]=1{\bf{v}}[1]=1, {\bf{v}}[4] = 1, and v[7]=1{\bf{v}}[7]=1. -Even though multiplemultiple was not used to generate the Bloom filter v{\bf{v}}, our query returns the false positive.

    Probability of false positives​

    For our analysis, we will assume that the probabilities that arise in our analysis are independent. -However, this assumption can be removed to gain the same approximation.

    We note that for a single hash function, the probability that a specific bit is flipped to 1 is 1/n1/n. -So, the probability that the specific bit is not flipped by the hash function is 1−1/n1-1/n. -Applying our assumption that the kk hash functions are 'independent,' +Moreover, the possibility of collisions makes it impossible to remove elements from the Bloom filter.

    +

    Complexity​

    +

    The storage of a Bloom filter requires constant space. +Specifically, the Bloom filter uses nn bits regardless of the size of the digest. +So, regardless of the number of elements that we append, the Bloom filter will use nn bits. +Further, if we assume that each of the kk hash functions runs in constant time, then we can append/query an entry in O(k)O(k).

    +

    Example​

    +

    Suppose that k=3k = 3 and n=10n = 10. +Our Bloom filter is initialized as v=(0000000000).\bf{v} = \begin{pmatrix}0&0&0&0&0&0&0&0&0&0\end{pmatrix}. +Now, we will append the words addadd, sumsum, and equalequal. +Suppose that

    +

    h0(add)=1h1(add)=4h2(add)=7h0(sum)=9h1(sum)=2h2(sum)=1h0(equal)=5h1(equal)=8h2(equal)=0.\begin{matrix} +h_0(add) = 1 & h_1(add) = 4 & h_2(add) = 7\\ +h_0(sum) = 9 & h_1(sum) = 2 & h_2(sum) = 1\\ +h_0(equal) = 5 & h_1(equal) = 8 & h_2(equal) = 0. +\end{matrix}

    +

    After appending these words, the Bloom filter is v=(1110110111).\bf{v} = \begin{pmatrix}1&1&1&0&1&1&0&1&1&1\end{pmatrix}.

    +

    Now, suppose that we query the words subtractsubtract and multiplemultiple so that

    +

    h0(subtract)=3h1(subtract)=5h2(subtract)=1h0(multiple)=7h1(multiple)=1h2(multiple)=4\begin{matrix} h_0(subtract) = 3 & h_1(subtract) = 5 & h_2(subtract) = 1\\ h_0(multiple) = 7 & h_1(multiple) = 1 & h_2(multiple) = 4\\ +\end{matrix}.

    +

    The query for subtractsubtract returns 0 since v[3]=0{\bf{v}}[3]=0. +On the other hand, the query for multiplemultiple returns 1 since v[1]=1,v[4]=1{\bf{v}}[1]=1, {\bf{v}}[4] = 1, and v[7]=1{\bf{v}}[7]=1. +Even though multiplemultiple was not used to generate the Bloom filter v{\bf{v}}, our query returns the false positive.

    +

    Probability of false positives​

    +

    For our analysis, we will assume that the probabilities that arise in our analysis are independent. +However, this assumption can be removed to gain the same approximation.

    +

    We note that for a single hash function, the probability that a specific bit is flipped to 1 is 1/n1/n. +So, the probability that the specific bit is not flipped by the hash function is 1−1/n1-1/n. +Applying our assumption that the kk hash functions are 'independent,' the probability that the specific bit is not flipped by any of the hash functions is -(1−1/n)k(1-1/n)^k.

    Recall the calculus fact lim⁡∞(1−1/n)n=e−1\lim_{\infty} (1-1/n)^n = e^{-1}. -That is, as we increase the number of bits that our Bloom filter uses, the approximate probability that a given bit is not flipped by any of the kk hash functions is e−k/ne^{-k/n}.

    Suppose that ℓ\ell entries have been added to the Bloom filter. -The probability that a specific bit is still 0 after the ℓ\ell entries have been added is approximately e−ℓk/ne^{-\ell k/n}. +(1−1/n)k(1-1/n)^k.

    +

    Recall the calculus fact lim⁡∞(1−1/n)n=e−1\lim_{\infty} (1-1/n)^n = e^{-1}. +That is, as we increase the number of bits that our Bloom filter uses, the approximate probability that a given bit is not flipped by any of the kk hash functions is e−k/ne^{-k/n}.

    +

    Suppose that ℓ\ell entries have been added to the Bloom filter. +The probability that a specific bit is still 0 after the ℓ\ell entries have been added is approximately e−ℓk/ne^{-\ell k/n}. The probability that a queried element is erroneously claimed as a member of the digest is approximately -(1−e−ℓk/n)k(1-e^{-\ell k/n})^k.

    The following table provides concrete values for these approximations.

    nnkkℓ\ell(1−e−ℓk/n)k(1-e^{-\ell k/n})^k
    32330.01474
    32370.11143
    323120.30802
    323170.50595
    323280.79804

    Notice that the probability of false positives increases as the number of elements (ℓ\ell) that have been added to the digest increases.

    Sliding-Window Bloom filter​

    Our toy example and table illustrated an issue concerning Bloom filters. -The number of entries that can be added to a Bloom filter is restricted by our choice of kk and nn. +(1−e−ℓk/n)k(1-e^{-\ell k/n})^k.

    +

    The following table provides concrete values for these approximations.

    +
    nnkkℓ\ell(1−e−ℓk/n)k(1-e^{-\ell k/n})^k
    32330.01474
    32370.11143
    323120.30802
    323170.50595
    323280.79804
    +

    Notice that the probability of false positives increases as the number of elements (ℓ\ell) that have been added to the digest increases.

    +

    Sliding-Window Bloom filter​

    +

    Our toy example and table illustrated an issue concerning Bloom filters. +The number of entries that can be added to a Bloom filter is restricted by our choice of kk and nn. Not only does the probability that false positives will occur increase, -but it is possible that our vector v{\bf{v}} can be a string of all 1s. -Szepieniec and Værge proposed a modification to Bloom filters to handle this.

    Instead of having a fixed number of bits for our Bloom filter, we dynamically allot memory based on the number of entries that have been added to the filter. -Given a predetermined threshold (bb) for the number of entries, we shift our 'window' of flipping bits by ss bits. +but it is possible that our vector v{\bf{v}} can be a string of all 1s. +Szepieniec and Værge proposed a modification to Bloom filters to handle this.

    +

    Instead of having a fixed number of bits for our Bloom filter, we dynamically allot memory based on the number of entries that have been added to the filter. +Given a predetermined threshold (bb) for the number of entries, we shift our 'window' of flipping bits by ss bits. Note that this means that it is necessary to keep track of when a given entry is added to the digest. -This means that querying the Sliding-Window Bloom filter will yield different results when different timestamps are used.

    This can be done with kk hash functions as we used earlier. -Alternatively, Szepieniec and Værge proposed using the same hash function but to produce kk entries in the current window. -Specifically, we obtain the bits we wish to flip to 1s by computing h(X∣∣i)h(X || i) for each i∈{0,…,k−1}i \in \{0,\dots, k-1\} and XX as we will define next. -For Sliding-Window Bloom filters, XX is more than just the element we wish to append to the filter. -Instead, XX consists of the element xx and a timestamp tt. -The timestamp tt is used to locate the correct window for bits, as we see below:

    • Append: Suppose that we wish to add the element xx with timestamp tt to the Sliding-Window Bloom filter.
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=h(x∣∣t∣∣i){\bf{b}}[i] := h(x||t||i) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
      • Update the binary string v[b[i]+⌊t/b⌋s]←1{\bf{v}}[{\bf{b}}[i]+\lfloor t/b \rfloor s] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
    • Query: Suppose that we wish to query the Bloom filter for element yy with timestamp tt.
      • Return 1 provided v[h(y∣∣t∣∣i)+⌊t/b⌋s]=1{\bf{v}}[h(y||t||i) + \lfloor t/b \rfloor s] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.

    By incorporating a shifting window, we maintain efficient querying and appending at the cost of constant space. -However, by losing constant space, we gain 'infinite' scalability.

    Cuckoo filters​

    A Cuckoo filter is a data structure for probabilistic membership proofs based on Cuckoo hash tables. +This means that querying the Sliding-Window Bloom filter will yield different results when different timestamps are used.

    +

    This can be done with kk hash functions as we used earlier. +Alternatively, Szepieniec and Værge proposed using the same hash function but to produce kk entries in the current window. +Specifically, we obtain the bits we wish to flip to 1s by computing h(X∣∣i)h(X || i) for each i∈{0,…,k−1}i \in \{0,\dots, k-1\} and XX as we will define next. +For Sliding-Window Bloom filters, XX is more than just the element we wish to append to the filter. +Instead, XX consists of the element xx and a timestamp tt. +The timestamp tt is used to locate the correct window for bits, as we see below:

    +
      +
    • +

      Append: Suppose that we wish to add the element xx with timestamp tt to the Sliding-Window Bloom filter.

      +
        +
      • Define the vector b∈{0,…,n−1}k{\bf{b}} \in \{0,\dots,n-1\}^k so that b[i]:=h(x∣∣t∣∣i){\bf{b}}[i] := h(x||t||i) for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      • Update the binary string v[b[i]+⌊t/b⌋s]←1{\bf{v}}[{\bf{b}}[i]+\lfloor t/b \rfloor s] \leftarrow 1 for each i∈{0,…,k−1}i \in \{0,\dots,k-1\}.
        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Bloom filter for element yy with timestamp tt.

      +
        +
      • Return 1 provided v[h(y∣∣t∣∣i)+⌊t/b⌋s]=1{\bf{v}}[h(y||t||i) + \lfloor t/b \rfloor s] = 1 for every i∈{0,…,k−1}i \in \{0,\dots,k-1\}. Otherwise, return 0.
        • +
      +
      • +
    +

    By incorporating a shifting window, we maintain efficient querying and appending at the cost of constant space. +However, by losing constant space, we gain 'infinite' scalability.

    +

    Cuckoo filters​

    +

    A Cuckoo filter is a data structure for probabilistic membership proofs based on Cuckoo hash tables. The specific design goal for Cuckoo filters is to address the inability to remove elements from a Bloom Filter. This is done by replacing a list of bits with a list of 'fingerprints.' A fingerprint can be thought of as the hash value for an entry in the digest. A Cuckoo filter is a fixed-length list of 'fingerprints.' -If the maximum number of entries that a Cuckoo filter can hold is nn and a fingerprint occupies ff bits, -then the Cuckoo filter occupies nfnf bits.

    Now, we describe the algorithms associated with the Cuckoo filter CC with hash function hash(X)hash(X) and fingerprint function fingerprint(X)fingerprint(X).

    • Append: Suppose that we wish to add the element xx to the Cuckoo filter.

      • If either position ix:=hash(x)i_x := hash(x) or jx:=i⊗hash(fingerprint(x))j_x := i \otimes hash(fingerprint(x)) of CC is empty, -then fingerprint(x)fingerprint(x) is inserted into an empty position.
      • If both ixi_x and jxj_x are occupied with a fingerprint that is distinct from fingerprint(x)fingerprint(x), -then we select either ixi_x or jxj_x to insert fingerprint(x)fingerprint(x). +If the maximum number of entries that a Cuckoo filter can hold is nn and a fingerprint occupies ff bits, +then the Cuckoo filter occupies nfnf bits.

        +

        Now, we describe the algorithms associated with the Cuckoo filter CC with hash function hash(X)hash(X) and fingerprint function fingerprint(X)fingerprint(X).

        +
          +
        • +

          Append: Suppose that we wish to add the element xx to the Cuckoo filter.

          +
            +
          • If either position ix:=hash(x)i_x := hash(x) or jx:=i⊗hash(fingerprint(x))j_x := i \otimes hash(fingerprint(x)) of CC is empty, +then fingerprint(x)fingerprint(x) is inserted into an empty position.
            • +
          • If both ixi_x and jxj_x are occupied with a fingerprint that is distinct from fingerprint(x)fingerprint(x), +then we select either ixi_x or jxj_x to insert fingerprint(x)fingerprint(x). The fingerprint that had previously occupied this position cannot be discarried. Instead, we insert this fingerprint into its alternate location. This reshuffling process either ends with fingerprints all having their own bucket or one that cannot be inserted. -In the case that we have a fingerprint that cannot be inserted, then the Cuckoo filter is overfilled.

        • Query: Suppose that we wish to query the Cuckoo filter for element yy.

          • Return 1 provided fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y.

        • Delete: Suppose that we wish to delete the element yy from the Cuckoo filter.

          • If yy has been added to the Cuckoo filter, then fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y. -We remove fingerprint(y)fingerprint(y) from the appropriate position.

        We note that false positives in Cuckoo filters only occur when an element shares a fingerprint and hash with a value that has already been added to the Cuckoo filter.

        Example​

        In this example, we will append the words addadd, sumsum, and equalequal to a Cuckoo filter with 8 slots.

        For each word xx, we compute two indices: -ix:=hash(x) and jx:=hash(x)⊗hash(fingerprint(x)).i_x := hash(x) \text{ and } j_x := hash(x) \otimes hash(fingerprint(x)). +In the case that we have a fingerprint that cannot be inserted, then the Cuckoo filter is overfilled.

        • +
      +
      • +
    • +

      Query: Suppose that we wish to query the Cuckoo filter for element yy.

      +
        +
      • Return 1 provided fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y.
        • +
      +
      • +
    • +

      Delete: Suppose that we wish to delete the element yy from the Cuckoo filter.

      +
        +
      • If yy has been added to the Cuckoo filter, then fingerprint(y)fingerprint(y) is either in position iyi_y or jyj_y. +We remove fingerprint(y)fingerprint(y) from the appropriate position.
        • +
      +
      • +
    +

    We note that false positives in Cuckoo filters only occur when an element shares a fingerprint and hash with a value that has already been added to the Cuckoo filter.

    +

    Example​

    +

    In this example, we will append the words addadd, sumsum, and equalequal to a Cuckoo filter with 8 slots.

    +

    For each word xx, we compute two indices: +ix:=hash(x) and jx:=hash(x)⊗hash(fingerprint(x)).i_x := hash(x) \text{ and } j_x := hash(x) \otimes hash(fingerprint(x)). Suppose that we have the following values for -our words:

    wordixi_xjxj_x
    addadd(0,1,0)(0,1,0)(1,0,0)(1,0,0)
    sumsum(1,0,1)(1,0,1)(1,1,0)(1,1,0)
    equalequal(0,1,0)(0,1,0)(1,0,1)(1,0,1)

    For clarity of the example, we append the words directly to the buckets instead of fingerprints of our data.

    01234567
    append addaddaddadd
    append sumsumaddaddsumsum

    Notice that both of the buckets (2 and 5) that equalequal can map to are occupied. -So, we select one of these buckets (say 2) to insert equalequal into. -Then, we have to insert addadd to its possible bucket (1). -This leaves us with the Cuckoo filter:

    01234567
    addaddequalequalsumsum

    Complexity​

    Notice that deletions and queries to Cuckoo filters are done in constant time. -Specifically, only two locations need to be checked for any data xx. +our words:

    +
    wordixi_xjxj_x
    addadd(0,1,0)(0,1,0)(1,0,0)(1,0,0)
    sumsum(1,0,1)(1,0,1)(1,1,0)(1,1,0)
    equalequal(0,1,0)(0,1,0)(1,0,1)(1,0,1)
    +

    For clarity of the example, we append the words directly to the buckets instead of fingerprints of our data.

    +
    01234567
    append addaddaddadd
    append sumsumaddaddsumsum
    +

    Notice that both of the buckets (2 and 5) that equalequal can map to are occupied. +So, we select one of these buckets (say 2) to insert equalequal into. +Then, we have to insert addadd to its possible bucket (1). +This leaves us with the Cuckoo filter:

    +
    01234567
    addaddequalequalsumsum
    +

    Complexity​

    +

    Notice that deletions and queries to Cuckoo filters are done in constant time. +Specifically, only two locations need to be checked for any data xx. Appends may require shuffling previously added elements to their alternate locations. -As such, the append does not run in constant time.

    Bloom filters vs Cuckoo filters​

    The design of Bloom filters is focused on space efficiency and quick query time. +As such, the append does not run in constant time.

    +

    Bloom filters vs Cuckoo filters​

    +

    The design of Bloom filters is focused on space efficiency and quick query time. Even though they occupy constant space, -Cuckoo filters require significantly more space for nn items than Bloom filters. +Cuckoo filters require significantly more space for nn items than Bloom filters. The worst-case append in a Cuckoo filter is slower than the append in a Bloom filter. However, an append that does not require any shuffling in a Cuckoo filter can be quicker than appends in Bloom filters. Cuckoo filters make up for these disadvantages with quicker query time and the ability to delete entries. -Further, the probability of false positives in Cuckoo filters is lower than the probability of false positives in Bloom filters.

    Combining Filters with RLN​

    In a series of posts (1,2,3), +Further, the probability of false positives in Cuckoo filters is lower than the probability of false positives in Bloom filters.

    +

    Combining Filters with RLN​

    +

    In a series of posts (1,2,3), various versons of rate limiting nullifiers (RLN) that are used by Waku has been discussed. RLN uses a sparse Merkle tree for the membership set. The computational power required to construct the Merkle tree prevent light clients from participating in verifying membership proofs. @@ -105,18 +213,30 @@ it was proposed to move the membership set on-chain so that it would not be nece Unfortunately, the naive approach is not practical as the gas limit for a single call is too restrictive for an appropriately sized tree. Instead, it was proposed to make utilize of subtrees. In this section, we provide a discussion of an alternate solution for light clients by using filters for the membership set. -The two parts of RLN that we will focus on are user registration and deletion.

    Both Bloom and Cuckoo filters support user registration as this is can be done as an append. +The two parts of RLN that we will focus on are user registration and deletion.

    +

    Both Bloom and Cuckoo filters support user registration as this is can be done as an append. The fixed size of these filters would restrict the total number of users that can register. This can be migitated by using Sliding-Window Bloom filter as this supports system growth. The Sliding-Window can be adapted to Cuckoo filters as well. In the case of a Sliding-Window filter, an user would maintain the epoch of when they registered. The registration of new users to Bloom filters can be done in constant time which is a significant improvement over appending to subtrees. -Unfortunately, the complexity of registration to Cuckoo filters cannot be as easily computed.

    A user could be slashed from the RLN by sending too many messages in a given epoch. +Unfortunately, the complexity of registration to Cuckoo filters cannot be as easily computed.

    +

    A user could be slashed from the RLN by sending too many messages in a given epoch. Unfortunately, Bloom filters do not support the deletion of members. -Luckily, Cuckoo filters allow for deletions that can performed in constant time.

    Cuckoo filter that use Sliding-Window could be used so that light clients are able to verify proofs of membership in the RLN. +Luckily, Cuckoo filters allow for deletions that can performed in constant time.

    +

    Cuckoo filter that use Sliding-Window could be used so that light clients are able to verify proofs of membership in the RLN. These proofs are not a substitute to the usual proofs that a heavy client can verify due to the allowance of false positives. -However, with the allowance of false positives, a light client can participate in verification RLN proofs in an efficient manner.

    References​

    - - +However, with the allowance of false positives, a light client can participate in verification RLN proofs in an efficient manner.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/waku-for-all/index.html b/rlog/waku-for-all/index.html index 8e350da4..43848e2e 100644 --- a/rlog/waku-for-all/index.html +++ b/rlog/waku-for-all/index.html @@ -1,72 +1,167 @@ - + - -Waku for All Decentralized Applications and Infrastructures | Vac Research + +Waku for All Decentralized Applications and Infrastructures | Vac Research - - - + + + -
    -

    Waku for All Decentralized Applications and Infrastructures

    by
    7 min read

    Waku is an open communication protocol and network. Decentralized apps and infrastructure can use Waku for their + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Waku for All Decentralized Applications and Infrastructures

    by
    7 min read

    Waku is an open communication protocol and network. Decentralized apps and infrastructure can use Waku for their communication needs. It is designed to enable dApps and decentralized infrastructure projects to have secure, private, scalable communication. Waku is available in several languages and platforms, from Web to mobile to desktop to cloud. Initially, We pushed Waku adoption to the Web ecosystem, we learned that Waku is usable in a variety of complex applications -and infrastructure projects. We have prioritized our effort to make Waku usable on various platforms and environments.

    Background​

    We have built Waku to be the communication layer for Web3. Waku is a collection of protocols to chose from for your +and infrastructure projects. We have prioritized our effort to make Waku usable on various platforms and environments.

    + +

    Background​

    +

    We have built Waku to be the communication layer for Web3. Waku is a collection of protocols to chose from for your messaging needs. It enables secure, censorship-resistant, privacy-preserving, spam-protected communication for its user. -It is designed to run on any device, from mobile to the cloud.

    Waku is available on many systems and environments and used by several applications and SDKs for decentralized communications.

    This involved research efforts in various domains: conversational security, protocol incentivization, zero-knowledge, -etc.

    Waku uses novel technologies. Hence, we knew that early dogfooding of Waku was necessary. Even if research +It is designed to run on any device, from mobile to the cloud.

    +

    Waku is available on many systems and environments and used by several applications and SDKs for decentralized communications.

    +

    This involved research efforts in various domains: conversational security, protocol incentivization, zero-knowledge, +etc.

    +

    Waku uses novel technologies. Hence, we knew that early dogfooding of Waku was necessary. Even if research was still in progress [1]. Thus, as soon as Waku protocols and software were usable, we started to push -for the adoption of Waku. This started back in 2021.

    Waku is the communication component of the Web3 trifecta. This trifecta was Ethereum (contracts), Swarm +for the adoption of Waku. This started back in 2021.

    +

    Waku is the communication component of the Web3 trifecta. This trifecta was Ethereum (contracts), Swarm (storage) and Whisper (communication). Hence, it made sense to first target dApps which already uses one of the pillars: -Ethereum.

    As most dApps are web apps, we started the development of js-waku for the browser.

    Once ready, we reached out to dApps to integrate Waku, added prizes to hackathons -and gave talks.

    We also assumed we would see patterns in the usage of Waku, that we would facilitate with the help of -SDKs.

    Finally, we created several web apps: +Ethereum.

    +

    As most dApps are web apps, we started the development of js-waku for the browser.

    +

    Once ready, we reached out to dApps to integrate Waku, added prizes to hackathons +and gave talks.

    +

    We also assumed we would see patterns in the usage of Waku, that we would facilitate with the help of +SDKs.

    +

    Finally, we created several web apps: examples -and PoCs.

    By discussing with Waku users and watching it being used, we learned a few facts:

    1. The potential use cases for Waku are varied and many:
    • Wallet <> dApp communication: WalletConnect, XMTP
    • Off-chain (and private) marketplace: +and PoCs.

      +

      By discussing with Waku users and watching it being used, we learned a few facts:

      +
        +
      1. The potential use cases for Waku are varied and many:
        2. +
      +
      1. Many projects are interested in having an embedded chat in their dApp,
      2. There are complex applications that need Waku as a solution. Taking RAILGUN as an example:
      • Web wallet
      • + React Native mobile wallet
      • + NodeJS node/backend.

      (1) means that it is not that easy to create SDKs for common use cases.

      (2) was a clear candidate for an SDK. Yet, building a chat app is a complex task. Hence, the Status app team tackled -this in the form of Status Web.

      Finally, (3) was the most important lesson. We learned that multi-tier applications need Waku for decentralized and +Decentralized Uber

      • +
    • Signature exchange for a multi-sign wallet: Gnosis Safe x Waku
      • +
    • Off-chain Game moves/actions: Super Card Game (EthOnline 2021)
      • +
    • Decentralized Pastebin: Debin
      • +
    +
      +
    1. Many projects are interested in having an embedded chat in their dApp,
      2. +
    2. There are complex applications that need Waku as a solution. Taking RAILGUN as an example:
      3. +
    +
      +
    • Web wallet
      • +
    • + React Native mobile wallet
      • +
    • + NodeJS node/backend.
      • +
    +

    (1) means that it is not that easy to create SDKs for common use cases.

    +

    (2) was a clear candidate for an SDK. Yet, building a chat app is a complex task. Hence, the Status app team tackled +this in the form of Status Web.

    +

    Finally, (3) was the most important lesson. We learned that multi-tier applications need Waku for decentralized and censorship-resistant communications. For these projects, js-waku is simply not enough. They need Waku to work in their -Golang backend, Unity desktop game and React Native mobile app.

    We understood that we should see the whole Waku software suite +Golang backend, Unity desktop game and React Native mobile app.

    +

    We understood that we should see the whole Waku software suite (js-waku, nwaku, go-waku, waku-react-native, etc) as an asset for its success. -That we should not limit outreach, marketing, documentation efforts to the web, but target all platforms.

    From a market perspective, we identified several actors:

    • platforms: Projects that uses Waku to handle communication,
    • operators: Operators run Waku nodes and are incentivized to do so,
    • developers: Developers are usually part of a platforms or solo hackers learning Web3,
    • contributors: Developers and researchers with interests in decentralization, privacy, censorship-resistance, -zero-knowledge, etc.

    Waku for All Decentralized Applications and Infrastructures​

    In 2022, we shifted our focus to make the various Waku implementations usable and used.

    We made Waku multi-plaform.

    We shifted Waku positioning to leverage all Waku implementations and better serve the user's needs:

    We are consolidating the documentation for all implementations on a single website (work in progress) -to improve developer experience.

    This year, we also started the operator outreach effort to push for users to run their own Waku nodes. We have +That we should not limit outreach, marketing, documentation efforts to the web, but target all platforms.

    +

    From a market perspective, we identified several actors:

    +
      +
    • platforms: Projects that uses Waku to handle communication,
      • +
    • operators: Operators run Waku nodes and are incentivized to do so,
      • +
    • developers: Developers are usually part of a platforms or solo hackers learning Web3,
      • +
    • contributors: Developers and researchers with interests in decentralization, privacy, censorship-resistance, +zero-knowledge, etc.
      • +
    +

    Waku for All Decentralized Applications and Infrastructures​

    +

    In 2022, we shifted our focus to make the various Waku implementations usable and used.

    +

    We made Waku multi-plaform.

    +

    We shifted Waku positioning to leverage all Waku implementations and better serve the user's needs:

    + +

    We are consolidating the documentation for all implementations on a single website (work in progress) +to improve developer experience.

    +

    This year, we also started the operator outreach effort to push for users to run their own Waku nodes. We have recently concluded our first operator trial run. Nwaku's documentation, stability and performance has improved. It is now easier to -run your own Waku node.

    Today, operator wannabes most likely run their own nodes to support or use the Waku network. +run your own Waku node.

    +

    Today, operator wannabes most likely run their own nodes to support or use the Waku network. We are dogfooding Waku RLN, our novel economic spam protection protocol, and looking at incentivizing the Waku Store protocol. -This way, we are adding reasons to run your own Waku node.

    For those who were following us in 2021, know that we are retiring the Waku Connect branding in favour of the Waku -branding.

    Waku for Your Project​

    As discussed, Waku is now available on various platforms. The question remains: How can Waku benefit your project?

    Here are a couple of use cases we recently investigated:

    Layer-2 Decentralization​

    Most ([2] [3] roll-ups use a centralized sequencer or equivalent. Running several sequencers is not as straightforward as running several execution nodes. -Waku can help:

    • Provide a neutral marketplace for a mempool: If sequencers compete for L2 tx fees, they may not be incentivized to +This way, we are adding reasons to run your own Waku node.

      +

      For those who were following us in 2021, know that we are retiring the Waku Connect branding in favour of the Waku +branding.

      +

      Waku for Your Project​

      +

      As discussed, Waku is now available on various platforms. The question remains: How can Waku benefit your project?

      +

      Here are a couple of use cases we recently investigated:

      +

      Layer-2 Decentralization​

      +

      Most ([2] [3] roll-ups use a centralized sequencer or equivalent. Running several sequencers is not as straightforward as running several execution nodes. +Waku can help:

      +
        +
      • Provide a neutral marketplace for a mempool: If sequencers compete for L2 tx fees, they may not be incentivized to share transactions with other sequencers. Waku nodes can act as a neutral network to enable all sequences to access -transactions.
      • Enable censorship-resistant wallet<>L2 communication,
      • Provide rate limiting mechanism for spam protection: Using RLN to prevent DDOS.

      Device pairing and communication​

      With Waku Device Pairing, a user can setup a secure encrypted communication channel +transactions.

      • +
    • Enable censorship-resistant wallet<>L2 communication,
      • +
    • Provide rate limiting mechanism for spam protection: Using RLN to prevent DDOS.
      • +
    +

    Device pairing and communication​

    +

    With Waku Device Pairing, a user can setup a secure encrypted communication channel between their devices. As this channel would operate over Waku, it would be censorship-resistant and privacy preserving. -These two devices could be:

    • Ethereum node and mobile phone to access a remote admin panel,
    • Alice's phone and Bob's phone for any kind of secure communication,
    • Mobile wallet and desktop/browser dApp for transaction and signature exchange.

    Check js-waku#950 for the latest update on this.

    Get Involved​

    Developer? Grab any of the Waku implementations and integrate it in your app: https://waku.org/platform.

    Researcher? See https://vac.dev/contribute to participate in Waku research.

    Tech-savvy? Try to run your own node: https://waku.org/operator.

    Otherwise, play around with the various web examples.

    If you want to help, we are hiring!

    Moving Forward​

    What you can expect next:

    • Scalability and performance studies and improvement across Waku software,
    • New websites to easily find documentation about Waku and its implementations,
    • New Waku protocols implemented in all code bases and cross client PoCs +These two devices could be:

      +
        +
      • Ethereum node and mobile phone to access a remote admin panel,
        • +
      • Alice's phone and Bob's phone for any kind of secure communication,
        • +
      • Mobile wallet and desktop/browser dApp for transaction and signature exchange.
        • +
      +

      Check js-waku#950 for the latest update on this.

      +

      Get Involved​

      +

      Developer? Grab any of the Waku implementations and integrate it in your app: https://waku.org/platform.

      +

      Researcher? See https://vac.dev/contribute to participate in Waku research.

      +

      Tech-savvy? Try to run your own node: https://waku.org/operator.

      +

      Otherwise, play around with the various web examples.

      +

      If you want to help, we are hiring!

      +

      Moving Forward​

      +

      What you can expect next:

      +

      References​

      • [1] Waku is modular; it is a suite of protocols; hence some Waku protocols may be mature, while +waku-rln-relay, etc),
        • +
      • Easier to run your own waku node, more operator trials,
        • +
      • Dogfooding and Improvement of existing protocols (e.g. Waku Filter),
        • +
      • Continue our focus Waku portability: Browser, +Raspberry Pi Zero and other restricted-resource environments,
        • +
      • More communication & marketing effort around Waku and the Waku developer community.
        • +
      +
      +

      References​

      +
        +
      • [1] Waku is modular; it is a suite of protocols; hence some Waku protocols may be mature, while new protocols are still being designed. Which means that research continues to be ongoing while -Waku is already used in production.
      • [2] The Optimism Foundation runs the only block produce on the Optimism network.
      • [3] Top 10 L2s are documented has having a centralized operator.
    - - +Waku is already used in production.
    • +
  • [2] The Optimism Foundation runs the only block produce on the Optimism network.
    • +
  • [3] Top 10 L2s are documented has having a centralized operator.
    • +
    \ No newline at end of file diff --git a/rlog/waku-update/index.html b/rlog/waku-update/index.html index 21d688a3..7b22e87a 100644 --- a/rlog/waku-update/index.html +++ b/rlog/waku-update/index.html @@ -1,27 +1,107 @@ - + - -Waku Update | Vac Research + +Waku Update | Vac Research - - - + + + -
    -

    Waku Update

    by
    6 min read

    A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?

    Waku is our fork of Whisper where we address the shortcomings of Whisper in an iterative manner. We've seen a in previous post that Whisper doesn't scale, and why. In this post we'll talk about what the current state of Waku is, how many users it can support, and future plans.

    Current state​

    Specs:

    We released Waku spec v0.3 this week! You can see the full changelog here.

    The main change from 0.2 is making the handshake more flexible. This enables us to communicate topic interest immediately without ambiguity. We also did the following:

    • added recommendation for DNS based discovery
    • added an upgradability and compatibility policy
    • cut the spec up into several components

    We cut the spec up in several components to make Vac as modular as possible. The components right now are:

    We can probably factor these out further as the main spec is getting quite big, but this is good enough for now.

    Clients:

    There are currently two clients that implement Waku v0.3, these are Nimbus (Update: now nim-waku) in Nim and status-go in Go.

    For more details on what each client support and don't, you can follow the work in progress checklist.

    Work is currently in progress to integrate it into the Status core app. Waku is expected to be part of their upcoming 1.1 release (see Status app roadmap (link deprecated)).

    Simulation:

    We have a simulation that verifies - or rather, fails to falsify - our scalability model. More on the simulation and what it shows below.

    How many users does Waku support?​

    This is our current understanding of how many users a network running Waku can support. Specifically in the context of the Status chat app, since that's the most immediate consumer of Waku. It should generalize fairly well to most deployments.

    tl;dr (for Status app):

    • beta: 100 DAU
    • v1: 1k DAU
    • v1.1 (waku only): 10k DAU (up to x10 with deployment hotfixes)
    • v1.2 (waku+dns): 100k DAU (can optionally be folded into v1.1)

    Assuming 10 concurrent users = 100 DAU. Estimate uncertainty increases for each order of magnitude until real-world data is observed.

    As far as we know right now, these are the bottlenecks we have:

    • Immediate bottleneck - Receive bandwidth for end user clients (aka ‘Fixing Whisper with Waku’)
    • Very likely bottleneck - Nodes and cluster capacity (aka ‘DNS based node discovery’)
    • Conjecture but not unlikely to appear- Full node traffic (aka ‘the routing / partition problem’)

    We've already seen the first bottleneck being discussed in the initial post. Dean wrote a post on DNS based discovery which explains how we will address the likely second bottleneck. More on the third one in future posts.

    For more details on these bottlenecks, see Scalability estimate: How many users can Waku and the Status app support?.

    Simulation​

    The ultimate test is real-world usage. Until then, we have a simulation thanks to Kim De Mey from the Nimbus team!

    We have two network topologies, Star and full mesh. Both networks have 6 full nodes, one traditional light node with bloom filter, and one Waku light node.

    One of the full nodes sends 1 envelope over 1 of the 100 topics that the two light nodes subscribe to. After that, it sends 10000 envelopes over random topics.

    For light node, bloom filter is set to almost 10% false positive (bloom filter: n=100, k=3, m=512). It shows the number of valid and invalid envelopes received for the different nodes.

    Star network:

    DescriptionPeersValidInvalid
    Master node7100010
    Full node 13100010
    Full node 21100010
    Full node 31100010
    Full node 41100010
    Full node 51100010
    Light node28150
    Waku light node210

    Full mesh:

    DescriptionPeersValidInvalid
    Full node 071000120676
    Full node 17100019554
    Full node 251000123304
    Full node 351000111983
    Full node 451000124425
    Full node 551000123472
    Light node2803803
    Waku light node211

    Things to note:

    • Whisper light node with ~10% false positive gets ~10% of total traffic
    • Waku light node gets ~1000x less envelopes than Whisper light node
    • Full mesh results in a lot more duplicate messages, expect for Waku light node

    Run the simulation yourself here. The parameters are configurable, and it is integrated with Prometheus and Grafana.

    Difference between Waku and Whisper​

    Summary of main differences between Waku v0 spec and Whisper v6, as described in EIP-627:

    • Handshake/Status message not compatible with shh/6 nodes; specifying options as association list
    • Include topic-interest in Status handshake
    • Upgradability policy
    • topic-interest packet code
    • RLPx subprotocol is changed from shh/6 to waku/0.
    • Light node capability is added.
    • Optional rate limiting is added.
    • Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits
    • Mail Server and Mail Client functionality is now part of the specification.
    • P2P Message packet contains a list of envelopes instead of a single envelope.

    Next steps and future plans​

    Several challenges remain to make Waku a robust and suitable base -communication protocol. Here we outline a few challenges that we are addressing and will continue to work on:

    • scalability of the network
    • incentived infrastructure and spam-resistance
    • build with resource restricted devices in mind, including nodes being mostly offline

    For the third bottleneck, a likely candidate for fixing this is Kademlia routing. This is similar to what is done in Swarm's PSS. We are in the early stages of experimenting with this over libp2p in nim-libp2p. More on this in a future post!

    Acknowledgements​

    Image from "caged sky" by mh.xbhd.org is licensed under CC BY 2.0 (https://ccsearch.creativecommons.org/photos/a9168311-78de-4cb7-a6ad-f92be8361d0e)

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Waku Update

    by
    6 min read

    A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?

    + +

    Waku is our fork of Whisper where we address the shortcomings of Whisper in an iterative manner. We've seen a in previous post that Whisper doesn't scale, and why. In this post we'll talk about what the current state of Waku is, how many users it can support, and future plans.

    +

    Current state​

    +

    Specs:

    +

    We released Waku spec v0.3 this week! You can see the full changelog here.

    +

    The main change from 0.2 is making the handshake more flexible. This enables us to communicate topic interest immediately without ambiguity. We also did the following:

    +
      +
    • added recommendation for DNS based discovery
      • +
    • added an upgradability and compatibility policy
      • +
    • cut the spec up into several components
      • +
    +

    We cut the spec up in several components to make Vac as modular as possible. The components right now are:

    + +

    We can probably factor these out further as the main spec is getting quite big, but this is good enough for now.

    +

    Clients:

    +

    There are currently two clients that implement Waku v0.3, these are Nimbus (Update: now nim-waku) in Nim and status-go in Go.

    +

    For more details on what each client support and don't, you can follow the work in progress checklist.

    +

    Work is currently in progress to integrate it into the Status core app. Waku is expected to be part of their upcoming 1.1 release (see Status app roadmap (link deprecated)).

    +

    Simulation:

    +

    We have a simulation that verifies - or rather, fails to falsify - our scalability model. More on the simulation and what it shows below.

    +

    How many users does Waku support?​

    +

    This is our current understanding of how many users a network running Waku can support. Specifically in the context of the Status chat app, since that's the most immediate consumer of Waku. It should generalize fairly well to most deployments.

    +

    tl;dr (for Status app):

    +
      +
    • beta: 100 DAU
      • +
    • v1: 1k DAU
      • +
    • v1.1 (waku only): 10k DAU (up to x10 with deployment hotfixes)
      • +
    • v1.2 (waku+dns): 100k DAU (can optionally be folded into v1.1)
      • +
    +

    Assuming 10 concurrent users = 100 DAU. Estimate uncertainty increases for each order of magnitude until real-world data is observed.

    +

    As far as we know right now, these are the bottlenecks we have:

    +
      +
    • Immediate bottleneck - Receive bandwidth for end user clients (aka ‘Fixing Whisper with Waku’)
      • +
    • Very likely bottleneck - Nodes and cluster capacity (aka ‘DNS based node discovery’)
      • +
    • Conjecture but not unlikely to appear- Full node traffic (aka ‘the routing / partition problem’)
      • +
    +

    We've already seen the first bottleneck being discussed in the initial post. Dean wrote a post on DNS based discovery which explains how we will address the likely second bottleneck. More on the third one in future posts.

    +

    For more details on these bottlenecks, see Scalability estimate: How many users can Waku and the Status app support?.

    +

    Simulation​

    +

    The ultimate test is real-world usage. Until then, we have a simulation thanks to Kim De Mey from the Nimbus team!

    +

    +

    We have two network topologies, Star and full mesh. Both networks have 6 full nodes, one traditional light node with bloom filter, and one Waku light node.

    +

    One of the full nodes sends 1 envelope over 1 of the 100 topics that the two light nodes subscribe to. After that, it sends 10000 envelopes over random topics.

    +

    For light node, bloom filter is set to almost 10% false positive (bloom filter: n=100, k=3, m=512). It shows the number of valid and invalid envelopes received for the different nodes.

    +

    Star network:

    +
    DescriptionPeersValidInvalid
    Master node7100010
    Full node 13100010
    Full node 21100010
    Full node 31100010
    Full node 41100010
    Full node 51100010
    Light node28150
    Waku light node210
    +

    Full mesh:

    +
    DescriptionPeersValidInvalid
    Full node 071000120676
    Full node 17100019554
    Full node 251000123304
    Full node 351000111983
    Full node 451000124425
    Full node 551000123472
    Light node2803803
    Waku light node211
    +

    Things to note:

    +
      +
    • Whisper light node with ~10% false positive gets ~10% of total traffic
      • +
    • Waku light node gets ~1000x less envelopes than Whisper light node
      • +
    • Full mesh results in a lot more duplicate messages, expect for Waku light node
      • +
    +

    Run the simulation yourself here. The parameters are configurable, and it is integrated with Prometheus and Grafana.

    +

    Difference between Waku and Whisper​

    +

    Summary of main differences between Waku v0 spec and Whisper v6, as described in EIP-627:

    +
      +
    • Handshake/Status message not compatible with shh/6 nodes; specifying options as association list
      • +
    • Include topic-interest in Status handshake
      • +
    • Upgradability policy
      • +
    • topic-interest packet code
      • +
    • RLPx subprotocol is changed from shh/6 to waku/0.
      • +
    • Light node capability is added.
      • +
    • Optional rate limiting is added.
      • +
    • Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits
      • +
    • Mail Server and Mail Client functionality is now part of the specification.
      • +
    • P2P Message packet contains a list of envelopes instead of a single envelope.
      • +
    +

    Next steps and future plans​

    +

    Several challenges remain to make Waku a robust and suitable base +communication protocol. Here we outline a few challenges that we are addressing and will continue to work on:

    +
      +
    • scalability of the network
      • +
    • incentived infrastructure and spam-resistance
      • +
    • build with resource restricted devices in mind, including nodes being mostly offline
      • +
    +

    For the third bottleneck, a likely candidate for fixing this is Kademlia routing. This is similar to what is done in Swarm's PSS. We are in the early stages of experimenting with this over libp2p in nim-libp2p. More on this in a future post!

    +

    Acknowledgements​

    +

    Image from "caged sky" by mh.xbhd.org is licensed under CC BY 2.0 (https://ccsearch.creativecommons.org/photos/a9168311-78de-4cb7-a6ad-f92be8361d0e)

    \ No newline at end of file diff --git a/rlog/waku-v1-v2-bandwidth-comparison/index.html b/rlog/waku-v1-v2-bandwidth-comparison/index.html index 36106c48..5adbab60 100644 --- a/rlog/waku-v1-v2-bandwidth-comparison/index.html +++ b/rlog/waku-v1-v2-bandwidth-comparison/index.html @@ -1,115 +1,187 @@ - + - -Waku v1 vs Waku v2: Bandwidth Comparison | Vac Research + +Waku v1 vs Waku v2: Bandwidth Comparison | Vac Research - - - + + + -
    -

    Waku v1 vs Waku v2: Bandwidth Comparison

    by
    10 min read

    A local comparison of bandwidth profiles showing significantly improved scalability in Waku v2 over Waku v1.

    Background​

    The original plan for Waku v2 suggested theoretical improvements in resource usage over Waku v1, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Waku v1 vs Waku v2: Bandwidth Comparison

    by
    10 min read

    A local comparison of bandwidth profiles showing significantly improved scalability in Waku v2 over Waku v1.

    + +

    Background​

    +

    The original plan for Waku v2 suggested theoretical improvements in resource usage over Waku v1, mainly as a result of the improved amplification factors provided by GossipSub. -In its turn, Waku v1 proposed improvements over its predecessor, Whisper.

    Given that Waku v2 is aimed at resource restricted environments, +In its turn, Waku v1 proposed improvements over its predecessor, Whisper.

    +

    Given that Waku v2 is aimed at resource restricted environments, we are specifically interested in its scalability and resource usage characteristics. However, the theoretical performance improvements of Waku v2 over Waku v1, -has never been properly benchmarked and tested.

    Although we're working towards a full performance evaluation of Waku v2, +has never been properly benchmarked and tested.

    +

    Although we're working towards a full performance evaluation of Waku v2, this would require significant planning and resources, if it were to simulate "real world" conditions faithfully and measure bandwidth and resource usage across different network connections, robustness against attacks/losses, message latencies, etc. (There already exists a fairly comprehensive evaluation of GossipSub v1.1, -on which 11/WAKU2-RELAY is based.)

    As a starting point, +on which 11/WAKU2-RELAY is based.)

    +

    As a starting point, this post contains a limited and local comparison of the bandwidth profile (only) between Waku v1 and Waku v2. It reuses and adapts existing network simulations for Waku v1 and Waku v2 -and compares bandwidth usage for similar message propagation scenarios.

    Theoretical improvements in Waku v2​

    Messages are propagated in Waku v1 using flood routing. +and compares bandwidth usage for similar message propagation scenarios.

    +

    Theoretical improvements in Waku v2​

    +

    Messages are propagated in Waku v1 using flood routing. This means that every peer will forward every new incoming message to all its connected peers (except the one it received the message from). This necessarily leads to unnecessary duplication (termed amplification factor), wasting bandwidth and resources. What's more, we expect this effect to worsen the larger the network becomes, as each connection will receive a copy of each message, -rather than a single copy per peer.

    Message routing in Waku v2 follows the libp2p GossipSub protocol, +rather than a single copy per peer.

    +

    Message routing in Waku v2 follows the libp2p GossipSub protocol, which lowers amplification factors by only sending full message contents to a subset of connected peers. As a Waku v2 network grows, each peer will limit its number of full-message ("mesh") peerings - libp2p suggests a maximum of 12 such connections per peer. This allows much better scalability than a flood-routed network. -From time to time, a Waku v2 peer will send metadata about the messages it has seen to other peers ("gossip" peers).

    See this explainer for a more detailed discussion.

    Methodology​

    The results below contain only some scenarios that provide an interesting contrast between Waku v1 and Waku v2. +From time to time, a Waku v2 peer will send metadata about the messages it has seen to other peers ("gossip" peers).

    +

    See this explainer for a more detailed discussion.

    +

    Methodology​

    +

    The results below contain only some scenarios that provide an interesting contrast between Waku v1 and Waku v2. For example, star network topologies do not show a substantial difference between Waku v1 and Waku v2. This is because each peer relies on a single connection to the central node for every message, which barely requires any routing: each connection receives a copy of every message for both Waku v1 and Waku v2. Hybrid topologies similarly show only a difference between Waku v1 and Waku v2 for network segments with mesh-like connections, -where routing decisions need to be made.

    For this reason, the following approach applies to all iterations:

    1. Simulations are run locally. +where routing decisions need to be made.

      +

      For this reason, the following approach applies to all iterations:

      +
        +
      1. Simulations are run locally. This limits the size of possible scenarios due to local resource constraints, -but is a way to quickly get an approximate comparison.
      2. Nodes are treated as a blackbox for which we only measure bandwidth, +but is a way to quickly get an approximate comparison.
        3. +
      3. Nodes are treated as a blackbox for which we only measure bandwidth, using an external bandwidth monitoring tool. -In other words, we do not consider differences in the size of the envelope (for v1) or the message (for v2).
      4. Messages are published at a rate of 50 new messages per second to each network, -except where explicitly stated otherwise.
      5. Each message propagated in the network carries 8 bytes of random payload, which is encrypted. -The same symmetric key cryptographic algorithm (with the same keys) are used in both Waku v1 and v2.
      6. Traffic in each network is generated from 10 nodes (randomly-selected) and published in a round-robin fashion to 10 topics (content topics for Waku v2). -In practice, we found no significant difference in average bandwidth usage when tweaking these two parameters (the number of traffic generating nodes and the number of topics).
      7. Peers are connected in a decentralized full mesh topology, +In other words, we do not consider differences in the size of the envelope (for v1) or the message (for v2).
        8. +
      8. Messages are published at a rate of 50 new messages per second to each network, +except where explicitly stated otherwise.
        9. +
      9. Each message propagated in the network carries 8 bytes of random payload, which is encrypted. +The same symmetric key cryptographic algorithm (with the same keys) are used in both Waku v1 and v2.
        10. +
      10. Traffic in each network is generated from 10 nodes (randomly-selected) and published in a round-robin fashion to 10 topics (content topics for Waku v2). +In practice, we found no significant difference in average bandwidth usage when tweaking these two parameters (the number of traffic generating nodes and the number of topics).
        11. +
      11. Peers are connected in a decentralized full mesh topology, i.e. each peer is connected to every other peer in the network. Waku v1 is expected to flood all messages across all existing connections. Waku v2 gossipsub will GRAFT some of these connections for full-message peerings, -with the rest being gossip-only peerings.
      12. After running each iteration, we verify that messages propagated to all peers (comparing the number of published messages to the metrics logged by each peer).

      For Waku v1, nodes are configured as "full" nodes (i.e. with full bloom filter), -while Waku v2 nodes are relay nodes, all subscribing and publishing to the same PubSub topic.

      Network size comparison​

      Iteration 1: 10 nodes​

      Let's start with a small network of 10 nodes only and see how Waku v1 bandwidth usage compares to that of Waku v2. +with the rest being gossip-only peerings.

      2. +
    2. After running each iteration, we verify that messages propagated to all peers (comparing the number of published messages to the metrics logged by each peer).
      3. +
    +

    For Waku v1, nodes are configured as "full" nodes (i.e. with full bloom filter), +while Waku v2 nodes are relay nodes, all subscribing and publishing to the same PubSub topic.

    +

    Network size comparison​

    +

    Iteration 1: 10 nodes​

    +

    Let's start with a small network of 10 nodes only and see how Waku v1 bandwidth usage compares to that of Waku v2. At this small scale we don't expect to see improved bandwidth usage in Waku v2 over Waku v1, since all connections, for both Waku v1 and Waku v2, will be full-message connections. The number of connections is low enough that Waku v2 nodes will likely GRAFT all connections to full-message peerings, essentially flooding every message on every connection in a similar fashion to Waku v1. If our expectations are confirmed, it helps validate our methodology, -showing that it gives more or less equivalent results between Waku v1 and Waku v2 networks.

    Sure enough, the figure shows that in this small-scale setup, +showing that it gives more or less equivalent results between Waku v1 and Waku v2 networks.

    +

    +

    Sure enough, the figure shows that in this small-scale setup, Waku v1 actually has a lower per-peer bandwidth usage than Waku v2. One reason for this may be the larger overall proportion of control messages in a gossipsub-routed network such as Waku v2. These play a larger role when the total network traffic is comparatively low, as in this iteration. -Also note that the average bandwidth remains more or less constant as long as the rate of published messages remains stable.

    Iteration 2: 30 nodes​

    Now, let's run the same scenario for a larger network of highly-connected nodes, this time consisting of 30 nodes. +Also note that the average bandwidth remains more or less constant as long as the rate of published messages remains stable.

    +

    Iteration 2: 30 nodes​

    +

    Now, let's run the same scenario for a larger network of highly-connected nodes, this time consisting of 30 nodes. At this point, the Waku v2 nodes will start pruning some connections to limit the number of full-message peerings (to a maximum of 12), while the Waku v1 nodes will continue flooding messages to all connected peers. -We therefore expect to see a somewhat improved bandwidth usage in Waku v2 over Waku v1.

    Bandwidth usage in Waku v2 has increased only slightly from the smaller network of 10 nodes (hovering between 2000 and 3000 kbps). +We therefore expect to see a somewhat improved bandwidth usage in Waku v2 over Waku v1.

    +

    +

    Bandwidth usage in Waku v2 has increased only slightly from the smaller network of 10 nodes (hovering between 2000 and 3000 kbps). This is because there are only a few more full-message peerings than before. -Compare this to the much higher increase in bandwidth usage for Waku v1, which now requires more than 4000 kbps on average.

    Iteration 3: 50 nodes​

    For an even larger network of 50 highly connected nodes, +Compare this to the much higher increase in bandwidth usage for Waku v1, which now requires more than 4000 kbps on average.

    +

    Iteration 3: 50 nodes​

    +

    For an even larger network of 50 highly connected nodes, the divergence between Waku v1 and Waku v2 is even larger. -The following figure shows comparative average bandwidth usage for a throughput of 50 messages per second.

    Average bandwidth usage (for the same message rate) has remained roughly the same for Waku v2 as it was for 30 nodes, -indicating that the number of full-message peerings per node has not increased.

    Iteration 4: 85 nodes​

    We already see a clear trend in the bandwidth comparisons above, +The following figure shows comparative average bandwidth usage for a throughput of 50 messages per second.

    +

    +

    Average bandwidth usage (for the same message rate) has remained roughly the same for Waku v2 as it was for 30 nodes, +indicating that the number of full-message peerings per node has not increased.

    +

    Iteration 4: 85 nodes​

    +

    We already see a clear trend in the bandwidth comparisons above, so let's confirm by running the test once more for a network of 85 nodes. Due to local resource constraints, the effective throughput for Waku v1 falls to below 50 messages per second, so the v1 results below have been normalized and are therefore approximate. -The local Waku v2 simulation maintains the message throughput rate without any problems.

    Iteration 5: 150 nodes​

    Finally, we simulate message propagation in a network of 150 nodes. +The local Waku v2 simulation maintains the message throughput rate without any problems.

    +

    +

    Iteration 5: 150 nodes​

    +

    Finally, we simulate message propagation in a network of 150 nodes. Due to local resource constraints, we run this simulation at a lower rate - 35 messages per second - -and for a shorter amount of time.

    Notice how the Waku v1 bandwidth usage is now more than 10 times worse than that of Waku v2. +and for a shorter amount of time.

    +

    +

    Notice how the Waku v1 bandwidth usage is now more than 10 times worse than that of Waku v2. This is to be expected, as each Waku v1 node will try to flood each new message to 149 other peers, -while the Waku v2 nodes limit their full-message peerings to no more than 12.

    Discussion​

    Let's summarize average bandwidth growth against network growth for a constant message propagation rate. +while the Waku v2 nodes limit their full-message peerings to no more than 12.

    +

    Discussion​

    +

    Let's summarize average bandwidth growth against network growth for a constant message propagation rate. Since we are particularly interested in how Waku v1 compares to Waku v2 in terms of bandwidth usage, -the results are normalised to the Waku v2 average bandwidth usage for each network size.

    Extrapolation is a dangerous game, +the results are normalised to the Waku v2 average bandwidth usage for each network size.

    +

    +

    Extrapolation is a dangerous game, but it's safe to deduce that the divergence will only grow for even larger network topologies. Although control signalling contributes more towards overall bandwidth for Waku v2 networks, this effect becomes less noticeable for larger networks. For network segments with more than ~18 densely connected nodes, -the advantage of using Waku v2 above Waku v1 becomes clear.

    Network traffic comparison​

    The analysis above controls the average message rate while network size grows. +the advantage of using Waku v2 above Waku v1 becomes clear.

    +

    Network traffic comparison​

    +

    The analysis above controls the average message rate while network size grows. In reality, however, active users (and therefore message rates) are likely to grow in conjunction with the network. This will have an effect on bandwidth for both Waku v1 and Waku v2, though not in equal measure. -Consider the impact of an increasing rate of messages in a network of constant size:

    The rate of increase in bandwidth for Waku v2 is slower than that for Waku v1 for a corresponding increase in message propagation rate. +Consider the impact of an increasing rate of messages in a network of constant size:

    +

    +

    The rate of increase in bandwidth for Waku v2 is slower than that for Waku v1 for a corresponding increase in message propagation rate. In fact, for a network of 30 densely-connected nodes, if the message propagation rate increases by 1 per second, Waku v1 requires an increased average bandwidth of almost 70kbps at each node. -A similar traffic increase in Waku v2 requires on average 40kbps more bandwidth per peer, just over half that of Waku v1.

    Conclusions​

    • Waku v2 scales significantly better than Waku v1 in terms of average bandwidth usage, -especially for densely connected networks.
    • E.g. for a network consisting of 150 or more densely connected nodes, -Waku v2 provides more than 10x better average bandwidth usage rates than Waku v1.
    • As the network continues to scale, both in absolute terms (number of nodes) and in network traffic (message rates) the disparity between Waku v2 and Waku v1 becomes even larger.

    Future work​

    Now that we've confirmed that Waku v2's bandwidth improvements over its predecessor matches theory, +A similar traffic increase in Waku v2 requires on average 40kbps more bandwidth per peer, just over half that of Waku v1.

    +

    Conclusions​

    +
      +
    • Waku v2 scales significantly better than Waku v1 in terms of average bandwidth usage, +especially for densely connected networks.
      • +
    • E.g. for a network consisting of 150 or more densely connected nodes, +Waku v2 provides more than 10x better average bandwidth usage rates than Waku v1.
      • +
    • As the network continues to scale, both in absolute terms (number of nodes) and in network traffic (message rates) the disparity between Waku v2 and Waku v1 becomes even larger.
      • +
    +

    Future work​

    +

    Now that we've confirmed that Waku v2's bandwidth improvements over its predecessor matches theory, we can proceed to a more in-depth characterisation of Waku v2's resource usage. -Some questions that we want to answer include:

    • What proportion of Waku v2's bandwidth usage is used to propagate payload versus bandwidth spent on control messaging to maintain the mesh?
    • To what extent is message latency (time until a message is delivered to its destination) affected by network size and message rate?
    • How reliable is message delivery in Waku v2 for different network sizes and message rates?
    • What are the resource usage profiles of other Waku v2 protocols (e.g.12/WAKU2-FILTER and 19/WAKU2-LIGHTPUSH)?

    Our aim is to get ever closer to a "real world" understanding of Waku v2's performance characteristics, +Some questions that we want to answer include:

    +
      +
    • What proportion of Waku v2's bandwidth usage is used to propagate payload versus bandwidth spent on control messaging to maintain the mesh?
      • +
    • To what extent is message latency (time until a message is delivered to its destination) affected by network size and message rate?
      • +
    • How reliable is message delivery in Waku v2 for different network sizes and message rates?
      • +
    • What are the resource usage profiles of other Waku v2 protocols (e.g.12/WAKU2-FILTER and 19/WAKU2-LIGHTPUSH)?
      • +
    +

    Our aim is to get ever closer to a "real world" understanding of Waku v2's performance characteristics, identify and fix vulnerabilities -and continually improve the efficiency of our suite of protocols.

    References​

    - - +and continually improve the efficiency of our suite of protocols.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/waku-v2-ethereum-coscup/index.html b/rlog/waku-v2-ethereum-coscup/index.html index 6743cba1..b291b8ed 100644 --- a/rlog/waku-v2-ethereum-coscup/index.html +++ b/rlog/waku-v2-ethereum-coscup/index.html @@ -1,78 +1,210 @@ - + - -[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging | Vac Research + +[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging | Vac Research - - - + + + -
    -

    [Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging

    by
    8 min read

    Learn more about Waku v2, its origins, goals, protocols, implementation and ongoing research. Understand how it is used and how it can be useful for messaging in Ethereum.

    This is the English version of a talk originally given in Chinese at COSCUP in Taipei.

    video recording with Chinese and English subtitles.

    Introduction​

    Hi everyone!

    Today I'll talk to you about Waku v2. What it is, what problems it is solving, + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    [Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging

    by
    8 min read

    Learn more about Waku v2, its origins, goals, protocols, implementation and ongoing research. Understand how it is used and how it can be useful for messaging in Ethereum.

    + +

    This is the English version of a talk originally given in Chinese at COSCUP in Taipei.

    +

    video recording with Chinese and English subtitles.

    +
    +

    Introduction​

    +

    Hi everyone!

    +

    Today I'll talk to you about Waku v2. What it is, what problems it is solving, and how it can be useful for things such as messaging in Ethereum. First, let me -start with some brief background.

    Brief history and background​

    Back when Ethereum got started, there used to be this concept of the "holy +start with some brief background.

    +

    Brief history and background​

    +

    Back when Ethereum got started, there used to be this concept of the "holy trinity". You had Ethereum for compute/consensus, Swarm for storage, and Whisper -for messaging. This is partly where the term Web3 comes from.

    Status started out as an app with the goal of being a window onto Ethereum and +for messaging. This is partly where the term Web3 comes from.

    +

    Status started out as an app with the goal of being a window onto Ethereum and a secure messenger. As one of the few, if not the only, apps using Whisper in production, not to mention on a mobile phone, we quickly realized there were problems with the underlying protocols and infrastructure. Protocols such as Whisper weren't quite ready for prime time yet when it came to things such as -scalability and working in the real world.

    As we started addressing some of these challenges, and moved from app +scalability and working in the real world.

    +

    As we started addressing some of these challenges, and moved from app developement to focusing on protocols, research and infrastructure, we created Vac. Vac is an r&d unit doing protocol research focused on creating modular p2p -messaging protocols for private, secure, censorship resistant communication.

    I won't go into too much detail on the issues with Whisper, if you are +messaging protocols for private, secure, censorship resistant communication.

    +

    I won't go into too much detail on the issues with Whisper, if you are interested in this check out this talk here or this -article.

    In a nutshell, we forked Whisper to address immediate shortcomings and this +article.

    +

    In a nutshell, we forked Whisper to address immediate shortcomings and this became Waku v1. Waku v2 is complete re-thought implementation from scratch on top -of libp2p. This will be the subject of today's talk.

    Waku v2​

    Overview​

    Waku v2 is a privacy-preserving peer-to-peer messaging protocol for resource -restricted devices. We can look at Waku v2 as several things:

    • Set of protocols
    • Set of implementations
    • Network of nodes

    Let's first look at what the goals are.

    Goals​

    Waku v2 provides a PubSub based messaging protocol with the following -characteristics:

    1. Generalized messaging. Applications that require a messaging protocol to -communicate human to human, machine to machine, or a mix.
    2. Peer-to-peer. For applications that require a p2p solution.
    3. Resource restricted. For example, running with limited bandwidth, being -mostly-offline, or in a browser.
    4. Privacy. Applications that have privacy requirements, such as pseudonymity, -metadata protection, etc.

    And to provide these properties in a modular fashion, where applications can -choose their desired trade-offs.

    Protocols​

    Waku v2 consists of several protocols. Here we highlight a few of the most -important ones:

    • 10/WAKU2 - main specification, details how all the pieces fit together
    • 11/RELAY - thin layer on top of GossipSub for message dissemination
    • 13/STORE - fetching of historical messages
    • 14/MESSAGE - message payload

    This is the recommended subset for a minimal Waku v2 client.

    In addition to this there are many other types of specifications at various +of libp2p. This will be the subject of today's talk.

    +

    Waku v2​

    +

    Overview​

    +

    Waku v2 is a privacy-preserving peer-to-peer messaging protocol for resource +restricted devices. We can look at Waku v2 as several things:

    +
      +
    • Set of protocols
      • +
    • Set of implementations
      • +
    • Network of nodes
      • +
    +

    Let's first look at what the goals are.

    +

    Goals​

    +

    Waku v2 provides a PubSub based messaging protocol with the following +characteristics:

    +
      +
    1. Generalized messaging. Applications that require a messaging protocol to +communicate human to human, machine to machine, or a mix.
      2. +
    2. Peer-to-peer. For applications that require a p2p solution.
      3. +
    3. Resource restricted. For example, running with limited bandwidth, being +mostly-offline, or in a browser.
      4. +
    4. Privacy. Applications that have privacy requirements, such as pseudonymity, +metadata protection, etc.
      5. +
    +

    And to provide these properties in a modular fashion, where applications can +choose their desired trade-offs.

    +

    Protocols​

    +

    Waku v2 consists of several protocols. Here we highlight a few of the most +important ones:

    +
      +
    • 10/WAKU2 - main specification, details how all the pieces fit together
      • +
    • 11/RELAY - thin layer on top of GossipSub for message dissemination
      • +
    • 13/STORE - fetching of historical messages
      • +
    • 14/MESSAGE - message payload
      • +
    +

    This is the recommended subset for a minimal Waku v2 client.

    +

    In addition to this there are many other types of specifications at various stages of maturity, such as: content based filtering, bridge mode to Waku v1, JSON RPC API, zkSNARKS based spam protection with RLN, accounting and settlements with SWAP, fault-tolerant store nodes, recommendations around topic -usage, and more.

    See https://rfc.vac.dev/ for a full overview.

    Implementations​

    Waku v2 consists of multiple implementations. This allows for client diversity, +usage, and more.

    +

    See https://rfc.vac.dev/ for a full overview.

    +

    Implementations​

    +

    Waku v2 consists of multiple implementations. This allows for client diversity, makes it easier to strengthen the protocols, and allow people to use Waku v2 in -different contexts.

    • nim-waku - the reference client written in Nim, most full-featured.
    • js-waku - allow usage of Waku v2 from browsers, focus on interacting with dapps.
    • go-waku - subset of Waku v2 to ease integration into the Status app.

    Testnet Huilong and dogfooding​

    In order to test the protocol we have setup a testnet across all implementations -called Huilong. Yes, that's the Taipei subway station!

    Among us core devs we have disabled the main #waku Discord channel used for -development, and people run their own node connected to this toy chat application.

    Feel free to join and say hi! Instructions can be found here:

    Research​

    While Waku v2 is being used today, we are actively researching improvements. +different contexts.

    +
      +
    • nim-waku - the reference client written in Nim, most full-featured.
      • +
    • js-waku - allow usage of Waku v2 from browsers, focus on interacting with dapps.
      • +
    • go-waku - subset of Waku v2 to ease integration into the Status app.
      • +
    +

    Testnet Huilong and dogfooding​

    +

    In order to test the protocol we have setup a testnet across all implementations +called Huilong. Yes, that's the Taipei subway station!

    +

    +

    Among us core devs we have disabled the main #waku Discord channel used for +development, and people run their own node connected to this toy chat application.

    +

    Feel free to join and say hi! Instructions can be found here:

    + +

    Research​

    +

    While Waku v2 is being used today, we are actively researching improvements. Since the design is modular, we can gracefully introduce new capabilities. Some -of these research areas are:

    • Privacy-preserving spam protection using zkSNARKs and RLN
    • Accounting and settlement of resource usage to incentivize nodes to provide services with SWAP
    • State synchronization for store protocol to make it easier to run a store node without perfect uptime
    • Better node discovery
    • More rigorous privacy analysis
    • Improving interaction with wallets and dapp

    Use cases​

    Let's look at where Waku v2 is and can be used.

    Prelude: Topics in Waku v2​

    To give some context, there are two different types of topics in Waku v2. One is +of these research areas are:

    +
      +
    • Privacy-preserving spam protection using zkSNARKs and RLN
      • +
    • Accounting and settlement of resource usage to incentivize nodes to provide services with SWAP
      • +
    • State synchronization for store protocol to make it easier to run a store node without perfect uptime
      • +
    • Better node discovery
      • +
    • More rigorous privacy analysis
      • +
    • Improving interaction with wallets and dapp
      • +
    +

    Use cases​

    +

    Let's look at where Waku v2 is and can be used.

    +

    Prelude: Topics in Waku v2​

    +

    To give some context, there are two different types of topics in Waku v2. One is a PubSub topic, for routing. The other is a content topic, which is used for -content based filtering. Here's an example of the default PubSub topic:

    /waku/2/default-waku/proto

    This is recommended as it increases privacy for participants and it is stored by -default, however this is up to the application.

    The second type of topic is a content topic, which is application specific. For -example, here's the content topic used in our testnet:

    /toychat/2/huilong/proto

    For more on topics, see https://rfc.vac.dev/waku/informational/23/topics

    Status app​

    In the Status protocol, content topics - topics in Whisper/Waku v1 - are used for several things:

    • Contact code topic to discover X3DH bundles for perfect forward secrecy
      • Partitioned into N (currently 5000) content topics to balance privacy with efficiency
    • Public chats correspond to hash of the plaintext name
    • Negotiated topic for 1:1 chat with DHKE derived content topic

    See more here http://rfc.vac.dev/status/deprecated/waku-usage

    Currently, Status app is in the process of migrating to and testing Waku v2.

    DappConnect: Ethereum messaging​

    It is easy to think of Waku as being for human messaging, since that's how it is +content based filtering. Here's an example of the default PubSub topic:

    +

    /waku/2/default-waku/proto

    +

    This is recommended as it increases privacy for participants and it is stored by +default, however this is up to the application.

    +

    The second type of topic is a content topic, which is application specific. For +example, here's the content topic used in our testnet:

    +

    /toychat/2/huilong/proto

    +

    For more on topics, see https://rfc.vac.dev/waku/informational/23/topics

    +

    Status app​

    +

    In the Status protocol, content topics - topics in Whisper/Waku v1 - are used for several things:

    +
      +
    • Contact code topic to discover X3DH bundles for perfect forward secrecy +
        +
      • Partitioned into N (currently 5000) content topics to balance privacy with efficiency
        • +
      +
      • +
    • Public chats correspond to hash of the plaintext name
      • +
    • Negotiated topic for 1:1 chat with DHKE derived content topic
      • +
    +

    See more here http://rfc.vac.dev/status/deprecated/waku-usage

    +

    Currently, Status app is in the process of migrating to and testing Waku v2.

    +

    DappConnect: Ethereum messaging​

    +

    It is easy to think of Waku as being for human messaging, since that's how it is primarily used in the Status app, but the goal is to be useful for generalized -messaging, which includes Machine-To-Machine (M2M) messaging.

    Recall the concept of the holy trinity with Ethereum/Swarm/Whisper and Web3 that +messaging, which includes Machine-To-Machine (M2M) messaging.

    +

    Recall the concept of the holy trinity with Ethereum/Swarm/Whisper and Web3 that we mentioned in the beginning. Messaging can be used as a building block for dapps, wallets, and users to communicate with each other. It can be used for -things such as:

    • Multisig and DAO vote transactions only needing one on-chain operation
    • Giving dapps ability to send push notifications to users
    • Giving users ability to directly respond to requests from dapps
    • Decentralized WalletConnect
    • Etc

    Basically anything that requires communication and doesn't have to be on-chain.

    WalletConnect v2​

    WalletConnect is an open protocol for connecting dapps to wallets with a QR +things such as:

    +
      +
    • Multisig and DAO vote transactions only needing one on-chain operation
      • +
    • Giving dapps ability to send push notifications to users
      • +
    • Giving users ability to directly respond to requests from dapps
      • +
    • Decentralized WalletConnect
      • +
    • Etc
      • +
    +

    Basically anything that requires communication and doesn't have to be on-chain.

    +

    WalletConnect v2​

    +

    WalletConnect is an open protocol for connecting dapps to wallets with a QR code. Version 2 is using Waku v2 as a communication channel to do so in a -decentralized and private fashion.

    See for more: https://docs.walletconnect.org/v/2.0/tech-spec

    WalletConnect v2 is currently in late alpha using Waku v2.

    More examples​

    • Gasless voting and vote aggregation off-chain
    • Dapp games using Waku as player discovery mechanism
    • Send encrypted message to someone with an Ethereum key
    • <Your dapp here>

    These are all things that are in progress / proof of concept stage.

    Contribute​

    We'd love to see contributions of any form!

    Conclusion​

    In this talk we've gone over the original vision for Web3 and how Waku came to +decentralized and private fashion.

    +

    +

    See for more: https://docs.walletconnect.org/v/2.0/tech-spec

    +

    WalletConnect v2 is currently in late alpha using Waku v2.

    +

    More examples​

    +
      +
    • Gasless voting and vote aggregation off-chain
      • +
    • Dapp games using Waku as player discovery mechanism
      • +
    • Send encrypted message to someone with an Ethereum key
      • +
    • <Your dapp here>
      • +
    +

    These are all things that are in progress / proof of concept stage.

    +

    Contribute​

    +

    We'd love to see contributions of any form!

    + +

    Conclusion​

    +

    In this talk we've gone over the original vision for Web3 and how Waku came to be. We've also looked at what Waku v2 aims to do. We looked at its protocols, implementations, the current testnet as well as briefly on some ongoing -research for Vac.

    We've also looked at some specific use cases for Waku. First we looked at how +research for Vac.

    +

    We've also looked at some specific use cases for Waku. First we looked at how Status uses it with different topics. Then we looked at how it can be useful for -messaging in Ethereum, including for things like WalletConnect.

    I hope this talk gives you a better idea of what Waku is, why it exists, and +messaging in Ethereum, including for things like WalletConnect.

    +

    I hope this talk gives you a better idea of what Waku is, why it exists, and that it inspires you to contribute, either to Waku itself or by using it in your -own project!

    - - +own project!

    \ No newline at end of file diff --git a/rlog/waku-v2-ethereum-messaging/index.html b/rlog/waku-v2-ethereum-messaging/index.html index 33e8fd7d..226231b7 100644 --- a/rlog/waku-v2-ethereum-messaging/index.html +++ b/rlog/waku-v2-ethereum-messaging/index.html @@ -1,30 +1,151 @@ - + - -[Talk] Vac, Waku v2 and Ethereum Messaging | Vac Research + +[Talk] Vac, Waku v2 and Ethereum Messaging | Vac Research - - - + + + -
    -

    [Talk] Vac, Waku v2 and Ethereum Messaging

    by
    10 min read

    Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging.

    The following post is a transcript of the talk given at the Taipei Ethereum meetup, November 5. There is also a video recording.

    0. Introduction​

    Hi! My name is Oskar and I'm the protocol research lead at Vac. This talk will be divided into two parts. First I'll talk about the journey from Whisper, to Waku v1 and now to Waku v2. Then I'll talk about messaging in Ethereum. After this talk, you should have an idea of what Waku v2 is, the problems it is trying to solve, as well as where it can be useful for messaging in Ethereum.

    PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2​

    1. Vac intro​

    First, what is Vac? Vac grew out of our efforts Status to create a window on to Ethereum and secure messenger. Vac is modular protocol stack for p2p secure messaging, paying special attention to resource restricted devices, privacy and censorship resistance.

    Today we are going to talk mainly about Waku v2, which is the transport privacy / routing aspect of the Vac protocol stack. It sits "above" the p2p overlay, such as libp2p dealing with transports etc, and below a conversational security layer dealing with messaging encryption, such as using Double Ratchet etc.

    2. Whisper to Waku v1​

    In the beginning, there was Whisper. Whisper was part of the holy trinity of Ethereum. You had Ethereum for consensus/computation, Whisper for messaging, and Swarm for storage.

    However, for various reasons, Whisper didn't get the attention it deserved. Development dwindled, it promised too much and it suffered from many issues, such as being extremely inefficient and not being suitable for running on e.g. mobile phone. Despite this, Status used it in its app from around 2017 to 2019. As far as I know, it was one of very few, if not the only, production uses of Whisper.

    In an effort to solve some of its immediate problems, we forked Whisper into Waku and formalized it with a proper specification. This solved immediate bandwidth issues for light nodes, introduced rate limiting for better spam protection, improved historical message support, etc.

    If you are interested in this journey, checkout the EthCC talk Dean and I gave in Paris earlier this year.

    Status upgraded to Waku v1 early 2020. What next?

    3. Waku v1 to v2​

    We were far from done. The changes we had made were quite incremental and done in order to get tangible improvements as quickly as possible. This meant we couldn't address more fundamental issues related to full node routing scalability, running with libp2p for more transports, better security, better spam protection and incentivization.

    This kickstarted Waku v2 efforts, which is what we've been working on since July. This work was and is initally centered around a few pieces:

    (a) Moving to libp2p

    (b) Better routing

    (c) Accounting and user-run nodes

    The general theme was: making the Waku network more scalable and robust.

    We also did a scalability study to show at what point the network would run into issues, due to the inherent lack of routing that Whisper and Waku v1 provided.

    You can read more about this here.

    3.5 Waku v2 - Design goals​

    Taking a step back, what problem does Waku v2 attempt to solve compared to all the other solutions that exists out there? What type of applications should use it and why? We have the following design goals:

    1. Generalized messaging. Many applications requires some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix.

    2. Peer-to-peer. These applications sometimes have requirements that make them suitable for peer-to-peer solutions.

    3. Resource restricted. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:

      • limited bandwidth, CPU, memory, disk, battery, etc
      • not being publicly connectable
      • only being intermittently connected; mostly-offline

    4. Privacy. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.

    As well as to do so in a modular fashion. Meaning you can find a reasonable trade-off depending on your exact requirements. For example, you usually have to trade off some bandwidth to get metadata protection, and vice versa.

    The concept of designing for resource restricted devices also leads to the concept of adaptive nodes, where you have more of a continuum between full nodes and light nodes. For example, if you switch your phone from mobile data to WiFi you might be able to handle more bandwidth, and so on.

    4. Waku v2 - Breakdown​

    Where is Waku v2 at now, and how is it structured?

    It is running over libp2p and we had our second internal testnet last week or so. As a side note, we name our testnets after subway stations in Taipei, the first one being Nangang, and the most recent one being Dingpu.

    The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:

    • Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.
    • Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.
    • Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.
    • Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.
    • Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.
    • Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.

    Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.

    You can read more about the breakdown in this update though some progress has been made since then, as well was in the main Waku v2 spec.

    5. Waku v2 - Upcoming​

    What's coming up next? There are a few things.

    For Status to use it in production, it needs to be integrated into the main app using the Nim Node API. The bridge also needs to be implemented and tested.

    For other users, we are currently overhauling the API to allow usage from a browser, e.g. To make this experience great, there are also a few underlying infrastructure things that we need in nim-libp2p, such as a more secure HTTP server in Nim, Websockets and WebRTC support.

    There are also some changes we made to at what level content encryption happens, and this needs to be made easier to use in the API. This means you can use a node without giving your keys to it, which is useful in some environments.

    More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:

    • Better scaling, by using topic sharding.
    • Accounting and user-run nodes, to account for and incentives full nodes.
    • Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.
    • Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.

    As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next.

    PART 2 - ETHEREUM MESSAGING​

    A lot of what follows is inspired by exploratory work that John Lea has done at Status, previously Head of UX Architecture at Ubuntu.

    6. Ethereum Messaging - Why?​

    It is easy to think that Waku v2 is only for human to human messaging, since that's how Waku is currently primarily used in the Status app. However, the goal is to be useful for generalized messaging, which includes other type of information as well as machine to machine messaging.

    What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:

    • Reducing on-chain transactions
    • Reduce latency for operations
    • Decentralize centrally coordinated services (like WalletConnect)
    • Improve UX of dapps
    • Broadcast live information
    • A message transport layer for state channels

    And so on.

    7. Ethereum Messaging - Why? (Cont)​

    What are some examples of practical things Waku as used for Ethereum Messaging could solve?

    • Multisig transfers only needing one on chain transaction
    • DAO votes only needing one one chain transaction
    • Giving dapps ability to direct push notifications to users
    • Giving users ability to directly respond to requests from daps
    • Decentralized Wallet Connect

    Etc.

    8. What's needed to deliver this?​

    We can break it down into our actors:

    • Decentralized M2M messaging system (Waku)
    • Native wallets (Argent, Metamask, Status, etc)
    • Dapps that benefit from M2M messaging
    • Users whose problems are being solved

    Each of these has a bunch of requirements in turn. The messaging system needs to be decentralized, scalable, robust, etc. Wallets need support for messaging layer, dapps need to integrate this, etc.

    This is a lot! Growing adoption is a challenge. There is a catch 22 in terms of justifying development efforts for wallets, when no dapps need it, and likewise for dapps when no wallets support Waku. In addition to this, there must be proven usage of Waku before it can be relied on, etc. How can we break this up into smaller pieces of work?

    9. Breaking up the problem and a high level roadmap​

    We can start small. It doesn't and need to be used for critical features first. A more hybrid approach can be taken where it acts more as nice-to-haves.

    1. Forking Whisper and solving scalablity, spam etc issues with it. -This is a work in progress. What we talked about in part 1.
    2. Expose messaging API for Dapp developers.
    3. Implement decentralized version of WalletConnect. -Currently wallets connect ot dapps with centralized service. Great UX.
    4. Solve DAO/Multi-Sig coordination problem. -E.g. send message to wallet-derived key when it is time to sign a transaction.
    5. Extend dapp-to-user and user-to-dapp communication to more dapps. -Use lessons learned and examples to drive adoptation for wallets/dapps.

    And then build up from there.

    10. We are hiring!​

    A lot of this will happen in Javascript and browsers, since that's the primarily environment for a lot of wallets and dapps. We are currently hiring for a Waku JS Wallet integration lead to help push this effort further.

    Come talk to me after or apply here.

    That's it! You can find us on Status, Telegram, vac.dev. I'm on twitter here.

    Questions?

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    [Talk] Vac, Waku v2 and Ethereum Messaging

    by
    10 min read

    Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging.

    + +

    The following post is a transcript of the talk given at the Taipei Ethereum meetup, November 5. There is also a video recording.

    +
    +

    0. Introduction​

    +

    Hi! My name is Oskar and I'm the protocol research lead at Vac. This talk will be divided into two parts. First I'll talk about the journey from Whisper, to Waku v1 and now to Waku v2. Then I'll talk about messaging in Ethereum. After this talk, you should have an idea of what Waku v2 is, the problems it is trying to solve, as well as where it can be useful for messaging in Ethereum.

    +

    PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2​

    +

    1. Vac intro​

    +

    First, what is Vac? Vac grew out of our efforts Status to create a window on to Ethereum and secure messenger. Vac is modular protocol stack for p2p secure messaging, paying special attention to resource restricted devices, privacy and censorship resistance.

    +

    Today we are going to talk mainly about Waku v2, which is the transport privacy / routing aspect of the Vac protocol stack. It sits "above" the p2p overlay, such as libp2p dealing with transports etc, and below a conversational security layer dealing with messaging encryption, such as using Double Ratchet etc.

    +

    2. Whisper to Waku v1​

    +

    In the beginning, there was Whisper. Whisper was part of the holy trinity of Ethereum. You had Ethereum for consensus/computation, Whisper for messaging, and Swarm for storage.

    +

    However, for various reasons, Whisper didn't get the attention it deserved. Development dwindled, it promised too much and it suffered from many issues, such as being extremely inefficient and not being suitable for running on e.g. mobile phone. Despite this, Status used it in its app from around 2017 to 2019. As far as I know, it was one of very few, if not the only, production uses of Whisper.

    +

    In an effort to solve some of its immediate problems, we forked Whisper into Waku and formalized it with a proper specification. This solved immediate bandwidth issues for light nodes, introduced rate limiting for better spam protection, improved historical message support, etc.

    +

    If you are interested in this journey, checkout the EthCC talk Dean and I gave in Paris earlier this year.

    +

    Status upgraded to Waku v1 early 2020. What next?

    +

    3. Waku v1 to v2​

    +

    We were far from done. The changes we had made were quite incremental and done in order to get tangible improvements as quickly as possible. This meant we couldn't address more fundamental issues related to full node routing scalability, running with libp2p for more transports, better security, better spam protection and incentivization.

    +

    This kickstarted Waku v2 efforts, which is what we've been working on since July. This work was and is initally centered around a few pieces:

    +

    (a) Moving to libp2p

    +

    (b) Better routing

    +

    (c) Accounting and user-run nodes

    +

    The general theme was: making the Waku network more scalable and robust.

    +

    We also did a scalability study to show at what point the network would run into issues, due to the inherent lack of routing that Whisper and Waku v1 provided.

    +

    You can read more about this here.

    +

    3.5 Waku v2 - Design goals​

    +

    Taking a step back, what problem does Waku v2 attempt to solve compared to all the other solutions that exists out there? What type of applications should use it and why? We have the following design goals:

    +
      +
    1. +

      Generalized messaging. Many applications requires some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix.

      +
      2. +
    2. +

      Peer-to-peer. These applications sometimes have requirements that make them suitable for peer-to-peer solutions.

      +
      3. +
    3. +

      Resource restricted. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:

      +
        +
      • limited bandwidth, CPU, memory, disk, battery, etc
        • +
      • not being publicly connectable
        • +
      • only being intermittently connected; mostly-offline
        • +
      +
      4. +
    4. +

      Privacy. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.

      +
      5. +
    +

    As well as to do so in a modular fashion. Meaning you can find a reasonable trade-off depending on your exact requirements. For example, you usually have to trade off some bandwidth to get metadata protection, and vice versa.

    +

    The concept of designing for resource restricted devices also leads to the concept of adaptive nodes, where you have more of a continuum between full nodes and light nodes. For example, if you switch your phone from mobile data to WiFi you might be able to handle more bandwidth, and so on.

    +

    4. Waku v2 - Breakdown​

    +

    Where is Waku v2 at now, and how is it structured?

    +

    It is running over libp2p and we had our second internal testnet last week or so. As a side note, we name our testnets after subway stations in Taipei, the first one being Nangang, and the most recent one being Dingpu.

    +

    The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:

    +
      +
    • Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.
      • +
    • Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.
      • +
    • Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.
      • +
    • Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.
      • +
    • Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.
      • +
    • Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.
      • +
    +

    Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.

    +

    You can read more about the breakdown in this update though some progress has been made since then, as well was in the main Waku v2 spec.

    +

    5. Waku v2 - Upcoming​

    +

    What's coming up next? There are a few things.

    +

    For Status to use it in production, it needs to be integrated into the main app using the Nim Node API. The bridge also needs to be implemented and tested.

    +

    For other users, we are currently overhauling the API to allow usage from a browser, e.g. To make this experience great, there are also a few underlying infrastructure things that we need in nim-libp2p, such as a more secure HTTP server in Nim, Websockets and WebRTC support.

    +

    There are also some changes we made to at what level content encryption happens, and this needs to be made easier to use in the API. This means you can use a node without giving your keys to it, which is useful in some environments.

    +

    More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:

    +
      +
    • Better scaling, by using topic sharding.
      • +
    • Accounting and user-run nodes, to account for and incentives full nodes.
      • +
    • Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.
      • +
    • Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.
      • +
    +

    As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next.

    +

    PART 2 - ETHEREUM MESSAGING​

    +

    A lot of what follows is inspired by exploratory work that John Lea has done at Status, previously Head of UX Architecture at Ubuntu.

    +

    6. Ethereum Messaging - Why?​

    +

    It is easy to think that Waku v2 is only for human to human messaging, since that's how Waku is currently primarily used in the Status app. However, the goal is to be useful for generalized messaging, which includes other type of information as well as machine to machine messaging.

    +

    What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:

    +
      +
    • Reducing on-chain transactions
      • +
    • Reduce latency for operations
      • +
    • Decentralize centrally coordinated services (like WalletConnect)
      • +
    • Improve UX of dapps
      • +
    • Broadcast live information
      • +
    • A message transport layer for state channels
      • +
    +

    And so on.

    +

    7. Ethereum Messaging - Why? (Cont)​

    +

    What are some examples of practical things Waku as used for Ethereum Messaging could solve?

    +
      +
    • Multisig transfers only needing one on chain transaction
      • +
    • DAO votes only needing one one chain transaction
      • +
    • Giving dapps ability to direct push notifications to users
      • +
    • Giving users ability to directly respond to requests from daps
      • +
    • Decentralized Wallet Connect
      • +
    +

    Etc.

    +

    8. What's needed to deliver this?​

    +

    We can break it down into our actors:

    +
      +
    • Decentralized M2M messaging system (Waku)
      • +
    • Native wallets (Argent, Metamask, Status, etc)
      • +
    • Dapps that benefit from M2M messaging
      • +
    • Users whose problems are being solved
      • +
    +

    Each of these has a bunch of requirements in turn. The messaging system needs to be decentralized, scalable, robust, etc. Wallets need support for messaging layer, dapps need to integrate this, etc.

    +

    This is a lot! Growing adoption is a challenge. There is a catch 22 in terms of justifying development efforts for wallets, when no dapps need it, and likewise for dapps when no wallets support Waku. In addition to this, there must be proven usage of Waku before it can be relied on, etc. How can we break this up into smaller pieces of work?

    +

    9. Breaking up the problem and a high level roadmap​

    +

    We can start small. It doesn't and need to be used for critical features first. A more hybrid approach can be taken where it acts more as nice-to-haves.

    +
      +
    1. Forking Whisper and solving scalablity, spam etc issues with it. +This is a work in progress. What we talked about in part 1.
      2. +
    2. Expose messaging API for Dapp developers.
      3. +
    3. Implement decentralized version of WalletConnect. +Currently wallets connect ot dapps with centralized service. Great UX.
      4. +
    4. Solve DAO/Multi-Sig coordination problem. +E.g. send message to wallet-derived key when it is time to sign a transaction.
      5. +
    5. Extend dapp-to-user and user-to-dapp communication to more dapps. +Use lessons learned and examples to drive adoptation for wallets/dapps.
      6. +
    +

    And then build up from there.

    +

    10. We are hiring!​

    +

    A lot of this will happen in Javascript and browsers, since that's the primarily environment for a lot of wallets and dapps. We are currently hiring for a Waku JS Wallet integration lead to help push this effort further.

    +

    Come talk to me after or apply here.

    +

    That's it! You can find us on Status, Telegram, vac.dev. I'm on twitter here.

    +

    Questions?

    +
    \ No newline at end of file diff --git a/rlog/waku-v2-plan/index.html b/rlog/waku-v2-plan/index.html index 8ff28300..b8568b3b 100644 --- a/rlog/waku-v2-plan/index.html +++ b/rlog/waku-v2-plan/index.html @@ -1,35 +1,173 @@ - + - -What's the Plan for Waku v2? | Vac Research + +What's the Plan for Waku v2? | Vac Research - - - + + + -
    -

    What's the Plan for Waku v2?

    by
    14 min read

    Read about our plans for Waku v2, moving to libp2p, better routing, adaptive nodes and accounting!

    tldr: The Waku network is fragile and doesn't scale. Here's how to solve it.

    NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.

    Problem​

    The Waku network is fragile and doesn't scale.

    As Status is moving into a user-acquisition phase and is improving retention rates for users they need the infrastructure to keep up, specifically when it comes to messaging.

    Based on user acquisition models, the initial goal is to support 100k DAU in September, with demand growing from there.

    With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.

    **1. Connection limits**. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.

    **2. Bandwidth as a bottleneck**. We notice that memory usage appears to not be -the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.

    **3. Amplification factors**. Reducing amplification factors with better routing, would have a high impact, but it is likely we'd need additional measures as well, such as topic sharding or similar. See Fig 8-13.

    Figure 1-5:

    -
    -
    -
    -

    See https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0 for the full report.

    What we need to do is:

    1. Reduce amplification factors
    2. Get more user-run full nodes

    Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:

    • keep increasing requirements for Internet speed for full nodes
    • are vulnerable to censorship and attacks
    • have to control the topology in an artifical manner to keep up with load
    • basically re-invent a traditional centralized client-server app with extra steps
    • deliberately ignore most of our principles
    • risk the network being shut down when we run out of cash

    Appetite​

    Our initial risk appetite for this is 6 weeks for a small team.

    The idea is that we want to make tangible progress towards the goal in a limited period of time, as opposed to getting bogged down in trying to find a theoretically perfect generalized solution. Fixed time, variable scope.

    It is likely some elements of a complete solution will be done separately. See later sections for that.

    Solution​

    There are two main parts of the solution. One is to reduce amplification factors, and the other is incentivization to get more user run full nodes with desktop, etc.

    What does a full node provide? It provides connectivity to the network, can act as a bandwidth "barrier" and be high or reasonably high availability. What this means right now is essentially topic interest and storing historical messages.

    The goal is here to improve the status quo, not get a perfect solution from the get go. All of this can be iterated on further, for stronger guarantees, as well as replaced by other new modules.

    Let's first look at the baseline, and then go into some of the tracks and their phases. Track 1 is best done first, after which track 2 and 3 can be executed in parallel. Track 1 gives us more options for track 2 and 3. The work in track 1 is currently more well-defined, so it is likely the specifics of track 2 and 3 will get refined at a later stage.

    Baseline​

    Here's where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can't easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.

    Track 1 - Move to libp2p​

    Moving to PubSub over libp2p wouldn't improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.

    See more on libp2p PubSub here: https://docs.libp2p.io/concepts/publish-subscribe/

    As part of this move, there are a few individual pieces that are needed.

    1. FloodSub​

    This is essentially what Waku over libp2p would look like in its most basic form.

    One difference that is worth noting is that the app topics would not be the same as Waku topics. Why? In Waku we currently don't use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.

    Why can't we use Waku topics for routing directly? PubSub over libp2p isn't built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.

    Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.

    Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: https://github.com/status-im/nim-waku

    2. Simplify the protocol​

    Due to Waku's origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (https://rfc.vac.dev/waku/standards/legacy/6/waku1/). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?

    Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.

    The essential characteristic that has to be maintained is that we don't need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.

    3. Core integration​

    As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.

    4. Topic interest behavior​

    While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (https://vac.dev/fixing-whisper-with-waku). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.

    5. Historical message caching​

    Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there's no reason to reimplement a poor API if we don't have to.

    Also see section below on adaptive nodes and capabilities.

    6. Waku v1 <> Libp2p bridge​

    To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.

    Track 2 - Better routing​

    This is where we improve the amplification factors.

    1. GossipSub​

    This is a subprotocol of FloodSub in the libp2p world. Moving to GossipSub would allow traffic between full nodes to go from an amplification factor of ~25 to ~6. This basically creates a mesh of stable bidirectional connections, together with some gossiping capabilities outside of this view.

    Explaining how GossipSub works is out of scope of this document. It is implemented in nim-libp2p and used by Nimbus as part of Eth2. You can read the specs here in more detail if you are interested: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md and https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md

    -
    -
    -

    While we technically could implement this over existing Waku, we'd have to re-implement it, and we'd lose out on all the other benefits libp2p would provide, as well as the ecosystem of people and projects working on improving the scalability and security of these protocols.

    2. Topic sharding​

    This one is slightly more speculative in terms of its ultimate impact. The basic idea is to split the application topic into N shards, say 10, and then each full node can choose which shards to listen to. This can reduce amplification factors by another factor of 10.

    -

    Note that this means a light node that listens to several topics would have to be connected to more full nodes to get connectivity. For a more exotic version of this, see https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47

    This is orthogonal from the choice of FloodSub or GossipSub, but due to GossipSub's more dynamic nature it is likely best combined with it.

    3. Other factors​

    Not a primary focus, but worth a look. Looking at the scaling model, there might be other easy wins to improve overall bandwidth consumption between full nodes. For example, can we reduce envelope size by a significant factor?

    Track 3 - Accounting and user-run nodes​

    This is where we make sure the network isn't fragile, become a true p2p app, get our users excited and engaged, and allow us to scale the network without creating an even bigger cluster.

    To work in practice, this has a soft dependency on node discovery such as DNS based discovery (https://eips.ethereum.org/EIPS/eip-1459) or Discovery v5 (https://vac.dev/feasibility-discv5).

    1. Adaptive nodes and capabilities​

    We want to make the gradation between light nodes, full nodes, storing (partial set of) historical messages, only acting for a specific shard, etc more flexible and explicit. This is required to identify and discover the nodes you want. See https://github.com/vacp2p/specs/issues/87

    Depending on how the other tracks come together, this design should allow for a desktop node to identify as a full relaying node for some some app topic shard, but also express waku topic interest and retrieve historical messages itself.

    E.g. Disc v5 can be used to supply node properties through ENR.

    2. Accounting​

    This is based on a few principles:

    1. Some nodes contribute a lot more than other nodes in the network
    2. We can account for the difference in contribution in some fashion
    3. We want to incentivize nodes to tell the true, and be incentivized not to lie

    Accounting here is a stepping stone, where accounting is the raw data upon which some settlement later occurs. It can have various forms of granularity. See https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31 for discussion.

    We also note that in GossipSub, the mesh is bidrectional. Additionally, it doesn't appears to be a high priority issue in terms of nodes misreporting. What is an issue is having people run full nodes in the first place. There are a few points to that. It has to be possible in the end-user UX, nodes have to be discovered, and it has to be profitable/visible that you are contributing. UX and discovery are out of scope for this work, whereas visibility/accounting is part of this scope. Settlement is a stretch goal here.

    The general shape of the solution is inspired by the Swarm model, where we do accounting separate from settlement. It doesn't require any specific proofs, but nodes are incentivized to tell the truth in the following way:

    1. Both full node and light node do accounting in a pairwise, local fashion
    2. If a light node doesn't ultimately pay or lie about reporting, they get disconnected (e.g.)
    3. If a full node doesn't provide its service the light node may pick another full node (e.g.)

    While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:

    • end to end delivery
    • online time
    • completeness of storage

    This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See https://discuss.status.im/t/network-incentivisation-first-draft/1037 for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51 for using validity proofs and removing liveness requirement for settlement.

    All of this is optional at this stage, because our goal here is to improve the status quo for user run nodes. Accounting at this stage should be visible and correspond to the net benefit a node provides to another.

    As a concrete example: a light node has some topic interest and cares about historical messages on some topic. A full node communicates envelopes as they come in, communicates their high availability (online time) and stores/forward stored messages. Both nodes have this information, and if they agree settlement (initially just a mock message) can be sending a payment to an address at some time interval / over some defined volume. See future sections for how this can be improved upon.

    Also see below in section 4, using constructs such as eigentrust as a local reputation mechanism.

    3. Relax high availability requirement​

    If we want desktop nodes to participate in the storing of historical messages, high availability is a problem. It is a problem for any node, especially if they lie about it, but assuming they are honest it is still an issue.

    By being connected to multiple nodes, we can get an overlapping online window. Then these can be combined together to get consistency. This is obviously experimental and would need to be tested before being deployed, but if it works it'd be very useful.

    Additionally or alternatively, instead of putting a high requirement on message availability, focus on detection of missing information. This likely requires re-thinking how we do data sync / replication.

    4. Incentivize light and full nodes to tell the truth (policy, etc)​

    In accounting phase it is largely assumed nodes are honest. What happens when they lie, and how do we incentivize them to be honest? In the case of Bittorrent this is done with tit-for-tat, however this is a different kind of relationship. What follows are some examples of how this can be done.

    For light nodes:

    • if they don't, they get disconnected
    • prepayment (especially to "high value" nodes)

    For full nodes:

    • multiple nodes reporting to agree, where truth becomes a shelling point
    • use eigentrust
    • staking for discovery visibility with slashing

    5. Settlement PoC​

    Can be done after phase 2 if so desired. Basically integrate payments based on accounting and policy.

    Out of scope​

    1. We assume the Status Base model requirements are accurate.
    2. We assume Core will improve retention rates.
    3. We assume the Stimbus production team will enable integration of nim-waku.
    4. We assume Discovery mechanisms such as DNS and Discovery v5 will be worked on separately.
    5. We assume Core will, at some point, provide an UX for integrating payment of services.
    6. We assume the desktop client is sufficiently usable.
    7. We assume Core and Infra will investigate ways of improving MaxPeers.
    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    What's the Plan for Waku v2?

    by
    14 min read

    Read about our plans for Waku v2, moving to libp2p, better routing, adaptive nodes and accounting!

    + +

    tldr: The Waku network is fragile and doesn't scale. Here's how to solve it.

    +

    NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.

    +

    Problem​

    +

    The Waku network is fragile and doesn't scale.

    +

    As Status is moving into a user-acquisition phase and is improving retention rates for users they need the infrastructure to keep up, specifically when it comes to messaging.

    +

    Based on user acquisition models, the initial goal is to support 100k DAU in September, with demand growing from there.

    +

    With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.

    +

    **1. Connection limits**. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.

    +

    **2. Bandwidth as a bottleneck**. We notice that memory usage appears to not be +the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.

    +

    **3. Amplification factors**. Reducing amplification factors with better routing, would have a high impact, but it is likely we'd need additional measures as well, such as topic sharding or similar. See Fig 8-13.

    +

    Figure 1-5:

    +

    +
    +
    +
    +

    +

    See https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0 for the full report.

    +

    What we need to do is:

    +
      +
    1. Reduce amplification factors
      2. +
    2. Get more user-run full nodes
      3. +
    +

    Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:

    +
      +
    • keep increasing requirements for Internet speed for full nodes
      • +
    • are vulnerable to censorship and attacks
      • +
    • have to control the topology in an artifical manner to keep up with load
      • +
    • basically re-invent a traditional centralized client-server app with extra steps
      • +
    • deliberately ignore most of our principles
      • +
    • risk the network being shut down when we run out of cash
      • +
    +

    Appetite​

    +

    Our initial risk appetite for this is 6 weeks for a small team.

    +

    The idea is that we want to make tangible progress towards the goal in a limited period of time, as opposed to getting bogged down in trying to find a theoretically perfect generalized solution. Fixed time, variable scope.

    +

    It is likely some elements of a complete solution will be done separately. See later sections for that.

    +

    Solution​

    +

    There are two main parts of the solution. One is to reduce amplification factors, and the other is incentivization to get more user run full nodes with desktop, etc.

    +

    What does a full node provide? It provides connectivity to the network, can act as a bandwidth "barrier" and be high or reasonably high availability. What this means right now is essentially topic interest and storing historical messages.

    +

    The goal is here to improve the status quo, not get a perfect solution from the get go. All of this can be iterated on further, for stronger guarantees, as well as replaced by other new modules.

    +

    Let's first look at the baseline, and then go into some of the tracks and their phases. Track 1 is best done first, after which track 2 and 3 can be executed in parallel. Track 1 gives us more options for track 2 and 3. The work in track 1 is currently more well-defined, so it is likely the specifics of track 2 and 3 will get refined at a later stage.

    +

    Baseline​

    +

    Here's where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can't easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.

    +

    +

    Track 1 - Move to libp2p​

    +

    Moving to PubSub over libp2p wouldn't improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.

    +

    See more on libp2p PubSub here: https://docs.libp2p.io/concepts/publish-subscribe/

    +

    As part of this move, there are a few individual pieces that are needed.

    +

    1. FloodSub​

    +

    This is essentially what Waku over libp2p would look like in its most basic form.

    +

    One difference that is worth noting is that the app topics would not be the same as Waku topics. Why? In Waku we currently don't use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.

    +

    Why can't we use Waku topics for routing directly? PubSub over libp2p isn't built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.

    +

    +

    Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.

    +

    Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: https://github.com/status-im/nim-waku

    +

    2. Simplify the protocol​

    +

    Due to Waku's origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (https://rfc.vac.dev/waku/standards/legacy/6/waku1/). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?

    +

    Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.

    +

    The essential characteristic that has to be maintained is that we don't need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.

    +

    3. Core integration​

    +

    As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.

    +

    4. Topic interest behavior​

    +

    While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (https://vac.dev/fixing-whisper-with-waku). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.

    +

    5. Historical message caching​

    +

    Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there's no reason to reimplement a poor API if we don't have to.

    +

    Also see section below on adaptive nodes and capabilities.

    +

    6. Waku v1 <\> Libp2p bridge​

    +

    To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.

    +

    Track 2 - Better routing​

    +

    This is where we improve the amplification factors.

    +

    1. GossipSub​

    +

    This is a subprotocol of FloodSub in the libp2p world. Moving to GossipSub would allow traffic between full nodes to go from an amplification factor of ~25 to ~6. This basically creates a mesh of stable bidirectional connections, together with some gossiping capabilities outside of this view.

    +

    Explaining how GossipSub works is out of scope of this document. It is implemented in nim-libp2p and used by Nimbus as part of Eth2. You can read the specs here in more detail if you are interested: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md and https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md

    +

    +

    +
    +
    +

    +

    While we technically could implement this over existing Waku, we'd have to re-implement it, and we'd lose out on all the other benefits libp2p would provide, as well as the ecosystem of people and projects working on improving the scalability and security of these protocols.

    +

    2. Topic sharding​

    +

    This one is slightly more speculative in terms of its ultimate impact. The basic idea is to split the application topic into N shards, say 10, and then each full node can choose which shards to listen to. This can reduce amplification factors by another factor of 10.

    +

    +

    +

    +

    Note that this means a light node that listens to several topics would have to be connected to more full nodes to get connectivity. For a more exotic version of this, see https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47

    +

    This is orthogonal from the choice of FloodSub or GossipSub, but due to GossipSub's more dynamic nature it is likely best combined with it.

    +

    3. Other factors​

    +

    Not a primary focus, but worth a look. Looking at the scaling model, there might be other easy wins to improve overall bandwidth consumption between full nodes. For example, can we reduce envelope size by a significant factor?

    +

    Track 3 - Accounting and user-run nodes​

    +

    This is where we make sure the network isn't fragile, become a true p2p app, get our users excited and engaged, and allow us to scale the network without creating an even bigger cluster.

    +

    To work in practice, this has a soft dependency on node discovery such as DNS based discovery (https://eips.ethereum.org/EIPS/eip-1459) or Discovery v5 (https://vac.dev/feasibility-discv5).

    +

    1. Adaptive nodes and capabilities​

    +

    We want to make the gradation between light nodes, full nodes, storing (partial set of) historical messages, only acting for a specific shard, etc more flexible and explicit. This is required to identify and discover the nodes you want. See https://github.com/vacp2p/specs/issues/87

    +

    Depending on how the other tracks come together, this design should allow for a desktop node to identify as a full relaying node for some some app topic shard, but also express waku topic interest and retrieve historical messages itself.

    +

    E.g. Disc v5 can be used to supply node properties through ENR.

    +

    2. Accounting​

    +

    This is based on a few principles:

    +
      +
    1. Some nodes contribute a lot more than other nodes in the network
      2. +
    2. We can account for the difference in contribution in some fashion
      3. +
    3. We want to incentivize nodes to tell the true, and be incentivized not to lie
      4. +
    +

    Accounting here is a stepping stone, where accounting is the raw data upon which some settlement later occurs. It can have various forms of granularity. See https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31 for discussion.

    +

    We also note that in GossipSub, the mesh is bidrectional. Additionally, it doesn't appears to be a high priority issue in terms of nodes misreporting. What is an issue is having people run full nodes in the first place. There are a few points to that. It has to be possible in the end-user UX, nodes have to be discovered, and it has to be profitable/visible that you are contributing. UX and discovery are out of scope for this work, whereas visibility/accounting is part of this scope. Settlement is a stretch goal here.

    +

    The general shape of the solution is inspired by the Swarm model, where we do accounting separate from settlement. It doesn't require any specific proofs, but nodes are incentivized to tell the truth in the following way:

    +
      +
    1. Both full node and light node do accounting in a pairwise, local fashion
      2. +
    2. If a light node doesn't ultimately pay or lie about reporting, they get disconnected (e.g.)
      3. +
    3. If a full node doesn't provide its service the light node may pick another full node (e.g.)
      4. +
    +

    While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:

    +
      +
    • end to end delivery
      • +
    • online time
      • +
    • completeness of storage
      • +
    +

    This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See https://discuss.status.im/t/network-incentivisation-first-draft/1037 for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51 for using validity proofs and removing liveness requirement for settlement.

    +

    All of this is optional at this stage, because our goal here is to improve the status quo for user run nodes. Accounting at this stage should be visible and correspond to the net benefit a node provides to another.

    +

    As a concrete example: a light node has some topic interest and cares about historical messages on some topic. A full node communicates envelopes as they come in, communicates their high availability (online time) and stores/forward stored messages. Both nodes have this information, and if they agree settlement (initially just a mock message) can be sending a payment to an address at some time interval / over some defined volume. See future sections for how this can be improved upon.

    +

    Also see below in section 4, using constructs such as eigentrust as a local reputation mechanism.

    +

    3. Relax high availability requirement​

    +

    If we want desktop nodes to participate in the storing of historical messages, high availability is a problem. It is a problem for any node, especially if they lie about it, but assuming they are honest it is still an issue.

    +

    By being connected to multiple nodes, we can get an overlapping online window. Then these can be combined together to get consistency. This is obviously experimental and would need to be tested before being deployed, but if it works it'd be very useful.

    +

    Additionally or alternatively, instead of putting a high requirement on message availability, focus on detection of missing information. This likely requires re-thinking how we do data sync / replication.

    +

    4. Incentivize light and full nodes to tell the truth (policy, etc)​

    +

    In accounting phase it is largely assumed nodes are honest. What happens when they lie, and how do we incentivize them to be honest? In the case of Bittorrent this is done with tit-for-tat, however this is a different kind of relationship. What follows are some examples of how this can be done.

    +

    For light nodes:

    +
      +
    • if they don't, they get disconnected
      • +
    • prepayment (especially to "high value" nodes)
      • +
    +

    For full nodes:

    +
      +
    • multiple nodes reporting to agree, where truth becomes a shelling point
      • +
    • use eigentrust
      • +
    • staking for discovery visibility with slashing
      • +
    +

    5. Settlement PoC​

    +

    Can be done after phase 2 if so desired. Basically integrate payments based on accounting and policy.

    +

    Out of scope​

    +
      +
    1. We assume the Status Base model requirements are accurate.
      2. +
    2. We assume Core will improve retention rates.
      3. +
    3. We assume the Stimbus production team will enable integration of nim-waku.
      4. +
    4. We assume Discovery mechanisms such as DNS and Discovery v5 will be worked on separately.
      5. +
    5. We assume Core will, at some point, provide an UX for integrating payment of services.
      6. +
    6. We assume the desktop client is sufficiently usable.
      7. +
    7. We assume Core and Infra will investigate ways of improving MaxPeers.
      8. +
    \ No newline at end of file diff --git a/rlog/waku-v2-update/index.html b/rlog/waku-v2-update/index.html index ebaea039..8a20d8fe 100644 --- a/rlog/waku-v2-update/index.html +++ b/rlog/waku-v2-update/index.html @@ -1,28 +1,106 @@ - + - -Waku v2 Update | Vac Research + +Waku v2 Update | Vac Research - - - - + + + + -
    -

    Waku v2 Update

    by
    8 min read

    A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?

    It has been a while since the last post. It is time for an update on Waku v2. Aside from getting more familiar with libp2p (specifically nim-libp2p) and some vacation, what have we been up to? In this post we'll talk about what we've gotten done since last time, and briefly talk about immediate next steps and future. But first, a recap.

    Recap​

    In the last post (Waku v2 plan) we explained the rationale of Waku v2 - the current Waku network is fragile and doesn't scale. To solve this, Waku v2 aims to reduce amplification factors and get more user run nodes. We broke the work down into three separate tracks.

    1. Track 1 - Move to libp2p
    2. Track 2 - Better routing
    3. Track 3 - Accounting and user-run nodes

    As well as various rough components for each track. The primary initial focus is track 1. This means things like: moving to FloodSub, simplify the protocol, core integration, topic interest behavior, historical message caching, and Waku v1<>v2 bridge.

    Current state​

    Let's talk about the state of specs and our main implementation nim-waku. Then we'll go over our recent testnet, Nangang, and finish off with a Web PoC.

    Specs​

    After some back and forth on how to best structure things, we ended up breaking down the specs into a few pieces. While Waku v2 is best thought of as a cohesive whole in terms of its capabilities, it is made up of several protocols. Here's a list of the current specs and their status:

    Raw means there is not yet an implementation that corresponds fully to the spec, and draft means there is an implementation that corresponds to the spec. In the interest of space, we won't go into too much detail on the specs here except to note a few things:

    • The relay spec is essentially a thin wrapper on top of PubSub/FloodSub/GossipSub
    • The filter protocol corresponds to previous light client mode in Waku v1
    • The store protocol corresponds to the previous mailserver construct in Waku v1

    The filter and store protocol allow for adaptive nodes, i.e. nodes that have various capabilities. For example, a node being mostly offline, or having limited bandwidth capacity. The bridge spec outlines how to bridge the Waku v1 and v2 networks.

    Implementation​

    The main implementation we are working on is nim-waku. This builds on top of libraries such as nim-libp2p and others that the Nimbus team have been working on as part of their Ethereum 2.0 client.

    Currently nim-waku implements the relay protocol, and is close to implementing filter and store protocol. It also exposes a Nim Node API that allows libraries such as nim-status to use it. Additionally, there is also a rudimentary JSON RPC API for command line scripting.

    Nangang testnet​

    Last week we launched a very rudimentary internal testnet called Nangang. The goal was to test basic connectivity and make sure things work end to end. It didn't have things like: client integration, encryption, bridging, multiple clients, store/filter protocol, or even a real interface. What it did do is allow Waku developers to "chat" via RPC calls and looking in the log output. Doing this meant we exposed and fixed a few blockers, such as connection issues, deployment, topic subscription management, protocol and node integration, and basic scripting/API usage. After this, we felt confident enough to upgrade the main and relay spec to "draft" status.

    Waku Web PoC​

    As a bonus, we wanted to see what it'd take to get Waku running in a browser. This is a very powerful capability that enables a lot of use cases, and something that libp2p enables with its multiple transport support.

    Using the current stack, with nim-waku, would require quite a lot of ground work with WASM, WebRTC, Websockets support etc. Instead, we decided to take a shortcut and hack together a JS implementation called Waku Web Chat. This quick hack wouldn't be possible without the people behind js-libp2p-examples and js-libp2p and all its libraries. These are people like Jacob Heun, Vasco Santos, and Cayman Nava. Thanks!

    It consists of a brower implementation, a NodeJS implementation and a bootstrap server that acts as a signaling server for WebRTC. It is largely a bastardized version of GossipSub, and while it isn't completely to spec, it does allow messages originating from a browser to eventually end up at a nim-waku node, and vice versa. Which is pretty cool.

    Coming up​

    Now that we know what the current state is, what is still missing? what are the next steps?

    Things that are missing​

    While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope:

    1. Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level.

    2. Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status (status-nim has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings -us too...

    3. Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 (1, 2). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc).

    4. Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented.

    5. General tightening up of specs and implementation.

    While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope.

    Upcoming​

    Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details.

    1. Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors.

    2. Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner.

    3. Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet -format can improve things like unlinkability.

    4. zkSnarks RLN for spam protection and incentivization. We studied this last year and recent developments have made this relevant to study again. Create an experimental spec/PoC as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this!

    5. Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we'd like to deliver on that.

    As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates.

    If you are feeling adventurous and want to use early stage alpha software, check out the docs. If you want to read the specs, head over to Waku spec. If you want to talk with us, join us on Status or on Telegram (they are bridged).

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Waku v2 Update

    by
    8 min read

    A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?

    + +

    It has been a while since the last post. It is time for an update on Waku v2. Aside from getting more familiar with libp2p (specifically nim-libp2p) and some vacation, what have we been up to? In this post we'll talk about what we've gotten done since last time, and briefly talk about immediate next steps and future. But first, a recap.

    +

    Recap​

    +

    In the last post (Waku v2 plan) we explained the rationale of Waku v2 - the current Waku network is fragile and doesn't scale. To solve this, Waku v2 aims to reduce amplification factors and get more user run nodes. We broke the work down into three separate tracks.

    +
      +
    1. Track 1 - Move to libp2p
      2. +
    2. Track 2 - Better routing
      3. +
    3. Track 3 - Accounting and user-run nodes
      4. +
    +

    As well as various rough components for each track. The primary initial focus is track 1. This means things like: moving to FloodSub, simplify the protocol, core integration, topic interest behavior, historical message caching, and Waku v1<\>v2 bridge.

    +

    Current state​

    +

    Let's talk about the state of specs and our main implementation nim-waku. Then we'll go over our recent testnet, Nangang, and finish off with a Web PoC.

    +

    Specs​

    +

    After some back and forth on how to best structure things, we ended up breaking down the specs into a few pieces. While Waku v2 is best thought of as a cohesive whole in terms of its capabilities, it is made up of several protocols. Here's a list of the current specs and their status:

    + +

    Raw means there is not yet an implementation that corresponds fully to the spec, and draft means there is an implementation that corresponds to the spec. In the interest of space, we won't go into too much detail on the specs here except to note a few things:

    +
      +
    • The relay spec is essentially a thin wrapper on top of PubSub/FloodSub/GossipSub
      • +
    • The filter protocol corresponds to previous light client mode in Waku v1
      • +
    • The store protocol corresponds to the previous mailserver construct in Waku v1
      • +
    +

    The filter and store protocol allow for adaptive nodes, i.e. nodes that have various capabilities. For example, a node being mostly offline, or having limited bandwidth capacity. The bridge spec outlines how to bridge the Waku v1 and v2 networks.

    +

    Implementation​

    +

    The main implementation we are working on is nim-waku. This builds on top of libraries such as nim-libp2p and others that the Nimbus team have been working on as part of their Ethereum 2.0 client.

    +

    Currently nim-waku implements the relay protocol, and is close to implementing filter and store protocol. It also exposes a Nim Node API that allows libraries such as nim-status to use it. Additionally, there is also a rudimentary JSON RPC API for command line scripting.

    +

    Nangang testnet​

    +

    Last week we launched a very rudimentary internal testnet called Nangang. The goal was to test basic connectivity and make sure things work end to end. It didn't have things like: client integration, encryption, bridging, multiple clients, store/filter protocol, or even a real interface. What it did do is allow Waku developers to "chat" via RPC calls and looking in the log output. Doing this meant we exposed and fixed a few blockers, such as connection issues, deployment, topic subscription management, protocol and node integration, and basic scripting/API usage. After this, we felt confident enough to upgrade the main and relay spec to "draft" status.

    +

    Waku Web PoC​

    +

    As a bonus, we wanted to see what it'd take to get Waku running in a browser. This is a very powerful capability that enables a lot of use cases, and something that libp2p enables with its multiple transport support.

    +

    Using the current stack, with nim-waku, would require quite a lot of ground work with WASM, WebRTC, Websockets support etc. Instead, we decided to take a shortcut and hack together a JS implementation called Waku Web Chat. This quick hack wouldn't be possible without the people behind js-libp2p-examples and js-libp2p and all its libraries. These are people like Jacob Heun, Vasco Santos, and Cayman Nava. Thanks!

    +

    It consists of a brower implementation, a NodeJS implementation and a bootstrap server that acts as a signaling server for WebRTC. It is largely a bastardized version of GossipSub, and while it isn't completely to spec, it does allow messages originating from a browser to eventually end up at a nim-waku node, and vice versa. Which is pretty cool.

    +

    Coming up​

    +

    Now that we know what the current state is, what is still missing? what are the next steps?

    +

    Things that are missing​

    +

    While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope:

    +
      +
    1. +

      Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level.

      +
      2. +
    2. +

      Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status (status-nim has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings +us too...

      +
      3. +
    3. +

      Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 (1, 2). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc).

      +
      4. +
    4. +

      Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented.

      +
      5. +
    5. +

      General tightening up of specs and implementation.

      +
      6. +
    +

    While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope.

    +

    Upcoming​

    +

    Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details.

    +
      +
    1. +

      Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors.

      +
      2. +
    2. +

      Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner.

      +
      3. +
    3. +

      Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet +format can improve things like unlinkability.

      +
      4. +
    4. +

      zkSnarks RLN for spam protection and incentivization. We studied this last year and recent developments have made this relevant to study again. Create an experimental spec/PoC as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this!

      +
      5. +
    5. +

      Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we'd like to deliver on that.

      +
      6. +
    +

    As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates.

    +
    +

    If you are feeling adventurous and want to use early stage alpha software, check out the docs. If you want to read the specs, head over to Waku spec. If you want to talk with us, join us on Status or on Telegram (they are bridged).

    \ No newline at end of file diff --git a/rlog/wakuv2-apd/index.html b/rlog/wakuv2-apd/index.html index 45a2b2a9..fd091774 100644 --- a/rlog/wakuv2-apd/index.html +++ b/rlog/wakuv2-apd/index.html @@ -1,32 +1,35 @@ - + - -Waku v2 Ambient Peer Discovery | Vac Research + +Waku v2 Ambient Peer Discovery | Vac Research - - - + + + -
    -

    Waku v2 Ambient Peer Discovery

    by
    18 min read

    Introducing and discussing ambient peer discovery methods currently used by Waku v2, as well as future plans in this area.

    Waku v2 comprises a set of modular protocols for secure, privacy preserving communication. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Waku v2 Ambient Peer Discovery

    by
    18 min read

    Introducing and discussing ambient peer discovery methods currently used by Waku v2, as well as future plans in this area.

    + +

    Waku v2 comprises a set of modular protocols for secure, privacy preserving communication. Avoiding centralization, these protocols exchange messages over a P2P network layer. In order to build a P2P network, participating nodes first have to discover peers within this network. This is where ambient peer discovery comes into play: -it allows nodes to find peers, making it an integral part of any decentralized application.

    In this post the term node to refers to our endpoint or the endpoint that takes action, +it allows nodes to find peers, making it an integral part of any decentralized application.

    +

    In this post the term node to refers to our endpoint or the endpoint that takes action, while the term peer refers to other endpoints in the P2P network. These endpoints can be any device connected to the Internet: e.g. servers, PCs, notebooks, mobile devices, or applications like a browser. -As such, nodes and peers are the same. We use these terms for the ease of explanation without loss of generality.

    In Waku's modular design, ambient peer discovery is an umbrella term for mechanisms that allow nodes to find peers. +As such, nodes and peers are the same. We use these terms for the ease of explanation without loss of generality.

    +

    In Waku's modular design, ambient peer discovery is an umbrella term for mechanisms that allow nodes to find peers. Various ambient peer discovery mechanisms are supported, and each is specified as a separate protocol. Where do these protocols fit into Waku's protocol stack? The P2P layer of Waku v2 builds on libp2p gossipsub. @@ -35,12 +38,15 @@ This mesh network is an Discovery v5 and DNS-based discovery. @@ -48,118 +54,240 @@ Waku v2 further supports Static Node Lists​

    The simplest method of learning about peers in a P2P network is via static node lists. +Besides these new protocols, we are working on integrating capability discovery in our existing ambient peer discovery protocols.

    +

    Static Node Lists​

    +

    The simplest method of learning about peers in a P2P network is via static node lists. These can be given to nodes as start-up parameters or listed in a config-file. They can also be provided in a script-parseable format, e.g. in JSON. While this method of providing bootstrap nodes is very easy to implement, it requires static peers, which introduce centralized elements. Also, updating static peer information introduces significant administrative overhead: code and/or config files have to be updated and released. -Typically, static node lists only hold a small number of bootstrap nodes, which may lead to high load on these nodes.

    DNS-based Discovery​

    Compared to static node lists, +Typically, static node lists only hold a small number of bootstrap nodes, which may lead to high load on these nodes.

    +

    DNS-based Discovery​

    +

    Compared to static node lists, DNS-based discovery (specified in EIP-1459) provides a more dynamic way of discovering bootstrap nodes. It is very efficient, can easily be handled by resource restricted devices and provides very good availability. -In addition to a naive DNS approach, Ethereum's DNS-based discovery introduces efficient authentication leveraging Merkle trees.

    A further advantage over static node lists is the separation of code/release management and bootstrap node management. -However, changing and updating the list of bootstrap nodes still requires administrative privileges because DNS records have to be added or updated.

    While this method of discovery still requires centralized elements, -node list management can be delegated to various DNS zones managed by other entities mitigating centralization.

    Discovery V5​

    A much more dynamic method of ambient peer discovery is Discovery v5, which is Ethereum's peer discovery protocol. +In addition to a naive DNS approach, Ethereum's DNS-based discovery introduces efficient authentication leveraging Merkle trees.

    +

    A further advantage over static node lists is the separation of code/release management and bootstrap node management. +However, changing and updating the list of bootstrap nodes still requires administrative privileges because DNS records have to be added or updated.

    +

    While this method of discovery still requires centralized elements, +node list management can be delegated to various DNS zones managed by other entities mitigating centralization.

    +

    Discovery V5​

    +

    A much more dynamic method of ambient peer discovery is Discovery v5, which is Ethereum's peer discovery protocol. It is based on the Kademlia distributed hashtable (DHT). An introduction to discv5 and its history, and a discv5 Waku v2 feasibility study -can be found in previous posts on this research log.

    We use Discovery v5 as an ambient peer discovery method for Waku v2 because it is decentralized, efficient, actively researched, and has web3 as its main application area. -Discv5 also offers mitigation techniques for various attacks, which we cover later in this post.

    Using a DHT (structured P2P network) as a means for ambient peer discovery, while using the gossipsub mesh network (unstructured P2P network) for transmitting actual messages, +can be found in previous posts on this research log.

    +

    We use Discovery v5 as an ambient peer discovery method for Waku v2 because it is decentralized, efficient, actively researched, and has web3 as its main application area. +Discv5 also offers mitigation techniques for various attacks, which we cover later in this post.

    +

    Using a DHT (structured P2P network) as a means for ambient peer discovery, while using the gossipsub mesh network (unstructured P2P network) for transmitting actual messages, Waku v2 leverages advantages from both worlds. One of the main benefits of DHTs is offering a global view over participating nodes. This, in turn, allows sampling random sets of nodes which is important for equally distributing load. Gossipsub, on the other hand, offers great robustness and resilience against attacks. -Even if discv5 discovery should not work in advent of a DoS attack, Waku v2 can still operate switching to different discovery methods.

    Discovery methods that use separate P2P networks still depend on bootstrapping, +Even if discv5 discovery should not work in advent of a DoS attack, Waku v2 can still operate switching to different discovery methods.

    +

    Discovery methods that use separate P2P networks still depend on bootstrapping, which Waku v2 does via parameters on start-up or via DNS-based discovery. This might raise the question of why such discovery methods are beneficial. The answer lies in the aforementioned global view of DHTs. Without discv5 and similar methods, the bootstrap nodes are used as part of the gossipsub mesh. This might put heavy load on these nodes and further, might open pathways to inference attacks. Discv5, on the other hand, uses the bootstrap nodes merely as an entry to the discovery network and can provide random sets of nodes (sampled from a global view) -for bootstrapping or expanding the mesh.

    DHT Background​

    Distributed Hash Tables are a class of structured P2P overlay networks. +for bootstrapping or expanding the mesh.

    +

    DHT Background​

    +

    Distributed Hash Tables are a class of structured P2P overlay networks. A DHT can be seen as a distributed node set of which each node is responsible for a part of the hash space. In contrast to unstructured P2P networks, e.g. the mesh network maintained by gossipsub, -DHTs have a global view over the node set and the hash space (assuming the participating nodes behave well).

    DHTs are susceptible to various kinds of attacks, especially Sybil attacks +DHTs have a global view over the node set and the hash space (assuming the participating nodes behave well).

    +

    DHTs are susceptible to various kinds of attacks, especially Sybil attacks and eclipse attacks. While security aspects have been addressed in various research papers, general practical solutions are not available. -However, discv5 introduced various practical mitigation techniques.

    Random Walk Discovery​

    While discv5 is based on the Kademlia DHT, it only uses the distributed node set aspect of DHTs. +However, discv5 introduced various practical mitigation techniques.

    +

    Random Walk Discovery​

    +

    While discv5 is based on the Kademlia DHT, it only uses the distributed node set aspect of DHTs. It does not map values (items) into the distributed hash space. This makes sense, because the main purpose of discv5 is discovering other nodes that support discv5, which are expected to be Ethereum nodes. Ethereum nodes that want to discover other Ethereum nodes simply query the discv5 network for a random set of peers. -If Waku v2 would do the same, only a small subset of the retrieved nodes would support Waku v2.

    A first naive solution for Waku v2 discv5 discovery is

    • retrieve a random node set, which is achieved by querying for a set of randomly chosen node IDs
    • filter the returned nodes on the query path based on Waku v2 capability via the Waku v2 ENR
    • repeat until enough Waku v2 capable nodes are found

    This query process boils down to random walk discovery, which is very resilient against attacks, but also very inefficient if the number of nodes supporting the desired capability is small. -We refer to this as the needle-in-the-haystack problem.

    Random Walk Performance Estimation​

    This subsection provides a rough estimation of the overhead introduced by random walk discovery.

    Given the following parameters:

    • nn number of total nodes participating in discv5
    • pp percentage of nodes supporting Waku
    • WW the event of having at least one Waku node in a random sample
    • kk the size of a random sample (default = 16)
    • α\alpha the number of parallel queries started
    • bb bits per hop
    • qq the number of queries

    A query takes log2bnlog_{2^b}n hops to retrieve a random sample of nodes.

    P(W)=1−(1−p/100)kP(W) = 1 - (1-p/100)^k is the probability of having at least one Waku node in the sample.

    P(Wq)=1−(1−p/100)kqP(W^q) = 1 - (1-p/100)^{kq} is the probability of having at least one Waku node in the union of qq samples.

    Expressing this in terms of qq, we can write: -P(Wq)=1−(1−p/100)kq  ⟺  q=log(1−p/100)k(1−P(Wq))P(W^q) = 1 - (1-p/100)^{kq} \iff q = log_{(1-p/100)^k}(1-P(W^q))

    Figure 1 shows a log-log plot for P(Wq)=90%P(W^q) = 90\%.

    Figure 1: log-log plot showing the number of queries necessary to retrieve a Waku v2 node with a probability of 90% in relation to the Waku v2 node concentration in the network.

    Assuming p=0.1p=0.1, we would need

    0.9=1−(1−0.1/100)16q=>q≈1440.9 = 1 - (1-0.1/100)^{16q} => q \approx 144

    queries to get a Waku node with 90% probability, which leads to ≈144∗18=2592\approx 144 * 18 = 2592 overlay hops. -Choosing b=3b=3 would reduce the number to ≈144∗6=864\approx 144 * 6 = 864. -Even when choosing α=10\alpha = 10 we would have to wait at least 80 RTTs. -This effort is just for retrieving a single Waku node. Ideally, we want at least 3 Waku nodes for bootstrapping a Waku relay.

    The discv5 doc roughly estimates p=1p=1% to be the threshold for acceptably efficient random walk discovery. -This is in line with our estimation:

    0.9=1−(1−1/100)16q=>q≈140.9 = 1 - (1-1/100)^{16q} => q \approx 14

    The number of necessary queries is linearly dependent on the percentage pp of Waku nodes. -The number of hops per query is logarithmically dependent on nn. -Thus, random walk searching is inefficient for small percentages pp. -Still, random walks are more resilient against attacks.

    We can conclude that a Waku node concentration below 1% renders vanilla discv5 unfit for our needs. -Our current solution and future plans for solving this issue are covered in the next subsections.

    Simple Solution: Separate Discovery Network​

    The simple solution we currently use for Waku v2 discv5 is a separate discv5 network. +If Waku v2 would do the same, only a small subset of the retrieved nodes would support Waku v2.

    +

    A first naive solution for Waku v2 discv5 discovery is

    +
      +
    • retrieve a random node set, which is achieved by querying for a set of randomly chosen node IDs
      • +
    • filter the returned nodes on the query path based on Waku v2 capability via the Waku v2 ENR
      • +
    • repeat until enough Waku v2 capable nodes are found
      • +
    +

    This query process boils down to random walk discovery, which is very resilient against attacks, but also very inefficient if the number of nodes supporting the desired capability is small. +We refer to this as the needle-in-the-haystack problem.

    +

    Random Walk Performance Estimation​

    +

    This subsection provides a rough estimation of the overhead introduced by random walk discovery.

    +

    Given the following parameters:

    +
      +
    • nn number of total nodes participating in discv5
      • +
    • pp percentage of nodes supporting Waku
      • +
    • WW the event of having at least one Waku node in a random sample
      • +
    • kk the size of a random sample (default = 16)
      • +
    • α\alpha the number of parallel queries started
      • +
    • bb bits per hop
      • +
    • qq the number of queries
      • +
    +

    A query takes log2bnlog_{2^b}n hops to retrieve a random sample of nodes.

    +

    P(W)=1−(1−p/100)kP(W) = 1 - (1-p/100)^k is the probability of having at least one Waku node in the sample.

    +

    P(Wq)=1−(1−p/100)kqP(W^q) = 1 - (1-p/100)^{kq} is the probability of having at least one Waku node in the union of qq samples.

    +

    Expressing this in terms of qq, we can write: +P(Wq)=1−(1−p/100)kq  ⟺  q=log(1−p/100)k(1−P(Wq))P(W^q) = 1 - (1-p/100)^{kq} \iff q = log_{(1-p/100)^k}(1-P(W^q))

    +

    Figure 1 shows a log-log plot for P(Wq)=90%P(W^q) = 90\%.

    +

    Figure 1: log-log plot showing the number of queries necessary to retrieve a Waku v2 node with a probability of 90% in relation to the Waku v2 node concentration in the network.

    +

    Assuming p=0.1p=0.1, we would need

    +

    0.9=1−(1−0.1/100)16q=>q≈1440.9 = 1 - (1-0.1/100)^{16q} => q \approx 144

    +

    queries to get a Waku node with 90% probability, which leads to ≈144∗18=2592\approx 144 * 18 = 2592 overlay hops. +Choosing b=3b=3 would reduce the number to ≈144∗6=864\approx 144 * 6 = 864. +Even when choosing α=10\alpha = 10 we would have to wait at least 80 RTTs. +This effort is just for retrieving a single Waku node. Ideally, we want at least 3 Waku nodes for bootstrapping a Waku relay.

    +

    The discv5 doc roughly estimates p=1p=1% to be the threshold for acceptably efficient random walk discovery. +This is in line with our estimation:

    +

    0.9=1−(1−1/100)16q=>q≈140.9 = 1 - (1-1/100)^{16q} => q \approx 14

    +

    The number of necessary queries is linearly dependent on the percentage pp of Waku nodes. +The number of hops per query is logarithmically dependent on nn. +Thus, random walk searching is inefficient for small percentages pp. +Still, random walks are more resilient against attacks.

    +

    We can conclude that a Waku node concentration below 1% renders vanilla discv5 unfit for our needs. +Our current solution and future plans for solving this issue are covered in the next subsections.

    +

    Simple Solution: Separate Discovery Network​

    +

    The simple solution we currently use for Waku v2 discv5 is a separate discv5 network. All (well behaving) nodes in this network support Waku v2, resulting in a very high query efficiency. -However, this solution reduces resilience because the difficulty of attacking a DHT scales with the number of participating nodes.

    Discv5 Topic Discovery​

    We did not base our solution on the current version of discv5 topic discovery, -because, similar to random walk discovery, it suffers from poor performance for relatively rare capabilities/topics.

    However, there is ongoing research in discv5 topic discovery which is close to ideas we explored when pondering efficient and resilient Waku discv5 solutions. -We keep a close eye on this research, give feedback, and make suggestions, as we plan to switch to this version of topic discovery in the future.

    In a nutshell, topic discovery will manage separate routing tables for each topic. +However, this solution reduces resilience because the difficulty of attacking a DHT scales with the number of participating nodes.

    +

    Discv5 Topic Discovery​

    +

    We did not base our solution on the current version of discv5 topic discovery, +because, similar to random walk discovery, it suffers from poor performance for relatively rare capabilities/topics.

    +

    However, there is ongoing research in discv5 topic discovery which is close to ideas we explored when pondering efficient and resilient Waku discv5 solutions. +We keep a close eye on this research, give feedback, and make suggestions, as we plan to switch to this version of topic discovery in the future.

    +

    In a nutshell, topic discovery will manage separate routing tables for each topic. These topic specific tables are initialized with nodes from the discv5 routing table. -While the buckets of the discv5 routing table represent distance intervals from the node's node ID, the topic table buckets represent distance intervals from topic IDs.

    Nodes that want to register a topic try to register that topic at one random peer per bucket. +While the buckets of the discv5 routing table represent distance intervals from the node's node ID, the topic table buckets represent distance intervals from topic IDs.

    +

    Nodes that want to register a topic try to register that topic at one random peer per bucket. This leads to registering the topic at peers in closer and closer neighbourhoods around the topic ID, which yields a very efficient and resilient compromise between random walk discovery and DHT discovery. -Peers in larger neighbourhoods around the topic ID are less efficient to discover, however more resilient against eclipse attacks and vice versa.

    Further, this works well with the overload and DoS protection discv5 employs. +Peers in larger neighbourhoods around the topic ID are less efficient to discover, however more resilient against eclipse attacks and vice versa.

    +

    Further, this works well with the overload and DoS protection discv5 employs. Discv5 limits the amount of nodes registered per topic on a single peer. Further, discv5 enforces a waiting time before nodes can register topics at peers. So, for popular topics, a node might fail to register the topic in a close neighbourhood. -However, because the topic is popular (has a high occurrence percentage pp), it can still be efficiently discovered.

    In the future, we also plan to integrate Waku v2 capability discovery, which will not only allow asking for nodes that support Waku v2, +However, because the topic is popular (has a high occurrence percentage pp), it can still be efficiently discovered.

    +

    In the future, we also plan to integrate Waku v2 capability discovery, which will not only allow asking for nodes that support Waku v2, but asking for Waku v2 nodes supporting specific Waku v2 protocols like filter or store. For the store protocol we envision sub-capabilities reflecting message topics and time frames of messages. -We will also investigate related security implications.

    Attacks on DHTs​

    In this post, we only briefly describe common attacks on DHTs. +We will also investigate related security implications.

    +

    Attacks on DHTs​

    +

    In this post, we only briefly describe common attacks on DHTs. These attacks are mainly used for denial of service (DoS), but can also used as parts of more sophisticated attacks, e.g. deanonymization attacks. -A future post on this research log will cover security aspects of ambient peer discovery with a focus on privacy and anonymity.

    Sybil Attack

    The power of an attacker in a DHT is proportional to the number of controlled nodes. -Controlling nodes comes at a high resource cost and/or requires controlling a botnet via a preliminary attack.

    In a Sybil attack, an attacker generates lots of virtual node identities. +A future post on this research log will cover security aspects of ambient peer discovery with a focus on privacy and anonymity.

    +

    Sybil Attack

    +

    The power of an attacker in a DHT is proportional to the number of controlled nodes. +Controlling nodes comes at a high resource cost and/or requires controlling a botnet via a preliminary attack.

    +

    In a Sybil attack, an attacker generates lots of virtual node identities. This allows the attacker to control a large portion of the ID space in a DHT at a relatively low cost. Sybil attacks are especially powerful when the attacker can freely choose the IDs of generated nodes, -because this allows positioning at chosen points in the DHT.

    Because Sybil attacks amplify the power of many attacks against DHTs, +because this allows positioning at chosen points in the DHT.

    +

    Because Sybil attacks amplify the power of many attacks against DHTs, making Sybil attacks as difficult as possible is the basis for resilient DHT operation. The typical abstract mitigation approach is binding node identities to physical network interfaces. To some extend, this can be achieved by introducing IP address based limits. Further, generating node IDs can be bound by proof of work (PoW), which, however, comes with a set of shortcomings, e.g. relatively high costs on resource restricted devices. The discv5 doc -describes both Sybil and eclipse attacks, as well as concrete mitigation techniques employed by discv5.

    Eclipse Attack

    In an eclipse attack, nodes controlled by the attacker poison the routing tables of other nodes in a way that parts of the DHT become eclipsed, i.e. invisible. +describes both Sybil and eclipse attacks, as well as concrete mitigation techniques employed by discv5.

    +

    Eclipse Attack

    +

    In an eclipse attack, nodes controlled by the attacker poison the routing tables of other nodes in a way that parts of the DHT become eclipsed, i.e. invisible. When a controlled node is asked for the next step in a path, it provides another controlled node as the next step, effectively navigating the querying node around or away from certain areas of the DHT. While several mitigation techniques have been researched, there is no definitive protection against eclipse attacks available as of yet. -One mitigation technique is increasing α\alpha, the number of parallel queries, and following each concurrent path independently for the lookup.

    The eclipse attack becomes very powerful in combination with a successful Sybil attack; -especially when the attacker can freely choose the position of the Sybil nodes.

    The aforementioned new topic discovery of discv5 provides a good balance between protection against eclipse attacks and query performance.

    Peer Exchange Protocol​

    While discv5 based ambient peer discovery has many desirable properties, resource restricted nodes and nodes behind restrictive NAT setups cannot run discv5 satisfactory. +One mitigation technique is increasing α\alpha, the number of parallel queries, and following each concurrent path independently for the lookup.

    +

    The eclipse attack becomes very powerful in combination with a successful Sybil attack; +especially when the attacker can freely choose the position of the Sybil nodes.

    +

    The aforementioned new topic discovery of discv5 provides a good balance between protection against eclipse attacks and query performance.

    +

    Peer Exchange Protocol​

    +

    While discv5 based ambient peer discovery has many desirable properties, resource restricted nodes and nodes behind restrictive NAT setups cannot run discv5 satisfactory. With these nodes in mind, we started working on a simple peer exchange protocol based on ideas proposed here. The peer exchange protocol will allow nodes to ask peers for additional peers. -Similar to discv5, the peer exchange protocol will also support capability discovery.

    The new peer exchange protocol can be seen as a simple replacement for the Rendezvous protocol, which Waku v2 does not support. +Similar to discv5, the peer exchange protocol will also support capability discovery.

    +

    The new peer exchange protocol can be seen as a simple replacement for the Rendezvous protocol, which Waku v2 does not support. While the rendezvous protocol involves nodes registering at rendezvous peers, the peer exchange protocol simply allows nodes to ask any peer for a list of peers (with a certain set of capabilities). -Rendezvous tends to introduce centralized elements as rendezvous peers have a super-peer role.

    In the future, we will investigate resource usage of Waku v2 discv5 and provide suggestions for minimal resources nodes should have to run discv5 satisfactory.

    Waku v2 comprises further protocols related to ambient peer discovery. We shortly mention them for context, even though they are not strictly ambient peer discovery protocols.

    Gossipsub Peer Exchange Protocol​

    Gossipsub provides an integrated peer exchange mechanism which is also supported by Waku v2. +Rendezvous tends to introduce centralized elements as rendezvous peers have a super-peer role.

    +

    In the future, we will investigate resource usage of Waku v2 discv5 and provide suggestions for minimal resources nodes should have to run discv5 satisfactory.

    + +

    Waku v2 comprises further protocols related to ambient peer discovery. We shortly mention them for context, even though they are not strictly ambient peer discovery protocols.

    +

    Gossipsub Peer Exchange Protocol​

    +

    Gossipsub provides an integrated peer exchange mechanism which is also supported by Waku v2. Gossipsub peer exchange works in a push manner. Nodes send peer lists to peers they prune from the active mesh. -This pruning is part of the gossipsub peer management, blurring the boundaries of peer management and ambient peer discovery.

    We will investigate anonymity implications of this protocol and might disable it in favour of more anonymity-preserving protocols. +This pruning is part of the gossipsub peer management, blurring the boundaries of peer management and ambient peer discovery.

    +

    We will investigate anonymity implications of this protocol and might disable it in favour of more anonymity-preserving protocols. Sending a list of peers discloses information about the sending node. -We consider restricting these peer lists to cached peers that are currently not used in the active gossipsub mesh.

    Capability Negotiation​

    Some of the ambient peer discovery methods used by Waku2 will support capability discovery. +We consider restricting these peer lists to cached peers that are currently not used in the active gossipsub mesh.

    +

    Capability Negotiation​

    +

    Some of the ambient peer discovery methods used by Waku2 will support capability discovery. This allows to narrow down the set of retrieved peers to peers that support specific capabilities. -This is efficient because it avoids establishing connections to nodes that we are not interested in.

    However, the ambient discovery interface does not require capability discovery, which will lead to nodes having peers with unknown capabilities in their peer lists. -We work on a capability negotiation protocol which allows nodes to ask peers

    • for their complete list of capabilities, and
    • whether they support a specific capability

    We will investigate security implications, especially when sending full capability lists.

    NAT traversal​

    For NAT traversal, Waku v2 currently supports the port mapping protocols UPnP and NAT-PMP / PCP.

    In the future, we plan to add support for parts of ICE, e.g. STUN. +This is efficient because it avoids establishing connections to nodes that we are not interested in.

    +

    However, the ambient discovery interface does not require capability discovery, which will lead to nodes having peers with unknown capabilities in their peer lists. +We work on a capability negotiation protocol which allows nodes to ask peers

    +
      +
    • for their complete list of capabilities, and
      • +
    • whether they support a specific capability
      • +
    +

    We will investigate security implications, especially when sending full capability lists.

    +

    NAT traversal​

    +

    For NAT traversal, Waku v2 currently supports the port mapping protocols UPnP and NAT-PMP / PCP.

    +

    In the future, we plan to add support for parts of ICE, e.g. STUN. We do not plan to support TURN because TURN relays would introduce a centralized element. A modified decentralized version of TURN featuring incentivization might be an option in the future; -strong peers could offer a relay service similar to TURN.

    There are plans to integrate more NAT traversal into discv5, in which we might participate. -So far, the only traversal technique supported by discv5 is nodes receiving their external IP address in pong messages.

    While NAT traversal is very important, adding more NAT traversal techniques is not a priority at the moment. +strong peers could offer a relay service similar to TURN.

    +

    There are plans to integrate more NAT traversal into discv5, in which we might participate. +So far, the only traversal technique supported by discv5 is nodes receiving their external IP address in pong messages.

    +

    While NAT traversal is very important, adding more NAT traversal techniques is not a priority at the moment. Nodes behind restrictive symmetric NAT setups cannot be discovered, but they can still discover peers in less restrictive setups. While we wish to have as many nodes as possible to be discoverable via ambient peer discovery, two nodes behind a restrictive symmetric NAT can still exchange Waku v2 messages if they discovered a shared peer. -This is one of the nice resilience related properties of flooding based routing algorithms.

    For mobile nodes, which suffer from changing IP addresses and double NAT setups, we plan using the peer exchange protocol to ask peers for more peers. -Besides saving resources on resource restricted devices, this approach works as long as peers are in less restrictive environments.

    Conclusion and Future Prospects​

    Ambient peer discovery is an integral part of decentralized applications. It allows nodes to learn about peers in the network. +This is one of the nice resilience related properties of flooding based routing algorithms.

    +

    For mobile nodes, which suffer from changing IP addresses and double NAT setups, we plan using the peer exchange protocol to ask peers for more peers. +Besides saving resources on resource restricted devices, this approach works as long as peers are in less restrictive environments.

    +

    Conclusion and Future Prospects​

    +

    Ambient peer discovery is an integral part of decentralized applications. It allows nodes to learn about peers in the network. As of yet, Waku v2 supports DNS-based discovery and a slightly modified version of discv5. We are working on further protocols, including a peer exchange protocol that allows resource restricted nodes to ask stronger peers for peer lists. -Further, we are working on adding capability discovery to our ambient discovery protocols, allowing nodes to find peers with desired properties.

    These protocols can be combined in a modular way and allow Waku v2 nodes to build a strong and resilient mesh network, -even if some discovery methods are not available in a given situation.

    We will investigate security properties of these discovery mechanisms with a focus on privacy and anonymity in a future post on this research log. +Further, we are working on adding capability discovery to our ambient discovery protocols, allowing nodes to find peers with desired properties.

    +

    These protocols can be combined in a modular way and allow Waku v2 nodes to build a strong and resilient mesh network, +even if some discovery methods are not available in a given situation.

    +

    We will investigate security properties of these discovery mechanisms with a focus on privacy and anonymity in a future post on this research log. As an outlook we can already state that DHT approaches typically allow inferring information about the querying node. Further, sending peer lists allows inferring the position of a node within the mesh, and by extension information about the node. Waku v2 already provides some mitigation, because the mesh for transmitting actual messages, and the peer discovery network are separate. -To mitigate information leakage by transmitting peer lists, we plan to only reply with lists of peers that nodes do not use in their active meshes.

    References​

    - - +To mitigate information leakage by transmitting peer lists, we plan to only reply with lists of peers that nodes do not use in their active meshes.

    +
    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/wakuv2-noise/index.html b/rlog/wakuv2-noise/index.html index 890b9ba7..6b96287a 100644 --- a/rlog/wakuv2-noise/index.html +++ b/rlog/wakuv2-noise/index.html @@ -1,28 +1,189 @@ - + - -Noise handshakes as key-exchange mechanism for Waku | Vac Research + +Noise handshakes as key-exchange mechanism for Waku | Vac Research - - - + + + -
    -

    Noise handshakes as key-exchange mechanism for Waku

    by
    22 min read

    We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2.

    Introduction​

    In this post we will provide an overview of how Waku v2 users can adopt Noise handshakes to agree on cryptographic keys used to securely encrypt messages.

    This process belongs to the class of key-exchange mechanisms, consisting of all those protocols that, with different levels of complexity and security guarantees, allow two parties to publicly agree on a secret without letting anyone else know what this secret is.

    But why do we need key-exchange mechanisms in the first place?

    With the advent of public-key cryptography, it become possible to decouple encryption from decryption through use of two distinct cryptographic keys: one public, used to encrypt information and that can be made available to anyone, and one private (kept secret), which enables decryption of messages encrypted with its corresponding public key. The same does not happen in the case of symmetric encryption schemes where, instead, the same key is used for both encryption and decryption operations and hence cannot be publicly revealed as for public keys.

    In order to address specific application needs, many different public, symmetric and hybrid cryptographic schemes were designed: Waku v1 and Waku v2, which inherits part of their design from the Ethereum messaging protocol Whisper, provide support to both public-key primitives (ECIES, ECDSA) and symmetric primitives (AES-256-GCM, KECCAK-256), used to sign, hash, encrypt and decrypt exchanged messages.

    In principle, when communications employ public-key based encryption schemes (ECIES, in the case of Waku), there is no need for a key-agreement among parties: messages can be directly encrypted using the recipient's public-key before being sent over the network. However, public-key encryption and decryption primitives are usually very inefficient in processing large amount of data, and this may constitute a bottleneck for many of today's applications. Symmetric encryption schemes such as AES-256-GCM, on the other hand, are much more efficient, but the encryption/decryption key needs to be shared among users beforehand any encrypted messages is exchanged.

    To counter the downsides given by each of these two approaches while taking advantage of their strengths, hybrid constructions were designed. In these, public-key primitives are employed to securely agree on a secret key which, in turn, is used with a symmetric cipher for encrypting messages. In other words, such constructions specify a (public-key based) key-agreement mechanism!

    Waku, up to payload version 1, does not implement nor recommend any protocol for exchanging symmetric ciphers' keys, leaving such task to the application layer. It is important to note that the kind of key-agreement employed has a direct impact on the security properties that can be granted on later encrypted messages, while security requirements usually depend on the specific application for which encryption is needed in the first place.

    In this regard, Status, which builds on top of Waku, implements a custom version of the X3DH key-agreement protocol, in order to allow users to instantiate end-to-end encrypted communication channels. However, although such a solution is optimal when applied to (distributed) E2E encrypted chats, it is not flexible enough to fit or simplify the variety of applications Waku aims to address. -Hence, proposing and implementing one or few key-agreements which provide certain (presumably strong) security guarantees, would inevitably degrade performances of all those applications for which, given their security requirements, more tailored and efficient key-exchange mechanisms can be employed.

    Guided by different examples, in the following sections we will overview Noise, a protocol framework we are currently integrating in Waku, for building secure key-agreements between two parties. One of the great advantage of using Noise is that it is possible to add support to new key-exchanges by just specifying users' actions from a predefined list, requiring none to minimal modifications to existing implementations. Furthermore, Noise provides a framework to systematically analyze protocols' security properties and the corresponding attacker threat models. This allows not only to easily design new key-agreements eventually optimized for specific applications we want to address, but also to easily analyze or even formally verify any of such custom protocol!

    We believe that with its enormous flexibility and features, Noise represents a perfect candidate for bringing key-exchange mechanisms in Waku.

    The Diffie-Hellman Key-exchange​

    The formalization of modern public-key cryptography started with the pioneering work of Whitefield Diffie and Martin Hellman, who detailed one of the earliest known key-agreement protocols: the famous Diffie-Hellman Key-Exchange.

    Diffie-Hellman (DH) key-exchange is largely used today and represents the main cryptographic building block on which Noise handshakes' security is based.

    In turn, the security of DH is based on a mathematical problem called discrete logarithm which is believed to be hard when the agreement is practically instantiated using certain elliptic curves EE defined over finite fields Fp\mathbb{F}_p.

    Informally, a DH exchange between Alice and Bob proceeds as follows:

    • Alice picks a secret scalar sA∈Fps_A\in\mathbb{F}_p and computes, using the underlying curve's arithmetic, the point PA=sA⋅P∈E(Fp)P_A = s_A\cdot P\in E(\mathbb{F}_p) for a certain pre-agreed public generator PP of the elliptic curve E(Fp)E(\mathbb{F}_p). She then sends PAP_A to Bob.
    • Similarly, Bob picks a secret scalar sB∈Fps_B\in\mathbb{F}_p, computes PB=sB⋅P∈E(Fp)P_B = s_B\cdot P\in E(\mathbb{F}_p) and sends PBP_B to Alice.
    • By commutativity of scalar multiplication, both Alice and Bob can now compute the point PAB=sAsB⋅PP_{AB} = s_As_B\cdot P, using the elliptic curve point received from the other party and their secret scalar.

    The assumed hardness of computing discrete logarithms in the elliptic curve, ensures that it is not possible to compute sAs_A or sBs_B from PAP_A and PBP_B, respectively. Another security assumption (named Computational Diffie-Hellman assumption) ensures that it is not possible to compute PABP_{AB} from PP, PAP_A and PBP_B. Hence the point PABP_{AB} shared by Alice and Bob at the end of the above protocol cannot be efficiently computed by an attacker intercepting PAP_A and PBP_B, and can then be used to generate a secret to be later employed, for example, as a symmetric encryption key.

    On a side note, this protocol shows the interplay between two components typical to public-key based schemes: the scalars sAs_A and sBs_B can be seen as private keys associated to the public keys PAP_A and PBP_B, respectively, which allow Alice and Bob only to compute the shared secret point PABP_{AB}.

    Ephemeral and Static Public Keys​

    Although we assumed that it is practically impossible for an attacker to compute the randomly picked secret scalar from the corresponding public elliptic curve point, it may happen that such scalar gets compromised or can be guessed due to a faulty employed random number generator. In such cases, an attacker will be able to recover the final shared secret and all encryption keys eventually derived from that, with clear catastrophic consequences for the privacy of exchanged messages.

    To mitigate such issues, multiple DH operations can be combined using two different types of exchanged elliptic curve points or, better, public keys: ephemeral keys, that is random keys used only once in a DH operation, and long-term static keys, used mainly for authentication purposes since employed multiple times.

    Just to provide an example, let us suppose Alice and Bob perform the following custom DH-based key-exchange protocol:

    • Alice generates an ephemeral key EA=eA⋅PE_A=e_A\cdot P by picking a random scalar eAe_A and sends EAE_A to Bob;
    • Similarly, Bob generates an ephemeral key EB=eB⋅PE_B=e_B\cdot P and sends EBE_B to Alice;
    • Alice and Bob computes EAB=eAeB⋅PE_{AB} = e_Ae_B \cdot P and from it derive a secret encryption key kk.
    • Bob sends to Alice his static key SB=sB⋅PS_B = s_B\cdot P encrypted with kk.
    • Alice encrypts with kk her static key SA=sA⋅PS_A = s_A\cdot P and sends it to Bob.
    • Alice and Bob decrypt the received static keys, compute the secret SAB=sAsB⋅PS_{AB} = s_As_B \cdot P and use it together with EABE_{AB} to derive a new encryption key k~\tilde{k} to be later used with a symmetric cipher.

    In this protocol, if Alice's and/or Bob's static keys get compromised, it would not possible to derive the final secret key k~\tilde{k}, since at least one ephemeral key among EAE_A and EBE_B has to be compromised too in order to recover the secret EABE_{AB}. Furthermore, since Alice's and Bob's long-term static keys are encrypted, an attacker intercepting exchanged (encrypted) public keys will not be able to link such communication to Alice or Bob, unless one of the ephemeral key is compromised (and, even in such case, none of the messages encrypted under the key k~\tilde{k} can be decrypted).

    The Noise Protocol Framework​

    In previous section we gave a small intuition on how multiple DH operations over ephemeral and static users' public keys can be combined to create different key-exchange protocols.

    The Noise Protocol Framework, defines various rules for building custom key-exchange protocols while allowing easy analysis of the security properties and threat models provided given the type and order of the DH operations employed.

    In Noise terminology, a key-agreement or Noise protocol consists of one or more Noise handshakes. During a Noise handshake, Alice and Bob exchange multiple (handshake) messages containing their ephemeral keys and/or static keys. These public keys are then used to perform a handshake-dependent sequence of Diffie-Hellman operations, whose results are all hashed into a shared secret key. Similarly as we have seen above, after a handshake is complete, each party will use the derived secret key to send and receive authenticated encrypted data by employing a symmetric cipher.

    Depending on the handshake pattern adopted, different security guarantees can be provided on messages encrypted using a handshake-derived key.

    The Noise handshakes we support in Waku all provide the following security properties:

    • Confidentiality: the adversary should not be able to learn what data is being sent between Alice and Bob.
    • Strong forward secrecy: an active adversary cannot decrypt messages nor infer any information on the employed encryption key, even in the case he has access to Alice's and Bob's long-term private keys (during or after their communication).
    • Authenticity: the adversary should not be able to cause either Alice or Bob to accept messages coming from a party different than their original senders.
    • Integrity: the adversary should not be able to cause Alice or Bob to accept data that has been tampered with.
    • Identity-hiding: once a secure communication channel is established, a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient by knowing their long-term static keys.

    We refer to Noise specification for more formal security definitions and precise threat models relative to Waku supported Noise Handshake patterns.

    Message patterns​

    Noise handshakes involving DH operations over ephemeral and static keys can be succinctly sketched using the following set of handshake message tokens: e,s,ee,se,es,ss.

    Tokens employing single letters denote (the type of) users' public keys: e refers to randomly generated ephemeral key(s), while s indicates the users' long-term static key(s).

    Two letters tokens, instead, denotes DH operations over the two users' public keys the token refers to, given that the left token letter refers to the handshake initiator's public key, while the right token letter indicates the used responder's public key. Thus, if Alice started a handshake with Bob, the es token will shortly represent a DH operation among Alice's ephemeral key e and Bob's static key s.

    Since, in order to perform any DH operations users need to share (or pre-share) the corresponding public keys, Noise compactly represents messages' exchanges using the two direction -> and <-, where the -> denotes a message (arbitrary and/or DH public key) from the initiator to the responder, while <- the opposite.

    Hence a message pattern consisting of a direction and one or multiple tokens such as <- e, s, es has to be interpreted one token at a time: in this example, the responder is sending his ephemeral and static key to the initiator and is then executing a DH operation over the initiator's ephemeral key e (shared in a previously exchanged message pattern) and his static key s. On the other hand, such message indicates also that the initiator received the responder's ephemeral and static keys e and s, respectively, and performed a DH operation over his ephemeral key and the responder's just received static key s. In this way, both parties will be able to derive at the end of each message pattern processed the same shared secret, which is eventually used to update any derived symmetric encryption keys computed so far.

    In some cases, DH public keys employed in a handshake are pre-shared before the handshake itself starts. In order to chronologically separate exchanged keys and DH operations performed before and during a handshake, Noise employs the ... delimiter.

    For example, the following message patterns

    <- e
    ...
    -> e, ee

    indicates that the initiator knew the responder's ephemeral key before he sends his own ephemeral key and executes a DH operation between both parties ephemeral keys (similarly, the responder receives the initiator's ephemeral key and does a ee DH operation).

    At this point it should be clear how such notation is able to compactly represent a large variety of DH based key-agreements. Nevertheless, we can easily define additional tokens and processing rules in order to address specific applications and security requirements, such as the psk token used to process arbitrary pre-shared key material.

    As an example of Noise flexibility, the custom protocol we detailed above can be shortly represented as (Alice is on the left):

    -> e
    <- e, ee, s
    -> s, ss

    where after each DH operation an encryption key is derived (along with the secrets computed by all previously executed DH operations) in order to encrypt/decrypt any subsequent sent/received message.

    Another example is given by the possibility to replicate within Noise the well established Signal's X3DH key-agreement protocols, thus making the latter a general framework to design and study security of many practical and widespread DH-based key-exchange protocols.

    The Noise State Objects​

    We mentioned multiple times that parties derive an encryption key each time they perform a DH operation, but how does this work in more details?

    Noise defines three state object: a Handshake State, a Symmetric State and a Cipher State, each encapsulated into each other and instantiated during the execution of a handshake.

    The Handshake State object stores the user's and other party's received ephemeral and static keys (if any) and embeds a Symmetric State object.

    The Symmetric State, instead, stores a handshake hash value h, iteratively updated with any message read/received and DH secret computed, and a chaining key ck, updated using a key derivation function every time a DH secret is computed. This object further embeds a Cipher State.

    Lastly, the Cipher State stores a symmetric encryption k key and a counter n used to encrypt and decrypt messages exchanged during the handshake (not only static keys, but also arbitrary payloads). These key and counter are refreshed every time the chaining key is updated.

    While processing each handshake's message pattern token, all these objects are updated according to some specific processing rules which employ a combination of public-key primitives, hash and key-derivation functions and symmetric ciphers. It is important to note, however, that at the end of each processed message pattern, the two users will share the same Symmetric and Cipher State embedded in their respective Handshake States.

    Once a handshake is complete, users derive two new Cipher States and can then discard the Handshake State object (and, thus, the embedded Symmetric State and Cipher State objects) -employed during the handshake.

    These two Cipher states are used to encrypt and decrypt all outbound and inbound after-handshake messages, respectively, and only to these will be granted the confidentiality, authenticity, integrity and identity-hiding properties we detailed above.

    For more details on processing rules, we refer to Noise specifications.

    Supported Noise Handshakes in Waku​

    The Noise handshakes we provided support to in Waku address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:

    • Alice and Bob know each others' static key.
    • Alice knows Bob's static key;
    • Alice and Bob share no key material and they don't know each others' static key.
    • Alice and Bob share some key material, but they don't know each others' static key.

    The possibility to have handshakes based on the reciprocal knowledge parties have of each other, allows designing Noise handshakes that can quickly reach the desired level of security on exchanged encrypted messages while keeping the number of interactions between Alice and Bob minimum.

    Nonetheless, due to the pure token-based nature of handshake processing rules, implementations can easily add support to any custom handshake pattern with minor modifications, in case more specific application use-cases need to be addressed.

    On a side note, we already mentioned that identity-hiding properties can be guaranteed against a passive attacker that only reads the communication occurring between Alice and Bob. However, an active attacker who compromised one party's static key and actively interferes with the parties' exchanged messages, may lower the identity-hiding security guarantees provided by some handshake patterns. In our security model we exclude such adversary, but, for completeness, in the following we report a summary of possible de-anonymization attacks that can be performed by such an active attacker.

    For more details on supported handshakes and on how these are implemented in Waku, we refer to WAKU2-NOISE RFC.

    The K1K1 Handshake​

    If Alice and Bob know each others' static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a K1K1 handshake. In Noise notation (Alice is on the left) this can be sketched as:

     K1K1:
        ->  s
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  se

    We note that here only ephemeral keys are exchanged. This handshake is useful in case Alice needs to instantiate a new separate encrypted communication channel with Bob, e.g. opening multiple parallel connections, file transfers, etc.

    Security considerations on identity-hiding (active attacker): no static key is transmitted, but an active attacker impersonating Alice can check candidates for Bob's static key.

    The XK1 Handshake​

    Here, Alice knows how to initiate a communication with Bob and she knows his public static key: such discovery can be achieved, for example, through a publicly accessible register of users' static keys, smart contracts, or through a previous public/private advertisement of Bob's static key.

    A Noise handshake pattern that suits this scenario is XK1:

     XK1:
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  s, se

    Within this handshake, Alice and Bob reciprocally authenticate their static keys s using ephemeral keys e. We note that while Bob's static key is assumed to be known to Alice (and hence is not transmitted), Alice's static key is sent to Bob encrypted with a key derived from both parties ephemeral keys and Bob's static key.

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob's static key against recorded/accepted exchanged handshake messages.

    The XX and XXpsk0 Handshakes​

    If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an XX handshake, where each party tranXmits to the other its own static key.

    The handshake goes as follows:

     XX:
        ->  e
        <-  e, ee, s, es
        ->  s, se

    We note that the main difference with XK1 is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange.

    This handshake can be slightly changed in case both Alice and Bob pre-shares some secret psk which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called XXpsk0, goes as follow:

     XXpsk0:
        ->  psk, e
        <-  e, ee, s, es
        ->  s, se

    The main difference with XX is that Alice's and Bob's static keys, when transmitted, would be encrypted with a key derived from psk as well.

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party for both XX and XXpsk0 handshakes. In XX, Bob's static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In XXpsk0, instead, Bob's secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret psk but not through any static key), provided that psk was not previously compromised (in such case identity-hiding properties provided by the XX handshake applies).

    Session Management and Multi-Device Support​

    When two users complete a Noise handshake, an encryption/decryption session - or Noise session - consisting of two Cipher States is instantiated.

    By identifying Noise session with a session-id derived from the handshake's cryptographic material, we can take advantage of the PubSub/GossipSub protocols used by Waku for relaying messages in order to manage instantiated Noise sessions.

    The core idea is to exchange after-handshake messages (encrypted with a Cipher State specific to the Noise session), over a content topic derived from the (secret) session-id the corresponding session refers to.

    This allows to decouple the handshaking phase from the actual encrypted communication, thus improving users' identity-hiding capabilities.

    Furthermore, by publicly revealing a value derived from session-id on the corresponding session content topic, a Noise session can be marked as stale, enabling peers to save resources by discarding any eventually stored message sent to such content topic.

    One relevant aspect in today's applications is the possibility for users to employ different devices in their communications. In some cases, this is non-trivial to achieve since, for example, encrypted messages might be required to be synced on different devices which do not necessarily share the necessary key material for decryption and may be temporarily offline.

    We address this by requiring each user's device to instantiate multiple Noise sessions either with all user's other devices which, in turn, all together share a Noise session with the other party, or by directly instantiating a Noise session with all other party's devices.

    We named these two approaches N11MN11M and NMNM, respectively, which are in turn loosely based on the paper “Multi-Device for Signal” and Signal’s Sesame Algorithm.

    Informally, in the N11MN11M session management scheme, once the first Noise session between any of Alice’s and Bob’s device is instantiated, its session information is securely propagated to all other devices using previously instantiated Noise sessions. Hence, all devices are able to send and receive new messages on the content topic associated to such session.

    In the NMNM session management scheme, instead, all pairs of Alice's and Bob's devices have a distinct Noise session: a message is then sent from the currently-in-use sender’s device to all recipient’s devices, by properly encrypting and sending it to the content topics of each corresponding Noise session. If sent messages should be available on all sender’s devices as well, we require each pair of sender’s devices to instantiate a Noise session used for syncing purposes.

    For more technical details on how Noise sessions are instantiated and managed within these two mechanisms and the different trade-offs provided by the latter, we refer to WAKU2-NOISE-SESSIONS.

    Conclusions​

    In this post we provided an overview of Noise, a protocol framework for designing Diffie-Hellman based key-exchange mechanisms allowing systematic security and threat model analysis.

    The flexibility provided by Noise components allows not only to fully replicate with same security guarantees well established key-exchange primitives such as X3DH, currently employed by Status TRANSPORT-SECURITY, but enables also optimizations based on the reciprocal knowledge parties have of each other while allowing easier protocols' security analysis and (formal) verification.

    Furthermore, different handshakes can be combined and executed one after each other, a particularly useful feature to authenticate multiple static keys employed by different applications but also to ease keys revocation.

    The possibility to manage Noise sessions over multiple devices and the fact that handshakes can be concretely instantiated using modern, fast and secure cryptographic primitives such as ChaChaPoly and BLAKE2b, make Noise one of the best candidates for efficiently and securely address the many different needs of applications built on top of Waku requiring key-agreement.

    Future steps​

    The available implementation of Noise in nwaku, although mostly complete, is still in its testing phase. As future steps we would like to:

    • have an extensively tested and robust Noise implementation;
    • formalize, implement and test performances of the two proposed N11MN11M and NMNM session management mechanisms and their suitability for common use-case scenarios;
    • provide Waku network nodes a native protocol to readily support key-exchanges, strongly-encrypted communication and multi-device session management mechanisms with none-to-little interaction besides applications' connection requests.

    References​

    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Noise handshakes as key-exchange mechanism for Waku

    by
    22 min read

    We provide an overview of the Noise Protocol Framework as a tool to design efficient and secure key-exchange mechanisms in Waku2.

    + +

    Introduction​

    +

    In this post we will provide an overview of how Waku v2 users can adopt Noise handshakes to agree on cryptographic keys used to securely encrypt messages.

    +

    This process belongs to the class of key-exchange mechanisms, consisting of all those protocols that, with different levels of complexity and security guarantees, allow two parties to publicly agree on a secret without letting anyone else know what this secret is.

    +

    But why do we need key-exchange mechanisms in the first place?

    +

    With the advent of public-key cryptography, it become possible to decouple encryption from decryption through use of two distinct cryptographic keys: one public, used to encrypt information and that can be made available to anyone, and one private (kept secret), which enables decryption of messages encrypted with its corresponding public key. The same does not happen in the case of symmetric encryption schemes where, instead, the same key is used for both encryption and decryption operations and hence cannot be publicly revealed as for public keys.

    +

    In order to address specific application needs, many different public, symmetric and hybrid cryptographic schemes were designed: Waku v1 and Waku v2, which inherits part of their design from the Ethereum messaging protocol Whisper, provide support to both public-key primitives (ECIES, ECDSA) and symmetric primitives (AES-256-GCM, KECCAK-256), used to sign, hash, encrypt and decrypt exchanged messages.

    +

    In principle, when communications employ public-key based encryption schemes (ECIES, in the case of Waku), there is no need for a key-agreement among parties: messages can be directly encrypted using the recipient's public-key before being sent over the network. However, public-key encryption and decryption primitives are usually very inefficient in processing large amount of data, and this may constitute a bottleneck for many of today's applications. Symmetric encryption schemes such as AES-256-GCM, on the other hand, are much more efficient, but the encryption/decryption key needs to be shared among users beforehand any encrypted messages is exchanged.

    +

    To counter the downsides given by each of these two approaches while taking advantage of their strengths, hybrid constructions were designed. In these, public-key primitives are employed to securely agree on a secret key which, in turn, is used with a symmetric cipher for encrypting messages. In other words, such constructions specify a (public-key based) key-agreement mechanism!

    +

    Waku, up to payload version 1, does not implement nor recommend any protocol for exchanging symmetric ciphers' keys, leaving such task to the application layer. It is important to note that the kind of key-agreement employed has a direct impact on the security properties that can be granted on later encrypted messages, while security requirements usually depend on the specific application for which encryption is needed in the first place.

    +

    In this regard, Status, which builds on top of Waku, implements a custom version of the X3DH key-agreement protocol, in order to allow users to instantiate end-to-end encrypted communication channels. However, although such a solution is optimal when applied to (distributed) E2E encrypted chats, it is not flexible enough to fit or simplify the variety of applications Waku aims to address. +Hence, proposing and implementing one or few key-agreements which provide certain (presumably strong) security guarantees, would inevitably degrade performances of all those applications for which, given their security requirements, more tailored and efficient key-exchange mechanisms can be employed.

    +

    Guided by different examples, in the following sections we will overview Noise, a protocol framework we are currently integrating in Waku, for building secure key-agreements between two parties. One of the great advantage of using Noise is that it is possible to add support to new key-exchanges by just specifying users' actions from a predefined list, requiring none to minimal modifications to existing implementations. Furthermore, Noise provides a framework to systematically analyze protocols' security properties and the corresponding attacker threat models. This allows not only to easily design new key-agreements eventually optimized for specific applications we want to address, but also to easily analyze or even formally verify any of such custom protocol!

    +

    We believe that with its enormous flexibility and features, Noise represents a perfect candidate for bringing key-exchange mechanisms in Waku.

    +

    The Diffie-Hellman Key-exchange​

    +

    The formalization of modern public-key cryptography started with the pioneering work of Whitefield Diffie and Martin Hellman, who detailed one of the earliest known key-agreement protocols: the famous Diffie-Hellman Key-Exchange.

    +

    Diffie-Hellman (DH) key-exchange is largely used today and represents the main cryptographic building block on which Noise handshakes' security is based.

    +

    In turn, the security of DH is based on a mathematical problem called discrete logarithm which is believed to be hard when the agreement is practically instantiated using certain elliptic curves EE defined over finite fields Fp\mathbb{F}_p.

    +

    Informally, a DH exchange between Alice and Bob proceeds as follows:

    +
      +
    • Alice picks a secret scalar sA∈Fps_A\in\mathbb{F}_p and computes, using the underlying curve's arithmetic, the point PA=sA⋅P∈E(Fp)P_A = s_A\cdot P\in E(\mathbb{F}_p) for a certain pre-agreed public generator PP of the elliptic curve E(Fp)E(\mathbb{F}_p). She then sends PAP_A to Bob.
      • +
    • Similarly, Bob picks a secret scalar sB∈Fps_B\in\mathbb{F}_p, computes PB=sB⋅P∈E(Fp)P_B = s_B\cdot P\in E(\mathbb{F}_p) and sends PBP_B to Alice.
      • +
    • By commutativity of scalar multiplication, both Alice and Bob can now compute the point PAB=sAsB⋅PP_{AB} = s_As_B\cdot P, using the elliptic curve point received from the other party and their secret scalar.
      • +
    +

    The assumed hardness of computing discrete logarithms in the elliptic curve, ensures that it is not possible to compute sAs_A or sBs_B from PAP_A and PBP_B, respectively. Another security assumption (named Computational Diffie-Hellman assumption) ensures that it is not possible to compute PABP_{AB} from PP, PAP_A and PBP_B. Hence the point PABP_{AB} shared by Alice and Bob at the end of the above protocol cannot be efficiently computed by an attacker intercepting PAP_A and PBP_B, and can then be used to generate a secret to be later employed, for example, as a symmetric encryption key.

    +

    On a side note, this protocol shows the interplay between two components typical to public-key based schemes: the scalars sAs_A and sBs_B can be seen as private keys associated to the public keys PAP_A and PBP_B, respectively, which allow Alice and Bob only to compute the shared secret point PABP_{AB}.

    +

    Ephemeral and Static Public Keys​

    +

    Although we assumed that it is practically impossible for an attacker to compute the randomly picked secret scalar from the corresponding public elliptic curve point, it may happen that such scalar gets compromised or can be guessed due to a faulty employed random number generator. In such cases, an attacker will be able to recover the final shared secret and all encryption keys eventually derived from that, with clear catastrophic consequences for the privacy of exchanged messages.

    +

    To mitigate such issues, multiple DH operations can be combined using two different types of exchanged elliptic curve points or, better, public keys: ephemeral keys, that is random keys used only once in a DH operation, and long-term static keys, used mainly for authentication purposes since employed multiple times.

    +

    Just to provide an example, let us suppose Alice and Bob perform the following custom DH-based key-exchange protocol:

    +
      +
    • Alice generates an ephemeral key EA=eA⋅PE_A=e_A\cdot P by picking a random scalar eAe_A and sends EAE_A to Bob;
      • +
    • Similarly, Bob generates an ephemeral key EB=eB⋅PE_B=e_B\cdot P and sends EBE_B to Alice;
      • +
    • Alice and Bob computes EAB=eAeB⋅PE_{AB} = e_Ae_B \cdot P and from it derive a secret encryption key kk.
      • +
    • Bob sends to Alice his static key SB=sB⋅PS_B = s_B\cdot P encrypted with kk.
      • +
    • Alice encrypts with kk her static key SA=sA⋅PS_A = s_A\cdot P and sends it to Bob.
      • +
    • Alice and Bob decrypt the received static keys, compute the secret SAB=sAsB⋅PS_{AB} = s_As_B \cdot P and use it together with EABE_{AB} to derive a new encryption key k~\tilde{k} to be later used with a symmetric cipher.
      • +
    +

    In this protocol, if Alice's and/or Bob's static keys get compromised, it would not possible to derive the final secret key k~\tilde{k}, since at least one ephemeral key among EAE_A and EBE_B has to be compromised too in order to recover the secret EABE_{AB}. Furthermore, since Alice's and Bob's long-term static keys are encrypted, an attacker intercepting exchanged (encrypted) public keys will not be able to link such communication to Alice or Bob, unless one of the ephemeral key is compromised (and, even in such case, none of the messages encrypted under the key k~\tilde{k} can be decrypted).

    +

    The Noise Protocol Framework​

    +

    In previous section we gave a small intuition on how multiple DH operations over ephemeral and static users' public keys can be combined to create different key-exchange protocols.

    +

    The Noise Protocol Framework, defines various rules for building custom key-exchange protocols while allowing easy analysis of the security properties and threat models provided given the type and order of the DH operations employed.

    +

    In Noise terminology, a key-agreement or Noise protocol consists of one or more Noise handshakes. During a Noise handshake, Alice and Bob exchange multiple (handshake) messages containing their ephemeral keys and/or static keys. These public keys are then used to perform a handshake-dependent sequence of Diffie-Hellman operations, whose results are all hashed into a shared secret key. Similarly as we have seen above, after a handshake is complete, each party will use the derived secret key to send and receive authenticated encrypted data by employing a symmetric cipher.

    +

    Depending on the handshake pattern adopted, different security guarantees can be provided on messages encrypted using a handshake-derived key.

    +

    The Noise handshakes we support in Waku all provide the following security properties:

    +
      +
    • Confidentiality: the adversary should not be able to learn what data is being sent between Alice and Bob.
      • +
    • Strong forward secrecy: an active adversary cannot decrypt messages nor infer any information on the employed encryption key, even in the case he has access to Alice's and Bob's long-term private keys (during or after their communication).
      • +
    • Authenticity: the adversary should not be able to cause either Alice or Bob to accept messages coming from a party different than their original senders.
      • +
    • Integrity: the adversary should not be able to cause Alice or Bob to accept data that has been tampered with.
      • +
    • Identity-hiding: once a secure communication channel is established, a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient by knowing their long-term static keys.
      • +
    +

    We refer to Noise specification for more formal security definitions and precise threat models relative to Waku supported Noise Handshake patterns.

    +

    Message patterns​

    +

    Noise handshakes involving DH operations over ephemeral and static keys can be succinctly sketched using the following set of handshake message tokens: e,s,ee,se,es,ss.

    +

    Tokens employing single letters denote (the type of) users' public keys: e refers to randomly generated ephemeral key(s), while s indicates the users' long-term static key(s).

    +

    Two letters tokens, instead, denotes DH operations over the two users' public keys the token refers to, given that the left token letter refers to the handshake initiator's public key, while the right token letter indicates the used responder's public key. Thus, if Alice started a handshake with Bob, the es token will shortly represent a DH operation among Alice's ephemeral key e and Bob's static key s.

    +

    Since, in order to perform any DH operations users need to share (or pre-share) the corresponding public keys, Noise compactly represents messages' exchanges using the two direction -> and <-, where the -> denotes a message (arbitrary and/or DH public key) from the initiator to the responder, while <- the opposite.

    +

    Hence a message pattern consisting of a direction and one or multiple tokens such as <- e, s, es has to be interpreted one token at a time: in this example, the responder is sending his ephemeral and static key to the initiator and is then executing a DH operation over the initiator's ephemeral key e (shared in a previously exchanged message pattern) and his static key s. On the other hand, such message indicates also that the initiator received the responder's ephemeral and static keys e and s, respectively, and performed a DH operation over his ephemeral key and the responder's just received static key s. In this way, both parties will be able to derive at the end of each message pattern processed the same shared secret, which is eventually used to update any derived symmetric encryption keys computed so far.

    +

    In some cases, DH public keys employed in a handshake are pre-shared before the handshake itself starts. In order to chronologically separate exchanged keys and DH operations performed before and during a handshake, Noise employs the ... delimiter.

    +

    For example, the following message patterns

    +
    <- e
    ...
    -> e, ee
    +

    indicates that the initiator knew the responder's ephemeral key before he sends his own ephemeral key and executes a DH operation between both parties ephemeral keys (similarly, the responder receives the initiator's ephemeral key and does a ee DH operation).

    +

    At this point it should be clear how such notation is able to compactly represent a large variety of DH based key-agreements. Nevertheless, we can easily define additional tokens and processing rules in order to address specific applications and security requirements, such as the psk token used to process arbitrary pre-shared key material.

    +

    As an example of Noise flexibility, the custom protocol we detailed above can be shortly represented as (Alice is on the left):

    +
    -> e
    <- e, ee, s
    -> s, ss
    +

    where after each DH operation an encryption key is derived (along with the secrets computed by all previously executed DH operations) in order to encrypt/decrypt any subsequent sent/received message.

    +

    Another example is given by the possibility to replicate within Noise the well established Signal's X3DH key-agreement protocols, thus making the latter a general framework to design and study security of many practical and widespread DH-based key-exchange protocols.

    +

    The Noise State Objects​

    +

    We mentioned multiple times that parties derive an encryption key each time they perform a DH operation, but how does this work in more details?

    +

    Noise defines three state object: a Handshake State, a Symmetric State and a Cipher State, each encapsulated into each other and instantiated during the execution of a handshake.

    +

    The Handshake State object stores the user's and other party's received ephemeral and static keys (if any) and embeds a Symmetric State object.

    +

    The Symmetric State, instead, stores a handshake hash value h, iteratively updated with any message read/received and DH secret computed, and a chaining key ck, updated using a key derivation function every time a DH secret is computed. This object further embeds a Cipher State.

    +

    Lastly, the Cipher State stores a symmetric encryption k key and a counter n used to encrypt and decrypt messages exchanged during the handshake (not only static keys, but also arbitrary payloads). These key and counter are refreshed every time the chaining key is updated.

    +

    While processing each handshake's message pattern token, all these objects are updated according to some specific processing rules which employ a combination of public-key primitives, hash and key-derivation functions and symmetric ciphers. It is important to note, however, that at the end of each processed message pattern, the two users will share the same Symmetric and Cipher State embedded in their respective Handshake States.

    +

    Once a handshake is complete, users derive two new Cipher States and can then discard the Handshake State object (and, thus, the embedded Symmetric State and Cipher State objects) +employed during the handshake.

    +

    These two Cipher states are used to encrypt and decrypt all outbound and inbound after-handshake messages, respectively, and only to these will be granted the confidentiality, authenticity, integrity and identity-hiding properties we detailed above.

    +

    For more details on processing rules, we refer to Noise specifications.

    +

    Supported Noise Handshakes in Waku​

    +

    The Noise handshakes we provided support to in Waku address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:

    +
      +
    • Alice and Bob know each others' static key.
      • +
    • Alice knows Bob's static key;
      • +
    • Alice and Bob share no key material and they don't know each others' static key.
      • +
    • Alice and Bob share some key material, but they don't know each others' static key.
      • +
    +

    The possibility to have handshakes based on the reciprocal knowledge parties have of each other, allows designing Noise handshakes that can quickly reach the desired level of security on exchanged encrypted messages while keeping the number of interactions between Alice and Bob minimum.

    +

    Nonetheless, due to the pure token-based nature of handshake processing rules, implementations can easily add support to any custom handshake pattern with minor modifications, in case more specific application use-cases need to be addressed.

    +

    On a side note, we already mentioned that identity-hiding properties can be guaranteed against a passive attacker that only reads the communication occurring between Alice and Bob. However, an active attacker who compromised one party's static key and actively interferes with the parties' exchanged messages, may lower the identity-hiding security guarantees provided by some handshake patterns. In our security model we exclude such adversary, but, for completeness, in the following we report a summary of possible de-anonymization attacks that can be performed by such an active attacker.

    +

    For more details on supported handshakes and on how these are implemented in Waku, we refer to WAKU2-NOISE RFC.

    +

    The K1K1 Handshake​

    +

    If Alice and Bob know each others' static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a K1K1 handshake. In Noise notation (Alice is on the left) this can be sketched as:

    +
     K1K1:
        ->  s
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  se
    +

    We note that here only ephemeral keys are exchanged. This handshake is useful in case Alice needs to instantiate a new separate encrypted communication channel with Bob, e.g. opening multiple parallel connections, file transfers, etc.

    +

    Security considerations on identity-hiding (active attacker): no static key is transmitted, but an active attacker impersonating Alice can check candidates for Bob's static key.

    +

    The XK1 Handshake​

    +

    Here, Alice knows how to initiate a communication with Bob and she knows his public static key: such discovery can be achieved, for example, through a publicly accessible register of users' static keys, smart contracts, or through a previous public/private advertisement of Bob's static key.

    +

    A Noise handshake pattern that suits this scenario is XK1:

    +
     XK1:
        <-  s
           ...
        ->  e
        <-  e, ee, es
        ->  s, se
    +

    Within this handshake, Alice and Bob reciprocally authenticate their static keys s using ephemeral keys e. We note that while Bob's static key is assumed to be known to Alice (and hence is not transmitted), Alice's static key is sent to Bob encrypted with a key derived from both parties ephemeral keys and Bob's static key.

    +

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob's static key against recorded/accepted exchanged handshake messages.

    +

    The XX and XXpsk0 Handshakes​

    +

    If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an XX handshake, where each party tranXmits to the other its own static key.

    +

    The handshake goes as follows:

    +
     XX:
        ->  e
        <-  e, ee, s, es
        ->  s, se
    +

    We note that the main difference with XK1 is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange.

    +

    This handshake can be slightly changed in case both Alice and Bob pre-shares some secret psk which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called XXpsk0, goes as follow:

    +
     XXpsk0:
        ->  psk, e
        <-  e, ee, s, es
        ->  s, se
    +

    The main difference with XX is that Alice's and Bob's static keys, when transmitted, would be encrypted with a key derived from psk as well.

    +

    Security considerations on identity-hiding (active attacker): Alice's static key is encrypted with forward secrecy to an authenticated party for both XX and XXpsk0 handshakes. In XX, Bob's static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In XXpsk0, instead, Bob's secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret psk but not through any static key), provided that psk was not previously compromised (in such case identity-hiding properties provided by the XX handshake applies).

    +

    Session Management and Multi-Device Support​

    +

    When two users complete a Noise handshake, an encryption/decryption session - or Noise session - consisting of two Cipher States is instantiated.

    +

    By identifying Noise session with a session-id derived from the handshake's cryptographic material, we can take advantage of the PubSub/GossipSub protocols used by Waku for relaying messages in order to manage instantiated Noise sessions.

    +

    The core idea is to exchange after-handshake messages (encrypted with a Cipher State specific to the Noise session), over a content topic derived from the (secret) session-id the corresponding session refers to.

    +

    This allows to decouple the handshaking phase from the actual encrypted communication, thus improving users' identity-hiding capabilities.

    +

    Furthermore, by publicly revealing a value derived from session-id on the corresponding session content topic, a Noise session can be marked as stale, enabling peers to save resources by discarding any eventually stored message sent to such content topic.

    +

    One relevant aspect in today's applications is the possibility for users to employ different devices in their communications. In some cases, this is non-trivial to achieve since, for example, encrypted messages might be required to be synced on different devices which do not necessarily share the necessary key material for decryption and may be temporarily offline.

    +

    We address this by requiring each user's device to instantiate multiple Noise sessions either with all user's other devices which, in turn, all together share a Noise session with the other party, or by directly instantiating a Noise session with all other party's devices.

    +

    We named these two approaches N11MN11M and NMNM, respectively, which are in turn loosely based on the paper “Multi-Device for Signal” and Signal’s Sesame Algorithm.

    +

    +

    Informally, in the N11MN11M session management scheme, once the first Noise session between any of Alice’s and Bob’s device is instantiated, its session information is securely propagated to all other devices using previously instantiated Noise sessions. Hence, all devices are able to send and receive new messages on the content topic associated to such session.

    +

    +

    In the NMNM session management scheme, instead, all pairs of Alice's and Bob's devices have a distinct Noise session: a message is then sent from the currently-in-use sender’s device to all recipient’s devices, by properly encrypting and sending it to the content topics of each corresponding Noise session. If sent messages should be available on all sender’s devices as well, we require each pair of sender’s devices to instantiate a Noise session used for syncing purposes.

    +

    For more technical details on how Noise sessions are instantiated and managed within these two mechanisms and the different trade-offs provided by the latter, we refer to WAKU2-NOISE-SESSIONS.

    +

    Conclusions​

    +

    In this post we provided an overview of Noise, a protocol framework for designing Diffie-Hellman based key-exchange mechanisms allowing systematic security and threat model analysis.

    +

    The flexibility provided by Noise components allows not only to fully replicate with same security guarantees well established key-exchange primitives such as X3DH, currently employed by Status TRANSPORT-SECURITY, but enables also optimizations based on the reciprocal knowledge parties have of each other while allowing easier protocols' security analysis and (formal) verification.

    +

    Furthermore, different handshakes can be combined and executed one after each other, a particularly useful feature to authenticate multiple static keys employed by different applications but also to ease keys revocation.

    +

    The possibility to manage Noise sessions over multiple devices and the fact that handshakes can be concretely instantiated using modern, fast and secure cryptographic primitives such as ChaChaPoly and BLAKE2b, make Noise one of the best candidates for efficiently and securely address the many different needs of applications built on top of Waku requiring key-agreement.

    +

    Future steps​

    +

    The available implementation of Noise in nwaku, although mostly complete, is still in its testing phase. As future steps we would like to:

    +
      +
    • have an extensively tested and robust Noise implementation;
      • +
    • formalize, implement and test performances of the two proposed N11MN11M and NMNM session management mechanisms and their suitability for common use-case scenarios;
      • +
    • provide Waku network nodes a native protocol to readily support key-exchanges, strongly-encrypted communication and multi-device session management mechanisms with none-to-little interaction besides applications' connection requests.
      • +
    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/wakuv2-relay-anon/index.html b/rlog/wakuv2-relay-anon/index.html index 69654196..c34328ab 100644 --- a/rlog/wakuv2-relay-anon/index.html +++ b/rlog/wakuv2-relay-anon/index.html @@ -1,153 +1,297 @@ - + - -Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay | Vac Research + +Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay | Vac Research - - - + + + -
    -

    Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay

    by
    17 min read

    Introducing a basic threat model and privacy/anonymity analysis for the Waku v2 relay protocol.

    Waku v2 enables secure, privacy preserving communication using a set of modular P2P protocols. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay

    by
    17 min read

    Introducing a basic threat model and privacy/anonymity analysis for the Waku v2 relay protocol.

    + +

    Waku v2 enables secure, privacy preserving communication using a set of modular P2P protocols. Waku v2 also aims at protecting the user's anonymity. This post is the first in a series about Waku v2 security, privacy, and anonymity. The goal is to eventually have a full privacy and anonymity analysis for each of the Waku v2 protocols, as well as covering the interactions of various Waku v2 protocols. -This provides transparency with respect to Waku's current privacy and anonymity guarantees, and also identifies weak points that we have to address.

    In this post, we first give an informal description of security, privacy and anonymity in the context of Waku v2. +This provides transparency with respect to Waku's current privacy and anonymity guarantees, and also identifies weak points that we have to address.

    +

    In this post, we first give an informal description of security, privacy and anonymity in the context of Waku v2. For each definition, we summarize Waku's current guarantees regarding the respective property. -We also provide attacker models, an attack-based threat model, and a first anonymity analysis of Waku v2 relay within the respective models.

    Waku comprises many protocols that can be combined in a modular way. +We also provide attacker models, an attack-based threat model, and a first anonymity analysis of Waku v2 relay within the respective models.

    +

    Waku comprises many protocols that can be combined in a modular way. For our privacy and anonymity analysis, we start with the relay protocol because it is at the core of Waku v2 enabling Waku's publish subscribe approach to P2P messaging. -In its current form, Waku relay is a minor extension of libp2p GossipSub.

    Figure 1: The Waku v2 relay mesh is based on the [GossipSub mesh](https://docs.libp2p.io/concepts/publish-subscribe#types-of-peering)

    Informal Definitions: Security, Privacy, and Anonymity​

    The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.

    Security​

    Of the three, Security has the clearest agreed upon definition, -at least regarding its key concepts: confidentiality, integrity, and availability.

    • confidentiality: data is not disclosed to unauthorized entities.
    • integrity: data is not modified by unauthorized entities.
    • availability: data is available, i.e. accessible by authorized entities.

    While these are the key concepts, the definition of information security has been extended over time including further concepts, +In its current form, Waku relay is a minor extension of libp2p GossipSub.

    +

    Figure 1: The Waku v2 relay mesh is based on the GossipSub mesh

    +

    Informal Definitions: Security, Privacy, and Anonymity​

    +

    The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.

    +

    Security​

    +

    Of the three, Security has the clearest agreed upon definition, +at least regarding its key concepts: confidentiality, integrity, and availability.

    +
      +
    • confidentiality: data is not disclosed to unauthorized entities.
      • +
    • integrity: data is not modified by unauthorized entities.
      • +
    • availability: data is available, i.e. accessible by authorized entities.
      • +
    +

    While these are the key concepts, the definition of information security has been extended over time including further concepts, e.g. authentication and non-repudiation. -We might cover these in future posts.

    Privacy​

    Privacy allows users to choose which data and information

    • they want to share
    • and with whom they want to share it.

    This includes data and information that is associated with and/or generated by users. +We might cover these in future posts.

    +

    Privacy​

    +

    Privacy allows users to choose which data and information

    +
      +
    • they want to share
      • +
    • and with whom they want to share it.
      • +
    +

    This includes data and information that is associated with and/or generated by users. Protected data also comprises metadata that might be generated without users being aware of it. This means, no further information about the sender or the message is leaked. Metadata that is protected as part of the privacy-preserving property does not cover protecting the identities of sender and receiver. -Identities are protected by the anonymity property.

    Often privacy is realized by the confidentiality property of security. +Identities are protected by the anonymity property.

    +

    Often privacy is realized by the confidentiality property of security. This neither makes privacy and security the same, nor the one a sub category of the other. While security is abstract itself (its properties can be realized in various ways), privacy lives on a more abstract level using security properties. Privacy typically does not use integrity and availability. -An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.

    Waku offers confidentiality via secure channels set up with the help of the Noise Protocol Framework. +An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.

    +

    Waku offers confidentiality via secure channels set up with the help of the Noise Protocol Framework. Using these secure channels, message content is only disclosed to the intended receivers. They also provide good metadata protection properties. However, we do not have a metadata protection analysis as of yet, -which is part of our privacy/anonymity roadmap.

    Anonymity​

    Privacy and anonymity are closely linked. +which is part of our privacy/anonymity roadmap.

    +

    Anonymity​

    +

    Privacy and anonymity are closely linked. Both the identity of a user and data that allows inferring a user's identity should be part of the privacy policy. -For the purpose of analysis, we want to have a clearer separation between these concepts.

    We define anonymity as unlinkablity of users' identities and their shared data and/or actions.

    We subdivide anonymity into receiver anonymity and sender anonymity.

    Receiver Anonymity​

    We define receiver anonymity as unlinkability of users' identities and the data they receive and/or related actions. +For the purpose of analysis, we want to have a clearer separation between these concepts.

    +

    We define anonymity as unlinkablity of users' identities and their shared data and/or actions.

    +

    We subdivide anonymity into receiver anonymity and sender anonymity.

    +

    Receiver Anonymity​

    +

    We define receiver anonymity as unlinkability of users' identities and the data they receive and/or related actions. The data transmitted via Waku relay must be a Waku message, which contains a content topic field. Because each message is associated with a content topic, and each receiver is interested in messages with specific content topics, receiver anonymity in the context of Waku corresponds to subscriber-topic unlinkability. -An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.

    The Waku message's content topic is not related to the libp2p pubsub topic. +An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.

    +

    The Waku message's content topic is not related to the libp2p pubsub topic. For now, Waku uses a single libp2p pubsub topic, which means messages are propagated via a single mesh of peers. With this, the receiver discloses its participation in Waku on the gossipsub layer. -We will leave the analysis of libp2p gossipsub to a future article within this series, and only provide a few hints and pointers here.

    Waku offers k-anonymity regarding content topic interest in the global adversary model. -K-anonymity in the context of Waku means an attacker can link receivers to content topics with a maximum certainty of 1/k1/k. -The larger kk, the less certainty the attacker gains. -Receivers basically hide in a pool of kk content topics, any subset of which could be topics they subscribed to. +We will leave the analysis of libp2p gossipsub to a future article within this series, and only provide a few hints and pointers here.

    +

    Waku offers k-anonymity regarding content topic interest in the global adversary model. +K-anonymity in the context of Waku means an attacker can link receivers to content topics with a maximum certainty of 1/k1/k. +The larger kk, the less certainty the attacker gains. +Receivers basically hide in a pool of kk content topics, any subset of which could be topics they subscribed to. The attacker does not know which of those the receiver actually subscribed to, and the receiver enjoys plausible deniability regarding content topic subscription. -Assuming there are nn Waku content topics, a receiver has nn-anonymity with respect to association to a specific content topic.

    Technically, Waku allows distributing messages over several libp2p pubsub topics. -This yields kk-anonymity, assuming kk content topics share the same pubsub topic. +Assuming there are nn Waku content topics, a receiver has nn-anonymity with respect to association to a specific content topic.

    +

    Technically, Waku allows distributing messages over several libp2p pubsub topics. +This yields kk-anonymity, assuming kk content topics share the same pubsub topic. However, if done wrongly, such sharding of pubsub topics can breach anonymity. -A formal specification of anonymity-preserving topic sharding building on the concepts of partitioned topics is part of our roadmap.

    Also, Waku is not directly concerned with 1:1 communication, so for this post, 1:1 communication is out of scope. +A formal specification of anonymity-preserving topic sharding building on the concepts of partitioned topics is part of our roadmap.

    +

    Also, Waku is not directly concerned with 1:1 communication, so for this post, 1:1 communication is out of scope. Channels for 1:1 communication can be implemented on top of Waku relay. In the future, a 1:1 communication protocol might be added to Waku. -Similar to topic sharding, it would maintain receiver anonymity leveraging partitioned topics.

    Sender Anonymity​

    We define sender anonymity as unlinkability of users' identities and the data they send and/or related actions. -Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.

    In summary, Waku offers weak sender anonymity because of Waku's strict no sign policy, +Similar to topic sharding, it would maintain receiver anonymity leveraging partitioned topics.

    +

    Sender Anonymity​

    +

    We define sender anonymity as unlinkability of users' identities and the data they send and/or related actions. +Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.

    +

    In summary, Waku offers weak sender anonymity because of Waku's strict no sign policy, which has its origins in the Ethereum consensus specs. -17/WAKU-RLN-RELAY and 18/WAKU2-SWAP mitigate replay and injection attacks.

    Waku currently does not offer sender anonymity in stronger attacker models, as well as cannot protect against targeted attacks in weaker attacker models like the single or multi node attacker. -We will cover this in more detail in later sections.

    Anonymity Trilemma​

    The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global on-net attacker model. +17/WAKU-RLN-RELAY and 18/WAKU2-SWAP mitigate replay and injection attacks.

    +

    Waku currently does not offer sender anonymity in stronger attacker models, as well as cannot protect against targeted attacks in weaker attacker models like the single or multi node attacker. +We will cover this in more detail in later sections.

    +

    Anonymity Trilemma​

    +

    The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global on-net attacker model. Waku's goal, being a modular set of protocols, is to offer any combination of two out of these three properties, as well as blends. -An example for blending is an adjustable number of pubsub topics and peers in the respective pubsub topic mesh; this allows tuning the trade-off between anonymity and bandwidth.

    Figure 2: Anonymity Trilemma: pick two.

    A fourth factor that influences the anonymity trilemma is frequency and patterns of messages. +An example for blending is an adjustable number of pubsub topics and peers in the respective pubsub topic mesh; this allows tuning the trade-off between anonymity and bandwidth.

    +

    Figure 2: Anonymity Trilemma: pick two.

    +

    A fourth factor that influences the anonymity trilemma is frequency and patterns of messages. The more messages there are, and the more randomly distributed they are, the better the anonymity protection offered by a given anonymous communication protocol. So, incentivising users to use the protocol, for instance by lowering entry barriers, helps protecting the anonymity of all users. -The frequency/patterns factor is also related to the above described k-anonymity.

    Censorship Resistance​

    Another security related property that Waku aims to offer is censorship resistance. +The frequency/patterns factor is also related to the above described k-anonymity.

    +

    Censorship Resistance​

    +

    Another security related property that Waku aims to offer is censorship resistance. Censorship resistance guarantees that users can participate even if an attacker tries to deny them access. So, censorship resistance ties into the availability aspect of security. In the context of Waku that means users should be able to send messages as well as receive all messages they are interested in, -even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.

    Currently, Waku only guarantees censorship resistance in the weak single node attacker model. +even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.

    +

    Currently, Waku only guarantees censorship resistance in the weak single node attacker model. While currently employed secure channels mitigate targeted censorship, e.g. blocking specific content topics, general censorship resistance in strong attacker models is part of our roadmap. -Among other options, we will investigate Pluggable Transports in future articles.

    Attacker Types​

    The following lists various attacker types with varying degrees of power. -The more power an attacker has, the more difficult it is to gain the respective attacker position.

    Each attacker type comes in a passive and an active variant. +Among other options, we will investigate Pluggable Transports in future articles.

    +

    Attacker Types​

    +

    The following lists various attacker types with varying degrees of power. +The more power an attacker has, the more difficult it is to gain the respective attacker position.

    +

    Each attacker type comes in a passive and an active variant. While a passive attacker can stay hidden and is not suspicious, -the respective active attacker has more (or at least the same) deanonymization power.

    We also distinguish between internal and external attackers.

    Internal​

    With respect to Waku relay, an internal attacker participates in the same pubsub topic as its victims. -Without additional measures on higher layer protocols, access to an internal position is easy to get.

    Single Node​

    This attacker controls a single node. -Because this position corresponds to normal usage of Waku relay, it is trivial to obtain.

    Multi Node​

    This attacker controls several nodes. We assume a smaller static number of controlled nodes. +the respective active attacker has more (or at least the same) deanonymization power.

    +

    We also distinguish between internal and external attackers.

    +

    Internal​

    +

    With respect to Waku relay, an internal attacker participates in the same pubsub topic as its victims. +Without additional measures on higher layer protocols, access to an internal position is easy to get.

    +

    Single Node​

    +

    This attacker controls a single node. +Because this position corresponds to normal usage of Waku relay, it is trivial to obtain.

    +

    Multi Node​

    +

    This attacker controls several nodes. We assume a smaller static number of controlled nodes. The multi node position can be achieved relatively easily by setting up multiple nodes. Botnets might be leveraged to increase the number of available hosts. Multi node attackers could use Sybil attacks to increase the number of controlled nodes. -A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.

    Linearly Scaling Nodes​

    This attacker controls a number of nodes that scales linearly with the number of nodes in the network. +A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.

    +

    Linearly Scaling Nodes​

    +

    This attacker controls a number of nodes that scales linearly with the number of nodes in the network. This attacker is especially interesting to investigate in the context of DHT security, -which Waku uses for ambient peer discovery.

    External​

    An external attacker can only see encrypted traffic (protected by a secure channel set up with WAKU2-NOISE). +which Waku uses for ambient peer discovery.

    +

    External​

    +

    An external attacker can only see encrypted traffic (protected by a secure channel set up with WAKU2-NOISE). Because an internal position can be easily obtained, in practice external attackers would mount combined attacks that leverage both internal an external attacks. -We cover this more below when describing attacks.

    Local​

    A local attacker has access to communication links in a local network segment. -This could be a rogue access point (with routing capability).

    AS​

    An AS attacker controls a single AS (autonomous system). +We cover this more below when describing attacks.

    +

    Local​

    +

    A local attacker has access to communication links in a local network segment. +This could be a rogue access point (with routing capability).

    +

    AS​

    +

    An AS attacker controls a single AS (autonomous system). A passive AS attacker can listen to traffic on arbitrary links within the AS. -An active AS attacker can drop, inject, and alter traffic on arbitrary links within the AS.

    In practice, a malicious ISP would be considered as an AS attacker. +An active AS attacker can drop, inject, and alter traffic on arbitrary links within the AS.

    +

    In practice, a malicious ISP would be considered as an AS attacker. A malicious ISP could also easily setup a set of nodes at specific points in the network, -gaining internal attack power similar to a strong multi node attacker.

    Global On-Net​

    A global on-net attacker has complete overview over the whole network. +gaining internal attack power similar to a strong multi node attacker.

    +

    Global On-Net​

    +

    A global on-net attacker has complete overview over the whole network. A passive global attacker can listen to traffic on all links, while the active global attacker basically carries the traffic: it can freely drop, inject, and alter traffic at all positions in the network. -This basically corresponds to the Dolev-Yao model.

    An entity with this power would, in practice, also have the power of the internal linearly scaling nodes attacker.

    Attack-based Threat Analysis​

    The following lists various attacks including the weakest attacker model in which the attack can be successfully performed. -The respective attack can be performed in all stronger attacker models as well.

    An attack is considered more powerful if it can be successfully performed in a weaker attacker model.

    If not stated otherwise, we look at these attacks with respect to their capability to deanonymize the message sender.

    Scope​

    In this post, we introduce a simple tightly scoped threat model for Waku v2 Relay, which will be extended in the course of this article series.

    In this first post, we will look at the relay protocol in isolation. +This basically corresponds to the Dolev-Yao model.

    +

    An entity with this power would, in practice, also have the power of the internal linearly scaling nodes attacker.

    +

    Attack-based Threat Analysis​

    +

    The following lists various attacks including the weakest attacker model in which the attack can be successfully performed. +The respective attack can be performed in all stronger attacker models as well.

    +

    An attack is considered more powerful if it can be successfully performed in a weaker attacker model.

    +

    If not stated otherwise, we look at these attacks with respect to their capability to deanonymize the message sender.

    +

    Scope​

    +

    In this post, we introduce a simple tightly scoped threat model for Waku v2 Relay, which will be extended in the course of this article series.

    +

    In this first post, we will look at the relay protocol in isolation. Even though many threats arise from layers Waku relay is based on, and layers that in turn live on top of relay, we want to first look at relay in isolation because it is at the core of Waku v2. -Addressing and trying to solve all security issues of a complex system at once is an overwhelming task, which is why we focus on the soundness of relay first.

    This also goes well with the modular design philosophy of Waku v2, as layers of varying levels of security guarantees can be built on top of relay, all of which can relay on the guarantees that Waku provides. -Instead of looking at a multiplicative explosion of possible interactions, we look at the core in this article, and cover the most relevant combinations in future posts.

    Further restricting the scope, we will look at the data field of a relay message as a black box. +Addressing and trying to solve all security issues of a complex system at once is an overwhelming task, which is why we focus on the soundness of relay first.

    +

    This also goes well with the modular design philosophy of Waku v2, as layers of varying levels of security guarantees can be built on top of relay, all of which can relay on the guarantees that Waku provides. +Instead of looking at a multiplicative explosion of possible interactions, we look at the core in this article, and cover the most relevant combinations in future posts.

    +

    Further restricting the scope, we will look at the data field of a relay message as a black box. In a second article on Waku v2 relay, we will look into the data field, which according to the specification of Waku v2 relay must be a Waku v2 message. -We only consider messages with version field 2, which indicates that the payload has to be encoded using WAKU2-NOISE.

    Prerequisite: Get a Specific Position in the Network​

    Some attacks require the attacker node(s) to be in a specific position in the network. -In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.

    In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. +We only consider messages with version field 2, which indicates that the payload has to be encoded using WAKU2-NOISE.

    +

    Prerequisite: Get a Specific Position in the Network​

    +

    Some attacks require the attacker node(s) to be in a specific position in the network. +In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.

    +

    In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. If the victim node still has open slots, the attacker gets the desired position. -This only requires the attacker to know the gossipsub multiaddress of the victim node.

    A linearly scaling nodes attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. -Waku v2 discv5 will employ countermeasures that mitigate the amplifying effect this attacker type can achieve.

    Replay Attack​

    In the scope we defined above, Waku v2 is resilient against replay attacks. +This only requires the attacker to know the gossipsub multiaddress of the victim node.

    +

    A linearly scaling nodes attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. +Waku v2 discv5 will employ countermeasures that mitigate the amplifying effect this attacker type can achieve.

    +

    Replay Attack​

    +

    In the scope we defined above, Waku v2 is resilient against replay attacks. GossipSub nodes, and by extension Waku relay nodes, feature a seen cache, and only relay messages they have not seen before. -Further, replay attacks will be punished by RLN and SWAP.

    Neighbourhood Surveillance​

    This attack can be performed by a single node attacker that is connected to all peers of the victim node vv with respect to a specific topic mesh. -The attacker also has to be connected to vv. -In this position, the attacker will receive messages mvm_v sent by vv both on the direct path from vv, and on indirect paths relayed by peers of vv. -It will also receive messages mxm_x that are not sent by vv. These messages mxm_x are relayed by both vv and the peers of vv. -Messages that are received (significantly) faster from vv than from any other of vv's peers are very likely messages that vv sent, -because for these messages the attacker is one hop closer to the source.

    The attacker can (periodically) measure latency between itself and vv, and between itself and the peers of vv to get more accurate estimates for the expected timings. -An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between vv and its well-behaving peers. -An active AS attacker could also increase the latency between vv and its peers to make the timing differences more prominent. -This, however, might lead to vv switching to other peers.

    This attack cannot (reliably) distinguish messages mvm_v sent by vv from messages mym_y relayed by peers of vv the attacker is not connected to. +Further, replay attacks will be punished by RLN and SWAP.

    +

    Neighbourhood Surveillance​

    +

    This attack can be performed by a single node attacker that is connected to all peers of the victim node vv with respect to a specific topic mesh. +The attacker also has to be connected to vv. +In this position, the attacker will receive messages mvm_v sent by vv both on the direct path from vv, and on indirect paths relayed by peers of vv. +It will also receive messages mxm_x that are not sent by vv. These messages mxm_x are relayed by both vv and the peers of vv. +Messages that are received (significantly) faster from vv than from any other of vv's peers are very likely messages that vv sent, +because for these messages the attacker is one hop closer to the source.

    +

    The attacker can (periodically) measure latency between itself and vv, and between itself and the peers of vv to get more accurate estimates for the expected timings. +An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between vv and its well-behaving peers. +An active AS attacker could also increase the latency between vv and its peers to make the timing differences more prominent. +This, however, might lead to vv switching to other peers.

    +

    This attack cannot (reliably) distinguish messages mvm_v sent by vv from messages mym_y relayed by peers of vv the attacker is not connected to. Still, there are hop-count variations that might be leveraged. -Messages mvm_v always have a hop-count of 1 on the path from vv to the attacker, while all other paths are longer. -Messages mym_y might have the same hop-count on the path from vv as well as on other paths.

    Controlled Neighbourhood​

    If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from vv.

    Observing Messages​

    If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving vv which have not been relayed to vv. -These are the messages sent by vv. -Waku relay protects against this attack by employing secure channels setup using Noise.

    Correlation​

    Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from vv. -This (alone) does not allow an external attacker to learn which message vv sent, but it allows identifying the respective traffic propagating through the network. -The more traffic in the network, the lower the success rate of this attack.

    Combined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.

    DoS​

    An active single node attacker could run a disruption attack by

    • (1) dropping messages that should be relayed
    • (2) flooding neighbours with bogus messages

    While (1) has a negative effect on availability, the impact is not significant. +Messages mvm_v always have a hop-count of 1 on the path from vv to the attacker, while all other paths are longer. +Messages mym_y might have the same hop-count on the path from vv as well as on other paths.

    +

    Controlled Neighbourhood​

    +

    If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from vv.

    +

    Observing Messages​

    +

    If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving vv which have not been relayed to vv. +These are the messages sent by vv. +Waku relay protects against this attack by employing secure channels setup using Noise.

    +

    Correlation​

    +

    Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from vv. +This (alone) does not allow an external attacker to learn which message vv sent, but it allows identifying the respective traffic propagating through the network. +The more traffic in the network, the lower the success rate of this attack.

    +

    Combined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.

    +

    DoS​

    +

    An active single node attacker could run a disruption attack by

    +
      +
    • (1) dropping messages that should be relayed
      • +
    • (2) flooding neighbours with bogus messages
      • +
    +

    While (1) has a negative effect on availability, the impact is not significant. A linearly scaling botnet attacker, however, could significantly disrupt the network with such an attack. (2) is thwarted by RLN. -Also SWAP helps mitigating DoS attacks.

    A local attacker can DoS Waku by dropping all Waku traffic within its controlled network segment. +Also SWAP helps mitigating DoS attacks.

    +

    A local attacker can DoS Waku by dropping all Waku traffic within its controlled network segment. An AS attacker can DoS Waku within its authority, while a global attacker can DoS the whole network. -A countermeasure are censorship resistance techniques like Pluggable Transports.

    Summary and Future Work​

    Currently, Waku v2 relay offers k-anonymity with respect to receiver anonymity. -This also includes k-anonymity towards legitimate members of the same topic.

    Waku v2 relay offers sender anonymity in the single node attacker model with its strict no sign policy. +A countermeasure are censorship resistance techniques like Pluggable Transports.

    +

    Summary and Future Work​

    +

    Currently, Waku v2 relay offers k-anonymity with respect to receiver anonymity. +This also includes k-anonymity towards legitimate members of the same topic.

    +

    Waku v2 relay offers sender anonymity in the single node attacker model with its strict no sign policy. Currently, Waku v2 does not guarantee sender anonymity in the multi node and stronger attacker models. However, we are working on modular anonymity-preserving protocols and building blocks as part of our privacy/anonymity roadmap. The goal is to allow tunable anonymity with respect to trade offs between strong anonymity, low bandwidth, and low latency. All of these cannot be fully guaranteed as the the anonymity trilemma states. Some applications have specific requirements, e.g. low latency, which require a compromise on anonymity. -Anonymity-preserving mechanisms we plan to investigate and eventually specify as pluggable anonymity protocols for Waku comprise

    • Dandelion++ for lightweight anonymity;
    • onion routing as a building block adding a low latency anonymization layer;
    • a mix network for providing strong anonymity (on top of onion routing) even in the strongest attacker model at the cost of higher latency.

    These pluggable anonymity-preserving protocols will form a sub-set of the Waku v2 protocol set. -As an intermediate step, we might directly employ Tor for onion-routing, and Nym as a mix-net layer.

    In future research log posts, we will cover further Waku v2 protocols and identify anonymity problems that will be added to our roadmap. -These protocols comprise

    • 13/WAKU2-STORE, which can violate receiver anonymity as it allows filtering by content topic. -A countermeasure is using the content topic exclusively for local filters.
    • 12/WAKU2-FILTER, which discloses nodes' interest in topics;
    • 19/WAKU2-LIGHTPUSH, which also discloses nodes' interest in topics and links the lightpush client as the sender of a message to the lightpush service node;
    • 21/WAKU2-FTSTORE, which discloses nodes' interest in specific time ranges allowing to infer information like online times.

    While these protocols are not necessary for the operation of Waku v2, and can be seen as pluggable features, -we aim to provide alternatives without the cost of lowering the anonymity level.

    References​

    - - +Anonymity-preserving mechanisms we plan to investigate and eventually specify as pluggable anonymity protocols for Waku comprise

    +
      +
    • Dandelion++ for lightweight anonymity;
      • +
    • onion routing as a building block adding a low latency anonymization layer;
      • +
    • a mix network for providing strong anonymity (on top of onion routing) even in the strongest attacker model at the cost of higher latency.
      • +
    +

    These pluggable anonymity-preserving protocols will form a sub-set of the Waku v2 protocol set. +As an intermediate step, we might directly employ Tor for onion-routing, and Nym as a mix-net layer.

    +

    In future research log posts, we will cover further Waku v2 protocols and identify anonymity problems that will be added to our roadmap. +These protocols comprise

    +
      +
    • 13/WAKU2-STORE, which can violate receiver anonymity as it allows filtering by content topic. +A countermeasure is using the content topic exclusively for local filters.
      • +
    • 12/WAKU2-FILTER, which discloses nodes' interest in topics;
      • +
    • 19/WAKU2-LIGHTPUSH, which also discloses nodes' interest in topics and links the lightpush client as the sender of a message to the lightpush service node;
      • +
    • 21/WAKU2-FTSTORE, which discloses nodes' interest in specific time ranges allowing to infer information like online times.
      • +
    +

    While these protocols are not necessary for the operation of Waku v2, and can be seen as pluggable features, +we aim to provide alternatives without the cost of lowering the anonymity level.

    +

    References​

    +
    \ No newline at end of file diff --git a/rlog/wechat-replacement-need/index.html b/rlog/wechat-replacement-need/index.html index 3503a3a5..fdcd0389 100644 --- a/rlog/wechat-replacement-need/index.html +++ b/rlog/wechat-replacement-need/index.html @@ -1,26 +1,189 @@ - + - -What Would a WeChat Replacement Need? | Vac Research + +What Would a WeChat Replacement Need? | Vac Research - - - + + + -
    -

    What Would a WeChat Replacement Need?

    by
    26 min read

    What would a self-sovereign, private, censorship-resistant and open alternative to WeChat look like?

    What would it take to replace WeChat? More specifically, what would a self-sovereign, private, censorship-resistant and open alternative look like? One that allows people to communicate, coordinate and transact freely.

    Background​

    What WeChat provides to the end-user​

    Let's first look at some of the things that WeChat providers. It is a lot:

    • Messaging: 1:1 and group chat. Text, as well as voice and video. Post gifs. Share location.
    • Group chat: Limited to 500 people; above 100 people people need to verify with a bank account. Also has group video chat and QR code to join a group.
    • Timeline/Moments: Post comments with attachments and have people like/comment on it.
    • Location Discovery: See WeChat users that are nearby.
    • Profile: Nickname and profile picture; can alias people.
    • "Broadcast" messages: Send one message to many contacts, up to 200 people (spam limited).
    • Contacts: Max 5000 contacts (people get around it with multiple accounts and sim cards).
    • App reach: Many diferent web apps, extensions, native apps, etc. Scan QR code to access web app from phone.
    • Selective posting: Decide who can view your posts and who can view your comments on other people's post.
    • Transact: Send money gifts through red envelopes.
    • Transact: Use WeChat pay to transfer money to friends and businesses; linked account with Alipay that is connected to your bank account.
    • Services: Find taxis and get notifications; book flights, train tickets, hotels etc.
    • Mini apps: API for all kinds of apps that allow you to provide services etc.
    • Picture in picture: allowing you to have a video call while using the app.

    And much more. Not going to through it all in detail, and there are probably many things I don't know about WeChat since I'm not a heavy user living in mainland China.

    How WeChat works - a toy model​

    This is an overly simplistic model of how WeChat works, but it is sufficient for our purposes. This general design applies to most traditional client-server apps today.

    To sign up for account you need a phone number or equivalent. To get access to some features you need to verify your identity further, for example with official ID and/or bank account.

    When you signup this creates an entry in the WeChat server, from now on treated as a black box. You authenticate with that box, and thats where you get your messages from. If you go online the app asks that box for messages you have received while you were offline. If you login from a different app your contacts and conversations are synced from that box.

    The box gives you an account, it deals with routing to your contacts, it stores messages and attachments and gives access to mini apps that people have uploaded. For transacting money, there is a partnership with a different company that has a different box which talks to your bank account.

    This is done in a such a way that they can support a billion users with the features above, no sweat.

    Whoever controls that box can sees who you are talking with and what the content of those messages are. There is no end to end encryption. If WeChat/Tencent disagrees with you for some reason they can ban you. This means you can't interact with the box under that name anymore.

    What do we want?​

    We want something that is self-sovereign, private, censorship-resistant and open that allows individuals and groups of people to communicate and transact freely. To explore what this means in more detail, without getting lost in the weeds, we provide the following list of properties. A lot of these are tied together, and some fall out of the other requirements. Some of them stand in slight opposition to each other.

    Self-sovereignity identity. Exercises authority within your own sphere. If you aren't harming anyone, you should be able to have an account and communicate with other people.

    Pseudonymity, and ideally total anonymity. Not having your identity tied to your real name (e.g. through phone number, bank account, ID, etc). This allows people to act more freely without being overly worried about censorship and coercion in the real world. While total anonymity is even more desirable - especially to break multiple hops to a true-name action - real-world constraints sometimes makes this more challenging.

    Private and secure communication. Your communication and who you transact with should be for your eyes only. This includes transactions (transfer of value) as a form of communication.

    Censorship-resistance. Not being able to easily censor individuals on the platform. Both at an individual, group and collective level. Not having single points of failure that allow service to be disrupted.

    Decentralization. Partly falls out of censorship-resistance and other properties. If infrastructure isn't decentralized it means there's a single point of failure that can be disrupted. This is more of a tool than a goal on its own, but it is an important tool.

    Built for mass adoption. Includes scalabiltiy, UX (latency, reliability, bandwidth consumption, UI etc), and allowing for people to stick around. One way of doing this is to allow users to discover people they want to talk to.

    Scalability. Infrastructure needs to support a lot of users to be a viabile alternative. Like, a billion of them (eventually).

    Fundamentals in place to support great user experience. To be a viable alternative, aside from good UI and distribution, fundamentals such as latency, bandwidth usage, consistency etc must support great UX to be a viable alternative.

    Works for resource restricted devices, including smartphones. Most people will use a smartphone to use this. This means it has to work well on them and similar devices, without becoming a second-class citizen where we ignore properties such as censorship-resistance and privacy. Some concession to reality will be necessary due to additional constraints, which leads us to...

    Adaptive nodes. Nodes will have different capabilities, and perhaps at different times. To maintain a lot of the properties described here it is desirable if as many participants as possible are first-class citizens. If a phone is switching from a limited data plan to a WiFi network or from battery to AC power it can do more useful work, and so on. Likewise for a laptop with a lot of free disk space and spare compute power, etc.

    Sustainable. If there's no centralized, top down ad-driven model, this means all the infrastructure has to be sustainable somehow. Since these are individual entitites, this means it has to be paid for. While altruistic modes and similar can be used, this likely requires some form of incentivization scheme for useful services provided in the network. Related: free rider problem.

    Spam resistant. Relates to sustainability, scalability and built for mass adoption. Made more difficult by pseudonymous identity due to whitewashing attacks.

    Trust-minimized. To know that properties are provided for and aren't compromised, various ways of minimizing trust requirements are useful. This also related to mass adoption and social cohesion. Examples include: open and audited protocols, open source, reproducible builds, etc. This also relates to how mini apps are provided for, since we may not know their source but want to be able to use them anyway.

    Open source. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn't been compromised, e.g. by uploading private data to a third party.

    Some of these are graded and a bit subtle, i.e.:

    • Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible.
    • Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience.
    • Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals.
    • "Discover people" and striving for "total anonymity" might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy.
    • Users often want some form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement.

    Counterpoint 1​

    We could refute the above by saying that the design goals are undesirable. We want a system where people can censor others, and where everyone is tied to their real identity. Or we could say something like, freedom of speech is a general concept, and it doesn't apply to Internet companies, even if they provide a vital service. You can survive without it and you should've read the terms of service. This roughly charactericizes the mainstream view.

    Additional factor here is the idea that a group of people know more about what's good for you then you do, so they are protecting you.

    Counterpoint 2​

    We could agree with all these design goals, but think they are too extreme in terms of their requirements. For example, we could operate as a non profit, take donations and volunteers, and then host the whole infrastructure ourselves. We could say we are in a friendly legislation, so we won't be a single point of failure. Since we are working on this and maybe even our designs are open, you can trust us and we'll provide service and infrastructure that gives you what you want without having to pay for it or solve all these complex decentralized computation and so on problems. If you don't trust us for some reason, you shouldn't use us regardless. Also, this is better than status quo. And we are more likely to survive by doing this, either by taking shortcuts or by being less ambituous in terms of scope.

    Principal components​

    There are many ways to skin a cat, but this is one way of breaking down the problem. We have a general direction with the properties listed above, together with some understanding of how WeChat works for the everday user. Now the question is, what infrastructure do we need to support this? How do we achieve the above properties, or at least get closer to them? We want to figure out the necessary building blocks, and one of doing this is to map out likely necessary components.

    Background: Ethereum and Web3 stack​

    It is worth noting that a lot of the required infrastructure has been developed, at least as concepts, in the original Ethereum / Web3 vision. In it there is Ethereum for consensus/compute/transact, storage through Swarm, and communication through Whisper. That said, the main focus has been on the Ethereum blockchain itself, and a lot of things have happened in the last 5y+ with respect to technology around privacy and scalabilty. It is worth revisiting things from a fresh point of view, with the WeChat alternative in mind as a clear use case.

    Account - self-sovereign identity and the perils of phone numbers​

    Starting from the most basic: what is an account and how do you get one? With most internet services today, WeChat and almost all popular messaging apps included, you need to signup with some centralized authority. Usually you also have to verify this with some data that ties this account to you as an individual. E.g. by requiring a phone number, which in most jurisdictions 1 means giving out your real ID. This also means you can be banned from using the service by a somewhat arbitrary process, with no due process.

    Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this.

    Justifications for requiring phone numbers are usually centered around three claims:

    1. Avoiding spam
    2. Tying your account to your real name, for various reasons
    3. Using as a commonly shared identifier as a social network discovery mechanism

    Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on.

    In contrast, a self-sovereign system would allow you to "create an account" completely on your own. This can easily be done with public key cryptograpy, and it also paves the way for end-to-end encryption to make your messages private.

    The main issue with this that you need to get more creative about avoiding spam (e.g. through white washing attacks), and ideally there is some other form of social discovery mechanism.

    Just having a public key as an account isn't enough though. If it goes through a central server, then nothing is stopping that server from arbitrarly blocking requests related to that public key. Of course, this also depends on how transparent such requests are. Fundamentally, lest we rely completely on goodwill, there needs to be multiple actors by which you can use the service. This naturally points to decentralization as a requirement. See counterpoint.

    Even so, if the system is closed source we don't know what it is doing. Perhaps the app communicating is also uploading data to another place, or somehow making it possible to see who is who and act accordingly.

    You might notice that just one simple property, self-sovereign identity, leads to a slew of other requirements and properties. You might also notice that WeChat is far from alone in this, even if their identity requirements might be a bit stringent than, say, Telegram. Their control aspects are also a bit more extreme, at least for someone with western sensibilities 2.

    Most user facing applications have similar issues, Google Apps/FB/Twitter etc. For popular tools that have this built in, we can look at git - which is truly decentralized and have keypair at the bottom. It is for a very specific technical domain, and even then people rely on Github. Key management is fairly difficult even for technical people, and for normal people even more so. Banks are generally far behind on this tech, relying on arcane procedures and special purpose hardware for 2FA. That's another big issue.

    Let's shift gears a bit and talk about some other functional requirements.

    Routing - packets from A to B​

    In order to get a lot of the features WeChat provides, we need the ability to do three things: communicate, store data, and transact with people. We need a bit more than that, but let's focus on this for now.

    To communicate with people, in the base case, we need to go from one phone to another phone that is separated by a large distance. This requires some form of routing. The most natural platform to build this on is the existing Internet, though not the only one. Most phones are resource restricted, and are only "on" for brief periods of time. This is needed to preserve battery and bandwidth. Additionally, Internet uses IPs as endpoints, which change as a phones move through space. NAT punching etc isn't always perfect either. This means we need a way to get a message from one public key to another, and through some intermediate nodes. We can think of these nodes as a form of service network. Similar to how a power grid works, or phone lines, or collection of ISPs.

    One important property here is to ensure we don't end up in a situation like the centralized capture scenario above, something we've seen with centralized ISPs 3 4 where they can choose which traffic is good and which is bad. We want to allow the use of different service nodes, just like if a restaurant gives you food poisioning you can go to the one next door and then the first one goes out of business after a while. And the circle of life continues.

    We shouldn't be naive though, and think that this is something nodes are likely to do for free. They need to be adequately compensated for their services, in some of incentivization scheme. That can either be monetary, or as in the case of Bittorrent, more of a barter situation where you use game theory to coordinate with strangers 5, and some form of reputation attached to it (for private trackers).

    There are many ways of doing routing, and we won't go into too much technical detail here. Suffice to say is that you likely want both a structured and unstructured alternative, and that these comes with several trade-offs when it comes to efficiency, metadata protection, ability to incentivize, compatibility with existing topologies, and suitability for mobilephones (mostly offline, bandwidth restricted, not directly connectable). Expect more on this in a future article.

    Some of these considerations naturally leads us into the storage and transaction components.

    Storage - available and persistant for later​

    If mobile phones are mostly offline, we need some way to store these messages so they can be retrieved when online again. The same goes for various kinds attachments as well, and for when people are switching devices. A user might control their timeline, but in the WeChat case that timeline is stored on Tencent's servers, and queried from there as well. This naturally needs to happen by some other service nodes. In the WeChat case, and for most IMs, the way these servers are paid for is through some indirect ad mechanism. The entity controlling these ads and so on is the same one as the one operating the servers for storage. A more direct model with different entities would see these services being compensated for their work.

    We also need storage for attachments, mini-apps, as well as a way of understanding the current state of consensus when it comes to the compute/transact module. In the WeChat case, this state is completely handled by the bank institution or one of their partners, such as Alibaba. When it comes to bearer instruments like cash, no state needs to be kept as that's a direct exchange in the physical world. This isn't directly compatible with transfering value over a distance.

    All of this state requires availability and persistance. It should be done in a trust minimized fashion and decentralized, which requires some form of incentivization for keeping data around. If it isn't, you are relying on social cohesion which breaks down at very large scales.

    Since data will be spread out across multiple nodes, you need a way to sync data and transfer it in the network. As well as being able to add and query data from it. All of this requires a routing component.

    To make it more censorship resistant it might be better to keep it as a general-purpose store, i.e. individuals don't need to know what they storing. Otherwise, you naturally end up in a situation where individual nodes can be pressured to not store certain content.

    Messaging - from me to you to all of us (not them)​

    This builds on top of routing, but it has a slightly different focus. The goal is to allow for individuals and groups to communicate in a private, secure and censorship-resistant manner.

    It also needs to provide a decent interface to the end user, in terms of dealing seamlessly with offline messages, providing reliable and timely messaging.

    In order to get closer to the ideal of total anonymity, it is useful to be able to hide metadata of who is talking to whom. This applies to both normal communication as well as for transactions. Ideally, no one but the parties involved can see who is taking part in a conversation. This can be achieved through various techniques such as mixnets, anonymous credentials, private information retrieval, and so on. Many of these techniques have a fundamental trade-off with latency and bandwidth, something that is a big concern for mobilephones. Being able to do some form of tuning, in an adaptive node manner, depending on your threat model and current capabilities is useful here.

    The baseline here is pseudonymity, and having tools to allow individuals to "cut off" ties to their real world identity and transactions. People act different in different circles in the real world, and this should be mimicked online as well. Your company, family or government shouldn't be able to know what exactly you use your paycheck for, and who you are talking to.

    Compute - transact, contract and settle​

    The most immediate need here is transaction from A to B. Direct exchange. There is also a more indirect need for private lawmaking and contracting.

    We talked about routing and storage and how they likely need to be incentivized to work properly. How are they going to be compensated? While this could in theory work via existing banking system and so on, this would be rather heavy. It'd also very likely require tying your identifier to your legal name, something that goes against what we want to achieve. What we want is something that acts more as right-to-access, similar to the way cash functions in a society 6. I pay for a fruit with something that is valuable to you and then I'm on my way.

    While there might be other candidates, such as pre-paid debit cards and so on, this transaction mode pretty much requires a cryptocurrency component. The alternative is to do it on a reputation basis, which might work for small communities, due to social cohesion, but quickly detoriates for large ones 7. Ad hoc models like private Bittorrent trackers are centralized and easy to censor.

    Now, none of the existing cryptocurrency models are ideal. They also all suffer from lack of widespread use, and it is difficult to get onboarded to them in the first place. Transactions in Bitcoin are slow. Ethereum is faster and has more capabilities, but it still suffers from linking payments over time, which makes the privacy part of this more difficult. Zcash, Monero and similar are interesting, but also require more use. For Zcash, shielded transactions appear to only account for less than 2% of all transactions in 2019 8 9.

    Another dimension is what sets general purpose cryptocurrencies like Ethereum apart. Aside from just paying from A to B, you can encode rules about when something should be paid out and not. This is very useful for doing a form of private lawmaking, contracting, for setting up service agreements with these nodes. If there's no trivial recourse as in the meatspace world, where you know someone's name and you can sue them, you need a different kind of model.

    What makes something like Zcash interesting is that it works more like digital cash. Instead of leaving a public trail for everyone, where someone can see where you got the initial money from and then trace you across various usage, for Zcash every hop is privacy preserving.

    To fulfill the general goals of being censorship resistance and secure, it is also vital that the system being used stays online and can't be easily disrupted. That points to disintermediation, as opposed to using gateways and exchanges. This is a case where something like cash, or gold, is more direct, since no one can censor this transaction without being physically present where this direct exchange is taking place. However, like before, this doesn't work over distance.

    Secure chat - just our business​

    Similar to the messaging module above. The distinction here is that we assume the network part has already taken place. Here we are interested in keeping the contents of messages private, so that means confidentiality/end-to-end encryption, integrity, authentication, as well as forward secrecy and plausible deniability. This means that even if there's some actor that gets some private key material, or confiscated your phone, there is some level of...ephemerality to your conversations. Another issue here in terms of scalable private group chat.

    Extensible mini apps​

    This relates to the compute and storage module above. Essentially we want to provide mini apps as in WeChat, but to do so in a way that is compatible with what we want to achieve more generally. This allows individuals and small businesses to create small tools for various purposes, and coordinate with strangers. E.g. booking a cab or getting an insurance, and so on.

    This has a higher dependency on the contracting/general computation aspect. I.e. often it isn't only a transaction, but you might want to encode some specific rules here that strangers can abide by without having too high trust requirements. As a simple example: escrows.

    This also needs an open API that anyone can use. It should be properly secured, so using one doesn't compromise the rest of the system it is operating in. To be censorship resistant it requires the routing and storage component to work properly.

    Where are we now?​

    Let's look back at some of desirable properties we set out in the beginning and see how close we are to building out the necessary components. Is it realistic at all or just a pipe dream? We'll see that there are many building blocks in place, and there's reason for hope.

    Self-sovereignity identity. Public key crypto and web of trust like constructs makes this possible.

    Pseudonymity, and ideally total anonymity. Pseudonymity can largely be achieved with public key crypto and open systems that allow for permissionless participation. For transactions, pseudonymity exists in most cryptocurrencies. The challenge is linkage across time, especially when interfacing with other "legacy" system. There are stronger constructs that are actively being worked on and are promising here, such as mixnets (Nym), mixers (Wasabi Wallet, Tornado.Cash) and zero knowledge proofs (Zcash, Ethereum, Starkware). This area of applied research has exploded over the last few years.

    Private and secure communication. Signal has pioneered a lot of this, following OTR. Double Ratchet, X3DH. E2EE is minimum these days, and properties like PFS and PD are getting better. For metadata protection, you have Tor, with its faults, and more active research on mixnets and private information retrieval, etc.

    Censorship-resistance. This covers a lot of ground across the spectrum. You have technologies like Bittorrent, Bitcoin/Ethereum, Tor obfuscated transports, E2EE by default, partial mesh networks in production, abilit to move/replicate host machines more quickly have all made this more of a reality than it used to be. this easier. Of course, techniques such as deep packet inspection and internet shutdowns have increased.

    Decentralization. Cryptocurrencies, projects like libp2p and IPFS. Need to be mindful here of many projects that claim decentralization but are still vulnerable to single points of failures, such as relying on gateways.

    Built for mass adoption. This one is more subjective. There's definitely a lot of work to be done here, both when it comes to fundamental performance, key management and things like social discoverability. Directionally these things are improving and becoming easier for the average person but there is a lot ot be done here.

    Scalability. With projects like Ethereum 2.0 and IPFS more and more resources are a being put into this, both at the consensus/compute layer as well as networking (gossip, scalable Kademlia) layer. Also various layer 2 solutions for transactions.

    Fundamentals in place to support great user experience. Similar to built for mass adoption. As scalability becomes more important, more applied research is being done in the p2p area to improve things like latency, bandwidth.

    Works for resource restricted devices, including smartphones. Work in progress and not enough focus here, generally an after thought. Also have stateless clients etc.

    Adaptive nodes. See above. With subprotocols and capabilities in Ethereum and libp2p, this is getting easier.

    Sustainable. Token economics is a thing. While a lot of it won't stay around, there are many more projects working on making themselves dispensable. Being open source, having an engaged community and enabling users run their own infrastructure. Users as stakeholders.

    Spam resistant. Tricky problem if you want to be pseudonymous, but some signs of hope with incentivization mechanisms, zero knowledge based signaling, etc. Together with various forms of rate limiting and better controlling of topology and network amplification. And just generally being battle-tested by real world attacks, such as historical Ethereum DDoS attacks.

    Trust minimized. Bitcoin. Zero knowledge provable computation. Open source. Reproducible builds. Signed binaries. Incentive compatible structures. Independent audits. Still a lot of work, but getting better.

    Open source. Big and only getting bigger. Including mainstream companies.

    What's next?​

    We've look at what WeChat provides and what we'd like an alternative to look like. We've also seen a few principal modules that are necessary to achieve those goals. To achieve all of this is a daunting task, and one might call it overly ambitiuous. We've also seen how far we've come with some of the goals, and how a lot of the pieces are there, in one form or another. Then it is a question of putting them all together in the right mix.

    The good news is that a lot of people are working all these building blocks and thinking about these problems. Compared to a few years ago we've come quite far when it comes to p2p infrastructure, privacy, security, scalability, and general developer mass and mindshare. If you want to join us in building some of these building blocks, and assembling them, check out our forum.

    PS. We are hiring protocol engineers. DS

    Acknowledgements​

    Corey, Dean, Jacek.

    References​

    1. Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws↩
    2. On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/↩
    3. Net Neutrality: https://www.eff.org/issues/net-neutrality↩
    4. ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/↩
    5. Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf↩
    6. The Case for Electronic Cash: https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf↩
    7. Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html↩
    8. Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions↩
    9. Shielded transactions usage (stats page 404s): https://z.cash/support/faq/↩
    - - + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    What Would a WeChat Replacement Need?

    by
    26 min read

    What would a self-sovereign, private, censorship-resistant and open alternative to WeChat look like?

    + +

    What would it take to replace WeChat? More specifically, what would a self-sovereign, private, censorship-resistant and open alternative look like? One that allows people to communicate, coordinate and transact freely.

    +

    Background​

    +

    What WeChat provides to the end-user​

    +

    Let's first look at some of the things that WeChat providers. It is a lot:

    +
      +
    • Messaging: 1:1 and group chat. Text, as well as voice and video. Post gifs. Share location.
      • +
    • Group chat: Limited to 500 people; above 100 people people need to verify with a bank account. Also has group video chat and QR code to join a group.
      • +
    • Timeline/Moments: Post comments with attachments and have people like/comment on it.
      • +
    • Location Discovery: See WeChat users that are nearby.
      • +
    • Profile: Nickname and profile picture; can alias people.
      • +
    • "Broadcast" messages: Send one message to many contacts, up to 200 people (spam limited).
      • +
    • Contacts: Max 5000 contacts (people get around it with multiple accounts and sim cards).
      • +
    • App reach: Many diferent web apps, extensions, native apps, etc. Scan QR code to access web app from phone.
      • +
    • Selective posting: Decide who can view your posts and who can view your comments on other people's post.
      • +
    • Transact: Send money gifts through red envelopes.
      • +
    • Transact: Use WeChat pay to transfer money to friends and businesses; linked account with Alipay that is connected to your bank account.
      • +
    • Services: Find taxis and get notifications; book flights, train tickets, hotels etc.
      • +
    • Mini apps: API for all kinds of apps that allow you to provide services etc.
      • +
    • Picture in picture: allowing you to have a video call while using the app.
      • +
    +

    And much more. Not going to through it all in detail, and there are probably many things I don't know about WeChat since I'm not a heavy user living in mainland China.

    +

    How WeChat works - a toy model​

    +

    This is an overly simplistic model of how WeChat works, but it is sufficient for our purposes. This general design applies to most traditional client-server apps today.

    +

    To sign up for account you need a phone number or equivalent. To get access to some features you need to verify your identity further, for example with official ID and/or bank account.

    +

    When you signup this creates an entry in the WeChat server, from now on treated as a black box. You authenticate with that box, and thats where you get your messages from. If you go online the app asks that box for messages you have received while you were offline. If you login from a different app your contacts and conversations are synced from that box.

    +

    The box gives you an account, it deals with routing to your contacts, it stores messages and attachments and gives access to mini apps that people have uploaded. For transacting money, there is a partnership with a different company that has a different box which talks to your bank account.

    +

    This is done in a such a way that they can support a billion users with the features above, no sweat.

    +

    Whoever controls that box can sees who you are talking with and what the content of those messages are. There is no end to end encryption. If WeChat/Tencent disagrees with you for some reason they can ban you. This means you can't interact with the box under that name anymore.

    +

    What do we want?​

    +

    We want something that is self-sovereign, private, censorship-resistant and open that allows individuals and groups of people to communicate and transact freely. To explore what this means in more detail, without getting lost in the weeds, we provide the following list of properties. A lot of these are tied together, and some fall out of the other requirements. Some of them stand in slight opposition to each other.

    +

    Self-sovereignity identity. Exercises authority within your own sphere. If you aren't harming anyone, you should be able to have an account and communicate with other people.

    +

    Pseudonymity, and ideally total anonymity. Not having your identity tied to your real name (e.g. through phone number, bank account, ID, etc). This allows people to act more freely without being overly worried about censorship and coercion in the real world. While total anonymity is even more desirable - especially to break multiple hops to a true-name action - real-world constraints sometimes makes this more challenging.

    +

    Private and secure communication. Your communication and who you transact with should be for your eyes only. This includes transactions (transfer of value) as a form of communication.

    +

    Censorship-resistance. Not being able to easily censor individuals on the platform. Both at an individual, group and collective level. Not having single points of failure that allow service to be disrupted.

    +

    Decentralization. Partly falls out of censorship-resistance and other properties. If infrastructure isn't decentralized it means there's a single point of failure that can be disrupted. This is more of a tool than a goal on its own, but it is an important tool.

    +

    Built for mass adoption. Includes scalabiltiy, UX (latency, reliability, bandwidth consumption, UI etc), and allowing for people to stick around. One way of doing this is to allow users to discover people they want to talk to.

    +

    Scalability. Infrastructure needs to support a lot of users to be a viabile alternative. Like, a billion of them (eventually).

    +

    Fundamentals in place to support great user experience. To be a viable alternative, aside from good UI and distribution, fundamentals such as latency, bandwidth usage, consistency etc must support great UX to be a viable alternative.

    +

    Works for resource restricted devices, including smartphones. Most people will use a smartphone to use this. This means it has to work well on them and similar devices, without becoming a second-class citizen where we ignore properties such as censorship-resistance and privacy. Some concession to reality will be necessary due to additional constraints, which leads us to...

    +

    Adaptive nodes. Nodes will have different capabilities, and perhaps at different times. To maintain a lot of the properties described here it is desirable if as many participants as possible are first-class citizens. If a phone is switching from a limited data plan to a WiFi network or from battery to AC power it can do more useful work, and so on. Likewise for a laptop with a lot of free disk space and spare compute power, etc.

    +

    Sustainable. If there's no centralized, top down ad-driven model, this means all the infrastructure has to be sustainable somehow. Since these are individual entitites, this means it has to be paid for. While altruistic modes and similar can be used, this likely requires some form of incentivization scheme for useful services provided in the network. Related: free rider problem.

    +

    Spam resistant. Relates to sustainability, scalability and built for mass adoption. Made more difficult by pseudonymous identity due to whitewashing attacks.

    +

    Trust-minimized. To know that properties are provided for and aren't compromised, various ways of minimizing trust requirements are useful. This also related to mass adoption and social cohesion. Examples include: open and audited protocols, open source, reproducible builds, etc. This also relates to how mini apps are provided for, since we may not know their source but want to be able to use them anyway.

    +

    Open source. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn't been compromised, e.g. by uploading private data to a third party.

    +

    Some of these are graded and a bit subtle, i.e.:

    +
      +
    • Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible.
      • +
    • Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience.
      • +
    • Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals.
      • +
    • "Discover people" and striving for "total anonymity" might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy.
      • +
    • Users often want some form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement.
      • +
    +

    Counterpoint 1​

    +

    We could refute the above by saying that the design goals are undesirable. We want a system where people can censor others, and where everyone is tied to their real identity. Or we could say something like, freedom of speech is a general concept, and it doesn't apply to Internet companies, even if they provide a vital service. You can survive without it and you should've read the terms of service. This roughly charactericizes the mainstream view.

    +

    Additional factor here is the idea that a group of people know more about what's good for you then you do, so they are protecting you.

    +

    Counterpoint 2​

    +

    We could agree with all these design goals, but think they are too extreme in terms of their requirements. For example, we could operate as a non profit, take donations and volunteers, and then host the whole infrastructure ourselves. We could say we are in a friendly legislation, so we won't be a single point of failure. Since we are working on this and maybe even our designs are open, you can trust us and we'll provide service and infrastructure that gives you what you want without having to pay for it or solve all these complex decentralized computation and so on problems. If you don't trust us for some reason, you shouldn't use us regardless. Also, this is better than status quo. And we are more likely to survive by doing this, either by taking shortcuts or by being less ambituous in terms of scope.

    +

    Principal components​

    +

    There are many ways to skin a cat, but this is one way of breaking down the problem. We have a general direction with the properties listed above, together with some understanding of how WeChat works for the everday user. Now the question is, what infrastructure do we need to support this? How do we achieve the above properties, or at least get closer to them? We want to figure out the necessary building blocks, and one of doing this is to map out likely necessary components.

    +

    Background: Ethereum and Web3 stack​

    +

    It is worth noting that a lot of the required infrastructure has been developed, at least as concepts, in the original Ethereum / Web3 vision. In it there is Ethereum for consensus/compute/transact, storage through Swarm, and communication through Whisper. That said, the main focus has been on the Ethereum blockchain itself, and a lot of things have happened in the last 5y+ with respect to technology around privacy and scalabilty. It is worth revisiting things from a fresh point of view, with the WeChat alternative in mind as a clear use case.

    +

    Account - self-sovereign identity and the perils of phone numbers​

    +

    Starting from the most basic: what is an account and how do you get one? With most internet services today, WeChat and almost all popular messaging apps included, you need to signup with some centralized authority. Usually you also have to verify this with some data that ties this account to you as an individual. E.g. by requiring a phone number, which in most jurisdictions 1 means giving out your real ID. This also means you can be banned from using the service by a somewhat arbitrary process, with no due process.

    +

    Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this.

    +

    Justifications for requiring phone numbers are usually centered around three claims:

    +
      +
    1. Avoiding spam
      2. +
    2. Tying your account to your real name, for various reasons
      3. +
    3. Using as a commonly shared identifier as a social network discovery mechanism
      4. +
    +

    Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on.

    +

    In contrast, a self-sovereign system would allow you to "create an account" completely on your own. This can easily be done with public key cryptograpy, and it also paves the way for end-to-end encryption to make your messages private.

    +

    The main issue with this that you need to get more creative about avoiding spam (e.g. through white washing attacks), and ideally there is some other form of social discovery mechanism.

    +

    Just having a public key as an account isn't enough though. If it goes through a central server, then nothing is stopping that server from arbitrarly blocking requests related to that public key. Of course, this also depends on how transparent such requests are. Fundamentally, lest we rely completely on goodwill, there needs to be multiple actors by which you can use the service. This naturally points to decentralization as a requirement. See counterpoint.

    +

    Even so, if the system is closed source we don't know what it is doing. Perhaps the app communicating is also uploading data to another place, or somehow making it possible to see who is who and act accordingly.

    +

    You might notice that just one simple property, self-sovereign identity, leads to a slew of other requirements and properties. You might also notice that WeChat is far from alone in this, even if their identity requirements might be a bit stringent than, say, Telegram. Their control aspects are also a bit more extreme, at least for someone with western sensibilities 2.

    +

    Most user facing applications have similar issues, Google Apps/FB/Twitter etc. For popular tools that have this built in, we can look at git - which is truly decentralized and have keypair at the bottom. It is for a very specific technical domain, and even then people rely on Github. Key management is fairly difficult even for technical people, and for normal people even more so. Banks are generally far behind on this tech, relying on arcane procedures and special purpose hardware for 2FA. That's another big issue.

    +

    Let's shift gears a bit and talk about some other functional requirements.

    +

    Routing - packets from A to B​

    +

    In order to get a lot of the features WeChat provides, we need the ability to do three things: communicate, store data, and transact with people. We need a bit more than that, but let's focus on this for now.

    +

    To communicate with people, in the base case, we need to go from one phone to another phone that is separated by a large distance. This requires some form of routing. The most natural platform to build this on is the existing Internet, though not the only one. Most phones are resource restricted, and are only "on" for brief periods of time. This is needed to preserve battery and bandwidth. Additionally, Internet uses IPs as endpoints, which change as a phones move through space. NAT punching etc isn't always perfect either. This means we need a way to get a message from one public key to another, and through some intermediate nodes. We can think of these nodes as a form of service network. Similar to how a power grid works, or phone lines, or collection of ISPs.

    +

    One important property here is to ensure we don't end up in a situation like the centralized capture scenario above, something we've seen with centralized ISPs 3 4 where they can choose which traffic is good and which is bad. We want to allow the use of different service nodes, just like if a restaurant gives you food poisioning you can go to the one next door and then the first one goes out of business after a while. And the circle of life continues.

    +

    We shouldn't be naive though, and think that this is something nodes are likely to do for free. They need to be adequately compensated for their services, in some of incentivization scheme. That can either be monetary, or as in the case of Bittorrent, more of a barter situation where you use game theory to coordinate with strangers 5, and some form of reputation attached to it (for private trackers).

    +

    There are many ways of doing routing, and we won't go into too much technical detail here. Suffice to say is that you likely want both a structured and unstructured alternative, and that these comes with several trade-offs when it comes to efficiency, metadata protection, ability to incentivize, compatibility with existing topologies, and suitability for mobilephones (mostly offline, bandwidth restricted, not directly connectable). Expect more on this in a future article.

    +

    Some of these considerations naturally leads us into the storage and transaction components.

    +

    Storage - available and persistant for later​

    +

    If mobile phones are mostly offline, we need some way to store these messages so they can be retrieved when online again. The same goes for various kinds attachments as well, and for when people are switching devices. A user might control their timeline, but in the WeChat case that timeline is stored on Tencent's servers, and queried from there as well. This naturally needs to happen by some other service nodes. In the WeChat case, and for most IMs, the way these servers are paid for is through some indirect ad mechanism. The entity controlling these ads and so on is the same one as the one operating the servers for storage. A more direct model with different entities would see these services being compensated for their work.

    +

    We also need storage for attachments, mini-apps, as well as a way of understanding the current state of consensus when it comes to the compute/transact module. In the WeChat case, this state is completely handled by the bank institution or one of their partners, such as Alibaba. When it comes to bearer instruments like cash, no state needs to be kept as that's a direct exchange in the physical world. This isn't directly compatible with transfering value over a distance.

    +

    All of this state requires availability and persistance. It should be done in a trust minimized fashion and decentralized, which requires some form of incentivization for keeping data around. If it isn't, you are relying on social cohesion which breaks down at very large scales.

    +

    Since data will be spread out across multiple nodes, you need a way to sync data and transfer it in the network. As well as being able to add and query data from it. All of this requires a routing component.

    +

    To make it more censorship resistant it might be better to keep it as a general-purpose store, i.e. individuals don't need to know what they storing. Otherwise, you naturally end up in a situation where individual nodes can be pressured to not store certain content.

    +

    Messaging - from me to you to all of us (not them)​

    +

    This builds on top of routing, but it has a slightly different focus. The goal is to allow for individuals and groups to communicate in a private, secure and censorship-resistant manner.

    +

    It also needs to provide a decent interface to the end user, in terms of dealing seamlessly with offline messages, providing reliable and timely messaging.

    +

    In order to get closer to the ideal of total anonymity, it is useful to be able to hide metadata of who is talking to whom. This applies to both normal communication as well as for transactions. Ideally, no one but the parties involved can see who is taking part in a conversation. This can be achieved through various techniques such as mixnets, anonymous credentials, private information retrieval, and so on. Many of these techniques have a fundamental trade-off with latency and bandwidth, something that is a big concern for mobilephones. Being able to do some form of tuning, in an adaptive node manner, depending on your threat model and current capabilities is useful here.

    +

    The baseline here is pseudonymity, and having tools to allow individuals to "cut off" ties to their real world identity and transactions. People act different in different circles in the real world, and this should be mimicked online as well. Your company, family or government shouldn't be able to know what exactly you use your paycheck for, and who you are talking to.

    +

    Compute - transact, contract and settle​

    +

    The most immediate need here is transaction from A to B. Direct exchange. There is also a more indirect need for private lawmaking and contracting.

    +

    We talked about routing and storage and how they likely need to be incentivized to work properly. How are they going to be compensated? While this could in theory work via existing banking system and so on, this would be rather heavy. It'd also very likely require tying your identifier to your legal name, something that goes against what we want to achieve. What we want is something that acts more as right-to-access, similar to the way cash functions in a society 6. I pay for a fruit with something that is valuable to you and then I'm on my way.

    +

    While there might be other candidates, such as pre-paid debit cards and so on, this transaction mode pretty much requires a cryptocurrency component. The alternative is to do it on a reputation basis, which might work for small communities, due to social cohesion, but quickly detoriates for large ones 7. Ad hoc models like private Bittorrent trackers are centralized and easy to censor.

    +

    Now, none of the existing cryptocurrency models are ideal. They also all suffer from lack of widespread use, and it is difficult to get onboarded to them in the first place. Transactions in Bitcoin are slow. Ethereum is faster and has more capabilities, but it still suffers from linking payments over time, which makes the privacy part of this more difficult. Zcash, Monero and similar are interesting, but also require more use. For Zcash, shielded transactions appear to only account for less than 2% of all transactions in 2019 8 9.

    +

    Another dimension is what sets general purpose cryptocurrencies like Ethereum apart. Aside from just paying from A to B, you can encode rules about when something should be paid out and not. This is very useful for doing a form of private lawmaking, contracting, for setting up service agreements with these nodes. If there's no trivial recourse as in the meatspace world, where you know someone's name and you can sue them, you need a different kind of model.

    +

    What makes something like Zcash interesting is that it works more like digital cash. Instead of leaving a public trail for everyone, where someone can see where you got the initial money from and then trace you across various usage, for Zcash every hop is privacy preserving.

    +

    To fulfill the general goals of being censorship resistance and secure, it is also vital that the system being used stays online and can't be easily disrupted. That points to disintermediation, as opposed to using gateways and exchanges. This is a case where something like cash, or gold, is more direct, since no one can censor this transaction without being physically present where this direct exchange is taking place. However, like before, this doesn't work over distance.

    +

    Secure chat - just our business​

    +

    Similar to the messaging module above. The distinction here is that we assume the network part has already taken place. Here we are interested in keeping the contents of messages private, so that means confidentiality/end-to-end encryption, integrity, authentication, as well as forward secrecy and plausible deniability. This means that even if there's some actor that gets some private key material, or confiscated your phone, there is some level of...ephemerality to your conversations. Another issue here in terms of scalable private group chat.

    +

    Extensible mini apps​

    +

    This relates to the compute and storage module above. Essentially we want to provide mini apps as in WeChat, but to do so in a way that is compatible with what we want to achieve more generally. This allows individuals and small businesses to create small tools for various purposes, and coordinate with strangers. E.g. booking a cab or getting an insurance, and so on.

    +

    This has a higher dependency on the contracting/general computation aspect. I.e. often it isn't only a transaction, but you might want to encode some specific rules here that strangers can abide by without having too high trust requirements. As a simple example: escrows.

    +

    This also needs an open API that anyone can use. It should be properly secured, so using one doesn't compromise the rest of the system it is operating in. To be censorship resistant it requires the routing and storage component to work properly.

    +

    Where are we now?​

    +

    Let's look back at some of desirable properties we set out in the beginning and see how close we are to building out the necessary components. Is it realistic at all or just a pipe dream? We'll see that there are many building blocks in place, and there's reason for hope.

    +

    Self-sovereignity identity. Public key crypto and web of trust like constructs makes this possible.

    +

    Pseudonymity, and ideally total anonymity. Pseudonymity can largely be achieved with public key crypto and open systems that allow for permissionless participation. For transactions, pseudonymity exists in most cryptocurrencies. The challenge is linkage across time, especially when interfacing with other "legacy" system. There are stronger constructs that are actively being worked on and are promising here, such as mixnets (Nym), mixers (Wasabi Wallet, Tornado.Cash) and zero knowledge proofs (Zcash, Ethereum, Starkware). This area of applied research has exploded over the last few years.

    +

    Private and secure communication. Signal has pioneered a lot of this, following OTR. Double Ratchet, X3DH. E2EE is minimum these days, and properties like PFS and PD are getting better. For metadata protection, you have Tor, with its faults, and more active research on mixnets and private information retrieval, etc.

    +

    Censorship-resistance. This covers a lot of ground across the spectrum. You have technologies like Bittorrent, Bitcoin/Ethereum, Tor obfuscated transports, E2EE by default, partial mesh networks in production, abilit to move/replicate host machines more quickly have all made this more of a reality than it used to be. this easier. Of course, techniques such as deep packet inspection and internet shutdowns have increased.

    +

    Decentralization. Cryptocurrencies, projects like libp2p and IPFS. Need to be mindful here of many projects that claim decentralization but are still vulnerable to single points of failures, such as relying on gateways.

    +

    Built for mass adoption. This one is more subjective. There's definitely a lot of work to be done here, both when it comes to fundamental performance, key management and things like social discoverability. Directionally these things are improving and becoming easier for the average person but there is a lot ot be done here.

    +

    Scalability. With projects like Ethereum 2.0 and IPFS more and more resources are a being put into this, both at the consensus/compute layer as well as networking (gossip, scalable Kademlia) layer. Also various layer 2 solutions for transactions.

    +

    Fundamentals in place to support great user experience. Similar to built for mass adoption. As scalability becomes more important, more applied research is being done in the p2p area to improve things like latency, bandwidth.

    +

    Works for resource restricted devices, including smartphones. Work in progress and not enough focus here, generally an after thought. Also have stateless clients etc.

    +

    Adaptive nodes. See above. With subprotocols and capabilities in Ethereum and libp2p, this is getting easier.

    +

    Sustainable. Token economics is a thing. While a lot of it won't stay around, there are many more projects working on making themselves dispensable. Being open source, having an engaged community and enabling users run their own infrastructure. Users as stakeholders.

    +

    Spam resistant. Tricky problem if you want to be pseudonymous, but some signs of hope with incentivization mechanisms, zero knowledge based signaling, etc. Together with various forms of rate limiting and better controlling of topology and network amplification. And just generally being battle-tested by real world attacks, such as historical Ethereum DDoS attacks.

    +

    Trust minimized. Bitcoin. Zero knowledge provable computation. Open source. Reproducible builds. Signed binaries. Incentive compatible structures. Independent audits. Still a lot of work, but getting better.

    +

    Open source. Big and only getting bigger. Including mainstream companies.

    +

    What's next?​

    +

    We've look at what WeChat provides and what we'd like an alternative to look like. We've also seen a few principal modules that are necessary to achieve those goals. To achieve all of this is a daunting task, and one might call it overly ambitiuous. We've also seen how far we've come with some of the goals, and how a lot of the pieces are there, in one form or another. Then it is a question of putting them all together in the right mix.

    +

    The good news is that a lot of people are working all these building blocks and thinking about these problems. Compared to a few years ago we've come quite far when it comes to p2p infrastructure, privacy, security, scalability, and general developer mass and mindshare. If you want to join us in building some of these building blocks, and assembling them, check out our forum.

    +

    PS. We are hiring protocol engineers. DS

    +

    Acknowledgements​

    +

    Corey, Dean, Jacek.

    +

    References​

    + +

    Footnotes​

    +
      +
    1. +

      Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws ↩

      +
      2. +
    2. +

      On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/ ↩

      +
      3. +
    3. +

      Net Neutrality: https://www.eff.org/issues/net-neutrality ↩

      +
      4. +
    4. +

      ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/ ↩

      +
      5. +
    5. +

      Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf ↩

      +
      6. +
    6. +

      The Case for Electronic Cash: https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf ↩

      +
      7. +
    7. +

      Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html ↩

      +
      8. +
    8. +

      Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions ↩

      +
      9. +
    9. +

      Shielded transactions usage (stats page 404s): https://z.cash/support/faq/ ↩

      +
      10. +
    +
    \ No newline at end of file diff --git a/rlog/zkVM-explorations/index.html b/rlog/zkVM-explorations/index.html index 07f480c7..8e116a7c 100644 --- a/rlog/zkVM-explorations/index.html +++ b/rlog/zkVM-explorations/index.html @@ -1,76 +1,323 @@ - + - -Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? | Vac Research + +Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? | Vac Research - - - - + + + + -
    -

    Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines?

    by
    24 min read

    Introduction

    The blockchain space is rapidly evolving, and with it, new technologies are emerging that promise enhanced privacy, scalability, and security. + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines?

    by
    24 min read
    +

    The blockchain space is rapidly evolving, and with it, new technologies are emerging that promise enhanced privacy, scalability, and security. As decentralized systems grow in complexity and usage, the need for secure and private computation has never been greater. Zero-knowledge virtual machines (zkVMs) are one such innovation, allowing for computations to be proven correct without revealing the underlying data. ZkVMs have enormous implications for privacy-preserving applications, decentralized finance (DeFi), and other blockchain-based use cases. -However, as the term "zkVM" becomes more widely adopted, it is critical to distinguish between projects that truly satisfy the stringent requirements of a zkVM and those that do not.

    What is a zkVM?

    A zkVM is a virtual machine that combines the principles of cryptographic proof generation and privacy preservation with the computational model +However, as the term "zkVM" becomes more widely adopted, it is critical to distinguish between projects that truly satisfy the stringent requirements of a zkVM and those that do not.

    +

    What is a zkVM?

    +

    A zkVM is a virtual machine that combines the principles of cryptographic proof generation and privacy preservation with the computational model of traditional virtual machines. Essentially, a zkVM enables the execution of arbitrary programs while generating cryptographic proofs—specifically, zero-knowledge proofs (ZKPs)—that can verify the correctness of these computations without revealing any sensitive information. This ensures that computations can be trusted while protecting the privacy of the data involved. -The key characteristics of a zkVM include:

    • Proof generation: The ability to produce ZKPs that verify the correct execution of programs. There are several types of cryptographic techniques used in zkVMs to +The key characteristics of a zkVM include:

      +
        +
      • Proof generation: The ability to produce ZKPs that verify the correct execution of programs. There are several types of cryptographic techniques used in zkVMs to generate these proofs, such as zk-SNARKs, zk-STARKs, and recursive proofs. A zkVM’s ability to generate these proofs determines how effectively it can ensure the integrity of computations -in a privacy-preserving manner.
      • Privacy preservation: The system must maintain privacy, ensuring that only the proof is revealed, not the underlying computation or data. Privacy-preserving zkVMs allow users to maintain +in a privacy-preserving manner.
        • +
      • Privacy preservation: The system must maintain privacy, ensuring that only the proof is revealed, not the underlying computation or data. Privacy-preserving zkVMs allow users to maintain confidentiality without compromising the security or verifiability of their operations. However, not all zkVMs achieve the same level of privacy. Some may focus more on proof generation -and scalability while deprioritizing privacy features, which can limit their use in certain privacy-sensitive applications.
      • Scalability and performance: zkVMs should offer scalable and efficient computation, leveraging advanced cryptographic techniques like zk-SNARKs, zk-STARKs, or recursive proofs. -A zkVM's performance must also be measured in terms of latency (time to generate and verify a proof) and throughput (number of computations processed within a certain time frame).
      • Verifiable computation: The zkVM should be able to prove the execution of arbitrary programs in a secure and verifiable manner. Verifiable computation ensures that zkVMs can be deployed -across a wide range of applications, from DeFi to private data-sharing platforms and more.

      Why zkVMs matter

      The rise of zkVMs is a crucial development for the future of blockchain and decentralized technologies. As more systems require the ability to scale while maintaining privacy and trust, -zkVMs provide a powerful solution. They offer the potential to reshape the way decentralized applications (dapps) handle sensitive information, enabling them to be both efficient and private.

      It is essential to distinguish between projects that fully realize the potential of zkVMs and those that do not. In the remainder of this post, we evaluate several zkVM projects, analyzing -whether they satisfy the criteria for being classified as zkVMs based on our research.

      Our methodology

      We analyzed each project’s documentation, source code, and available benchmarks to determine whether they meet the definition of a zkVM. -Our criteria focus on the key capabilities of zkVMs—proof generation, privacy, scalability, and integration with existing systems.

      ZkVM project analysis

      1. [SP1]​

      • Overview: SP1 [1] is a developer-friendly zkVM designed to enable ZKP execution for LLVM-based languages like C, C++, Rust, and others. It supports a RISC-V-like instruction set architecture (ISA), -which makes it compatible with various programming languages compiled through LLVM.
      • Main focus: The main focus of SP1 is scalability, open-source contributions, and accessibility for developers. It prioritizes performance over privacy, -making it a good fit for environments where privacy isn't the primary concern.
      • Privacy: Not explicitly mentioned, making it less suitable for privacy-preserving applications.
      • Performance: SP1 has demonstrated up to 5.4x better performance than similar zkVMs like RISC0 for specific computations such as Fibonacci sequence generation.
      • Integration: SP1 is highly adaptable for rollups, light client verifiers, oracles, and even web2 projects like verifying the originality of images.
      • Conclusion: Yes, SP1 is a zkVM, but it does not prioritize zero-knowledge privacy, focusing more on scalability and performance.

      2. [Nexus]​

      • Overview: Nexus [2] is a highly modular zkVM designed to process up to a trillion CPU cycles per second. It relies on RISC-V instructions for computation, making it extensible and scalable. -However, it currently lacks full ZKP capabilities due to its use of Spartan proofs.
      • Main focus: Nexus focuses on high performance and scalability, aiming to create an efficient execution environment for computationally intensive tasks.
      • Privacy: Although zero-knowledge privacy isn't the primary feature of Nexus, the project hints at potential privacy enhancements in the future.
      • Performance: Nexus has a high theoretical throughput, but it has yet to demonstrate benchmarks on zero-knowledge privacy.
      • Integration: Nexus is a good fit for high-performance environments that do not necessarily require full privacy.
      • Conclusion: Yes, Nexus qualifies as a zkVM in terms of scalability and proof generation, but it does not yet achieve full zero-knowledge privacy.

      3. [RISC0]​

      • Overview: Risc0 [3] is a general-purpose zkVM with strong developer support. It allows for the execution of Rust and C code on a RISC-V virtual machine -and generates zk-SNARK and zk-STARK proofs for these computations.
      • Main focus: Risc0 is focused on ease of use for developers by abstracting away the complexities of circuit generation, making it accessible for a wide range of use cases.
      • Privacy: Full zero-knowledge privacy is supported via zk-SNARK and zk-STARK proofs, with Groth16 used for constant-size proof generation.
      • Performance: Risc0 offers strong benchmarks across different hardware setups, making it one of the most versatile zkVMs in terms of performance and scalability.
      • Integration: Risc0 integrates with several ecosystems, including Ethereum, and supports verifiable execution of Rust-based programs.
      • Conclusion: Yes, Risc0 qualifies as a zkVM, offering a balance of developer usability, scalability, and privacy.

      4. [Powdr]​

      • Overview: Powdr [4] is a toolkit for creating custom zkVMs. It allows developers to select from various front-end and back-end components to create zkVMs tailored to specific needs.
      • Main focus: Powdr is focused on providing a modular architecture for zkVM creation. It enables flexibility by allowing the combination of different ZK-proof backends like Halo2 or Valida.
      • Privacy: Powdr itself does not generate ZKPs, but it facilitates the creation of zkVMs that do.
      • Performance: The performance depends on the components chosen by the developer, as Powdr itself is more of a framework.
      • Integration: Powdr is highly customizable and can integrate with existing zkVM frameworks to extend their capabilities.
      • Conclusion: No, Powdr is not a zkVM itself, but it is a powerful tool for building customized zkVMs with different privacy and performance needs.

      5. [ZkMIPS]​

      • Overview: ZkMIPS [5] uses zk-STARKs to ensure privacy during computation, ensuring that private inputs are preserved while still proving correctness.
      • Performance: ZkMIPS is built for scalability, though explicit benchmarks are not widely published.
      • Integration: ZkMIPS can be integrated into systems that rely on MIPS architecture, making it versatile for legacy codebases that require privacy.
      • Conclusion: Yes, zkMIPS is a zkVM focused on scalability and privacy for MIPS-based architectures.

      6. [Valida]​

      • Overview: Valida [6] is a performance-oriented zkVM that generates proofs for programs using a custom ISA designed to optimize zkVM implementation. -It uses Plonky3 for its proof system.
      • Main focus: Valida is centered around optimizing prover performance and extensibility, making it a valuable tool for generating proofs efficiently.
      • Privacy: While Valida is focused on performance, it does not prioritize zero-knowledge privacy as much as other zkVMs.
      • Performance: Valida has benchmarks indicating its performance advantages in proving computations quickly, particularly through parallel processing.
      • Integration: Valida is specialized and may not integrate as seamlessly into general-purpose systems, as it is optimized for performance over broad applicability.
      • Conclusion: Yes, Valida qualifies as a zkVM based on proof generation, but its lack of focus on privacy makes it less suitable for privacy-first use cases.

      7. [Jolt]​

      • Overview: Jolt [7] is a zkVM built to optimize prover performance using a modified Hyrax polynomial commitment system. It relies on RISC-V instructions for computation -but falls short of full zero-knowledge capabilities.
      • Main focus: Jolt's main goal is to optimize the speed of proving program execution, making it suitable for high-performance applications where privacy isn't the primary concern.
      • Privacy: Jolt does not fully achieve zero-knowledge privacy due to the choice of polynomial commitment schemes.
      • Performance: Jolt offers strong performance, with benchmarks highlighting its ability to process proofs efficiently.
      • Integration: Jolt can be integrated with systems that prioritize speed over privacy, particularly where rapid proof generation is essential.
      • Conclusion: Yes, Jolt qualifies as a zkVM based on proof generation, though it does not provide full zero-knowledge privacy.

      8. [ZkWASM]​

      • Overview: ZkWASM [8] is a zkVM designed to execute WebAssembly (WASM) code in a privacy-preserving and scalable manner. It uses zk-SNARKs to prove the correctness of WASM -program execution while ensuring privacy.
      • Main focus: ZkWASM focuses on scalability and privacy for WebAssembly, making it ideal for dapps that require verifiable computation without compromising privacy.
      • Privacy: Full zero-knowledge privacy is provided through zk-SNARKs, ensuring that the execution of WASM programs remains confidential.
      • Performance: ZkWASM is optimized for running WASM programs efficiently, with offchain computation and onchain verification to enhance performance.
      • Integration: ZkWASM is ideal for dapps, particularly those that use WebAssembly and require verifiable execution.
      • Conclusion: Yes, zkWASM qualifies as a zkVM, providing strong privacy, scalability, and verifiable execution for WebAssembly code.

      9. [Aleo]​

      • Overview: Aleo's [9] snarkVM converts code into Aleo instructions, which are then compiled into bytecode executable on its zkVM. Aleo emphasizes building private, scalable dapps.
      • Main focus: Aleo prioritizes privacy and scalability for dapps, providing a robust framework for developers building private dapps.
      • Privacy: Aleo offers full privacy through zk-SNARK proofs, making it suitable for building fully private applications.
      • Performance: Aleo focuses on scalability through efficient proof systems, though detailed performance benchmarks are not widely available.
      • Integration: Aleo is built for privacy-first dapps and integrates with other zkVM-based systems.
      • Conclusion: Yes, Aleo qualifies as a zkVM, offering a comprehensive solution for private and scalable dapps.

      10. [Ola]​

      • Overview: Ola [10] is a ZK-friendly, high-performance layer-2 (L2) rollup platform that is still under development. It is designed to execute computations offchain while generating -validity proofs for these computations, ensuring that they are correctly executed without compromising security.
      • Privacy: Ola does not specifically prioritize privacy in the same way that zkVMs do. While it leverages ZKPs for scalability, its focus is on proving the correctness of -transactions and computations rather than ensuring that the data remains private.
      • Performance: Ola is designed to achieve high performance, particularly in terms of transaction throughput.
      • Integration: Ola is designed to be interoperable with various layer-1 blockchains. The platform supports a hybrid ZK-rollup architecture and is expected to include bridges for cross-chain -interoperability, enabling assets and data to move seamlessly between the layer-1 blockchain and the Ola rollup.
      • Conclusion: No, Ola is not a zkVM. While it leverages ZKPs (in the form of ZK-rollups) to ensure the validity of offchain computations, its primary focus is on scalability and performance +and scalability while deprioritizing privacy features, which can limit their use in certain privacy-sensitive applications.
        • +
      • Scalability and performance: zkVMs should offer scalable and efficient computation, leveraging advanced cryptographic techniques like zk-SNARKs, zk-STARKs, or recursive proofs. +A zkVM's performance must also be measured in terms of latency (time to generate and verify a proof) and throughput (number of computations processed within a certain time frame).
        • +
      • Verifiable computation: The zkVM should be able to prove the execution of arbitrary programs in a secure and verifiable manner. Verifiable computation ensures that zkVMs can be deployed +across a wide range of applications, from DeFi to private data-sharing platforms and more.
        • +
      +

      Why zkVMs matter

      +

      The rise of zkVMs is a crucial development for the future of blockchain and decentralized technologies. As more systems require the ability to scale while maintaining privacy and trust, +zkVMs provide a powerful solution. They offer the potential to reshape the way decentralized applications (dapps) handle sensitive information, enabling them to be both efficient and private.

      +

      It is essential to distinguish between projects that fully realize the potential of zkVMs and those that do not. In the remainder of this post, we evaluate several zkVM projects, analyzing +whether they satisfy the criteria for being classified as zkVMs based on our research.

      +

      Our methodology

      +

      We analyzed each project’s documentation, source code, and available benchmarks to determine whether they meet the definition of a zkVM. +Our criteria focus on the key capabilities of zkVMs—proof generation, privacy, scalability, and integration with existing systems.

      +

      ZkVM project analysis

      +

      1. [SP1]​

      +
        +
      • Overview: SP1 [1] is a developer-friendly zkVM designed to enable ZKP execution for LLVM-based languages like C, C++, Rust, and others. It supports a RISC-V-like instruction set architecture (ISA), +which makes it compatible with various programming languages compiled through LLVM.
        • +
      • Main focus: The main focus of SP1 is scalability, open-source contributions, and accessibility for developers. It prioritizes performance over privacy, +making it a good fit for environments where privacy isn't the primary concern.
        • +
      • Privacy: Not explicitly mentioned, making it less suitable for privacy-preserving applications.
        • +
      • Performance: SP1 has demonstrated up to 5.4x better performance than similar zkVMs like RISC0 for specific computations such as Fibonacci sequence generation.
        • +
      • Integration: SP1 is highly adaptable for rollups, light client verifiers, oracles, and even web2 projects like verifying the originality of images.
        • +
      • Conclusion: Yes, SP1 is a zkVM, but it does not prioritize zero-knowledge privacy, focusing more on scalability and performance.
        • +
      +

      2. [Nexus]​

      +
        +
      • Overview: Nexus [2] is a highly modular zkVM designed to process up to a trillion CPU cycles per second. It relies on RISC-V instructions for computation, making it extensible and scalable. +However, it currently lacks full ZKP capabilities due to its use of Spartan proofs.
        • +
      • Main focus: Nexus focuses on high performance and scalability, aiming to create an efficient execution environment for computationally intensive tasks.
        • +
      • Privacy: Although zero-knowledge privacy isn't the primary feature of Nexus, the project hints at potential privacy enhancements in the future.
        • +
      • Performance: Nexus has a high theoretical throughput, but it has yet to demonstrate benchmarks on zero-knowledge privacy.
        • +
      • Integration: Nexus is a good fit for high-performance environments that do not necessarily require full privacy.
        • +
      • Conclusion: Yes, Nexus qualifies as a zkVM in terms of scalability and proof generation, but it does not yet achieve full zero-knowledge privacy.
        • +
      +

      3. [RISC0]​

      +
        +
      • Overview: Risc0 [3] is a general-purpose zkVM with strong developer support. It allows for the execution of Rust and C code on a RISC-V virtual machine +and generates zk-SNARK and zk-STARK proofs for these computations.
        • +
      • Main focus: Risc0 is focused on ease of use for developers by abstracting away the complexities of circuit generation, making it accessible for a wide range of use cases.
        • +
      • Privacy: Full zero-knowledge privacy is supported via zk-SNARK and zk-STARK proofs, with Groth16 used for constant-size proof generation.
        • +
      • Performance: Risc0 offers strong benchmarks across different hardware setups, making it one of the most versatile zkVMs in terms of performance and scalability.
        • +
      • Integration: Risc0 integrates with several ecosystems, including Ethereum, and supports verifiable execution of Rust-based programs.
        • +
      • Conclusion: Yes, Risc0 qualifies as a zkVM, offering a balance of developer usability, scalability, and privacy.
        • +
      +

      4. [Powdr]​

      +
        +
      • Overview: Powdr [4] is a toolkit for creating custom zkVMs. It allows developers to select from various front-end and back-end components to create zkVMs tailored to specific needs.
        • +
      • Main focus: Powdr is focused on providing a modular architecture for zkVM creation. It enables flexibility by allowing the combination of different ZK-proof backends like Halo2 or Valida.
        • +
      • Privacy: Powdr itself does not generate ZKPs, but it facilitates the creation of zkVMs that do.
        • +
      • Performance: The performance depends on the components chosen by the developer, as Powdr itself is more of a framework.
        • +
      • Integration: Powdr is highly customizable and can integrate with existing zkVM frameworks to extend their capabilities.
        • +
      • Conclusion: No, Powdr is not a zkVM itself, but it is a powerful tool for building customized zkVMs with different privacy and performance needs.
        • +
      +

      5. [ZkMIPS]​

      +
        +
      • Overview: ZkMIPS [5] uses zk-STARKs to ensure privacy during computation, ensuring that private inputs are preserved while still proving correctness.
        • +
      • Performance: ZkMIPS is built for scalability, though explicit benchmarks are not widely published.
        • +
      • Integration: ZkMIPS can be integrated into systems that rely on MIPS architecture, making it versatile for legacy codebases that require privacy.
        • +
      • Conclusion: Yes, zkMIPS is a zkVM focused on scalability and privacy for MIPS-based architectures.
        • +
      +

      6. [Valida]​

      +
        +
      • Overview: Valida [6] is a performance-oriented zkVM that generates proofs for programs using a custom ISA designed to optimize zkVM implementation. +It uses Plonky3 for its proof system.
        • +
      • Main focus: Valida is centered around optimizing prover performance and extensibility, making it a valuable tool for generating proofs efficiently.
        • +
      • Privacy: While Valida is focused on performance, it does not prioritize zero-knowledge privacy as much as other zkVMs.
        • +
      • Performance: Valida has benchmarks indicating its performance advantages in proving computations quickly, particularly through parallel processing.
        • +
      • Integration: Valida is specialized and may not integrate as seamlessly into general-purpose systems, as it is optimized for performance over broad applicability.
        • +
      • Conclusion: Yes, Valida qualifies as a zkVM based on proof generation, but its lack of focus on privacy makes it less suitable for privacy-first use cases.
        • +
      +

      7. [Jolt]​

      +
        +
      • Overview: Jolt [7] is a zkVM built to optimize prover performance using a modified Hyrax polynomial commitment system. It relies on RISC-V instructions for computation +but falls short of full zero-knowledge capabilities.
        • +
      • Main focus: Jolt's main goal is to optimize the speed of proving program execution, making it suitable for high-performance applications where privacy isn't the primary concern.
        • +
      • Privacy: Jolt does not fully achieve zero-knowledge privacy due to the choice of polynomial commitment schemes.
        • +
      • Performance: Jolt offers strong performance, with benchmarks highlighting its ability to process proofs efficiently.
        • +
      • Integration: Jolt can be integrated with systems that prioritize speed over privacy, particularly where rapid proof generation is essential.
        • +
      • Conclusion: Yes, Jolt qualifies as a zkVM based on proof generation, though it does not provide full zero-knowledge privacy.
        • +
      +

      8. [ZkWASM]​

      +
        +
      • Overview: ZkWASM [8] is a zkVM designed to execute WebAssembly (WASM) code in a privacy-preserving and scalable manner. It uses zk-SNARKs to prove the correctness of WASM +program execution while ensuring privacy.
        • +
      • Main focus: ZkWASM focuses on scalability and privacy for WebAssembly, making it ideal for dapps that require verifiable computation without compromising privacy.
        • +
      • Privacy: Full zero-knowledge privacy is provided through zk-SNARKs, ensuring that the execution of WASM programs remains confidential.
        • +
      • Performance: ZkWASM is optimized for running WASM programs efficiently, with offchain computation and onchain verification to enhance performance.
        • +
      • Integration: ZkWASM is ideal for dapps, particularly those that use WebAssembly and require verifiable execution.
        • +
      • Conclusion: Yes, zkWASM qualifies as a zkVM, providing strong privacy, scalability, and verifiable execution for WebAssembly code.
        • +
      +

      9. [Aleo]​

      +
        +
      • Overview: Aleo's [9] snarkVM converts code into Aleo instructions, which are then compiled into bytecode executable on its zkVM. Aleo emphasizes building private, scalable dapps.
        • +
      • Main focus: Aleo prioritizes privacy and scalability for dapps, providing a robust framework for developers building private dapps.
        • +
      • Privacy: Aleo offers full privacy through zk-SNARK proofs, making it suitable for building fully private applications.
        • +
      • Performance: Aleo focuses on scalability through efficient proof systems, though detailed performance benchmarks are not widely available.
        • +
      • Integration: Aleo is built for privacy-first dapps and integrates with other zkVM-based systems.
        • +
      • Conclusion: Yes, Aleo qualifies as a zkVM, offering a comprehensive solution for private and scalable dapps.
        • +
      +

      10. [Ola]​

      +
        +
      • Overview: Ola [10] is a ZK-friendly, high-performance layer-2 (L2) rollup platform that is still under development. It is designed to execute computations offchain while generating +validity proofs for these computations, ensuring that they are correctly executed without compromising security.
        • +
      • Privacy: Ola does not specifically prioritize privacy in the same way that zkVMs do. While it leverages ZKPs for scalability, its focus is on proving the correctness of +transactions and computations rather than ensuring that the data remains private.
        • +
      • Performance: Ola is designed to achieve high performance, particularly in terms of transaction throughput.
        • +
      • Integration: Ola is designed to be interoperable with various layer-1 blockchains. The platform supports a hybrid ZK-rollup architecture and is expected to include bridges for cross-chain +interoperability, enabling assets and data to move seamlessly between the layer-1 blockchain and the Ola rollup.
        • +
      • Conclusion: No, Ola is not a zkVM. While it leverages ZKPs (in the form of ZK-rollups) to ensure the validity of offchain computations, its primary focus is on scalability and performance rather than privacy or verifiable execution of arbitrary programs. Ola is more accurately described as a ZK-rollup platform aimed at improving transaction throughput and reducing transaction costs on -layer-1 blockchains.

      11. [Miden]​

      • Overview: Miden zkVM [11] is a zk-STARK-based virtual machine that converts code into Miden VM instructions and proves the execution of these instructions with zero-knowledge privacy.
      • Main focus: Miden focuses on scalability and privacy for ZK-rollups, offering efficient proof generation for dapps.
      • Privacy: Miden ensures privacy for transactions and programs via zk-STARK proofs, making it suitable for private dapps.
      • Performance: Miden is optimized for scalability, with benchmarks showing its ability to handle up to 1,000 transactions per second (TPS).
      • Integration: Miden integrates well with ZK-rollup solutions, making it ideal for L2 scaling solutions on blockchains like Ethereum.
      • Conclusion: Yes, Miden qualifies as a zkVM, providing strong privacy and scalability for dapps and ZK-rollups.

      12. [ZkOS]​

      • Overview: ZkOS [12] is a verifiable operating system focused on running zkApps in a decentralized manner. It is built on the RISC-V architecture and aims to create -a world computer where all untrusted executions can be verified.
      • Main focus: ZkOS is primarily designed to offer a proof-of-concept operating system where all executions can be verified in a trustless manner. -However, its focus is more on the infrastructure for verifiable applications rather than being a traditional zkVM.
      • Privacy: ZkOS does not focus on privacy guarantees such as those found in zkVMs that generate ZKPs.
      • Performance: ZkOS focuses on the efficient execution of dapps, but performance benchmarks specific to ZKP generation are not provided.
      • Integration: ZkOS supports the execution of zkApps, but it is more of a verifiable operating system rather than a zkVM, making it distinct in its functionality.
      • Conclusion: No, zkOS is not a zkVM. It is a verifiable operating system focused on the infrastructure to support zkApps but does not directly generate ZKPs or focus on privacy preservation.

      13. [Triton]​

      • Overview: Triton [13] is a domain-specific language (DSL) and compiler designed primarily for high-performance GPU kernels, particularly those used in deep learning applications.
      • Main focus: The primary goal of Triton is to optimize computation for machine learning and GPU workloads. It is focused on enhancing performance and efficiency in processing data -rather than on ZKPs or verifiable computation.
      • Privacy: Triton does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on high-performance computation rather than cryptographic verifiability.
      • Performance: Triton is highly optimized for GPU execution, offering significant improvements in performance for computationally intensive tasks such as those found in deep learning.
      • Integration: Triton is integrated with GPU-based computation environments and is highly specialized for optimizing low-level operations on hardware rather -than being a general-purpose virtual machine.
      • Conclusion: No, Triton is not a zkVM. It is a specialized tool for optimizing GPU workloads, focusing on performance rather than privacy or ZKPs.

      14. [Cairo]​

      • Overview: Cairo zkVM [14] uses a custom language that compiles to an optimized STARK-based proof system, ensuring verifiable computation. It is primarily used in systems like Starknet.
      • Main focus: Cairo focuses on scalability and performance, using zk-STARK proofs to ensure the verifiable and secure execution of programs.
      • Privacy: Cairo provides privacy through zk-STARKs, but it focuses more on scalability and performance than privacy-first use cases.
      • Performance: Cairo is highly optimized for performance, making it well-suited for scalable applications on Starknet.
      • Integration: Cairo integrates deeply with systems like Starknet, supporting verifiable computation in a highly scalable and efficient manner.
      • Conclusion: Yes, Cairo qualifies as a zkVM, focusing on performance and verifiable execution while being ZK-friendly.

      15. [SnarkOS]​

      • Overview: SnarkOS [15] is a decentralized operating system designed to power Aleo's network, enabling secure and private dapps. -It manages transactions and consensus, making it a critical infrastructure component for Aleo's zkVM-based ecosystem.
      • Main focus: SnarkOS primarily focuses on securing Aleo's network through consensus mechanisms and privacy-preserving transactions rather than acting as a -zkVM that directly proves program execution.
      • Privacy: SnarkOS supports zero-knowledge privacy through its integration with Aleo's zkVM, but the operating system itself does not generate ZKPs for arbitrary computations.
      • Performance: SnarkOS is optimized for managing dapps on the Aleo network and handling private transactions, but its focus is more on infrastructure -and consensus than on proof generation.
      • Integration: SnarkOS integrates seamlessly with Aleo's zkVM to support private dapps and transactions, but its primary role is as a consensus layer.
      • Conclusion: No, SnarkOS is not a zkVM. It serves as an operating system for Aleo's decentralized network, focusing on privacy and consensus rather than on generating ZKPs for computations.

      16. [Lurk]​

      • Overview: Lurk [16] is a Turing-complete programming language designed for recursive zk-SNARKs. It focuses on enabling developers to build complex, -recursive ZKPs efficiently through a custom language tailored for verifiable computation.
      • Main focus: Lurk is centered around recursive proof generation rather than serving as a traditional virtual machine. Its purpose is to facilitate the creation of complex zk-SNARK-based proofs, -making it a specialized tool for cryptographic proofs rather than general-purpose computation.
      • Privacy: Lurk is built for generating zk-SNARKs, which inherently provide privacy. However, Lurk itself is a language and not a zkVM that executes arbitrary programs and generates ZKPs for them.
      • Performance: Lurk is optimized for recursive zk-SNARK generation, but specific performance metrics are tied to its proof-generation capabilities rather than traditional execution environments.
      • Integration: Lurk is specialized for zk-SNARKs and may not easily integrate with other general-purpose systems, as it focuses on specific cryptographic tasks.
      • Conclusion: No, Lurk is not a zkVM. It is a programming language designed for recursive zk-SNARKs and focuses on proof generation rather than program execution in a virtual machine environment.

      17. [Piecrust]​

      • Overview: Piecrust [17] is a WASM-based zkVM designed to run on the Dusk Network. It supports concurrent execution and focuses on providing privacy and scalability for smart contracts.
      • Main focus: Piecrust is designed to provide private and efficient execution of smart contracts through the use of ZKPs.
      • Privacy: Piecrust supports ZK-friendly computations and enhances privacy through cryptographic primitives such as Merkle trees.
      • Performance: Piecrust is designed to be scalable and concurrent, allowing multiple sessions to run simultaneously, which improves overall performance.
      • Integration: Piecrust integrates with the Dusk Network and supports private smart contracts, making it ideal for dapps.
      • Conclusion: Yes, Piecrust qualifies as a zkVM, offering scalability, privacy, and support for succinct proof generation.

      18. [Ceno]​

      • Overview: Ceno [18] is a zkVM that provides a theoretical framework for reducing proving time by grouping common portions of code together. It uses recursive proofs to enhance prover efficiency.
      • Main focus: Ceno aims to optimize prover performance through recursive proofs, making it a powerful tool for handling complex computations efficiently.
      • Privacy: Ceno supports zero-knowledge privacy through recursive proofs and is designed to handle large-scale computations securely.
      • Performance: Ceno's recursive proof framework ensures that it can efficiently prove the execution of programs, reducing the time required for proof generation.
      • Integration: Ceno can be integrated into systems that require high efficiency and privacy, particularly those handling complex, repeated computations.
      • Conclusion: Yes, Ceno qualifies as a zkVM, providing efficient and private computation through the use of recursive proofs.

      19. [Stellar]​

      • Overview: Stellar [19] is a decentralized protocol designed to facilitate cross-border transactions between digital and fiat currencies.
      • Main focus: Stellar's primary goal is to improve financial transactions by enabling decentralized, low-cost currency transfers. It does not aim to provide ZKPs or run verifiable computations -like a zkVM.
      • Privacy: Stellar focuses on confidentiality and security for financial transactions, but it does not employ ZKPs in the way zkVMs do for verifying computation without revealing data.
      • Performance: Stellar prioritizes the performance of financial transactions, ensuring low latency and high throughput across its decentralized network. -However, this performance focus is specific to transactions rather than general-purpose program execution.
      • Integration: Stellar is designed for integration with financial systems, enabling currency conversions and transfers, but it is not built for executing smart contracts or verifiable computations.
      • Conclusion: No, Stellar is not a zkVM. It is a decentralized financial protocol focused on facilitating cross-border payments rather than verifiable or privacy-preserving computation.

      20. [NovaNet]​

      • Overview: NovaNet [20] is an open peer-to-peer network that aims to build upon concepts of non-uniform incremental verifiable computation.
      • Main focus: NovaNet's focus is on peer-to-peer networking and decentralized computing rather than on proving the execution of programs in a zero-knowledge manner.
      • Privacy: NovaNet does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on decentralized networking and computation.
      • Performance: NovaNet prioritizes efficient decentralized computation but does not focus on privacy or performance benchmarks related to ZKPs.
      • Integration: NovaNet is built for decentralized networks but is not designed to integrate with systems requiring verifiable computation or ZKP generation.
      • Conclusion: No, NovaNet is not a zkVM. It is a decentralized peer-to-peer network focused on distributed computing rather than zero-knowledge computation.

      21. [ZkLLVM]​

      • Overview: ZkLLVM [21] is a compiler that transforms C++ or Rust code into circuits for use in zk-SNARK or zk-STARK systems. Its primary purpose is to bridge high-level programming -languages with ZKP systems by compiling code into arithmetic circuits that can be used to generate and verify proofs.
      • Main focus: ZkLLVM focuses on making ZKPs accessible to developers by enabling them to write code in familiar languages (C++, Rust) and then compile that code into ZK circuits.
      • Privacy: ZkLLVM enables the generation of ZKPs by compiling high-level code into ZK-compatible circuits. It plays a crucial role in privacy-preserving applications but does not act -as a zkVM itself.
      • Performance: ZkLLVM allows for the performance of ZKPs to be closely tied to the complexity of the compiled circuits. The performance depends on the underlying -zk-SNARK or zk-STARK system used.
      • Integration: ZkLLVM integrates with zk-SNARK and zk-STARK proof systems, making it useful for a variety of privacy-focused applications, but it does not serve as a zkVM -for general-purpose computation.
      • Conclusion: No, zkLLVM is not a zkVM. It is a compiler that transforms high-level code into ZK circuits, enabling ZKPs but not acting as a virtual machine for executing and proving programs.

      22. [ZkMove]​

      • Overview: ZkMove [22] is a zkVM designed to execute smart contracts written in the Move language. It utilizes ZKPs to ensure that the execution of these contracts remains verifiable and secure.
      • Main focus: ZkMove focuses on privacy and verifiable execution for Move-based smart contracts, providing a framework for ZK-friendly computation.
      • Privacy: ZkMove ensures that smart contract execution remains private through ZKPs, making it suitable for privacy-preserving applications.
      • Performance: ZkMove is optimized for verifiable execution, ensuring that contracts can be proven correct while preserving privacy.
      • Integration: ZkMove integrates well with systems that use the Move language, particularly in environments that require private smart contract execution.
      • Conclusion: Yes, zkMove qualifies as a zkVM, offering ZK-friendly execution and privacy for smart contracts written in the Move language.

      23. [O1VM]​

      • Overview: O1VM [23] is a general-purpose zkVM developed by o1Labs. It is designed to prove the execution of MIPS programs efficiently through a combination of zk-SNARKs -and specialized techniques like folding schemes and RAMLookups.
      • Main focus: O1VM focuses on scalability and verifiable computation for MIPS-based programs, making it a strong contender for executing and proving complex programs efficiently.
      • Privacy: O1VM ensures privacy through zk-SNARK proofs, keeping the details of the computation private while proving its correctness.
      • Performance: O1VM is optimized for handling long execution traces and complex computations, making it highly scalable.
      • Integration: O1VM integrates well with MIPS-based architectures and systems that require privacy-preserving computation.
      • Conclusion: Yes, o1VM qualifies as a zkVM, providing privacy, scalability, and strong proof generation for MIPS programs.

      Summary of findings

      Project nameZkVM statusZero knowledgeReasoning/comments
      SP1YesNoProves execution of LLVM-based programs but lacks privacy features.
      NexusYesNoStrong proof generation but lacks zero-knowledge privacy due to Spartan.
      Risc0YesYesSupports full ZKP generation for Rust programs.
      PowdrNoYesToolkit for creating custom zkVMs, not a zkVM itself.
      ZkMIPSYesYesSupports MIPS-like architecture with full zero-knowledge and proof generation.
      ValidaYesNoPerformance-focused zkVM, lacks privacy guarantees.
      JoltYesNoPerformance-focused zkVM, does not achieve zero-knowledge privacy.
      ZkWASMYesYesFull zero-knowledge and verifiable execution of WebAssembly code.
      AleoYesYesFully private and scalable dapps.
      OlaNoNoPrimarily a ZK-rollup platform, not a zkVM, focusing on scalability and performance rather than privacy.
      MidenYesYesZk-STARK-based zkVM with strong privacy and scalability.
      ZkOSNoNoVerifiable operating system focused on zkApps, not a zkVM.
      TritonNoNoOptimizes GPU workloads but not designed for ZKPs.
      CairoYesZK-friendlyCustom Rust-based language with zk-STARK proof generation.
      SnarkOSNoYesDecentralized OS for Aleo's network, focuses on consensus rather than verifiable computation.
      LurkNoNoProgramming language for recursive zk-SNARKs, not a zkVM.
      PiecrustYesZK-friendlyZkVM with recursive SNARK capabilities, focused on succinct proof generation.
      CenoYesYesTheoretical zkVM improving prover efficiency through recursive proofs.
      StellarNoNoFocuses on cross-border transactions, not ZK-proof generation or verifiable computation.
      NovaNetNoNoPeer-to-peer network focused on distributed computing, not zero-knowledge computation.
      ZkLLVMNoYes, in some casesCompiler for generating ZK-circuits, not a zkVM.
      ZkMoveYesZK-friendlyZkVM supporting Move language with ZKP execution.
      O1VMYesYesMIPS-based zkVM with strong privacy, scalability, and proof generation.

      Insights and conclusions

      Our analysis reveals that many of the projects labeled as zkVMs do meet the core criteria for zkVMs, offering verifiable computation and proof generation +layer-1 blockchains.

      • +
    +

    11. [Miden]​

    +
      +
    • Overview: Miden zkVM [11] is a zk-STARK-based virtual machine that converts code into Miden VM instructions and proves the execution of these instructions with zero-knowledge privacy.
      • +
    • Main focus: Miden focuses on scalability and privacy for ZK-rollups, offering efficient proof generation for dapps.
      • +
    • Privacy: Miden ensures privacy for transactions and programs via zk-STARK proofs, making it suitable for private dapps.
      • +
    • Performance: Miden is optimized for scalability, with benchmarks showing its ability to handle up to 1,000 transactions per second (TPS).
      • +
    • Integration: Miden integrates well with ZK-rollup solutions, making it ideal for L2 scaling solutions on blockchains like Ethereum.
      • +
    • Conclusion: Yes, Miden qualifies as a zkVM, providing strong privacy and scalability for dapps and ZK-rollups.
      • +
    +

    12. [ZkOS]​

    +
      +
    • Overview: ZkOS [12] is a verifiable operating system focused on running zkApps in a decentralized manner. It is built on the RISC-V architecture and aims to create +a world computer where all untrusted executions can be verified.
      • +
    • Main focus: ZkOS is primarily designed to offer a proof-of-concept operating system where all executions can be verified in a trustless manner. +However, its focus is more on the infrastructure for verifiable applications rather than being a traditional zkVM.
      • +
    • Privacy: ZkOS does not focus on privacy guarantees such as those found in zkVMs that generate ZKPs.
      • +
    • Performance: ZkOS focuses on the efficient execution of dapps, but performance benchmarks specific to ZKP generation are not provided.
      • +
    • Integration: ZkOS supports the execution of zkApps, but it is more of a verifiable operating system rather than a zkVM, making it distinct in its functionality.
      • +
    • Conclusion: No, zkOS is not a zkVM. It is a verifiable operating system focused on the infrastructure to support zkApps but does not directly generate ZKPs or focus on privacy preservation.
      • +
    +

    13. [Triton]​

    +
      +
    • Overview: Triton [13] is a domain-specific language (DSL) and compiler designed primarily for high-performance GPU kernels, particularly those used in deep learning applications.
      • +
    • Main focus: The primary goal of Triton is to optimize computation for machine learning and GPU workloads. It is focused on enhancing performance and efficiency in processing data +rather than on ZKPs or verifiable computation.
      • +
    • Privacy: Triton does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on high-performance computation rather than cryptographic verifiability.
      • +
    • Performance: Triton is highly optimized for GPU execution, offering significant improvements in performance for computationally intensive tasks such as those found in deep learning.
      • +
    • Integration: Triton is integrated with GPU-based computation environments and is highly specialized for optimizing low-level operations on hardware rather +than being a general-purpose virtual machine.
      • +
    • Conclusion: No, Triton is not a zkVM. It is a specialized tool for optimizing GPU workloads, focusing on performance rather than privacy or ZKPs.
      • +
    +

    14. [Cairo]​

    +
      +
    • Overview: Cairo zkVM [14] uses a custom language that compiles to an optimized STARK-based proof system, ensuring verifiable computation. It is primarily used in systems like Starknet.
      • +
    • Main focus: Cairo focuses on scalability and performance, using zk-STARK proofs to ensure the verifiable and secure execution of programs.
      • +
    • Privacy: Cairo provides privacy through zk-STARKs, but it focuses more on scalability and performance than privacy-first use cases.
      • +
    • Performance: Cairo is highly optimized for performance, making it well-suited for scalable applications on Starknet.
      • +
    • Integration: Cairo integrates deeply with systems like Starknet, supporting verifiable computation in a highly scalable and efficient manner.
      • +
    • Conclusion: Yes, Cairo qualifies as a zkVM, focusing on performance and verifiable execution while being ZK-friendly.
      • +
    +

    15. [SnarkOS]​

    +
      +
    • Overview: SnarkOS [15] is a decentralized operating system designed to power Aleo's network, enabling secure and private dapps. +It manages transactions and consensus, making it a critical infrastructure component for Aleo's zkVM-based ecosystem.
      • +
    • Main focus: SnarkOS primarily focuses on securing Aleo's network through consensus mechanisms and privacy-preserving transactions rather than acting as a +zkVM that directly proves program execution.
      • +
    • Privacy: SnarkOS supports zero-knowledge privacy through its integration with Aleo's zkVM, but the operating system itself does not generate ZKPs for arbitrary computations.
      • +
    • Performance: SnarkOS is optimized for managing dapps on the Aleo network and handling private transactions, but its focus is more on infrastructure +and consensus than on proof generation.
      • +
    • Integration: SnarkOS integrates seamlessly with Aleo's zkVM to support private dapps and transactions, but its primary role is as a consensus layer.
      • +
    • Conclusion: No, SnarkOS is not a zkVM. It serves as an operating system for Aleo's decentralized network, focusing on privacy and consensus rather than on generating ZKPs for computations.
      • +
    +

    16. [Lurk]​

    +
      +
    • Overview: Lurk [16] is a Turing-complete programming language designed for recursive zk-SNARKs. It focuses on enabling developers to build complex, +recursive ZKPs efficiently through a custom language tailored for verifiable computation.
      • +
    • Main focus: Lurk is centered around recursive proof generation rather than serving as a traditional virtual machine. Its purpose is to facilitate the creation of complex zk-SNARK-based proofs, +making it a specialized tool for cryptographic proofs rather than general-purpose computation.
      • +
    • Privacy: Lurk is built for generating zk-SNARKs, which inherently provide privacy. However, Lurk itself is a language and not a zkVM that executes arbitrary programs and generates ZKPs for them.
      • +
    • Performance: Lurk is optimized for recursive zk-SNARK generation, but specific performance metrics are tied to its proof-generation capabilities rather than traditional execution environments.
      • +
    • Integration: Lurk is specialized for zk-SNARKs and may not easily integrate with other general-purpose systems, as it focuses on specific cryptographic tasks.
      • +
    • Conclusion: No, Lurk is not a zkVM. It is a programming language designed for recursive zk-SNARKs and focuses on proof generation rather than program execution in a virtual machine environment.
      • +
    +

    17. [Piecrust]​

    +
      +
    • Overview: Piecrust [17] is a WASM-based zkVM designed to run on the Dusk Network. It supports concurrent execution and focuses on providing privacy and scalability for smart contracts.
      • +
    • Main focus: Piecrust is designed to provide private and efficient execution of smart contracts through the use of ZKPs.
      • +
    • Privacy: Piecrust supports ZK-friendly computations and enhances privacy through cryptographic primitives such as Merkle trees.
      • +
    • Performance: Piecrust is designed to be scalable and concurrent, allowing multiple sessions to run simultaneously, which improves overall performance.
      • +
    • Integration: Piecrust integrates with the Dusk Network and supports private smart contracts, making it ideal for dapps.
      • +
    • Conclusion: Yes, Piecrust qualifies as a zkVM, offering scalability, privacy, and support for succinct proof generation.
      • +
    +

    18. [Ceno]​

    +
      +
    • Overview: Ceno [18] is a zkVM that provides a theoretical framework for reducing proving time by grouping common portions of code together. It uses recursive proofs to enhance prover efficiency.
      • +
    • Main focus: Ceno aims to optimize prover performance through recursive proofs, making it a powerful tool for handling complex computations efficiently.
      • +
    • Privacy: Ceno supports zero-knowledge privacy through recursive proofs and is designed to handle large-scale computations securely.
      • +
    • Performance: Ceno's recursive proof framework ensures that it can efficiently prove the execution of programs, reducing the time required for proof generation.
      • +
    • Integration: Ceno can be integrated into systems that require high efficiency and privacy, particularly those handling complex, repeated computations.
      • +
    • Conclusion: Yes, Ceno qualifies as a zkVM, providing efficient and private computation through the use of recursive proofs.
      • +
    +

    19. [Stellar]​

    +
      +
    • Overview: Stellar [19] is a decentralized protocol designed to facilitate cross-border transactions between digital and fiat currencies.
      • +
    • Main focus: Stellar's primary goal is to improve financial transactions by enabling decentralized, low-cost currency transfers. It does not aim to provide ZKPs or run verifiable computations +like a zkVM.
      • +
    • Privacy: Stellar focuses on confidentiality and security for financial transactions, but it does not employ ZKPs in the way zkVMs do for verifying computation without revealing data.
      • +
    • Performance: Stellar prioritizes the performance of financial transactions, ensuring low latency and high throughput across its decentralized network. +However, this performance focus is specific to transactions rather than general-purpose program execution.
      • +
    • Integration: Stellar is designed for integration with financial systems, enabling currency conversions and transfers, but it is not built for executing smart contracts or verifiable computations.
      • +
    • Conclusion: No, Stellar is not a zkVM. It is a decentralized financial protocol focused on facilitating cross-border payments rather than verifiable or privacy-preserving computation.
      • +
    +

    20. [NovaNet]​

    +
      +
    • Overview: NovaNet [20] is an open peer-to-peer network that aims to build upon concepts of non-uniform incremental verifiable computation.
      • +
    • Main focus: NovaNet's focus is on peer-to-peer networking and decentralized computing rather than on proving the execution of programs in a zero-knowledge manner.
      • +
    • Privacy: NovaNet does not provide ZKPs or privacy features typically associated with zkVMs. Its focus is on decentralized networking and computation.
      • +
    • Performance: NovaNet prioritizes efficient decentralized computation but does not focus on privacy or performance benchmarks related to ZKPs.
      • +
    • Integration: NovaNet is built for decentralized networks but is not designed to integrate with systems requiring verifiable computation or ZKP generation.
      • +
    • Conclusion: No, NovaNet is not a zkVM. It is a decentralized peer-to-peer network focused on distributed computing rather than zero-knowledge computation.
      • +
    +

    21. [ZkLLVM]​

    +
      +
    • Overview: ZkLLVM [21] is a compiler that transforms C++ or Rust code into circuits for use in zk-SNARK or zk-STARK systems. Its primary purpose is to bridge high-level programming +languages with ZKP systems by compiling code into arithmetic circuits that can be used to generate and verify proofs.
      • +
    • Main focus: ZkLLVM focuses on making ZKPs accessible to developers by enabling them to write code in familiar languages (C++, Rust) and then compile that code into ZK circuits.
      • +
    • Privacy: ZkLLVM enables the generation of ZKPs by compiling high-level code into ZK-compatible circuits. It plays a crucial role in privacy-preserving applications but does not act +as a zkVM itself.
      • +
    • Performance: ZkLLVM allows for the performance of ZKPs to be closely tied to the complexity of the compiled circuits. The performance depends on the underlying +zk-SNARK or zk-STARK system used.
      • +
    • Integration: ZkLLVM integrates with zk-SNARK and zk-STARK proof systems, making it useful for a variety of privacy-focused applications, but it does not serve as a zkVM +for general-purpose computation.
      • +
    • Conclusion: No, zkLLVM is not a zkVM. It is a compiler that transforms high-level code into ZK circuits, enabling ZKPs but not acting as a virtual machine for executing and proving programs.
      • +
    +

    22. [ZkMove]​

    +
      +
    • Overview: ZkMove [22] is a zkVM designed to execute smart contracts written in the Move language. It utilizes ZKPs to ensure that the execution of these contracts remains verifiable and secure.
      • +
    • Main focus: ZkMove focuses on privacy and verifiable execution for Move-based smart contracts, providing a framework for ZK-friendly computation.
      • +
    • Privacy: ZkMove ensures that smart contract execution remains private through ZKPs, making it suitable for privacy-preserving applications.
      • +
    • Performance: ZkMove is optimized for verifiable execution, ensuring that contracts can be proven correct while preserving privacy.
      • +
    • Integration: ZkMove integrates well with systems that use the Move language, particularly in environments that require private smart contract execution.
      • +
    • Conclusion: Yes, zkMove qualifies as a zkVM, offering ZK-friendly execution and privacy for smart contracts written in the Move language.
      • +
    +

    23. [O1VM]​

    +
      +
    • Overview: O1VM [23] is a general-purpose zkVM developed by o1Labs. It is designed to prove the execution of MIPS programs efficiently through a combination of zk-SNARKs +and specialized techniques like folding schemes and RAMLookups.
      • +
    • Main focus: O1VM focuses on scalability and verifiable computation for MIPS-based programs, making it a strong contender for executing and proving complex programs efficiently.
      • +
    • Privacy: O1VM ensures privacy through zk-SNARK proofs, keeping the details of the computation private while proving its correctness.
      • +
    • Performance: O1VM is optimized for handling long execution traces and complex computations, making it highly scalable.
      • +
    • Integration: O1VM integrates well with MIPS-based architectures and systems that require privacy-preserving computation.
      • +
    • Conclusion: Yes, o1VM qualifies as a zkVM, providing privacy, scalability, and strong proof generation for MIPS programs.
      • +
    +

    Summary of findings

    +
    Project nameZkVM statusZero knowledgeReasoning/comments
    SP1YesNoProves execution of LLVM-based programs but lacks privacy features.
    NexusYesNoStrong proof generation but lacks zero-knowledge privacy due to Spartan.
    Risc0YesYesSupports full ZKP generation for Rust programs.
    PowdrNoYesToolkit for creating custom zkVMs, not a zkVM itself.
    ZkMIPSYesYesSupports MIPS-like architecture with full zero-knowledge and proof generation.
    ValidaYesNoPerformance-focused zkVM, lacks privacy guarantees.
    JoltYesNoPerformance-focused zkVM, does not achieve zero-knowledge privacy.
    ZkWASMYesYesFull zero-knowledge and verifiable execution of WebAssembly code.
    AleoYesYesFully private and scalable dapps.
    OlaNoNoPrimarily a ZK-rollup platform, not a zkVM, focusing on scalability and performance rather than privacy.
    MidenYesYesZk-STARK-based zkVM with strong privacy and scalability.
    ZkOSNoNoVerifiable operating system focused on zkApps, not a zkVM.
    TritonNoNoOptimizes GPU workloads but not designed for ZKPs.
    CairoYesZK-friendlyCustom Rust-based language with zk-STARK proof generation.
    SnarkOSNoYesDecentralized OS for Aleo's network, focuses on consensus rather than verifiable computation.
    LurkNoNoProgramming language for recursive zk-SNARKs, not a zkVM.
    PiecrustYesZK-friendlyZkVM with recursive SNARK capabilities, focused on succinct proof generation.
    CenoYesYesTheoretical zkVM improving prover efficiency through recursive proofs.
    StellarNoNoFocuses on cross-border transactions, not ZK-proof generation or verifiable computation.
    NovaNetNoNoPeer-to-peer network focused on distributed computing, not zero-knowledge computation.
    ZkLLVMNoYes, in some casesCompiler for generating ZK-circuits, not a zkVM.
    ZkMoveYesZK-friendlyZkVM supporting Move language with ZKP execution.
    O1VMYesYesMIPS-based zkVM with strong privacy, scalability, and proof generation.
    +

    Insights and conclusions

    +

    Our analysis reveals that many of the projects labeled as zkVMs do meet the core criteria for zkVMs, offering verifiable computation and proof generation as foundational features. However, a number of these projects fall short of delivering full zero-knowledge privacy. Projects like Risc0, Aleo, and Miden stand out as leading zkVM frameworks -that balance proof generation, privacy, and scalability, offering strong platforms for developers seeking to build privacy-preserving applications.

    Conversely, projects like SP1 and Nexus excel in generating verifiable proofs but currently lack comprehensive zero-knowledge privacy mechanisms. These platforms are excellent for -scenarios where proof generation and scalability are paramount, but privacy is not a primary concern.

    As zkVM technology continues to evolve, we expect to see more projects integrating enhanced privacy-preserving mechanisms while simultaneously improving performance and scalability. +that balance proof generation, privacy, and scalability, offering strong platforms for developers seeking to build privacy-preserving applications.

    +

    Conversely, projects like SP1 and Nexus excel in generating verifiable proofs but currently lack comprehensive zero-knowledge privacy mechanisms. These platforms are excellent for +scenarios where proof generation and scalability are paramount, but privacy is not a primary concern.

    +

    As zkVM technology continues to evolve, we expect to see more projects integrating enhanced privacy-preserving mechanisms while simultaneously improving performance and scalability. This ongoing development will likely broaden the application of zkVMs across the blockchain ecosystem, particularly in privacy-sensitive sectors such as finance, data security, -and decentralized applications.

    What are your thoughts on our zkVM analysis? Do you agree with our findings, or do you know of other zkVM projects that should be on our radar? We would love to hear your insights, questions, -or suggestions! Feel free to join the discussion on our forum.

    References

    [1] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    [2] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    [3] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    [4] Powdr. Retrieved from https://docs.powdr.org/

    [5] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    [6] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    [7] Building Jolt: A fast, easy-to-use zkVM. Retrieved from https://a16zcrypto.com/posts/article/building-jolt/

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    [9] Aleo. Retrieved from https://aleo.org/blog/

    [10] OlaVM Whitepaper V2. Retrieved from https://github.com/Sin7Y/olavm-whitepaper-v2/tree/master

    [11] Polygon Miden VM. Retrieved from https://0xpolygonmiden.github.io/miden-vm/intro/main.html

    [12] The Adventures of OS: Making a RISC-V Operating System using Rust. Retrieved from https://osblog.stephenmarz.com/index.html

    [13] Triton VM. Retrieved from https://triton-vm.org/spec/

    [14] How does the original Cairo VM work?. Retrieved from https://github.com/lambdaclass/cairo-vm/blob/main/docs/python_vm/README.md

    [15] Aleo completes security audits of snarkOS & snarkVM. Retrieved from https://aleo.org/post/aleo-completes-security-audits-of-snarkos-and-snarkvm/

    [16] Lurk zkVM. Retrieved from https://github.com/lurk-lab

    [17] Piecrust VM. Retrieved from https://docs.rs/piecrust/latest/piecrust/

    [18] Ceno: Non-uniform, Segment and Parallel Zero-knowledge Virtual Machine. Retrieved from https://eprint.iacr.org/2024/387

    [19] ZkVM: a new design for fast, confidential smart contracts. Retrieved from https://stellar.org/blog/developers/zkvm-a-new-design-for-fast-confidential-smart-contracts

    [20] Novanet. Retrieved from https://www.novanet.xyz/blog

    [21] ZKLLVM. Retrieved from https://github.com/NilFoundation/zkLLVM

    [22] zkMove 0.2.0 - Achieving Full Bytecode Compatibility with Move. Retrieved from https://www.zkmove.net/2023-06-20-zkMove-0.2.0-Achieving-Full-Bytecode-Compatibility-with-Move/

    [23] O1VM. Retrieved from https://github.com/o1-labs/proof-systems/tree/master/o1vm

    - - +and decentralized applications.

    +

    What are your thoughts on our zkVM analysis? Do you agree with our findings, or do you know of other zkVM projects that should be on our radar? We would love to hear your insights, questions, +or suggestions! Feel free to join the discussion on our forum.

    +

    References

    +

    [1] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    +

    [2] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    +

    [3] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    +

    [4] Powdr. Retrieved from https://docs.powdr.org/

    +

    [5] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    +

    [6] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    +

    [7] Building Jolt: A fast, easy-to-use zkVM. Retrieved from https://a16zcrypto.com/posts/article/building-jolt/

    +

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    +

    [9] Aleo. Retrieved from https://aleo.org/blog/

    +

    [10] OlaVM Whitepaper V2. Retrieved from https://github.com/Sin7Y/olavm-whitepaper-v2/tree/master

    +

    [11] Polygon Miden VM. Retrieved from https://0xpolygonmiden.github.io/miden-vm/intro/main.html

    +

    [12] The Adventures of OS: Making a RISC-V Operating System using Rust. Retrieved from https://osblog.stephenmarz.com/index.html

    +

    [13] Triton VM. Retrieved from https://triton-vm.org/spec/

    +

    [14] How does the original Cairo VM work?. Retrieved from https://github.com/lambdaclass/cairo-vm/blob/main/docs/python_vm/README.md

    +

    [15] Aleo completes security audits of snarkOS & snarkVM. Retrieved from https://aleo.org/post/aleo-completes-security-audits-of-snarkos-and-snarkvm/

    +

    [16] Lurk zkVM. Retrieved from https://github.com/lurk-lab

    +

    [17] Piecrust VM. Retrieved from https://docs.rs/piecrust/latest/piecrust/

    +

    [18] Ceno: Non-uniform, Segment and Parallel Zero-knowledge Virtual Machine. Retrieved from https://eprint.iacr.org/2024/387

    +

    [19] ZkVM: a new design for fast, confidential smart contracts. Retrieved from https://stellar.org/blog/developers/zkvm-a-new-design-for-fast-confidential-smart-contracts

    +

    [20] Novanet. Retrieved from https://www.novanet.xyz/blog

    +

    [21] ZKLLVM. Retrieved from https://github.com/NilFoundation/zkLLVM

    +

    [22] zkMove 0.2.0 - Achieving Full Bytecode Compatibility with Move. Retrieved from https://www.zkmove.net/2023-06-20-zkMove-0.2.0-Achieving-Full-Bytecode-Compatibility-with-Move/

    +

    [23] O1VM. Retrieved from https://github.com/o1-labs/proof-systems/tree/master/o1vm

    \ No newline at end of file diff --git a/rlog/zkVM-testing/index.html b/rlog/zkVM-testing/index.html index 1056722d..89c80a88 100644 --- a/rlog/zkVM-testing/index.html +++ b/rlog/zkVM-testing/index.html @@ -1,79 +1,296 @@ - + - -zkVM Testing Report: Evaluating Zero-Knowledge Virtual Machines for Nescience | Vac Research + +zkVM Testing Report: Evaluating Zero-Knowledge Virtual Machines for Nescience | Vac Research - - - - + + + + -
    -

    zkVM Testing Report: Evaluating Zero-Knowledge Virtual Machines for Nescience

    by
    12 min read

    Introduction

    Following our initial exploration of zkVMs in our previous blog post [1], + ) .lsd-dropdown__trigger:focus .lsd-dropdown__option-label{-webkit-text-decoration:underline;text-decoration:underline;}.lsd-dropdown__label{display:block;}.lsd-dropdown__button-container{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;}.lsd-dropdown__trigger{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:justify;-webkit-justify-content:space-between;justify-content:space-between;border:none;cursor:pointer;background:none;}.lsd-dropdown__trigger:focus{outline:none;}.lsd-dropdown__option-label{cursor:inherit;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}.lsd-dropdown__icons{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;gap:8px;}.lsd-dropdown__icon{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}.lsd-dropdown__supporting-text{margin:6px 14px;}.lsd-dropdown--error .lsd-dropdown__option-label{-webkit-text-decoration:line-through;text-decoration:line-through;}.lsd-dropdown--disabled{opacity:0.34;cursor:initial;}.lsd-dropdown--large{width:208px;}.lsd-dropdown--large.lsd-dropdown--error{width:230px;}.lsd-dropdown--large .lsd-dropdown__label{margin:0 0 6px 18px;}.lsd-dropdown--large .lsd-dropdown__button-container{height:40px;}.lsd-dropdown--large .lsd-dropdown__trigger{padding:10px 0px 10px 18px;}.lsd-dropdown--large .lsd-dropdown__icons{padding:0px 14px;}.lsd-dropdown--medium{width:188px;}.lsd-dropdown--medium.lsd-dropdown--error{width:210px;}.lsd-dropdown--medium .lsd-dropdown__label{margin:0 0 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__button-container{height:32px;}.lsd-dropdown--medium .lsd-dropdown__trigger{padding:6px 0px 6px 14px;}.lsd-dropdown--medium .lsd-dropdown__icons{padding:0px 12px;}.lsd-dropdown--small{width:164px;}.lsd-dropdown--small.lsd-dropdown--error{width:186px;}.lsd-dropdown--small .lsd-dropdown__label{margin:0 0 6px 12px;}.lsd-dropdown--small .lsd-dropdown__button-container{height:28px;}.lsd-dropdown--small .lsd-dropdown__trigger{padding:6px 0px 6px 12px;}.lsd-dropdown--small .lsd-dropdown__icons{padding:0px 10px;}.lsd-dropdown--outlined .lsd-dropdown__button-container{border:1px solid rgb(var(--lsd-border-primary));}.lsd-dropdown--underlined .lsd-dropdown__button-container{border:1px solid transparent;border-bottom:1px solid rgb(var(--lsd-border-primary));}

    zkVM Testing Report: Evaluating Zero-Knowledge Virtual Machines for Nescience

    by
    12 min read
    +

    Following our initial exploration of zkVMs in our previous blog post [1], we have conducted a series of tests to identify the most suitable zkVM for the Nescience architecture [2]. This post outlines the testing process, results, and conclusions. Additionally, the full test suite and scripts can be found on our GitHub page [3], allowing others to replicate the results or explore the candidates further. Please note that we chose not to use hardware acceleration in our benchmarks, as our project is aimed at a broad audience. -Particularly, we cannot assume AVX512 support by default, as it is typically available only in high-end CPUs.

    We've shortlisted the following zkVMs for testing:

    • SP1 [4]
    • RISC0 [5]
    • Nexus [6]
    • ZkMIPS [7]
    • ZkWASM [8]
    • Valida [9]

    Why these candidates?

    When narrowing down the zkVMs, we focused on key factors:

    • True zero-knowledge functionality: The zkVMs had to demonstrate or be close to demonstrating the ability to generate and verify zero-knowledge proofs.
    • Performance baselines: We sought zkVMs with solid benchmarks in performance, particularly in speed and efficiency.
    • Specific functionalities: For Nescience, functionalities like lookup tables, precompiles, and recursive capabilities are critical.

    We need a zkVM that supports these to enable robust project development.

    Preliminary information on the candidates

    1. SP1 is a performant, open-source zkVM that verifies the execution of arbitrary Rust (or any LLVM-compiled language) programs. +Particularly, we cannot assume AVX512 support by default, as it is typically available only in high-end CPUs.

      +

      We've shortlisted the following zkVMs for testing:

      +
        +
      • SP1 [4]
        • +
      • RISC0 [5]
        • +
      • Nexus [6]
        • +
      • ZkMIPS [7]
        • +
      • ZkWASM [8]
        • +
      • Valida [9]
        • +
      +

      Why these candidates?

      +

      When narrowing down the zkVMs, we focused on key factors:

      +
        +
      • True zero-knowledge functionality: The zkVMs had to demonstrate or be close to demonstrating the ability to generate and verify zero-knowledge proofs.
        • +
      • Performance baselines: We sought zkVMs with solid benchmarks in performance, particularly in speed and efficiency.
        • +
      • Specific functionalities: For Nescience, functionalities like lookup tables, precompiles, and recursive capabilities are critical.
        • +
      +

      We need a zkVM that supports these to enable robust project development.

      +

      Preliminary information on the candidates

      +
        +
      1. +

        SP1 is a performant, open-source zkVM that verifies the execution of arbitrary Rust (or any LLVM-compiled language) programs. SP1 utilizes Plonky3, enabling recursive proofs and supporting a wide range of cryptographic algorithms, including ECC-based ones like Groth16. -While it supports aggregation, it appears not to support zero knowledge in a conventional manner.

      2. RISC0 zkVM allows one to prove the correct execution of arbitrary Rust code. Built on a RISC-V architecture, it is inherently adaptable +While it supports aggregation, it appears not to support zero knowledge in a conventional manner.

        +
        3. +
      3. +

        RISC0 zkVM allows one to prove the correct execution of arbitrary Rust code. Built on a RISC-V architecture, it is inherently adaptable for implementing standard cryptographic hash functions such as SHA-256 and ECDSA. RISC0 employs STARKs, providing a security level of 98 bits. -It supports multiple programming languages, including C and Rust, thanks to its compatibility with LLVM and WASM.

      4. Nexus is a modular, extensible, open-source, highly parallelized, prover-optimized, and contributor-friendly zkVM written in Rust. +It supports multiple programming languages, including C and Rust, thanks to its compatibility with LLVM and WASM.

        +
        5. +
      5. +

        Nexus is a modular, extensible, open-source, highly parallelized, prover-optimized, and contributor-friendly zkVM written in Rust. It focuses on performance and security, using the Nova folding scheme, which is particularly effective for recursive proofs. -Nexus also supports precompiles and targeted compilation, and besides Rust, it offers C++ support.

      6. ZkMIPS is a general verifiable computing infrastructure based on Plonky2 and the MIPS microarchitecture, aiming to empower Ethereum -as a global settlement layer. It can run arbitrary Rust code as well. Notably, zkMIPS is the only zkVM in this list that utilizes the MIPS opcode set.

      7. ZkWASM adheres to and supports the unmodified standard WASM bytecode specification. Since Rust code can be compiled to WASM bytecode, -one could theoretically run any Rust code on a zkWASM machine, providing flexibility and broad language support.

      8. Valida is a STARK-based virtual machine aiming to improve upon the state of the art in several categories:

        • Code reuse: The VM has a RISC-inspired instruction set, simplifying the targeting of conventional programming languages. +Nexus also supports precompiles and targeted compilation, and besides Rust, it offers C++ support.

          +
          • +
        • +

          ZkMIPS is a general verifiable computing infrastructure based on Plonky2 and the MIPS microarchitecture, aiming to empower Ethereum +as a global settlement layer. It can run arbitrary Rust code as well. Notably, zkMIPS is the only zkVM in this list that utilizes the MIPS opcode set.

          +
          • +
        • +

          ZkWASM adheres to and supports the unmodified standard WASM bytecode specification. Since Rust code can be compiled to WASM bytecode, +one could theoretically run any Rust code on a zkWASM machine, providing flexibility and broad language support.

          +
          • +
        • +

          Valida is a STARK-based virtual machine aiming to improve upon the state of the art in several categories:

          +
            +
          • Code reuse: The VM has a RISC-inspired instruction set, simplifying the targeting of conventional programming languages. A backend compiler is being developed to compile LLVM IR to the Valida ISA, enabling the proving of programs written in Rust, -Go, C++, and others with minimal to no changes in source code.
          • Prover performance: Engineered to maximize prover performance, Valida is compatible with a 31-bit field, restricted to degree 3 constraints, +Go, C++, and others with minimal to no changes in source code.
            • +
          • Prover performance: Engineered to maximize prover performance, Valida is compatible with a 31-bit field, restricted to degree 3 constraints, and features minimal instruction decoding. It operates directly on memory without general-purpose registers or a dedicated stack, -utilizing newer lookup arguments to reduce trace overhead involved in cross-chip communication.
          • Extensibility: Designed to be customizable, Valida can easily be extended to include an arbitrary number of user-defined instructions. -Procedural macros are used to construct the desired machine at compile time, avoiding any runtime penalties.

      Valida appears to be in the early stages of development but already showcases respectable performance metrics.

      Testing plan

      To thoroughly evaluate each zkVM, we devised a two-stage testing process:

      • Stage 1: Arithmetic operations

        The first phase focused on evaluating the zkVMs’ ability to handle basic arithmetic operations: addition, subtraction, multiplication, +utilizing newer lookup arguments to reduce trace overhead involved in cross-chip communication.

        • +
      • Extensibility: Designed to be customizable, Valida can easily be extended to include an arbitrary number of user-defined instructions. +Procedural macros are used to construct the desired machine at compile time, avoiding any runtime penalties.
        • +
      +
      2. +
    +

    Valida appears to be in the early stages of development but already showcases respectable performance metrics.

    +

    Testing plan

    +

    To thoroughly evaluate each zkVM, we devised a two-stage testing process:

    +
      +
    • +

      Stage 1: Arithmetic operations

      +

      The first phase focused on evaluating the zkVMs’ ability to handle basic arithmetic operations: addition, subtraction, multiplication, division, modulus division, and square root calculations. We designed the test around heptagonal numbers, which required zkVMs to process multiple arithmetic operations simultaneously. By using this method, we could measure efficiency and speed in handling complex mathematical calculations – -a crucial element for zkVM performance.

    • Stage 2: Memory consumption

      For the second phase, we evaluated each zkVM’s ability to manage memory under heavy loads. We tested several data structures, including lists, -hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent tests for the following operations:

      • Insert: How quickly can the zkVM add data to structures?
      • Delete: Does the zkVM handle memory release effectively?
      • Append: Can the zkVM efficiently grow data structures?
      • Search: How fast and efficient is the zkVM when retrieving stored data?

    The purpose of this stage was to identify any memory bottlenecks and to determine whether a zkVM could manage high-intensity tasks efficiently, -something vital for the Nescience project’s complex, data-heavy processes.

    Machine specifications

    The tests were conducted on the following hardware configuration:

    • CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
    • RAM: 600GiB DDR4 3200MHz ECC RAM, distributed across 16 DIMMs
    • Host OS: Proxmox 8.3
    • Hypervisor: KVM
    • Network layer: Open vSwitch
    • Machine model: Supermicro AS-2024US-TRT

    Results

    1. SP1​

    SP1 does not provide zero-knowledge capability in its proofs but delivers respectable performance, though slightly behind its main competitor. +a crucial element for zkVM performance.

    + +
  • +

    Stage 2: Memory consumption

    +

    For the second phase, we evaluated each zkVM’s ability to manage memory under heavy loads. We tested several data structures, including lists, +hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent tests for the following operations:

    +
      +
    • Insert: How quickly can the zkVM add data to structures?
      • +
    • Delete: Does the zkVM handle memory release effectively?
      • +
    • Append: Can the zkVM efficiently grow data structures?
      • +
    • Search: How fast and efficient is the zkVM when retrieving stored data?
      • +
    +
    • + +

    The purpose of this stage was to identify any memory bottlenecks and to determine whether a zkVM could manage high-intensity tasks efficiently, +something vital for the Nescience project’s complex, data-heavy processes.

    +

    Machine specifications

    +

    The tests were conducted on the following hardware configuration:

    +
      +
    • CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
      • +
    • RAM: 600GiB DDR4 3200MHz ECC RAM, distributed across 16 DIMMs
      • +
    • Host OS: Proxmox 8.3
      • +
    • Hypervisor: KVM
      • +
    • Network layer: Open vSwitch
      • +
    • Machine model: Supermicro AS-2024US-TRT
      • +
    +

    Results

    +

    1. SP1​

    +

    SP1 does not provide zero-knowledge capability in its proofs but delivers respectable performance, though slightly behind its main competitor. Memory leaks were minimal, staying below the 700 KB threshold. Interestingly, SP1 consumed more RAM during the basic arithmetic test than in memory allocation tests, showcasing the team's effective handling of memory under load. In the basic test, -allocations were primarily in the 9-16 B, 33-64 B, and 65-128 B ranges. For memory allocations, most fell into the 129-256 B range.

    • Stage 1: Hept 100 test
      • Proof size: 3.108 MB
      • Proof time: 16.95 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 3.17 MB
      • Proof time: 20.85 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    2. RISC0​

    RISC0 stands out with exceptional performance in proof size and generation time, ranking among the best +allocations were primarily in the 9-16 B, 33-64 B, and 65-128 B ranges. For memory allocations, most fell into the 129-256 B range.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 3.108 MB
        • +
      • Proof time: 16.95 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 3.17 MB
        • +
      • Proof time: 20.85 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    2. RISC0​

    +

    RISC0 stands out with exceptional performance in proof size and generation time, ranking among the best (with the exception of Valida and zkWASM's basic test). It also handles memory well, with minor leaks under 0.5 MB -and controlled RAM consumption staying below 2.2 GB. RISC0's memory allocations were consistent, primarily in the 17-32 B and 33-64 B ranges.

    • Stage 1: Hept 100 test
      • Proof size: 217.4 KB
      • Proof time: 9.73 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 217.4 KB
      • Proof time: 16.63 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    Based on these results, RISC0 is a solid candidate for Nescience.

    3. Nexus​

    Nexus' performance offers interesting insights into folding scheme-based zkVMs. Surprisingly, proof sizes remained consistent +and controlled RAM consumption staying below 2.2 GB. RISC0's memory allocations were consistent, primarily in the 17-32 B and 33-64 B ranges.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 217.4 KB
        • +
      • Proof time: 9.73 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 217.4 KB
        • +
      • Proof time: 16.63 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +

    Based on these results, RISC0 is a solid candidate for Nescience.

    +
    +

    3. Nexus​

    +

    Nexus' performance offers interesting insights into folding scheme-based zkVMs. Surprisingly, proof sizes remained consistent regardless of workload, with no significant memory leaks (under 700 KB). However, while RAM consumption increased slightly with higher -workloads (up to 1.2 GB), Nexus performed poorly during memory allocation tests, making it unsuitable for our use case.

    • Allocation details:

      • Basic test: Most allocations concentrated in 65-128 B
      • Memory-heavy test: Allocations in the 129-256 B range

    • Stage 1: Hept 100 test

      • Proof size: 46 MB
      • Proof time: 12.06 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 46 MB
      • Proof time: 56 minutes
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    4. ZkMIPS​

    ZkMIPS presents an intriguing case. While it shows good results in terms of proof size and generation time during the basic test, +workloads (up to 1.2 GB), Nexus performed poorly during memory allocation tests, making it unsuitable for our use case.

    +
      +
    • +

      Allocation details:

      +
        +
      • Basic test: Most allocations concentrated in 65-128 B
        • +
      • Memory-heavy test: Allocations in the 129-256 B range
        • +
      +
      • +
    • +

      Stage 1: Hept 100 test

      +
        +
      • Proof size: 46 MB
        • +
      • Proof time: 12.06 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 46 MB
        • +
      • Proof time: 56 minutes
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    4. ZkMIPS​

    +

    ZkMIPS presents an intriguing case. While it shows good results in terms of proof size and generation time during the basic test, these come at the cost of significant RAM usage and memory leaks. The memory allocation test revealed a concerning 6.7 GB memory leak, with 0.5 GB leaked during the basic test. Despite this, RAM consumption (while high at 17+ GB) remains stable under higher workloads. -Allocation sizes are spread across several ranges, with notable concentrations in the 17-32 B, 65-128 B, and 257-512 B slots.

    • Stage 1: Hept 100 test
      • Proof size: 4.3 MB
      • Proof time: 9.32 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 4.898 MB
      • Proof time: 42.57 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    This zkVM provides mixed results with strong proof generation but concerning memory management issues.

    5. ZkWASM​

    ZkWASM, unfortunately, performed poorly in both stages regarding proof size and generation time. RAM consumption was particularly high, +Allocation sizes are spread across several ranges, with notable concentrations in the 17-32 B, 65-128 B, and 257-512 B slots.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 4.3 MB
        • +
      • Proof time: 9.32 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 4.898 MB
        • +
      • Proof time: 42.57 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +

    This zkVM provides mixed results with strong proof generation but concerning memory management issues.

    +
    +

    5. ZkWASM​

    +

    ZkWASM, unfortunately, performed poorly in both stages regarding proof size and generation time. RAM consumption was particularly high, exceeding 7 GB in the basic test, and an astounding 57 GB during memory allocation tests. Despite its impressive memory usage, the proof sizes were relatively large at 18 KB and 334 KB respectively. Allocation sizes were mainly concentrated in the 33-64 B range, -with neighboring slots contributing small but notable amounts.

    • Stage 1: Hept 100 test
      • Proof size: 18 KB
      • Proof time: 42.7 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    • Stage 2: Vec 10000 test
      • Proof size: 334 KB
      • Proof time: 323 seconds
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5

    6. Valida​

    Valida delivered impressive results in proof generation speed and size, with a proof size of 280 KB and a proof time of < 1 second. +with neighboring slots contributing small but notable amounts.

    +
      +
    • Stage 1: Hept 100 test +
        +
      • Proof size: 18 KB
        • +
      • Proof time: 42.7 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
      +
    • Stage 2: Vec 10000 test +
        +
      • Proof size: 334 KB
        • +
      • Proof time: 323 seconds
        • +
      +
      • +
    +
    Image 1
    Image 2
    Image 3
    Image 4
    Image 5
    +
    +

    6. Valida​

    +

    Valida delivered impressive results in proof generation speed and size, with a proof size of 280 KB and a proof time of < 1 second. However, profiling was not possible due to Valida's limited Rust support. Valida currently compiles Rust using the LLVM backend, transpiling LLVM IR to leverage its C/C++ implementation, which leads to errors when handling Rust-specific data structures or dependencies. As a result, complex memory interactions couldn't be tested, and using Valida with Rust code is currently not advisable. -A GitHub issue has been opened to address this.

    Summary table​

    Stage 1​

    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP116.95 s3.108 MB2.1 GB656.8 KB
    RISC09.73 s217.4 KB1.9 GB470.5 KB
    Nexus12.06 s46 MB9.7 MB646.5 KB
    ZkMIPS9.32 s4.3 MB17.3 GB453.8 MB
    ZkWASM42.7 s18 KB8.2 GB259.4 KB
    Valida< 1 s280 KBN/AN/A

    Stage 2​

    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP120.85 s3.17 MB1.9 GB616 KB
    RISC016.63 s217.4 KB2.3 GB485.3 KB
    Nexus56 m46 MB1.9 GB616 KB
    ZkMIPS42.57 s4.898 MB18.9 GB6.9 GB
    ZkWASM323 s334 KB58.8 GB259.4 KB
    ValidaN/AN/AN/AN/A

    Summary

    After an extensive evaluation of six zkVM candidates for the Nescience project, RISC0 emerged as the top choice. +A GitHub issue has been opened to address this.

    +
    +

    Summary table​

    +

    Stage 1​

    +
    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP116.95 s3.108 MB2.1 GB656.8 KB
    RISC09.73 s217.4 KB1.9 GB470.5 KB
    Nexus12.06 s46 MB9.7 MB646.5 KB
    ZkMIPS9.32 s4.3 MB17.3 GB453.8 MB
    ZkWASM42.7 s18 KB8.2 GB259.4 KB
    Valida< 1 s280 KBN/AN/A
    +
    +

    Stage 2​

    +
    zkVMProof timeProof sizePeak RAM consumptionMemory leaked
    SP120.85 s3.17 MB1.9 GB616 KB
    RISC016.63 s217.4 KB2.3 GB485.3 KB
    Nexus56 m46 MB1.9 GB616 KB
    ZkMIPS42.57 s4.898 MB18.9 GB6.9 GB
    ZkWASM323 s334 KB58.8 GB259.4 KB
    ValidaN/AN/AN/AN/A
    +
    +

    Summary

    +

    After an extensive evaluation of six zkVM candidates for the Nescience project, RISC0 emerged as the top choice. It excels in both proof generation time and size while maintaining a reasonable memory footprint. With strong zero-knowledge proof capabilities and support for multiple programming languages, it aligns well with our project's needs for privacy, -performance, and flexibility. Its overall balance between performance and efficiency makes it the most viable zkVM at this stage.

    Valida, while promising with its potential for high prover performance, is still in early development and suffers from Rust integration issues. +performance, and flexibility. Its overall balance between performance and efficiency makes it the most viable zkVM at this stage.

    +

    Valida, while promising with its potential for high prover performance, is still in early development and suffers from Rust integration issues. The current LLVM IR transpilation limitations mean it cannot handle complex memory interactions, disqualifying it for now. -However, once its development matures, Valida could become a strong alternative, and we plan to revisit it as it evolves.

    SP1, though initially interesting, failed to meet the zero-knowledge proof requirement. Its performance in arithmetic operations was -respectable but insufficient to justify further consideration given its lack of ZK functionality – critical for our privacy-first objectives.

    Nexus demonstrated consistent proof sizes and manageable memory usage, but its lackluster performance during memory-intensive tasks and +However, once its development matures, Valida could become a strong alternative, and we plan to revisit it as it evolves.

    +

    SP1, though initially interesting, failed to meet the zero-knowledge proof requirement. Its performance in arithmetic operations was +respectable but insufficient to justify further consideration given its lack of ZK functionality – critical for our privacy-first objectives.

    +

    Nexus demonstrated consistent proof sizes and manageable memory usage, but its lackluster performance during memory-intensive tasks and its proof size (especially for larger workloads) disqualified it from being a top contender. While zkMIPS delivered solid proof times, -the memory issues were too significant to ignore, making it unsuitable.

    Finally, zkWASM exhibited the poorest results, struggling both in proof size and generation time. Despite its potential for WASM bytecode support, -the excessive RAM consumption (up to 57 GB in the memory test) rendered it impractical for Nescience’s use case.

    In conclusion, RISC0 is the best fit for Nescience at this stage, but Valida remains a future candidate as its development progresses.

    In the future, we plan to compare RISC0 and SP1 with CUDA acceleration. Ideally, by that time, more zkVMs will include similar acceleration capabilities, -enabling a fairer and more comprehensive comparison across platforms.

    We’d love to hear your thoughts on our zkVM testing process and results! Do you agree with our conclusions, or do you think we missed a promising zkVM? -We’re always open to feedback, insights, and suggestions from the community.

    Join the discussion and share your perspectives on +the memory issues were too significant to ignore, making it unsuitable.

    +

    Finally, zkWASM exhibited the poorest results, struggling both in proof size and generation time. Despite its potential for WASM bytecode support, +the excessive RAM consumption (up to 57 GB in the memory test) rendered it impractical for Nescience’s use case.

    +

    In conclusion, RISC0 is the best fit for Nescience at this stage, but Valida remains a future candidate as its development progresses.

    +

    In the future, we plan to compare RISC0 and SP1 with CUDA acceleration. Ideally, by that time, more zkVMs will include similar acceleration capabilities, +enabling a fairer and more comprehensive comparison across platforms.

    +

    We’d love to hear your thoughts on our zkVM testing process and results! Do you agree with our conclusions, or do you think we missed a promising zkVM? +We’re always open to feedback, insights, and suggestions from the community.

    +

    Join the discussion and share your perspectives on our forum or try out the -tests yourself through our GitHub page!

    References

    [1] Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? Retrieved from https://vac.dev/rlog/zkVM-explorations/

    [2] Nescience: A User-Centric State-Separation Architecture. Retrieved from https://vac.dev/rlog/Nescience-state-separation-architecture

    [3] Our GitHub Page for zkVM Testing. Retrieved from https://github.com/vacp2p/nescience-zkvm-testing

    [4] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    [5] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    [6] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    [7] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    [9] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    - - +tests yourself through our GitHub page!

    +

    References

    +

    [1] Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines? Retrieved from https://vac.dev/rlog/zkVM-explorations/

    +

    [2] Nescience: A User-Centric State-Separation Architecture. Retrieved from https://vac.dev/rlog/Nescience-state-separation-architecture

    +

    [3] Our GitHub Page for zkVM Testing. Retrieved from https://github.com/vacp2p/nescience-zkvm-testing

    +

    [4] Introducing SP1: A performant, 100% open-source, contributor-friendly zkVM. Retrieved from https://blog.succinct.xyz/introducing-sp1/

    +

    [5] The first general purpose zkVM. Retrieved from https://www.risczero.com/zkvm

    +

    [6] The Nexus 2.0 zkVM. Retrieved from https://docs.nexus.xyz/

    +

    [7] ZKM Architecture. Retrieved from https://docs.zkm.io/zkm-architecture

    +

    [8] ZK-WASM. Retrieved from https://delphinuslab.com/zk-wasm/

    +

    [9] Valida zkVM Design. Retrieved from https://delendum.xyz/writings/2023-05-10-zkvm-design.html

    \ No newline at end of file diff --git a/search-index.json b/search-index.json index 5a6fb29d..6c8e8566 100644 --- a/search-index.json +++ b/search-index.json @@ -1 +1 @@ -[{"documents":[{"i":1,"t":"Vac 2024 Year in Review","u":"/rlog/2024-recap","b":["Research Blog"]},{"i":57,"t":"","u":"/rlog/archive","b":["Research Blog"]},{"i":58,"t":"Building Privacy-Protecting Infrastructure","u":"/rlog/building-privacy-protecting-infrastructure","b":["Research Blog"]},{"i":98,"t":"Device Pairing in Js-waku and Go-waku","u":"/rlog/device-pairing-in-js-waku-and-go-waku","b":["Research Blog"]},{"i":106,"t":"Opinion: Pseudo-ethics in the Surveillance Tech Industry","u":"/rlog/ethics-surveillance-tech","b":["Research Blog"]},{"i":125,"t":"DNS Based Discovery","u":"/rlog/dns-based-discovery","b":["Research Blog"]},{"i":127,"t":"Vac 101: Climbing Merkle Trees","u":"/rlog/climbing-merkle-trees","b":["Research Blog"]},{"i":151,"t":"Feasibility Study: Discv5","u":"/rlog/feasibility-discv5","b":["Research Blog"]},{"i":173,"t":"Feasibility Study: Semaphore rate limiting through zkSNARKs","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","b":["Research Blog"]},{"i":200,"t":"Fixing Whisper with Waku","u":"/rlog/fixing-whisper-with-waku","b":["Research Blog"]},{"i":221,"t":"The Future of Waku Network: Scaling, Incentivization, and Heterogeneity","u":"/rlog/future-of-waku-network","b":["Research Blog"]},{"i":241,"t":"Libp2p GossipSub IDONTWANT Message Performance Impact","u":"/rlog/gsub-idontwant-perf-eval","b":["Research Blog"]},{"i":251,"t":"GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks","u":"/rlog/GossipSub Improvements","b":["Research Blog"]},{"i":271,"t":"Large Message Handling in GossipSub: Potential Improvements","u":"/rlog/gsub-largemsg-improvements","b":["Research Blog"]},{"i":292,"t":"From Kademlia to Discv5","u":"/rlog/kademlia-to-discv5","b":["Research Blog"]},{"i":304,"t":"Introducing nwaku","u":"/rlog/introducing-nwaku","b":["Research Blog"]},{"i":338,"t":"Nescience - A zkVM leveraging hiding properties","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","b":["Research Blog"]},{"i":374,"t":"P2P Data Sync for Mobile","u":"/rlog/p2p-data-sync-for-mobile","b":["Research Blog"]},{"i":396,"t":"","u":"/rlog/page/2","b":["Research Blog"]},{"i":417,"t":"","u":"/rlog/page/3","b":["Research Blog"]},{"i":438,"t":"","u":"/rlog/page/4","b":["Research Blog"]},{"i":459,"t":"","u":"/rlog/page/5","b":["Research Blog"]},{"i":462,"t":"The MDSECheck method: choosing secure square MDS matrices for P-SP-networks","u":"/rlog/mdsecheck-method","b":["Research Blog"]},{"i":480,"t":"P2P Data Sync with a Remote Log","u":"/rlog/remote-log","b":["Research Blog"]},{"i":495,"t":"Nescience: A User-Centric State-Separation Architecture","u":"/rlog/Nescience-state-separation-architecture","b":["Research Blog"]},{"i":573,"t":"Verifying RLN Proofs in Light Clients with Subtrees","u":"/rlog/rln-light-verifiers","b":["Research Blog"]},{"i":603,"t":"Presenting JS-Waku: Waku v2 in the Browser","u":"/rlog/presenting-js-waku","b":["Research Blog"]},{"i":613,"t":"Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku","u":"/rlog/rln-anonymous-dos-prevention","b":["Research Blog"]},{"i":639,"t":"RLN-v3: Towards a Flexible and Cost-Efficient Implementation","u":"/rlog/rln-v3","b":["Research Blog"]},{"i":668,"t":"Privacy-preserving p2p economic spam protection in Waku v2","u":"/rlog/rln-relay","b":["Research Blog"]},{"i":710,"t":"Vac - A Rough Overview","u":"/rlog/vac-overview","b":["Research Blog"]},{"i":718,"t":"Waku for All Decentralized Applications and Infrastructures","u":"/rlog/waku-for-all","b":["Research Blog"]},{"i":735,"t":"Waku Update","u":"/rlog/waku-update","b":["Research Blog"]},{"i":749,"t":"Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument","u":"/rlog/vac101-fiat-shamir","b":["Research Blog"]},{"i":777,"t":"Waku v1 vs Waku v2: Bandwidth Comparison","u":"/rlog/waku-v1-v2-bandwidth-comparison","b":["Research Blog"]},{"i":806,"t":"Vac 101: Membership with Bloom Filters and Cuckoo Filters","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","b":["Research Blog"]},{"i":832,"t":"[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging","u":"/rlog/waku-v2-ethereum-coscup","b":["Research Blog"]},{"i":867,"t":"[Talk] Vac, Waku v2 and Ethereum Messaging","u":"/rlog/waku-v2-ethereum-messaging","b":["Research Blog"]},{"i":896,"t":"Waku v2 Update","u":"/rlog/waku-v2-update","b":["Research Blog"]},{"i":916,"t":"What's the Plan for Waku v2?","u":"/rlog/waku-v2-plan","b":["Research Blog"]},{"i":962,"t":"Waku v2 Ambient Peer Discovery","u":"/rlog/wakuv2-apd","b":["Research Blog"]},{"i":996,"t":"Noise handshakes as key-exchange mechanism for Waku","u":"/rlog/wakuv2-noise","b":["Research Blog"]},{"i":1026,"t":"What Would a WeChat Replacement Need?","u":"/rlog/wechat-replacement-need","b":["Research Blog"]},{"i":1065,"t":"Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay","u":"/rlog/wakuv2-relay-anon","b":["Research Blog"]},{"i":1107,"t":"About Vac","u":"/","b":["About Vac"]},{"i":1109,"t":"Exploring zkVMs: Which Projects Truly Qualify as Zero-Knowledge Virtual Machines?","u":"/rlog/zkVM-explorations","b":["Research Blog"]},{"i":1171,"t":"Join the community","u":"/community","b":["Community"]},{"i":1173,"t":"zkVM Testing Report: Evaluating Zero-Knowledge Virtual Machines for Nescience","u":"/rlog/zkVM-testing","b":["Research Blog"]},{"i":1206,"t":"Current job openings","u":"/join-us","b":["Join Us"]},{"i":1208,"t":"Contribute","u":"/contribute","b":[]},{"i":1213,"t":"","u":"/principles","b":[]},{"i":1237,"t":"","u":"/media","b":[]},{"i":1239,"t":"Privacy Policy","u":"/privacy-policy","b":[]},{"i":1259,"t":"Vac Research","u":"/research","b":[]},{"i":1261,"t":"Publications","u":"/publications","b":[]},{"i":1266,"t":"Security","u":"/security","b":[]},{"i":1268,"t":"Vac RFC Process","u":"/rfcprocess","b":[]},{"i":1270,"t":"Vac R&D Service Units","u":"/vsus","b":[]},{"i":1290,"t":"Terms of Use","u":"/terms","b":[]},{"i":1314,"t":"Vac Incubator Projects","u":"/vips","b":[]}],"index":{"version":"2.3.9","fields":["t"],"fieldVectors":[["t/1",[0,1.619,1,3.785,2,3.785,3,3.785]],["t/57",[]],["t/58",[4,3.785,5,2.663,6,3.263,7,3.263]],["t/98",[8,3.159,9,3.159,10,2.723,11,1.607,12,3.159]],["t/106",[13,3.159,14,3.159,15,3.159,16,3.159,17,3.159,18,3.159]],["t/125",[19,4.201,20,4.201,21,3.622]],["t/127",[0,1.473,22,2.656,23,3.444,24,3.444,25,3.444]],["t/151",[26,3.622,27,3.622,28,3.622]],["t/173",[26,2.515,27,2.515,29,2.918,30,2.515,31,2.515,32,2.918,33,2.918]],["t/200",[11,1.483,34,4.201,35,4.201]],["t/221",[11,1.115,36,3.159,37,2.437,38,3.159,39,3.159,40,3.159]],["t/241",[41,3.159,42,2.437,43,3.159,44,2.051,45,3.159,46,3.159]],["t/251",[37,1.831,42,1.831,44,1.541,47,2.046,48,2.374,49,2.374,50,2.374,51,2.374,52,2.374,53,1.67]],["t/271",[42,2.437,44,2.051,47,2.723,54,3.159,55,3.159,56,3.159]],["t/292",[28,4.07,57,4.721]],["t/304",[58,4.721,59,4.721]],["t/338",[60,2.656,61,2.656,62,3.444,63,3.444,64,3.444]],["t/374",[53,2.663,65,3.263,66,3.263,67,3.785]],["t/396",[]],["t/417",[]],["t/438",[]],["t/459",[]],["t/462",[37,1.831,68,2.374,69,2.374,70,2.374,71,2.046,72,2.374,73,2.374,74,2.374,75,2.374,76,2.374]],["t/480",[53,2.423,65,2.969,66,2.969,77,3.444,78,3.444]],["t/495",[60,2.437,79,3.159,80,3.159,81,3.159,82,3.159,83,3.159]],["t/573",[84,3.159,85,2.723,86,3.159,87,3.159,88,3.159,89,3.159]],["t/603",[10,2.723,11,1.607,90,3.159,91,1.68,92,3.159]],["t/613",[11,0.956,30,2.337,31,2.337,93,2.711,94,2.337,95,2.711,96,2.711,97,2.711]],["t/639",[85,2.515,98,2.918,99,2.918,100,2.918,101,2.918,102,2.918,103,2.918]],["t/668",[5,1.907,6,2.337,11,0.956,53,1.907,91,1.442,104,2.711,105,2.711,106,2.711]],["t/710",[0,1.797,107,4.201,108,4.201]],["t/718",[7,3.263,11,1.336,109,3.785,110,3.785]],["t/735",[11,1.666,111,4.07]],["t/749",[0,1.248,22,2.25,112,2.918,113,2.918,114,2.918,115,2.918,116,2.918]],["t/777",[11,1.516,91,1.552,117,2.918,118,2.918,119,2.918,120,2.918]],["t/806",[0,1.248,22,2.25,121,2.918,122,2.918,123,4.297,124,2.918]],["t/832",[0,1.248,11,1.03,44,1.895,91,1.552,125,2.515,126,2.918,127,2.515]],["t/867",[0,1.351,11,1.115,44,2.051,91,1.68,125,2.723,127,2.723]],["t/896",[11,1.483,91,2.235,111,3.622]],["t/916",[11,1.336,91,2.013,128,3.785,129,3.785]],["t/962",[11,1.215,21,2.969,91,1.832,130,3.444,131,3.444]],["t/996",[11,1.115,132,3.159,133,3.159,134,3.159,135,3.159,136,3.159]],["t/1026",[137,4.201,138,4.201,139,4.201]],["t/1065",[5,1.907,11,1.436,94,2.337,140,2.711,141,2.711,142,2.711,143,2.711]],["t/1107",[0,2.304]],["t/1109",[61,1.952,144,2.531,145,2.182,146,2.531,147,2.531,148,2.182,149,2.182,150,2.182,151,2.182]],["t/1171",[152,4.721,153,4.721]],["t/1173",[60,1.952,61,1.952,148,2.182,149,2.182,150,2.182,151,2.182,154,2.531,155,2.531,156,2.531]],["t/1206",[157,4.201,158,4.201,159,4.201]],["t/1208",[160,5.386]],["t/1213",[]],["t/1237",[]],["t/1239",[5,3.321,161,4.721]],["t/1259",[0,2.019,162,4.721]],["t/1261",[163,5.386]],["t/1266",[71,4.644]],["t/1268",[0,1.797,164,4.201,165,4.201]],["t/1270",[0,1.619,166,3.785,167,3.785,168,3.785]],["t/1290",[169,4.721,170,4.721]],["t/1314",[0,1.797,145,3.622,171,4.201]]],"invertedIndex":[["101",{"_index":22,"t":{"127":{"position":[[4,4]]},"749":{"position":[[4,4]]},"806":{"position":[[4,4]]}}}],["2024",{"_index":1,"t":{"1":{"position":[[4,4]]}}}],["ambient",{"_index":130,"t":{"962":{"position":[[8,7]]}}}],["analysi",{"_index":140,"t":{"1065":{"position":[[27,8]]}}}],["anonym",{"_index":94,"t":{"613":{"position":[[14,9]]},"1065":{"position":[[17,9]]}}}],["applic",{"_index":110,"t":{"718":{"position":[[27,12]]}}}],["architectur",{"_index":83,"t":{"495":{"position":[[43,12]]}}}],["argument",{"_index":116,"t":{"749":{"position":[[66,8]]}}}],["bandwidth",{"_index":119,"t":{"777":{"position":[[20,9]]}}}],["base",{"_index":20,"t":{"125":{"position":[[4,5]]}}}],["bloom",{"_index":122,"t":{"806":{"position":[[25,5]]}}}],["browser",{"_index":92,"t":{"603":{"position":[[35,7]]}}}],["build",{"_index":4,"t":{"58":{"position":[[0,8]]}}}],["centric",{"_index":80,"t":{"495":{"position":[[18,7]]}}}],["choos",{"_index":70,"t":{"462":{"position":[[22,8]]}}}],["client",{"_index":88,"t":{"573":{"position":[[30,7]]}}}],["climb",{"_index":23,"t":{"127":{"position":[[9,8]]}}}],["commun",{"_index":153,"t":{"1171":{"position":[[9,9]]}}}],["comparison",{"_index":120,"t":{"777":{"position":[[30,10]]}}}],["contribut",{"_index":160,"t":{"1208":{"position":[[0,10]]}}}],["coscup",{"_index":126,"t":{"832":{"position":[[9,7]]}}}],["cost",{"_index":101,"t":{"639":{"position":[[31,4]]}}}],["cuckoo",{"_index":124,"t":{"806":{"position":[[43,6]]}}}],["current",{"_index":157,"t":{"1206":{"position":[[0,7]]}}}],["data",{"_index":65,"t":{"374":{"position":[[4,4]]},"480":{"position":[[4,4]]}}}],["decentr",{"_index":109,"t":{"718":{"position":[[13,13]]}}}],["definit",{"_index":142,"t":{"1065":{"position":[[44,11]]}}}],["design",{"_index":50,"t":{"251":{"position":[[45,6]]}}}],["devic",{"_index":8,"t":{"98":{"position":[[0,6]]}}}],["discoveri",{"_index":21,"t":{"125":{"position":[[10,9]]},"962":{"position":[[21,9]]}}}],["discv5",{"_index":28,"t":{"151":{"position":[[19,6]]},"292":{"position":[[17,6]]}}}],["dissemin",{"_index":51,"t":{"251":{"position":[[64,13]]}}}],["dn",{"_index":19,"t":{"125":{"position":[[0,3]]}}}],["do",{"_index":95,"t":{"613":{"position":[[24,3]]}}}],["econom",{"_index":105,"t":{"668":{"position":[[23,8]]}}}],["effici",{"_index":102,"t":{"639":{"position":[[36,9]]}}}],["ethereum",{"_index":127,"t":{"832":{"position":[[34,8]]},"867":{"position":[[24,8]]}}}],["ethic",{"_index":15,"t":{"106":{"position":[[16,6]]}}}],["evalu",{"_index":156,"t":{"1173":{"position":[[21,10]]}}}],["evolut",{"_index":48,"t":{"251":{"position":[[24,9]]}}}],["exchang",{"_index":135,"t":{"996":{"position":[[24,8]]}}}],["explor",{"_index":144,"t":{"1109":{"position":[[0,9]]}}}],["feasibl",{"_index":26,"t":{"151":{"position":[[0,11]]},"173":{"position":[[0,11]]}}}],["filter",{"_index":123,"t":{"806":{"position":[[31,7],[50,7]]}}}],["fix",{"_index":34,"t":{"200":{"position":[[0,6]]}}}],["flexibl",{"_index":100,"t":{"639":{"position":[[18,8]]}}}],["futur",{"_index":36,"t":{"221":{"position":[[4,6]]}}}],["go",{"_index":12,"t":{"98":{"position":[[30,2]]}}}],["gossipsub",{"_index":42,"t":{"241":{"position":[[7,9]]},"251":{"position":[[0,9]]},"271":{"position":[[26,10]]}}}],["handl",{"_index":55,"t":{"271":{"position":[[14,8]]}}}],["handshak",{"_index":133,"t":{"996":{"position":[[6,10]]}}}],["heterogen",{"_index":40,"t":{"221":{"position":[[58,13]]}}}],["hide",{"_index":63,"t":{"338":{"position":[[30,6]]}}}],["idontw",{"_index":43,"t":{"241":{"position":[[17,9]]}}}],["impact",{"_index":46,"t":{"241":{"position":[[47,6]]}}}],["implement",{"_index":103,"t":{"639":{"position":[[46,14]]}}}],["improv",{"_index":47,"t":{"251":{"position":[[10,13]]},"271":{"position":[[47,12]]}}}],["incentiv",{"_index":39,"t":{"221":{"position":[[37,16]]}}}],["incub",{"_index":171,"t":{"1314":{"position":[[4,9]]}}}],["industri",{"_index":18,"t":{"106":{"position":[[48,8]]}}}],["infrastructur",{"_index":7,"t":{"58":{"position":[[28,14]]},"718":{"position":[[44,15]]}}}],["interact",{"_index":113,"t":{"749":{"position":[[25,11]]}}}],["introduc",{"_index":58,"t":{"304":{"position":[[0,11]]}}}],["job",{"_index":158,"t":{"1206":{"position":[[8,3]]}}}],["join",{"_index":152,"t":{"1171":{"position":[[0,4]]}}}],["js",{"_index":10,"t":{"98":{"position":[[18,2]]},"603":{"position":[[11,2]]}}}],["kademlia",{"_index":57,"t":{"292":{"position":[[5,8]]}}}],["key",{"_index":134,"t":{"996":{"position":[[20,3]]}}}],["knowledg",{"_index":149,"t":{"1109":{"position":[[54,9]]},"1173":{"position":[[37,9]]}}}],["larg",{"_index":54,"t":{"271":{"position":[[0,5]]}}}],["leverag",{"_index":62,"t":{"338":{"position":[[19,10]]}}}],["libp2p",{"_index":41,"t":{"241":{"position":[[0,6]]}}}],["light",{"_index":87,"t":{"573":{"position":[[24,5]]}}}],["limit",{"_index":31,"t":{"173":{"position":[[34,8]]},"613":{"position":[[49,8]]}}}],["log",{"_index":78,"t":{"480":{"position":[[28,3]]}}}],["machin",{"_index":151,"t":{"1109":{"position":[[72,9]]},"1173":{"position":[[55,8]]}}}],["matric",{"_index":74,"t":{"462":{"position":[[49,8]]}}}],["md",{"_index":73,"t":{"462":{"position":[[45,3]]}}}],["mdsecheck",{"_index":68,"t":{"462":{"position":[[4,9]]}}}],["mechan",{"_index":136,"t":{"996":{"position":[[33,9]]}}}],["membership",{"_index":121,"t":{"806":{"position":[[9,10]]}}}],["merkl",{"_index":24,"t":{"127":{"position":[[18,6]]}}}],["messag",{"_index":44,"t":{"241":{"position":[[27,7]]},"251":{"position":[[56,7]]},"271":{"position":[[6,7]]},"832":{"position":[[43,9]]},"867":{"position":[[33,9]]}}}],["method",{"_index":69,"t":{"462":{"position":[[14,7]]}}}],["mobil",{"_index":67,"t":{"374":{"position":[[18,6]]}}}],["need",{"_index":139,"t":{"1026":{"position":[[32,5]]}}}],["nescienc",{"_index":60,"t":{"338":{"position":[[0,9]]},"495":{"position":[[0,10]]},"1173":{"position":[[68,9]]}}}],["network",{"_index":37,"t":{"221":{"position":[[19,8]]},"251":{"position":[[98,8]]},"462":{"position":[[67,8]]}}}],["nois",{"_index":132,"t":{"996":{"position":[[0,5]]}}}],["noninteract",{"_index":115,"t":{"749":{"position":[[51,14]]}}}],["nullifi",{"_index":97,"t":{"613":{"position":[[58,10]]}}}],["nwaku",{"_index":59,"t":{"304":{"position":[[12,5]]}}}],["open",{"_index":159,"t":{"1206":{"position":[[12,8]]}}}],["opinion",{"_index":13,"t":{"106":{"position":[[0,8]]}}}],["overlay",{"_index":49,"t":{"251":{"position":[[37,7]]}}}],["overview",{"_index":108,"t":{"710":{"position":[[14,8]]}}}],["p",{"_index":75,"t":{"462":{"position":[[62,1]]}}}],["p2p",{"_index":53,"t":{"251":{"position":[[94,3]]},"374":{"position":[[0,3]]},"480":{"position":[[0,3]]},"668":{"position":[[19,3]]}}}],["pair",{"_index":9,"t":{"98":{"position":[[7,7]]}}}],["part",{"_index":141,"t":{"1065":{"position":[[36,4]]}}}],["peer",{"_index":131,"t":{"962":{"position":[[16,4]]}}}],["perform",{"_index":45,"t":{"241":{"position":[[35,11]]}}}],["plan",{"_index":129,"t":{"916":{"position":[[11,4]]}}}],["polici",{"_index":161,"t":{"1239":{"position":[[8,6]]}}}],["potenti",{"_index":56,"t":{"271":{"position":[[37,9]]}}}],["present",{"_index":90,"t":{"603":{"position":[[0,10]]}}}],["preserv",{"_index":104,"t":{"668":{"position":[[8,10]]}}}],["prevent",{"_index":96,"t":{"613":{"position":[[28,10]]}}}],["privaci",{"_index":5,"t":{"58":{"position":[[9,7]]},"668":{"position":[[0,7]]},"1065":{"position":[[5,7]]},"1239":{"position":[[0,7]]}}}],["process",{"_index":165,"t":{"1268":{"position":[[8,7]]}}}],["project",{"_index":145,"t":{"1109":{"position":[[23,8]]},"1314":{"position":[[14,8]]}}}],["proof",{"_index":86,"t":{"573":{"position":[[14,6]]}}}],["properti",{"_index":64,"t":{"338":{"position":[[37,10]]}}}],["protect",{"_index":6,"t":{"58":{"position":[[17,10]]},"668":{"position":[[37,10]]}}}],["protocol",{"_index":114,"t":{"749":{"position":[[37,8]]}}}],["pseudo",{"_index":14,"t":{"106":{"position":[[9,6]]}}}],["public",{"_index":163,"t":{"1261":{"position":[[0,12]]}}}],["qualifi",{"_index":147,"t":{"1109":{"position":[[38,7]]}}}],["r&d",{"_index":166,"t":{"1270":{"position":[[4,3]]}}}],["rate",{"_index":30,"t":{"173":{"position":[[29,4]]},"613":{"position":[[44,4]]}}}],["relay",{"_index":143,"t":{"1065":{"position":[[65,5]]}}}],["remot",{"_index":77,"t":{"480":{"position":[[21,6]]}}}],["replac",{"_index":138,"t":{"1026":{"position":[[20,11]]}}}],["report",{"_index":155,"t":{"1173":{"position":[[13,7]]}}}],["research",{"_index":162,"t":{"1259":{"position":[[4,8]]}}}],["review",{"_index":3,"t":{"1":{"position":[[17,6]]}}}],["rfc",{"_index":164,"t":{"1268":{"position":[[4,3]]}}}],["rln",{"_index":85,"t":{"573":{"position":[[10,3]]},"639":{"position":[[0,3]]}}}],["rough",{"_index":107,"t":{"710":{"position":[[8,5]]}}}],["scale",{"_index":38,"t":{"221":{"position":[[28,8]]}}}],["secur",{"_index":71,"t":{"462":{"position":[[31,6]]},"1266":{"position":[[0,8]]}}}],["semaphor",{"_index":29,"t":{"173":{"position":[[19,9]]}}}],["separ",{"_index":82,"t":{"495":{"position":[[32,10]]}}}],["servic",{"_index":167,"t":{"1270":{"position":[[8,7]]}}}],["sp",{"_index":76,"t":{"462":{"position":[[64,2]]}}}],["spam",{"_index":106,"t":{"668":{"position":[[32,4]]}}}],["squar",{"_index":72,"t":{"462":{"position":[[38,6]]}}}],["state",{"_index":81,"t":{"495":{"position":[[26,5]]}}}],["strengthen",{"_index":93,"t":{"613":{"position":[[0,13]]}}}],["studi",{"_index":27,"t":{"151":{"position":[[12,6]]},"173":{"position":[[12,6]]}}}],["subtre",{"_index":89,"t":{"573":{"position":[[43,8]]}}}],["surveil",{"_index":16,"t":{"106":{"position":[[30,12]]}}}],["sync",{"_index":66,"t":{"374":{"position":[[9,4]]},"480":{"position":[[9,4]]}}}],["talk",{"_index":125,"t":{"832":{"position":[[0,5]]},"867":{"position":[[0,6]]}}}],["tech",{"_index":17,"t":{"106":{"position":[[43,4]]}}}],["term",{"_index":169,"t":{"1290":{"position":[[0,5]]}}}],["test",{"_index":154,"t":{"1173":{"position":[[5,7]]}}}],["through",{"_index":32,"t":{"173":{"position":[[43,7]]}}}],["toward",{"_index":99,"t":{"639":{"position":[[8,7]]}}}],["transform",{"_index":112,"t":{"749":{"position":[[9,12]]}}}],["tree",{"_index":25,"t":{"127":{"position":[[25,5]]}}}],["truli",{"_index":146,"t":{"1109":{"position":[[32,5]]}}}],["unit",{"_index":168,"t":{"1270":{"position":[[16,5]]}}}],["unstructur",{"_index":52,"t":{"251":{"position":[[81,12]]}}}],["updat",{"_index":111,"t":{"735":{"position":[[5,6]]},"896":{"position":[[8,6]]}}}],["us",{"_index":170,"t":{"1290":{"position":[[9,3]]}}}],["user",{"_index":79,"t":{"495":{"position":[[13,4]]}}}],["v1",{"_index":117,"t":{"777":{"position":[[5,2]]}}}],["v2",{"_index":91,"t":{"603":{"position":[[25,2]]},"668":{"position":[[56,2]]},"777":{"position":[[16,3]]},"832":{"position":[[27,2]]},"867":{"position":[[17,2]]},"896":{"position":[[5,2]]},"916":{"position":[[25,3]]},"962":{"position":[[5,2]]}}}],["v3",{"_index":98,"t":{"639":{"position":[[4,3]]}}}],["vac",{"_index":0,"t":{"1":{"position":[[0,3]]},"127":{"position":[[0,3]]},"710":{"position":[[0,3]]},"749":{"position":[[0,3]]},"806":{"position":[[0,3]]},"832":{"position":[[17,4]]},"867":{"position":[[7,4]]},"1107":{"position":[[6,3]]},"1259":{"position":[[0,3]]},"1268":{"position":[[0,3]]},"1270":{"position":[[0,3]]},"1314":{"position":[[0,3]]}}}],["verifi",{"_index":84,"t":{"573":{"position":[[0,9]]}}}],["virtual",{"_index":150,"t":{"1109":{"position":[[64,7]]},"1173":{"position":[[47,7]]}}}],["vs",{"_index":118,"t":{"777":{"position":[[8,2]]}}}],["waku",{"_index":11,"t":{"98":{"position":[[21,4],[33,4]]},"200":{"position":[[20,4]]},"221":{"position":[[14,4]]},"603":{"position":[[14,5],[20,4]]},"613":{"position":[[72,4]]},"668":{"position":[[51,4]]},"718":{"position":[[0,4]]},"735":{"position":[[0,4]]},"777":{"position":[[0,4],[11,4]]},"832":{"position":[[22,4]]},"867":{"position":[[12,4]]},"896":{"position":[[0,4]]},"916":{"position":[[20,4]]},"962":{"position":[[0,4]]},"996":{"position":[[47,4]]},"1065":{"position":[[0,4],[60,4]]}}}],["wechat",{"_index":137,"t":{"1026":{"position":[[13,6]]}}}],["what'",{"_index":128,"t":{"916":{"position":[[0,6]]}}}],["whisper",{"_index":35,"t":{"200":{"position":[[7,7]]}}}],["year",{"_index":2,"t":{"1":{"position":[[9,4]]}}}],["zero",{"_index":148,"t":{"1109":{"position":[[49,4]]},"1173":{"position":[[32,4]]}}}],["zksnark",{"_index":33,"t":{"173":{"position":[[51,8]]}}}],["zkvm",{"_index":61,"t":{"338":{"position":[[14,4]]},"1109":{"position":[[10,6]]},"1173":{"position":[[0,4]]}}}]],"pipeline":["stemmer"]}},{"documents":[{"i":3,"t":"Nescience","u":"/rlog/2024-recap","h":"#nescience","p":1},{"i":5,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights","p":1},{"i":7,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward","p":1},{"i":9,"t":"Token Economics (TKE)","u":"/rlog/2024-recap","h":"#token-economics-tke","p":1},{"i":11,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-1","p":1},{"i":13,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-1","p":1},{"i":15,"t":"Quality Assurance (QA)","u":"/rlog/2024-recap","h":"#quality-assurance-qa","p":1},{"i":17,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-2","p":1},{"i":19,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-2","p":1},{"i":21,"t":"RFC","u":"/rlog/2024-recap","h":"#rfc","p":1},{"i":23,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-3","p":1},{"i":25,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-3","p":1},{"i":27,"t":"Applied Cryptography and ZK (ACZ)","u":"/rlog/2024-recap","h":"#applied-cryptography-and-zk-acz","p":1},{"i":29,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-4","p":1},{"i":31,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-4","p":1},{"i":33,"t":"P2P","u":"/rlog/2024-recap","h":"#p2p","p":1},{"i":35,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-5","p":1},{"i":37,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-5","p":1},{"i":39,"t":"Distributed Systems Testing (DST)","u":"/rlog/2024-recap","h":"#distributed-systems-testing-dst","p":1},{"i":41,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-6","p":1},{"i":43,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-6","p":1},{"i":45,"t":"Nim","u":"/rlog/2024-recap","h":"#nim","p":1},{"i":47,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-7","p":1},{"i":49,"t":"Smart Contracts (SC)","u":"/rlog/2024-recap","h":"#smart-contracts-sc","p":1},{"i":51,"t":"Highlights","u":"/rlog/2024-recap","h":"#highlights-8","p":1},{"i":53,"t":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-7","p":1},{"i":55,"t":"Heading into 2025","u":"/rlog/2024-recap","h":"#heading-into-2025","p":1},{"i":60,"t":"Intro","u":"/rlog/building-privacy-protecting-infrastructure","h":"#intro","p":58},{"i":62,"t":"About","u":"/rlog/building-privacy-protecting-infrastructure","h":"#about","p":58},{"i":64,"t":"Why build privacy-protecting infrastructure?","u":"/rlog/building-privacy-protecting-infrastructure","h":"#why-build-privacy-protecting-infrastructure","p":58},{"i":66,"t":"Web3 infrastructure","u":"/rlog/building-privacy-protecting-infrastructure","h":"#web3-infrastructure","p":58},{"i":68,"t":"ZK for privacy-protecting infrastructure","u":"/rlog/building-privacy-protecting-infrastructure","h":"#zk-for-privacy-protecting-infrastructure","p":58},{"i":70,"t":"Waku","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku","p":58},{"i":72,"t":"Waku - adaptive nodes","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku---adaptive-nodes","p":58},{"i":74,"t":"Waku - protocol interactions","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku---protocol-interactions","p":58},{"i":76,"t":"Waku - Network","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku---network","p":58},{"i":78,"t":"Dealing with network spam and RLN Relay","u":"/rlog/building-privacy-protecting-infrastructure","h":"#dealing-with-network-spam-and-rln-relay","p":58},{"i":80,"t":"RLN - Overview and Flow","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln---overview-and-flow","p":58},{"i":82,"t":"RLN - Circuit","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln---circuit","p":58},{"i":84,"t":"RLN - Shamir's secret sharing","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln---shamirs-secret-sharing","p":58},{"i":86,"t":"RLN Relay","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln-relay","p":58},{"i":88,"t":"RLN Relay cross-client testnet","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln-relay-cross-client-testnet","p":58},{"i":90,"t":"Private settlement / Service credentials","u":"/rlog/building-privacy-protecting-infrastructure","h":"#private-settlement--service-credentials","p":58},{"i":92,"t":"Zerokit","u":"/rlog/building-privacy-protecting-infrastructure","h":"#zerokit","p":58},{"i":94,"t":"Other research","u":"/rlog/building-privacy-protecting-infrastructure","h":"#other-research","p":58},{"i":96,"t":"Summary","u":"/rlog/building-privacy-protecting-infrastructure","h":"#summary","p":58},{"i":100,"t":"Device Pairing","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"#device-pairing","p":98},{"i":102,"t":"Conclusion","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"#conclusion","p":98},{"i":104,"t":"References","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"#references","p":98},{"i":108,"t":"Preface","u":"/rlog/ethics-surveillance-tech","h":"#preface","p":106},{"i":110,"t":"Spotlight on an industry","u":"/rlog/ethics-surveillance-tech","h":"#spotlight-on-an-industry","p":106},{"i":112,"t":"A typical response","u":"/rlog/ethics-surveillance-tech","h":"#a-typical-response","p":106},{"i":114,"t":"Ethics != the law","u":"/rlog/ethics-surveillance-tech","h":"#ethics--the-law","p":106},{"i":115,"t":"The law is trailing behind","u":"/rlog/ethics-surveillance-tech","h":"#the-law-is-trailing-behind","p":106},{"i":117,"t":"The law depends on ethics","u":"/rlog/ethics-surveillance-tech","h":"#the-law-depends-on-ethics","p":106},{"i":119,"t":"International law is vague and exploitable","u":"/rlog/ethics-surveillance-tech","h":"#international-law-is-vague-and-exploitable","p":106},{"i":121,"t":"Conclusion","u":"/rlog/ethics-surveillance-tech","h":"#conclusion","p":106},{"i":123,"t":"References","u":"/rlog/ethics-surveillance-tech","h":"#references","p":106},{"i":129,"t":"Introduction","u":"/rlog/climbing-merkle-trees","h":"#introduction","p":127},{"i":131,"t":"Tree structure","u":"/rlog/climbing-merkle-trees","h":"#tree-structure","p":127},{"i":133,"t":"Merkle trees","u":"/rlog/climbing-merkle-trees","h":"#merkle-trees","p":127},{"i":135,"t":"Construction","u":"/rlog/climbing-merkle-trees","h":"#construction","p":127},{"i":137,"t":"Merkle tree intregrity","u":"/rlog/climbing-merkle-trees","h":"#merkle-tree-intregrity","p":127},{"i":139,"t":"Proof of membership","u":"/rlog/climbing-merkle-trees","h":"#proof-of-membership","p":127},{"i":141,"t":"Extensions of Merkle trees","u":"/rlog/climbing-merkle-trees","h":"#extensions-of-merkle-trees","p":127},{"i":143,"t":"Sparse Merkle trees","u":"/rlog/climbing-merkle-trees","h":"#sparse-merkle-trees","p":127},{"i":145,"t":"Proof of nonmembership","u":"/rlog/climbing-merkle-trees","h":"#proof-of-nonmembership","p":127},{"i":147,"t":"Verkle Trees","u":"/rlog/climbing-merkle-trees","h":"#verkle-trees","p":127},{"i":149,"t":"References","u":"/rlog/climbing-merkle-trees","h":"#references","p":127},{"i":153,"t":"Motivating Problem","u":"/rlog/feasibility-discv5","h":"#motivating-problem","p":151},{"i":155,"t":"Overview","u":"/rlog/feasibility-discv5","h":"#overview","p":151},{"i":157,"t":"Feasbility","u":"/rlog/feasibility-discv5","h":"#feasbility","p":151},{"i":159,"t":"CPU & Memory Usage","u":"/rlog/feasibility-discv5","h":"#cpu--memory-usage","p":151},{"i":161,"t":"NAT on Cellular Data","u":"/rlog/feasibility-discv5","h":"#nat-on-cellular-data","p":151},{"i":163,"t":"Topic Tables","u":"/rlog/feasibility-discv5","h":"#topic-tables","p":151},{"i":165,"t":"Finding a node","u":"/rlog/feasibility-discv5","h":"#finding-a-node","p":151},{"i":167,"t":"Results","u":"/rlog/feasibility-discv5","h":"#results","p":151},{"i":169,"t":"General Thoughts","u":"/rlog/feasibility-discv5","h":"#general-thoughts","p":151},{"i":171,"t":"Acknowledgements","u":"/rlog/feasibility-discv5","h":"#acknowledgements","p":151},{"i":175,"t":"Motivating problem","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#motivating-problem","p":173},{"i":177,"t":"Related problems","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#related-problems","p":173},{"i":179,"t":"Overview","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#overview","p":173},{"i":180,"t":"Basic terminology","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#basic-terminology","p":173},{"i":182,"t":"Basic flow","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#basic-flow","p":173},{"i":184,"t":"Rate limiting example","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#rate-limiting-example","p":173},{"i":186,"t":"Briefly on scope of 'approved users'","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#briefly-on-scope-of-approved-users","p":173},{"i":188,"t":"Technical details","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#technical-details","p":173},{"i":190,"t":"Feasibility","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#feasibility","p":173},{"i":192,"t":"Technical feasibility","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#technical-feasibility","p":173},{"i":194,"t":"Social feasibility","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#social-feasibility","p":173},{"i":196,"t":"General thoughts","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#general-thoughts","p":173},{"i":198,"t":"Acknowledgement","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#acknowledgement","p":173},{"i":202,"t":"Introduction","u":"/rlog/fixing-whisper-with-waku","h":"#introduction","p":200},{"i":204,"t":"Whisper theoretical scalability model","u":"/rlog/fixing-whisper-with-waku","h":"#whisper-theoretical-scalability-model","p":200},{"i":206,"t":"Caveats","u":"/rlog/fixing-whisper-with-waku","h":"#caveats","p":200},{"i":208,"t":"Goals","u":"/rlog/fixing-whisper-with-waku","h":"#goals","p":200},{"i":210,"t":"Model","u":"/rlog/fixing-whisper-with-waku","h":"#model","p":200},{"i":212,"t":"Takeaways","u":"/rlog/fixing-whisper-with-waku","h":"#takeaways","p":200},{"i":214,"t":"Introducing Waku","u":"/rlog/fixing-whisper-with-waku","h":"#introducing-waku","p":200},{"i":215,"t":"Motivation for a new protocol","u":"/rlog/fixing-whisper-with-waku","h":"#motivation-for-a-new-protocol","p":200},{"i":217,"t":"Briefly on Waku mode","u":"/rlog/fixing-whisper-with-waku","h":"#briefly-on-waku-mode","p":200},{"i":219,"t":"Progress so far","u":"/rlog/fixing-whisper-with-waku","h":"#progress-so-far","p":200},{"i":223,"t":"DOS Mitigation for Status Communities","u":"/rlog/future-of-waku-network","h":"#dos-mitigation-for-status-communities","p":221},{"i":225,"t":"A Heterogeneous Waku Network","u":"/rlog/future-of-waku-network","h":"#a-heterogeneous-waku-network","p":221},{"i":227,"t":"Reversing the Incentivization Question","u":"/rlog/future-of-waku-network","h":"#reversing-the-incentivization-question","p":221},{"i":229,"t":"RLN and Service Credentials","u":"/rlog/future-of-waku-network","h":"#rln-and-service-credentials","p":221},{"i":231,"t":"Waku Network: Ethereum or Cosmos?","u":"/rlog/future-of-waku-network","h":"#waku-network-ethereum-or-cosmos","p":221},{"i":233,"t":"Scalability and Discovery Protocols","u":"/rlog/future-of-waku-network","h":"#scalability-and-discovery-protocols","p":221},{"i":235,"t":"Open Problems","u":"/rlog/future-of-waku-network","h":"#open-problems","p":221},{"i":237,"t":"The Evolving Waku Network","u":"/rlog/future-of-waku-network","h":"#the-evolving-waku-network","p":221},{"i":239,"t":"References","u":"/rlog/future-of-waku-network","h":"#references","p":221},{"i":243,"t":"Overview","u":"/rlog/gsub-idontwant-perf-eval","h":"#overview","p":241},{"i":245,"t":"Experiments","u":"/rlog/gsub-idontwant-perf-eval","h":"#experiments","p":241},{"i":247,"t":"Findings","u":"/rlog/gsub-idontwant-perf-eval","h":"#findings","p":241},{"i":249,"t":"References","u":"/rlog/gsub-idontwant-perf-eval","h":"#references","p":241},{"i":253,"t":"Motivitation","u":"/rlog/GossipSub Improvements","h":"#motivitation","p":251},{"i":255,"t":"Introduction","u":"/rlog/GossipSub Improvements","h":"#introduction","p":251},{"i":257,"t":"GOAL1: Low Latency Operation","u":"/rlog/GossipSub Improvements","h":"#goal1-low-latency-operation","p":251},{"i":259,"t":"GOAL2: Considering Heterogeneity In Overlay Design","u":"/rlog/GossipSub Improvements","h":"#goal2-considering-heterogeneity-in-overlay-design","p":251},{"i":261,"t":"GOAL3: Bandwidth Optimization","u":"/rlog/GossipSub Improvements","h":"#goal3-bandwidth-optimization","p":251},{"i":263,"t":"GOAL4: Handling Large Messages","u":"/rlog/GossipSub Improvements","h":"#goal4-handling-large-messages","p":251},{"i":265,"t":"GOAL5: Scalability","u":"/rlog/GossipSub Improvements","h":"#goal5-scalability","p":251},{"i":267,"t":"Summary","u":"/rlog/GossipSub Improvements","h":"#summary","p":251},{"i":269,"t":"References","u":"/rlog/GossipSub Improvements","h":"","p":251},{"i":273,"t":"Motivation","u":"/rlog/gsub-largemsg-improvements","h":"#motivation","p":271},{"i":275,"t":"Problem description","u":"/rlog/gsub-largemsg-improvements","h":"#problem-description","p":271},{"i":277,"t":"Possible improvements","u":"/rlog/gsub-largemsg-improvements","h":"#possible-improvements","p":271},{"i":278,"t":"1. Minimizing transfer time for large messages","u":"/rlog/gsub-largemsg-improvements","h":"#1-minimizing-transfer-time-for-large-messages","p":271},{"i":280,"t":"2. Mitigating transport issues","u":"/rlog/gsub-largemsg-improvements","h":"#2-mitigating-transport-issues","p":271},{"i":282,"t":"3. Eliminating redundant transmissions","u":"/rlog/gsub-largemsg-improvements","h":"#3-eliminating-redundant-transmissions","p":271},{"i":284,"t":"4. Message prioritization","u":"/rlog/gsub-largemsg-improvements","h":"#4-message-prioritization","p":271},{"i":286,"t":"5. Maximizing benefits from IWANT messages","u":"/rlog/gsub-largemsg-improvements","h":"#5-maximizing-benefits-from-iwant-messages","p":271},{"i":288,"t":"Summary","u":"/rlog/gsub-largemsg-improvements","h":"#summary","p":271},{"i":290,"t":"References","u":"/rlog/gsub-largemsg-improvements","h":"#references","p":271},{"i":294,"t":"The Beginning","u":"/rlog/kademlia-to-discv5","h":"#the-beginning","p":292},{"i":296,"t":"Distributed Hash Tables","u":"/rlog/kademlia-to-discv5","h":"#distributed-hash-tables","p":292},{"i":298,"t":"Kademlia","u":"/rlog/kademlia-to-discv5","h":"#kademlia","p":292},{"i":300,"t":"Discv4","u":"/rlog/kademlia-to-discv5","h":"#discv4","p":292},{"i":302,"t":"Discv5","u":"/rlog/kademlia-to-discv5","h":"#discv5","p":292},{"i":306,"t":"Background","u":"/rlog/introducing-nwaku","h":"#background","p":304},{"i":308,"t":"1. nim-waku is now known as nwaku","u":"/rlog/introducing-nwaku","h":"#1-nim-waku-is-now-known-as-nwaku","p":304},{"i":310,"t":"2. Improvements in stability and performance","u":"/rlog/introducing-nwaku","h":"#2-improvements-in-stability-and-performance","p":304},{"i":312,"t":"3. Improvements in interoperability","u":"/rlog/introducing-nwaku","h":"#3-improvements-in-interoperability","p":304},{"i":314,"t":"4. Peer discovery","u":"/rlog/introducing-nwaku","h":"#4-peer-discovery","p":304},{"i":316,"t":"DNS-based discovery","u":"/rlog/introducing-nwaku","h":"#dns-based-discovery","p":304},{"i":318,"t":"GossipSub peer exchange","u":"/rlog/introducing-nwaku","h":"#gossipsub-peer-exchange","p":304},{"i":320,"t":"Waku Node Discovery Protocol v5","u":"/rlog/introducing-nwaku","h":"#waku-node-discovery-protocol-v5","p":304},{"i":322,"t":"5. Spam protection using RLN","u":"/rlog/introducing-nwaku","h":"#5-spam-protection-using-rln","p":304},{"i":324,"t":"Future work","u":"/rlog/introducing-nwaku","h":"#future-work","p":304},{"i":326,"t":"Reaching out to operators:","u":"/rlog/introducing-nwaku","h":"#reaching-out-to-operators","p":304},{"i":328,"t":"Better conversational security layer guarantees","u":"/rlog/introducing-nwaku","h":"#better-conversational-security-layer-guarantees","p":304},{"i":330,"t":"Protocol incentivization","u":"/rlog/introducing-nwaku","h":"#protocol-incentivization","p":304},{"i":332,"t":"Improved store capacity","u":"/rlog/introducing-nwaku","h":"#improved-store-capacity","p":304},{"i":334,"t":"Multipurpose discovery","u":"/rlog/introducing-nwaku","h":"#multipurpose-discovery","p":304},{"i":336,"t":"References","u":"/rlog/introducing-nwaku","h":"#references","p":304},{"i":340,"t":"Introduction","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#introduction","p":338},{"i":342,"t":"Goal 1: Create a State Separation Architecture","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-1-create-a-state-separation-architecture","p":338},{"i":344,"t":"Rationale Behind the Dual Architecture:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-rationale-behind-the-dual-architecture-","p":338},{"i":346,"t":"Harmonizing the Two Systems:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-harmonizing-the-two-systems-","p":338},{"i":348,"t":"Challenges and Solutions","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-challenges-and-solutions-","p":338},{"i":350,"t":"Goal 2: Virtual Machine Creation","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-2-virtual-machine-creation","p":338},{"i":352,"t":"Optimization Concerns for WASM and RISC-V:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-optimization-concerns-for-wasm-and-risc-v-","p":338},{"i":354,"t":"Emerging ZKP Optimized Systems Considerations:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-emerging-zkp-optimized-systems-considerations-","p":338},{"i":356,"t":"Strategies for Integration:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-strategies-for-integration-","p":338},{"i":358,"t":"Goal 3: Proofs Creation and Verification","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-3-proofs-creation-and-verification","p":338},{"i":360,"t":"Goal 4: Kernel-based Architecture Implementation","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-4-kernel-based-architecture-implementation","p":338},{"i":362,"t":"Goal 5: Seamless Interaction Design","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-5-seamless-interaction-design","p":338},{"i":364,"t":"Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-6-integration-of-defi-protocols-with-a-privacy-preserving-framework","p":338},{"i":366,"t":"Strategic Roadmap for Goal 6","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-strategic-roadmap-for-goal-6-","p":338},{"i":368,"t":"Summary of the Architecture","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#summary-of-the-architecture","p":338},{"i":370,"t":"Current State","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#current-state","p":338},{"i":372,"t":"References","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"","p":338},{"i":376,"t":"Problem motivation","u":"/rlog/p2p-data-sync-for-mobile","h":"#problem-motivation","p":374},{"i":378,"t":"Why p2p?","u":"/rlog/p2p-data-sync-for-mobile","h":"#why-p2p","p":374},{"i":380,"t":"Why reliably sync data?","u":"/rlog/p2p-data-sync-for-mobile","h":"#why-reliably-sync-data","p":374},{"i":382,"t":"Why mobilephones?","u":"/rlog/p2p-data-sync-for-mobile","h":"#why-mobilephones","p":374},{"i":384,"t":"Minimal Requirements","u":"/rlog/p2p-data-sync-for-mobile","h":"#minimal-requirements","p":374},{"i":386,"t":"MVDS - a minimium viable version","u":"/rlog/p2p-data-sync-for-mobile","h":"#mvds---a-minimium-viable-version","p":374},{"i":388,"t":"Basic simulation","u":"/rlog/p2p-data-sync-for-mobile","h":"#basic-simulation","p":374},{"i":390,"t":"Mostly-offline devices","u":"/rlog/p2p-data-sync-for-mobile","h":"#mostly-offline-devices","p":374},{"i":392,"t":"Basic calculations for bandwidth multiplier","u":"/rlog/p2p-data-sync-for-mobile","h":"#basic-calculations-for-bandwidth-multiplier","p":374},{"i":394,"t":"Future work","u":"/rlog/p2p-data-sync-for-mobile","h":"#future-work","p":374},{"i":397,"t":"RLN-v3: Towards a Flexible and Cost-Efficient Implementation","u":"/rlog/page/2","h":"","p":396},{"i":399,"t":"Verifying RLN Proofs in Light Clients with Subtrees","u":"/rlog/page/2","h":"","p":396},{"i":401,"t":"Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku","u":"/rlog/page/2","h":"","p":396},{"i":403,"t":"GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks","u":"/rlog/page/2","h":"","p":396},{"i":405,"t":"Nescience - A zkVM leveraging hiding properties","u":"/rlog/page/2","h":"","p":396},{"i":407,"t":"Device Pairing in Js-waku and Go-waku","u":"/rlog/page/2","h":"","p":396},{"i":409,"t":"The Future of Waku Network: Scaling, Incentivization, and Heterogeneity","u":"/rlog/page/2","h":"","p":396},{"i":411,"t":"Waku for All Decentralized Applications and Infrastructures","u":"/rlog/page/2","h":"","p":396},{"i":413,"t":"Building Privacy-Protecting Infrastructure","u":"/rlog/page/2","h":"","p":396},{"i":415,"t":"Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay","u":"/rlog/page/2","h":"","p":396},{"i":418,"t":"Noise handshakes as key-exchange mechanism for Waku","u":"/rlog/page/3","h":"","p":417},{"i":420,"t":"Waku v2 Ambient Peer Discovery","u":"/rlog/page/3","h":"","p":417},{"i":422,"t":"Introducing nwaku","u":"/rlog/page/3","h":"","p":417},{"i":424,"t":"Opinion: Pseudo-ethics in the Surveillance Tech Industry","u":"/rlog/page/3","h":"","p":417},{"i":426,"t":"Waku v1 vs Waku v2: Bandwidth Comparison","u":"/rlog/page/3","h":"","p":417},{"i":428,"t":"[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging","u":"/rlog/page/3","h":"","p":417},{"i":430,"t":"Presenting JS-Waku: Waku v2 in the Browser","u":"/rlog/page/3","h":"","p":417},{"i":432,"t":"Privacy-preserving p2p economic spam protection in Waku v2","u":"/rlog/page/3","h":"","p":417},{"i":434,"t":"[Talk] Vac, Waku v2 and Ethereum Messaging","u":"/rlog/page/3","h":"","p":417},{"i":436,"t":"Waku v2 Update","u":"/rlog/page/3","h":"","p":417},{"i":439,"t":"What's the Plan for Waku v2?","u":"/rlog/page/4","h":"","p":438},{"i":441,"t":"Feasibility Study: Discv5","u":"/rlog/page/4","h":"","p":438},{"i":443,"t":"What Would a WeChat Replacement Need?","u":"/rlog/page/4","h":"","p":438},{"i":445,"t":"From Kademlia to Discv5","u":"/rlog/page/4","h":"","p":438},{"i":447,"t":"Waku Update","u":"/rlog/page/4","h":"","p":438},{"i":449,"t":"DNS Based Discovery","u":"/rlog/page/4","h":"","p":438},{"i":451,"t":"Fixing Whisper with Waku","u":"/rlog/page/4","h":"","p":438},{"i":453,"t":"Feasibility Study: Semaphore rate limiting through zkSNARKs","u":"/rlog/page/4","h":"","p":438},{"i":455,"t":"P2P Data Sync with a Remote Log","u":"/rlog/page/4","h":"","p":438},{"i":457,"t":"Vac - A Rough Overview","u":"/rlog/page/4","h":"","p":438},{"i":460,"t":"P2P Data Sync for Mobile","u":"/rlog/page/5","h":"","p":459},{"i":464,"t":"Introduction","u":"/rlog/mdsecheck-method","h":"#introduction","p":462},{"i":466,"t":"MDS matrix: how to define and construct","u":"/rlog/mdsecheck-method","h":"#mds-matrix-how-to-define-and-construct","p":462},{"i":468,"t":"Partial substitution-permutation networks","u":"/rlog/mdsecheck-method","h":"#partial-substitution-permutation-networks","p":462},{"i":470,"t":"Square MDS matrix security check in the context of P-SPNs","u":"/rlog/mdsecheck-method","h":"#square-mds-matrix-security-check-in-the-context-of-p-spns","p":462},{"i":472,"t":"MDSECheck method: getting rid of the matrix powers","u":"/rlog/mdsecheck-method","h":"#mdsecheck-method-getting-rid-of-the-matrix-powers","p":462},{"i":474,"t":"MDSECheck library crate: implementation in Rust","u":"/rlog/mdsecheck-method","h":"#mdsecheck-library-crate-implementation-in-rust","p":462},{"i":476,"t":"Conclusion","u":"/rlog/mdsecheck-method","h":"#conclusion","p":462},{"i":478,"t":"References","u":"/rlog/mdsecheck-method","h":"#references","p":462},{"i":482,"t":"Remote log","u":"/rlog/remote-log","h":"#remote-log","p":480},{"i":484,"t":"Definitions","u":"/rlog/remote-log","h":"#definitions","p":480},{"i":486,"t":"Roles","u":"/rlog/remote-log","h":"#roles","p":480},{"i":488,"t":"Flow","u":"/rlog/remote-log","h":"#flow","p":480},{"i":489,"t":"Data format","u":"/rlog/remote-log","h":"#data-format","p":480},{"i":491,"t":"Interaction with MVDS","u":"/rlog/remote-log","h":"#interaction-with-mvds","p":480},{"i":493,"t":"Future work","u":"/rlog/remote-log","h":"#future-work","p":480},{"i":497,"t":"A. Introduction","u":"/rlog/Nescience-state-separation-architecture","h":"","p":495},{"i":499,"t":"Introducing NSSA: A user-centric approach","u":"/rlog/Nescience-state-separation-architecture","h":"#introducing-nssa-a-user-centric-approach","p":495},{"i":501,"t":"Why NSSA differs from other hybrid execution platforms","u":"/rlog/Nescience-state-separation-architecture","h":"#why-nssa-differs-from-other-hybrid-execution-platforms","p":495},{"i":503,"t":"How Nescience state-separation architecture can be used","u":"/rlog/Nescience-state-separation-architecture","h":"#how-nescience-state-separation-architecture-can-be-used","p":495},{"i":505,"t":"Use case: Adding privacy to existing dapps","u":"/rlog/Nescience-state-separation-architecture","h":"#use-case-adding-privacy-to-existing-dapps","p":495},{"i":507,"t":"Key advantage: Decoupling from the host chain","u":"/rlog/Nescience-state-separation-architecture","h":"#key-advantage-decoupling-from-the-host-chain","p":495},{"i":509,"t":"Conclusion","u":"/rlog/Nescience-state-separation-architecture","h":"#conclusion","p":495},{"i":511,"t":"B. Design","u":"/rlog/Nescience-state-separation-architecture","h":"","p":495},{"i":513,"t":"1. Architecture's components","u":"/rlog/Nescience-state-separation-architecture","h":"#1-architectures-components","p":495},{"i":515,"t":"a) Public state","u":"/rlog/Nescience-state-separation-architecture","h":"#a-public-state","p":495},{"i":517,"t":"b) Private state","u":"/rlog/Nescience-state-separation-architecture","h":"#b-private-state","p":495},{"i":519,"t":"c) ZkVM (zero-knowledge virtual machine)","u":"/rlog/Nescience-state-separation-architecture","h":"#c-zkvm-zero-knowledge-virtual-machine","p":495},{"i":521,"t":"d) Execution types in NSSA","u":"/rlog/Nescience-state-separation-architecture","h":"#d-execution-types-in-nssa","p":495},{"i":523,"t":"e) Nescience users","u":"/rlog/Nescience-state-separation-architecture","h":"#e-nescience-users","p":495},{"i":525,"t":"f) Smart contracts in NSSA","u":"/rlog/Nescience-state-separation-architecture","h":"#f-smart-contracts-in-nssa","p":495},{"i":527,"t":"2. General execution overview","u":"/rlog/Nescience-state-separation-architecture","h":"#2-general-execution-overview","p":495},{"i":529,"t":"User actions","u":"/rlog/Nescience-state-separation-architecture","h":"#user-actions","p":495},{"i":531,"t":"Sequencer actions","u":"/rlog/Nescience-state-separation-architecture","h":"#sequencer-actions","p":495},{"i":533,"t":"3. Execution processes and UTXO management","u":"/rlog/Nescience-state-separation-architecture","h":"#3-execution-processes-and-utxo-management","p":495},{"i":535,"t":"a) Components of a Nescience UTXO","u":"/rlog/Nescience-state-separation-architecture","h":"#a-components-of-a-nescience-utxo","p":495},{"i":537,"t":"b) UTXO lifecycle: From generation to consumption","u":"/rlog/Nescience-state-separation-architecture","h":"#b-utxo-lifecycle-from-generation-to-consumption","p":495},{"i":539,"t":"Summary of UTXO consumption in NSSA","u":"/rlog/Nescience-state-separation-architecture","h":"#summary-of-utxo-consumption-in-nssa","p":495},{"i":541,"t":"C. Cryptographic primitives in NSSA","u":"/rlog/Nescience-state-separation-architecture","h":"","p":495},{"i":543,"t":"a) Hash functions in Nescience","u":"/rlog/Nescience-state-separation-architecture","h":"#a-hash-functions-in-nescience","p":495},{"i":545,"t":"Use case: How to use the Pedersen hash to create the UTXO commitment","u":"/rlog/Nescience-state-separation-architecture","h":"#use-case-how-to-use-the-pedersen-hash-to-create-the-utxo-commitment","p":495},{"i":547,"t":"b) Key management and addresses in Nescience","u":"/rlog/Nescience-state-separation-architecture","h":"#-b-key-management-and-addresses-in-nescience","p":495},{"i":549,"t":"I. Spending key","u":"/rlog/Nescience-state-separation-architecture","h":"#i-spending-key","p":495},{"i":551,"t":"II. Private keys","u":"/rlog/Nescience-state-separation-architecture","h":"#ii-private-keys","p":495},{"i":553,"t":"III. Public keys","u":"/rlog/Nescience-state-separation-architecture","h":"#iii-public-keys","p":495},{"i":555,"t":"IV. Viewing key","u":"/rlog/Nescience-state-separation-architecture","h":"#iv-viewing-key","p":495},{"i":557,"t":"V. Ephemeral key","u":"/rlog/Nescience-state-separation-architecture","h":"#-v-ephemeral-key","p":495},{"i":559,"t":"VI. Nescience addresses","u":"/rlog/Nescience-state-separation-architecture","h":"#vi-nescience-addresses","p":495},{"i":561,"t":"VII. Conclusion","u":"/rlog/Nescience-state-separation-architecture","h":"#vii-conclusion","p":495},{"i":563,"t":"c) Trees in NSSA","u":"/rlog/Nescience-state-separation-architecture","h":"#c-trees-in-nssa","p":495},{"i":565,"t":"e) Recursive-friendly privacy-preserving zk-zkVM","u":"/rlog/Nescience-state-separation-architecture","h":"#e-recursive-friendly-privacy-preserving-zk-zkvm","p":495},{"i":567,"t":"f) MPC-based synchronization mechanism (under review)","u":"/rlog/Nescience-state-separation-architecture","h":"#f-mpc-based-synchronization-mechanism-under-review","p":495},{"i":569,"t":"D. Future plans for Nescience","u":"/rlog/Nescience-state-separation-architecture","h":"","p":495},{"i":571,"t":"References","u":"/rlog/Nescience-state-separation-architecture","h":"","p":495},{"i":575,"t":"Introduction","u":"/rlog/rln-light-verifiers","h":"#introduction","p":573},{"i":577,"t":"Constraints and requirements","u":"/rlog/rln-light-verifiers","h":"#constraints-and-requirements","p":573},{"i":579,"t":"Metrics on sync time for a tree with 2,653 leaves","u":"/rlog/rln-light-verifiers","h":"#metrics-on-sync-time-for-a-tree-with-2653-leaves","p":573},{"i":581,"t":"Test bench","u":"/rlog/rln-light-verifiers","h":"#test-bench","p":573},{"i":583,"t":"Metrics","u":"/rlog/rln-light-verifiers","h":"#metrics","p":573},{"i":585,"t":"Proposed solution","u":"/rlog/rln-light-verifiers","h":"#proposed-solution","p":573},{"i":587,"t":"Insertion","u":"/rlog/rln-light-verifiers","h":"#insertion","p":573},{"i":589,"t":"Syncing","u":"/rlog/rln-light-verifiers","h":"#syncing","p":573},{"i":591,"t":"Gas costs","u":"/rlog/rln-light-verifiers","h":"#gas-costs","p":573},{"i":593,"t":"Events","u":"/rlog/rln-light-verifiers","h":"#events","p":573},{"i":595,"t":"Proof of concept","u":"/rlog/rln-light-verifiers","h":"#proof-of-concept","p":573},{"i":597,"t":"Future work","u":"/rlog/rln-light-verifiers","h":"#future-work","p":573},{"i":599,"t":"Conclusion","u":"/rlog/rln-light-verifiers","h":"#conclusion","p":573},{"i":601,"t":"References","u":"/rlog/rln-light-verifiers","h":"#references","p":573},{"i":605,"t":"Waku v2","u":"/rlog/presenting-js-waku","h":"#waku-v2","p":603},{"i":607,"t":"Waku v2 in the browser","u":"/rlog/presenting-js-waku","h":"#waku-v2-in-the-browser","p":603},{"i":609,"t":"Achievements so far","u":"/rlog/presenting-js-waku","h":"#achievements-so-far","p":603},{"i":611,"t":"What's next?","u":"/rlog/presenting-js-waku","h":"#whats-next","p":603},{"i":615,"t":"Introduction","u":"/rlog/rln-anonymous-dos-prevention","h":"#introduction","p":613},{"i":617,"t":"RLN Protocol parameters","u":"/rlog/rln-anonymous-dos-prevention","h":"#rln-protocol-parameters","p":613},{"i":619,"t":"Malicious User secret interpolation mechanism","u":"/rlog/rln-anonymous-dos-prevention","h":"#malicious-user-secret-interpolation-mechanism","p":613},{"i":621,"t":"Waku's problem with DoS","u":"/rlog/rln-anonymous-dos-prevention","h":"#wakus-problem-with-dos","p":613},{"i":623,"t":"DoS prevention with user metadata","u":"/rlog/rln-anonymous-dos-prevention","h":"#dos-prevention-with-user-metadata","p":613},{"i":625,"t":"Performance analysis","u":"/rlog/rln-anonymous-dos-prevention","h":"#performance-analysis","p":613},{"i":627,"t":"Security analysis","u":"/rlog/rln-anonymous-dos-prevention","h":"#security-analysis","p":613},{"i":629,"t":"Storage analysis","u":"/rlog/rln-anonymous-dos-prevention","h":"#storage-analysis","p":613},{"i":631,"t":"The bare minimum requirements to run RLN","u":"/rlog/rln-anonymous-dos-prevention","h":"#the-bare-minimum-requirements-to-run-rln","p":613},{"i":633,"t":"RLN usage guide","u":"/rlog/rln-anonymous-dos-prevention","h":"#rln-usage-guide","p":613},{"i":635,"t":"Future work","u":"/rlog/rln-anonymous-dos-prevention","h":"#future-work","p":613},{"i":637,"t":"References","u":"/rlog/rln-anonymous-dos-prevention","h":"#references","p":613},{"i":641,"t":"Introduction","u":"/rlog/rln-v3","h":"#introduction","p":639},{"i":643,"t":"Theory","u":"/rlog/rln-v3","h":"#theory","p":639},{"i":644,"t":"Modification to leaves set in the membership Merkle tree","u":"/rlog/rln-v3","h":"#modification-to-leaves-set-in-the-membership-merkle-tree","p":639},{"i":646,"t":"Modification to circuit inputs","u":"/rlog/rln-v3","h":"#modification-to-circuit-inputs","p":639},{"i":648,"t":"Additional circuit constraints","u":"/rlog/rln-v3","h":"#additional-circuit-constraints","p":639},{"i":650,"t":"Modifications to external epoch validation (Waku, etc.)","u":"/rlog/rln-v3","h":"#modifications-to-external-epoch-validation-waku-etc","p":639},{"i":652,"t":"Modifications to double signaling detection scheme (Waku, etc.)","u":"/rlog/rln-v3","h":"#modifications-to-double-signaling-detection-scheme-waku-etc","p":639},{"i":654,"t":"The implementation","u":"/rlog/rln-v3","h":"#the-implementation","p":639},{"i":656,"t":"Comments on performance","u":"/rlog/rln-v3","h":"#comments-on-performance","p":639},{"i":658,"t":"Proving","u":"/rlog/rln-v3","h":"#proving","p":639},{"i":660,"t":"Verification","u":"/rlog/rln-v3","h":"#verification","p":639},{"i":662,"t":"Conclusion","u":"/rlog/rln-v3","h":"#conclusion","p":639},{"i":664,"t":"Future work","u":"/rlog/rln-v3","h":"#future-work","p":639},{"i":666,"t":"References","u":"/rlog/rln-v3","h":"#references","p":639},{"i":670,"t":"Introduction","u":"/rlog/rln-relay","h":"#introduction","p":668},{"i":672,"t":"What do we mean by spamming?","u":"/rlog/rln-relay","h":"#what-do-we-mean-by-spamming","p":668},{"i":674,"t":"Possible Solutions","u":"/rlog/rln-relay","h":"#possible-solutions","p":668},{"i":676,"t":"Centralized Messaging Systems","u":"/rlog/rln-relay","h":"#centralized-messaging-systems","p":668},{"i":678,"t":"P2P Systems","u":"/rlog/rln-relay","h":"#p2p-systems","p":668},{"i":680,"t":"Technical Terms","u":"/rlog/rln-relay","h":"#technical-terms","p":668},{"i":682,"t":"Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers","u":"/rlog/rln-relay","h":"#overview-economic-incentive-spam-protection-through-rate-limiting-nullifiers","p":668},{"i":684,"t":"Flow","u":"/rlog/rln-relay","h":"#flow","p":668},{"i":686,"t":"Setup and Registration","u":"/rlog/rln-relay","h":"#setup-and-registration","p":668},{"i":688,"t":"Maintaining the membership Merkle Tree","u":"/rlog/rln-relay","h":"#maintaining-the-membership-merkle-tree","p":668},{"i":690,"t":"Publishing","u":"/rlog/rln-relay","h":"#publishing","p":668},{"i":692,"t":"Routing","u":"/rlog/rln-relay","h":"#routing","p":668},{"i":694,"t":"Spam Detection and Slashing","u":"/rlog/rln-relay","h":"#spam-detection-and-slashing","p":668},{"i":696,"t":"Feasibility and Open Issues","u":"/rlog/rln-relay","h":"#feasibility-and-open-issues","p":668},{"i":698,"t":"Storage overhead per peer","u":"/rlog/rln-relay","h":"#storage-overhead-per-peer","p":668},{"i":700,"t":"Cost-effective way of member insertion and deletion","u":"/rlog/rln-relay","h":"#cost-effective-way-of-member-insertion-and-deletion","p":668},{"i":702,"t":"Exceeding the messaging rate via multiple registrations","u":"/rlog/rln-relay","h":"#exceeding-the-messaging-rate-via-multiple-registrations","p":668},{"i":704,"t":"Conclusion and Future Steps","u":"/rlog/rln-relay","h":"#conclusion-and-future-steps","p":668},{"i":706,"t":"Acknowledgement","u":"/rlog/rln-relay","h":"#acknowledgement","p":668},{"i":708,"t":"References","u":"/rlog/rln-relay","h":"#references","p":668},{"i":712,"t":"Basic terms","u":"/rlog/vac-overview","h":"#basic-terms","p":710},{"i":714,"t":"Protocol stack","u":"/rlog/vac-overview","h":"#protocol-stack","p":710},{"i":716,"t":"Problems and rough priorities","u":"/rlog/vac-overview","h":"#problems-and-rough-priorities","p":710},{"i":720,"t":"Background","u":"/rlog/waku-for-all","h":"#background","p":718},{"i":723,"t":"Waku for Your Project","u":"/rlog/waku-for-all","h":"#waku-for-your-project","p":718},{"i":725,"t":"Layer-2 Decentralization","u":"/rlog/waku-for-all","h":"#layer-2-decentralization","p":718},{"i":727,"t":"Device pairing and communication","u":"/rlog/waku-for-all","h":"#device-pairing-and-communication","p":718},{"i":729,"t":"Get Involved","u":"/rlog/waku-for-all","h":"#get-involved","p":718},{"i":731,"t":"Moving Forward","u":"/rlog/waku-for-all","h":"#moving-forward","p":718},{"i":733,"t":"References","u":"/rlog/waku-for-all","h":"#references","p":718},{"i":737,"t":"Current state","u":"/rlog/waku-update","h":"#current-state","p":735},{"i":739,"t":"How many users does Waku support?","u":"/rlog/waku-update","h":"#how-many-users-does-waku-support","p":735},{"i":741,"t":"Simulation","u":"/rlog/waku-update","h":"#simulation","p":735},{"i":743,"t":"Difference between Waku and Whisper","u":"/rlog/waku-update","h":"#difference-between-waku-and-whisper","p":735},{"i":745,"t":"Next steps and future plans","u":"/rlog/waku-update","h":"#next-steps-and-future-plans","p":735},{"i":747,"t":"Acknowledgements","u":"/rlog/waku-update","h":"#acknowledgements","p":735},{"i":751,"t":"Introduction","u":"/rlog/vac101-fiat-shamir","h":"#introduction","p":749},{"i":753,"t":"Sigma Protocols","u":"/rlog/vac101-fiat-shamir","h":"#sigma-protocols","p":749},{"i":755,"t":"The Schnorr Protocol","u":"/rlog/vac101-fiat-shamir","h":"#the-schnorr-protocol","p":749},{"i":757,"t":"Chaum-Pedersen protocol","u":"/rlog/vac101-fiat-shamir","h":"#chaum-pedersen-protocol","p":749},{"i":759,"t":"Hash Functions","u":"/rlog/vac101-fiat-shamir","h":"#hash-functions","p":749},{"i":761,"t":"The Fiat-Shamir heuristic","u":"/rlog/vac101-fiat-shamir","h":"#the-fiat-shamir-heuristic","p":749},{"i":763,"t":"Schnorr Protocol with the strong Fiat-Shamir","u":"/rlog/vac101-fiat-shamir","h":"#schnorr-protocol-with-the-strong-fiat-shamir","p":749},{"i":765,"t":"Chaum-Pedersen Protocol with the strong Fiat-Shamir","u":"/rlog/vac101-fiat-shamir","h":"#chaum-pedersen-protocol-with-the-strong-fiat-shamir","p":749},{"i":767,"t":"Improper use of the Fiat-Shamir heuristic","u":"/rlog/vac101-fiat-shamir","h":"#improper-use-of-the-fiat-shamir-heuristic","p":749},{"i":769,"t":"Schnorr protocol with the weak Fiat-Shamir heuristic","u":"/rlog/vac101-fiat-shamir","h":"#schnorr-protocol-with-the-weak-fiat-shamir-heuristic","p":749},{"i":771,"t":"Chaum-Pedersen protocol with the Fiat-Shamir heuristic","u":"/rlog/vac101-fiat-shamir","h":"#chaum-pedersen-protocol-with-the-fiat-shamir-heuristic","p":749},{"i":773,"t":"Conclusion","u":"/rlog/vac101-fiat-shamir","h":"#conclusion","p":749},{"i":775,"t":"References","u":"/rlog/vac101-fiat-shamir","h":"#references","p":749},{"i":779,"t":"Background","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#background","p":777},{"i":781,"t":"Theoretical improvements in Waku v2","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#theoretical-improvements-in-waku-v2","p":777},{"i":783,"t":"Methodology","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#methodology","p":777},{"i":785,"t":"Network size comparison","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#network-size-comparison","p":777},{"i":786,"t":"Iteration 1: 10 nodes","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#iteration-1-10-nodes","p":777},{"i":788,"t":"Iteration 2: 30 nodes","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#iteration-2-30-nodes","p":777},{"i":790,"t":"Iteration 3: 50 nodes","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#iteration-3-50-nodes","p":777},{"i":792,"t":"Iteration 4: 85 nodes","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#iteration-4-85-nodes","p":777},{"i":794,"t":"Iteration 5: 150 nodes","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#iteration-5-150-nodes","p":777},{"i":796,"t":"Discussion","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#discussion","p":777},{"i":798,"t":"Network traffic comparison","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#network-traffic-comparison","p":777},{"i":800,"t":"Conclusions","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#conclusions","p":777},{"i":802,"t":"Future work","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#future-work","p":777},{"i":804,"t":"References","u":"/rlog/waku-v1-v2-bandwidth-comparison","h":"#references","p":777},{"i":808,"t":"Membership with Bloom Filters and Cuckoo Filters","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#membership-with-bloom-filters-and-cuckoo-filters","p":806},{"i":810,"t":"Bloom filters","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#bloom-filters","p":806},{"i":812,"t":"Complexity","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#complexity","p":806},{"i":814,"t":"Example","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#example","p":806},{"i":816,"t":"Probability of false positives","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#probability-of-false-positives","p":806},{"i":818,"t":"Sliding-Window Bloom filter","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#sliding-window-bloom-filter","p":806},{"i":820,"t":"Cuckoo filters","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#cuckoo-filters","p":806},{"i":822,"t":"Example","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#example-1","p":806},{"i":824,"t":"Complexity","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#complexity-1","p":806},{"i":826,"t":"Bloom filters vs Cuckoo filters","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#bloom-filters-vs-cuckoo-filters","p":806},{"i":828,"t":"Combining Filters with RLN","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#combining-filters-with-rln","p":806},{"i":830,"t":"References","u":"/rlog/vac101-membership-with-bloom-filters-and-cuckoo-filters","h":"#references","p":806},{"i":834,"t":"Introduction","u":"/rlog/waku-v2-ethereum-coscup","h":"#introduction","p":832},{"i":836,"t":"Brief history and background","u":"/rlog/waku-v2-ethereum-coscup","h":"#brief-history-and-background","p":832},{"i":838,"t":"Waku v2","u":"/rlog/waku-v2-ethereum-coscup","h":"#waku-v2","p":832},{"i":839,"t":"Overview","u":"/rlog/waku-v2-ethereum-coscup","h":"#overview","p":832},{"i":841,"t":"Goals","u":"/rlog/waku-v2-ethereum-coscup","h":"#goals","p":832},{"i":843,"t":"Protocols","u":"/rlog/waku-v2-ethereum-coscup","h":"#protocols","p":832},{"i":845,"t":"Implementations","u":"/rlog/waku-v2-ethereum-coscup","h":"#implementations","p":832},{"i":847,"t":"Testnet Huilong and dogfooding","u":"/rlog/waku-v2-ethereum-coscup","h":"#testnet-huilong-and-dogfooding","p":832},{"i":849,"t":"Research","u":"/rlog/waku-v2-ethereum-coscup","h":"#research","p":832},{"i":851,"t":"Use cases","u":"/rlog/waku-v2-ethereum-coscup","h":"#use-cases","p":832},{"i":853,"t":"Prelude: Topics in Waku v2","u":"/rlog/waku-v2-ethereum-coscup","h":"#prelude-topics-in-waku-v2","p":832},{"i":855,"t":"Status app","u":"/rlog/waku-v2-ethereum-coscup","h":"#status-app","p":832},{"i":857,"t":"DappConnect: Ethereum messaging","u":"/rlog/waku-v2-ethereum-coscup","h":"#dappconnect-ethereum-messaging","p":832},{"i":859,"t":"WalletConnect v2","u":"/rlog/waku-v2-ethereum-coscup","h":"#walletconnect-v2","p":832},{"i":861,"t":"More examples","u":"/rlog/waku-v2-ethereum-coscup","h":"#more-examples","p":832},{"i":863,"t":"Contribute","u":"/rlog/waku-v2-ethereum-coscup","h":"#contribute","p":832},{"i":865,"t":"Conclusion","u":"/rlog/waku-v2-ethereum-coscup","h":"#conclusion","p":832},{"i":869,"t":"0. Introduction","u":"/rlog/waku-v2-ethereum-messaging","h":"#0-introduction","p":867},{"i":871,"t":"PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2","u":"/rlog/waku-v2-ethereum-messaging","h":"#part-1---vac-and-the-journey-from-whisper-to-waku-v1-to-waku-v2","p":867},{"i":872,"t":"1. Vac intro","u":"/rlog/waku-v2-ethereum-messaging","h":"#1-vac-intro","p":867},{"i":874,"t":"2. Whisper to Waku v1","u":"/rlog/waku-v2-ethereum-messaging","h":"#2-whisper-to-waku-v1","p":867},{"i":876,"t":"3. Waku v1 to v2","u":"/rlog/waku-v2-ethereum-messaging","h":"#3-waku-v1-to-v2","p":867},{"i":878,"t":"3.5 Waku v2 - Design goals","u":"/rlog/waku-v2-ethereum-messaging","h":"#35-waku-v2---design-goals","p":867},{"i":880,"t":"4. Waku v2 - Breakdown","u":"/rlog/waku-v2-ethereum-messaging","h":"#4-waku-v2---breakdown","p":867},{"i":882,"t":"5. Waku v2 - Upcoming","u":"/rlog/waku-v2-ethereum-messaging","h":"#5-waku-v2---upcoming","p":867},{"i":884,"t":"PART 2 - ETHEREUM MESSAGING","u":"/rlog/waku-v2-ethereum-messaging","h":"#part-2---ethereum-messaging","p":867},{"i":886,"t":"6. Ethereum Messaging - Why?","u":"/rlog/waku-v2-ethereum-messaging","h":"#6-ethereum-messaging---why","p":867},{"i":888,"t":"7. Ethereum Messaging - Why? (Cont)","u":"/rlog/waku-v2-ethereum-messaging","h":"#7-ethereum-messaging---why-cont","p":867},{"i":890,"t":"8. What's needed to deliver this?","u":"/rlog/waku-v2-ethereum-messaging","h":"#8-whats-needed-to-deliver-this","p":867},{"i":892,"t":"9. Breaking up the problem and a high level roadmap","u":"/rlog/waku-v2-ethereum-messaging","h":"#9-breaking-up-the-problem-and-a-high-level-roadmap","p":867},{"i":894,"t":"10. We are hiring!","u":"/rlog/waku-v2-ethereum-messaging","h":"#10-we-are-hiring","p":867},{"i":898,"t":"Recap","u":"/rlog/waku-v2-update","h":"#recap","p":896},{"i":900,"t":"Current state","u":"/rlog/waku-v2-update","h":"#current-state","p":896},{"i":902,"t":"Specs","u":"/rlog/waku-v2-update","h":"#specs","p":896},{"i":904,"t":"Implementation","u":"/rlog/waku-v2-update","h":"#implementation","p":896},{"i":906,"t":"Nangang testnet","u":"/rlog/waku-v2-update","h":"#nangang-testnet","p":896},{"i":908,"t":"Waku Web PoC","u":"/rlog/waku-v2-update","h":"#waku-web-poc","p":896},{"i":910,"t":"Coming up","u":"/rlog/waku-v2-update","h":"#coming-up","p":896},{"i":912,"t":"Things that are missing","u":"/rlog/waku-v2-update","h":"#things-that-are-missing","p":896},{"i":914,"t":"Upcoming","u":"/rlog/waku-v2-update","h":"#upcoming","p":896},{"i":918,"t":"Problem","u":"/rlog/waku-v2-plan","h":"#problem","p":916},{"i":920,"t":"Appetite","u":"/rlog/waku-v2-plan","h":"#appetite","p":916},{"i":922,"t":"Solution","u":"/rlog/waku-v2-plan","h":"#solution","p":916},{"i":924,"t":"Baseline","u":"/rlog/waku-v2-plan","h":"#baseline","p":916},{"i":926,"t":"Track 1 - Move to libp2p","u":"/rlog/waku-v2-plan","h":"#track-1---move-to-libp2p","p":916},{"i":928,"t":"1. FloodSub","u":"/rlog/waku-v2-plan","h":"#1-floodsub","p":916},{"i":930,"t":"2. Simplify the protocol","u":"/rlog/waku-v2-plan","h":"#2-simplify-the-protocol","p":916},{"i":932,"t":"3. Core integration","u":"/rlog/waku-v2-plan","h":"#3-core-integration","p":916},{"i":934,"t":"4. Topic interest behavior","u":"/rlog/waku-v2-plan","h":"#4-topic-interest-behavior","p":916},{"i":936,"t":"5. Historical message caching","u":"/rlog/waku-v2-plan","h":"#5-historical-message-caching","p":916},{"i":938,"t":"6. Waku v1 <> Libp2p bridge","u":"/rlog/waku-v2-plan","h":"#6-waku-v1--libp2p-bridge","p":916},{"i":940,"t":"Track 2 - Better routing","u":"/rlog/waku-v2-plan","h":"#track-2---better-routing","p":916},{"i":942,"t":"1. GossipSub","u":"/rlog/waku-v2-plan","h":"#1-gossipsub","p":916},{"i":944,"t":"2. Topic sharding","u":"/rlog/waku-v2-plan","h":"#2-topic-sharding","p":916},{"i":946,"t":"3. Other factors","u":"/rlog/waku-v2-plan","h":"#3-other-factors","p":916},{"i":948,"t":"Track 3 - Accounting and user-run nodes","u":"/rlog/waku-v2-plan","h":"#track-3---accounting-and-user-run-nodes","p":916},{"i":950,"t":"1. Adaptive nodes and capabilities","u":"/rlog/waku-v2-plan","h":"#1-adaptive-nodes-and-capabilities","p":916},{"i":952,"t":"2. Accounting","u":"/rlog/waku-v2-plan","h":"#2-accounting","p":916},{"i":954,"t":"3. Relax high availability requirement","u":"/rlog/waku-v2-plan","h":"#3-relax-high-availability-requirement","p":916},{"i":956,"t":"4. Incentivize light and full nodes to tell the truth (policy, etc)","u":"/rlog/waku-v2-plan","h":"#4-incentivize-light-and-full-nodes-to-tell-the-truth-policy-etc","p":916},{"i":958,"t":"5. Settlement PoC","u":"/rlog/waku-v2-plan","h":"#5-settlement-poc","p":916},{"i":960,"t":"Out of scope","u":"/rlog/waku-v2-plan","h":"#out-of-scope","p":916},{"i":964,"t":"Static Node Lists","u":"/rlog/wakuv2-apd","h":"#static-node-lists","p":962},{"i":966,"t":"DNS-based Discovery","u":"/rlog/wakuv2-apd","h":"#dns-based-discovery","p":962},{"i":968,"t":"Discovery V5","u":"/rlog/wakuv2-apd","h":"#discovery-v5","p":962},{"i":970,"t":"DHT Background","u":"/rlog/wakuv2-apd","h":"#dht-background","p":962},{"i":972,"t":"Random Walk Discovery","u":"/rlog/wakuv2-apd","h":"#random-walk-discovery","p":962},{"i":974,"t":"Random Walk Performance Estimation","u":"/rlog/wakuv2-apd","h":"#random-walk-performance-estimation","p":962},{"i":976,"t":"Simple Solution: Separate Discovery Network","u":"/rlog/wakuv2-apd","h":"#simple-solution-separate-discovery-network","p":962},{"i":978,"t":"Discv5 Topic Discovery","u":"/rlog/wakuv2-apd","h":"#discv5-topic-discovery","p":962},{"i":980,"t":"Attacks on DHTs","u":"/rlog/wakuv2-apd","h":"#attacks-on-dhts","p":962},{"i":982,"t":"Peer Exchange Protocol","u":"/rlog/wakuv2-apd","h":"#peer-exchange-protocol","p":962},{"i":984,"t":"Further Protocols Related to Discovery","u":"/rlog/wakuv2-apd","h":"#further-protocols-related-to-discovery","p":962},{"i":986,"t":"Gossipsub Peer Exchange Protocol","u":"/rlog/wakuv2-apd","h":"#gossipsub-peer-exchange-protocol","p":962},{"i":988,"t":"Capability Negotiation","u":"/rlog/wakuv2-apd","h":"#capability-negotiation","p":962},{"i":990,"t":"NAT traversal","u":"/rlog/wakuv2-apd","h":"#nat-traversal","p":962},{"i":992,"t":"Conclusion and Future Prospects","u":"/rlog/wakuv2-apd","h":"#conclusion-and-future-prospects","p":962},{"i":994,"t":"References","u":"/rlog/wakuv2-apd","h":"#references","p":962},{"i":998,"t":"Introduction","u":"/rlog/wakuv2-noise","h":"#introduction","p":996},{"i":1000,"t":"The Diffie-Hellman Key-exchange","u":"/rlog/wakuv2-noise","h":"#the-diffie-hellman-key-exchange","p":996},{"i":1002,"t":"Ephemeral and Static Public Keys","u":"/rlog/wakuv2-noise","h":"#ephemeral-and-static-public-keys","p":996},{"i":1004,"t":"The Noise Protocol Framework","u":"/rlog/wakuv2-noise","h":"#the-noise-protocol-framework","p":996},{"i":1006,"t":"Message patterns","u":"/rlog/wakuv2-noise","h":"#message-patterns","p":996},{"i":1008,"t":"The Noise State Objects","u":"/rlog/wakuv2-noise","h":"#the-noise-state-objects","p":996},{"i":1010,"t":"Supported Noise Handshakes in Waku","u":"/rlog/wakuv2-noise","h":"#supported-noise-handshakes-in-waku","p":996},{"i":1012,"t":"The K1K1 Handshake","u":"/rlog/wakuv2-noise","h":"#the-k1k1-handshake","p":996},{"i":1014,"t":"The XK1 Handshake","u":"/rlog/wakuv2-noise","h":"#the-xk1-handshake","p":996},{"i":1016,"t":"The XX and XXpsk0 Handshakes","u":"/rlog/wakuv2-noise","h":"#the-xx-and-xxpsk0-handshakes","p":996},{"i":1018,"t":"Session Management and Multi-Device Support","u":"/rlog/wakuv2-noise","h":"#session-management-and-multi-device-support","p":996},{"i":1020,"t":"Conclusions","u":"/rlog/wakuv2-noise","h":"#conclusions","p":996},{"i":1022,"t":"Future steps","u":"/rlog/wakuv2-noise","h":"#future-steps","p":996},{"i":1024,"t":"References","u":"/rlog/wakuv2-noise","h":"#references","p":996},{"i":1028,"t":"Background","u":"/rlog/wechat-replacement-need","h":"#background","p":1026},{"i":1029,"t":"What WeChat provides to the end-user","u":"/rlog/wechat-replacement-need","h":"#what-wechat-provides-to-the-end-user","p":1026},{"i":1031,"t":"How WeChat works - a toy model","u":"/rlog/wechat-replacement-need","h":"#how-wechat-works---a-toy-model","p":1026},{"i":1033,"t":"What do we want?","u":"/rlog/wechat-replacement-need","h":"#what-do-we-want","p":1026},{"i":1035,"t":"Counterpoint 1","u":"/rlog/wechat-replacement-need","h":"#counterpoint-1","p":1026},{"i":1037,"t":"Counterpoint 2","u":"/rlog/wechat-replacement-need","h":"#counterpoint-2","p":1026},{"i":1039,"t":"Principal components","u":"/rlog/wechat-replacement-need","h":"#principal-components","p":1026},{"i":1041,"t":"Background: Ethereum and Web3 stack","u":"/rlog/wechat-replacement-need","h":"#background-ethereum-and-web3-stack","p":1026},{"i":1043,"t":"Account - self-sovereign identity and the perils of phone numbers","u":"/rlog/wechat-replacement-need","h":"#account---self-sovereign-identity-and-the-perils-of-phone-numbers","p":1026},{"i":1045,"t":"Routing - packets from A to B","u":"/rlog/wechat-replacement-need","h":"#routing---packets-from-a-to-b","p":1026},{"i":1047,"t":"Storage - available and persistant for later","u":"/rlog/wechat-replacement-need","h":"#storage---available-and-persistant-for-later","p":1026},{"i":1049,"t":"Messaging - from me to you to all of us (not them)","u":"/rlog/wechat-replacement-need","h":"#messaging---from-me-to-you-to-all-of-us-not-them","p":1026},{"i":1051,"t":"Compute - transact, contract and settle","u":"/rlog/wechat-replacement-need","h":"#compute---transact-contract-and-settle","p":1026},{"i":1053,"t":"Secure chat - just our business","u":"/rlog/wechat-replacement-need","h":"#secure-chat---just-our-business","p":1026},{"i":1055,"t":"Extensible mini apps","u":"/rlog/wechat-replacement-need","h":"#extensible-mini-apps","p":1026},{"i":1057,"t":"Where are we now?","u":"/rlog/wechat-replacement-need","h":"#where-are-we-now","p":1026},{"i":1059,"t":"What's next?","u":"/rlog/wechat-replacement-need","h":"#whats-next","p":1026},{"i":1061,"t":"Acknowledgements","u":"/rlog/wechat-replacement-need","h":"#acknowledgements","p":1026},{"i":1063,"t":"References","u":"/rlog/wechat-replacement-need","h":"#references","p":1026},{"i":1067,"t":"Informal Definitions: Security, Privacy, and Anonymity","u":"/rlog/wakuv2-relay-anon","h":"#informal-definitions-security-privacy-and-anonymity","p":1065},{"i":1069,"t":"Security","u":"/rlog/wakuv2-relay-anon","h":"#security","p":1065},{"i":1071,"t":"Privacy","u":"/rlog/wakuv2-relay-anon","h":"#privacy","p":1065},{"i":1073,"t":"Anonymity","u":"/rlog/wakuv2-relay-anon","h":"#anonymity","p":1065},{"i":1075,"t":"Anonymity Trilemma","u":"/rlog/wakuv2-relay-anon","h":"#anonymity-trilemma","p":1065},{"i":1077,"t":"Censorship Resistance","u":"/rlog/wakuv2-relay-anon","h":"#censorship-resistance","p":1065},{"i":1079,"t":"Attacker Types","u":"/rlog/wakuv2-relay-anon","h":"#attacker-types","p":1065},{"i":1081,"t":"Internal","u":"/rlog/wakuv2-relay-anon","h":"#internal","p":1065},{"i":1083,"t":"External","u":"/rlog/wakuv2-relay-anon","h":"#external","p":1065},{"i":1085,"t":"Attack-based Threat Analysis","u":"/rlog/wakuv2-relay-anon","h":"#attack-based-threat-analysis","p":1065},{"i":1087,"t":"Scope","u":"/rlog/wakuv2-relay-anon","h":"#scope","p":1065},{"i":1089,"t":"Prerequisite: Get a Specific Position in the Network","u":"/rlog/wakuv2-relay-anon","h":"#prerequisite-get-a-specific-position-in-the-network","p":1065},{"i":1091,"t":"Replay Attack","u":"/rlog/wakuv2-relay-anon","h":"#replay-attack","p":1065},{"i":1093,"t":"Neighbourhood Surveillance","u":"/rlog/wakuv2-relay-anon","h":"#neighbourhood-surveillance","p":1065},{"i":1095,"t":"Controlled Neighbourhood","u":"/rlog/wakuv2-relay-anon","h":"#controlled-neighbourhood","p":1065},{"i":1097,"t":"Observing Messages","u":"/rlog/wakuv2-relay-anon","h":"#observing-messages","p":1065},{"i":1099,"t":"Correlation","u":"/rlog/wakuv2-relay-anon","h":"#correlation","p":1065},{"i":1101,"t":"DoS","u":"/rlog/wakuv2-relay-anon","h":"#dos","p":1065},{"i":1103,"t":"Summary and Future Work","u":"/rlog/wakuv2-relay-anon","h":"#summary-and-future-work","p":1065},{"i":1105,"t":"References","u":"/rlog/wakuv2-relay-anon","h":"#references","p":1065},{"i":1110,"t":"Introduction","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1112,"t":"What is a zkVM?","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1114,"t":"Why zkVMs matter","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1116,"t":"Our methodology","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1118,"t":"ZkVM project analysis","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1119,"t":"1. [SP1]","u":"/rlog/zkVM-explorations","h":"#1-sp1","p":1109},{"i":1121,"t":"2. [Nexus]","u":"/rlog/zkVM-explorations","h":"#2-nexus","p":1109},{"i":1123,"t":"3. [RISC0]","u":"/rlog/zkVM-explorations","h":"#3-risc0","p":1109},{"i":1125,"t":"4. [Powdr]","u":"/rlog/zkVM-explorations","h":"#4-powdr","p":1109},{"i":1127,"t":"5. [ZkMIPS]","u":"/rlog/zkVM-explorations","h":"#5-zkmips","p":1109},{"i":1129,"t":"6. [Valida]","u":"/rlog/zkVM-explorations","h":"#6-valida","p":1109},{"i":1131,"t":"7. [Jolt]","u":"/rlog/zkVM-explorations","h":"#7-jolt","p":1109},{"i":1133,"t":"8. [ZkWASM]","u":"/rlog/zkVM-explorations","h":"#8-zkwasm","p":1109},{"i":1135,"t":"9. [Aleo]","u":"/rlog/zkVM-explorations","h":"#9-aleo","p":1109},{"i":1137,"t":"10. [Ola]","u":"/rlog/zkVM-explorations","h":"#10-ola","p":1109},{"i":1139,"t":"11. [Miden]","u":"/rlog/zkVM-explorations","h":"#11-miden","p":1109},{"i":1141,"t":"12. [ZkOS]","u":"/rlog/zkVM-explorations","h":"#12-zkos","p":1109},{"i":1143,"t":"13. [Triton]","u":"/rlog/zkVM-explorations","h":"#13-triton","p":1109},{"i":1145,"t":"14. [Cairo]","u":"/rlog/zkVM-explorations","h":"#14-cairo","p":1109},{"i":1147,"t":"15. [SnarkOS]","u":"/rlog/zkVM-explorations","h":"#15-snarkos","p":1109},{"i":1149,"t":"16. [Lurk]","u":"/rlog/zkVM-explorations","h":"#16-lurk","p":1109},{"i":1151,"t":"17. [Piecrust]","u":"/rlog/zkVM-explorations","h":"#17-piecrust","p":1109},{"i":1153,"t":"18. [Ceno]","u":"/rlog/zkVM-explorations","h":"#18-ceno","p":1109},{"i":1155,"t":"19. [Stellar]","u":"/rlog/zkVM-explorations","h":"#19-stellar","p":1109},{"i":1157,"t":"20. [NovaNet]","u":"/rlog/zkVM-explorations","h":"#20-novanet","p":1109},{"i":1159,"t":"21. [ZkLLVM]","u":"/rlog/zkVM-explorations","h":"#21-zkllvm","p":1109},{"i":1161,"t":"22. [ZkMove]","u":"/rlog/zkVM-explorations","h":"#22-zkmove","p":1109},{"i":1163,"t":"23. [O1VM]","u":"/rlog/zkVM-explorations","h":"#23-o1vm","p":1109},{"i":1165,"t":"Summary of findings","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1167,"t":"Insights and conclusions","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1169,"t":"References","u":"/rlog/zkVM-explorations","h":"","p":1109},{"i":1174,"t":"Introduction","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1176,"t":"Why these candidates?","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1178,"t":"Preliminary information on the candidates","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1180,"t":"Testing plan","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1182,"t":"Machine specifications","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1184,"t":"Results","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1185,"t":"1. SP1","u":"/rlog/zkVM-testing","h":"#1-sp1","p":1173},{"i":1187,"t":"2. RISC0","u":"/rlog/zkVM-testing","h":"#2-risc0","p":1173},{"i":1189,"t":"3. Nexus","u":"/rlog/zkVM-testing","h":"#3-nexus","p":1173},{"i":1191,"t":"4. ZkMIPS","u":"/rlog/zkVM-testing","h":"#4-zkmips","p":1173},{"i":1193,"t":"5. ZkWASM","u":"/rlog/zkVM-testing","h":"#5-zkwasm","p":1173},{"i":1195,"t":"6. Valida","u":"/rlog/zkVM-testing","h":"#6-valida","p":1173},{"i":1197,"t":"Summary table","u":"/rlog/zkVM-testing","h":"#summary-table","p":1173},{"i":1198,"t":"Stage 1","u":"/rlog/zkVM-testing","h":"#stage-1","p":1173},{"i":1200,"t":"Stage 2","u":"/rlog/zkVM-testing","h":"#stage-2","p":1173},{"i":1202,"t":"Summary","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1204,"t":"References","u":"/rlog/zkVM-testing","h":"","p":1173},{"i":1209,"t":"How to Contribute","u":"/contribute","h":"#how-to-contribute","p":1208},{"i":1211,"t":"What to Contribute","u":"/contribute","h":"#what-to-contribute","p":1208},{"i":1215,"t":"Principles","u":"/principles","h":"","p":1213},{"i":1217,"t":"I. Liberty","u":"/principles","h":"#i-liberty","p":1213},{"i":1219,"t":"II. Censorship resistance","u":"/principles","h":"#ii-censorship-resistance","p":1213},{"i":1221,"t":"III. Security","u":"/principles","h":"#iii-security","p":1213},{"i":1223,"t":"IV. Privacy","u":"/principles","h":"#iv-privacy","p":1213},{"i":1225,"t":"V. Transparency","u":"/principles","h":"#v-transparency","p":1213},{"i":1227,"t":"VI. Openness","u":"/principles","h":"#vi-openness","p":1213},{"i":1229,"t":"VII. Decentralisation","u":"/principles","h":"#vii-decentralisation","p":1213},{"i":1231,"t":"VIII. Inclusivity","u":"/principles","h":"#viii-inclusivity","p":1213},{"i":1233,"t":"IX. Continuance","u":"/principles","h":"#ix-continuance","p":1213},{"i":1235,"t":"X. Resourcefulness","u":"/principles","h":"#x-resourcefulness","p":1213},{"i":1241,"t":"1) Who we are","u":"/privacy-policy","h":"#1-who-we-are","p":1239},{"i":1243,"t":"2) We limit the collection and processing of personal data from your use of the Website","u":"/privacy-policy","h":"#2-we-limit-the-collection-and-processing-of-personal-data-from-your-use-of-the-website","p":1239},{"i":1245,"t":"3) Third party processing of personal data","u":"/privacy-policy","h":"#3-third-party-processing-of-personal-data","p":1239},{"i":1247,"t":"4) Security measures we take in respect of the Website","u":"/privacy-policy","h":"#4-security-measures-we-take-in-respect-of-the-website","p":1239},{"i":1249,"t":"5) Exporting data outside the European Union and Switzerland","u":"/privacy-policy","h":"#5-exporting-data-outside-the-european-union-and-switzerland","p":1239},{"i":1251,"t":"6) Your choices and rights","u":"/privacy-policy","h":"#6-your-choices-and-rights","p":1239},{"i":1253,"t":"7) Third party links","u":"/privacy-policy","h":"#7-third-party-links","p":1239},{"i":1255,"t":"8) This Privacy Policy might change","u":"/privacy-policy","h":"#8-this-privacy-policy-might-change","p":1239},{"i":1257,"t":"9) Contact information","u":"/privacy-policy","h":"#9-contact-information","p":1239},{"i":1262,"t":"Papers","u":"/publications","h":"#papers","p":1261},{"i":1264,"t":"Write-ups","u":"/publications","h":"#write-ups","p":1261},{"i":1272,"t":"P2P","u":"/vsus","h":"#p2p","p":1270},{"i":1274,"t":"Token Economics (TKE)","u":"/vsus","h":"#token-economics-tke","p":1270},{"i":1276,"t":"Distributed Systems Testing (DST)","u":"/vsus","h":"#distributed-systems-testing-dst","p":1270},{"i":1278,"t":"Quality Assurance (QA)","u":"/vsus","h":"#quality-assurance-qa","p":1270},{"i":1280,"t":"Smart Contracts (SC)","u":"/vsus","h":"#smart-contracts-sc","p":1270},{"i":1282,"t":"Nim","u":"/vsus","h":"#nim","p":1270},{"i":1284,"t":"Applied Cryptography & ZK (ACZ)","u":"/vsus","h":"#applied-cryptography--zk-acz","p":1270},{"i":1286,"t":"RFC","u":"/vsus","h":"#rfc","p":1270},{"i":1288,"t":"Security","u":"/vsus","h":"#security","p":1270},{"i":1292,"t":"1) Who we are","u":"/terms","h":"#1-who-we-are","p":1290},{"i":1294,"t":"2) Disclaimers","u":"/terms","h":"#2-disclaimers","p":1290},{"i":1296,"t":"3) Forward looking statements","u":"/terms","h":"#3-forward-looking-statements","p":1290},{"i":1298,"t":"4) Intellectual property rights","u":"/terms","h":"#4-intellectual-property-rights","p":1290},{"i":1300,"t":"5) Third-party website links","u":"/terms","h":"#5-third-party-website-links","p":1290},{"i":1302,"t":"6) Limitation of liability","u":"/terms","h":"#6-limitation-of-liability","p":1290},{"i":1304,"t":"7) Indemnity","u":"/terms","h":"#7-indemnity","p":1290},{"i":1306,"t":"8) Modifications","u":"/terms","h":"#8-modifications","p":1290},{"i":1308,"t":"9) Governing law","u":"/terms","h":"#9-governing-law","p":1290},{"i":1310,"t":"10) Disputes","u":"/terms","h":"#10-disputes","p":1290},{"i":1312,"t":"11) About these Website Terms of Use","u":"/terms","h":"#11-about-these-website-terms-of-use","p":1290},{"i":1316,"t":"Nescience","u":"/vips","h":"#nescience","p":1314}],"index":{"version":"2.3.9","fields":["t"],"fieldVectors":[["t/3",[0,5.47]],["t/5",[1,5.605]],["t/7",[2,4.655,3,4.543]],["t/9",[4,5.256,5,4.694,6,5.256]],["t/11",[1,5.605]],["t/13",[2,4.655,3,4.543]],["t/15",[7,5.256,8,5.256,9,5.256]],["t/17",[1,5.605]],["t/19",[2,4.655,3,4.543]],["t/21",[10,7.403]],["t/23",[1,5.605]],["t/25",[2,4.655,3,4.543]],["t/27",[11,4.591,12,4.591,13,4.1,14,4.591]],["t/29",[1,5.605]],["t/31",[2,4.655,3,4.543]],["t/33",[15,5.755]],["t/35",[1,5.605]],["t/37",[2,4.655,3,4.543]],["t/39",[16,4.31,17,3.793,18,4.1,19,4.591]],["t/41",[1,5.605]],["t/43",[2,4.655,3,4.543]],["t/45",[20,6.95]],["t/47",[1,5.605]],["t/49",[21,4.935,22,4.694,23,5.256]],["t/51",[1,5.605]],["t/53",[2,4.655,3,4.543]],["t/55",[24,6.719,25,6.719]],["t/60",[26,7.403]],["t/62",[]],["t/64",[27,4.591,28,3.247,29,3.793,30,3.932]],["t/66",[30,5.266,31,6.148]],["t/68",[13,4.1,28,3.247,29,3.793,30,3.932]],["t/70",[32,3.466]],["t/72",[32,2.461,33,5.256,34,3.717]],["t/74",[32,2.461,35,3.199,36,4.935]],["t/76",[32,2.879,37,4.348]],["t/78",[37,2.882,38,4.453,39,3.366,40,2.772,41,3.639]],["t/80",[40,3.576,42,4.086,43,4.694]],["t/82",[40,4.182,44,5.771]],["t/84",[40,3.123,45,5.017,46,4.591,47,5.017]],["t/86",[40,4.182,41,5.49]],["t/88",[40,2.772,41,3.639,48,4.453,49,4.075,50,3.825]],["t/90",[51,3.825,52,4.075,53,3.49,54,4.075,55,4.075]],["t/92",[56,8.091]],["t/94",[57,7.403]],["t/96",[58,5.605]],["t/100",[59,5.266,60,5.771]],["t/102",[61,5.036]],["t/104",[62,4.276]],["t/108",[63,8.091]],["t/110",[64,6.719,65,6.148]],["t/112",[66,6.719,67,6.719]],["t/114",[53,4.502,68,4.935,69,4.502]],["t/115",[69,4.502,70,5.745,71,5.256]],["t/117",[68,4.935,69,4.502,72,5.745]],["t/119",[69,3.932,73,4.591,74,5.017,75,5.017]],["t/121",[61,5.036]],["t/123",[62,4.276]],["t/129",[76,4.862]],["t/131",[77,4.543,78,6.719]],["t/133",[77,4.543,79,5.079]],["t/135",[80,7.403]],["t/137",[77,3.884,79,4.343,81,5.745]],["t/139",[82,5.266,83,5.49]],["t/141",[77,3.884,79,4.343,84,5.256]],["t/143",[77,3.884,79,4.343,85,5.745]],["t/145",[82,5.266,86,6.719]],["t/147",[77,4.543,87,6.719]],["t/149",[62,4.276]],["t/153",[88,5.266,89,4.543]],["t/155",[42,5.755]],["t/157",[90,8.091]],["t/159",[53,3.932,91,5.017,92,5.017,93,4.591]],["t/161",[94,5.256,95,5.745,96,4.086]],["t/163",[97,5.266,98,5.771]],["t/165",[34,4.348,99,5.771]],["t/167",[100,7.403]],["t/169",[101,5.49,102,6.148]],["t/171",[103,6.341]],["t/175",[88,5.266,89,4.543]],["t/177",[89,4.543,104,6.148]],["t/179",[42,5.755]],["t/180",[105,5.266,106,6.719]],["t/182",[43,5.49,105,5.266]],["t/184",[107,4.502,108,4.343,109,4.694]],["t/186",[110,4.591,111,4.31,112,5.017,113,3.476]],["t/188",[114,5.771,115,6.719]],["t/190",[116,6.116]],["t/192",[114,5.771,116,5.079]],["t/194",[116,5.079,117,6.719]],["t/196",[101,5.49,102,6.148]],["t/198",[103,6.341]],["t/202",[76,4.862]],["t/204",[118,3.932,119,4.591,120,4.31,121,4.31]],["t/206",[122,8.091]],["t/208",[123,5.47]],["t/210",[121,6.95]],["t/212",[124,8.091]],["t/214",[32,2.879,125,5.771]],["t/215",[35,3.199,88,4.502,126,5.745]],["t/217",[32,2.461,110,5.256,127,5.745]],["t/219",[128,6.719,129,6.148]],["t/223",[130,3.932,131,4.591,132,4.591,133,4.591]],["t/225",[32,2.461,37,3.717,134,4.935]],["t/227",[135,5.745,136,4.694,137,5.745]],["t/229",[40,3.576,54,5.256,55,5.256]],["t/231",[32,2.15,37,3.247,138,3.569,139,5.017]],["t/233",[35,3.199,120,4.935,140,3.644]],["t/235",[89,4.543,141,5.771]],["t/237",[32,2.461,37,3.717,142,5.745]],["t/239",[62,4.276]],["t/243",[42,5.755]],["t/245",[143,8.091]],["t/247",[99,6.95]],["t/249",[62,4.276]],["t/253",[144,8.091]],["t/255",[76,4.862]],["t/257",[145,5.017,146,5.017,147,5.017,148,4.591]],["t/259",[134,3.825,149,4.453,150,4.453,151,4.075,152,3.49]],["t/261",[153,5.745,154,4.935,155,4.935]],["t/263",[156,5.017,157,5.017,158,4.591,159,2.966]],["t/265",[120,5.771,160,6.719]],["t/267",[58,5.605]],["t/269",[62,4.276]],["t/273",[88,6.341]],["t/275",[89,4.543,161,6.719]],["t/277",[162,6.148,163,5.079]],["t/278",[158,3.663,159,2.366,164,2.366,165,3.663,166,4.003,167,3.663]],["t/280",[131,4.591,168,2.919,169,5.017,170,4.591]],["t/282",[171,3.123,172,5.017,173,5.017,174,5.017]],["t/284",[159,3.396,175,3.797,176,5.745]],["t/286",[159,2.632,177,2.944,178,4.453,179,4.453,180,4.453]],["t/288",[58,5.605]],["t/290",[62,4.276]],["t/294",[181,8.091]],["t/296",[16,4.935,98,4.935,182,4.694]],["t/298",[183,7.403]],["t/300",[184,8.091]],["t/302",[185,6.612]],["t/306",[186,5.924]],["t/308",[20,3.439,32,1.715,164,2.366,187,3.663,188,4.003,189,3.663]],["t/310",[163,3.793,168,2.919,190,5.017,191,4.1]],["t/312",[163,4.343,171,3.576,192,5.745]],["t/314",[140,3.644,175,3.797,193,4.343]],["t/316",[140,3.644,194,4.935,195,4.343]],["t/318",[193,4.343,196,4.694,197,4.502]],["t/320",[32,1.908,34,2.882,35,2.48,140,2.825,198,4.075]],["t/322",[29,3.366,39,3.366,40,2.772,177,2.944,199,3.168]],["t/324",[200,4.182,201,4.655]],["t/326",[148,5.256,202,5.745,203,5.256]],["t/328",[204,4.075,205,4.453,206,3.085,207,4.075,208,4.453]],["t/330",[35,3.741,136,5.49]],["t/332",[163,4.343,209,5.745,210,5.745]],["t/334",[140,4.262,211,6.719]],["t/336",[62,4.276]],["t/340",[76,4.862]],["t/342",[123,2.707,164,2.366,212,3.663,213,2.848,214,3.439,215,3.138]],["t/344",[71,4.591,215,3.932,216,5.017,217,5.017]],["t/346",[17,4.343,218,5.745,219,5.745]],["t/348",[220,6.719,221,5.266]],["t/350",[123,3.011,168,2.591,222,4.075,223,3.825,224,4.075]],["t/352",[155,3.825,225,4.453,226,4.453,227,4.453,228,3.825]],["t/354",[17,3.366,155,3.825,229,4.453,230,4.453,231,4.453]],["t/356",[232,6.719,233,5.771]],["t/358",[82,3.49,123,3.011,171,2.772,224,4.075,234,4.075]],["t/360",[123,2.707,175,2.646,195,3.026,215,3.138,235,4.003,236,3.026]],["t/362",[36,3.825,123,3.011,152,3.49,177,2.944,237,4.453]],["t/364",[28,2.155,35,1.854,123,2.252,233,2.861,238,2.369,239,3.33,240,2.861,241,3.047]],["t/366",[123,3.392,238,3.569,242,5.017,243,4.591]],["t/368",[58,4.655,215,5.266]],["t/370",[213,4.779,244,5.771]],["t/372",[62,4.276]],["t/376",[88,5.266,89,4.543]],["t/378",[15,5.755]],["t/380",[96,4.086,245,5.745,246,4.502]],["t/382",[247,8.091]],["t/384",[165,6.148,248,5.49]],["t/386",[249,4.591,250,5.017,251,5.017,252,5.017]],["t/388",[105,5.266,253,6.148]],["t/390",[59,4.502,254,5.745,255,5.745]],["t/392",[105,3.932,154,4.31,256,5.017,257,5.017]],["t/394",[200,4.182,201,4.655]],["t/397",[40,2.263,236,2.749,258,3.636,259,3.636,260,3.636,261,3.123,262,3.636]],["t/399",[40,2.492,49,3.663,82,3.138,263,4.003,264,3.663,265,4.003]],["t/401",[32,1.427,107,2.61,108,2.518,130,2.61,266,3.33,267,2.61,268,3.047,269,3.047]],["t/403",[15,2.028,37,1.845,151,2.609,152,2.235,159,1.685,163,2.155,196,2.33,270,2.851,271,2.851,272,2.851]],["t/405",[0,3.011,273,3.366,274,4.453,275,4.453,276,4.075]],["t/407",[32,2.633,59,3.138,60,3.439,277,3.663,278,4.003]],["t/409",[32,1.715,37,2.591,134,3.439,136,3.271,200,2.492,279,4.003]],["t/411",[30,3.932,32,2.15,280,4.591,281,5.017]],["t/413",[27,4.591,28,3.247,29,3.793,30,3.932]],["t/415",[28,2.155,32,2.279,41,2.721,267,2.61,282,2.518,283,2.861,284,2.861]],["t/418",[32,1.715,197,3.138,285,3.271,286,3.138,287,2.707,288,3.439]],["t/420",[32,1.908,140,2.825,193,3.366,289,2.552,290,4.453]],["t/422",[125,5.771,189,6.148]],["t/424",[65,3.663,68,3.439,291,4.003,292,4.003,293,3.663,294,4.003]],["t/426",[32,2.443,154,3.123,289,2.084,295,2.85,296,3.327,297,3.123]],["t/428",[32,1.558,138,2.586,159,2.149,289,2.084,298,3.327,299,3.636,300,2.85]],["t/430",[32,2.633,277,3.663,289,2.294,301,4.003,302,3.663]],["t/432",[5,2.721,15,2.369,28,2.155,29,2.518,32,1.427,39,2.518,240,2.861,289,1.909]],["t/434",[32,1.715,138,2.848,159,2.366,289,2.294,298,3.663,300,3.138]],["t/436",[32,2.461,289,3.292,303,5.256]],["t/439",[32,2.15,289,2.875,304,4.1,305,4.1]],["t/441",[116,4.343,185,4.694,306,5.256]],["t/443",[307,4.935,308,5.745,309,5.256]],["t/445",[183,6.148,185,5.49]],["t/447",[32,2.879,303,6.148]],["t/449",[140,3.644,194,4.935,195,4.343]],["t/451",[32,2.461,118,4.502,310,5.745]],["t/453",[107,2.85,108,2.749,116,2.749,306,3.327,311,3.636,312,3.327,313,3.636]],["t/455",[15,3.168,96,3.168,246,3.49,314,4.075,315,4.075]],["t/457",[42,4.086,300,4.502,316,5.256]],["t/460",[15,3.569,96,3.569,246,3.932,317,5.017]],["t/464",[76,4.862]],["t/466",[80,4.591,318,4.591,319,4.31,320,5.017]],["t/468",[37,3.247,321,5.017,322,5.017,323,5.017]],["t/470",[206,2.307,318,3.047,319,2.861,324,3.33,325,3.33,326,3.33,327,3.33,328,3.33]],["t/472",[319,3.439,329,3.663,330,4.003,331,4.003,332,4.003,333,4.003]],["t/474",[236,3.366,329,4.075,334,4.453,335,4.453,336,4.453]],["t/476",[61,5.036]],["t/478",[62,4.276]],["t/482",[314,6.148,315,6.148]],["t/484",[284,6.95]],["t/486",[337,8.091]],["t/488",[43,6.612]],["t/489",[96,4.779,338,6.719]],["t/491",[36,5.771,249,6.148]],["t/493",[200,4.182,201,4.655]],["t/497",[76,4.862]],["t/499",[113,3.085,125,3.825,339,3.26,340,4.453,341,4.453]],["t/501",[339,3.26,342,4.075,343,4.453,344,3.639,345,4.453]],["t/503",[0,3.011,199,3.168,213,3.168,214,3.825,215,3.49]],["t/505",[28,2.591,199,2.848,346,3.439,347,4.003,348,4.003,349,4.003]],["t/507",[287,3.011,350,4.453,351,4.453,352,4.453,353,4.453]],["t/509",[61,5.036]],["t/511",[152,5.266,354,5.266]],["t/513",[164,3.396,355,5.745,356,4.935]],["t/515",[213,4.779,357,5.771]],["t/517",[51,4.935,213,4.086,354,4.502]],["t/519",[222,3.663,223,3.439,273,3.026,358,3.439,359,4.003,360,4.003]],["t/521",[339,3.673,344,4.1,361,4.591,362,4.591]],["t/523",[0,3.884,113,3.98,363,5.256]],["t/525",[21,4.31,22,4.1,339,3.673,364,4.591]],["t/527",[42,3.569,101,4.1,168,2.919,344,4.1]],["t/529",[113,4.655,365,6.148]],["t/531",[365,6.148,366,6.719]],["t/533",[171,2.772,344,3.639,367,3.825,368,3.49,369,3.825]],["t/535",[0,3.884,356,4.935,368,4.502]],["t/537",[101,3.639,354,3.49,368,3.49,370,4.453,371,4.075]],["t/539",[58,3.476,339,3.673,368,3.932,371,4.591]],["t/541",[339,3.673,358,4.31,372,5.017,373,5.017]],["t/543",[0,3.884,182,4.694,374,5.256]],["t/545",[182,2.721,199,3.784,212,3.047,346,2.861,368,2.61,375,2.721,376,3.33]],["t/547",[0,3.011,287,3.011,354,3.49,369,3.825,377,4.075]],["t/549",[287,4.543,378,6.719]],["t/551",[51,4.935,287,3.884,379,5.256]],["t/553",[287,3.884,357,4.935,380,5.256]],["t/555",[287,3.884,381,5.256,382,5.745]],["t/557",[228,4.935,287,3.884,383,5.256]],["t/559",[0,3.884,377,5.256,384,5.256]],["t/561",[61,4.182,385,6.148]],["t/563",[77,3.884,339,4.206,358,4.935]],["t/565",[13,2.971,28,2.353,240,3.123,273,2.749,363,3.327,386,3.636,387,3.636]],["t/567",[195,2.749,288,3.123,364,3.327,388,3.636,389,3.636,390,3.636,391,3.636]],["t/569",[0,3.392,200,3.123,305,4.1,361,4.591]],["t/571",[62,4.276]],["t/575",[76,4.862]],["t/577",[248,5.49,392,6.148]],["t/579",[77,2.707,167,3.663,246,3.138,393,3.663,394,4.003,395,3.663]],["t/581",[18,5.49,396,6.719]],["t/583",[393,7.403]],["t/585",[221,5.266,397,6.719]],["t/587",[398,7.403]],["t/589",[246,6.341]],["t/591",[261,5.771,399,6.719]],["t/593",[400,8.091]],["t/595",[82,5.266,401,6.719]],["t/597",[200,4.182,201,4.655]],["t/599",[61,5.036]],["t/601",[62,4.276]],["t/605",[32,2.879,289,3.851]],["t/607",[32,2.461,289,3.292,302,5.256]],["t/609",[129,6.148,402,6.719]],["t/611",[304,5.49,403,5.771]],["t/615",[76,4.862]],["t/617",[35,3.199,40,3.576,404,5.745]],["t/619",[46,4.075,113,3.085,288,3.825,405,4.453,406,4.453]],["t/621",[89,3.884,130,4.502,407,5.745]],["t/623",[113,3.476,130,3.932,268,4.591,408,5.017]],["t/625",[191,5.49,282,5.079]],["t/627",[206,4.655,282,5.079]],["t/629",[282,5.079,409,5.771]],["t/631",[40,2.772,248,3.639,410,4.453,411,4.453,412,4.075]],["t/633",[40,3.576,93,5.256,413,5.745]],["t/635",[200,4.182,201,4.655]],["t/637",[62,4.276]],["t/641",[76,4.862]],["t/643",[414,8.091]],["t/644",[77,2.707,79,3.026,83,3.271,395,3.663,415,3.138,416,4.003]],["t/646",[44,4.935,415,4.502,417,5.745]],["t/648",[44,4.935,392,5.256,418,5.745]],["t/650",[32,1.715,415,3.138,419,3.663,420,4.003,421,4.003,422,3.439]],["t/652",[32,1.558,415,2.85,422,3.123,423,3.636,424,3.636,425,3.327,426,3.636]],["t/654",[236,6.116]],["t/656",[191,5.49,427,6.719]],["t/658",[428,8.091]],["t/660",[234,7.403]],["t/662",[61,5.036]],["t/664",[200,4.182,201,4.655]],["t/666",[62,4.276]],["t/670",[76,4.862]],["t/672",[39,5.079,429,6.719]],["t/674",[162,6.148,221,5.266]],["t/676",[17,4.343,159,3.396,430,5.745]],["t/678",[15,4.779,17,5.079]],["t/680",[114,5.771,431,5.771]],["t/682",[5,2.51,29,2.322,39,2.322,42,2.185,107,2.408,108,2.322,269,2.811,312,2.811,432,3.072]],["t/684",[43,6.612]],["t/686",[433,6.719,434,6.148]],["t/688",[77,3.392,79,3.793,83,4.1,435,5.017]],["t/690",[436,8.091]],["t/692",[437,6.95]],["t/694",[39,4.343,425,5.256,438,5.745]],["t/696",[116,4.343,141,4.935,170,5.256]],["t/698",[193,3.793,409,4.31,439,5.017,440,5.017]],["t/700",[261,3.439,398,3.663,441,4.003,442,4.003,443,4.003,444,4.003]],["t/702",[107,3.138,159,2.366,434,3.663,445,4.003,446,4.003,447,4.003]],["t/704",[61,3.576,200,3.576,448,4.935]],["t/706",[103,6.341]],["t/708",[62,4.276]],["t/712",[105,5.266,431,5.771]],["t/714",[35,3.741,449,6.148]],["t/716",[89,3.884,316,5.256,450,5.745]],["t/720",[186,5.924]],["t/723",[32,2.879,451,6.148]],["t/725",[168,3.343,207,5.256,280,5.256]],["t/727",[59,4.502,60,4.935,133,5.256]],["t/729",[452,8.091]],["t/731",[3,4.543,453,6.148]],["t/733",[62,4.276]],["t/737",[213,4.779,244,5.771]],["t/739",[32,2.15,113,3.476,454,5.017,455,4.31]],["t/741",[253,7.403]],["t/743",[32,2.15,118,3.932,342,4.591,456,5.017]],["t/745",[200,3.123,305,4.1,403,4.31,448,4.31]],["t/747",[103,6.341]],["t/751",[76,4.862]],["t/753",[35,3.741,457,6.719]],["t/755",[35,3.741,458,5.771]],["t/757",[35,3.199,375,4.694,459,4.935]],["t/759",[182,5.49,374,6.148]],["t/761",[460,4.343,461,4.343,462,4.694]],["t/763",[35,2.48,458,3.825,460,3.366,461,3.366,463,4.075]],["t/765",[35,2.229,375,3.271,459,3.439,460,3.026,461,3.026,463,3.663]],["t/767",[199,3.168,460,3.366,461,3.366,462,3.639,464,4.453]],["t/769",[35,2.229,458,3.439,460,3.026,461,3.026,462,3.271,465,4.003]],["t/771",[35,2.229,375,3.271,459,3.439,460,3.026,461,3.026,462,3.271]],["t/773",[61,5.036]],["t/775",[62,4.276]],["t/779",[186,5.924]],["t/781",[32,2.15,119,4.591,163,3.793,289,2.875]],["t/783",[466,7.403]],["t/785",[37,3.717,297,4.935,467,5.745]],["t/786",[34,3.247,164,2.966,468,3.932,469,4.1]],["t/788",[34,3.247,168,2.919,468,3.932,470,5.017]],["t/790",[34,3.247,171,3.123,468,3.932,471,5.017]],["t/792",[34,3.247,175,3.316,468,3.932,472,5.017]],["t/794",[34,3.247,177,3.316,468,3.932,473,5.017]],["t/796",[474,8.091]],["t/798",[37,3.717,297,4.935,475,5.745]],["t/800",[61,5.036]],["t/802",[200,4.182,201,4.655]],["t/804",[62,4.276]],["t/808",[83,3.639,476,3.639,477,5.036,478,3.825]],["t/810",[476,5.49,477,5.079]],["t/812",[479,7.403]],["t/814",[109,6.612]],["t/816",[480,5.745,481,5.745,482,5.256]],["t/818",[476,4.1,477,3.793,483,5.017,484,5.017]],["t/820",[477,5.079,478,5.771]],["t/822",[109,6.612]],["t/824",[479,7.403]],["t/826",[296,4.075,476,3.639,477,5.036,478,3.825]],["t/828",[40,3.576,477,4.343,485,5.745]],["t/830",[62,4.276]],["t/834",[76,4.862]],["t/836",[186,4.206,486,5.745,487,5.745]],["t/838",[32,2.879,289,3.851]],["t/839",[42,5.755]],["t/841",[123,5.47]],["t/843",[35,4.505]],["t/845",[236,6.116]],["t/847",[50,4.935,488,5.745,489,5.745]],["t/849",[57,7.403]],["t/851",[199,4.779,346,5.771]],["t/853",[32,2.15,97,3.932,289,2.875,490,5.017]],["t/855",[132,6.148,491,6.148]],["t/857",[138,4.086,159,3.396,492,5.745]],["t/859",[289,3.851,493,6.719]],["t/861",[109,5.49,494,6.719]],["t/863",[495,6.95]],["t/865",[61,5.036]],["t/869",[76,4.037,496,6.719]],["t/871",[32,2.136,118,2.408,164,1.816,283,2.639,289,1.761,295,2.408,300,2.408,497,3.072]],["t/872",[26,5.256,164,3.396,300,4.502]],["t/874",[32,2.15,118,3.932,168,2.919,295,3.932]],["t/876",[32,2.15,171,3.123,289,2.875,295,3.932]],["t/878",[32,1.908,123,3.011,152,3.49,289,2.552,498,4.453]],["t/880",[32,2.15,175,3.316,289,2.875,499,5.017]],["t/882",[32,2.15,177,3.316,289,2.875,500,4.591]],["t/884",[138,3.569,159,2.966,168,2.919,283,4.31]],["t/886",[138,4.086,159,3.396,238,4.086]],["t/888",[138,3.569,159,2.966,501,4.1,502,5.017]],["t/890",[304,4.1,309,4.591,503,4.1,504,5.017]],["t/892",[89,2.458,243,3.327,505,2.971,506,3.636,507,3.123,508,3.327,509,3.636]],["t/894",[469,5.49,510,6.719]],["t/898",[511,8.091]],["t/900",[213,4.779,244,5.771]],["t/902",[512,8.091]],["t/904",[236,6.116]],["t/906",[50,5.771,513,6.719]],["t/908",[32,2.461,514,5.745,515,5.256]],["t/910",[507,5.771,516,6.719]],["t/912",[517,6.719,518,6.719]],["t/914",[500,7.403]],["t/918",[89,5.47]],["t/920",[519,8.091]],["t/922",[221,6.341]],["t/924",[520,8.091]],["t/926",[164,2.966,453,4.591,521,4.31,522,4.591]],["t/928",[164,3.972,523,6.719]],["t/930",[35,3.199,168,3.343,524,5.745]],["t/932",[171,3.576,233,4.935,525,5.745]],["t/934",[97,3.932,175,3.316,526,5.017,527,5.017]],["t/936",[159,2.966,177,3.316,528,5.017,529,5.017]],["t/938",[32,1.715,53,3.138,238,2.848,295,3.138,522,3.663,530,4.003]],["t/940",[168,2.919,204,4.591,437,4.31,521,4.31]],["t/942",[164,3.972,196,5.49]],["t/944",[97,4.502,168,3.343,531,5.745]],["t/946",[171,4.182,532,6.719]],["t/948",[34,2.591,113,2.773,171,2.492,412,3.663,521,3.439,533,3.439]],["t/950",[33,4.591,34,3.247,164,2.966,534,4.591]],["t/952",[168,3.909,533,5.771]],["t/954",[171,2.772,248,3.639,508,4.075,535,4.453,536,4.075]],["t/956",[34,1.988,136,2.51,175,2.031,264,2.811,422,2.639,537,3.072,538,3.072,539,3.072,540,2.811]],["t/958",[52,5.256,177,3.797,515,5.256]],["t/960",[111,5.771,203,6.148]],["t/964",[34,3.717,541,5.256,542,5.745]],["t/966",[140,3.644,194,4.935,195,4.343]],["t/968",[140,4.262,198,6.148]],["t/970",[186,4.919,543,6.148]],["t/972",[140,3.644,544,5.256,545,5.256]],["t/974",[191,4.1,544,4.591,545,4.591,546,5.017]],["t/976",[37,2.882,140,2.825,214,3.825,221,3.49,547,4.453]],["t/978",[97,4.502,140,3.644,185,4.694]],["t/980",[543,6.148,548,5.49]],["t/982",[35,3.199,193,4.343,197,4.502]],["t/984",[35,2.794,104,4.591,140,3.182,549,5.017]],["t/986",[35,2.794,193,3.793,196,4.1,197,3.932]],["t/988",[534,6.148,550,6.719]],["t/990",[94,6.148,551,6.719]],["t/992",[61,3.576,200,3.576,552,5.745]],["t/994",[62,4.276]],["t/998",[76,4.862]],["t/1000",[197,3.932,287,3.392,553,5.017,554,5.017]],["t/1002",[287,3.392,357,4.31,383,4.591,541,4.591]],["t/1004",[35,3.199,241,5.256,285,4.694]],["t/1006",[159,3.972,555,6.719]],["t/1008",[213,4.086,285,4.694,556,5.745]],["t/1010",[32,2.15,285,4.1,286,3.932,455,4.31]],["t/1012",[286,5.266,557,6.719]],["t/1014",[286,5.266,558,6.719]],["t/1016",[286,4.502,559,5.745,560,5.745]],["t/1018",[59,3.49,369,3.825,455,3.825,561,4.453,562,4.453]],["t/1020",[61,5.036]],["t/1022",[200,4.182,448,5.771]],["t/1024",[62,4.276]],["t/1028",[186,5.924]],["t/1029",[113,3.476,307,4.31,563,5.017,564,5.017]],["t/1031",[121,4.31,201,3.476,307,4.31,565,5.017]],["t/1033",[566,8.091]],["t/1035",[164,3.972,567,6.148]],["t/1037",[168,3.909,567,6.148]],["t/1039",[356,5.771,568,6.719]],["t/1041",[31,4.591,138,3.569,186,3.673,449,4.591]],["t/1043",[533,3.123,569,3.636,570,3.636,571,3.636,572,3.636,573,3.636,574,3.636]],["t/1045",[354,4.502,437,4.935,575,5.745]],["t/1047",[409,4.31,536,4.591,576,5.017,577,5.017]],["t/1049",[159,4.783]],["t/1051",[22,4.1,578,5.017,579,5.017,580,5.017]],["t/1053",[206,3.98,581,5.745,582,5.745]],["t/1055",[84,5.256,491,5.256,583,5.745]],["t/1057",[187,7.403]],["t/1059",[304,5.49,403,5.771]],["t/1061",[103,6.341]],["t/1063",[62,4.276]],["t/1067",[28,2.882,206,3.085,267,3.49,284,3.825,584,3.825]],["t/1069",[206,5.605]],["t/1071",[28,5.236]],["t/1073",[267,6.341]],["t/1075",[267,5.266,585,6.719]],["t/1077",[586,6.148,587,6.148]],["t/1079",[362,6.148,548,5.49]],["t/1081",[73,7.403]],["t/1083",[419,7.403]],["t/1085",[195,3.793,282,3.793,548,4.1,588,5.017]],["t/1087",[111,6.95]],["t/1089",[37,3.247,482,4.591,589,5.017,590,4.591]],["t/1091",[548,5.49,591,6.719]],["t/1093",[293,6.148,592,6.148]],["t/1095",[592,6.148,593,6.719]],["t/1097",[159,3.972,594,6.719]],["t/1099",[595,8.091]],["t/1101",[130,6.341]],["t/1103",[58,3.98,200,3.576,201,3.98]],["t/1105",[62,4.276]],["t/1110",[76,4.862]],["t/1112",[273,6.116]],["t/1114",[273,5.079,596,6.719]],["t/1116",[466,7.403]],["t/1118",[273,4.343,282,4.343,451,5.256]],["t/1119",[164,3.972,597,6.148]],["t/1121",[168,3.909,598,6.148]],["t/1123",[171,4.182,599,6.148]],["t/1125",[175,4.441,600,6.719]],["t/1127",[177,4.441,601,6.148]],["t/1129",[238,4.779,602,6.148]],["t/1131",[501,5.49,603,6.719]],["t/1133",[503,5.49,604,6.148]],["t/1135",[505,5.49,605,6.719]],["t/1137",[469,5.49,606,6.719]],["t/1139",[607,6.148,608,6.719]],["t/1141",[609,6.719,610,6.719]],["t/1143",[611,6.719,612,6.719]],["t/1145",[613,6.719,614,6.719]],["t/1147",[615,6.719,616,6.719]],["t/1149",[617,6.719,618,6.719]],["t/1151",[619,6.719,620,6.719]],["t/1153",[621,6.719,622,6.719]],["t/1155",[623,6.719,624,6.719]],["t/1157",[625,6.719,626,6.719]],["t/1159",[627,6.719,628,6.719]],["t/1161",[629,6.719,630,6.719]],["t/1163",[631,6.719,632,6.719]],["t/1165",[58,4.655,99,5.771]],["t/1167",[61,4.182,633,6.719]],["t/1169",[62,4.276]],["t/1174",[76,4.862]],["t/1176",[634,7.403]],["t/1178",[584,4.935,634,5.256,635,5.745]],["t/1180",[18,5.49,305,5.49]],["t/1182",[223,5.771,590,6.148]],["t/1184",[100,7.403]],["t/1185",[164,3.972,597,6.148]],["t/1187",[168,3.909,599,6.148]],["t/1189",[171,4.182,598,6.148]],["t/1191",[175,4.441,601,6.148]],["t/1193",[177,4.441,604,6.148]],["t/1195",[238,4.779,602,6.148]],["t/1197",[58,4.655,98,5.771]],["t/1198",[164,3.972,636,6.148]],["t/1200",[168,3.909,636,6.148]],["t/1202",[58,5.605]],["t/1204",[62,4.276]],["t/1209",[495,6.95]],["t/1211",[495,6.95]],["t/1215",[637,8.091]],["t/1217",[638,8.091]],["t/1219",[379,5.256,586,5.256,587,5.256]],["t/1221",[206,4.655,380,6.148]],["t/1223",[28,4.348,381,6.148]],["t/1225",[228,5.771,639,6.719]],["t/1227",[141,5.771,384,6.148]],["t/1229",[385,6.148,640,6.719]],["t/1231",[641,6.719,642,6.719]],["t/1233",[643,6.719,644,6.719]],["t/1235",[645,6.719,646,6.719]],["t/1241",[164,4.783]],["t/1243",[96,2.369,108,2.518,168,1.938,199,2.369,367,2.861,647,3.33,648,3.047,649,2.721]],["t/1245",[96,2.848,171,2.492,367,3.439,648,3.663,650,3.439,651,3.439]],["t/1247",[175,2.646,206,2.773,649,3.271,652,4.003,653,4.003,654,4.003]],["t/1249",[96,2.586,177,2.403,655,3.636,656,3.636,657,3.636,658,3.636,659,3.636]],["t/1251",[238,4.086,660,5.745,661,5.256]],["t/1253",[501,4.1,650,4.31,651,4.31,662,4.591]],["t/1255",[28,3.247,503,4.1,540,4.591,663,5.017]],["t/1257",[505,4.694,584,4.935,664,5.745]],["t/1262",[665,8.091]],["t/1264",[507,5.771,666,6.719]],["t/1272",[15,5.755]],["t/1274",[4,5.256,5,4.694,6,5.256]],["t/1276",[16,4.31,17,3.793,18,4.1,19,4.591]],["t/1278",[7,5.256,8,5.256,9,5.256]],["t/1280",[21,4.935,22,4.694,23,5.256]],["t/1282",[20,6.95]],["t/1284",[11,4.075,12,4.075,13,3.639,14,4.075,53,3.49]],["t/1286",[10,7.403]],["t/1288",[206,5.605]],["t/1292",[164,4.783]],["t/1294",[168,3.909,667,6.719]],["t/1296",[2,3.476,3,3.392,171,3.123,668,5.017]],["t/1298",[175,3.316,276,4.591,661,4.591,669,5.017]],["t/1300",[177,2.944,649,3.639,650,3.825,651,3.825,662,4.075]],["t/1302",[108,4.343,238,4.086,670,5.745]],["t/1304",[501,5.49,671,6.719]],["t/1306",[415,5.266,503,5.49]],["t/1308",[69,4.502,505,4.694,672,5.745]],["t/1310",[469,5.49,673,6.719]],["t/1312",[199,3.569,431,4.31,607,4.591,649,4.1]],["t/1316",[0,5.47]]],"invertedIndex":[["",{"_index":53,"t":{"90":{"position":[[19,1]]},"114":{"position":[[7,2]]},"159":{"position":[[4,1]]},"938":{"position":[[11,2]]},"1284":{"position":[[21,1]]}}}],["0",{"_index":496,"t":{"869":{"position":[[0,2]]}}}],["1",{"_index":164,"t":{"278":{"position":[[0,2]]},"308":{"position":[[0,2]]},"342":{"position":[[5,2]]},"513":{"position":[[0,2]]},"786":{"position":[[10,2]]},"871":{"position":[[5,1]]},"872":{"position":[[0,2]]},"926":{"position":[[6,1]]},"928":{"position":[[0,2]]},"942":{"position":[[0,2]]},"950":{"position":[[0,2]]},"1035":{"position":[[13,1]]},"1119":{"position":[[0,2]]},"1185":{"position":[[0,2]]},"1198":{"position":[[6,1]]},"1241":{"position":[[0,2]]},"1292":{"position":[[0,2]]}}}],["10",{"_index":469,"t":{"786":{"position":[[13,2]]},"894":{"position":[[0,3]]},"1137":{"position":[[0,3]]},"1310":{"position":[[0,3]]}}}],["11",{"_index":607,"t":{"1139":{"position":[[0,3]]},"1312":{"position":[[0,3]]}}}],["12",{"_index":609,"t":{"1141":{"position":[[0,3]]}}}],["13",{"_index":611,"t":{"1143":{"position":[[0,3]]}}}],["14",{"_index":613,"t":{"1145":{"position":[[0,3]]}}}],["15",{"_index":615,"t":{"1147":{"position":[[0,3]]}}}],["150",{"_index":473,"t":{"794":{"position":[[13,3]]}}}],["16",{"_index":617,"t":{"1149":{"position":[[0,3]]}}}],["17",{"_index":619,"t":{"1151":{"position":[[0,3]]}}}],["18",{"_index":621,"t":{"1153":{"position":[[0,3]]}}}],["19",{"_index":623,"t":{"1155":{"position":[[0,3]]}}}],["2",{"_index":168,"t":{"280":{"position":[[0,2]]},"310":{"position":[[0,2]]},"350":{"position":[[5,2]]},"527":{"position":[[0,2]]},"725":{"position":[[6,1]]},"788":{"position":[[10,2]]},"874":{"position":[[0,2]]},"884":{"position":[[5,1]]},"930":{"position":[[0,2]]},"940":{"position":[[6,1]]},"944":{"position":[[0,2]]},"952":{"position":[[0,2]]},"1037":{"position":[[13,1]]},"1121":{"position":[[0,2]]},"1187":{"position":[[0,2]]},"1200":{"position":[[6,1]]},"1243":{"position":[[0,2]]},"1294":{"position":[[0,2]]}}}],["2,653",{"_index":394,"t":{"579":{"position":[[37,5]]}}}],["20",{"_index":625,"t":{"1157":{"position":[[0,3]]}}}],["2025",{"_index":25,"t":{"55":{"position":[[13,4]]}}}],["21",{"_index":627,"t":{"1159":{"position":[[0,3]]}}}],["22",{"_index":629,"t":{"1161":{"position":[[0,3]]}}}],["23",{"_index":631,"t":{"1163":{"position":[[0,3]]}}}],["3",{"_index":171,"t":{"282":{"position":[[0,2]]},"312":{"position":[[0,2]]},"358":{"position":[[5,2]]},"533":{"position":[[0,2]]},"790":{"position":[[10,2]]},"876":{"position":[[0,2]]},"932":{"position":[[0,2]]},"946":{"position":[[0,2]]},"948":{"position":[[6,1]]},"954":{"position":[[0,2]]},"1123":{"position":[[0,2]]},"1189":{"position":[[0,2]]},"1245":{"position":[[0,2]]},"1296":{"position":[[0,2]]}}}],["3.5",{"_index":498,"t":{"878":{"position":[[0,3]]}}}],["30",{"_index":470,"t":{"788":{"position":[[13,2]]}}}],["4",{"_index":175,"t":{"284":{"position":[[0,2]]},"314":{"position":[[0,2]]},"360":{"position":[[5,2]]},"792":{"position":[[10,2]]},"880":{"position":[[0,2]]},"934":{"position":[[0,2]]},"956":{"position":[[0,2]]},"1125":{"position":[[0,2]]},"1191":{"position":[[0,2]]},"1247":{"position":[[0,2]]},"1298":{"position":[[0,2]]}}}],["5",{"_index":177,"t":{"286":{"position":[[0,2]]},"322":{"position":[[0,2]]},"362":{"position":[[5,2]]},"794":{"position":[[10,2]]},"882":{"position":[[0,2]]},"936":{"position":[[0,2]]},"958":{"position":[[0,2]]},"1127":{"position":[[0,2]]},"1193":{"position":[[0,2]]},"1249":{"position":[[0,2]]},"1300":{"position":[[0,2]]}}}],["50",{"_index":471,"t":{"790":{"position":[[13,2]]}}}],["6",{"_index":238,"t":{"364":{"position":[[5,2]]},"366":{"position":[[27,1]]},"886":{"position":[[0,2]]},"938":{"position":[[0,2]]},"1129":{"position":[[0,2]]},"1195":{"position":[[0,2]]},"1251":{"position":[[0,2]]},"1302":{"position":[[0,2]]}}}],["7",{"_index":501,"t":{"888":{"position":[[0,2]]},"1131":{"position":[[0,2]]},"1253":{"position":[[0,2]]},"1304":{"position":[[0,2]]}}}],["8",{"_index":503,"t":{"890":{"position":[[0,2]]},"1133":{"position":[[0,2]]},"1255":{"position":[[0,2]]},"1306":{"position":[[0,2]]}}}],["85",{"_index":472,"t":{"792":{"position":[[13,2]]}}}],["9",{"_index":505,"t":{"892":{"position":[[0,2]]},"1135":{"position":[[0,2]]},"1257":{"position":[[0,2]]},"1308":{"position":[[0,2]]}}}],["account",{"_index":533,"t":{"948":{"position":[[10,10]]},"952":{"position":[[3,10]]},"1043":{"position":[[0,7]]}}}],["achiev",{"_index":402,"t":{"609":{"position":[[0,12]]}}}],["acknowledg",{"_index":103,"t":{"171":{"position":[[0,16]]},"198":{"position":[[0,15]]},"706":{"position":[[0,15]]},"747":{"position":[[0,16]]},"1061":{"position":[[0,16]]}}}],["action",{"_index":365,"t":{"529":{"position":[[5,7]]},"531":{"position":[[10,7]]}}}],["acz",{"_index":14,"t":{"27":{"position":[[28,5]]},"1284":{"position":[[26,5]]}}}],["ad",{"_index":347,"t":{"505":{"position":[[10,6]]}}}],["adapt",{"_index":33,"t":{"72":{"position":[[7,8]]},"950":{"position":[[3,8]]}}}],["addit",{"_index":418,"t":{"648":{"position":[[0,10]]}}}],["address",{"_index":377,"t":{"547":{"position":[[22,9]]},"559":{"position":[[14,9]]}}}],["advantag",{"_index":350,"t":{"507":{"position":[[4,10]]}}}],["aleo",{"_index":605,"t":{"1135":{"position":[[3,6]]}}}],["ambient",{"_index":290,"t":{"420":{"position":[[8,7]]}}}],["analysi",{"_index":282,"t":{"415":{"position":[[27,8]]},"625":{"position":[[12,8]]},"627":{"position":[[9,8]]},"629":{"position":[[8,8]]},"1085":{"position":[[20,8]]},"1118":{"position":[[13,8]]}}}],["anonym",{"_index":267,"t":{"401":{"position":[[14,9]]},"415":{"position":[[17,9]]},"1067":{"position":[[45,9]]},"1073":{"position":[[0,9]]},"1075":{"position":[[0,9]]}}}],["app",{"_index":491,"t":{"855":{"position":[[7,3]]},"1055":{"position":[[16,4]]}}}],["appetit",{"_index":519,"t":{"920":{"position":[[0,8]]}}}],["appli",{"_index":11,"t":{"27":{"position":[[0,7]]},"1284":{"position":[[0,7]]}}}],["applic",{"_index":281,"t":{"411":{"position":[[27,12]]}}}],["approach",{"_index":341,"t":{"499":{"position":[[33,8]]}}}],["approv",{"_index":112,"t":{"186":{"position":[[20,9]]}}}],["architectur",{"_index":215,"t":{"342":{"position":[[34,12]]},"344":{"position":[[26,13]]},"360":{"position":[[21,12]]},"368":{"position":[[15,12]]},"503":{"position":[[31,12]]}}}],["architecture'",{"_index":355,"t":{"513":{"position":[[3,14]]}}}],["assur",{"_index":8,"t":{"15":{"position":[[8,9]]},"1278":{"position":[[8,9]]}}}],["attack",{"_index":548,"t":{"980":{"position":[[0,7]]},"1079":{"position":[[0,8]]},"1085":{"position":[[0,6]]},"1091":{"position":[[7,6]]}}}],["avail",{"_index":536,"t":{"954":{"position":[[14,12]]},"1047":{"position":[[10,9]]}}}],["b",{"_index":354,"t":{"511":{"position":[[0,2]]},"517":{"position":[[0,2]]},"537":{"position":[[0,2]]},"547":{"position":[[0,2]]},"1045":{"position":[[28,1]]}}}],["background",{"_index":186,"t":{"306":{"position":[[0,10]]},"720":{"position":[[0,10]]},"779":{"position":[[0,10]]},"836":{"position":[[18,10]]},"970":{"position":[[4,10]]},"1028":{"position":[[0,10]]},"1041":{"position":[[0,11]]}}}],["bandwidth",{"_index":154,"t":{"261":{"position":[[7,9]]},"392":{"position":[[23,9]]},"426":{"position":[[20,9]]}}}],["bare",{"_index":410,"t":{"631":{"position":[[4,4]]}}}],["base",{"_index":195,"t":{"316":{"position":[[4,5]]},"360":{"position":[[15,5]]},"449":{"position":[[4,5]]},"567":{"position":[[7,5]]},"966":{"position":[[4,5]]},"1085":{"position":[[7,5]]}}}],["baselin",{"_index":520,"t":{"924":{"position":[[0,8]]}}}],["basic",{"_index":105,"t":{"180":{"position":[[0,5]]},"182":{"position":[[0,5]]},"388":{"position":[[0,5]]},"392":{"position":[[0,5]]},"712":{"position":[[0,5]]}}}],["begin",{"_index":181,"t":{"294":{"position":[[4,9]]}}}],["behavior",{"_index":527,"t":{"934":{"position":[[18,8]]}}}],["behind",{"_index":71,"t":{"115":{"position":[[20,6]]},"344":{"position":[[10,6]]}}}],["bench",{"_index":396,"t":{"581":{"position":[[5,5]]}}}],["benefit",{"_index":179,"t":{"286":{"position":[[14,8]]}}}],["better",{"_index":204,"t":{"328":{"position":[[0,6]]},"940":{"position":[[10,6]]}}}],["between",{"_index":456,"t":{"743":{"position":[[11,7]]}}}],["bloom",{"_index":476,"t":{"808":{"position":[[16,5]]},"810":{"position":[[0,5]]},"818":{"position":[[15,5]]},"826":{"position":[[0,5]]}}}],["break",{"_index":506,"t":{"892":{"position":[[3,8]]}}}],["breakdown",{"_index":499,"t":{"880":{"position":[[13,9]]}}}],["bridg",{"_index":530,"t":{"938":{"position":[[21,6]]}}}],["brief",{"_index":486,"t":{"836":{"position":[[0,5]]}}}],["briefli",{"_index":110,"t":{"186":{"position":[[0,7]]},"217":{"position":[[0,7]]}}}],["browser",{"_index":302,"t":{"430":{"position":[[35,7]]},"607":{"position":[[15,7]]}}}],["build",{"_index":27,"t":{"64":{"position":[[4,5]]},"413":{"position":[[0,8]]}}}],["busi",{"_index":582,"t":{"1053":{"position":[[23,8]]}}}],["c",{"_index":358,"t":{"519":{"position":[[0,2]]},"541":{"position":[[0,2]]},"563":{"position":[[0,2]]}}}],["cach",{"_index":529,"t":{"936":{"position":[[22,7]]}}}],["cairo",{"_index":614,"t":{"1145":{"position":[[4,7]]}}}],["calcul",{"_index":256,"t":{"392":{"position":[[6,12]]}}}],["candid",{"_index":634,"t":{"1176":{"position":[[10,11]]},"1178":{"position":[[31,10]]}}}],["capabl",{"_index":534,"t":{"950":{"position":[[22,12]]},"988":{"position":[[0,10]]}}}],["capac",{"_index":210,"t":{"332":{"position":[[15,8]]}}}],["case",{"_index":346,"t":{"505":{"position":[[4,5]]},"545":{"position":[[4,5]]},"851":{"position":[[4,5]]}}}],["caveat",{"_index":122,"t":{"206":{"position":[[0,7]]}}}],["cellular",{"_index":95,"t":{"161":{"position":[[7,8]]}}}],["ceno",{"_index":622,"t":{"1153":{"position":[[4,6]]}}}],["censorship",{"_index":586,"t":{"1077":{"position":[[0,10]]},"1219":{"position":[[4,10]]}}}],["central",{"_index":430,"t":{"676":{"position":[[0,11]]}}}],["centric",{"_index":340,"t":{"499":{"position":[[25,7]]}}}],["chain",{"_index":353,"t":{"507":{"position":[[40,5]]}}}],["challeng",{"_index":220,"t":{"348":{"position":[[0,10]]}}}],["chang",{"_index":663,"t":{"1255":{"position":[[29,6]]}}}],["chat",{"_index":581,"t":{"1053":{"position":[[7,4]]}}}],["chaum",{"_index":459,"t":{"757":{"position":[[0,5]]},"765":{"position":[[0,5]]},"771":{"position":[[0,5]]}}}],["check",{"_index":325,"t":{"470":{"position":[[27,5]]}}}],["choic",{"_index":660,"t":{"1251":{"position":[[8,7]]}}}],["circuit",{"_index":44,"t":{"82":{"position":[[6,7]]},"646":{"position":[[16,7]]},"648":{"position":[[11,7]]}}}],["client",{"_index":49,"t":{"88":{"position":[[16,6]]},"399":{"position":[[30,7]]}}}],["collect",{"_index":647,"t":{"1243":{"position":[[16,10]]}}}],["combin",{"_index":485,"t":{"828":{"position":[[0,9]]}}}],["come",{"_index":516,"t":{"910":{"position":[[0,6]]}}}],["comment",{"_index":427,"t":{"656":{"position":[[0,8]]}}}],["commit",{"_index":376,"t":{"545":{"position":[[58,10]]}}}],["commun",{"_index":133,"t":{"223":{"position":[[26,11]]},"727":{"position":[[19,13]]}}}],["comparison",{"_index":297,"t":{"426":{"position":[[30,10]]},"785":{"position":[[13,10]]},"798":{"position":[[16,10]]}}}],["complex",{"_index":479,"t":{"812":{"position":[[0,10]]},"824":{"position":[[0,10]]}}}],["compon",{"_index":356,"t":{"513":{"position":[[18,10]]},"535":{"position":[[3,10]]},"1039":{"position":[[10,10]]}}}],["comput",{"_index":578,"t":{"1051":{"position":[[0,7]]}}}],["concept",{"_index":401,"t":{"595":{"position":[[9,7]]}}}],["concern",{"_index":225,"t":{"352":{"position":[[13,8]]}}}],["conclus",{"_index":61,"t":{"102":{"position":[[0,10]]},"121":{"position":[[0,10]]},"476":{"position":[[0,10]]},"509":{"position":[[0,10]]},"561":{"position":[[5,10]]},"599":{"position":[[0,10]]},"662":{"position":[[0,10]]},"704":{"position":[[0,10]]},"773":{"position":[[0,10]]},"800":{"position":[[0,11]]},"865":{"position":[[0,10]]},"992":{"position":[[0,10]]},"1020":{"position":[[0,11]]},"1167":{"position":[[13,11]]}}}],["consid",{"_index":150,"t":{"259":{"position":[[7,11]]}}}],["consider",{"_index":231,"t":{"354":{"position":[[31,15]]}}}],["constraint",{"_index":392,"t":{"577":{"position":[[0,11]]},"648":{"position":[[19,11]]}}}],["construct",{"_index":80,"t":{"135":{"position":[[0,12]]},"466":{"position":[[30,9]]}}}],["consumpt",{"_index":371,"t":{"537":{"position":[[38,11]]},"539":{"position":[[16,11]]}}}],["cont",{"_index":502,"t":{"888":{"position":[[29,6]]}}}],["contact",{"_index":664,"t":{"1257":{"position":[[3,7]]}}}],["context",{"_index":326,"t":{"470":{"position":[[40,7]]}}}],["continu",{"_index":644,"t":{"1233":{"position":[[4,11]]}}}],["contract",{"_index":22,"t":{"49":{"position":[[6,9]]},"525":{"position":[[9,9]]},"1051":{"position":[[20,8]]},"1280":{"position":[[6,9]]}}}],["contribut",{"_index":495,"t":{"863":{"position":[[0,10]]},"1209":{"position":[[7,10]]},"1211":{"position":[[8,10]]}}}],["control",{"_index":593,"t":{"1095":{"position":[[0,10]]}}}],["convers",{"_index":205,"t":{"328":{"position":[[7,14]]}}}],["core",{"_index":525,"t":{"932":{"position":[[3,4]]}}}],["correl",{"_index":595,"t":{"1099":{"position":[[0,11]]}}}],["coscup",{"_index":299,"t":{"428":{"position":[[9,7]]}}}],["cosmo",{"_index":139,"t":{"231":{"position":[[26,7]]}}}],["cost",{"_index":261,"t":{"397":{"position":[[31,4]]},"591":{"position":[[4,5]]},"700":{"position":[[0,4]]}}}],["counterpoint",{"_index":567,"t":{"1035":{"position":[[0,12]]},"1037":{"position":[[0,12]]}}}],["cpu",{"_index":91,"t":{"159":{"position":[[0,3]]}}}],["crate",{"_index":335,"t":{"474":{"position":[[18,6]]}}}],["creat",{"_index":212,"t":{"342":{"position":[[8,6]]},"545":{"position":[[42,6]]}}}],["creation",{"_index":224,"t":{"350":{"position":[[24,8]]},"358":{"position":[[15,8]]}}}],["credenti",{"_index":55,"t":{"90":{"position":[[29,11]]},"229":{"position":[[16,11]]}}}],["cross",{"_index":48,"t":{"88":{"position":[[10,5]]}}}],["cryptograph",{"_index":372,"t":{"541":{"position":[[3,13]]}}}],["cryptographi",{"_index":12,"t":{"27":{"position":[[8,12]]},"1284":{"position":[[8,12]]}}}],["cuckoo",{"_index":478,"t":{"808":{"position":[[34,6]]},"820":{"position":[[0,6]]},"826":{"position":[[17,6]]}}}],["current",{"_index":244,"t":{"370":{"position":[[0,7]]},"737":{"position":[[0,7]]},"900":{"position":[[0,7]]}}}],["d",{"_index":361,"t":{"521":{"position":[[0,2]]},"569":{"position":[[0,2]]}}}],["dapp",{"_index":349,"t":{"505":{"position":[[37,5]]}}}],["dappconnect",{"_index":492,"t":{"857":{"position":[[0,12]]}}}],["data",{"_index":96,"t":{"161":{"position":[[16,4]]},"380":{"position":[[18,5]]},"455":{"position":[[4,4]]},"460":{"position":[[4,4]]},"489":{"position":[[0,4]]},"1243":{"position":[[54,4]]},"1245":{"position":[[38,4]]},"1249":{"position":[[13,4]]}}}],["deal",{"_index":38,"t":{"78":{"position":[[0,7]]}}}],["decentr",{"_index":280,"t":{"411":{"position":[[13,13]]},"725":{"position":[[8,16]]}}}],["decentralis",{"_index":640,"t":{"1229":{"position":[[5,16]]}}}],["decoupl",{"_index":351,"t":{"507":{"position":[[15,10]]}}}],["defi",{"_index":239,"t":{"364":{"position":[[23,4]]}}}],["defin",{"_index":320,"t":{"466":{"position":[[19,6]]}}}],["definit",{"_index":284,"t":{"415":{"position":[[44,11]]},"484":{"position":[[0,11]]},"1067":{"position":[[9,12]]}}}],["delet",{"_index":444,"t":{"700":{"position":[[43,8]]}}}],["deliv",{"_index":504,"t":{"890":{"position":[[20,7]]}}}],["depend",{"_index":72,"t":{"117":{"position":[[8,7]]}}}],["descript",{"_index":161,"t":{"275":{"position":[[8,11]]}}}],["design",{"_index":152,"t":{"259":{"position":[[44,6]]},"362":{"position":[[29,6]]},"403":{"position":[[45,6]]},"511":{"position":[[3,6]]},"878":{"position":[[14,6]]}}}],["detail",{"_index":115,"t":{"188":{"position":[[10,7]]}}}],["detect",{"_index":425,"t":{"652":{"position":[[34,9]]},"694":{"position":[[5,9]]}}}],["devic",{"_index":59,"t":{"100":{"position":[[0,6]]},"390":{"position":[[15,7]]},"407":{"position":[[0,6]]},"727":{"position":[[0,6]]},"1018":{"position":[[29,6]]}}}],["dht",{"_index":543,"t":{"970":{"position":[[0,3]]},"980":{"position":[[11,4]]}}}],["differ",{"_index":342,"t":{"501":{"position":[[9,7]]},"743":{"position":[[0,10]]}}}],["diffi",{"_index":553,"t":{"1000":{"position":[[4,6]]}}}],["disclaim",{"_index":667,"t":{"1294":{"position":[[3,11]]}}}],["discoveri",{"_index":140,"t":{"233":{"position":[[16,9]]},"314":{"position":[[8,9]]},"316":{"position":[[10,9]]},"320":{"position":[[10,9]]},"334":{"position":[[13,9]]},"420":{"position":[[21,9]]},"449":{"position":[[10,9]]},"966":{"position":[[10,9]]},"968":{"position":[[0,9]]},"972":{"position":[[12,9]]},"976":{"position":[[26,9]]},"978":{"position":[[13,9]]},"984":{"position":[[29,9]]}}}],["discuss",{"_index":474,"t":{"796":{"position":[[0,10]]}}}],["discv4",{"_index":184,"t":{"300":{"position":[[0,6]]}}}],["discv5",{"_index":185,"t":{"302":{"position":[[0,6]]},"441":{"position":[[19,6]]},"445":{"position":[[17,6]]},"978":{"position":[[0,6]]}}}],["disput",{"_index":673,"t":{"1310":{"position":[[4,8]]}}}],["dissemin",{"_index":271,"t":{"403":{"position":[[64,13]]}}}],["distribut",{"_index":16,"t":{"39":{"position":[[0,11]]},"296":{"position":[[0,11]]},"1276":{"position":[[0,11]]}}}],["dn",{"_index":194,"t":{"316":{"position":[[0,3]]},"449":{"position":[[0,3]]},"966":{"position":[[0,3]]}}}],["do",{"_index":130,"t":{"223":{"position":[[0,3]]},"401":{"position":[[24,3]]},"621":{"position":[[20,3]]},"623":{"position":[[0,3]]},"1101":{"position":[[0,3]]}}}],["dogfood",{"_index":489,"t":{"847":{"position":[[20,10]]}}}],["doubl",{"_index":423,"t":{"652":{"position":[[17,6]]}}}],["dst",{"_index":19,"t":{"39":{"position":[[28,5]]},"1276":{"position":[[28,5]]}}}],["dual",{"_index":217,"t":{"344":{"position":[[21,4]]}}}],["e",{"_index":363,"t":{"523":{"position":[[0,2]]},"565":{"position":[[0,2]]}}}],["econom",{"_index":5,"t":{"9":{"position":[[6,9]]},"432":{"position":[[23,8]]},"682":{"position":[[10,8]]},"1274":{"position":[[6,9]]}}}],["effect",{"_index":441,"t":{"700":{"position":[[5,9]]}}}],["effici",{"_index":262,"t":{"397":{"position":[[36,9]]}}}],["elimin",{"_index":172,"t":{"282":{"position":[[3,11]]}}}],["emerg",{"_index":229,"t":{"354":{"position":[[0,8]]}}}],["end",{"_index":564,"t":{"1029":{"position":[[28,3]]}}}],["ephemer",{"_index":383,"t":{"557":{"position":[[3,9]]},"1002":{"position":[[0,9]]}}}],["epoch",{"_index":420,"t":{"650":{"position":[[26,5]]}}}],["estim",{"_index":546,"t":{"974":{"position":[[24,10]]}}}],["etc",{"_index":422,"t":{"650":{"position":[[50,5]]},"652":{"position":[[58,5]]},"956":{"position":[[63,4]]}}}],["ethereum",{"_index":138,"t":{"231":{"position":[[14,8]]},"428":{"position":[[34,8]]},"434":{"position":[[24,8]]},"857":{"position":[[13,8]]},"884":{"position":[[9,8]]},"886":{"position":[[3,8]]},"888":{"position":[[3,8]]},"1041":{"position":[[12,8]]}}}],["ethic",{"_index":68,"t":{"114":{"position":[[0,6]]},"117":{"position":[[19,6]]},"424":{"position":[[16,6]]}}}],["european",{"_index":657,"t":{"1249":{"position":[[30,8]]}}}],["event",{"_index":400,"t":{"593":{"position":[[0,6]]}}}],["evolut",{"_index":270,"t":{"403":{"position":[[24,9]]}}}],["evolv",{"_index":142,"t":{"237":{"position":[[4,8]]}}}],["exampl",{"_index":109,"t":{"184":{"position":[[14,7]]},"814":{"position":[[0,7]]},"822":{"position":[[0,7]]},"861":{"position":[[5,8]]}}}],["exceed",{"_index":445,"t":{"702":{"position":[[0,9]]}}}],["exchang",{"_index":197,"t":{"318":{"position":[[15,8]]},"418":{"position":[[24,8]]},"982":{"position":[[5,8]]},"986":{"position":[[15,8]]},"1000":{"position":[[23,8]]}}}],["execut",{"_index":344,"t":{"501":{"position":[[35,9]]},"521":{"position":[[3,9]]},"527":{"position":[[11,9]]},"533":{"position":[[3,9]]}}}],["exist",{"_index":348,"t":{"505":{"position":[[28,8]]}}}],["experi",{"_index":143,"t":{"245":{"position":[[0,11]]}}}],["exploit",{"_index":75,"t":{"119":{"position":[[31,11]]}}}],["export",{"_index":655,"t":{"1249":{"position":[[3,9]]}}}],["extens",{"_index":84,"t":{"141":{"position":[[0,10]]},"1055":{"position":[[0,10]]}}}],["extern",{"_index":419,"t":{"650":{"position":[[17,8]]},"1083":{"position":[[0,8]]}}}],["f",{"_index":364,"t":{"525":{"position":[[0,2]]},"567":{"position":[[0,2]]}}}],["factor",{"_index":532,"t":{"946":{"position":[[9,7]]}}}],["fals",{"_index":481,"t":{"816":{"position":[[15,5]]}}}],["far",{"_index":129,"t":{"219":{"position":[[12,3]]},"609":{"position":[[16,3]]}}}],["feasbl",{"_index":90,"t":{"157":{"position":[[0,10]]}}}],["feasibl",{"_index":116,"t":{"190":{"position":[[0,11]]},"192":{"position":[[10,11]]},"194":{"position":[[7,11]]},"441":{"position":[[0,11]]},"453":{"position":[[0,11]]},"696":{"position":[[0,11]]}}}],["fiat",{"_index":460,"t":{"761":{"position":[[4,4]]},"763":{"position":[[33,4]]},"765":{"position":[[40,4]]},"767":{"position":[[20,4]]},"769":{"position":[[31,4]]},"771":{"position":[[33,4]]}}}],["filter",{"_index":477,"t":{"808":{"position":[[22,7],[41,7]]},"810":{"position":[[6,7]]},"818":{"position":[[21,6]]},"820":{"position":[[7,7]]},"826":{"position":[[6,7],[24,7]]},"828":{"position":[[10,7]]}}}],["find",{"_index":99,"t":{"165":{"position":[[0,7]]},"247":{"position":[[0,8]]},"1165":{"position":[[11,8]]}}}],["fix",{"_index":310,"t":{"451":{"position":[[0,6]]}}}],["flexibl",{"_index":260,"t":{"397":{"position":[[18,8]]}}}],["floodsub",{"_index":523,"t":{"928":{"position":[[3,8]]}}}],["flow",{"_index":43,"t":{"80":{"position":[[19,4]]},"182":{"position":[[6,4]]},"488":{"position":[[0,4]]},"684":{"position":[[0,4]]}}}],["format",{"_index":338,"t":{"489":{"position":[[5,6]]}}}],["forward",{"_index":3,"t":{"7":{"position":[[8,7]]},"13":{"position":[[8,7]]},"19":{"position":[[8,7]]},"25":{"position":[[8,7]]},"31":{"position":[[8,7]]},"37":{"position":[[8,7]]},"43":{"position":[[8,7]]},"53":{"position":[[8,7]]},"731":{"position":[[7,7]]},"1296":{"position":[[3,7]]}}}],["framework",{"_index":241,"t":{"364":{"position":[[64,9]]},"1004":{"position":[[19,9]]}}}],["friendli",{"_index":387,"t":{"565":{"position":[[13,8]]}}}],["full",{"_index":537,"t":{"956":{"position":[[25,4]]}}}],["function",{"_index":374,"t":{"543":{"position":[[8,9]]},"759":{"position":[[5,9]]}}}],["further",{"_index":549,"t":{"984":{"position":[[0,7]]}}}],["futur",{"_index":200,"t":{"324":{"position":[[0,6]]},"394":{"position":[[0,6]]},"409":{"position":[[4,6]]},"493":{"position":[[0,6]]},"569":{"position":[[3,6]]},"597":{"position":[[0,6]]},"635":{"position":[[0,6]]},"664":{"position":[[0,6]]},"704":{"position":[[15,6]]},"745":{"position":[[15,6]]},"802":{"position":[[0,6]]},"992":{"position":[[15,6]]},"1022":{"position":[[0,6]]},"1103":{"position":[[12,6]]}}}],["ga",{"_index":399,"t":{"591":{"position":[[0,3]]}}}],["gener",{"_index":101,"t":{"169":{"position":[[0,7]]},"196":{"position":[[0,7]]},"527":{"position":[[3,7]]},"537":{"position":[[24,10]]}}}],["get",{"_index":331,"t":{"472":{"position":[[18,7]]}}}],["go",{"_index":278,"t":{"407":{"position":[[30,2]]}}}],["goal",{"_index":123,"t":{"208":{"position":[[0,5]]},"342":{"position":[[0,4]]},"350":{"position":[[0,4]]},"358":{"position":[[0,4]]},"360":{"position":[[0,4]]},"362":{"position":[[0,4]]},"364":{"position":[[0,4]]},"366":{"position":[[22,4]]},"841":{"position":[[0,5]]},"878":{"position":[[21,5]]}}}],["goal1",{"_index":145,"t":{"257":{"position":[[0,6]]}}}],["goal2",{"_index":149,"t":{"259":{"position":[[0,6]]}}}],["goal3",{"_index":153,"t":{"261":{"position":[[0,6]]}}}],["goal4",{"_index":156,"t":{"263":{"position":[[0,6]]}}}],["goal5",{"_index":160,"t":{"265":{"position":[[0,6]]}}}],["gossipsub",{"_index":196,"t":{"318":{"position":[[0,9]]},"403":{"position":[[0,9]]},"942":{"position":[[3,9]]},"986":{"position":[[0,9]]}}}],["govern",{"_index":672,"t":{"1308":{"position":[[3,9]]}}}],["guarante",{"_index":208,"t":{"328":{"position":[[37,10]]}}}],["guid",{"_index":413,"t":{"633":{"position":[[10,5]]}}}],["handl",{"_index":157,"t":{"263":{"position":[[7,8]]}}}],["handshak",{"_index":286,"t":{"418":{"position":[[6,10]]},"1010":{"position":[[16,10]]},"1012":{"position":[[9,9]]},"1014":{"position":[[8,9]]},"1016":{"position":[[18,10]]}}}],["harmon",{"_index":218,"t":{"346":{"position":[[0,11]]}}}],["hash",{"_index":182,"t":{"296":{"position":[[12,4]]},"543":{"position":[[3,4]]},"545":{"position":[[34,4]]},"759":{"position":[[0,4]]}}}],["head",{"_index":24,"t":{"55":{"position":[[0,7]]}}}],["hellman",{"_index":554,"t":{"1000":{"position":[[11,7]]}}}],["heterogen",{"_index":134,"t":{"225":{"position":[[2,13]]},"259":{"position":[[19,13]]},"409":{"position":[[58,13]]}}}],["heurist",{"_index":462,"t":{"761":{"position":[[16,9]]},"767":{"position":[[32,9]]},"769":{"position":[[43,9]]},"771":{"position":[[45,9]]}}}],["hide",{"_index":275,"t":{"405":{"position":[[30,6]]}}}],["high",{"_index":508,"t":{"892":{"position":[[33,4]]},"954":{"position":[[9,4]]}}}],["highlight",{"_index":1,"t":{"5":{"position":[[0,10]]},"11":{"position":[[0,10]]},"17":{"position":[[0,10]]},"23":{"position":[[0,10]]},"29":{"position":[[0,10]]},"35":{"position":[[0,10]]},"41":{"position":[[0,10]]},"47":{"position":[[0,10]]},"51":{"position":[[0,10]]}}}],["hire",{"_index":510,"t":{"894":{"position":[[11,7]]}}}],["histor",{"_index":528,"t":{"936":{"position":[[3,10]]}}}],["histori",{"_index":487,"t":{"836":{"position":[[6,7]]}}}],["host",{"_index":352,"t":{"507":{"position":[[35,4]]}}}],["huilong",{"_index":488,"t":{"847":{"position":[[8,7]]}}}],["hybrid",{"_index":343,"t":{"501":{"position":[[28,6]]}}}],["ident",{"_index":571,"t":{"1043":{"position":[[25,8]]}}}],["ii",{"_index":379,"t":{"551":{"position":[[0,3]]},"1219":{"position":[[0,3]]}}}],["iii",{"_index":380,"t":{"553":{"position":[[0,4]]},"1221":{"position":[[0,4]]}}}],["implement",{"_index":236,"t":{"360":{"position":[[34,14]]},"397":{"position":[[46,14]]},"474":{"position":[[25,14]]},"654":{"position":[[4,14]]},"845":{"position":[[0,15]]},"904":{"position":[[0,14]]}}}],["improp",{"_index":464,"t":{"767":{"position":[[0,8]]}}}],["improv",{"_index":163,"t":{"277":{"position":[[9,12]]},"310":{"position":[[3,12]]},"312":{"position":[[3,12]]},"332":{"position":[[0,8]]},"403":{"position":[[10,13]]},"781":{"position":[[12,12]]}}}],["incent",{"_index":432,"t":{"682":{"position":[[19,9]]}}}],["incentiv",{"_index":136,"t":{"227":{"position":[[14,15]]},"330":{"position":[[9,15]]},"409":{"position":[[37,16]]},"956":{"position":[[3,11]]}}}],["inclus",{"_index":642,"t":{"1231":{"position":[[6,11]]}}}],["indemn",{"_index":671,"t":{"1304":{"position":[[3,9]]}}}],["industri",{"_index":65,"t":{"110":{"position":[[16,8]]},"424":{"position":[[48,8]]}}}],["inform",{"_index":584,"t":{"1067":{"position":[[0,8]]},"1178":{"position":[[12,11]]},"1257":{"position":[[11,11]]}}}],["infrastructur",{"_index":30,"t":{"64":{"position":[[29,15]]},"66":{"position":[[5,14]]},"68":{"position":[[26,14]]},"411":{"position":[[44,15]]},"413":{"position":[[28,14]]}}}],["input",{"_index":417,"t":{"646":{"position":[[24,6]]}}}],["insert",{"_index":398,"t":{"587":{"position":[[0,9]]},"700":{"position":[[29,9]]}}}],["insight",{"_index":633,"t":{"1167":{"position":[[0,8]]}}}],["integr",{"_index":233,"t":{"356":{"position":[[15,12]]},"364":{"position":[[8,11]]},"932":{"position":[[8,11]]}}}],["intellectu",{"_index":669,"t":{"1298":{"position":[[3,12]]}}}],["interact",{"_index":36,"t":{"74":{"position":[[16,12]]},"362":{"position":[[17,11]]},"491":{"position":[[0,11]]}}}],["interest",{"_index":526,"t":{"934":{"position":[[9,8]]}}}],["intern",{"_index":73,"t":{"119":{"position":[[0,13]]},"1081":{"position":[[0,8]]}}}],["interoper",{"_index":192,"t":{"312":{"position":[[19,16]]}}}],["interpol",{"_index":406,"t":{"619":{"position":[[22,13]]}}}],["intregr",{"_index":81,"t":{"137":{"position":[[12,10]]}}}],["intro",{"_index":26,"t":{"60":{"position":[[0,5]]},"872":{"position":[[7,5]]}}}],["introduc",{"_index":125,"t":{"214":{"position":[[0,11]]},"422":{"position":[[0,11]]},"499":{"position":[[0,11]]}}}],["introduct",{"_index":76,"t":{"129":{"position":[[0,12]]},"202":{"position":[[0,12]]},"255":{"position":[[0,12]]},"340":{"position":[[0,12]]},"464":{"position":[[0,12]]},"497":{"position":[[3,12]]},"575":{"position":[[0,12]]},"615":{"position":[[0,12]]},"641":{"position":[[0,12]]},"670":{"position":[[0,12]]},"751":{"position":[[0,12]]},"834":{"position":[[0,12]]},"869":{"position":[[3,12]]},"998":{"position":[[0,12]]},"1110":{"position":[[0,12]]},"1174":{"position":[[0,12]]}}}],["involv",{"_index":452,"t":{"729":{"position":[[4,8]]}}}],["issu",{"_index":170,"t":{"280":{"position":[[24,6]]},"696":{"position":[[21,6]]}}}],["iter",{"_index":468,"t":{"786":{"position":[[0,9]]},"788":{"position":[[0,9]]},"790":{"position":[[0,9]]},"792":{"position":[[0,9]]},"794":{"position":[[0,9]]}}}],["iv",{"_index":381,"t":{"555":{"position":[[0,3]]},"1223":{"position":[[0,3]]}}}],["iwant",{"_index":180,"t":{"286":{"position":[[28,5]]}}}],["ix",{"_index":643,"t":{"1233":{"position":[[0,3]]}}}],["jolt",{"_index":603,"t":{"1131":{"position":[[3,6]]}}}],["journey",{"_index":497,"t":{"871":{"position":[[21,7]]}}}],["js",{"_index":277,"t":{"407":{"position":[[18,2]]},"430":{"position":[[11,2]]}}}],["k1k1",{"_index":557,"t":{"1012":{"position":[[4,4]]}}}],["kademlia",{"_index":183,"t":{"298":{"position":[[0,8]]},"445":{"position":[[5,8]]}}}],["kernel",{"_index":235,"t":{"360":{"position":[[8,6]]}}}],["key",{"_index":287,"t":{"418":{"position":[[20,3]]},"507":{"position":[[0,3]]},"547":{"position":[[3,3]]},"549":{"position":[[12,3]]},"551":{"position":[[12,4]]},"553":{"position":[[12,4]]},"555":{"position":[[12,3]]},"557":{"position":[[13,3]]},"1000":{"position":[[19,3]]},"1002":{"position":[[28,4]]}}}],["knowledg",{"_index":360,"t":{"519":{"position":[[14,9]]}}}],["known",{"_index":188,"t":{"308":{"position":[[19,5]]}}}],["larg",{"_index":158,"t":{"263":{"position":[[16,5]]},"278":{"position":[[32,5]]}}}],["latenc",{"_index":147,"t":{"257":{"position":[[11,7]]}}}],["later",{"_index":577,"t":{"1047":{"position":[[39,5]]}}}],["law",{"_index":69,"t":{"114":{"position":[[14,3]]},"115":{"position":[[4,3]]},"117":{"position":[[4,3]]},"119":{"position":[[14,3]]},"1308":{"position":[[13,3]]}}}],["layer",{"_index":207,"t":{"328":{"position":[[31,5]]},"725":{"position":[[0,5]]}}}],["leav",{"_index":395,"t":{"579":{"position":[[43,6]]},"644":{"position":[[16,6]]}}}],["level",{"_index":509,"t":{"892":{"position":[[38,5]]}}}],["leverag",{"_index":274,"t":{"405":{"position":[[19,10]]}}}],["liabil",{"_index":670,"t":{"1302":{"position":[[17,9]]}}}],["liberti",{"_index":638,"t":{"1217":{"position":[[3,7]]}}}],["libp2p",{"_index":522,"t":{"926":{"position":[[18,6]]},"938":{"position":[[14,6]]}}}],["librari",{"_index":334,"t":{"474":{"position":[[10,7]]}}}],["lifecycl",{"_index":370,"t":{"537":{"position":[[8,10]]}}}],["light",{"_index":264,"t":{"399":{"position":[[24,5]]},"956":{"position":[[15,5]]}}}],["limit",{"_index":108,"t":{"184":{"position":[[5,8]]},"401":{"position":[[49,8]]},"453":{"position":[[34,8]]},"682":{"position":[[58,8]]},"1243":{"position":[[6,5]]},"1302":{"position":[[3,10]]}}}],["link",{"_index":662,"t":{"1253":{"position":[[15,5]]},"1300":{"position":[[23,5]]}}}],["list",{"_index":542,"t":{"964":{"position":[[12,5]]}}}],["log",{"_index":315,"t":{"455":{"position":[[28,3]]},"482":{"position":[[7,3]]}}}],["look",{"_index":2,"t":{"7":{"position":[[0,7]]},"13":{"position":[[0,7]]},"19":{"position":[[0,7]]},"25":{"position":[[0,7]]},"31":{"position":[[0,7]]},"37":{"position":[[0,7]]},"43":{"position":[[0,7]]},"53":{"position":[[0,7]]},"1296":{"position":[[11,7]]}}}],["low",{"_index":146,"t":{"257":{"position":[[7,3]]}}}],["lurk",{"_index":618,"t":{"1149":{"position":[[4,6]]}}}],["machin",{"_index":223,"t":{"350":{"position":[[16,7]]},"519":{"position":[[32,8]]},"1182":{"position":[[0,7]]}}}],["maintain",{"_index":435,"t":{"688":{"position":[[0,11]]}}}],["malici",{"_index":405,"t":{"619":{"position":[[0,9]]}}}],["manag",{"_index":369,"t":{"533":{"position":[[32,10]]},"547":{"position":[[7,10]]},"1018":{"position":[[8,10]]}}}],["mani",{"_index":454,"t":{"739":{"position":[[4,4]]}}}],["matrix",{"_index":319,"t":{"466":{"position":[[4,7]]},"470":{"position":[[11,6]]},"472":{"position":[[37,6]]}}}],["matter",{"_index":596,"t":{"1114":{"position":[[10,6]]}}}],["maxim",{"_index":178,"t":{"286":{"position":[[3,10]]}}}],["md",{"_index":318,"t":{"466":{"position":[[0,3]]},"470":{"position":[[7,3]]}}}],["mdsecheck",{"_index":329,"t":{"472":{"position":[[0,9]]},"474":{"position":[[0,9]]}}}],["mean",{"_index":429,"t":{"672":{"position":[[11,4]]}}}],["measur",{"_index":652,"t":{"1247":{"position":[[12,8]]}}}],["mechan",{"_index":288,"t":{"418":{"position":[[33,9]]},"567":{"position":[[29,9]]},"619":{"position":[[36,9]]}}}],["member",{"_index":443,"t":{"700":{"position":[[22,6]]}}}],["membership",{"_index":83,"t":{"139":{"position":[[9,10]]},"644":{"position":[[34,10]]},"688":{"position":[[16,10]]},"808":{"position":[[0,10]]}}}],["memori",{"_index":92,"t":{"159":{"position":[[6,6]]}}}],["merkl",{"_index":79,"t":{"133":{"position":[[0,6]]},"137":{"position":[[0,6]]},"141":{"position":[[14,6]]},"143":{"position":[[7,6]]},"644":{"position":[[45,6]]},"688":{"position":[[27,6]]}}}],["messag",{"_index":159,"t":{"263":{"position":[[22,8]]},"278":{"position":[[38,8]]},"284":{"position":[[3,7]]},"286":{"position":[[34,8]]},"403":{"position":[[56,7]]},"428":{"position":[[43,9]]},"434":{"position":[[33,9]]},"676":{"position":[[12,9]]},"702":{"position":[[14,9]]},"857":{"position":[[22,9]]},"884":{"position":[[18,9]]},"886":{"position":[[12,9]]},"888":{"position":[[12,9]]},"936":{"position":[[14,7]]},"1006":{"position":[[0,7]]},"1049":{"position":[[0,9]]},"1097":{"position":[[10,8]]}}}],["metadata",{"_index":408,"t":{"623":{"position":[[25,8]]}}}],["method",{"_index":330,"t":{"472":{"position":[[10,7]]}}}],["methodolog",{"_index":466,"t":{"783":{"position":[[0,11]]},"1116":{"position":[[4,11]]}}}],["metric",{"_index":393,"t":{"579":{"position":[[0,7]]},"583":{"position":[[0,7]]}}}],["miden",{"_index":608,"t":{"1139":{"position":[[4,7]]}}}],["mini",{"_index":583,"t":{"1055":{"position":[[11,4]]}}}],["minim",{"_index":165,"t":{"278":{"position":[[3,10]]},"384":{"position":[[0,7]]}}}],["minimium",{"_index":250,"t":{"386":{"position":[[9,8]]}}}],["minimum",{"_index":411,"t":{"631":{"position":[[9,7]]}}}],["miss",{"_index":518,"t":{"912":{"position":[[16,7]]}}}],["mitig",{"_index":131,"t":{"223":{"position":[[4,10]]},"280":{"position":[[3,10]]}}}],["mobil",{"_index":317,"t":{"460":{"position":[[18,6]]}}}],["mobilephon",{"_index":247,"t":{"382":{"position":[[4,13]]}}}],["mode",{"_index":127,"t":{"217":{"position":[[16,4]]}}}],["model",{"_index":121,"t":{"204":{"position":[[32,5]]},"210":{"position":[[0,5]]},"1031":{"position":[[25,5]]}}}],["modif",{"_index":415,"t":{"644":{"position":[[0,12]]},"646":{"position":[[0,12]]},"650":{"position":[[0,13]]},"652":{"position":[[0,13]]},"1306":{"position":[[3,13]]}}}],["more",{"_index":494,"t":{"861":{"position":[[0,4]]}}}],["mostli",{"_index":254,"t":{"390":{"position":[[0,6]]}}}],["motiv",{"_index":88,"t":{"153":{"position":[[0,10]]},"175":{"position":[[0,10]]},"215":{"position":[[0,10]]},"273":{"position":[[0,10]]},"376":{"position":[[8,10]]}}}],["motivit",{"_index":144,"t":{"253":{"position":[[0,12]]}}}],["move",{"_index":453,"t":{"731":{"position":[[0,6]]},"926":{"position":[[10,4]]}}}],["mpc",{"_index":388,"t":{"567":{"position":[[3,3]]}}}],["multi",{"_index":562,"t":{"1018":{"position":[[23,5]]}}}],["multipl",{"_index":447,"t":{"702":{"position":[[33,8]]}}}],["multipli",{"_index":257,"t":{"392":{"position":[[33,10]]}}}],["multipurpos",{"_index":211,"t":{"334":{"position":[[0,12]]}}}],["mvd",{"_index":249,"t":{"386":{"position":[[0,4]]},"491":{"position":[[17,4]]}}}],["nangang",{"_index":513,"t":{"906":{"position":[[0,7]]}}}],["nat",{"_index":94,"t":{"161":{"position":[[0,3]]},"990":{"position":[[0,3]]}}}],["need",{"_index":309,"t":{"443":{"position":[[32,5]]},"890":{"position":[[10,6]]}}}],["negoti",{"_index":550,"t":{"988":{"position":[[11,11]]}}}],["neighbourhood",{"_index":592,"t":{"1093":{"position":[[0,13]]},"1095":{"position":[[11,13]]}}}],["nescienc",{"_index":0,"t":{"3":{"position":[[0,9]]},"405":{"position":[[0,9]]},"503":{"position":[[4,9]]},"523":{"position":[[3,9]]},"535":{"position":[[19,9]]},"543":{"position":[[21,9]]},"547":{"position":[[35,9]]},"559":{"position":[[4,9]]},"569":{"position":[[20,9]]},"1316":{"position":[[0,9]]}}}],["network",{"_index":37,"t":{"76":{"position":[[7,7]]},"78":{"position":[[13,7]]},"225":{"position":[[21,7]]},"231":{"position":[[5,8]]},"237":{"position":[[18,7]]},"403":{"position":[[98,8]]},"409":{"position":[[19,8]]},"468":{"position":[[33,8]]},"785":{"position":[[0,7]]},"798":{"position":[[0,7]]},"976":{"position":[[36,7]]},"1089":{"position":[[45,7]]}}}],["new",{"_index":126,"t":{"215":{"position":[[17,3]]}}}],["next",{"_index":403,"t":{"611":{"position":[[7,5]]},"745":{"position":[[0,4]]},"1059":{"position":[[7,5]]}}}],["nexu",{"_index":598,"t":{"1121":{"position":[[3,7]]},"1189":{"position":[[3,5]]}}}],["nim",{"_index":20,"t":{"45":{"position":[[0,3]]},"308":{"position":[[3,3]]},"1282":{"position":[[0,3]]}}}],["node",{"_index":34,"t":{"72":{"position":[[16,5]]},"165":{"position":[[10,4]]},"320":{"position":[[5,4]]},"786":{"position":[[16,5]]},"788":{"position":[[16,5]]},"790":{"position":[[16,5]]},"792":{"position":[[16,5]]},"794":{"position":[[17,5]]},"948":{"position":[[34,5]]},"950":{"position":[[12,5]]},"956":{"position":[[30,5]]},"964":{"position":[[7,4]]}}}],["nois",{"_index":285,"t":{"418":{"position":[[0,5]]},"1004":{"position":[[4,5]]},"1008":{"position":[[4,5]]},"1010":{"position":[[10,5]]}}}],["nonmembership",{"_index":86,"t":{"145":{"position":[[9,13]]}}}],["novanet",{"_index":626,"t":{"1157":{"position":[[4,9]]}}}],["now",{"_index":187,"t":{"308":{"position":[[15,3]]},"1057":{"position":[[13,4]]}}}],["nssa",{"_index":339,"t":{"499":{"position":[[12,5]]},"501":{"position":[[4,4]]},"521":{"position":[[22,4]]},"525":{"position":[[22,4]]},"539":{"position":[[31,4]]},"541":{"position":[[31,4]]},"563":{"position":[[12,4]]}}}],["nullifi",{"_index":269,"t":{"401":{"position":[[58,10]]},"682":{"position":[[67,10]]}}}],["number",{"_index":574,"t":{"1043":{"position":[[58,7]]}}}],["nwaku",{"_index":189,"t":{"308":{"position":[[28,5]]},"422":{"position":[[12,5]]}}}],["o1vm",{"_index":632,"t":{"1163":{"position":[[4,6]]}}}],["object",{"_index":556,"t":{"1008":{"position":[[16,7]]}}}],["observ",{"_index":594,"t":{"1097":{"position":[[0,9]]}}}],["offlin",{"_index":255,"t":{"390":{"position":[[7,7]]}}}],["ola",{"_index":606,"t":{"1137":{"position":[[4,5]]}}}],["open",{"_index":141,"t":{"235":{"position":[[0,4]]},"696":{"position":[[16,4]]},"1227":{"position":[[4,8]]}}}],["oper",{"_index":148,"t":{"257":{"position":[[19,9]]},"326":{"position":[[16,10]]}}}],["opinion",{"_index":291,"t":{"424":{"position":[[0,8]]}}}],["optim",{"_index":155,"t":{"261":{"position":[[17,12]]},"352":{"position":[[0,12]]},"354":{"position":[[13,9]]}}}],["out",{"_index":203,"t":{"326":{"position":[[9,3]]},"960":{"position":[[0,3]]}}}],["outsid",{"_index":656,"t":{"1249":{"position":[[18,7]]}}}],["overhead",{"_index":439,"t":{"698":{"position":[[8,8]]}}}],["overlay",{"_index":151,"t":{"259":{"position":[[36,7]]},"403":{"position":[[37,7]]}}}],["overview",{"_index":42,"t":{"80":{"position":[[6,8]]},"155":{"position":[[0,8]]},"179":{"position":[[0,8]]},"243":{"position":[[0,8]]},"457":{"position":[[14,8]]},"527":{"position":[[21,8]]},"682":{"position":[[0,9]]},"839":{"position":[[0,8]]}}}],["p",{"_index":327,"t":{"470":{"position":[[51,1]]}}}],["p2p",{"_index":15,"t":{"33":{"position":[[0,3]]},"378":{"position":[[4,4]]},"403":{"position":[[94,3]]},"432":{"position":[[19,3]]},"455":{"position":[[0,3]]},"460":{"position":[[0,3]]},"678":{"position":[[0,3]]},"1272":{"position":[[0,3]]}}}],["packet",{"_index":575,"t":{"1045":{"position":[[10,7]]}}}],["pair",{"_index":60,"t":{"100":{"position":[[7,7]]},"407":{"position":[[7,7]]},"727":{"position":[[7,7]]}}}],["paper",{"_index":665,"t":{"1262":{"position":[[0,6]]}}}],["paramet",{"_index":404,"t":{"617":{"position":[[13,10]]}}}],["part",{"_index":283,"t":{"415":{"position":[[36,4]]},"871":{"position":[[0,4]]},"884":{"position":[[0,4]]}}}],["parti",{"_index":651,"t":{"1245":{"position":[[9,5]]},"1253":{"position":[[9,5]]},"1300":{"position":[[9,5]]}}}],["partial",{"_index":321,"t":{"468":{"position":[[0,7]]}}}],["pattern",{"_index":555,"t":{"1006":{"position":[[8,8]]}}}],["pedersen",{"_index":375,"t":{"545":{"position":[[25,8]]},"757":{"position":[[6,8]]},"765":{"position":[[6,8]]},"771":{"position":[[6,8]]}}}],["peer",{"_index":193,"t":{"314":{"position":[[3,4]]},"318":{"position":[[10,4]]},"420":{"position":[[16,4]]},"698":{"position":[[21,4]]},"982":{"position":[[0,4]]},"986":{"position":[[10,4]]}}}],["per",{"_index":440,"t":{"698":{"position":[[17,3]]}}}],["perform",{"_index":191,"t":{"310":{"position":[[33,11]]},"625":{"position":[[0,11]]},"656":{"position":[[12,11]]},"974":{"position":[[12,11]]}}}],["peril",{"_index":572,"t":{"1043":{"position":[[42,6]]}}}],["permut",{"_index":323,"t":{"468":{"position":[[21,11]]}}}],["persist",{"_index":576,"t":{"1047":{"position":[[24,10]]}}}],["person",{"_index":648,"t":{"1243":{"position":[[45,8]]},"1245":{"position":[[29,8]]}}}],["phone",{"_index":573,"t":{"1043":{"position":[[52,5]]}}}],["piecrust",{"_index":620,"t":{"1151":{"position":[[4,10]]}}}],["plan",{"_index":305,"t":{"439":{"position":[[11,4]]},"569":{"position":[[10,5]]},"745":{"position":[[22,5]]},"1180":{"position":[[8,4]]}}}],["platform",{"_index":345,"t":{"501":{"position":[[45,9]]}}}],["poc",{"_index":515,"t":{"908":{"position":[[9,3]]},"958":{"position":[[14,3]]}}}],["polici",{"_index":540,"t":{"956":{"position":[[54,8]]},"1255":{"position":[[16,6]]}}}],["posit",{"_index":482,"t":{"816":{"position":[[21,9]]},"1089":{"position":[[29,8]]}}}],["possibl",{"_index":162,"t":{"277":{"position":[[0,8]]},"674":{"position":[[0,8]]}}}],["powdr",{"_index":600,"t":{"1125":{"position":[[3,7]]}}}],["power",{"_index":333,"t":{"472":{"position":[[44,6]]}}}],["prefac",{"_index":63,"t":{"108":{"position":[[0,7]]}}}],["preliminari",{"_index":635,"t":{"1178":{"position":[[0,11]]}}}],["prelud",{"_index":490,"t":{"853":{"position":[[0,8]]}}}],["prerequisit",{"_index":589,"t":{"1089":{"position":[[0,13]]}}}],["present",{"_index":301,"t":{"430":{"position":[[0,10]]}}}],["preserv",{"_index":240,"t":{"364":{"position":[[53,10]]},"432":{"position":[[8,10]]},"565":{"position":[[30,10]]}}}],["prevent",{"_index":268,"t":{"401":{"position":[[28,10]]},"623":{"position":[[4,10]]}}}],["primit",{"_index":373,"t":{"541":{"position":[[17,10]]}}}],["princip",{"_index":568,"t":{"1039":{"position":[[0,9]]}}}],["principl",{"_index":637,"t":{"1215":{"position":[[0,10]]}}}],["priorit",{"_index":176,"t":{"284":{"position":[[11,14]]}}}],["prioriti",{"_index":450,"t":{"716":{"position":[[19,10]]}}}],["privaci",{"_index":28,"t":{"64":{"position":[[10,7]]},"68":{"position":[[7,7]]},"364":{"position":[[45,7]]},"413":{"position":[[9,7]]},"415":{"position":[[5,7]]},"432":{"position":[[0,7]]},"505":{"position":[[17,7]]},"565":{"position":[[22,7]]},"1067":{"position":[[32,8]]},"1071":{"position":[[0,7]]},"1223":{"position":[[4,7]]},"1255":{"position":[[8,7]]}}}],["privat",{"_index":51,"t":{"90":{"position":[[0,7]]},"517":{"position":[[3,7]]},"551":{"position":[[4,7]]}}}],["probabl",{"_index":480,"t":{"816":{"position":[[0,11]]}}}],["problem",{"_index":89,"t":{"153":{"position":[[11,7]]},"175":{"position":[[11,7]]},"177":{"position":[[8,8]]},"235":{"position":[[5,8]]},"275":{"position":[[0,7]]},"376":{"position":[[0,7]]},"621":{"position":[[7,7]]},"716":{"position":[[0,8]]},"892":{"position":[[19,7]]},"918":{"position":[[0,7]]}}}],["process",{"_index":367,"t":{"533":{"position":[[13,9]]},"1243":{"position":[[31,10]]},"1245":{"position":[[15,10]]}}}],["progress",{"_index":128,"t":{"219":{"position":[[0,8]]}}}],["project",{"_index":451,"t":{"723":{"position":[[14,7]]},"1118":{"position":[[5,7]]}}}],["proof",{"_index":82,"t":{"139":{"position":[[0,5]]},"145":{"position":[[0,5]]},"358":{"position":[[8,6]]},"399":{"position":[[14,6]]},"595":{"position":[[0,5]]}}}],["properti",{"_index":276,"t":{"405":{"position":[[37,10]]},"1298":{"position":[[16,8]]}}}],["propos",{"_index":397,"t":{"585":{"position":[[0,8]]}}}],["prospect",{"_index":552,"t":{"992":{"position":[[22,9]]}}}],["protect",{"_index":29,"t":{"64":{"position":[[18,10]]},"68":{"position":[[15,10]]},"322":{"position":[[8,10]]},"413":{"position":[[17,10]]},"432":{"position":[[37,10]]},"682":{"position":[[34,10]]}}}],["protocol",{"_index":35,"t":{"74":{"position":[[7,8]]},"215":{"position":[[21,8]]},"233":{"position":[[26,9]]},"320":{"position":[[20,8]]},"330":{"position":[[0,8]]},"364":{"position":[[28,9]]},"617":{"position":[[4,8]]},"714":{"position":[[0,8]]},"753":{"position":[[6,9]]},"755":{"position":[[12,8]]},"757":{"position":[[15,8]]},"763":{"position":[[8,8]]},"765":{"position":[[15,8]]},"769":{"position":[[8,8]]},"771":{"position":[[15,8]]},"843":{"position":[[0,9]]},"930":{"position":[[16,8]]},"982":{"position":[[14,8]]},"984":{"position":[[8,9]]},"986":{"position":[[24,8]]},"1004":{"position":[[10,8]]}}}],["prove",{"_index":428,"t":{"658":{"position":[[0,7]]}}}],["provid",{"_index":563,"t":{"1029":{"position":[[12,8]]}}}],["pseudo",{"_index":292,"t":{"424":{"position":[[9,6]]}}}],["public",{"_index":357,"t":{"515":{"position":[[3,6]]},"553":{"position":[[5,6]]},"1002":{"position":[[21,6]]}}}],["publish",{"_index":436,"t":{"690":{"position":[[0,10]]}}}],["qa",{"_index":9,"t":{"15":{"position":[[18,4]]},"1278":{"position":[[18,4]]}}}],["qualiti",{"_index":7,"t":{"15":{"position":[[0,7]]},"1278":{"position":[[0,7]]}}}],["question",{"_index":137,"t":{"227":{"position":[[30,8]]}}}],["random",{"_index":544,"t":{"972":{"position":[[0,6]]},"974":{"position":[[0,6]]}}}],["rate",{"_index":107,"t":{"184":{"position":[[0,4]]},"401":{"position":[[44,4]]},"453":{"position":[[29,4]]},"682":{"position":[[53,4]]},"702":{"position":[[24,4]]}}}],["rational",{"_index":216,"t":{"344":{"position":[[0,9]]}}}],["reach",{"_index":202,"t":{"326":{"position":[[0,8]]}}}],["recap",{"_index":511,"t":{"898":{"position":[[0,5]]}}}],["recurs",{"_index":386,"t":{"565":{"position":[[3,9]]}}}],["redund",{"_index":173,"t":{"282":{"position":[[15,9]]}}}],["refer",{"_index":62,"t":{"104":{"position":[[0,10]]},"123":{"position":[[0,10]]},"149":{"position":[[0,10]]},"239":{"position":[[0,10]]},"249":{"position":[[0,10]]},"269":{"position":[[0,10]]},"290":{"position":[[0,10]]},"336":{"position":[[0,10]]},"372":{"position":[[0,10]]},"478":{"position":[[0,10]]},"571":{"position":[[0,10]]},"601":{"position":[[0,10]]},"637":{"position":[[0,10]]},"666":{"position":[[0,10]]},"708":{"position":[[0,10]]},"733":{"position":[[0,10]]},"775":{"position":[[0,10]]},"804":{"position":[[0,10]]},"830":{"position":[[0,10]]},"994":{"position":[[0,10]]},"1024":{"position":[[0,10]]},"1063":{"position":[[0,10]]},"1105":{"position":[[0,10]]},"1169":{"position":[[0,10]]},"1204":{"position":[[0,10]]}}}],["registr",{"_index":434,"t":{"686":{"position":[[10,12]]},"702":{"position":[[42,13]]}}}],["relat",{"_index":104,"t":{"177":{"position":[[0,7]]},"984":{"position":[[18,7]]}}}],["relax",{"_index":535,"t":{"954":{"position":[[3,5]]}}}],["relay",{"_index":41,"t":{"78":{"position":[[34,5]]},"86":{"position":[[4,5]]},"88":{"position":[[4,5]]},"415":{"position":[[65,5]]}}}],["reliabl",{"_index":245,"t":{"380":{"position":[[4,8]]}}}],["remot",{"_index":314,"t":{"455":{"position":[[21,6]]},"482":{"position":[[0,6]]}}}],["replac",{"_index":308,"t":{"443":{"position":[[20,11]]}}}],["replay",{"_index":591,"t":{"1091":{"position":[[0,6]]}}}],["requir",{"_index":248,"t":{"384":{"position":[[8,12]]},"577":{"position":[[16,12]]},"631":{"position":[[17,12]]},"954":{"position":[[27,11]]}}}],["research",{"_index":57,"t":{"94":{"position":[[6,8]]},"849":{"position":[[0,8]]}}}],["resist",{"_index":587,"t":{"1077":{"position":[[11,10]]},"1219":{"position":[[15,10]]}}}],["resourc",{"_index":646,"t":{"1235":{"position":[[3,15]]}}}],["respect",{"_index":654,"t":{"1247":{"position":[[32,7]]}}}],["respons",{"_index":67,"t":{"112":{"position":[[10,8]]}}}],["result",{"_index":100,"t":{"167":{"position":[[0,7]]},"1184":{"position":[[0,7]]}}}],["revers",{"_index":135,"t":{"227":{"position":[[0,9]]}}}],["review",{"_index":391,"t":{"567":{"position":[[46,7]]}}}],["rfc",{"_index":10,"t":{"21":{"position":[[0,3]]},"1286":{"position":[[0,3]]}}}],["rid",{"_index":332,"t":{"472":{"position":[[26,3]]}}}],["right",{"_index":661,"t":{"1251":{"position":[[20,6]]},"1298":{"position":[[25,6]]}}}],["risc",{"_index":227,"t":{"352":{"position":[[35,4]]}}}],["risc0",{"_index":599,"t":{"1123":{"position":[[3,7]]},"1187":{"position":[[3,5]]}}}],["rln",{"_index":40,"t":{"78":{"position":[[30,3]]},"80":{"position":[[0,3]]},"82":{"position":[[0,3]]},"84":{"position":[[0,3]]},"86":{"position":[[0,3]]},"88":{"position":[[0,3]]},"229":{"position":[[0,3]]},"322":{"position":[[25,3]]},"397":{"position":[[0,3]]},"399":{"position":[[10,3]]},"617":{"position":[[0,3]]},"631":{"position":[[37,3]]},"633":{"position":[[0,3]]},"828":{"position":[[23,3]]}}}],["roadmap",{"_index":243,"t":{"366":{"position":[[10,7]]},"892":{"position":[[44,7]]}}}],["role",{"_index":337,"t":{"486":{"position":[[0,5]]}}}],["rough",{"_index":316,"t":{"457":{"position":[[8,5]]},"716":{"position":[[13,5]]}}}],["rout",{"_index":437,"t":{"692":{"position":[[0,7]]},"940":{"position":[[17,7]]},"1045":{"position":[[0,7]]}}}],["run",{"_index":412,"t":{"631":{"position":[[33,3]]},"948":{"position":[[30,3]]}}}],["rust",{"_index":336,"t":{"474":{"position":[[43,4]]}}}],["sc",{"_index":23,"t":{"49":{"position":[[16,4]]},"1280":{"position":[[16,4]]}}}],["scalabl",{"_index":120,"t":{"204":{"position":[[20,11]]},"233":{"position":[[0,11]]},"265":{"position":[[7,11]]}}}],["scale",{"_index":279,"t":{"409":{"position":[[28,8]]}}}],["scheme",{"_index":426,"t":{"652":{"position":[[44,6]]}}}],["schnorr",{"_index":458,"t":{"755":{"position":[[4,7]]},"763":{"position":[[0,7]]},"769":{"position":[[0,7]]}}}],["scope",{"_index":111,"t":{"186":{"position":[[11,5]]},"960":{"position":[[7,5]]},"1087":{"position":[[0,5]]}}}],["seamless",{"_index":237,"t":{"362":{"position":[[8,8]]}}}],["secret",{"_index":46,"t":{"84":{"position":[[15,6]]},"619":{"position":[[15,6]]}}}],["secur",{"_index":206,"t":{"328":{"position":[[22,8]]},"470":{"position":[[18,8]]},"627":{"position":[[0,8]]},"1053":{"position":[[0,6]]},"1067":{"position":[[22,9]]},"1069":{"position":[[0,8]]},"1221":{"position":[[5,8]]},"1247":{"position":[[3,8]]},"1288":{"position":[[0,8]]}}}],["self",{"_index":569,"t":{"1043":{"position":[[10,4]]}}}],["semaphor",{"_index":311,"t":{"453":{"position":[[19,9]]}}}],["separ",{"_index":214,"t":{"342":{"position":[[23,10]]},"503":{"position":[[20,10]]},"976":{"position":[[17,8]]}}}],["sequenc",{"_index":366,"t":{"531":{"position":[[0,9]]}}}],["servic",{"_index":54,"t":{"90":{"position":[[21,7]]},"229":{"position":[[8,7]]}}}],["session",{"_index":561,"t":{"1018":{"position":[[0,7]]}}}],["set",{"_index":416,"t":{"644":{"position":[[23,3]]}}}],["settl",{"_index":580,"t":{"1051":{"position":[[33,6]]}}}],["settlement",{"_index":52,"t":{"90":{"position":[[8,10]]},"958":{"position":[[3,10]]}}}],["setup",{"_index":433,"t":{"686":{"position":[[0,5]]}}}],["shamir",{"_index":461,"t":{"761":{"position":[[9,6]]},"763":{"position":[[38,6]]},"765":{"position":[[45,6]]},"767":{"position":[[25,6]]},"769":{"position":[[36,6]]},"771":{"position":[[38,6]]}}}],["shamir'",{"_index":45,"t":{"84":{"position":[[6,8]]}}}],["shard",{"_index":531,"t":{"944":{"position":[[9,8]]}}}],["share",{"_index":47,"t":{"84":{"position":[[22,7]]}}}],["sigma",{"_index":457,"t":{"753":{"position":[[0,5]]}}}],["signal",{"_index":424,"t":{"652":{"position":[[24,9]]}}}],["simpl",{"_index":547,"t":{"976":{"position":[[0,6]]}}}],["simplifi",{"_index":524,"t":{"930":{"position":[[3,8]]}}}],["simul",{"_index":253,"t":{"388":{"position":[[6,10]]},"741":{"position":[[0,10]]}}}],["size",{"_index":467,"t":{"785":{"position":[[8,4]]}}}],["slash",{"_index":438,"t":{"694":{"position":[[19,8]]}}}],["slide",{"_index":483,"t":{"818":{"position":[[0,7]]}}}],["smart",{"_index":21,"t":{"49":{"position":[[0,5]]},"525":{"position":[[3,5]]},"1280":{"position":[[0,5]]}}}],["snarko",{"_index":616,"t":{"1147":{"position":[[4,9]]}}}],["social",{"_index":117,"t":{"194":{"position":[[0,6]]}}}],["solut",{"_index":221,"t":{"348":{"position":[[15,9]]},"585":{"position":[[9,8]]},"674":{"position":[[9,9]]},"922":{"position":[[0,8]]},"976":{"position":[[7,9]]}}}],["sovereign",{"_index":570,"t":{"1043":{"position":[[15,9]]}}}],["sp1",{"_index":597,"t":{"1119":{"position":[[3,5]]},"1185":{"position":[[3,3]]}}}],["spam",{"_index":39,"t":{"78":{"position":[[21,4]]},"322":{"position":[[3,4]]},"432":{"position":[[32,4]]},"672":{"position":[[19,9]]},"682":{"position":[[29,4]]},"694":{"position":[[0,4]]}}}],["spars",{"_index":85,"t":{"143":{"position":[[0,6]]}}}],["spec",{"_index":512,"t":{"902":{"position":[[0,5]]}}}],["specif",{"_index":590,"t":{"1089":{"position":[[20,8]]},"1182":{"position":[[8,14]]}}}],["spend",{"_index":378,"t":{"549":{"position":[[3,8]]}}}],["spn",{"_index":328,"t":{"470":{"position":[[53,4]]}}}],["spotlight",{"_index":64,"t":{"110":{"position":[[0,9]]}}}],["squar",{"_index":324,"t":{"470":{"position":[[0,6]]}}}],["stabil",{"_index":190,"t":{"310":{"position":[[19,9]]}}}],["stack",{"_index":449,"t":{"714":{"position":[[9,5]]},"1041":{"position":[[30,5]]}}}],["stage",{"_index":636,"t":{"1198":{"position":[[0,5]]},"1200":{"position":[[0,5]]}}}],["state",{"_index":213,"t":{"342":{"position":[[17,5]]},"370":{"position":[[8,5]]},"503":{"position":[[14,5]]},"515":{"position":[[10,5]]},"517":{"position":[[11,5]]},"737":{"position":[[8,5]]},"900":{"position":[[8,5]]},"1008":{"position":[[10,5]]}}}],["statement",{"_index":668,"t":{"1296":{"position":[[19,10]]}}}],["static",{"_index":541,"t":{"964":{"position":[[0,6]]},"1002":{"position":[[14,6]]}}}],["statu",{"_index":132,"t":{"223":{"position":[[19,6]]},"855":{"position":[[0,6]]}}}],["stellar",{"_index":624,"t":{"1155":{"position":[[4,9]]}}}],["step",{"_index":448,"t":{"704":{"position":[[22,5]]},"745":{"position":[[5,5]]},"1022":{"position":[[7,5]]}}}],["storag",{"_index":409,"t":{"629":{"position":[[0,7]]},"698":{"position":[[0,7]]},"1047":{"position":[[0,7]]}}}],["store",{"_index":209,"t":{"332":{"position":[[9,5]]}}}],["strateg",{"_index":242,"t":{"366":{"position":[[0,9]]}}}],["strategi",{"_index":232,"t":{"356":{"position":[[0,10]]}}}],["strengthen",{"_index":266,"t":{"401":{"position":[[0,13]]}}}],["strong",{"_index":463,"t":{"763":{"position":[[26,6]]},"765":{"position":[[33,6]]}}}],["structur",{"_index":78,"t":{"131":{"position":[[5,9]]}}}],["studi",{"_index":306,"t":{"441":{"position":[[12,6]]},"453":{"position":[[12,6]]}}}],["substitut",{"_index":322,"t":{"468":{"position":[[8,12]]}}}],["subtre",{"_index":265,"t":{"399":{"position":[[43,8]]}}}],["summari",{"_index":58,"t":{"96":{"position":[[0,7]]},"267":{"position":[[0,7]]},"288":{"position":[[0,7]]},"368":{"position":[[0,7]]},"539":{"position":[[0,7]]},"1103":{"position":[[0,7]]},"1165":{"position":[[0,7]]},"1197":{"position":[[0,7]]},"1202":{"position":[[0,7]]}}}],["support",{"_index":455,"t":{"739":{"position":[[25,8]]},"1010":{"position":[[0,9]]},"1018":{"position":[[36,7]]}}}],["surveil",{"_index":293,"t":{"424":{"position":[[30,12]]},"1093":{"position":[[14,12]]}}}],["switzerland",{"_index":659,"t":{"1249":{"position":[[49,11]]}}}],["sync",{"_index":246,"t":{"380":{"position":[[13,4]]},"455":{"position":[[9,4]]},"460":{"position":[[9,4]]},"579":{"position":[[11,4]]},"589":{"position":[[0,7]]}}}],["synchron",{"_index":389,"t":{"567":{"position":[[13,15]]}}}],["system",{"_index":17,"t":{"39":{"position":[[12,7]]},"346":{"position":[[20,8]]},"354":{"position":[[23,7]]},"676":{"position":[[22,7]]},"678":{"position":[[4,7]]},"1276":{"position":[[12,7]]}}}],["tabl",{"_index":98,"t":{"163":{"position":[[6,6]]},"296":{"position":[[17,6]]},"1197":{"position":[[8,5]]}}}],["take",{"_index":653,"t":{"1247":{"position":[[24,4]]}}}],["takeaway",{"_index":124,"t":{"212":{"position":[[0,9]]}}}],["talk",{"_index":298,"t":{"428":{"position":[[0,5]]},"434":{"position":[[0,6]]}}}],["tech",{"_index":294,"t":{"424":{"position":[[43,4]]}}}],["technic",{"_index":114,"t":{"188":{"position":[[0,9]]},"192":{"position":[[0,9]]},"680":{"position":[[0,9]]}}}],["tell",{"_index":538,"t":{"956":{"position":[[39,4]]}}}],["term",{"_index":431,"t":{"680":{"position":[[10,5]]},"712":{"position":[[6,5]]},"1312":{"position":[[24,5]]}}}],["terminolog",{"_index":106,"t":{"180":{"position":[[6,11]]}}}],["test",{"_index":18,"t":{"39":{"position":[[20,7]]},"581":{"position":[[0,4]]},"1180":{"position":[[0,7]]},"1276":{"position":[[20,7]]}}}],["testnet",{"_index":50,"t":{"88":{"position":[[23,7]]},"847":{"position":[[0,7]]},"906":{"position":[[8,7]]}}}],["theoret",{"_index":119,"t":{"204":{"position":[[8,11]]},"781":{"position":[[0,11]]}}}],["theori",{"_index":414,"t":{"643":{"position":[[0,6]]}}}],["thing",{"_index":517,"t":{"912":{"position":[[0,6]]}}}],["third",{"_index":650,"t":{"1245":{"position":[[3,5]]},"1253":{"position":[[3,5]]},"1300":{"position":[[3,5]]}}}],["thought",{"_index":102,"t":{"169":{"position":[[8,8]]},"196":{"position":[[8,8]]}}}],["threat",{"_index":588,"t":{"1085":{"position":[[13,6]]}}}],["through",{"_index":312,"t":{"453":{"position":[[43,7]]},"682":{"position":[[45,7]]}}}],["time",{"_index":167,"t":{"278":{"position":[[23,4]]},"579":{"position":[[16,4]]}}}],["tke",{"_index":6,"t":{"9":{"position":[[16,5]]},"1274":{"position":[[16,5]]}}}],["token",{"_index":4,"t":{"9":{"position":[[0,5]]},"1274":{"position":[[0,5]]}}}],["topic",{"_index":97,"t":{"163":{"position":[[0,5]]},"853":{"position":[[9,6]]},"934":{"position":[[3,5]]},"944":{"position":[[3,5]]},"978":{"position":[[7,5]]}}}],["toward",{"_index":259,"t":{"397":{"position":[[8,7]]}}}],["toy",{"_index":565,"t":{"1031":{"position":[[21,3]]}}}],["track",{"_index":521,"t":{"926":{"position":[[0,5]]},"940":{"position":[[0,5]]},"948":{"position":[[0,5]]}}}],["traffic",{"_index":475,"t":{"798":{"position":[[8,7]]}}}],["trail",{"_index":70,"t":{"115":{"position":[[11,8]]}}}],["transact",{"_index":579,"t":{"1051":{"position":[[10,9]]}}}],["transfer",{"_index":166,"t":{"278":{"position":[[14,8]]}}}],["transmiss",{"_index":174,"t":{"282":{"position":[[25,13]]}}}],["transpar",{"_index":639,"t":{"1225":{"position":[[3,12]]}}}],["transport",{"_index":169,"t":{"280":{"position":[[14,9]]}}}],["travers",{"_index":551,"t":{"990":{"position":[[4,9]]}}}],["tree",{"_index":77,"t":{"131":{"position":[[0,4]]},"133":{"position":[[7,5]]},"137":{"position":[[7,4]]},"141":{"position":[[21,5]]},"143":{"position":[[14,5]]},"147":{"position":[[7,5]]},"563":{"position":[[3,5]]},"579":{"position":[[27,4]]},"644":{"position":[[52,4]]},"688":{"position":[[34,4]]}}}],["trilemma",{"_index":585,"t":{"1075":{"position":[[10,8]]}}}],["triton",{"_index":612,"t":{"1143":{"position":[[4,8]]}}}],["truth",{"_index":539,"t":{"956":{"position":[[48,5]]}}}],["two",{"_index":219,"t":{"346":{"position":[[16,3]]}}}],["type",{"_index":362,"t":{"521":{"position":[[13,5]]},"1079":{"position":[[9,5]]}}}],["typic",{"_index":66,"t":{"112":{"position":[[2,7]]}}}],["under",{"_index":390,"t":{"567":{"position":[[39,6]]}}}],["union",{"_index":658,"t":{"1249":{"position":[[39,5]]}}}],["unstructur",{"_index":272,"t":{"403":{"position":[[81,12]]}}}],["up",{"_index":507,"t":{"892":{"position":[[12,2]]},"910":{"position":[[7,2]]},"1264":{"position":[[6,3]]}}}],["upcom",{"_index":500,"t":{"882":{"position":[[13,8]]},"914":{"position":[[0,8]]}}}],["updat",{"_index":303,"t":{"436":{"position":[[8,6]]},"447":{"position":[[5,6]]}}}],["us",{"_index":199,"t":{"322":{"position":[[19,5]]},"503":{"position":[[51,4]]},"505":{"position":[[0,3]]},"545":{"position":[[0,3],[17,3]]},"767":{"position":[[9,3]]},"851":{"position":[[0,3]]},"1243":{"position":[[69,3]]},"1312":{"position":[[33,3]]}}}],["usag",{"_index":93,"t":{"159":{"position":[[13,5]]},"633":{"position":[[4,5]]}}}],["user",{"_index":113,"t":{"186":{"position":[[30,6]]},"499":{"position":[[20,4]]},"523":{"position":[[13,5]]},"529":{"position":[[0,4]]},"619":{"position":[[10,4]]},"623":{"position":[[20,4]]},"739":{"position":[[9,5]]},"948":{"position":[[25,4]]},"1029":{"position":[[32,4]]}}}],["utxo",{"_index":368,"t":{"533":{"position":[[27,4]]},"535":{"position":[[29,4]]},"537":{"position":[[3,4]]},"539":{"position":[[11,4]]},"545":{"position":[[53,4]]}}}],["v",{"_index":228,"t":{"352":{"position":[[40,2]]},"557":{"position":[[0,2]]},"1225":{"position":[[0,2]]}}}],["v1",{"_index":295,"t":{"426":{"position":[[5,2]]},"871":{"position":[[50,2]]},"874":{"position":[[19,2]]},"876":{"position":[[8,2]]},"938":{"position":[[8,2]]}}}],["v2",{"_index":289,"t":{"420":{"position":[[5,2]]},"426":{"position":[[16,3]]},"428":{"position":[[27,2]]},"430":{"position":[[25,2]]},"432":{"position":[[56,2]]},"434":{"position":[[17,2]]},"436":{"position":[[5,2]]},"439":{"position":[[25,3]]},"605":{"position":[[5,2]]},"607":{"position":[[5,2]]},"781":{"position":[[33,2]]},"838":{"position":[[5,2]]},"853":{"position":[[24,2]]},"859":{"position":[[14,2]]},"871":{"position":[[61,2]]},"876":{"position":[[14,2]]},"878":{"position":[[9,2]]},"880":{"position":[[8,2]]},"882":{"position":[[8,2]]}}}],["v3",{"_index":258,"t":{"397":{"position":[[4,3]]}}}],["v5",{"_index":198,"t":{"320":{"position":[[29,2]]},"968":{"position":[[10,2]]}}}],["vac",{"_index":300,"t":{"428":{"position":[[17,4]]},"434":{"position":[[7,4]]},"457":{"position":[[0,3]]},"871":{"position":[[9,3]]},"872":{"position":[[3,3]]}}}],["vagu",{"_index":74,"t":{"119":{"position":[[21,5]]}}}],["valid",{"_index":421,"t":{"650":{"position":[[32,10]]}}}],["valida",{"_index":602,"t":{"1129":{"position":[[3,8]]},"1195":{"position":[[3,6]]}}}],["verif",{"_index":234,"t":{"358":{"position":[[28,12]]},"660":{"position":[[0,12]]}}}],["verifi",{"_index":263,"t":{"399":{"position":[[0,9]]}}}],["verkl",{"_index":87,"t":{"147":{"position":[[0,6]]}}}],["version",{"_index":252,"t":{"386":{"position":[[25,7]]}}}],["vi",{"_index":384,"t":{"559":{"position":[[0,3]]},"1227":{"position":[[0,3]]}}}],["via",{"_index":446,"t":{"702":{"position":[[29,3]]}}}],["viabl",{"_index":251,"t":{"386":{"position":[[18,6]]}}}],["view",{"_index":382,"t":{"555":{"position":[[4,7]]}}}],["vii",{"_index":385,"t":{"561":{"position":[[0,4]]},"1229":{"position":[[0,4]]}}}],["viii",{"_index":641,"t":{"1231":{"position":[[0,5]]}}}],["virtual",{"_index":222,"t":{"350":{"position":[[8,7]]},"519":{"position":[[24,7]]}}}],["vs",{"_index":296,"t":{"426":{"position":[[8,2]]},"826":{"position":[[14,2]]}}}],["waku",{"_index":32,"t":{"70":{"position":[[0,4]]},"72":{"position":[[0,4]]},"74":{"position":[[0,4]]},"76":{"position":[[0,4]]},"214":{"position":[[12,4]]},"217":{"position":[[11,4]]},"225":{"position":[[16,4]]},"231":{"position":[[0,4]]},"237":{"position":[[13,4]]},"308":{"position":[[7,4]]},"320":{"position":[[0,4]]},"401":{"position":[[72,4]]},"407":{"position":[[21,4],[33,4]]},"409":{"position":[[14,4]]},"411":{"position":[[0,4]]},"415":{"position":[[0,4],[60,4]]},"418":{"position":[[47,4]]},"420":{"position":[[0,4]]},"426":{"position":[[0,4],[11,4]]},"428":{"position":[[22,4]]},"430":{"position":[[14,5],[20,4]]},"432":{"position":[[51,4]]},"434":{"position":[[12,4]]},"436":{"position":[[0,4]]},"439":{"position":[[20,4]]},"447":{"position":[[0,4]]},"451":{"position":[[20,4]]},"605":{"position":[[0,4]]},"607":{"position":[[0,4]]},"650":{"position":[[43,6]]},"652":{"position":[[51,6]]},"723":{"position":[[0,4]]},"739":{"position":[[20,4]]},"743":{"position":[[19,4]]},"781":{"position":[[28,4]]},"838":{"position":[[0,4]]},"853":{"position":[[19,4]]},"871":{"position":[[45,4],[56,4]]},"874":{"position":[[14,4]]},"876":{"position":[[3,4]]},"878":{"position":[[4,4]]},"880":{"position":[[3,4]]},"882":{"position":[[3,4]]},"908":{"position":[[0,4]]},"938":{"position":[[3,4]]},"1010":{"position":[[30,4]]}}}],["waku'",{"_index":407,"t":{"621":{"position":[[0,6]]}}}],["walk",{"_index":545,"t":{"972":{"position":[[7,4]]},"974":{"position":[[7,4]]}}}],["walletconnect",{"_index":493,"t":{"859":{"position":[[0,13]]}}}],["want",{"_index":566,"t":{"1033":{"position":[[11,5]]}}}],["wasm",{"_index":226,"t":{"352":{"position":[[26,4]]}}}],["way",{"_index":442,"t":{"700":{"position":[[15,3]]}}}],["weak",{"_index":465,"t":{"769":{"position":[[26,4]]}}}],["web",{"_index":514,"t":{"908":{"position":[[5,3]]}}}],["web3",{"_index":31,"t":{"66":{"position":[[0,4]]},"1041":{"position":[[25,4]]}}}],["websit",{"_index":649,"t":{"1243":{"position":[[80,7]]},"1247":{"position":[[47,7]]},"1300":{"position":[[15,7]]},"1312":{"position":[[16,7]]}}}],["wechat",{"_index":307,"t":{"443":{"position":[[13,6]]},"1029":{"position":[[5,6]]},"1031":{"position":[[4,6]]}}}],["what'",{"_index":304,"t":{"439":{"position":[[0,6]]},"611":{"position":[[0,6]]},"890":{"position":[[3,6]]},"1059":{"position":[[0,6]]}}}],["whisper",{"_index":118,"t":{"204":{"position":[[0,7]]},"451":{"position":[[7,7]]},"743":{"position":[[28,7]]},"871":{"position":[[34,7]]},"874":{"position":[[3,7]]}}}],["window",{"_index":484,"t":{"818":{"position":[[8,6]]}}}],["work",{"_index":201,"t":{"324":{"position":[[7,4]]},"394":{"position":[[7,4]]},"493":{"position":[[7,4]]},"597":{"position":[[7,4]]},"635":{"position":[[7,4]]},"664":{"position":[[7,4]]},"802":{"position":[[7,4]]},"1031":{"position":[[11,5]]},"1103":{"position":[[19,4]]}}}],["write",{"_index":666,"t":{"1264":{"position":[[0,5]]}}}],["x",{"_index":645,"t":{"1235":{"position":[[0,2]]}}}],["xk1",{"_index":558,"t":{"1014":{"position":[[4,3]]}}}],["xx",{"_index":559,"t":{"1016":{"position":[[4,2]]}}}],["xxpsk0",{"_index":560,"t":{"1016":{"position":[[11,6]]}}}],["zero",{"_index":359,"t":{"519":{"position":[[8,5]]}}}],["zerokit",{"_index":56,"t":{"92":{"position":[[0,7]]}}}],["zk",{"_index":13,"t":{"27":{"position":[[25,2]]},"68":{"position":[[0,2]]},"565":{"position":[[41,2]]},"1284":{"position":[[23,2]]}}}],["zkllvm",{"_index":628,"t":{"1159":{"position":[[4,8]]}}}],["zkmip",{"_index":601,"t":{"1127":{"position":[[3,8]]},"1191":{"position":[[3,6]]}}}],["zkmove",{"_index":630,"t":{"1161":{"position":[[4,8]]}}}],["zko",{"_index":610,"t":{"1141":{"position":[[4,6]]}}}],["zkp",{"_index":230,"t":{"354":{"position":[[9,3]]}}}],["zksnark",{"_index":313,"t":{"453":{"position":[[51,8]]}}}],["zkvm",{"_index":273,"t":{"405":{"position":[[14,4]]},"519":{"position":[[3,4]]},"565":{"position":[[44,4]]},"1112":{"position":[[10,5]]},"1114":{"position":[[4,5]]},"1118":{"position":[[0,4]]}}}],["zkwasm",{"_index":604,"t":{"1133":{"position":[[3,8]]},"1193":{"position":[[3,6]]}}}]],"pipeline":["stemmer"]}},{"documents":[{"i":2,"t":"In this post, we recap Vac's achievements in 2024 and look forward to 2025. With 2024 now behind us and a new year ahead, Vac is proud to reflect on the milestones and breakthroughs that defined another year of researching and developing free and open digital public goods for the Institute of Free Technology and wider web3 ecosystem. Vac comprises various subteams and service units, each with its own focus. Below, we celebrate each unit's achievements and look forward to its 2025 plans.","s":"Vac 2024 Year in Review","u":"/rlog/2024-recap","h":"","p":1},{"i":4,"t":"Nescience is our state separation architecture that aims to enable private transactions and provide a general-purpose execution environment for classical applications.","s":"Nescience","u":"/rlog/2024-recap","h":"#nescience","p":1},{"i":6,"t":"This year, the Nescience state separation architecture moved from exploration to real progress, taking significant steps towards building a functional and reliable system. The team focused on turning ideas into something real, testing the proposed architecture, and understanding its strengths and weaknesses. ZkVM exploration and benchmarks Published deep reviews of 23 existing zkVMs Benchmarked the performance of the six zkVMs that best fit Nescience Defined the NSSA architecture Brought clarity to NSSA’s design and explained the system’s architecture in a lengthy exploratory blog post Built the sandboxed testnet Designed the first version of the node specification All core components (execution types, UTXOs, cryptographic primitives) implemented and being tested Testing the performance of all execution types in various scenarios We also made progress on the essential parts of NSSA’s system, including: Key protocol for secure key management Execution types and circuits for reliable computation UTXO specification to manage state transitions effectively Cryptography module to ensure privacy and security","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights","p":1},{"i":8,"t":"In 2025, the Nescience team plans to double down on what works, fix what doesn’t, and push NSSA closer to real-world use. Sandboxed testnet data analysis – the sandboxed testnet will be our primary data source that we will analyse to identify issues, limitations, and areas for improvement. Expanding the node – expand sandboxed components into a full node implementation with rigorous testing and iterative optimization (to bridge the gap between proof of concept and production readiness). Finalizing the architecture and RFC – after completing NSSA’s architecture, we will draft an RFC to ensure transparency and enable greater collaboration with the broader ecosystem. Testing real-life scenarios – applying NSSA to diverse, practical use cases to assess its adaptability, performance, and impact. Ongoing optimization – ensure NSSA is robust, efficient, and ready to scale.","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward","p":1},{"i":10,"t":"The TKE Service Unit works closely with IFT portfolio projects to design and implement crypto-economic incentive structures.","s":"Token Economics (TKE)","u":"/rlog/2024-recap","h":"#token-economics-tke","p":1},{"i":12,"t":"Formalized and implemented Codex economic incentives in the Litepaper and simulations Orchestrated Status Network incentive structure and smart contract implementation Started building Nomos’s economic model Consulted and provided analysis of incentives for the Logos Operators ordinals project Drove discussions on the economic sustainability of Waku; helped define RLN membership and its payment mechanism","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-1","p":1},{"i":14,"t":"In 2025, TKE will continue to support IFT portfolio projects, working toward economic sustainability while strengthening relationships within the organization. Additionally, the service unit aims to continue building its external reputation through partnerships and publications of relevant work on the Vac forum.","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-1","p":1},{"i":16,"t":"The QA Service Unit focuses on the development and execution of comprehensive test plans, including implementing unit and interoperability testing.","s":"Quality Assurance (QA)","u":"/rlog/2024-recap","h":"#quality-assurance-qa","p":1},{"i":18,"t":"Matured Waku interoperability testing framework with coverage for all major protocols and features Began collaboration with Nomos, contributing to unit and integration testing Partnered with the Status team to test message reliability under unstable network conditions","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-2","p":1},{"i":20,"t":"Extend collaboration with the Waku team on go-waku bindings and message reliability testing Cement working relationship with the Nomos team through the building of an E2E testing framework for higher-level node validation Work closely with Status’s QA team to enhance the functional testing framework Continue work on nim-libp2p testing Expand collaboration to additional projects","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-2","p":1},{"i":22,"t":"The RFC Service Unit takes on the responsibility of shepherding and editing specifications for IFT projects. The unit acts as a linchpin for ensuring standardized and interoperable protocols within the IFT ecosystem.","s":"RFC","u":"/rlog/2024-recap","h":"#rfc","p":1},{"i":24,"t":"Working to implement RFC culture across the IFT ecosystem Began editorial work for several IFT portfolio projects: Status, Nomos, Waku, and Codex. Reworked our standards with regard to writing RFCs to a consensus-oriented specification system","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-3","p":1},{"i":26,"t":"Continue to implement RFC culture across the IFT ecosystem Broaden the number of RFCs produced – particularly for IFT portfolio projects nearing public releases Include new projects with the rfc-index Encourage external projects requiring RFCs to establish relationships with the service unit","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-3","p":1},{"i":28,"t":"The ACZ Service Unit focuses on cryptographic solutions and zero-knowledge proofs, enhancing the security, privacy, and trustworthiness of IFT portfolio projects and contributing to the overall integrity and resilience of the decentralized web ecosystem.","s":"Applied Cryptography and ZK (ACZ)","u":"/rlog/2024-recap","h":"#applied-cryptography-and-zk-acz","p":1},{"i":30,"t":"Researched a libp2p mix protocol and first proof-of-concept implementation (including ping and GossipSub over mix) Researched a decentralized version of MLS (message layer security) with a first proof of concept Released Zerokit v0.6.0 and v0.5.0 Added gnark RLN implementation Released Stealth Address Kit v0.3.1, v0.2.0, and v0.1.0 Published: Verifying RLN Proofs in Light Clients with Subtrees RLN-v3: Towards a Flexible and Cost-Efficient Implementation","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-4","p":1},{"i":32,"t":"Ensure libp2p mix protocol is production-ready and support with the publishing of a paper and blog posts Ensure decentralized MLS is production-ready and support with the publishing of a paper and blog posts Begin explorations of additional research topics Release Zerokit v0.7 and future versions","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-4","p":1},{"i":34,"t":"The P2P Service Unit specializes in peer-to-peer technologies and develops nim-libp2p, improves the libp2p GossipSub protocol, and assists IFT portfolio projects with the integration of P2P network layers.","s":"P2P","u":"/rlog/2024-recap","h":"#p2p","p":1},{"i":36,"t":"Analysis and work on libp2p GossipSub improvements Published: Libp2p GossipSub IDONTWANT Message Performance Impact PR to libp2p specifications about specific lib2p GossipSub improvements we researched and tested https://github.com/libp2p/specs/pull/654","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-5","p":1},{"i":38,"t":"Add new features to nim-libp2p: QUIC transport, web transport Update specifications for libp2p GossipSub, aiming to significantly improve its performance","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-5","p":1},{"i":40,"t":"The DST Service Unit’s primary objective is to assist IFT portfolio projects in understanding the scaling behavior of their nodes within larger networks. By conducting thorough regression testing, the DST unit helps ensure the reliability and stability of projects.","s":"Distributed Systems Testing (DST)","u":"/rlog/2024-recap","h":"#distributed-systems-testing-dst","p":1},{"i":42,"t":"DST compute resources transitioned from a hosted environment to a dedicated Vac Lab, enabling better customization of resources and adding significantly more compute power – enabled much higher and more stable simulations (several hundred nodes to several thousand) and enhanced environmental control. Maintained monthly regression simulations for both Waku and Nim-libp2p, helping us to detect several issues and ensure that future versions do not introduce new ones. Successfully simulated and obtained results for all Waku protocols, relaying feedback to the team.","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-6","p":1},{"i":44,"t":"More testing and simulations for Codex and Nomos Develop useful tools for all IFT portfolio projects – e.g. a Log Parser tool and data dashboard","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-6","p":1},{"i":46,"t":"Several IFT portfolio projects use the Nim ecosystem for its efficiency. The Nim Service Unit is responsible for the development and maintenance of Nim tooling.","s":"Nim","u":"/rlog/2024-recap","h":"#nim","p":1},{"i":48,"t":"Released Nim-libp2p (v1.7.1, v1.7.0, v1.6.0, v1.5.0, v1.4.0, v1.3.0, v1.2.0) Introduced SAT solver to the Nimble package manager that significantly improves dependency resolution Nim VSCode Extension Stabilized Nim Language Server","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-7","p":1},{"i":50,"t":"Vac's Smart Contracts Service Unit ensures the smart contracts deployed across the various IFT portfolio projects are secure, robust, and aligned with project requirements.","s":"Smart Contracts (SC)","u":"/rlog/2024-recap","h":"#smart-contracts-sc","p":1},{"i":52,"t":"Deployed the SNT staking protocol testnet following Status's governance vote to develop SNT staking and Status Network Wrote specifications for Codex's architectural components and Status's staking contracts Delivered several learn-up sessions on a variety of topics for IFT contributors, including: Stealth addresses Tokenized vaults Rental NFTs Merkle trees Account abstraction EVM deep dive","s":"Highlights","u":"/rlog/2024-recap","h":"#highlights-8","p":1},{"i":54,"t":"Deploy the SNT staking protocol on the Status Network testnet Encourage community security audits via contests Provide smart contract consultation services for IFT portfolio products Engage in more learn-up sessions to promote org-wide knowledge sharing.","s":"Looking forward","u":"/rlog/2024-recap","h":"#looking-forward-7","p":1},{"i":56,"t":"This year has seen Vac involved with many research, development, and testing undertakings in support of IFT portfolio projects. The digital public goods that emerge from our efforts not only support the organization itself but are open and free to use by any project that would benefit. As we move into 2025, we aim to nurture a stronger RFC culture across the IFT to encourage greater collaboration and knowledge sharing among portfolio projects. Our goal is to serve as an internal conduit of expertise within the organization, supported by a strong RFC culture, maintaining a repository of internal knowledge creation, and identifying and facilitating IFT project synergies. Such an approach should lead to greater efficiencies across the organization. We also aim to establish a diverse research community around Vac, and our efforts in this regard are already underway. In the final quarter of 2024, Vac stepped up its collaboration with the libp2p community and made a concerted effort to engage the community on the Vac forum. In 2025, we aim to continue working closely with those communities to which we already have ties, such as the libp2p, Ethereum, and Nim ecosystems. We look forward to continuing our journey with you! Follow Vac on X, join us in the Vac Discord, or take part in the discussions on the Vac forum to stay up to date with our research and development progress.","s":"Heading into 2025","u":"/rlog/2024-recap","h":"#heading-into-2025","p":1},{"i":59,"t":"What is privacy-protecting infrastructure? Why do we need it and how we can build it? We'll look at Waku, the communication layer for Web3. We'll see how it uses ZKPs to incentivize and protect the Waku network. We'll also look at Zerokit, a library that makes it easier to use ZKPs in different environments. After reading this, I hope you'll better understand the importance of privacy-protecting infrastructure and how we can build it. This write-up is based on a talk given at DevCon 6 in Bogota, a video can be found here","s":"Building Privacy-Protecting Infrastructure","u":"/rlog/building-privacy-protecting-infrastructure","h":"","p":58},{"i":61,"t":"In this write-up, we are going to talk about building privacy-protecting infrastructure. What is it, why do we need it and how can we build it? We'll look at Waku, the communication layer for Web3. We'll look at how we are using Zero Knowledge (ZK) technology to incentivize and protect the Waku network. We'll also look at Zerokit, a library we are writing to make ZKP easier to use in different environments. At the end of this write-up, I hope you'll come away with an understanding of the importance of privacy-protecting infrastructure and how we can build it.","s":"Intro","u":"/rlog/building-privacy-protecting-infrastructure","h":"#intro","p":58},{"i":63,"t":"First, briefly about Vac. We build public good protocols for the decentralized web, with a focus on privacy and communication. We do applied research based on which we build protocols, libraries and publications. We are also the custodians of protocols that reflect a set of principles. It has its origins in the Status app and trying to improve the underlying protocols and infrastructure. We build Waku, among other things.","s":"About","u":"/rlog/building-privacy-protecting-infrastructure","h":"#about","p":58},{"i":65,"t":"Privacy is the power to selectively reveal yourself. It is a requirement for freedom and self-determination. Just like you need decentralization in order to get censorship-resistance, you need privacy to enable freedom of expression. To build applications that are decentralized and privacy-protecting, you need the base layer, the infrastructure itself, to have those properties. We see this a lot. It is easier to make trade-offs at the application layer than doing them at the base layer. You can build custodial solutions on top of a decentralized and non-custodial network where participants control their own keys, but you can't do the opposite. If you think about it, buildings can be seen as a form of privacy-protecting infrastructure. It is completely normal and obvious in many ways, but when it comes to the digital realm our mental models and way of speaking about it hasn't caught up yet for most people. I'm not going too much more into the need for privacy or what happens when you don't have it, but suffice to say it is an important property for any open society. When we have conversations, true peer-to-peer offline conversations, we can talk privately. If we use cash to buy things we can do commerce privately. On the Internet, great as it is, there are a lot of forces that makes this natural state of things not the default. Big Tech has turned users into a commodity, a product, and monetized user's attention for advertising. To optimize for your attention they need to surveil your habits and activities, and hence breach your privacy. As opposed to more old-fashioned models, where someone is buying a useful service from a company and the incentives are more aligned. We need to build credibly neutral infrastructure that protects your privacy at the base layer, in order to truly enable applications that are censorship-resistant and encourage meaningful freedom of expression.","s":"Why build privacy-protecting infrastructure?","u":"/rlog/building-privacy-protecting-infrastructure","h":"#why-build-privacy-protecting-infrastructure","p":58},{"i":67,"t":"Infrastructure is what lies underneath. Many ways of looking at this but I'll keep it simple as per the original Web3 vision. You had Ethereum for compute/consensus, Swarm for storage, and Whisper for messaging. Waku has taken over the mantle from Whisper and is a lot more usable today than Whisper ever was, for many reasons. On the privacy-front, we see how Ethereum is struggling. It is a big UX problem, especially when you try to add privacy back \"on top\". It takes a lot of effort and it is easier to censor. We see this with recent action around Tornado Cash. Compare this with something like Zcash or Monero, where privacy is there by default. There are also problems when it comes to the p2p networking side of things, for example with Ethereum validator privacy and hostile actors and jurisdictions. If someone can easily find out where a certain validator is physically located, that's a problem in many parts of the world. Being able to have stronger privacy-protection guarantees would be very useful for high-value targets. This doesn't begin to touch on the so called \"dapps\" that make a lot of sacrifices in how they function, from the way domains work, to how websites are hosted and the reliance on centralized services for communication. We see this time and time again, where centralized, single points of failure systems work for a while, but then eventually fail. In many cases an individual user might not care enough though, and for platforms the lure to take shortcuts is strong. That is why it is important to be principled, but also pragmatic in terms of the trade-offs that you allow on top. We'll touch more on this in the design goals around modularity that Waku has.","s":"Web3 infrastructure","u":"/rlog/building-privacy-protecting-infrastructure","h":"#web3-infrastructure","p":58},{"i":69,"t":"ZKPs are a wonderful new tool. Just like smart contracts enables programmable money, ZKPs allow us to express fundamentally new things. In line with the great tradition of trust-minimization, we can prove statement while revealing the absolute minimum information necessary. This fits the definition of privacy, the power to selectively reveal yourself, perfectly. I'm sure I don't need to tell anyone reading this but this is truly revolutionary. The technology is advancing extremely fast and often it is our imagination that is the limit.","s":"ZK for privacy-protecting infrastructure","u":"/rlog/building-privacy-protecting-infrastructure","h":"#zk-for-privacy-protecting-infrastructure","p":58},{"i":71,"t":"What is Waku? It is a set of modular protocols for p2p communication. It has a focus on privacy, security and being able to run anywhere. It is the spiritual success to Whisper. By modular we mean that you can pick and choose protocols and how you use them depending on constraints and trade-offs. For example, bandwidth usage vs privacy. It is designed to work in resource restricted environments, such as mobile phones and in web browsers. It is important that infrastructure meets users where they are and supports their real-world use cases. Just like you don't need your own army and a castle to have your own private bathroom, you shouldn't need to have a powerful always-on node to get reasonable privacy and censorship-resistance. We might call this self-sovereignty.","s":"Waku","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku","p":58},{"i":73,"t":"One way of looking at Waku is as an open service network. There are nodes with varying degrees of capabilities and requirements. For example when it comes to bandwidth usage, storage, uptime, privacy requirements, latency requirements, and connectivity restrictions. We have a concept of adaptive nodes that can run a variety of protocols. A node operator can choose which protocols they want to run. Naturally, there'll be some nodes that do more consumption and other nodes that do more provisioning. This gives rise to the idea of a service network, where services are provided for and consumed.","s":"Waku - adaptive nodes","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku---adaptive-nodes","p":58},{"i":75,"t":"There are many protocols that interact. Waku Relay protocol is based on libp2p GossipSub for p2p messaging. We have filter for bandwidth-restricted nodes to only receive subset of messages. Lightpush for nodes with short connection windows to push messages into network. Store for nodes that want to retrieve historical messages. On the payload layer, we provide support for Noise handshakes/key-exchanges. This means that as a developers, you can get end-to-end encryption and expected guarantees out of the box. We have support for setting up a secure channel from scratch, and all of this paves the way for providing Signal's Double Ratchet at the protocol level much easier. We also have experimental support for multi-device usage. Similar features have existed in for example the Status app for a while, but with this we make it easier for any platform using Waku to use it. There are other protocols too, related to peer discovery, topic usage, etc. See specs for more details.","s":"Waku - protocol interactions","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku---protocol-interactions","p":58},{"i":77,"t":"For the Waku network, there are a few problems. For example, when it comes to network spam and incentivizing service nodes. We want to address these while keeping privacy-guarantees of the base layer. I'm going to go into both of these. The spam problem arises on the gossip layer when anyone can overwhelm the network with messages. The service incentivization is a problem when nodes don't directly benefit from the provisioning of a certain service. This can happen if they are not using the protocol directly themselves as part of normal operation, or if they aren't socially inclined to provide a certain service. This depends a lot on how an individual platform decides to use the network.","s":"Waku - Network","u":"/rlog/building-privacy-protecting-infrastructure","h":"#waku---network","p":58},{"i":79,"t":"Since the p2p relay network is open to anyone, there is a problem with spam. If we look at existing solutions for dealing with spam in traditional messaging systems, a lot of entities like Google, Facebook, Twitter, Telegram, Discord use phone number verification. While this is largely sybil-resistant, it is centralized and not private at all. Historically, Whisper used PoW which isn't good for heterogenerous networks. Peer scoring is open to sybil attacks and doesn't directly address spam protection in an anonymous p2p network. The key idea here is to use RLN for private economic spam protection using zkSNARKs. I'm not going to go into too much detail of RLN here. If you are interested, I gave a talk in Amsterdam at Devconnect about this. We have some write-ups on RLN here by Sanaz who has been pushing a lot of this from our side. There's also another talk at Devcon by Tyler going into RLN in more detail. Finally, here's the RLN spec. I'll briefly go over what it is, the interface and circuit and then talk about how it is used in Waku.","s":"Dealing with network spam and RLN Relay","u":"/rlog/building-privacy-protecting-infrastructure","h":"#dealing-with-network-spam-and-rln-relay","p":58},{"i":81,"t":"RLN stands for Rate Limiting Nullifier. It is an anonyomous rate limiting mechanism based on zkSNARKs. By rate limiting we mean you can only send N messages in a given period. By anonymity we mean that you can't link message to a publisher. We can think of it as a voting booth, where you are only allowed to vote once every election. It can be used for spam protection in p2p messaging systems, and also rate limiting in general, such as for a decentralized captcha. There are three parts to it. You register somewhere, then you can signal and finally there's a verification/slashing phase. You put some capital at risk, either economic or social, and if you double signal you get slashed.","s":"RLN - Overview and Flow","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln---overview-and-flow","p":58},{"i":83,"t":"Here's what the private and public inputs to the circuit look like. The identity secret is generated locally, and we create an identity commitment that is inserted into a Merkle tree. We then use Merkle proofs to prove membership. Registered member can only signal once for a given epoch or external nullifier, for example every ten seconds in Unix time. RLN identifer is for a specific RLN app. We also see what the circuit output looks like. This is calculated locally. y is a share of the secret equation, and the (internal) nullifier acts as a unique fingerprint for a given app/user/epoch combination. How do we calculate y and the internal nullifier? // Private input signal input identity_secret; signal input path_elements[n_levels][1]; signal input identity_path_index[n_levels]; // Public input signal input x; // signal_hash signal input epoch; // external_nullifier signal input rln_identifier; // Circuit output signal output y; signal output root; signal output nullifier;","s":"RLN - Circuit","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln---circuit","p":58},{"i":85,"t":"This is done using Shamir's secret sharing. Shamir’s secret sharing is based on idea of splitting a secret into shares. This is how we enable slashing of funds. In this case, we have two shares. If a given identity a0 signals twice in epoch/external nullifier, a1 is the same. For a given RLN app, internal_nullifier then stays the same. x is signal hash which is different, and y is public, so we can reconstruct identity_secret. With the identity secret revealed, this gives access to e.g. financial stake. a_0 = identity_secret // secret S a_1 = poseidonHash([a0, external_nullifier]) y = a_0 + x * a_1 internal_nullifier = poseidonHash([a_1, rln_identifier])","s":"RLN - Shamir's secret sharing","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln---shamirs-secret-sharing","p":58},{"i":87,"t":"This is how RLN is used with Relay/GossipSub protocol. A node registers and locks up funds, and after that it can send messages. It publishes a message containing the Zero Knowledge proof and some other details. Each relayer node listens to the membership contract for new members, and it also keeps track of relevant metadata and merkle tree. Metadata is needed to be able to detect double signaling and perform slashing. Before forwarding a message, it does some verification checks to ensure there are no duplicate messages, ZKP is valid and no double signaling has occured. It is worth noting that this can be combined with peer scoring, for example for duplicate messages or invalid ZK proofs. In line of Waku's goals of modularity, RLN Relay is applied on a specific subset of pubsub and content topics. You can think of it as an extra secure channel.","s":"RLN Relay","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln-relay","p":58},{"i":89,"t":"Where are we with RLN Relay deployment? We've recently launched our second testnet. This is using RLN Relay with a smart contract on Goerli. It integrates with our example p2p chat application, and it does so through three different clients, nwaku, go-waku and js-waku for browsers. This is our first p2p cross-client testnet for RLN Relay. Here's a video that shows a user registering in a browser, signaling through JS-Waku. It then gets relayed to a nwaku node, that verifies the proof. The second video shows what happens in the spam case. when more than one message is sent in a given epoch, it detects it as spam and discards it. Slashing hasn't been implemented fully yet in the client and is a work in progress. If you are curious and want to participate, you can join the effort on our Vac Discord. We also have tutorials setup for all clients so you can play around with it. As part of this, and to make it work in multiple different environments, we've also been developing a new library called Zerokit. I'll talk about this a bit later.","s":"RLN Relay cross-client testnet","u":"/rlog/building-privacy-protecting-infrastructure","h":"#rln-relay-cross-client-testnet","p":58},{"i":91,"t":"Going back to the service network idea, let's talk about service credentials. The idea behind service credentials and private settlement is to enable two actors to pay for and provide services without compromising their privacy. We do not want the payment to create a direct public link between the service provider and requester. Recall the Waku service network illustration with adaptive nodes that choose which protocols they want to run. Many of these protocols aren't very heavy and just work by default. For example the relay protocol is enabled by default. Other protocols are much heavier to provide, such as storing historical messages. It is desirable to have additional incentives for this, especially for platforms that aren't community-based where some level of altruism can be assumed (e.g. Status Communities, or WalletConnect cloud infrastructure). You have a node Alice that is often offline and wants to consume historical messages on some specific content topics. You have another node Bob that runs a server at home where they store historical messages for the last several weeks. Bob is happy to provide this service for free because he's excited about running privacy-preserving infrastructure and he's using it himself, but his node is getting overwhelmed by freeloaders and he feels like he should be paid something for continuing to provide this service. Alice deposits some funds in a smart contract which registers it in a tree, similar to certain other private settlement mechanisms. A fee is taken or burned. In exchange, she gets a set of tokens or service credentials. When she wants to do a query with some criteria, she sends this to Bob. Bob responds with size of response, cost, and receiver address. Alice then sends a proof of delegation of a service token as a payment. Bob verifies the proof and resolves the query. The end result is that Alice has consumed some service from Bob, and Bob has received payment for this. There's no direct transaction link between Alice and Bob, and gas fees can be minimized by extending the period before settling on chain. This can be complemented with altruistic service provisioning, for example by splitting the peer pool into two slots, or only providing a few cheap queries for free. The service provisioning is general, and can be generalized for any kind of request/response service provisoning that we want to keep private. This isn't a perfect solution, but it is an incremental improvement on top of the status quo. It can be augmented with more advanced techniques such as better non-repudiable node reputation, proof of correct service provisioning, etc. We are currently in the raw spec / proof of concept stage of this. We expect to launch a testnet of this later this year or early next year.","s":"Private settlement / Service credentials","u":"/rlog/building-privacy-protecting-infrastructure","h":"#private-settlement--service-credentials","p":58},{"i":93,"t":"Zerokit is a set of Zero Knowledge modules, written in Rust and designed to be used in many different environments. The initial goal is to get the best of both worlds with Circom/Solidity/JS and Rust/ZK ecosystem. This enables people to leverage Circom-based constructs from non-JS environments. For the RLN module, it is using Circom circuits via ark-circom and Rust for scaffolding. It exposes a C FFI API that can be used through other system programming environments, like Nim and Go. It also exposes an experimental WASM API that can be used through web browsers. Waku is p2p infrastructure running in many different environments, such as Nim/JS/Go/Rust, so this a requirement for us. Circom and JS strengths are access to Dapp developers, tooling, generating verification code, circuits etc. Rust strengths is that it is systems-based and easy to interface with other language runtime such as Nim, Go, Rust, C. It also gives access to other Rust ZK ecosystems such as arkworks. This opens door for using other constructs, such as Halo2. This becomes especially relevant for constructs where you don't want to do a trusted setup or where circuits are more complex/custom and performance requirements are higher. In general with Zerokit, we want to make it easy to build and use ZKP in a multitude of environments, such as mobile phones and web browsers. Currently it is too complex to write privacy-protecting infrastructure with ZKPs considering all the languages and tools you have to learn, from JS, Solidity and Circom to Rust, WASM and FFI. And that isn't even touching on things like secure key storage or mobile dev. Luckily more and more projects are working on this, including writing DSLs etc. It'd also be exciting if we can make a useful toolstack for JS-less ZK dev to reduce cognitive overhead, similar to what we have with something like Foundry.","s":"Zerokit","u":"/rlog/building-privacy-protecting-infrastructure","h":"#zerokit","p":58},{"i":95,"t":"I also want to mention a few other things we are doing. One thing is protocol specifications. We think this is very important for p2p infra, and we see a lot of other projects that claim to do it p2p infrastructure but they aren't clear about guarantees or how stable something is. That makes it hard to have multiple implementations, to collaborate across different projects, and to analyze things objectively. Related to that is publishing papers. We've put out three so far, related to Waku and RLN-Relay. This makes it easier to interface with academia. There's a lot of good researchers out there and we want to build a better bridge between academia and industry. Another thing is network privacy. Waku is modular with respect to privacy guarantees, and there are a lot of knobs to turn here depending on specific deployments. For example, if you are running the full relay protocol you currently have much stronger receiver anonymity than if you are running filter protocol from a bandwidth or connectivity-restricted node. We aim to make this pluggable depending on user needs. E.g. mixnets such as Nym come with some trade-offs but are a useful tool in the arsenal. A good mental model to keep in mind is the anonymity trilemma, where you can only pick 2/3 out of low latency, low bandwidth usage and strong anonymity. We are currently exploring Dandelion-like additions to the relay/gossip protocol, which would provide for stronger sender anonymity, especially in a multi-node/botnet attacker model. As part of this we are looking into different parameters choices and general possibilities for lower latency usage. This could make it more amenable for latency sensitive environments, such as validator privacy, for specific threat models. The general theme here is we want to be rigorous with the guarantees we provide, under what conditions and for what threat models. Another thing mentioned earlier is Noise payload encryption, and specifically things like allowing for pairing different devices with e.g. QR codes. This makes it easier for developers to provide secure messaging in many realistic scenarios in a multi-device world.","s":"Other research","u":"/rlog/building-privacy-protecting-infrastructure","h":"#other-research","p":58},{"i":97,"t":"We've gone over what privacy-protecting infrastructure is, why we want it and how we can build it. We've seen how ZK is a fundamental building block for this. We've looked at Waku, the communication layer for Web3, and how it uses Zero Knowledge proofs to stay private and function better. We've also looked at Zerokit and how we can make it easier to do ZKP in different environments. Finally we also looked at some other research we've been doing. All of the things mentioned in this article, and more, is available as write-ups, specs, or discussions on our forum or Github. If you find any of this exciting to work on, feel free to reach out on our Discord. We are also hiring, and we have started expanding into other privacy infrastructure tech like private and provable computation with ZK-WASM.","s":"Summary","u":"/rlog/building-privacy-protecting-infrastructure","h":"#summary","p":58},{"i":99,"t":"Device pairing and secure message exchange using Waku and noise protocol. As the world becomes increasingly connected through the internet, the need for secure and reliable communication becomes paramount. In this article it is described how the Noise protocol can be used as a key-exchange mechanism for Waku. Recently, this feature was introduced in js-waku and go-waku, providing a simple API for developers to implement secure communication protocols using the Noise Protocol framework. These open-source libraries provide a solid foundation for building secure and decentralized applications that prioritize data privacy and security. This functionality is designed to be simple and easy to use, even for developers who are not experts in cryptography. The library offers a clear and concise API that abstracts away the complexity of the Noise Protocol framework and provides an straightforward interface for developers to use. Using this, developers can effortlessly implement secure communication protocols on top of their JavaScript and Go applications, without having to worry about the low-level details of cryptography. One of the key benefits of using Noise is that it provides end-to-end encryption, which means that the communication between two parties is encrypted from start to finish. This is essential for ensuring the security and privacy of sensitive information","s":"Device Pairing in Js-waku and Go-waku","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"","p":98},{"i":101,"t":"In today's digital world, device pairing has become an integral part of our lives. Whether it's connecting our smartphones with other computers or web applications, the need for secure device pairing has become more crucial than ever. With the increasing threat of cyber-attacks and data breaches, it's essential to implement secure protocols for device pairing to ensure data privacy and prevent unauthorized access. To demonstrate how device pairing can be achieved using Waku and Noise, we have examples available at https://examples.waku.org/noise-js/. You can try pairing different devices, such as mobile and desktop, via a web application. This can be done by scanning a QR code or opening a URL that contains the necessary data for a secure handshake. The process works as follows: Actors: Alice the initiator Bob the responder The first step in achieving secure device pairing using Noise and Waku is for Bob generate the pairing information which could be transmitted out-of-band. For this, Bob opens https://examples.waku.org/noise-js/ and a QR code is generated, containing the data required to do the handshake. This pairing QR code is timeboxed, meaning that after 2 minutes, it will become invalid and a new QR code must be generated Alice scans the QR code using a mobile phone. This will open the app with the QR code parameters initiating the handshake process which is described in WAKU2-DEVICE-PAIRING. These messages are exchanged between two devices over Waku to establish a secure connection. The handshake messages consist of three main parts: the initiator's message, the responder's message, and the final message, which are exchanged to establish a secure connection. While using js-noise, the developer is abstracted of this process, since the messaging happens automatically depending on the actions performed by the actors in the pairing process. Both Alice and Bob will be asked to verify each other's identity. This is done by confirming if an 8-digits authorization code match in both devices. If both actors confirm that the authorization code is valid, the handshake concludes succesfully Alice and Bob receive a set of shared keys that can be used to start exchanging encrypted messages. The shared secret keys generated during the handshake process are used to encrypt and decrypt messages sent between the devices. This ensures that the messages exchanged between the devices are secure and cannot be intercepted or modified by an attacker. The above example demonstrates device pairing using js-waku. Additionally, You can also try building and experimenting with other noise implementations like nwaku, or go-waku, with an example available at https://github.com/waku-org/go-waku/tree/master/examples/noise in which the same flow described before is done with Bob (the receiver) using go-waku instead of js-waku.","s":"Device Pairing","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"#device-pairing","p":98},{"i":103,"t":"With its easy to use API built on top of the Noise Protocol framework and the LibP2P networking stack, if you are a developer looking to implement secure messaging in their applications that are both decentralized and censorship resistant, Waku is definitely an excellent choice worth checking out! Waku is also Open source with a MIT and APACHEv2 licenses, which means that developers are encouraged to contribute code, report bugs, and suggest improvements to make it even better. Don't hesitate to try the live example at https://examples.waku.org/noise-js and build your own webapp using https://github.com/waku-org/js-noise, https://github.com/waku-org/js-waku and https://github.com/waku-org/go-waku. This will give you a hands-on experience of implementing secure communication protocols using the Noise Protocol framework in a practical setting. Happy coding!","s":"Conclusion","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"#conclusion","p":98},{"i":105,"t":"Noise handshakes as key-exchange mechanism for Waku Noise Protocols for Waku Payload Encryption Session Management for Waku Noise Device pairing and secure transfers with Noise go-waku Noise's example js-waku Noise's example js-noise go-noise","s":"References","u":"/rlog/device-pairing-in-js-waku-and-go-waku","h":"#references","p":98},{"i":107,"t":"A look at typical ethical shortfalls in the global surveillance tech industry. This is an opinion piece by pseudonymous contributor, circe.","s":"Opinion: Pseudo-ethics in the Surveillance Tech Industry","u":"/rlog/ethics-surveillance-tech","h":"","p":106},{"i":109,"t":"The Vac team aims to provide a public good in the form of freely available, open source tools and protocols for decentralized communication. As such, we value our independence and the usefulness of our protocols for a wide range of applications. At the same time, we realize that all technical development, including ours, has a moral component. As a diverse team we are guided by a shared devotion to the principles of human rights and liberty. This explains why we place such a high premium on security, censorship-resistance and privacy - a stance we share with the wider Status Network. The post below takes a different approach from our usual more technical analyses, by starting to peel back the curtain on the ethical shortfalls of the global surveillance tech industry.","s":"Preface","u":"/rlog/ethics-surveillance-tech","h":"#preface","p":106},{"i":111,"t":"Apple's announcement of their lawsuit against Israel's NSO Group marks the latest in a series of recent setbacks for the surveillance tech company. In early November, the United States blacklisted the firm, citing concerns about the use of their spyware by foreign governments targeting civilians such as \"journalists, businesspeople, activists\" and more. The company is already embroiled in a lawsuit with Whatsapp over their exploit of the chat app's video calling service to install malware on target devices. NSO Group's most infamous product, Pegasus, operates as a hidden exploit installed on victims' mobile phones, sometimes without even requiring as much as an unguarded click on a malicious link. It has the potential to lay bare, and report to its owners, everything within the reach of the infected device. For most people this amounts to a significant portion of their private lives and thoughts. Pegasus can read your private messages (even encrypted), collect your passwords, record calls, track your location and access your device's microphone and camera. No activity or application on an infected phone would be hidden. The latest controversies are perhaps less because of the novelty of the revelations - the existence of Pegasus has been known to civil activists since at least 2016. Rather, the public was reminded again of the potential scope of surveillance tech in the indiscriminate use of Pegasus on private citizens. This has far-reaching implications for human freedoms worldwide. Earlier this year, a leaked list of over 50,000 targets, or possible targets, of Pegasus included the phone numbers of human rights advocates, independent journalists, lawyers and political activists. This should have come as no surprise. The type of autocratically inclined agents, and governments, who would venture to buy and use such invasive cyber-arms often target those they find politically inconvenient. Pegasus, and similar technologies, simply extend the reach and capacity of such individuals and governments - no border or distance, no political rank or social advantage, no sanctity of profession or regard for dignity, provide any indemnity from becoming a victim. Your best hope is to remain uninteresting enough to escape consideration. The NSO Group has, of course, denied allegations of culpability and questions the authenticity of the list. At this stage, the latter is almost beside the point: Amnesty International's cybersecurity team, Security Lab, did find forensic evidence of Pegasus on the phones of several volunteers whose numbers appeared on the original list, including those of journalists and human rights activists. (Security Lab has since opened up their infection finding tool to the public.) French intelligence has similarly inspected and confirmed infection of at least three devices belonging to journalists. The phones of several people who were close to the Saudi-American journalist, Jamal Khashoggi, were confirmed hacked both before and after Khashoggi's brutal murder at the Saudi embassy in Istanbul in 2018. More reports of confirmed Pegasus hacks are still published with some regularity. It is now an open secret that many authoritarian governments have bought Pegasus. It's not difficult to extrapolate from existing reports and such clients' track records what the potential injuries to human freedoms are that they can inflict with access to such a powerful cyberweapon.","s":"Spotlight on an industry","u":"/rlog/ethics-surveillance-tech","h":"#spotlight-on-an-industry","p":106},{"i":113,"t":"NSO's response to the allegations follows a textbook approach of avoiding earnest ethical introspection on the manufacturing, and selling, of cyber-arms. Firstly, shift ethical responsibility to a predetermined process, a list of checkboxes of your own making. The Group, for example, claims to sell only to \"vetted governments\", following a classification process of which they have now published some procedural details but no tangible criteria. The next step is to reaffirm continuously, and repetitively, your dedication to the legal combat against crime, \"legitimate law enforcement agencies\" (note the almost tautological phrasing), adherence to international arms trade laws, compliance clauses in customer contracts, etc. Thirdly, having been absolved of any moral suspicions that might exist about product and process, from conception to engineering to trade, distance yourself from the consequences of its use in the world. \"NSO does not operate its technology, does not collect, nor possesses, nor has any access to any kind of data of its customers.\" It is interesting that directly after this statement they claim with contradictory confidence that their \"technology was not associated in any way with the heinous murder of Jamal Khashoggi\". The unapologetic tone seems hardly appropriate when the same document confirms that the Group had to shut down customers' systems due to \"confirmed misuse\" and have had to do so \"multiple times\" in the past. Given all this, the response manages to evade any serious interrogation of the \"vetting\" process itself, which forced the company to reject \"approximately 15% of potential new opportunities for Pegasus\" in one year. Courageous. We have heard this all before. There exists a multi-billion dollar industry of private companies and engineering firms thriving on proceeds from selling surveillance tools and cyber-arms to dubious agencies and foreign governments. In turn, the most power-hungry and oppressive regimes often rely on such technological innovations - for which they lack the in-country engineering expertise - to maintain control, suppress uprisings, intimidate opposing journalists, and track their citizens. It's a lucrative business opportunity, and resourceful companies have sprung up everywhere to supply this demand, often in countries where citizens, including employees of the company, would be horrified if they were similarly subject to the oppressions of their own products. When, in 2014, Italy's HackingTeam were pulsed by the United Nations about their (then alleged) selling of spyware to Sudan, which would have been a contravention of the UN's weapon export ban, they simply replied that their product was not controlled as a weapon and therefore not subject to such scrutiny. They remained within their legal bounds, technically. Furthermore, they similarly shifted ethical responsibility to external standards of legitimacy, claiming their \"software is not sold to governments that are blacklisted by the EU, the US, NATO, and similar international organizations\". When the company themselves were hacked in 2015, revelations (confirmations, that is) of widespread misuse by repressive governments were damaging enough to force them to disappear and rebrand as Memento Labs. Their website boasts an impressive list of statutes, regulations, procedures, export controls and legal frameworks, all of which the rebranded hackers proudly comply with. Surely no further ethical scrutiny is necessary?","s":"A typical response","u":"/rlog/ethics-surveillance-tech","h":"#a-typical-response","p":106},{"i":116,"t":"Such recourse to the legality of your action as ethical justification is moot for several reasons. The first is glaringly obvious - our laws are ill-equipped to address the implications of modern technology. Legal systems are a cumbersome inheritance built over generations. This is especially true of the statutes and regulations governing international trade, behind which these companies so often hide. Our best legal systems are trailing miles behind the technology for which we seek guidelines. Legislators are still struggling to make sense of technologies like face recognition, the repercussions of smart devices acting \"on their own\" and biases in algorithms. To claim you are performing ethical due diligence by resorting to an outdated and incomplete system of legal codes is disingenuous.","s":"The law is trailing behind","u":"/rlog/ethics-surveillance-tech","h":"#the-law-is-trailing-behind","p":106},{"i":118,"t":"The second reason is more central to my argument, and an important flaw in these sleight of hand justifications appearing from time to time in the media. Ethics can in no way be confused as synonymous with legality or legitimacy. These are incommensurable concepts. In an ideal world, of course, the law is meant to track the minimum standards of ethical conduct in a society. Laws are often drafted exactly from some ethical, and practical, impulse to minimize harmful conduct and provide for corrective and punitive measures where transgressions do occur. The law, however, has a much narrower scope than ethics. It can be just or unjust. In fact, it is in need of ethics to constantly reform. Ethics and values are born out of collective self-reflection. It develops in our conversation with ourselves and others about the type of society we strive for. As such, an ethical worldview summarizes our deepest intuitions about how we should live and measure our impact on the world. For this reason, ethics is primarily enforced by social and internal pressures, not legal boundaries - our desire to do what ought to be done, however we define that. Ethics is therefore a much grander scheme than global legal systems and the diplomatic frameworks that grants legitimacy to governments. These are but one limited outflow of the human aspiration to form societies in accordance with our ideologies and ethics.","s":"The law depends on ethics","u":"/rlog/ethics-surveillance-tech","h":"#the-law-depends-on-ethics","p":106},{"i":120,"t":"Of course, the cyber-arms trade has a favorite recourse, international law, which is even more limited. Since such products are seldomly sold to governments and agencies within the country of production, it enables a further distancing from consequences. Many private surveillance companies are based in fairly liberal societies with (seemingly) strict emphases on human rights in their domestic laws. International laws are much more complicated - for opportunists a synonym for \"more grey areas in which to hide\". Company conduct can now be governed, and excused, by a system that follows the whims of autocrats with exploitative intent and vastly different ethical conceptions from the company's purported aims. International law, and the ways it is most often enforced by way of, say, UN-backed sanctions, have long been shaped by the compromises of international diplomacy. To be blunt: these laws are weak and subject to exactly the sort of narrow interests behind which mercenaries have always hidden. The surveillance tech industry is no exception.","s":"International law is vague and exploitable","u":"/rlog/ethics-surveillance-tech","h":"#international-law-is-vague-and-exploitable","p":106},{"i":122,"t":"My point is simple: selling cyber-arms with the potential to become vast tools of oppression to governments and bodies with blatant histories of human rights violations, and all but the publicly announced intention to continue operating in this way, is categorically unconscionable. This seems obvious no matter what ethics system you argue from, provided it harbors any consideration for human dignity and freedom. It is a sign of poor moral discourse that such recourses to law and legitimacy are often considered synonymous with ethical justification. \"I have acted within the bounds of law\", \"We supply only to legitimate law enforcement agencies\", etc. are no substitutes. Ethical conduct requires an honest evaluation of an action against some conception of \"the good\", however you define that. Too often the surveillance tech industry precisely sidesteps this question, both in internal processes and external rationalisations to a concerned public. John Locke, he of the life-liberty-and-property, articulated the idea that government exists solely through the consent of the governed. Towards the end of the 17th century, he wrote in his Second Treatise on Civil Government, \"[w]henever legislators endeavor to take away, and destroy the property of the people, or to reduce them to slavery under arbitrary power, they put themselves in a state of war with the people, who are thereupon absolved from any further obedience\". The inference is straightforward and humanist in essence: legitimacy is not something that is conferred by governments and institutions. Rather, they derive their legitimacy from us, their citizens, holding them to standards of ethics and societal ideals. This legitimacy only remains in tact as long as this mandate is honored and continuously extended by a well-informed public. This is the principle of informed consent on which all reciprocal ethics is based. The surveillance tech industry may well have nothing more or less noble in mind than profit-making within legal bounds when developing and selling their products. However, when such companies are revealed again and again to have supplied tools of gross human rights violations to known human rights violators, they will do well to remember that ethics always precedes requirements of legality and legitimacy. It is a fallacy to take normative guidance from the concept of \"legitimacy\" if the concept itself depends on such normative guidelines for definition. Without examining the ethical standards by which institutions, governments, and laws, were created, no value-judgements about their legitimacy can be made. Hiding behind legal compliance as substitute for moral justification is not enough. Targets of increasingly invasive governmental snooping are too often chosen precisely to suppress the mechanisms from which the legitimacy of such governments flow - the consent of ordinary civilians. Free and fair elections, free speech, free media, freedom of thought are all at risk.","s":"Conclusion","u":"/rlog/ethics-surveillance-tech","h":"#conclusion","p":106},{"i":124,"t":"Status Principles Federal Register: Addition of Certain Entities to the Entity List forbiddenstories.org: The Pegasus Project theguardian.com: The Pegasus Project amnesty.org Forensic Methodology Report: How to catch NSO Group’s Pegasus Apple sues NSO Group to curb the abuse of state-sponsored spyware bbc.com: Who are the hackers who cracked the iPhone? bbc.com: Pegasus: Who are the alleged victims of spyware targeting? citizenlab.ca: Mapping Hacking Team’s “Untraceable” Spyware economist.com: Offering software for snooping to governments is a booming business Memento Labs Mobile Verification Toolkit to identify compromised devices NSO Group: Transparency and Responsibility Report 2021 reuters.com: WhatsApp sues Israel's NSO for allegedly helping spies hack phones around the world wired.com: Hacking Team Breach Shows a Global Spying Firm Run Amok","s":"References","u":"/rlog/ethics-surveillance-tech","h":"#references","p":106},{"i":126,"t":"A look at EIP-1459 and the benefits of DNS based discovery. Discovery in p2p networks is the process of how nodes find each other and specific resources they are looking for. Popular discovery protocols, such as Kademlia which utilizes a distributed hash table or DHT, are highly inefficient for resource restricted devices. These methods use short connection windows, and it is quite battery intensive to keep establishing connections. Additionally, we cannot expect a mobile phone for example to synchronize an entire DHT using cellular data. Another issue is how we do the initial bootstrapping. In other words, how does a client find its first node to then discover the rest of the network? In most applications, including Status right now, this is done with a static list of nodes that a client can connect to. In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable. How do we do this? EIP 1459: Node Discovery via DNS, which is one of the strategies we are using for discovering waku nodes. EIP-1459 is a DNS-based discovery protocol that stores merkle trees in DNS records which contain connection information for nodes. Waku is our fork of Whisper. Oskar recently wrote an entire post explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification here DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be. > dig dnsseed.bluematt.me +short 129.226.73.12 107.180.78.111 169.255.56.123 91.216.149.28 85.209.240.91 66.232.124.232 207.55.53.96 86.149.241.168 193.219.38.57 190.198.210.139 74.213.232.234 158.181.226.33 176.99.2.207 202.55.87.45 37.205.10.3 90.133.4.73 176.191.182.3 109.207.166.232 45.5.117.59 178.211.170.2 160.16.0.30 The above displays the result of querying on of these DNS seeds. All the nodes are stored as A records for the given domain name. This is quite a simple solution which Bitcoin almost soley relies on since removing the IRC bootstrapping method in v0.8.2. What makes this DNS based discovery useful? It allows us to have a mutable list of bootstrap nodes without needing to ship a new version of the client every time a list is mutated. It also allows for a more lightweight method of discovering nodes, something very important for resource restricted devices. Additionally, DNS provides us with a robust and scalable infrastructure. This is due to its hierarchical architecture. This hierarchical architecture also already makes it distributed such that the failure of one DNS server does not result in us no longer being able to resolve our name. As with every solution though, there is a trade-off. By storing the list in DNS name an adversary would simply need to censor the DNS records for a specific name. This would prevent any new client trying to join the network from being able to do so. One thing you notice when looking at EIP-1459 is that it is a lot more technically complex than Bitcoin's way of doing this. So if Bitcoin uses this simple method and has proven that it works, why did we need a new method? There are multiple reasons, but the main one is security. In the Bitcoin example, an attacker could create a new list and no one querying would be able to tell. This is however mitigated in EIP-1459 where we can verify the integrity of the entire returned list by storing an entire merkle tree in the DNS records. Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party. So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using merkle trees as follows: The root of the tree is stored in a TXT record, this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above. Additionally, there is also a root hash to a second tree called a link tree, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names. The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version. Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a TXT record. The branch record contains all the hashes of the branch's leafs. Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at EIP-778, or read a short blog post I wrote explaining them here. Below is the zone file taken from the EIP-1459, displaying how this looks in practice. ; name ttl class type content @ 60 IN TXT enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA C7HRFPF3BLGF3YR4DY5KX3SMBE 86900 IN TXT enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org JWXYDBPXYWG6FX3GMDIBFA6CJ4 86900 IN TXT enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24 2XS2367YHAXJFGLZHVAWLQD4ZY 86900 IN TXT enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA H4FHT4B454P6UXFD7JCYQ5PWDY 86900 IN TXT enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI MHTDO6TMUBRIA2XWG5LUDACK24 86900 IN TXT enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o All of this has already been introduced into go-ethereum with the pull request #20094, created by Felix Lange. There's a lot of tooling around it that already exists too which is really cool. So if your project is written in Golang and wants to use this, it's relatively simple! Additionally, here's a proof of concept that shows what this might look like with libp2p on github. I hope this was a helpful explainer into DNS based discovery, and shows EIP-1459's benefits over more traditional DNS-based discovery schemes.","s":"DNS Based Discovery","u":"/rlog/dns-based-discovery","h":"","p":125},{"i":128,"t":"In this post, we introduce a crucial data structure used throughout web3.","s":"Vac 101: Climbing Merkle Trees","u":"/rlog/climbing-merkle-trees","h":"","p":127},{"i":130,"t":"A large amount of data is swapped between users on a blockchain in the form of transactions. Over the entire life of a blockchain, the storage space required to maintain a copy of every transaction becomes untenable for most users. However, the integrity of a blockchain relies on a large pool of users that can validate the blockchain's history from its inception to its present state. The data representing the blockchain's state is compressed. This compression addresses the issue of scalability that would otherwise greatly restrict the pool of users. Data compression alone is not the end goal. As mentioned, it is essential for users to be able to validate the blockchain's history. The property of compression and validation was solved in Bitcoin by the use of Merkle trees. Merkle trees were introduced first by Ralph Merkle in his dissertation [1]. A Merkle tree is a data structure that compresses a digest of data to a constant size while still providing a method for proving membership of elements of the digest. A previous rlog[2] described how Merkle trees with their proof of membership could be used for lightweight clients for RLN.","s":"Introduction","u":"/rlog/climbing-merkle-trees","h":"#introduction","p":127},{"i":132,"t":"A tree is a special data structure that organizes nodes so that there is exactly one path between any two nodes. The trees that we consider can be arranged in layers with multiple nodes (children) merged into a single node (parent) in the preceding layer. A single node exists in the base layer; this special node is called the root node. The highest level of the tree consists of childless nodes called leaves. A binary tree has one additional property: each nonleaf node has exactly two children nodes. That is, we assume that nodes in a binary tree are either a parent node with two children or a leaf. As strange as it sounds, each child node has exactly one parental node. A binary tree with 2n2^n2n leaves consists of n+1n+1n+1 layers. Additionally, such a tree has 2n+1−12^{n+1}-12n+1−1 nodes.","s":"Tree structure","u":"/rlog/climbing-merkle-trees","h":"#tree-structure","p":127},{"i":134,"t":"A Merkle tree is a specialized tree in which each node contains the evaluation of a hash function. Merkle trees are usually taken to have a binary tree structure. As such, the presentation we provide in this section will be for binary trees.","s":"Merkle trees","u":"/rlog/climbing-merkle-trees","h":"#merkle-trees","p":127},{"i":136,"t":"In this section, we show how Merkle trees are constructed to compress a digest DDD. Suppose that the digest DDD consists of 2n2^n2n entries; we assume that the digest DDD has this many entries since a Merkle tree is a binary tree. Additionally, each digest can be padded to ensure that DDD has the desired number of entries. Each leaf of the Merkle tree contains the hash of a digest entry. Each parent node contains the hash of the concatenation of their child nodes. Through this iterative construction, we reach the root of the tree. The value contained in the root node is called the root hash. The root hash is a compressed representation of the digest DDD. Each node in the Merkle tree is computed by taking a hash. Since a binary tree with 2n2^n2n leaves has 2n+1−12^{n+1}-12n+1−1 nodes, then we need to evaluate 2n+1−12^{n+1}-12n+1−1 hashes to construct the Merkle tree.","s":"Construction","u":"/rlog/climbing-merkle-trees","h":"#construction","p":127},{"i":138,"t":"A large quantity of data can be compressed to a single hash value. A natural question to ask is: could a clever party find another digest that yields a Merkle tree with the same root hash? If possible, this would compromise the ledger since the blockchain's history could be altered. Fortunately, Merkle trees are quite secure. In fact, Merkle trees can be used to both bind and hide a digest. The Merkle tree is able to bind a digest with one of the properties of hash functions (see our previous Vac 101 [3] for information on hash functions). A hash function is collision resistant; it is infeasible for a malicious party to find two values share the same hash value. This collision resistance property, essentially, fixes the input to each leaf and into their parent, their parent's parent, and so on. In certain applications, it may be desirable for the digest of a Merkle tree to be kept confidential. This is achieved with the preimage resistant property of hash functions. A hash function is preimage resistant provided that it is difficult to reverse the hashing operation. It would be necessary for a malicious party to find preimages to each node starting from the root node to determine the original digest. Now, we see that Merkle trees are secured structures that are tamper resistant.","s":"Merkle tree intregrity","u":"/rlog/climbing-merkle-trees","h":"#merkle-tree-intregrity","p":127},{"i":140,"t":"An interesting and critical property of Merkle trees is their ability to prove that any piece of data is part of its digest. This can be done with logarithmic storage and logarithmic computation time. Suppose that we want to show that data ℓ\\ellℓ is part of the Merkle tree's digest. Additionally, suppose that hash\\mathsf{hash}hash is the hash function used to construct the tree. We assume that the hash function hash\\mathsf{hash}hash can be computed in constant-time for any input. Suppose that a prover provides data ℓ\\ellℓ to a verifier, and tells the verifier that ℓ\\ellℓ corresponds to the iiith leaf of the Merkle tree. For the verifier to be convinced that ℓ\\ellℓ is part of the digest, he needs to be able to construct the tree's root hash using hash\\mathsf{hash}hash, iii, ℓ\\ellℓ and some additional information from the prover. Specifically, the prover must provide the sibling hashes for each value that the verifier can compute. This enables the verifier to compute the parents of the siblings that the prover provides and the values that he was able to produce himself. The last of the computed parents is the root. The leaf index iii indicates whether a hash value provided by the prover is a left or right sibling. This is done by looking at the binary expansion of iii. The verifier can compute the leaf h0=hash(ℓ)h_0 = \\mathsf{hash}(\\ell)h0​=hash(ℓ). Next, using h0h_0h0​'s sibling, h0′h'_0h0′​, provided by the prover, the verifier can compute h1=hash(h0∥h0′)h_1 = \\mathsf{hash}(h_0 \\|h'_0)h1​=hash(h0​∥h0′​) or h1=hash(h0′∥h0)h_1 = \\mathsf{hash}(h'_0 \\| h_0)h1​=hash(h0′​∥h0​) depending on whether h0′h'_0h0′​ is a left or right sibling. This pathing continues until the verifier either successfully computes the root hash (in n+1n+1n+1 hashes) or fails to do so. The prover has to provide nnn sibling nodes for the proof of membership. There is a key detail that is essential for the proof of membership to be secure. The root hash has to be provided to the verifier prior to the selection of data ℓ\\ellℓ. Otherwise, the prover could generate a series of hash values (with the corresponding root hash) to forge a proof of membership. Capped proof of membership​ Polygon provides an implementation [4] of a shortened proof of membership with a slight modification. A specific layer of the Merkle tree is published instead of just the root hash. By doing this, a capped proof of membership is just the path from leaf to the published layer.","s":"Proof of membership","u":"/rlog/climbing-merkle-trees","h":"#proof-of-membership","p":127},{"i":142,"t":"Merkle trees can be extended in multiple ways. In this section, we explore a select few of these extensions.","s":"Extensions of Merkle trees","u":"/rlog/climbing-merkle-trees","h":"#extensions-of-merkle-trees","p":127},{"i":144,"t":"A sparse Merkle tree (SMT) is a special Merkle tree that can be used to represent digests with nonconsecutive entries. Specifically, each digest entry has a particular leaf index. For simplicity, we assume that the index value is computed by taking the hash of the entry. We note that this is a sorted SMT. Let nnn denote the number of bits that a hash value can possess. This means that our SMT can have at most 2n2^n2n leaves. An SMT is treated as a Merkle tree in which each entry is placed in the leaf corresponding to its hash value, and the other entries have a null\\mathsf{null}null marker inserted in. This means that we can prove membership in the way described. However, we can also prove nonmembership of an element by showing that null\\mathsf{null}null is located in the element's hash location. The crucial difference between a sorted and unsorted SMT is that the unsorted variant cannot be used to prove nonmembership. We can take advantage of the sparse nature of SMTs to provide shortened proofs. Specifically, it is unlikely for entries to cluster together. Thus, it is efficient to maintain a list of values: Null values d0:=Hash(null),d_0 := \\mathsf{Hash(null)},d0​:=Hash(null), d1:=Hash(d0∥∥d0),d_1 := \\mathsf{Hash(d_0 \\|\\| d_0)},d1​:=Hash(d0​∥∥d0​), d2:=Hash(d1∥∥d1),d_2 := \\mathsf{Hash(d_1 \\|\\| d_1)},d2​:=Hash(d1​∥∥d1​), ⋮\\vdots⋮ dn−1:=Hash(dn−2∥∥dn−2).d_{n-1} := \\mathsf{Hash(d_{n-2} \\|\\| d_{n-2})}.dn−1​:=Hash(dn−2​∥∥dn−2​). Each of the did_idi​'s represents the root hash of a Merkle tree with 2i2^i2i leaves containing null\\mathsf{null}null. These values can be used to shorten the time needed to construct an SMT and the length of proofs.","s":"Sparse Merkle trees","u":"/rlog/climbing-merkle-trees","h":"#sparse-merkle-trees","p":127},{"i":146,"t":"In the first Vac 101 [5], we examined Bloom and Cuckoo filters that could be used for proof of membership and nonmembership. However, the proof of membership may result in false positives due to collisions. This would affect nonmembership proofs as well. Sparse Merkle trees can be adapted to provide greater assurance that a given piece of data is not a member of the digest. Why is sorting essential? The sorting mechanism of data can be arbitrarily chosen. However, it is essential that there are no gaps in the ordering. The maximum number of elements that could ever exist in the digest must be known. A simple method for this is to use a hash function to provide fingerprints to the data. Each hash using either SHA-256 or Keccak has 256-bits. Our entire digest could consist of a maximum of 22562^{256}2256 entries. This assumes that our digest does not contain collisions. The fingerprint of a piece of data ℓ\\ellℓ indicates which leaf of the SMT it is contained in. This means that a nonmembership of ℓ\\ellℓ in the SMT becomes a matter of proving that null\\mathsf{null}null is contained in ℓ\\ellℓ's location. It is crucial for the SMT to be sorted. Otherwise, a malicious party can append the entry ℓ\\ellℓ to a random location. This allows for the malicious party to provide contradictory proofs that prove both membership and nonmembership. We note that the requirement that an SMT is sorted may be too strong of an assumption in centralized cases. However, sortedness is a necessary property of SMTs for decentralized systems.","s":"Proof of nonmembership","u":"/rlog/climbing-merkle-trees","h":"#proof-of-nonmembership","p":127},{"i":148,"t":"A proof of membership grows in length as the Merkle tree grows. The most obvious approach to remedy this scalability issue is to use Merkle trees in which each node has more than two children. However, this does not fix the issue. A proof of membership in a kkk-nary Merkle tree [6] (each node has kkk children) has a proof size log⁡k(n)(k−1)\\log_k(n)(k-1)logk​(n)(k−1). The multiple k−1k-1k−1 is the number of silbings that a node has on each layer. Hence, the proof size grows faster than a logarithmic function of the digest size. An alternate approach is to use a different data structure: Verkle trees [6]. A Verkle tree replaces hash functions with polynomial commitments [7, 8]. We will explore Verkle trees in a future Vac 101 edition.","s":"Verkle Trees","u":"/rlog/climbing-merkle-trees","h":"#verkle-trees","p":127},{"i":150,"t":"Secrecy, Authentication, and Public Key Systems Verifying RLN Proofs in Light Clients with Subtrees Vac 101: Transforming an Interactive Protocol to a Noninteractive Argument Capped merkle tree in Plonky2 Vac 101: Membership with Bloom Filters and Cuckoo Filters Verkle Trees Using polynomial commitments to replace state roots KZG polynomial commitments O1 labs' Verkle Tree repo","s":"References","u":"/rlog/climbing-merkle-trees","h":"#references","p":127},{"i":152,"t":"Looking at discv5 and the theoretical numbers behind finding peers. Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers. This post serves as a more authoritative overview of the discv5 study, for a discussionary post providing more context make sure to check out the corresponding discuss post. Additionally, if you are unfamiliar with discv5, check out my previous write-up: \"From Kademlia to Discv5\".","s":"Feasibility Study: Discv5","u":"/rlog/feasibility-discv5","h":"","p":151},{"i":154,"t":"The discovery method currently used by Status, is made up of various components and grew over time to solve a mix of problems. We want to simplify this while maintaining some of the properties we currently have. Namely, we want to ensure censorship resistance to state-level adversaries. One of the issues Status had which caused us them add to their discovery method was the fact that addresses from providers like AWS and GCP were blocked both in Russia and China. Additionally, one of the main factors required is the ability to function on resource restricted devices. Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider: Battery consumption - constant connections like websockets consume a lot of battery life. CPU usage - certain discovery methods may be CPU incentive, slowing an app down and making it unusable. Bandwidth consumption - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans. Short connection windows - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time. Not publicly connectable - There is a good chance that most resource restricted devices are not publicly connectable. For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node. With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation. In order to answer our questions, we formulated the problem with a simple method for testing. The \"needle in a haystack\" problem was formulated to figure out how easily a specific node can be found within a given network. This issue was fully formulated in vacp2p/research#15.","s":"Motivating Problem","u":"/rlog/feasibility-discv5","h":"#motivating-problem","p":151},{"i":156,"t":"The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer: We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia. We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through topic tables.","s":"Overview","u":"/rlog/feasibility-discv5","h":"#overview","p":151},{"i":158,"t":"To be able to investigate the feasibility of discv5, we used various methods including rough calculations which can be found in the notebook, and a simulation isolated in vacp2p/research#19.","s":"Feasbility","u":"/rlog/feasibility-discv5","h":"#feasbility","p":151},{"i":160,"t":"The experimental discv5 has already been used within Status, however what was noticed was that the CPU and memory usage was rather high. It therefore should be investiaged if this is still the case, and if it is, it should be isolated where this stems from. Additionally it is worth looking at whether or not this is the case with both the go and nim implementation. See details: vacp2p/research#31","s":"CPU & Memory Usage","u":"/rlog/feasibility-discv5","h":"#cpu--memory-usage","p":151},{"i":162,"t":"If a peer is not publically connectable it can not participate in the DHT both ways. A lot of mobile phones are behind symmetric NATs which UDP hole-punching close to impossible. It should be investigated whether or not mobile phones will be able to participate both ways and if there are good methods for doing hole-punching. See details: vacp2p/research#29","s":"NAT on Cellular Data","u":"/rlog/feasibility-discv5","h":"#nat-on-cellular-data","p":151},{"i":164,"t":"Topic Tables allow us the ability to efficiently find nodes given a specific topic. However, they are not implemented in the status-im/nim-eth implementation nor are they fully finalized in the spec. These are important if the network grows past a size where the concentration of specific nodes is relatively low making them hard to find. See details: vacp2p/research#26","s":"Topic Tables","u":"/rlog/feasibility-discv5","h":"#topic-tables","p":151},{"i":166,"t":"It is important to note, that given a network is relatively small sized, eg 100-500 nodes, then finding a node given a specific address is relatively managable. Additionally, if the concentration of a specific capability in a network is reasonable, then finding a node advertising its capabilities using an ENR rather than the topic table is also managable. A reasonable concentration for example would be 10%, which would give us an 80% chance of getting a node with that capability in the first lookup request. This can be explored more using our discv5 notebook.","s":"Finding a node","u":"/rlog/feasibility-discv5","h":"#finding-a-node","p":151},{"i":168,"t":"Research has shown that finding a node in the DHT has a relatively low effect on bandwidth, both inbound and outbound. For example when trying to find a node in a network of 100 nodes, it would take roughly 5668 bytes total. Additionally if we assume 100ms latency per request it would range at ≈ 300ms latency, translating to 3 requests to find a specific node.","s":"Results","u":"/rlog/feasibility-discv5","h":"#results","p":151},{"i":170,"t":"One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more \"features\" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.","s":"General Thoughts","u":"/rlog/feasibility-discv5","h":"#general-thoughts","p":151},{"i":172,"t":"Oskar Thoren Dmitry Shmatko Kim De Mey Corey Petty","s":"Acknowledgements","u":"/rlog/feasibility-discv5","h":"#acknowledgements","p":151},{"i":174,"t":"A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks. tldr: Moon math promising for solving spam in Whisper, but to get there we need to invest more in performance work and technical upskilling.","s":"Feasibility Study: Semaphore rate limiting through zkSNARKs","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"","p":173},{"i":176,"t":"In open p2p networks for messaging, one big problem is spam-resistance. Existing solutions, such as Whisper's proof of work, are insufficient, especially for heterogeneous nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy. One possible solution is to use a right-to-access staking-based method, where a node is only able to send a message, signal, at a certain rate, and otherwise they can be slashed. One problem with this is in terms of privacy-preservation, where we specifically don't want a user to be tied to a specific payment or unique fingerprint.","s":"Motivating problem","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#motivating-problem","p":173},{"i":178,"t":"In addition to above, there are a lot of related problems that share similarities in terms of their structure and proposed solution. Private transactions (Zcash, AZTEC) Private voting (Semaphore) Private group membership (Semaphore) Layer 2 scaling, poss layer 1 (ZK Rollup; StarkWare/Eth2-3)","s":"Related problems","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#related-problems","p":173},{"i":181,"t":"A zero-knowledge proof allows a prover to show a verifier that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id. zkSNARKs is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more. Semaphore is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See current Github home for more. Circom is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See Github for more.","s":"Basic terminology","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#basic-terminology","p":173},{"i":183,"t":"We start with a private voting example, and then extend it to the slashable rate limiting example. A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are. When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some public outputs, and can be verified by anyone (including a smart contract). Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as \"2020-01-01 vote on Foo Bar for president\". Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once. As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified.","s":"Basic flow","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#basic-flow","p":173},{"i":185,"t":"In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat. NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same. Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s. Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key). This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them. As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it.","s":"Rate limiting example","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#rate-limiting-example","p":173},{"i":187,"t":"In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration.","s":"Briefly on scope of 'approved users'","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#briefly-on-scope-of-approved-users","p":173},{"i":189,"t":"For a fairly self-contained set of examples above, see exploration in Vac research repo. Note that the Shamir secret sharing is not inside the SNARK, but out-of-band for now. The current version of Semaphore is using NodeJS and Circom from Iden3 for Snarks. For more on rate limiting idea, see ethresearch post.","s":"Technical details","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#technical-details","p":173},{"i":191,"t":"The above repo was used to exercise the basic paths and to gain intution of feasibility. Based on it and related reading we outline a few blockers and things that require further study.","s":"Feasibility","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#feasibility","p":173},{"i":193,"t":"Proof time​ Prove time for Semaphore (https://github.com/kobigurk/semaphore) zKSNARKs using circom, groth and snarkjs is currently way too long. It takes on the order of ~10m to generate a proof. With Websnark, it is likely to take 30s, which might still be too long. We should experiment with native code on mobile here. See details. Proving key size​ Prover key size is ~110mb for Semaphore. Assuming this is embedded on mobile device, it bloats the APK a lot. Current APK size is ~30mb and even that might be high for people with limited bandwidth. See details. Trusted setup​ Using zkSNARKs a trusted setup is required to generate prover and verifier keys. As part of this setup, a toxic parameter lambda is generated. If a party gets access to this lambda, they can prove anything. This means people using zKSNARKs usually have an elaborate MPC ceremony to ensure this parameter doesn't get discovered. See details. Shamir logic in SNARK​ For Semaphore RLN we need to embed the Shamir logic inside the SNARK in order to do slashing for spam. Currently the implementation is trusted and very hacky. See details. End to end integation​ Currently is standalone and doesn't touch multiple users, deployed contract with merkle tree and verification, actual transactions, a mocked network, add/remove members, etc. There are bound to be edge cases and unknown unknowns here. See details. Licensing issues​ Currently Circom uses a GPL license, which can get tricky when it comes to the App Store etc. See details. Alternative ZKPs?​ Some of the isolated blockers for zKSNARKs (#7, #8, #9) might be mitigated by the use of other ZKP technology. However, they likely have their own issues. See details.","s":"Technical feasibility","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#technical-feasibility","p":173},{"i":195,"t":"Technical skill​ zkSNARKs and related technologies are quite new. To learn how they work and get an intuition for them requires individuals to dedicate a lot of time to studying them. This means we must make getting competence in these technologies if we wish to use them to our advantage. Time and resources​ In order for this and related projects (such as private transaction) to get anywhere, it must be made an explicit area of focus for an extend period of time.","s":"Social feasibility","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#social-feasibility","p":173},{"i":197,"t":"Similar to Whisper, and in line with moving towards protocol and infrastructure, we need to upskill and invest resources into this. This doesn't mean developing all of the technologies ourselves, but gaining enough competence to leverage and extend existing solutions by the growing ZKP community. For example, this might also include leveraging largely ready made solutions such as AZTEC for private transaction; more fundamental research into ZK rollup and similar; using Semaphore for private group membership and private voting; Nim based wrapper aronud Bellman, etc.","s":"General thoughts","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#general-thoughts","p":173},{"i":199,"t":"Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues. Peacock header image from [Tonos]()._","s":"Acknowledgement","u":"/rlog/feasibility-semaphore-rate-limiting-zksnarks","h":"#acknowledgement","p":173},{"i":201,"t":"A research log. Why Whisper doesn't scale and how to fix it. This post will introduce Waku. Waku is a fork of Whisper that attempts to addresses some of Whisper's shortcomings in an iterative fashion. We will also introduce a theoretical scaling model for Whisper that shows why it doesn't scale, and what can be done about it.","s":"Fixing Whisper with Waku","u":"/rlog/fixing-whisper-with-waku","h":"","p":200},{"i":203,"t":"Whisper is a gossip-based communication protocol or an ephemeral key-value store depending on which way you look at it. Historically speaking, it is the messaging pilllar of Web3, together with Ethereum for consensus and Swarm for storage. Whisper, being a somewhat esoteric protocol and with some fundamental issues, hasn't seen a lot of usage. However, applications such as Status are using it, and have been making minor ad hoc modifications to it to make it run on mobile devices. What are these fundamental issues? In short: scalability, most immediately when it comes to bandwidth usage spam-resistance, proof of work is a poor mechanism for heterogeneous nodes no incentivized infrastructure, leading to centralized choke points lack of formal and unambiguous specification makes it hard to analyze and implement running over devp2p, which limits where it can run and how In this post, we'll focus on the first problem, which is scalability through bandwidth usage.","s":"Introduction","u":"/rlog/fixing-whisper-with-waku","h":"#introduction","p":200},{"i":205,"t":"(Feel free to skip this section if you want to get right to the results). There's widespread implicit knowledge that Whisper \"doesn't scale\", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see Status Whisper usage spec).","s":"Whisper theoretical scalability model","u":"/rlog/fixing-whisper-with-waku","h":"#whisper-theoretical-scalability-model","p":200},{"i":207,"t":"First, some caveats: this model likely contains bugs, has wrong assumptions, or completely misses certain dimensions. However, it acts as a form of existence proof for unscalability, with clear reasons. If certain assumptions are wrong, then we can challenge them and reason about them in isolation. It doesn’t mean things will definitely work as the model predicts, and that there aren’t unknown unknowns. The model also only deals with receiving bandwidth for end nodes, uses mostly static assumptions of averages, and doesn’t deal with spam resistance, privacy guarantees, accounting, intermediate node or network wide failures.","s":"Caveats","u":"/rlog/fixing-whisper-with-waku","h":"#caveats","p":200},{"i":209,"t":"Ensure network scales by being user or usage bound, as opposed to bandwidth growing in proportion to network size. Staying with in a reasonable bandwidth limit for limited data plans. Do the above without materially impacting existing nodes. It proceeds through various case with clear assumptions behind them, starting from the most naive assumptions. It shows results for 100 users, 10k users and 1m users.","s":"Goals","u":"/rlog/fixing-whisper-with-waku","h":"#goals","p":200},{"i":211,"t":"Case 1. Only receiving messages meant for you [naive case] Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A4. Only receiving messages meant for you. For 100 users, receiving bandwidth is 1000.0KB/day For 10k users, receiving bandwidth is 1000.0KB/day For 1m users, receiving bandwidth is 1000.0KB/day ------------------------------------------------------------ Case 2. Receiving messages for everyone [naive case] Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A5. Received messages for everyone. For 100 users, receiving bandwidth is 97.7MB/day For 10k users, receiving bandwidth is 9.5GB/day For 1m users, receiving bandwidth is 953.7GB/day ------------------------------------------------------------ Case 3. All private messages go over one discovery topic [naive case] Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A6. Proportion of private messages (static): 0.5 - A7. Public messages only received by relevant recipients (static). - A8. All private messages are received by everyone (same topic) (static). For 100 users, receiving bandwidth is 49.3MB/day For 10k users, receiving bandwidth is 4.8GB/day For 1m users, receiving bandwidth is 476.8GB/day ------------------------------------------------------------ Case 4. All private messages are partitioned into shards [naive case] Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A6. Proportion of private messages (static): 0.5 - A7. Public messages only received by relevant recipients (static). - A9. Private messages partitioned across partition shards (static), n=5000 For 100 users, receiving bandwidth is 1000.0KB/day For 10k users, receiving bandwidth is 1.5MB/day For 1m users, receiving bandwidth is 98.1MB/day ------------------------------------------------------------ Case 5. 4 + Bloom filter with false positive rate Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A6. Proportion of private messages (static): 0.5 - A7. Public messages only received by relevant recipients (static). - A9. Private messages partitioned across partition shards (static), n=5000 - A10. Bloom filter size (m) (static): 512 - A11. Bloom filter hash functions (k) (static): 3 - A12. Bloom filter elements, i.e. topics, (n) (static): 100 - A13. Bloom filter assuming optimal k choice (sensitive to m, n). - A14. Bloom filter false positive proportion of full traffic, p=0.1 For 100 users, receiving bandwidth is 10.7MB/day For 10k users, receiving bandwidth is 978.0MB/day For 1m users, receiving bandwidth is 95.5GB/day NOTE: Traffic extremely sensitive to bloom false positives This completely dominates network traffic at scale. With p=1% we get 10k users ~100MB/day and 1m users ~10gb/day) ------------------------------------------------------------ Case 6. Case 5 + Benign duplicate receives Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A6. Proportion of private messages (static): 0.5 - A7. Public messages only received by relevant recipients (static). - A9. Private messages partitioned across partition shards (static), n=5000 - A10. Bloom filter size (m) (static): 512 - A11. Bloom filter hash functions (k) (static): 3 - A12. Bloom filter elements, i.e. topics, (n) (static): 100 - A13. Bloom filter assuming optimal k choice (sensitive to m, n). - A14. Bloom filter false positive proportion of full traffic, p=0.1 - A15. Benign duplicate receives factor (static): 2 - A16. No bad envelopes, bad PoW, expired, etc (static). For 100 users, receiving bandwidth is 21.5MB/day For 10k users, receiving bandwidth is 1.9GB/day For 1m users, receiving bandwidth is 190.9GB/day ------------------------------------------------------------ Case 7. 6 + Mailserver under good conditions; small bloom fp; mostly offline Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A6. Proportion of private messages (static): 0.5 - A7. Public messages only received by relevant recipients (static). - A9. Private messages partitioned across partition shards (static), n=5000 - A10. Bloom filter size (m) (static): 512 - A11. Bloom filter hash functions (k) (static): 3 - A12. Bloom filter elements, i.e. topics, (n) (static): 100 - A13. Bloom filter assuming optimal k choice (sensitive to m, n). - A14. Bloom filter false positive proportion of full traffic, p=0.1 - A15. Benign duplicate receives factor (static): 2 - A16. No bad envelopes, bad PoW, expired, etc (static). - A17. User is offline p% of the time (static) p=0.9 - A18. No bad request, dup messages for mailservers; overlap perfect (static). - A19. Mailserver requests can change false positive rate to be p=0.01 For 100 users, receiving bandwidth is 3.9MB/day For 10k users, receiving bandwidth is 284.8MB/day For 1m users, receiving bandwidth is 27.8GB/day ------------------------------------------------------------ Case 8. No metadata protection w bloom filter; 1 node connected; static shard Aka waku mode. Next step up is to either only use contact code, or shard more aggressively. Note that this requires change of other nodes behavior, not just local node. Assumptions: - A1. Envelope size (static): 1024kb - A2. Envelopes / message (static): 10 - A3. Received messages / day (static): 100 - A6. Proportion of private messages (static): 0.5 - A7. Public messages only received by relevant recipients (static). - A9. Private messages partitioned across partition shards (static), n=5000 For 100 users, receiving bandwidth is 1000.0KB/day For 10k users, receiving bandwidth is 1.5MB/day For 1m users, receiving bandwidth is 98.1MB/day ------------------------------------------------------------ See source for more detail on the model and its assumptions.","s":"Model","u":"/rlog/fixing-whisper-with-waku","h":"#model","p":200},{"i":213,"t":"Whisper as it currently works doesn’t scale, and we quickly run into unacceptable bandwidth usage. There are a few factors of this, but largely it boils down to noisy topics usage and use of bloom filters. Duplicate (e.g. see Whisper vs PSS) and bad envelopes are also factors, but this depends a bit more on specific deployment configurations. Waku mode (case 8) is an additional capability that doesn’t require other nodes to change, for nodes that put a premium on performance. The next bottleneck after this is the partitioned topics (app/network specific), which either needs to gracefully (and potentially quickly) grow, or an alternative way of consuming those messages needs to be deviced. The results are summarized in the graph above. Notice the log-log scale. The colored backgrounds correspond to the following bandwidth usage: Blue: <10mb/d (<~300mb/month) Green: <30mb/d (<~1gb/month) Yellow: <100mb/d (<~3gb/month) Red: >100mb/d (>3gb/month) These ranges are somewhat arbitrary, but are based on user requirements for users on a limited data plan, with comparable usage for other messaging apps.","s":"Takeaways","u":"/rlog/fixing-whisper-with-waku","h":"#takeaways","p":200},{"i":216,"t":"Apps such as Status will likely use something like Whisper for the forseeable future, and we want to enable them to use it with more users on mobile devices without bandwidth exploding with minimal changes. Additionally, there's not a clear cut alternative that maps cleanly to the desired use cases (p2p, multicast, privacy-preserving, open, etc). We are actively researching, developing and collaborating with more greenfield approaches. It is likely that Waku will either converge to those, or Waku will lay the groundwork (clear specs, common issues/components) necessary to make switching to another protocol easier. In this project we want to emphasize iterative work with results on the order of weeks.","s":"Motivation for a new protocol","u":"/rlog/fixing-whisper-with-waku","h":"#motivation-for-a-new-protocol","p":200},{"i":218,"t":"Doesn’t impact existing clients, it’s just a separate node and capability. Other nodes can still use Whisper as is, like a full node. Sacrifices metadata protection and incurs higher connectivity/availability requirements for scalbility Requirements: Exposes API to get messages from a set of list of topics (no bloom filter) Way of being identified as a Waku node (e.g. through version string) Option to statically encode this node in app, e.g. similar to custom bootnodes/mailserver Only node that needs to be connected to, possibly as Whisper relay / mailserver hybrid Provides: likely provides scalability of up to 10k users and beyond with some enhancements to partition topic logic, can possibly scale up to 1m users (app/network specific) Caveats: hasn’t been tested in a large-scale simulation other network and intermediate node bottlenecks might become apparent (e.g. full bloom filter and private cluster capacity; can likely be dealt with in isolation using known techniques, e.g. load balancing) (deployment specific)","s":"Briefly on Waku mode","u":"/rlog/fixing-whisper-with-waku","h":"#briefly-on-waku-mode","p":200},{"i":220,"t":"In short, we have a Waku version 0 spec up as well as a PoC for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for Waku mode. See rough roadmap, project board [link deprecated] and progress thread on the Vac forum. The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously ad hoc implemented features, such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the geth (Go), status/whisper (Go) and nim-eth (Nim) versions, specifically around light node usage and the handshake. If you are interested in this effort, please check out our forum for questions, comments and proposals. We already have some discussion for better spam protection (see previous post for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement.","s":"Progress so far","u":"/rlog/fixing-whisper-with-waku","h":"#progress-so-far","p":200},{"i":222,"t":"Learn how the Waku Network is evolving through scaling, incentivization, and diverse ecosystem development and what the future might look like. Waku is preparing for production with a focus on the Status Communities use case. In this blog post, we will provide an overview of recent discussions and research outputs, aiming to give you a better understanding of how the Waku network may look like in terms of scaling and incentivization.","s":"The Future of Waku Network: Scaling, Incentivization, and Heterogeneity","u":"/rlog/future-of-waku-network","h":"","p":221},{"i":224,"t":"Waku is actively exploring DOS mitigation mechanisms suitable for Status Communities. While RLN (Rate Limiting Nullifiers) remains the go-to DOS protection solution due to its privacy-preserving and censorship-resistant properties, there is still more work to be done. We are excited to collaborate with PSE (Privacy & Scaling Explorations) in this endeavor. Learn more about their latest progress in this tweet.","s":"DOS Mitigation for Status Communities","u":"/rlog/future-of-waku-network","h":"#dos-mitigation-for-status-communities","p":221},{"i":226,"t":"As we noted in a previous forum post, Waku's protocol incentivization model needs to be flexible to accommodate various business models. Flexibility ensures that projects can choose how they want to use Waku based on their specific needs.","s":"A Heterogeneous Waku Network","u":"/rlog/future-of-waku-network","h":"#a-heterogeneous-waku-network","p":221},{"i":228,"t":"Traditionally, the question of incentivization revolves around how to incentivize operators to run nodes. We'd like to reframe the question and instead ask, \"How do we pay for the infrastructure?\" Waku does not intend to offer a free lunch. Ethereum's infrastructure is supported by transaction fees and inflation, with validators receiving rewards from both sources. However, this model does not suit a communication network like Waku. Users and platforms would not want to pay for every single message they send. Additionally, Waku aims to support instant ephemeral messages that do not require consensus or long-term storage. Projects that use Waku to enable user interactions, whether for chat messages, gaming, private DeFi, notifications, or inter-wallet communication, may have different value extraction models. Some users might provide services for the project and expect to receive value by running nodes, while others may pay for the product or run infrastructure to contribute back. Waku aims to support each of these use cases, which means there will be various ways to \"pay for the infrastructure.\" In his talk, Oskar addressed two strategies: RLN and service credentials.","s":"Reversing the Incentivization Question","u":"/rlog/future-of-waku-network","h":"#reversing-the-incentivization-question","p":221},{"i":230,"t":"RLN enables DOS protection across the network in a privacy-preserving and permission-less manner: stake in a contract, and you can send messages. Service credentials establish a customer-provider relationship. Users might pay to have messages they are interested in stored and served by a provider. Alternatively, a community owner could pay a service provider to host their community. Providers could offer trial or limited free services to Waku users, similar to Slack or Discord. Once a trial is expired or outgrown, a community owner could pay for more storage or bandwidth, similar to Slack's model. Alternatively, individual users could contribute financially, akin to Discord's Server Boost, or by sharing their own resources with their community. We anticipate witnessing various scenarios across the spectrum: from users sharing resources to users paying for access to the network and everything in between.","s":"RLN and Service Credentials","u":"/rlog/future-of-waku-network","h":"#rln-and-service-credentials","p":221},{"i":232,"t":"Another perspective is to consider whether the Waku network will resemble Ethereum or Cosmos. For those not familiar with the difference between both, in a very concise manner: Ethereum is a set of protocols and software that are designed to operate on one common network and infrastructure Cosmos is a set of protocols and software (SDKs) designed to be deployed in separate yet interoperable networks and infrastructures by third parties We want Waku to be decentralized to provide censorship resistance and privacy-preserving communication. If each application has to deploy its own network, we will not achieve this goal. Therefore, we aim Waku to be not only an open source set of protocols, but also a shared infrastructure that anyone can leverage to build applications on top, with some guarantees in terms of decentralization and anonymity. This approach is closer in spirit to Ethereum than Cosmos. Do note that, similarly to Ethereum, anyone is free to take Waku software and protocols and deploy their own network. Yet, because of the difference in the fee model, the Waku Network is unlikely to be as unified as Ethereum's. We currently assume that there will be separate gossipsub networks with different funding models. Since there is no consensus on Waku, each individual operator can decide which network to support, enabling Waku to maintain its permission-less property. Most likely, the Waku network will be heterogeneous, and node operators will choose the incentivization model they prefer.","s":"Waku Network: Ethereum or Cosmos?","u":"/rlog/future-of-waku-network","h":"#waku-network-ethereum-or-cosmos","p":221},{"i":234,"t":"To enable scalability, the flow of messages in the Waku network will be divided in shards, so that not every node has to forward every message of the whole network. Discovery protocols will facilitate users connecting to the right nodes to receive the messages they are interested in. Different shards could be subject to a variety of rate limiting techniques (globally, targeted to that shard or something in-between). Marketplace protocols may also be developed to help operators understand how they can best support the network and where their resources are most needed. However, we are still far from establishing or even assert that such a marketplace will be needed.","s":"Scalability and Discovery Protocols","u":"/rlog/future-of-waku-network","h":"#scalability-and-discovery-protocols","p":221},{"i":236,"t":"Splitting traffic between shards reduces bandwidth consumption for every Waku Relay node. This improvement increases the likelihood that users with home connections can participate and contribute to the gossipsub network without encountering issues. However, it does not cap traffic. There are still open problems regarding how to guarantee that someone can use Waku with lower Internet bandwidth or run critical services, such as a validation node, on the same connection. We have several ongoing initiatives: Analyzing the Status Community protocol to confirm efficient usage of Waku [4] Simulating the Waku Network to measure actual bandwidth usage [5] Segregating chat messages from control and media messages [6] The final solution will likely be a combination of protocols that reduce bandwidth usage or mitigate the risk of DOS attacks, providing flexibility for users and platforms to enable the best experience.","s":"Open Problems","u":"/rlog/future-of-waku-network","h":"#open-problems","p":221},{"i":238,"t":"The definition of the \"Waku Network\" will likely change over time. In the near future, it will transition from a single gossipsub network to a sharded set of networks unified by a common discovery layer. This change will promote scalability and allow various payment models to coexist within the Waku ecosystem. In conclusion, the future of Waku Network entails growth, incentivization, and heterogeneity while steadfastly maintaining its core principles. As Waku continues to evolve, we expect it to accommodate a diverse range of use cases and business models, all while preserving privacy, resisting censorship, avoiding surveillance, and remaining accessible to devices with limited resources.","s":"The Evolving Waku Network","u":"/rlog/future-of-waku-network","h":"#the-evolving-waku-network","p":221},{"i":240,"t":"WAKU2-RELAY-SHARDING 57/STATUS-Simple-Scaling RLN-V2 Scaling Status Communities: Potential Problems Waku Network Testing WAKU2-RELAY-SHARDING: Control Message Shards","s":"References","u":"/rlog/future-of-waku-network","h":"#references","p":221},{"i":242,"t":"This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains.","s":"Libp2p GossipSub IDONTWANT Message Performance Impact","u":"/rlog/gsub-idontwant-perf-eval","h":"","p":241},{"i":244,"t":"IDONTWANT messages are introduced to curtail redundant transmissions without compromising resilience. Cutting down on duplicates can potentially render two significant advantages: Reducing bandwidth requirements Reducing message dissemination time (latency) For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message. Duplicates investigation reveals that the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network. This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, showing a potential for a significant drop in bandwidth utilization. On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers.","s":"Overview","u":"/rlog/gsub-idontwant-perf-eval","h":"#overview","p":241},{"i":246,"t":"We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time. The experiments are performed on nim-libp2p using the shadow simulator. The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages. In all experiments, ten messages are transmitted to the network, i.e., ten peers (publishers) are selected as the message transmitters. Every publisher transmits exactly one message, and inter-packet spacing (delay) is set to four seconds for each published message. For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class. At the start of each experiment, two additional messages are transmitted to increase the TCP CwndC_{wnd}Cwnd​. These messages are not included in latency computations. The simulation details are presented in the table below. Parameter Value Parameter Value Peers 2000 Publishers 10 Peer bandwidth 50-150 Mbps Link latency 40-130 ms Message size 1KB, 50KB, 500KB, 1MB DDD 8 Heartbeat interval 700ms, 1000ms, 1500ms DlowD_{low}Dlow​ 6 FloodPublish False DhighD_{high}Dhigh​ 12 Gossip factor 0.05 Muxer yamux","s":"Experiments","u":"/rlog/gsub-idontwant-perf-eval","h":"#experiments","p":241},{"i":248,"t":"We use bandwidth utilization and latency as evaluation metrics. Bandwidth utilization represents total network-wide traffic (including gossip and other control messages). Latency refers to network-wide message dissemination time. The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights. Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. A higher drop is seen with a higher heartbeat interval. Interestingly, a relatively low bandwidth reduction (12-20%) is seen for 1MB messages, compared to 500KB messages (18-21%). This is because downloading a large message may consume several hundred milliseconds. During this time, a receiver will likely generate multiple IWANT requests for the same message, increasing bandwidth utilization. Moreover, a peer can generate IDONTWANTs only after it has finished downloading the message. A longer download time will result in simultaneous reception of the same message from other mesh members. These IWANT requests mainly overwhelm early message receivers, which can negatively impact message dissemination time on some occasions. Therefore, a similar message dissemination time is seen with and without IDONTWANT messages. Similar results are seen on our large-scale deployment runs (running Waku nodes in Kubernetes). Please feel free to join the discussion and leave feedback regarding this post in the VAC forum.","s":"Findings","u":"/rlog/gsub-idontwant-perf-eval","h":"#findings","p":241},{"i":250,"t":"GossipSub Specifications v1.2 GossipSub v1.2: IDONTWANT Control Message Number Duplicate Messages in Ethereum’s Gossipsub Network IWANT Messages Impact on Latency Large Message Handling (IDONTWANT + IMReceiving) IDONTWANT Message Impact Before/After Message Validation GossipSub for Big Messages Regression Test Results: nim-libp2p","s":"References","u":"/rlog/gsub-idontwant-perf-eval","h":"#references","p":241},{"i":252,"t":"GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks","s":"GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks","u":"/rlog/GossipSub Improvements","h":"","p":251},{"i":254,"t":"We have been recently working on analyzing and improving the performance of the GossipSub protocol for large messages, as in the case of Ethereum Improvement Proposal EIP-4844. This work led to a comprehensive study of unstructured P2P networks. The intention was to identify the best practices that can serve as guidelines for performance improvement and scalability of P2P networks.","s":"Motivitation","u":"/rlog/GossipSub Improvements","h":"#motivitation","p":251},{"i":256,"t":"Nodes in an unstructured p2p network form self-organizing overlay(s) on top of the IP infrastructure to facilitate different services like information dissemination, query propagation, file sharing, etc. The overlay(s) can be as optimal as a tree-like structure or as enforcing as a fully connected mesh. Due to peer autonomy and a trustless computing environment, some peers may deviate from the expected operation or even leave the network. At the same time, the underlying IP layer is unreliable. Therefore, tree-like overlays are not best suited for reliable information propagation. Moreover, tree-based solutions usually result in significantly higher message dissemination latency due to suboptimal branches. Flooding-based solutions, on the other hand, result in maximum resilience against adversaries and achieve minimal message dissemination latency because the message propagates through all (including the optimal) paths. Redundant transmissions help maintain the integrity and security of the network in the presence of adversaries and high node failure but significantly increase network-wide bandwidth utilization, cramming the bottleneck links. An efficient alternative is to lower the number of redundant transmissions by D-regular broadcasting, where a peer will likely receive (or relay) a message from up to DDD random peers. Publishing through a D-regular overlay triggers approximately N×DN \\times DN×D transmissions. Reducing DDD reduces the redundant transmissions but compromises reachability and latency. Sharing metadata through a K-regular overlay (where K>DK > DK>D) allows nodes to pull missing messages. GossipSub [1] benefits from full-message (D-regular) and metadata-only (k-regular) overlays. Alternatively, a metadata-only overlay can be used, requiring a pull-based operation that significantly minimizes bandwidth utilization at the cost of increased latency. Striking the right balance between parameters like D,KD, KD,K, pull-based operation, etc., can yield application-specific performance tuning, but scalability remains a problem. At the same time, many other aspects can significantly contribute to the network's performance and scalability. One option is to realize peers' suitability and continuously changing capabilities while forming overlays. For instance, a low-bandwidth link near a publisher can significantly demean the entire network's performance. Reshuffling of peering links according to the changing network conditions can lead to superior performance. Laying off additional responsibilities to more capable nodes (super nodes) can alleviate peer cramming, but it makes the network susceptible to adversaries/peer churn. Grouping multiple super nodes to form virtual node(s) can solve this problem. Similarly, flat (single-tier) overlays cannot address the routing needs in large (geographically dispersed) networks. Hierarchical (Multi-tier) overlays with different intra/inter-overlay routing solutions can better address these needs. Moreover, using message aggregation schemes for grouping multiple messages can save bandwidth and provide better resilience against adversaries/peer churn. This article's primary objective is to investigate the possible choices that can empower an unstructured P2P network to achieve superior performance for the broadest set of applications. We look into different constraints imposed by application-specific needs (performance goals) and investigate various choices that can augment the network's performance. We explore overlay designs/freshness, peer selection approaches, message-relaying mechanisms, and resilience against adversaries/peer churn. We consider GossipSub a baseline protocol to explore various possibilities and decisively commit to the ones demonstrating superior performance. We also discuss the current state and, where applicable, propose a strategic plan for embedding new features to the GossipSub protocol.","s":"Introduction","u":"/rlog/GossipSub Improvements","h":"#introduction","p":251},{"i":258,"t":"Different applications, like blockchain, streaming, etc., impose strict time bounds on network-wide message dissemination latency. A message delivered after the imposed time bounds is considered as dropped. An early message delivery in applications like live streaming can further enhance the viewing quality. The properties and nature of the overlay network topology significantly impact the performance of services and applications executed on top of them. Studying and devising mechanisms for better overlay design and message dissemination is paramount to achieving superior performance. Interestingly, shortest-path message delivery trees have many limitations: 1) Changing network dynamics requires a quicker and continuous readjustment of the multicast tree. 2) The presence of resource-constrained (bandwidth/compute, etc.) nodes in the overlay can result in congestion. 3) Node failure can result in partitions, making many segments unreachable. 4) Assuring a shortest-path tree-like structure requires a detailed view of the underlying (and continuously changing) network topology. Solutions involve creating multiple random trees to add redundancy [2]. Alternatives involve building an overlay mesh and forwarding messages through the multicast delivery tree (eager push). Metadata is shared through the overlay links so that the nodes can ask for missing messages (lazy push or pull-based operation) through the overlay links. New nodes are added from the overlay on node failure, but it requires non-faulty node selection. GossipSub uses eager push (through overlay mesh) and lazy push (through IWANT messages). The mesh degree DLow≤D≤DHighD_{Low} \\leq D \\leq D_{High}DLow​≤D≤DHigh​ is crucial in deciding message dissemination latency. A smaller value for DDD results in higher latency due to increased rounds, whereas a higher DDD reduces latency on the cost of increased bandwidth. At the same time, keeping DDD independent of the growing network size (NNN) may increase network-wide message dissemination latency. Adjusting DDD with NNN maintains similar latency on the cost of increased workload for peers. Authors in [3] suggest only a logarithmic increase in DDD to maintain a manageable workload for peers. In [4], it is reported that the average mesh degree should not exceed Davg=ln⁡(N)+CD_{avg} = \\ln(N) + CDavg​=ln(N)+C for an optimal operation, where CCC is a small constant. Moreover, quicker shuffling of peers results in better performance in the presence of resource-constrained nodes or node failure [4].","s":"GOAL1: Low Latency Operation","u":"/rlog/GossipSub Improvements","h":"#goal1-low-latency-operation","p":251},{"i":260,"t":"Random peering connections in P2P overlays represent a stochastic process. It is inherently difficult to precisely model the performance of such systems. Most of the research on P2P networks provides simulation results assuming nodes with similar capabilities. The aspect of dissimilar capabilities and resource-constrained nodes is less explored. It is discussed in GOAL1 that overlay mesh results in better performance if DavgD_{avg}Davg​ does not exceed ln⁡(N)+C\\ln(N) + Cln(N)+C. Enforcing all the nodes to have approximately ln⁡(N)+C\\ln(N) + Cln(N)+C peers makes resource-rich nodes under-utilized, while resource-constrained nodes are overloaded. At the same time, connecting high-bandwidth nodes through a low-bandwidth node undermines the network's performance. Ideally, the workload on any node should not exceed its available resources. A better solution involves a two-phased operation: Every node computes its available bandwidth and selects a node degree DDD proportional to its available bandwidth [4]. Different bandwidth estimation approaches are suggested in literature [5,6]. Simple bandwidth estimation approaches like variable packet size probing [6] yield similar results with less complexity. It is also worth mentioning that many nodes may want to allocate only a capped share of their bandwidth to the network. Lowering DDD according to the available bandwidth can still prove helpful. Additionally, bandwidth preservation at the transport layer through approaches like µTP can be useful. To further conform to the suggested mesh-degree average DavgD_{avg}Davg​, every node tries achieving this average within its neighborhood, resulting in an overall similar DavgD_{avg}Davg​. From the available local view, every node tries connecting peers with the lowest latency until DDD connections are made. We suggest referring to the peering solution discussed in GOAL5 to avoid network partitioning. The current GossipSub design considers homogeneous peers, and every node tries maintaining DLow≤D≤DHighD_{Low} \\leq D \\leq D_{High}DLow​≤D≤DHigh​ connections.","s":"GOAL2: Considering Heterogeneity In Overlay Design","u":"/rlog/GossipSub Improvements","h":"#goal2-considering-heterogeneity-in-overlay-design","p":251},{"i":262,"t":"Redundant message transmissions are essential for handling adversaries/node failure. However, these transmissions result in traffic bursts, cramming many overlay links. This not only adds to the network-wide message dissemination latency but a significant share of the network's bandwidth is wasted on (usually) unnecessary transmissions. It is essential to explore solutions that can minimize the number of redundant transmissions while assuring resilience against node failures. Many efforts have been made to minimize the impact of redundant transmissions. These solutions include multicast delivery trees, metadata sharing to enable pull-based operation, in-network information caching, etc. [7,8]. GossipSub employs a hybrid of eager push (message dissemination through the overlay) and lazy push (a pull-based operation by the nodes requiring information through IWANT messages). A better alternative to simple redundant transmission is to use message aggregation [9,10,11] for the GossipSub protocol. As a result, redundant message transmissions can serve as a critical advantage of the GossipSub protocol. Suppose that we have three equal-length messages x1,x2,x3x1, x2, x3x1,x2,x3. Assuming an XOR coding function, we know two trivial properties: x1⊕x2⊕x2=x1x1 \\oplus x2 \\oplus x2 = x1x1⊕x2⊕x2=x1 and ∣x1∣=∣x1⊕x2⊕x2∣\\vert x1 \\vert = \\vert x1 \\oplus x2 \\oplus x2 \\vert∣x1∣=∣x1⊕x2⊕x2∣. This implies that instead of sending messages individually, we can encode and transmit composite message(s) to the network. The receiver can reconstruct the original message from encoded segments. As a result, fewer transmissions are sufficient for sending more messages to the network. However, sharing linear combinations of messages requires organizing messages in intervals, and devising techniques to identify all messages belonging to each interval. In addition, combining messages from different publishers requires more complex arrangements, involving embedding publisher/message IDs, delayed forwarding (to accommodate more messages), and mechanisms to ensure the decoding of messages at all peers. Careful application-specific need analysis can help decide the benefits against the added complexity.","s":"GOAL3: Bandwidth Optimization","u":"/rlog/GossipSub Improvements","h":"#goal3-bandwidth-optimization","p":251},{"i":264,"t":"Many applications require transferring large messages for their successful operation. For instance, database/blockchain transactions [12]. This introduces two challenges: 1) Redundant large message transmissions result in severe network congestion. 2) Message transmissions follow a store/forward process at all peers, which is inefficient in the case of large messages. The above-mentioned challenges result in a noticeable increase in message dissemination latency and bandwidth wastage. Most of the work done for handling large messages involves curtailing redundant transmissions using multicast delivery trees, reducing the number of fanout nodes, employing in-network message caching, pull-based operation, etc. Approaches like message aggregation also prove helpful in minimizing bandwidth wastage. Our recent work on GossipSub improvements (still a work in progress) suggests the following solutions to deal with large message transmissions: Using IDontWant message proposal [13] and staggered sending. IDontWant message helps curtail redundant transmissions by letting other peers know we have already received the message. Staggered sending enables relaying the message to a short subset of peers in each round. We argue that simultaneously relaying a message to all peers hampers the effectiveness of the IDontWant message. Therefore, using the IDontWant message with staggered sending can yield better results by allowing timely reception and processing of IDontWant messages. Message transmissions follow a store/forward process at all peers that is inefficient in the case of large messages. We can parallelize message transmission by partitioning large messages into smaller fragments, letting intermediate peers relay these fragments as soon as they receive them.","s":"GOAL4: Handling Large Messages","u":"/rlog/GossipSub Improvements","h":"#goal4-handling-large-messages","p":251},{"i":266,"t":"P2P networks are inherently scalable because every incoming node brings in bandwidth and compute resources. In other words, we can keep adding nodes to the network as long as every incoming node brings at-least R×DR \\times DR×D bandwidth, where RRR is average data arrival rate. It is worth mentioning that network-wide message dissemination requires at-least ⌈log⁡D(N)⌉\\lceil \\log_D (N) \\rceil⌈logD​(N)⌉ hops. Therefore, increasing network size increases message dissemination latency, assuming D is independent of the network size. Additionally, problems like peer churn, adversaries, heterogeneity, distributed operation, etc., significantly hamper the network's performance. Most efforts for bringing scalability to the P2P systems have focused on curtailing redundant transmissions and flat overlay adjustments. Hierarchical overlay designs, on the other hand, are less explored. Placing a logical structure in unstructured P2P systems can help scale P2P networks. One possible solution is to use a hierarchical overlay inspired by the approaches [14,15,16]. An abstract operation of such overlay design is provided below: Clustering nodes based on locality, assuming that such peers will have relatively lower intra-cluster latency and higher bandwidth. For this purpose, every node tries connecting peers with the lowest latency until DDD connections are made or the cluster limit is reached. A small subset of nodes having the highest bandwidth and compute resources is selected from each cluster. These super nodes form a fully connected mesh and jointly act as a virtual node, mitigating the problem of peer churn among super nodes. Virtual nodes form a fully connected mesh to construct a hierarchical overlay. Each virtual node is essentially a collection of super nodes; a link to any of the constituent super nodes represents a link to the virtual node. One possible idea is to use GossipSub for intra-cluster message dissemination and FloodSub for inter-cluster message dissemination.","s":"GOAL5: Scalability","u":"/rlog/GossipSub Improvements","h":"#goal5-scalability","p":251},{"i":268,"t":"Overlay acts as a virtual backbone for a P2P network. A flat overlay is more straightforward and allows effortless readjustment to application needs. On the other hand, a hierarchical overlay can bring scalability at the cost of increased complexity. Regardless of the overlay design, a continuous readjustment to appropriate peering links is essential for superior performance. At the same time, bandwidth preservation (through message aggregation, caching at strategic locations, metadata sharing, pull-based operation, etc.) can help minimize latency. However, problems like peer churn and in-network adversaries can be best alleviated through balanced redundant coverage, and frequent reshuffling of the peering links.","s":"Summary","u":"/rlog/GossipSub Improvements","h":"#summary","p":251},{"i":270,"t":"[1] D. Vyzovitis, Y. Napora, D. McCormick, D. Dias, and Y. Psaras, “Gossipsub: Attack-resilient message propagation in the filecoin and eth2. 0 networks,” arXiv preprint arXiv:2007.02754, 2020. Retrieved from https://arxiv.org/pdf/2007.02754.pdf [2] M. Matos, V. Schiavoni, P. Felber, R. Oliveira, and E. Riviere, “Brisa: Combining efficiency and reliability in epidemic data dissemination,” in 2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 2012, pp. 983–994. Retrieved from https://ieeexplore.ieee.org/abstract/document/6267905 [3] P. T. Eugster, R. Guerraoui, A. M. Kermarrec, and L. Massouli, “Epidemic information dissemination in distributed systems,” IEEE Computer, vol. 37, no. 5, 2004. Retrieved from https://infoscience.epfl.ch/record/83478/files/EugGueKerMas04IEEEComp.pdf [4] D. Frey, “Epidemic protocols: From large scale to big data,” Ph.D. dissertation, Universite De Rennes 1, 2019. Retrieved from https://inria.hal.science/tel-02375909/document [5] M. Jain and C. Dovrolis, “End-to-end available bandwidth: measurement methodology, dynamics, and relation with tcp throughput,” IEEE/ACM Transactions on networking, vol. 11, no. 4, pp. 537–549, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1224454 [6] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy, “Bandwidth estimation: metrics, measurement techniques, and tools,” IEEE network, vol. 17, no. 6, pp. 27–35, 2003. Retrieved from https://ieeexplore.ieee.org/abstract/document/1248658 [7] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High bandwidth data dissemination using an overlay mesh,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, pp. 282–297. Retrieved from https://dl.acm.org/doi/abs/10.1145/945445.945473 [8] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. E. Mohr, “Chainsaw: Eliminating trees from overlay multicast,” in Peer-to-Peer Systems IV: 4th International Workshop, IPTPS 2005, Ithaca, NY, USA, February 24-25, 2005. Revised Selected Papers 4. Springer, 2005, pp. 127–140. Retrieved from https://link.springer.com/chapter/10.1007/11558989_12 [9] Y.-D. Bromberg, Q. Dufour, and D. Frey, “Multisource rumor spreading with network coding,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 2019, pp. 2359–2367. Retrieved from https://ieeexplore.ieee.org/abstract/document/8737576 [10] B. Haeupler, “Analyzing network coding gossip made easy,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, 2011, pp. 293–302. Retrieved from https://dl.acm.org/doi/abs/10.1145/1993636.1993676 [11] S. Yu and Z. Li, “Massive data delivery in unstructured peer-to-peer networks with network coding,” in 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007). IEEE, 2007, pp. 592–597. Retrieved from https://ieeexplore.ieee.org/abstract/document/4276446 [12] V. Buterin, D. Feist, D. Loerakker, G. Kadianakis, M. Garnett, M. Taiwo, and A. Dietrichs, “Eip-4844: Shard blob transactions scale data-availability of ethereum in a simple, forwards-compatible manner,” 2022. Retrieved from https://eips.ethereum.org/EIPS/eip-4844 [13] A. Manning, “Gossipsub extension for epidemic meshes (v1.2.0),” 2022. Retrieved from https://github.com/libp2p/specs/pull/413 [14] Z. Duan, C. Tian, M. Zhou, X. Wang, N. Zhang, H. Du, and L. Wang, “Two-layer hybrid peer-to-peer networks,” Peer-to-Peer Networking and Applications, vol. 10, pp. 1304–1322, 2017. Retrieved from https://link.springer.com/article/10.1007/s12083-016-0460-5 [15] W. Hao, J. Zeng, X. Dai, J. Xiao, Q. Hua, H. Chen, K.-C. Li, and H. Jin, “Blockp2p: Enabling fast blockchain broadcast with scalable peer-to-peer network topology,” in Green, Pervasive, and Cloud Computing: 14th International Conference, GPC 2019, Uberlandia, Brazil, May 26–28, 2019, Proceedings 14. Springer, 2019, pp. 223–237. Retrieved from https://link.springer.com/chapter/10.1007/978-3-030-19223-5_16 [16] H. Qiu, T. Ji, S. Zhao, X. Chen, J. Qi, H. Cui, and S. Wang, “A geography-based p2p overlay network for fast and robust blockchain systems,” IEEE Transactions on Services Computing, 2022. Retrieved from https://ieeexplore.ieee.org/abstract/document/9826458","s":"References","u":"/rlog/GossipSub Improvements","h":"","p":251},{"i":272,"t":"Large Message Handling in GossipSub: Potential Improvements","s":"Large Message Handling in GossipSub: Potential Improvements","u":"/rlog/gsub-largemsg-improvements","h":"","p":271},{"i":274,"t":"The challenge of large message transmissions in GossipSub leads to longer than expected network-wide message dissemination times (and relatively higher fluctuations). It is particularly relevant for applications that require on-time, network-wide dissemination of large messages, such as Ethereum and Waku [1,2]. This matter has been extensively discussed in the libp2p community [3, 4, 5, 6, 7, 8], and numerous improvements have been considered (or even incorporated) for the GossipSub protocol to enable efficient large-message propagation [3, 7, 9, 10].","s":"Motivation","u":"/rlog/gsub-largemsg-improvements","h":"#motivation","p":271},{"i":276,"t":"Sending a message to N peers involves approximately ⌈log⁡D(N)⌉\\lceil \\log_D(N) \\rceil⌈logD​(N)⌉ rounds, with approximately (D−1)X−1×D(D-1)^{X-1} \\times D(D−1)X−1×D transmissions in each round, where X,DX, DX,D represent the round number and mesh size. Transmitting to a higher number of peers (floodpublish) can theoretically reduce latency by increasing the transmissions in each round to (D−1)X−1×(F+D)(D-1)^{X-1} \\times (F+D)(D−1)X−1×(F+D), where FFF represents the number of peers included in floodpublish. This arrangement works fine for relatively small/moderate message sizes. However, as message sizes increase, significant rises and fluctuations in network-wide message dissemination time are seen. Interestingly, a higher DDD or FFF can also degrade performance in this situation. Several aspects contribute to this behavior: Ideally, a message transmission to a single peer concludes in τ1=LR+P\\tau_1 = \\frac {L}{R}+Pτ1​=RL​+P (ignoring any message processing time), where L,R,PL, R, PL,R,P represent message size, data rate, and link latency. Therefore, the time required for sending a message on a 100Mbps link with 100ms latency jumps from τ110KB=100.8ms\\tau_1^{10KB} = 100.8msτ110KB​=100.8ms for a 10KB message to τ11MB=180ms\\tau_1^{1MB} = 180msτ11MB​=180ms for a 1MB message. For DDD peers, the transmission time increases to τD1MB=(80×D)+100ms\\tau_D^{1MB} = (80 \\times D) + 100msτD1MB​=(80×D)+100ms, triggering additional queuing delays (proportional to the transmission queue size) during each transmission round. In practice, τ11MB\\tau_1^{1MB}τ11MB​ sometimes rises to several hundred milliseconds, further exaggerating the abovementioned queuing delays. This rise is because TCP congestion avoidance algorithms usually limit maximum in-flight bytes in a round trip time (RTT) based on the congestion window (CwndC_{wnd}Cwnd​) and maximum segment size (MSS) to approximately Cwnd×MSS{C_{wnd} \\times MSS}Cwnd​×MSS, with CwndC_{wnd}Cwnd​ rising with the data transfer for each flow. Consequently, sending the same message through a newly established (cold) connection takes longer. The message transfer time lowers as the CwndC_{wnd}Cwnd​ grows. Therefore, performance-improvement practices such as floodpublish, frequent mesh adjustment, and lazy sending typically result in longer than expected message dissemination times for large messages (due to cold connections). It is also worth mentioning that some TCP variants reset their CwndC_{wnd}Cwnd​ after different periods of inactivity. Theoretically, the message transmission time to D peers (τD)(\\tau_D)(τD​) remains the same even if the message is relayed sequentially to all peers or simultaneous transmissions are carried out, i.e., τD=∑i=1Dτi\\tau_D = \\sum_{i=1}^{D} \\tau_iτD​=∑i=1D​τi​ However, sequential transmissions finish early for individual peers, allowing them to relay early. This may result in quicker network-wide message dissemination. A realistic network comprises nodes with dissimilar capabilities (bandwidth, link latency, compute, etc.). As the message disseminates, it's not uncommon for some peers to receive it much earlier than others. Early gossip (IHAVE announcements) may bring in many IWANT requests to the early receivers (even from peers already receiving the same message), which adds to their workload. A busy peer (with a sizeable outgoing message queue) will enqueue (or simultaneously transfer) newly scheduled outgoing messages. As a result, already scheduled messages are prioritized over those published by the peer itself, introducing a significant initial delay to the locally published messages. Enqueuing IWANT replies to the outgoing message queue can further exaggerate the problem. The lack of adaptiveness and standardization in outgoing message prioritization are key factors that can lead to noticeable inconsistency in message dissemination latency at each hop, even in similar network conditions. Message size directly contributes to peers' workloads in terms of processing and transmission time. It also raises the probability of simultaneous redundant transmissions to the same peer, resulting in bandwidth wastage, congestion, and slow message propagation to the network. Moreover, the benefits of sequential message relaying can be compromised by prioritizing slow (or busy) peers. Most use cases necessitate validating received messages before forwarding them to the next-hop peers. For a higher message transfer time (τ)(\\tau )(τ), this store-and-forward delay accumulates across the hops traveled by the message.","s":"Problem description","u":"/rlog/gsub-largemsg-improvements","h":"#problem-description","p":271},{"i":279,"t":"The impact of message size and achievable data rate on message transmit time τ\\tauτ is crucial as this time accumulates due to the store-and-forward delay introduced at intermediate hops. Some possible improvements to minimize overall message dissemination latency include: a. Message fragmentation​ In a homogeneous network, network-wide message dissemination time (ignoring any processing delays) can be simplified to roughly δ≈δTx+Ph\\delta \\approx \\delta_{Tx} + P_hδ≈δTx​+Ph​, where δTx\\delta_{Tx}δTx​ represents accumulative message transmit time denoted as δTx=SR×h\\delta_{Tx} = \\frac{S}{R} \\times hδTx​=RS​×h, with S,RS, RS,R being the data size and data rate, and h,Phh, P_hh,Ph​ being the number of hops in the longest path and message propagation time through the longest path. Partitioning a large message into n fragments reduces a single fragment transmit time to δTxn\\frac{\\delta_{Tx}}{n}nδTx​​. As a received fragment can be immediately relayed by the receiver (while the sender is still transmitting the remaining fragments), it reduces the transmit time to δTx=SR×2h−1n\\delta_{Tx} = \\frac{S}{R} \\times \\frac{2h-1}{n}δTx​=RS​×n2h−1​. This time reduction is mainly attributed to the smaller store-and-forward delay involved in fragment transmissions. However, it is worth noting that many applications require each fragment to be individually verifiable. At the same time, message fragmentation allows a malicious peer to never relay some fragments of a message, which can lead to a significant rise in the application's receive buffer size. Therefore, message fragmentation requires a careful tradeoff analysis between time and risks. b. Message staggering​ Considering the same bandwidth, the time τD\\tau_DτD​ required for sending a message to D peers stays the same, even if we relay to all peers in parallel or send sequentially to the peers, i.e., τD=∑i=1Dτi\\tau_D = \\sum_{i=1}^{D} \\tau_iτD​=∑i=1D​τi​. However, sequential relaying results in quicker message reception at individual peers (τ1≈τDD\\tau_1 \\approx \\frac{\\tau_D}{D}τ1​≈DτD​​) due to bandwidth concentration for a particular peer. So, the receiver can start relaying early to its mesh members while the original sender is still sending the message to other peers. As a result, after every τDD\\frac{\\tau_D}{D}DτD​​ milliseconds, the number of peers receiving the message increases by 2X ∀ X∈{0,D−1}2^X\\ \\forall\\ X \\in \\{0, D-1\\}2X ∀ X∈{0,D−1} and by ∑k=X−DX−1λk ∀ X≥D\\sum_{k=X-D}^{X-1} \\lambda_k\\ \\forall\\ X \\geq D∑k=X−DX−1​λk​ ∀ X≥D. Here, XXX represents message transmission round X=i⋅τDD∣i∈N0X = i \\cdot \\frac{\\tau_D}{D} \\mid i \\in \\mathbb{N}_0X=i⋅DτD​​∣i∈N0​, and λk\\lambda_kλk​ represents the number of peers that received the message in round kkk. It is worth noting that a realistic network imposes certain constraints on staggering for peers. For instance, in a network with dissimilar peer capabilities, placing a slow peer (also in cases where many senders simultaneously select the same peer) at the head of the transmission queue may result in head-of-line blocking for the message queue. At the same time, early receivers get many IWANT requests, increasing their workload. c. Message prioritization for slow senders​ A slow peer often struggles with a backlog of messages in the outgoing message queue(s) for mesh members. Any new message transmission at this stage (especially the locally published messages) gets delayed. Adaptive message-forwarding can help such peers prioritize traffic to minimize latency for essential message transfers. For instance, any GossipSub peer will likely receive every message from multiple senders, leading to redundant transmissions [11]. Implementing efficient strategies (only for slow senders) like lazy sending and prioritizing locally published messages/IWANT replies over already queued messages can help minimize outgoing message queue sizes and optimize bandwidth for essential message transfers. A peer can identify itself as a slow peer by using any bandwidth estimation approach or simply setting an outgoing message queue threshold for all mesh members. Eliminating/deprioritizing some messages can lower a peer's score, but it also earns the peer an overall better score by achieving some early message transfers. For instance, sending many near-first messages can only save a peer from a deficit penalty. On the other hand, sending only one message (assuming MeshMessageDeliveriesThreshold defaults to 1) as the first delivered message can add to the accumulative peer score.","s":"1. Minimizing transfer time for large messages","u":"/rlog/gsub-largemsg-improvements","h":"#1-minimizing-transfer-time-for-large-messages","p":271},{"i":281,"t":"Congestion avoidance algorithms used in various TCP versions directly influence achievable throughput and message transfer time as maximum unacknowledged in-flight bytes are based on the congestion window (Cwnd)(C_{wnd})(Cwnd​) size. Rapid adaptation of CwndC_{wnd}Cwnd​ to the available network conditions can help lower message dissemination latency. Therefore, selecting a more suitable TCP variant like BBR, which is known for its ability to dynamically adjust the congestion window based on network conditions, can significantly enhance GossipSub's performance. At the same time, parameters like receive window scaling and initial CwndC_{wnd}Cwnd​ also impact message transfer time, but these are usually OS-specific system-wide choices. One possible solution is to raise CwndC_{wnd}Cwnd​ by exchanging data over the newly established connection. This data may involve useful details like peer exchange information and gossip to build initial trust, or GossipSub can use some dummy data to raise CwndC_{wnd}Cwnd​ to a reasonable level. It's important to understand that some TCP variants reset CwndC_{wnd}Cwnd​ after specific periods of inactivity [12]. This can lead to a decline in TCP's performance for applications that generate traffic after intervals long enough to trigger the resetting of the congestion window. Implementing straightforward measures like transport-level ping-pong messages can effectively mitigate this problem [13]. The limitations faced with CwndC_{wnd}Cwnd​ scaling also impact some performance optimizations in GossipSub. For instance, floodpublishing is an optimization relying on additional transmissions by the publisher to minimize message dissemination latency. However, a small CwndC_{wnd}Cwnd​ value in (new/cold) TCP connections established with floodpublish peers significantly increases message transmission time [4]. Usually, these peers also receive the same message from other sources during this time, wasting the publisher's bandwidth. The same is the case with IWANT replies. Maintaining a bigger mesh (with warm TCP connections) and relaying to DDD peers can be a better alternative to this problem.","s":"2. Mitigating transport issues","u":"/rlog/gsub-largemsg-improvements","h":"#2-mitigating-transport-issues","p":271},{"i":283,"t":"For every received packet, a peer makes roughly DDD transmissions to contribute its fair share to the spread of messages. However, the fact that many recipients had already received the message (from some other peer) makes this message propagation inefficient. Although the DDD-spread is attributed to quicker dissemination and resilience against non-conforming peers, many potential solutions can still minimize redundant transmissions while preserving the resilience of GossipSub. These solutions, ranging from probabilistic to more knowledgeful elimination of messages from the outgoing message queue, not only address the issue of redundancy but also provide an opportunity for bandwidth optimization, especially for resource-constrained peers. For instance, an IDONTWANT message, a key component of GossipSub (v1.2) [10], can significantly reduce redundant transmissions. It allows any node to notify its mesh members that it has already received a message, thereby preventing them from resending the same message. This functionality is useful when a node receives a message larger than a specified threshold. In such cases, the node promptly informs its mesh peers about the successful reception of the message by sending IDONTWANT messages. It's important to note that an IDONTWANT message is essentially an IHAVE message, but with a crucial difference, i.e., IHAVEs are only transmitted during the heartbeat intervals, whereas IDONTWANTs are sent immediately after receiving a large message. This prompt notification helps curtail redundant large message transmissions without compromising the GossipSub resilience. However, the use of IDONTWANT messages alone has an inherent limitation. For instance, a peer can only send an IDONTWANT after receiving the complete message. A large message transmission consumes significant time. For example, transmitting a 1MB message at 100 Mbps bandwidth may consume 80 to several hundred milliseconds (depending upon CwndC_{wnd}Cwnd​ and latency). As a result, other mesh members may also start transmitting the same message during this interval. A few potential solutions include: a. Staggering with IDONTWANT messages​ As previously discussed, staggering can significantly reduce network-wide message dissemination latency. This is primarily due to the relatively smaller store-and-forward delays that are inherent in this approach. Using both staggering and IDONTWANT messages can further enhance efficiency by reducing redundant transmissions. This is because a node only saturates its bandwidth for a small subset of mesh peers, leading to early transmissions and prompt IDONTWANT message notifications to the mesh members. It is worth highlighting that staggering can be implemented in various ways. For example, it can be applied to peers (peer staggering) where a node sequentially relays the same message to all peers one by one. Alternatively, a node can send a different message to every peer (message staggering or rotational sending), allowing IDONTWANTs for other messages to arrive during this time. The message staggering approach is beneficial when several messages are introduced to the network within a short interval of time. As the peers in staggered sending are sequentially covered (with a faster speed due to bandwidth concentration), this leads to another problem. The early covered peers send IHAVE (during their heartbeat intervals) for the messages they have received. IHAVE announcements for newly received large messages trigger IWANTs from nodes (including those already receiving the same message), leading to an additional workload for early receivers [14]. Potential solutions to mitigate these problems include: 1) Defering IHAVE announcements for large messages. Deferring IHAVE announcements can indirectly prioritize message transmission to the mesh peers over IWANT replies. However, deciding on a suitable deferred interval is crucial for optimal performance. One possible solution is to generate IHAVEs only after the message is relayed to all the mesh peers. 2) Defering IWANT requests for messages that are currently being received. This requires prior knowledge of msgIDs for the messages under reception. Knowing the message length is also essential in deciding a suitable defer interval to handle situations where a sender starts sending a message and never completes the transmission. 3) Not issuing IWANT for a message if at least KKK peers have transmitted IDONTWANT for the same message (as this indicates that these peers will eventually relay this message). However, this approach can inadvertently empower a group of non-conforming mesh peers to send IDONTWANT for a message and never complete message transmission. A delayed IWANT, along with negative peer scoring, can remedy this problem. b. IMReceiving message​ A peer can issue an IDONTWANT only after it has received the entire message. However, a large message transmission may take several hundred milliseconds to complete. During this time, many other mesh members may start relaying the same message. Therefore, the probability of simultaneously receiving the same message from multiple senders increases with the message size, significantly compromising the effectiveness of IDONTWANT messages. Sending a short preamble (containing msgID and length) before the message transmission can provide valuable information about the message. If a receiver is already receiving the same message from another sender, the receiver can request to defer this transmission by sending a brief IMReceiving message. An IDONTWANT from the receiver will indicate successful message reception. Otherwise, the waiting sender can initiate transmission after a specific wait interval. However, waiting for IMReceiving after sending the preamble can delay the message transmission. On the other hand, proceeding with message transfer (after sending the preamble) leads to another problem: it is difficult to cancel ongoing message transmission after receiving IMReceiving for the same message. To streamline this process, a peer can immediately send an IMReceiving message (for every received preamble), urging other mesh peers to defer sending the same message [15, 16]. The other peers can send this message if IDONTWANT is not received from the receiver during the wait interval. This approach can boost IDONTWANT benefits by considering ongoing transmissions for large messages. While IMReceiving messages can bring about substantial improvements in terms of latency and bandwidth utilization, it's crucial to be aware of the potential risks. A malicious user can exploit this approach to disrupt message transmission either by never completing a message or by intentionally sending a message at an extremely slow rate to numerous peers. This could ultimately result in network-wide slow message propagation. However, carefully calibrating the deferring interval (based on message size) and negative peer scoring can help mitigate these risks. c. IDONTWANT message with reduced forwarding​ It is common for slow peers to pile up outgoing message queues, especially for large message transfers. This results in a significant queuing delay for outgoing messages. Reduced message forwarding can help decrease the workload of slower peers. On receiving a message longer than the specified threshold, a slow peer can relay it to only K∈DK \\in DK∈D peers and send an IDONTWANT message to all the peers in DDD. In this arrangement, the IDONTWANT message serves an additional purpose: to promptly announce data availability, reinforcing redundancy in the presence of adversaries. When a peer receives an IDONTWANT for an unseen message, it learns about the new message and can request it by sending an IWANT request without waiting for the heartbeat (gossip) interval. As a result, a significantly smaller number of transmissions is sufficient for propagating the message to the entire network. This approach conserves peer bandwidth by minimizing redundant transmissions while ensuring GossipSub resilience at the cost of one RTT (for missing peers). Interestingly, curtailing queuing delays can also help lower network-wide message dissemination latency (for huge messages). However, finding an appropriate value for KKK is crucial for optimal performance. A smaller KKK saves peer bandwidth, while a larger KKK achieves quicker spread until outgoing message queues pile up. Setting K=DlowK = D_{low}K=Dlow​ can be one option. It is worth mentioning that such behavior may negatively impact peer scoring (by missing message delivery rewards from D−KD-KD−K peers). However, a minimized workload enables early message dissemination to the remaining peers. These early transmissions and randomized KKK set selection can help achieve an overall better peer score.","s":"3. Eliminating redundant transmissions","u":"/rlog/gsub-largemsg-improvements","h":"#3-eliminating-redundant-transmissions","p":271},{"i":285,"t":"Despite the standardized specifications of the GossipSub protocol, the message forwarding mechanisms can significantly impact network-wide message dissemination latency and bandwidth utilization. It is worth mentioning that every node is responsible for transmitting different types of packets, including control messages, locally published messages, messages received from mesh members, IWANT replies, etc. As long as traffic volume is lower than the available data rate, the message forwarding mechanisms yield similar results due to negligible queuing delays. However, when the traffic volume increases and exceeds the available peer bandwidth (even for short traffic bursts), the outgoing message queue(s) sizes rise, potentially impacting the network's performance. In this scenario, FIFO-based traffic forwarding can lead to locally published messages being placed at the end of the outgoing message queue, introducing a queuing delay proportional to the queue size. The same applies to other delay-sensitive messages like IDONTWANT, PRUNE, etc. On the other hand, the segregation of traffic into priority and non-priority queues can potentially starve low-priority messages. One possible solution is to use weighted queues for a fair spread of messages. Message prioritization can be a powerful tool to ensure that important messages reach their intended recipients on time and allow for customizable message handling. For example, staggering between peers and messages can be better managed by using priority queues. However, it is important to note that message prioritization also introduces additional complexity to the system, necessitating sophisticated algorithms for better message handling.","s":"4. Message prioritization","u":"/rlog/gsub-largemsg-improvements","h":"#4-message-prioritization","p":271},{"i":287,"t":"During heartbeat intervals, GossipSub nodes transmit IHAVE messages (carrying IDs of seen messages) to the peers not included in the full-message mesh. These peers can use IWANT messages to request any missing messages. A budget counter ensures these messages never exceed a specified threshold during each heartbeat interval. The IHAVE/IWANT messages are a crucial tool in maintaining network connectivity. They bridge the information gap between nearby and far-off peers, ensuring that information can be disseminated to peers outside the mesh. This function is essential in protecting against network partitions and indirectly aids in safeguarding against Sybil and eclipse attacks. However, it is essential to understand that high transmission times for large messages require careful due diligence when using IWANT messages for reasons not limited to: 1) A large message reception may take several hundred milliseconds to complete. During this time, an IHAVE message announcing the same message ID will trigger an IWANT request. 2) A peer can send IWANT requests for the same message to multiple nodes, leading to simultaneous transmissions of the same message. 3) Replying to (potentially many) IWANT requests can delay the transmission of the same message to mesh peers, resulting in lower peer scores and slower message propagation. A few possible solutions to mitigate this problem may include: 1) Issuing IHAVE announcements only after the message is delivered to many mesh peers. 2) Allocating a volume-based budget to service IWANT requests during each heartbeat interval. 3) Deferring IWANT requests for messages that are currently being received. 4) Deferring IWANT requests if at least KKK IDONTWANTs are received for the same message. 5) A large message transmission can yield high CwndC_{wnd}Cwnd​; preferring such peers during mesh maintenance can be helpful.","s":"5. Maximizing benefits from IWANT messages","u":"/rlog/gsub-largemsg-improvements","h":"#5-maximizing-benefits-from-iwant-messages","p":271},{"i":289,"t":"This study investigates the pressing issue of considerable fluctuations and rises in network-wide dissemination times for large messages. We delve into multiple factors, such as increased message transmit times, store-and-forward delays, congestion avoidance mechanisms, and prioritization between messages, to establish a comprehensive understanding of the problem. The study also explores the performance of optimization efforts like floodpublishing, IHAVE/IWANT messages, and message forwarding strategies in the wake of large message transmissions. A key finding is that most congestion avoidance algorithms lack optimization for peer-to-peer networks. Coupling this constraint with increased message transmission times results in notable store-and-forward delays accumulating at each hop. Furthermore, the probabilistic message-forwarding nature of GossipSub further exacerbates the situation by utilizing a considerable share of available bandwidth on redundant transmissions. Therefore, approaches focused on eliminating redundant transmissions (IDONTWANT, IMReceiving, lazy sending, etc.) can prove helpful. At the same time, strategies aimed at reducing store-and-forward delays (fragmentation, staggering, prioritization, etc.) can prove beneficial. It is worth mentioning that many of the strategies suggested in this post are ideas at different stages. Some of these have already been explored and discussed to some extent [5, 17, 18]. We are nearing the completion of a comprehensive performance evaluation of these approaches and will soon share the results of our findings. Please feel free to join the discussion and leave feedback regarding this post in the VAC forum.","s":"Summary","u":"/rlog/gsub-largemsg-improvements","h":"#summary","p":271},{"i":291,"t":"[1] EIP-4844: Shard Blob Transactions. Retrieved from https://eips.ethereum.org/EIPS/eip-4844 [2] Message Propagation Times With Waku-RLN. Retrieved from https://docs.waku.org/research/research-and-studies/message-propagation/ [3] Lenient Flood Publishing. Retrieved from https://github.com/libp2p/rust-libp2p/pull/3666 [4] Disable Flood Publishing. Retrieved from https://github.com/sigp/lighthouse/pull/4383 [5] GossipSub for Big Messages. Retrieved from https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw [6] GossipSub: Lazy Sending. Retrieved from https://github.com/status-im/nim-libp2p/issues/850 [7] GossipSub: Limit Flood Publishing. Retrieved from https://github.com/vacp2p/nim-libp2p/pull/911 [8] GossipSub: Lazy Prefix Detection. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/859 [9] Potential Gossip Improvement List for EIP4844. Retrieved from https://hackmd.io/@gRwfloEASH6NWWS_KJxFGQ/B18wdnNDh [10] GossipSub Specifications v1.2: IDONTWANT Message. Retrieved from https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52 [11] Number of Duplicate Messages in Ethereum’s GossipSub Network. Retrieved from https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921 [12] TCP Congestion Control: Re-starting Idle Connections. Retrieved from https://datatracker.ietf.org/doc/html/rfc2581#section-4.1 [13] PING/PONG Control Messages. Retrieved from https://github.com/libp2p/specs/pull/558 [14] IHAVE/IWANT Message Impact. Retrieved from https://github.com/vacp2p/nim-libp2p/issues/1101 [15] Large Message Handling IDONTWANT + IMReceiving Messages. Retrieved from https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281 [16] IDONTWANT Message Impact. Retrieved from https://forum.vac.dev/t/idontwant-message-impact/283 [17] IWANT Message Impact. Retrieved from https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366 [18] IDONTWANT Message Performance. Retrieved from https://vac.dev/rlog/gsub-idontwant-perf-eval/","s":"References","u":"/rlog/gsub-largemsg-improvements","h":"#references","p":271},{"i":293,"t":"A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ. If you've been working on Ethereum or adjacent technologies you've probably heard of discv4 or discv5. But what are they actually? How do they work and what makes them different? To answer these questions, we need to start at the beginning, so this post will assume that there is little knowledge on the subject so the post should be accessible for anyone.","s":"From Kademlia to Discv5","u":"/rlog/kademlia-to-discv5","h":"","p":292},{"i":295,"t":"Let's start right at the beginning: the problem of discovery and organization of nodes in peer-to-peer networks. Early P2P file sharing technologies, such as Napster, would share information about who holds what file using a single server. A node would connect to the central server and give it a list of the files it owns. Another node would then connect to that central server, find a node that has the file it is looking for and contact that node. This however was a flawed system -- it was vulnerable to attacks and left a single party open to lawsuits. It became clear that another solution was needed, and after years of research and experimentation, we were given the distributed hash table or DHT.","s":"The Beginning","u":"/rlog/kademlia-to-discv5","h":"#the-beginning","p":292},{"i":297,"t":"In 2001 4 new protocols for such DHTs were conceived, Tapestry, Chord, CAN and Pastry, all of which made various trade-offs and changes in their core functionality, giving them unique characteristics. But as said, they're all DHTs. So what is a DHT? A distributed hash table (DHT) is essentially a distributed key-value list. Nodes participating in the DHT can easily retrieve the value for a key. If we have a network with 9 key-value pairs and 3 nodes, ideally each node would store 3 (optimally 6 for redundancy) of those key-value pairs, meaning that if a key-value pair were to be updated, only part of the network would responsible for ensuring that it is. The idea is that any node in the network would know where to find the specific key-value pair it is looking for based on how things are distributed amongst the nodes.","s":"Distributed Hash Tables","u":"/rlog/kademlia-to-discv5","h":"#distributed-hash-tables","p":292},{"i":299,"t":"So now that we know what DHTs are, let's get to Kademlia, the predecessor of discv4. Kademlia was created by Petar Maymounkov and David Mazières in 2002. I will naively say that this is probably one of the most popular and most used DHT protocols. It's quite simple in how it works, so let's look at it. In Kademlia, nodes and values are arranged by distance (in a very mathematical definition). This distance is not a geographical one, but rather based on identifiers. It is calculated how far 2 identifiers are from eachother using some distance function. Kademlia uses an XOR as its distance function. An XOR is a function that outputs true only when inputs differ. Here is an example with some binary identifiers: XOR 10011001 00110010 -------- 10101011 The top in decimal numbers means that the distance between 153 and 50 is 171. There are several reasons why XOR was taken: The distance from one ID to itself will be 0. Distance is symmetric, A to B is the same as B to A. Follows triangle inequality, if A, B and C are points on a triangle then the distance A to B is closer or equal to that of A to C plus the one from B to C. In summary, this distance function allows a node to decide what is \"close\" to it and make decisions based on that \"closeness\". Kademlia nodes store a routing table. This table contains multiple lists. Each subsequent list contains nodes which are a little further distanced than the ones included in the previous list. Nodes maintain detailed knowledge about nodes closest to them, and the further away a node is, the less knowledge the node maintains about it. So let's say I want to find a specific node. What I would do is go to any node which I already know and ask them for all their neighbours closest to my target. I repeat this process for the returned neighbours until I find my target. The same thing happens for values. Values have a certain distance from nodes and their IDs are structured the same way so we can calculate this distance. If I want to find a value, I simply look for the neighbours closest to that value's key until I find the one storing said value. For Kademlia nodes to support these functions, there are several messages with which the protocol communicates. PING - Used to check whether a node is still running. STORE - Stores a value with a given key on a node. FINDNODE - Returns the closest nodes requested to a given ID. FINDVALUE - The same as FINDNODE, except if a node stores the specific value it will return it directly. This is a very simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the paper or a more in-depth design specification","s":"Kademlia","u":"/rlog/kademlia-to-discv5","h":"#kademlia","p":292},{"i":301,"t":"Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT. Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the FINDVALUE and STORE commands described by Kademlia. The lookup method previously described by Kademlia describes how a node gets its peers. A node contacts some node and asks it for the nodes closest to itself. It does so until it can no longer find any new nodes. Additionally, discv4 adds mutual endpoint verification. This is meant to ensure that a peer calling FINDNODE also participates in the discovery protocol. Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called ENRRequest. If you want some more details on ENRs, check out one of my posts \"Network Addresses in Ethereum\" Discv4 comes with its own range of problems however. Let's look at a few of them. Firstly, the way discv4 works right now, there is no way to differentiate between node sub-protocols. This means for example that an Ethereum node could add an Ethereum Classic Node, Swarm or Whisper node to its DHT without realizing that it is invalid until more communication has happened. This inability to differentiate sub-protocols makes it harder to find specific nodes, such as Ethereum nodes with light-client support. Next, in order to prevent replay attacks, discv4 uses timestamps. This however can lead to various issues when a host's clock is wrong. For more details, see the \"Known Issues\" section of the discv4 specification. Finally, we have an issue with the way mutual endpoint verification works. Messages can get dropped and there is no way to tell if both peers have verified eachother. This means that we could consider our peer verified while it does not consider us so making them drop the FINDNODE packet.","s":"Discv4","u":"/rlog/kademlia-to-discv5","h":"#discv4","p":292},{"i":303,"t":"Finally, let's look at discv5. The next iteration of discv4 and the discovery protocol which will be used by Eth 2.0. It aims at fixing various issues present in discv4. The first change is the way FINDNODE works. In traditional Kademlia as well as in discv5, we pass an identifier. However, in discv5 we instead pass the logarithmic distance, meaning that a FINDNODE request gets a response containing all nodes at the specified logarithmic distance from the called node. Logarithmic distance means we first calculate the distance and then run it through our log base 2 function. See: log2(A xor B) And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are first in first out lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering ads on their peers. As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place ads on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network. Additionally, it is unclear how many peers a node should place these ads on and exactly which peers to place them on. For more details, check out the issue devp2p#136. There are a bunch more smaller changes to the protocol, but they are less important hence they were ommitted from this summary. Nevertheless, discv5 still does not resolve a couple issues present in discv4, such as unreliable endpoint verification. As of writing this post, there is currently no new method in discv5 to improve the endpoint verification process. As you can see discv5 is still a work in progress and has a few large challenges to overcome. However if it does, it will most likely be a large improvement to a more naive Kademlia implementations. Hopefully this article helped explain what these discovery protocols are and how they work. If you're interested in their full specifications you can find them on github.","s":"Discv5","u":"/rlog/kademlia-to-discv5","h":"#discv5","p":292},{"i":305,"t":"Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.","s":"Introducing nwaku","u":"/rlog/introducing-nwaku","h":"","p":304},{"i":307,"t":"If you've been following our research log, you'll know that many things have happened in the world of Waku v2 since our last general update. In line with our long term goals, we've introduced new protocols, tweaked our existing protocols and expanded our team. We've also shown in a series of practical experiments that Waku v2 does indeed deliver on some of the theoretical advantages it was designed to have over its predecessor, Waku v1. A sustainability and business workshop led to the formulation of a clearer vision for Vac as a team. From the beginning, our protocol development has been complemented by various client implementations of these protocols, first in Nim, but later also in JavaScript and Go. A follow-up post will clarify the purposes, similarities and differences between these three clients. The Nim client, is our reference implementation, developed by the research team in parallel with the specs and building on a home-grown implementation of libp2p. The Nim client is suitable to run as a standalone adaptive node, managed by individual operators or as an encapsulated service node in other applications. This post looks at some recent developments within the Nim client.","s":"Background","u":"/rlog/introducing-nwaku","h":"#background","p":304},{"i":309,"t":"Pronounced NWHA-koo. You may already have seen us refer to \"nwaku\" on Vac communication channels, but it is now official: The nim-waku Waku v2 client has been named nwaku. Why? Well, we needed a recognizable name for our client that could easily be referred to in everyday conversations and nim-waku just didn't roll off the tongue. We've followed the example of the closely related nimbus project to find a punchier name that explicitly links the client to both the Waku set of protocols and the Nim language.","s":"1. nim-waku is now known as nwaku","u":"/rlog/introducing-nwaku","h":"#1-nim-waku-is-now-known-as-nwaku","p":304},{"i":311,"t":"The initial implementation of Waku v2 demonstrated how the suite of protocols can be applied to form a generalized, peer-to-peer messaging network, while addressing a wide range of adaptive requirements. This allowed us to lift several protocol specifications from raw to draft status, indicating that a reference implementation exists for each. However, as internal dogfooding increased and more external applications started using nwaku, we stepped up our focus on the client's stability and performance. This is especially true where we want nwaku to run unsupervised in a production environment without any degradation in the services it provides. Some of the more significant productionization efforts over the last couple of months included: Reworking the store implementation to maintain stable memory usage while storing historical messages and serving multiple clients querying history simultaneously. Previously, a store node would see gradual service degradation due to inefficient memory usage when responding to history queries. Queries that often took longer than 8 mins now complete in under 100 ms. Improved peer management. For example, filter nodes will now remove unreachable clients after a number of connection failures, whereas they would previously keep accumulating dead peers. Improved disk usage. nwaku nodes that persist historical messages on disk now manage their own storage size based on the --store-capacity. This can significantly improve node start-up times. More stability issues may be addressed in future as nwaku matures, but we've noticed a marked improvement in the reliability of running nwaku nodes. These include environments where nwaku nodes are expected to run with a long uptime. Vac currently operates two long-running fleets of nwaku nodes, wakuv2.prod and wakuv2.test, for internal dogfooding and to serve as experimental bootstrapping nodes. Status has also recently deployed similar fleets for production and testing based on nwaku. Our goal is to have nwaku be stable, performant and flexible enough to be an attractive option for operators to run and maintain their own Waku v2 nodes. See also the future work section below for more on our general goal of nwaku for operators.","s":"2. Improvements in stability and performance","u":"/rlog/introducing-nwaku","h":"#2-improvements-in-stability-and-performance","p":304},{"i":313,"t":"We've implemented several features that improve nwaku's usability in different environments and its interoperability with other Waku v2 clients. One major step forward here was adding support for both secure and unsecured WebSocket connections as libp2p transports. This allows direct connectivity with js-waku and paves the way for native browser usage. We've also added support for parsing and resolving DNS-type multiaddrs, i.e. multiaddress protocol schemes dns, dns4, dns6 and dnsaddr. A nwaku node can now also be configured with its own IPv4 DNS domain name allowing dynamic IP address allocation without impacting a node's reachability by its peers.","s":"3. Improvements in interoperability","u":"/rlog/introducing-nwaku","h":"#3-improvements-in-interoperability","p":304},{"i":315,"t":"Peer discovery is the method by which nodes become aware of each other’s existence. The question of peer discovery in a Waku v2 network has been a focus area since the protocol was first conceptualized. Since then several different approaches to discovery have been proposed and investigated. We've implemented three discovery mechanisms in nwaku so far:","s":"4. Peer discovery","u":"/rlog/introducing-nwaku","h":"#4-peer-discovery","p":304},{"i":317,"t":"nwaku nodes can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. Our implementation is based on EIP-1459.","s":"DNS-based discovery","u":"/rlog/introducing-nwaku","h":"#dns-based-discovery","p":304},{"i":319,"t":"GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism whereby a pruning peer may provide a pruned peer with a set of alternative peers where it can connect to reform its mesh. This is a very suitable mechanism to gradually discover more peers from an initial connection to a small set of bootstrap peers. It is enabled in a nwaku node by default.","s":"GossipSub peer exchange","u":"/rlog/introducing-nwaku","h":"#gossipsub-peer-exchange","p":304},{"i":321,"t":"This is a DHT-based discovery mechanism adapted to store and relay node records. Our implementation is based on Ethereum's Discovery v5 protocol with some minor modifications to isolate our discovery network from that of Ethereum. The decision to separate the Waku Discovery v5 network from Ethereum's was made on considerations of lookup efficiency. This comes at a possible tradeoff in network resilience. We are considering merging with the Ethereum Discovery v5 network in future, or even implement a hybrid solution. This post explains the decision and future steps.","s":"Waku Node Discovery Protocol v5","u":"/rlog/introducing-nwaku","h":"#waku-node-discovery-protocol-v5","p":304},{"i":323,"t":"An early addition to our suite of protocols was an extension of 11/WAKU-RELAY that provided spam protection using Rate Limiting Nullifiers (RLN). The nwaku client now contains a working demonstration and integration of RLN relay. Check out this tutorial to see the protocol in action using a toy chat application built on nwaku. We'd love for people to join us in dogfooding RLN spam protection as part of our operator incentive testnet. Feel free to join our Vac Discord server and head to the #rln channel for more information.","s":"5. Spam protection using RLN","u":"/rlog/introducing-nwaku","h":"#5-spam-protection-using-rln","p":304},{"i":325,"t":"As we continue working towards our goal of a fully decentralized, generalized and censorship-resistant messaging protocol, these are some of the current and future focus areas for nwaku:","s":"Future work","u":"/rlog/introducing-nwaku","h":"#future-work","p":304},{"i":327,"t":"We are starting to push for operators to run and maintain their own Waku v2 nodes, preferably contributing to the default Waku v2 network as described by the default pubsub topic (/waku/2/default-waku/proto). Amongst other things, a large fleet of stable operator-run Waku v2 nodes will help secure the network, provide valuable services to a variety of applications and ensure the future sustainability of both Vac as a research organization and the Waku suite of protocols. We are targeting nwaku as the main option for operator-run nodes. Specifically, we aim to provide through nwaku: a lightweight and robust Waku v2 client. This client must be first in line to support innovative and new Waku v2 protocols, but configurable enough to serve the adaptive needs of various operators. an easy-to-follow guide for operators to configure, set up and maintain their own nodes a set of operator-focused tools to monitor and maintain a running node","s":"Reaching out to operators:","u":"/rlog/introducing-nwaku","h":"#reaching-out-to-operators","p":304},{"i":329,"t":"Conversational security guarantees in Waku v2 are currently designed around the Status application. Developers building their own applications on top of Waku would therefore either have to reimplement a set of tools similar to Status or build their own security solutions on the application layer above Waku. We are working on a set of features built into Waku that will provide the general security properties Waku users may desire and do so in a modern and simple way. This is useful for applications outside of Status that want similar security guarantees. As a first step, we've already made good progress toward integrating noise handshakes as a key exchange mechanism in Waku v2.","s":"Better conversational security layer guarantees","u":"/rlog/introducing-nwaku","h":"#better-conversational-security-layer-guarantees","p":304},{"i":331,"t":"We want to design incentivization around our protocols to encourage desired behaviors in the Waku network, rewarding nodes providing costly services and punishing adversarial actions. This will increase the overall security of the network and encourage operators to run their own Waku nodes. In turn, the sustainability of Vac as an organization will be better guaranteed. As such, protocol incentivization was a major focus in our recent Vac Sustainability and Business Workshop. Our first step here is to finish integrating RLN relay into Waku with blockchain interaction to manage members, punish spammers and reward spam detectors. After this, we want to design monetary incentivization for providers of store, lightpush and filter services. This may also tie into a reputation mechanism for service nodes based on a network-wide consensus on service quality. A big challenge for protocol incentivization is doing it in a private fashion, so we can keep similar metadata protection guarantees as the Waku base layer. This ties into our focus on Zero Knowledge tech.","s":"Protocol incentivization","u":"/rlog/introducing-nwaku","h":"#protocol-incentivization","p":304},{"i":333,"t":"The nwaku store currently serves as an efficient in-memory store for historical messages, dimensioned by the maximum number of messages the store node is willing to keep. This makes the nwaku store appropriate for keeping history over a short term without any time-based guarantees, but with the advantage of providing fast responses to history queries. Some applications, such as Status, require longer-term historical message storage with time-based dimensioning to guarantee that messages will be stored for a specified minimum period. Because of the relatively high cost of memory compared to disk space, a higher capacity store, with time guarantees, should operate as a disk-only database of historical messages. This is an ongoing effort.","s":"Improved store capacity","u":"/rlog/introducing-nwaku","h":"#improved-store-capacity","p":304},{"i":335,"t":"In addition to the three discovery methods already implemented in nwaku, we are working on improving discovery on at least three fronts: Capability discovery:​ Waku v2 nodes may be interested in peers with specific capabilities, for example: peers within a specific pubsub topic mesh, peers with store capability, store peers with x days of history for a specific content topic, etc. Capability discovery entails mechanisms by which such capabilities can be advertised and discovered/negotiated. One major hurdle to overcome is the increased complexity of finding a node with specific capabilities within the larger network (a needle in a haystack). See the original problem statement for more. Improvements in Discovery v5​ Of the implemented discovery methods, Discovery v5 best addresses our need for a decentralized and scalable discovery mechanism. With the basic implementation done, there are some improvements planned for Discovery v5, including methods to increase security such as merging with the Ethereum Discovery v5 network, introducing explicit NAT traversal and utilizing topic advertisement. The Waku v2 Discovery v5 Roadmap contains more details. Generalized peer exchange​ nwaku already implements GossipSub peer exchange. We now need a general request-response mechanism outside of GossipSub by which a node may learn about other Waku v2 nodes by requesting and receiving a list of peers from a neighbor. This could, for example, be a suitable way for resource-restricted devices to request a stronger peer to perform a random Discovery v5 lookup on their behalf or simply to be informed of a subset of the peers known to that neighbor. See this issue for more. This concludes a general outline of some of the main recent developments in the nwaku client and a summary of the current and future focus areas. Much more is happening behind the scenes, of course, so for more information, or to join the conversation, feel free to join our Vac Discord server or to check out the nwaku repo on Github. You can also view the changelog for past releases here.","s":"Multipurpose discovery","u":"/rlog/introducing-nwaku","h":"#multipurpose-discovery","p":304},{"i":337,"t":"17/WAKU-RLN-RELAY 32/RLN 33/WAKU2-DISCV5 Capabilities advertising Configuring a domain name Conversational security Discovery v5 Topic Advertisement EIP-1459 GossipSub Peer Exchange go-waku js-waku multiaddr formats nimbus-eth2 nim-libp2p nim-waku nim-waku releases Node Discovery Protocol v5 - Theory Noise handshakes RLN tutorial Vac <3 ZK Vac About page Vac Research log Vac RFC site Vac Sustainability and Business Workshop Waku Update Waku v1 vs Waku v2: Bandwidth Comparison Waku v2 Peer Exchange Waku v2 Discovery v5 Roadmap What's the Plan for Waku v2?","s":"References","u":"/rlog/introducing-nwaku","h":"#references","p":304},{"i":339,"t":"Nescience, a privacy-first blockchain zkVM.","s":"Nescience - A zkVM leveraging hiding properties","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"","p":338},{"i":341,"t":"Nescience is a privacy-first blockchain project that aims to enable private transactions and provide a general-purpose execution environment for classical applications. The goals include creating a state separation architecture for public/private computation, designing a versatile virtual machine based on mainstream instruction sets, creating proofs for private state updates, implementing a kernel-based architecture for correct execution of private functions, and implementing core DeFi protocols such as AMMs and staking from a privacy perspective. It intends to create a user experience that is similar to public blockchains, but with additional privacy features that users can leverage at will. To achieve this goal, Nescience will implement a versatile virtual machine that can be used to implement existing blockchain applications, while also enabling the development of privacy-centric protocols such as private staking and private DEXs. To ensure minimal trust assumptions and prevent information leakage, Nescience proposes a proof system that allows users to create proofs for private state updates, while the verification of the proofs and the execution of the public functions inside the virtual machine can be delegated to an external incentivised prover. It also aims to implement a seamless interaction between public and private state, enabling composability between contracts, and private and public functions. Finally, Nescience intends to implement permissive licensing, which means that the source code will be open-source, and developers will be able to use and modify the code without any restriction. Our primary objective is the construction of the Zero-Knowledge Virtual Machine (zkVM). This document serves as a detailed exploration of the multifaceted challenges, potential solutions, and alternatives that lay ahead. Each step is a testament to our commitment to thoroughness; we systematically test various possibilities and decisively commit to the one that demonstrates paramount performance and utility. For instance, as we progress towards achieving Goal 2, we are undertaking a rigorous benchmarking of the Nova proof system against its contemporaries. Should Nova showcase superior performance metrics, we stand ready to integrate it as our proof system of choice. Through such meticulous approaches, we not only reinforce the foundation of our project but also ensure its scalability and robustness in the ever-evolving landscape of blockchain technology.","s":"Introduction","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#introduction","p":338},{"i":343,"t":"The initial goal revolves around crafting a distinctive architecture that segregates public and private computations, employing an account-based framework for the public state and a UTXO-based structure for the private state. The UTXO model [1,2], notably utilized in Bitcoin, generates new UTXOs to serve future transactions, while the account-based paradigm assigns balances to accounts that transactions can modify. Although the UTXO model bolsters privacy by concealing comprehensive balances, the pursuit of a dual architecture mandates a meticulous synchronization of these state models, ensuring that private transactions remain inconspicuous in the wider public network state. This task is further complicated by the divergent transaction processing methods intrinsic to each model, necessitating a thoughtful and innovative approach to harmonize their functionality. To seamlessly bring together the dual architecture, harmonizing the account-based model for public state with the UTXO-based model for private state, a comprehensive strategy is essential. The concept of blending an account-based structure with a UTXO-based model for differentiating between public and private states is intriguing. It seeks to leverage the strengths of both models: the simplicity and directness of the account-based model with the privacy enhancements of the UTXO model. Here's a breakdown and a potential strategy for harmonizing these models:","s":"Goal 1: Create a State Separation Architecture","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-1-create-a-state-separation-architecture","p":338},{"i":345,"t":"Account-Based Model: This model is intuitive and easy to work with. Every participant has an account, and transactions directly modify the balances of these accounts. It's conducive for smart contracts and a broad range of applications. UTXO-Based Model: This model treats every transaction as a new output, which can then be used as an input for future transactions. By not explicitly associating transaction outputs with user identities, it offers a degree of privacy.","s":"Rationale Behind the Dual Architecture:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-rationale-behind-the-dual-architecture-","p":338},{"i":347,"t":"Translation Layer Role: Interface between UTXO and account-based states. UTXO-to-Account Adapter: When UTXOs are spent, the adapter can translate these into the corresponding account balance modifications. This could involve creating a temporary 'pseudo-account' that mirrors the UTXO's attributes. Account-to-UTXO Adapter: When an account wishes to make a private transaction, it would initiate a process converting a part of its balance to a UTXO, facilitating a privacy transaction. Unified Identity Management Role: Maintain a unified identity (or address) system that works across both state models, allowing users to easily manage their public and private states without requiring separate identities. Deterministic Wallets: Use Hierarchical Deterministic (HD) wallets [3,4], enabling users to generate multiple addresses (both UTXO and account-based) from a single seed. This ensures privacy while keeping management centralized for the user. State Commitments Role: Use cryptographic commitments to commit to the state of both models. This can help in efficiently validating cross-model transactions. Verkle Trees: Verkle Trees combine Vector Commitment and the KZG polynomial commitment scheme to produce a structure that's efficient in terms of both proofs and verification. Verkle proofs are considerably small in size (less data to store and transmit), where Transaction and state verifications can be faster due to the smaller proof sizes and computational efficiencies. Mimblewimble-style Aggregation [5]: For UTXOs, techniques similar to those used in Mimblewimble can be used to aggregate transactions, keeping the state compact and enhancing privacy. Batch Processing & Anonymity Sets Role: Group several UTXO-based private transactions into a single public account-based transaction. This can provide a level of obfuscation and can make synchronization between the two models more efficient. CoinJoin Technique [6]: As seen in Bitcoin, multiple users can combine their UTXO transactions into one, enhancing privacy. Tornado Cash Principle [7]: For account-based systems wanting to achieve privacy, methods like those used in Tornado Cash can be implemented, providing zk-SNARKs-based private transactions. Event Hooks & Smart Contracts Role: Implement event-driven mechanisms that trigger specific actions in one model based on events in the other. For instance, a private transaction (UTXO-based) can trigger a corresponding public notification or event in the account-based model. Conditional Execution: Smart contracts could be set to execute based on events in the UTXO system. For instance, a smart contract might release funds (account-based) once a specific UTXO is spent. Privacy Smart Contracts: Using zk-SNARKs or zk-STARKs to bring privacy to the smart contract layer, allowing for private logic execution.","s":"Harmonizing the Two Systems:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-harmonizing-the-two-systems-","p":338},{"i":349,"t":"Synchronization Overhead Challenge: Combining two distinct transaction models creates an inherent synchronization challenge. State Channels: By allowing transactions to be conducted off-chain between participants, state channels can alleviate synchronization stresses. Only the final state needs to be settled on-chain, drastically reducing the amount of data and frequency of updates required. Sidechains: These act as auxiliary chains to the main blockchain. Transactions can be processed on the sidechain and then periodically synced with the main chain. This structure helps reduce the immediate load on the primary system. Checkpointing: Introduce periodic checkpoints where the two systems' states are verified and harmonized. This can ensure consistency without constant synchronization. Double Spending Challenge: With two models operating in tandem, there's an increased risk of double-spending attacks. Multi-Signature Transactions: Implementing transactions that require signatures from both systems can prevent unauthorized movements. Cross-Verification Mechanisms: Before finalizing a transaction, it undergoes verification in both UTXO and account-based systems. If discrepancies arise, the transaction can be halted. Timestamping: By attaching a timestamp to each transaction, it's possible to order them sequentially, making it easier to spot and prevent double spending. Complexity in User Experience Challenge: The dual model, while powerful, is inherently complex. Abstracted User Interfaces: Design UIs that handle the complexity behind the scenes, allowing users to make transactions without needing to understand the nuances of the dual model. Guided Tutorials: Offer onboarding tutorials to acquaint users with the system's features, especially emphasizing when and why they might choose one transaction type over the other. Feedback Systems: Implement systems where users can provide feedback on any complexities or challenges they encounter. This real-time feedback can be invaluable for iterative design improvements. Security Challenge: Merging two systems can introduce unforeseen vulnerabilities. Threat Modeling: Regularly conduct threat modeling exercises to anticipate potential attack vectors, especially those that might exploit the interaction between the two systems. Layered Security Protocols: Beyond regular audits, introduce multiple layers of security checks. Each layer can act as a fail-safe if a potential threat bypasses another. Decentralized Watchtowers: These are third-party services that monitor the network for malicious activities. If any suspicious activity is detected, they can take corrective measures or raise alerts. Gas & Fee Management: Challenge: A dual model can lead to convoluted fee structures. Dynamic Fee Adjustment: Implement algorithms that adjust fees based on network congestion and transaction type. This can ensure fairness and prevent network abuse. Fee Estimation Tools: Provide tools that can estimate fees before a transaction is initiated. This helps users understand potential costs upfront. Unified Gas Stations: Design platforms where users can purchase or allocate gas for both transaction types simultaneously, simplifying the gas acquisition process. By addressing these challenges head-on with a detailed and systematic approach, it's possible to unlock the full potential of a dual-architecture system, combining the strengths of both UTXO and account-based models without their standalone limitations. Aspect Details Harmony - Advanced VM Development: Design tailored for private smart contracts. - Leverage Established Architectures: Use WASM or RISC-V to harness their versatile and encompassing nature suitable for zero-knowledge applications. - Support for UTXO & Account-Based Models: Enhance adaptability across various blockchain structures. Challenges - Adaptation Concerns: WASM and RISC-V weren't designed with zero-knowledge proofs as a primary focus, posing integration challenges. - Complexities with Newer Systems: Systems like (Super)Nova, STARKs, and Sangria are relatively nascent, adding another layer of intricacy to the integration. - Optimization Concerns: Ensuring that these systems are optimized for zero-knowledge proofs. Proposed Solutions - Integration of Nova: Consider Nova's proof system for its potential alignment with project goals. - Comprehensive Testing: Rigorously test and benchmark against alternatives like Halo2, Plonky, and Starky to validate choices. - Poseidon Recursion Technique: To conduct exhaustive performance tests, providing insights into each system's efficiency and scalability.","s":"Challenges and Solutions","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-challenges-and-solutions-","p":338},{"i":351,"t":"The second goal entails the creation of an advanced virtual machine by leveraging established mainstream instruction sets like WASM or RISC-V. Alternatively, the objective involves pioneering a new, specialized instruction set meticulously optimized for Zero-Knowledge applications. This initiative seeks to foster a versatile and efficient environment for executing computations within the privacy-focused context of the project. Both WASM and RISC-V exhibit adaptability to both UTXO and account-based models due to their encompassing nature as general-purpose instruction set architectures. WASM, operating as a low-level virtual machine, possesses the capacity to execute code derived from a myriad of high-level programming languages, and boasts seamless integration across diverse blockchain platforms. Meanwhile, RISC-V emerges as a versatile option, accommodating both models, and can be seamlessly integrated with secure enclaves like SGX or TEE, elevating the levels of security and privacy. However, it is crucial to acknowledge that employing WASM or RISC-V might present challenges, given their original design without specific emphasis on optimizing for Zero-Knowledge Proofs (ZKPs). Further complexity arises with the consideration of more potent proof systems like (Super)Nova, STARKs, and Sangria, which, while potentially addressing optimization concerns, necessitate extensive research and testing due to their relatively nascent status within the field. This accentuates the need for a judicious balance between established options and innovative solutions in pursuit of an architecture harmoniously amalgamating privacy, security, and performance. The ambition to build a powerful virtual machine tailored to zero-knowledge (ZK) applications is both commendable and intricate. The combination of two renowned instruction sets, WASM and RISC-V, in tandem with ZK, is an innovation that could redefine privacy standards in blockchain. Let's dissect the challenges and possibilities inherent in this goal: Established Mainstream Instruction Sets - WASM and RISC-V Strengths: WASM: Rooted in its ability to execute diverse high-level language codes, its potential for cross-chain compatibility makes it a formidable contender. Serving as a low-level virtual machine, its role in the blockchain realm is analogous to that of the Java Virtual Machine in the traditional computing landscape. RISC-V: This open-standard instruction set architecture has made waves due to its customizable nature. Its adaptability to both UTXO and account-based structures coupled with its compatibility with trusted execution environments like SGX and TEE augments its appeal, especially in domains that prioritize security and privacy. Challenges: Neither WASM nor RISC-V was primarily designed with ZKPs in mind. While they offer flexibility, they might lack the necessary optimizations for ZK-centric tasks. Adjustments to these architectures might demand intensive R&D efforts. Pioneering a New, Specialized Instruction Set Strengths: A bespoke instruction set can be meticulously designed from the ground up with ZK in focus, potentially offering unmatched performance and optimizations tailored to the project's requirements. Challenges: Crafting a new instruction set is a monumental task requiring vast resources, including expertise, time, and capital. It would also need to garner community trust and support over time. Contemporary Proof Systems - (Super)Nova, STARKs, Sangria Strengths: These cutting-edge systems, being relatively new, might offer breakthrough cryptographic efficiencies that older systems lack: designed with modern challenges in mind, they could potentially bridge the gap where WASM and RISC-V might falter in terms of ZKP optimization. Challenges: Their nascent nature implies a dearth of exhaustive testing, peer reviews, and potentially limited community support. The unknowns associated with these systems could introduce unforeseen vulnerabilities or complexities. While they could offer optimizations that address challenges presented by WASM and RISC-V, their young status demands rigorous vetting and testing. Mainstream (WASM, RISC-V) ZK-optimized (New Instruction Set) Existing Tooling YES NO Blockchain-focused NO YES Performant DEPENDS YES","s":"Goal 2: Virtual Machine Creation","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-2-virtual-machine-creation","p":338},{"i":353,"t":"Cryptography Libraries: ZKP applications rely heavily on specific cryptographic primitives. Neither WASM nor RISC-V natively supports all of these primitives. Thus, a comprehensive library of cryptographic functions, optimized for these platforms, needs to be developed. Parallel Execution: Given the heavy computational demands of ZKPs, leveraging parallel processing capabilities can optimize the time taken. Both WASM and RISC-V would need modifications to handle parallel execution of ZKP processes efficiently. Memory Management: ZKP computations can sometimes require significant amounts of memory, especially during the proof generation phase. Fine-tuned memory management mechanisms are essential to prevent bottlenecks.","s":"Optimization Concerns for WASM and RISC-V:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-optimization-concerns-for-wasm-and-risc-v-","p":338},{"i":355,"t":"Proof Size: Different systems generate proofs of varying sizes. A smaller proof size is preferable for blockchain applications to save on storage and bandwidth. The trade-offs between proof size, computational efficiency, and security need to be balanced. Universality: Some systems can support any computational statement (universal), while others might be tailored to specific tasks. A universal system can be more versatile for diverse applications on the blockchain. Setup Requirements: Certain ZKP systems, like zk-SNARKs, require a trusted setup, which can be a security concern. Alternatives like zk-STARKs don't have this requirement but come with other trade-offs.","s":"Emerging ZKP Optimized Systems Considerations:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-emerging-zkp-optimized-systems-considerations-","p":338},{"i":357,"t":"Iterative Development: Given the complexities, an iterative development approach can be beneficial. Start with a basic integration of WASM or RISC-V for general tasks and gradually introduce specialized ZKP functionalities. Benchmarking: Establish benchmark tests specifically for ZKP operations. This will provide continuous feedback on the performance of the system as modifications are made, ensuring optimization. External Audits & Research: Regular checks from cryptographic experts and collaboration with academic researchers can help in staying updated and ensuring secure implementations.","s":"Strategies for Integration:","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-strategies-for-integration-","p":338},{"i":359,"t":"The process of generating proofs for private state updates is vested in the hands of the user, aligning with our commitment to minimizing trust assumptions and enhancing privacy. Concurrently, the responsibility of verifying these proofs and executing public functions within the virtual machine can be effectively delegated to an external prover, a role that is incentivized to operate with utmost honesty and integrity. This intricate balance seeks to safeguard against information leakage, preserving the confidentiality of private transactions. Integral to this mechanism is the establishment of a robust incentivization framework. To ensure the prover’s steadfast commitment to performing tasks with honesty, we should introduce a mechanism that facilitates both rewards for sincere behavior and penalties for any deviation from the expected standards. This two-pronged approach serves as a compelling deterrent against dishonest behavior and fosters an environment of accountability. In addition to incentivization, a crucial consideration is the economic aspect of verification and execution. The verification process has been intentionally designed to be more cost-effective than execution. This strategic approach prevents potential malicious actors from exploiting the system by flooding it with spurious proofs, a scenario that could arise when the costs align favorably. By maintaining a cost balance that favors verification, we bolster the system’s resilience against fraudulent activities while ensuring its efficiency. In sum, our multifaceted approach endeavors to strike an intricate equilibrium between user-initiated proof creation, external verification, and incentivization. This delicate interplay of mechanisms ensures a level of trustworthiness that hinges on transparency, accountability, and economic viability. As a result, we are poised to cultivate an ecosystem where users’ privacy is preserved, incentives are aligned, and the overall integrity of the system is fortified against potential adversarial actions. To achieve the goals of user-initiated proof creation, external verification, incentivization, and cost-effective verification over execution, several options and mechanisms can be employed: User-Initiated Proof Creation: Users are entrusted with the generation of proofs for private state updates, thus ensuring greater privacy and reducing trust dependencies. Challenges: Maintaining the quality and integrity of the proofs generated by users. Ensuring that users have the tools and knowledge to produce valid proofs. Solutions: Offer extensive documentation, tutorials, and user-friendly tools to streamline the proof-generation process. Implement checks at the verifier's end to ensure the quality of proofs. External Verification by Provers: An external prover verifies the proofs and executes public functions within the virtual machine. Challenges: Ensuring that the external prover acts honestly. Avoiding centralized points of failure. Solutions: Adopt a decentralized verification approach, with multiple provers cross-verifying each other’s work. Use reputation systems to rank provers based on their past performances, creating a trust hierarchy. Incentivization Framework: A system that rewards honesty and penalizes dishonest actions, ensuring provers' commitment to the task. Challenges: Determining the right balance of rewards and penalties. Ensuring that the system cannot be gamed for undue advantage. Solutions1: Implement a dynamic reward system that adjusts based on network metrics and provers' performance. Use a staking mechanism where provers need to lock up a certain amount of assets. Honest behavior earns rewards, while dishonest behavior could lead to loss of staked assets. Economic Viability through Cost Dynamics: Making verification more cost-effective than execution to deter spamming and malicious attacks. Challenges: Setting the right cost metrics for both verification and execution. Ensuring that genuine users aren’t priced out of the system. Solutions: Use a dynamic pricing model, adjusting costs in real-time based on network demand. Implement gas-like mechanisms to differentiate operation costs and ensure fairness. Maintaining Trustworthiness: Create a system that's transparent, holds all actors accountable, and is economically sound. Challenges: Keeping the balance where users feel their privacy is intact, while provers feel incentivized. Ensuring the system remains resilient against adversarial attacks. Solutions: Implement layered checks and balances. Foster community involvement, allowing them to participate in decision-making, potentially through a decentralized autonomous organization (DAO). Each of these options can be combined or customized to suit the specific requirements of your project, striking a balance between user incentives, cost dynamics, and verification integrity. A thoughtful combination of these mechanisms ensures that the system remains robust, resilient, and conducive to the objectives of user-initiated proof creation, incentivized verification, and cost- effective validation. Aspect Details Design Principle - User Responsibility: Generating proofs for private state updates. - External Prover: Delegated the task of verifying proofs and executing public VM functions. Trust & Privacy - Minimized Trust Assumptions: Place proof generation in users' hands. - Enhanced Privacy: Ensure confidentiality of private transactions and prevent information leakage. Incentivization Framework - Rewards: Compensate honest behavior. - Penalties: Deter and penalize dishonest behavior. Economic Considerations - Verification vs. Execution: Make verification more cost-effective than execution to prevent spurious proofs flooding. - Cost Balance: Strengthen resilience against fraudulent activities and maintain efficiency. Outcome An ecosystem where: - Users' privacy is paramount. - Incentives are appropriately aligned. - The system is robust against adversarial actions.","s":"Goal 3: Proofs Creation and Verification","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-3-proofs-creation-and-verification","p":338},{"i":361,"t":"This goal centers on the establishment of a kernel-based architecture, akin to the model observed in ZEXE, to facilitate the attestation of accurate private function executions. This innovative approach employs recursion to construct a call stack, which is then validated through iterative recursive computations. At its core, this technique harnesses a recursive Succinct Non-Interactive Argument of Knowledge (SNARK) mechanism, where each function call’s proof accumulates within the call stack. The subsequent verification of this stack’s authenticity leverages recursive SNARK validation. While this method offers robust verification of private function executions, it’s essential to acknowledge its associated intricacies. The generation of SNARK proofs necessitates a substantial computational effort, which, in turn, may lead to elevated gas fees for users. Moreover, the iterative recursive computations could potentially exhibit computational expansion as the depth of recursion increases. This calls for a meticulous balance between the benefits of recursive verification and the resource implications it may entail. In essence, Goal 4 embodies a pursuit of enhanced verification accuracy through a kernel-based architecture. By weaving recursion and iterative recursive computations into the fabric of our system, we aim to establish a mechanism that accentuates the trustworthiness of private function executions, while conscientiously navigating the computational demands that ensue. To accomplish the goal of implementing a kernel-based architecture for recursive verification of private function executions, several strategic steps and considerations can be undertaken: recursion handling and depth management. Recursion Handling Call Stack Management: Implement a data structure to manage the call stack, recording each recursive function call’s details, parameters, and state. Proof Accumulation: Design a mechanism to accumulate proof data for each function call within the call stack. This includes cryptographic commitments, intermediate results, and cryptographic challenges. Ensure that the accumulated proof data remains secure and tamper-resistant throughout the recursion process. Intermediary SNARK Proofs: Develop an intermediary SNARK proof for each function call’s correctness within the call stack. This proof should demonstrate that the function executed correctly and produced expected outputs. Ensure that the intermediary SNARK proof for each recursive call can be aggregated and verified together, maintaining the integrity of the entire call stack. Depth management Depth Limitation: Define a threshold for the maximum allowable recursion depth based on the system’s computational capacity, gas limitations, and performance considerations. Implement a mechanism to prevent further recursion beyond the defined depth limit, safeguarding against excessive computational growth. Graceful Degradation: Design a strategy for graceful degradation when the recursion depth approaches or reaches the defined limit. This may involve transitioning to alternative execution modes or optimization techniques. Communicate the degradation strategy to users and ensure that the system gracefully handles scenarios where recursion must be curtailed. Resource Monitoring: Develop tools to monitor resource consumption (such as gas usage and computational time) as recursion progresses. Provide real-time feedback to users about the cost and impact of recursive execution. Dynamic Depth Adjustment: Consider implementing adaptive depth management that dynamically adjusts the recursion depth based on network conditions, transaction fees, and available resources. Utilize algorithms to assess the optimal recursion depth for efficient execution while adhering to gas cost constraints. Fallback Mechanisms: Create fallback mechanisms that activate if the recursion depth limit is reached or if the system encounters resource constraints. These mechanisms could involve alternative verification methods or delayed execution. User Notifications: Notify users when the recursion depth limit is approaching, enabling them to make informed decisions about the complexity of their transactions and potential resource usage. Goal 4 underscores the project's ambition to integrate the merits of a kernel-based architecture with recursive verifications to bolster the reliability of private function executions. While the approach promises robust outcomes, it's pivotal to maneuver through its intricacies with astute strategies, ensuring computational efficiency and economic viability. By striking this balance, the architecture can realize its full potential in ensuring trustworthy and efficient private function executions.","s":"Goal 4: Kernel-based Architecture Implementation","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-4-kernel-based-architecture-implementation","p":338},{"i":363,"t":"Goal 5 revolves around the meticulous design of a seamless interaction between public and private states within the blockchain ecosystem. This objective envisions achieving not only composability between contracts but also the harmonious integration of private and public functions. A notable challenge in this endeavor lies in the intricate interplay between public and private states, wherein the potential linkage of a private transaction to a public one raises concerns about unintended information leakage. The essence of this goal entails crafting an architecture that facilitates the dynamic interaction of different states while ensuring that the privacy and confidentiality of private transactions remain unbreached. This involves the formulation of mechanisms that enable secure composability between contracts, guaranteeing the integrity of interactions across different layers of functionality. A key focus of this goal is to surmount the challenge of information leakage by implementing robust safeguards. The solution involves devising strategies to mitigate the risk of revealing private transaction details when connected to corresponding public actions. By creating a nuanced framework that com- partmentalizes private and public interactions, the architecture aims to uphold privacy while facilitating seamless interoperability. Goal 5 encapsulates a multifaceted undertaking, calling for the creation of an intricate yet transparent framework that empowers users to confidently engage in both public and private functions, without compromising the confidentiality of private transactions. The successful realization of this vision hinges on a delicate blend of architectural ingenuity, cryptographic sophistication, and user-centric design. To achieve seamless interaction between public and private states, composability, and privacy preservation, a combination of solutions and approaches can be employed. In the table below, a comprehensive list of solutions that address these objectives: Solution Category Description Layer 2 Solutions Employ zk-Rollups, Optimistic Rollups, and state channels to handle private interactions off-chain and settle them on-chain periodically. Boost scalability and cut transaction costs. Intermediary Smart Contracts Craft smart contracts as intermediaries for secure public-private interactions. Use these to manage data exchange confidentially. Decentralized Identity & Pseudonymity Implement decentralized identity systems for pseudonymous interactions. Validate identity using cryptographic proofs. Confidential Sidechains & Cross-Chain Set up confidential sidechains and employ cross-chain protocols to ensure private and composability across blockchains. Temporal Data Structures Create chronological data structures for secure interactions. Utilize cryptographic methods for data integrity and privacy. Homomorphic Encryption & MPC Apply homomorphic encryption and MPC for computations on encrypted data and interactions between state layers. Commit-Reveal Schemes Introduce commit-reveal mechanisms for private transactions, revealing data only post necessary public actions. Auditability & Verifiability Use on-chain tools for auditing and verifying interactions. Utilize cryptographic commitments for third-party validation. Data Fragmentation & Sharding Fragment data across shards for private interactions and curtailed data exposure. Bridge shards securely with cryptography. Ring Signatures & CoinJoin Incorporate ring signatures and CoinJoin protocols to mask transaction details and mix transactions collaboratively.","s":"Goal 5: Seamless Interaction Design","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-5-seamless-interaction-design","p":338},{"i":365,"t":"The primary aim of Goal 6 is to weave key DeFi protocols, such as AMMs and staking, into a user-centric environment that accentuates privacy. This endeavor comes with inherent challenges, especially considering the heterogeneity of existing DeFi protocols, predominantly built on Ethereum. These variations in programming languages and VMs exacerbate the quest for interoperability. Furthermore, the success and functionality of DeFi protocols is closely tied to liquidity, which in turn is influenced by user engagement and the amount of funds locked into the system.","s":"Goal 6: Integration of DeFi Protocols with a Privacy-Preserving Framework","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#goal-6-integration-of-defi-protocols-with-a-privacy-preserving-framework","p":338},{"i":367,"t":"Pioneering Privacy-Centric DeFi Models: Initiate the development of AMMs and staking solutions that are inherently protective of users' transactional privacy and identity. Specialized Smart Contracts with Privacy: Architect distinct smart contracts infused with privacy elements, setting the stage for secure user interactions within this new, confidential DeFi landscape. Optimized User Interfaces: Craft interfaces that resonate with user needs, simplifying the journey through the private DeFi space without compromising on security. Tackling Interoperability: Deploy advanced bridge technologies and middleware tools to foster efficient data exchanges and guarantee operational harmony across a spectrum of programming paradigms and virtual environments. Design and enforce universal communication guidelines that bridge the privacy-centric DeFi entities with the larger DeFi world seamlessly. Enhancing and Sustaining Liquidity: Unveil innovative liquidity stimuli and yield farming incentives, compelling users to infuse liquidity into the private DeFi space. Incorporate adaptive liquidity frameworks that continually adjust based on the evolving market demands, ensuring consistent liquidity. Forge robust alliances with other DeFi stalwarts, jointly maximizing liquidity stores and honing sustainable token distribution strategies. Amplifying Community Engagement: Design and roll out enticing incentive schemes to rally users behind privacy-focused AMMs and staking systems, thereby nurturing a vibrant, privacy-advocating DeFi community. Through the integration of these approaches, we aim to achieve Goal 6, providing users with a privacy-focused platform for engaging effortlessly in core DeFi functions such as AMMs and staking, all while effectively overcoming the obstacles related to interoperability and liquidity concerns.","s":"Strategic Roadmap for Goal 6","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#-strategic-roadmap-for-goal-6-","p":338},{"i":369,"t":"In our quest to optimize privacy, we're proposing a Zero-Knowledge Virtual Machine (Zkvm) that harnesses the power of Zero-Knowledge Proofs (ZKPs). These proofs ensure that while private state data remains undisclosed, public state transitions can still be carried out and subsequently verified by third parties. This blend of public and private state is envisaged to be achieved through a state tree representing the public state, while the encrypted state leaves stand for the private state. Each user's private state indicates validity through the absence of a corresponding nullifier. A nullifier is a unique cryptographic value generated in privacy-preserving blockchain transactions to prevent double-spending, ensuring that each private transaction is spent only once without revealing its details. Private functions' execution mandates users to offer a proof underscoring the accurate execution of all encapsulated private calls. For validating a singular private function call, we're leaning into the kernel-based model inspired by the ZEXE protocol. Defined as kernel circuits, these functions validate the correct execution of each private function call. Due to their recursive circuit structure, a succession of private function calls can be executed by calculating proofs in an iterative manner. Execution-relevant data, like private and public call stacks and additions to the state tree, are incorporated as public inputs. Our method integrates the verification keys for these functions within a merkle tree. Here's the innovation: a user's ZKP showcases the existence of the verification key in this tree, yet keeps the executed function concealed. The unique function identifier can be presented as the verification key, with all contracts merkleized for hiding functionalities. We suggest a nuanced shift from the ZEXE protocol's identity function, which crafts an identity for smart contracts delineating its behavior, access timeframes, and other functionalities. Instead of the ZEXE protocol's structure, our approach pivots to a method anchored in the security of a secret combined with the uniqueness from hashing with the contract address. The underlying rationale is straightforward: the sender, equipped with a unique nonce and salt for the transaction, hashes the secret, payload, nonce, and salt. This result is then hashed with the contract address for the final value. The hash function's unidirectional nature ensures that the input cannot be deduced easily from its output. A specific concern, however, is the potential repetition of secret and payload values across transactions, which could jeopardize privacy. Yet, by embedding the function's hash within the hash of the contract address, users can validate a specific function's execution without divulging the function, navigating this limitation. Alternative routes do exist: We could employ signature schemes like ECDSA, focusing on uniqueness and authenticity, albeit at the cost of complex key management. Fully Homomorphic Encryption (FHE) offers another pathway, enabling function execution on encrypted data, or Multi-Party Computation (MPC) which guarantees non-disclosure of function or inputs. Yet, integrating ZKPs with either FHE or MPC presents a challenge. Combining cryptographic functions like SHA-3 and BLAKE2 can also bolster security and uniqueness. It's imperative to entertain these alternatives, especially when hashing might not serve large input/output functions effectively or might fall short in guaranteeing uniqueness.","s":"Summary of the Architecture","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#summary-of-the-architecture","p":338},{"i":371,"t":"Our aim is to revolutionize the privacy and security paradigms through Nescience. As we strive to set milestones and achieve groundbreaking advancements, our current focus narrows onto the realization of Goal 2 and Goal 3. Our endeavors to build a powerful virtual machine tailored for Zero-Knowledge applications have led us down the path of rigorous exploration and testing. We believe that integrating the right proof system is pivotal to our project's success, which brings us to Nova [8]. In our project journey, we have opted to integrate the Nova proof system, recognizing its potential alignment with our overarching goals. However, as part of our meticulous approach to innovation and optimization, we acknowledge the need to thoroughly examine Nova’s performance capabilities, particularly due to its status as a pioneering and relatively unexplored proof system. This critical evaluation entails a comprehensive process of benchmarking and comparative analysis [9], pitting Nova against other prominent proof systems in the field, including Halo2 [10], Plonky2 [11], and Starky [12]. This ongoing and methodical initiative is designed to ensure a fair and impartial assessment, enabling us to draw meaningful conclusions about Nova’s strengths and limitations in relation to its counterparts. By leveraging the Poseidon recursion technique, we are poised to conduct an exhaustive performance test that delves into intricate details. Through this testing framework, we aim to discern whether Nova possesses the potential to outshine its contemporaries in terms of efficiency, scalability, and overall performance. The outcome of this rigorous evaluation will be pivotal in shaping our strategic decisions moving forward. Armed with a comprehensive understanding of Nova’s performance metrics vis-à-vis other proof systems, we can confidently chart a course that maximizes the benefits of our project’s optimization efforts. Moreover, as we ambitiously pursue the establishment of a robust mechanism for proof creation and verification, our focus remains resolute on preserving user privacy, incentivizing honest behaviour, and ensuring the cost-effective verification of transactions. At the heart of this endeavor is our drive to empower users by allowing them the autonomy of generating proofs for private state updates, thereby reducing dependencies and enhancing privacy. We would like to actively work on providing comprehensive documentation, user-friendly tools, and tutorials to aid users in this intricate process. Parallelly, we're looking into decentralized verification processes, harnessing the strength of multiple external provers that cross-verify each other's work. Our commitment is further cemented by our efforts to introduce a dynamic reward system that adjusts based on network metrics and prover performance. This intricate balance, while challenging, aims to fortify our system against potential adversarial actions, aligning incentives, and preserving the overall integrity of the project.","s":"Current State","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"#current-state","p":338},{"i":373,"t":"[1] Nakamoto, S. (2008). Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved from https://bitcoin.org/bitcoin.pdf [2] Sanchez, F. (2021). Cardano’s Extended UTXO accounting model. Retrived from https://iohk.io/en/blog/posts/2021/03/11/cardanos-extended-utxo-accounting-model/ [3] Morgan, D. (2020). HD Wallets Explained: From High Level to Nuts and Bolts. Retrieved from https://medium.com/mycrypto/the-journey-from-mnemonic-phrase-to-address-6c5e86e11e14 [4] Wuille, P. (012). Bitcoin Improvement Proposal (BIP) 44. Retrieved from https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki [5] Jedusor, T. (2020). Introduction to Mimblewimble and Grin. Retrieved from https://github.com/mimblewimble/grin/blob/master/doc/intro.md [6] Bitcoin's official wiki overview of the CoinJoin method. Retrieved from https://en.bitcoin.it/wiki/CoinJoin [7] TornadoCash official Github page. Retrieved from https://github.com/tornadocash/tornado-classic-ui [8] Kothapalli, A., Setty, S., Tzialla, I. (2021). Nova: Recursive Zero-Knowledge Arguments from Folding Schemes. Retrieved from https://eprint.iacr.org/2021/370 [9] ZKvm Github page. Retrieved from https://github.com/vacp2p/zk-explorations [10] Electric Coin Company (2020). Explaining Halo 2. Retrieved from https://electriccoin.co/blog/explaining-halo-2/ [11] Polygon Labs (2022). Introducing Plonky2. Retrieved from https://polygon.technology/blog/introducing-plonky2 [12] StarkWare (2021). ethSTARK Documentation. Retrieved from https://eprint.iacr.org/2021/582 Incentive Mechanisms: Token Rewards: Design a token-based reward system where honest provers are compensated with tokens for their verification services. This incentivizes participation and encourages integrity. Staking and Slashing: Introduce a staking mechanism where provers deposit tokens as collateral. Dishonest behavior results in slashing (partial or complete loss) of the staked tokens, while honest actions are rewarded. Proof of Work/Proof of Stake: Implement a proof-of-work or proof-of- stake consensus mechanism for verification, aligning incentives with the blockchain’s broader consensus mechanism. ↩","s":"References","u":"/rlog/Nescience-A-zkVM-leveraging-hiding-properties","h":"","p":338},{"i":375,"t":"A research log. Reliable and decentralized, pick two. Together with decanus, I've been working on the problem of data sync lately. In building p2p messaging systems, one problem you quickly come across is the problem of reliably transmitting data. If there's no central server with high availability guarantees, you can't meaningfully guarantee that data has been transmitted. One way of solving this problem is through a synchronization protocol. There are many synchronization protocols out there and I won't go into detail of how they differ with our approach here. Some common examples are Git and Bittorrent, but there are also projects like IPFS, Swarm, Dispersy, Matrix, Briar, SSB, etc.","s":"P2P Data Sync for Mobile","u":"/rlog/p2p-data-sync-for-mobile","h":"","p":374},{"i":377,"t":"Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices.","s":"Problem motivation","u":"/rlog/p2p-data-sync-for-mobile","h":"#problem-motivation","p":374},{"i":379,"t":"For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from. Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat.","s":"Why p2p?","u":"/rlog/p2p-data-sync-for-mobile","h":"#why-p2p","p":374},{"i":381,"t":"At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on.","s":"Why reliably sync data?","u":"/rlog/p2p-data-sync-for-mobile","h":"#why-reliably-sync-data","p":374},{"i":383,"t":"Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence. More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc.","s":"Why mobilephones?","u":"/rlog/p2p-data-sync-for-mobile","h":"#why-mobilephones","p":374},{"i":385,"t":"For requirements or design goals for a solution, here's what we came up with. MUST sync data reliably between devices. By reliably we mean having the ability to deal with messages being out of order, dropped, duplicated, or delayed. MUST NOT rely on any centralized services for reliability. By centralized services we mean any single point of failure that isn’t one of the endpoint devices. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices that are resource restricted, mostly-offline and often changing network. MAY use helper services in order to be more mobile-friendly. Examples of helper services are decentralized file storage solutions such as IPFS and Swarm. These help with availability and latency of data for mostly-offline devices. MUST have the ability to provide casual consistency. By casual consistency we mean the commonly accepted definition in distributed systems literature. This means messages that are casually related can achieve a partial ordering. MUST support ephemeral messages that don’t need replication. That is, allow for messages that don’t need to be reliabily transmitted but still needs to be transmitted between devices. MUST allow for privacy-preserving messages and extreme data loss. By privacy-preserving we mean things such as exploding messages (self-destructing messages). By extreme data loss we mean the ability for two trusted devices to recover from a, deliberate or accidental, removal of data. MUST be agnostic to whatever transport it is running on. It should not rely on specific semantics of the transport it is running on, nor be tightly coupled with it. This means a transport can be swapped out without loss of reliability between devices.","s":"Minimal Requirements","u":"/rlog/p2p-data-sync-for-mobile","h":"#minimal-requirements","p":374},{"i":387,"t":"The first minimum viable version is in an alpha stage, and it has a specification, implementation and we have deployed it in a console client for end to end functionality. It's heavily inspired by Bramble Sync Protocol. The spec is fairly minimal. You have nodes that exchange records over some secure transport. These records are of different types, such as OFFER, MESSAGE, REQUEST, and ACK. A peer keep tracks of the state of message for each node it is interacting with. There's also logic for message retransmission with exponential delay. The positive ACK and retransmission model is quite similar to how TCP is designed. There are two different modes of syncing, interactive and batch mode. See sequence diagrams below. Interactive mode: Batch mode: Which mode should you choose? It's a tradeoff of latency and bandwidth. If you want to minimize latency, batch mode is better. If you care about preserving bandwidth interactive mode is better. The choice is up to each node.","s":"MVDS - a minimium viable version","u":"/rlog/p2p-data-sync-for-mobile","h":"#mvds---a-minimium-viable-version","p":374},{"i":389,"t":"Initial ad hoc bandwidth and latency testing shows some issues with a naive approach. Running with the default simulation settings: communicating nodes: 2 nodes using interactive mode: 2 interval between messages: 5s time node is offine: 90% nodes each node is sharing with: 2 we notice a huge overhead. More specifically, we see a ~5 minute latency overhead and a bandwidth multiplier of x100-1000, i.e. 2-3 orders of magnitude just for receiving a message with interactive mode, without acks. Now, that seems terrible. A moment of reflection will reveal why that is. If each node is offline uniformly 90% of the time, that means that each record will be lost 90% of the time. Since interactive mode requires offer, request, payload (and then ack), that's three links just for Bob to receive the actual message. Each failed attempt implies another retransmission. That means we have (1/0.1)^3 = 1000 expected overhead to receive a message in interactive mode. The latency follows naturally from that, with the retransmission logic.","s":"Basic simulation","u":"/rlog/p2p-data-sync-for-mobile","h":"#basic-simulation","p":374},{"i":391,"t":"The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost. There are a few ways of getting around this issue. One is having a store and forward model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status. Another, arguably more pure and robust, way is having a remote log, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS. What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are required to get reliable data transmission.","s":"Mostly-offline devices","u":"/rlog/p2p-data-sync-for-mobile","h":"#mostly-offline-devices","p":374},{"i":393,"t":"While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions: two nodes exchanging a single message in batch mode 10% uniformly random uptime for each node in HA cache case, 100% uptime of a piece of infrastructure C retransmission every epoch (with constant or exponential backoff) only looking at average (p50) case First case, no helper services​ A sends a message to B, and B acks it. A message -> B (10% chance of arrival) A <- ack B (10% chance of arrival) With a constant backoff, A will send messages at epoch 1, 2, 3, .... With exponential backoff and a multiplier of 2, this would be 1, 2, 4, 8, .... Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier. There's a difference between time to receive and time to stop sending. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore: A will send messages until it receives an ACK. B will send ACK if it receives a message. To get an average of one ack through, A needs to send 100 messages, and B send on average 10 acks. That's a multiplier of roughly a 100. That's roughly what we saw with the simulation above for receiving a message in interactive mode. Second case, high-availability caching layer​ Let's introduce a helper node or piece of infrastructure, C. Whenever A or B sends a message, it also sends it to C. Whenever A or B comes online, it queries for messages with C. A message -> B (10% chance of arrival) A message -> C (100% chance of arrival) B <- req/res -> C (100% chance of arrival) A <- ack B (10% chance of arrival) C <- ack B (100% chance of arrival) A <- req/res -> C (100% chance of arrival) What's the probability that A's messages will arrive at B? Directly, it's still 10%. But we can assume it's 100% that C picks up the message. (Giving C a 90% chance success rate doesn't materially change the numbers). B will pick up A's message from C after an average of 10 epochs. Then B will send ack to A, which will also be picked up by C 100% of the time. Once A comes online again, it'll query C and receive B's ack. Assuming we use exponential backoff with a multiplier of 2, A will send a message directly to B at epoch 1, 2, 4, 8 (assuming it is online). At this point, epoch 10, B will be online in the average case. These direct sends will likely fail, but B will pick the message up from C and send one ack, both directly to A and to be picked up by C. Once A comes online, it'll query C and receive the ack from B, which means it won't do any more retransmits. How many messages have been sent? Not counting interactions with C, A sends 4 (at most) and B 1. Depending on if the interaction with C is direct or indirect (i.e. multicast), the factor for interaction with C will be ~2. This means the total bandwidth multiplier is likely to be <10, which is a lot more acceptable. Since the syncing semantics are end-to-end, this is without relying on the reliablity of C. Caveat​ Note that both of these are probabilistic argument. They are also based on heuristics. More formal analysis would be desirable, as well as better simulations to experimentally verify them. In fact, the calculations could very well be wrong!","s":"Basic calculations for bandwidth multiplier","u":"/rlog/p2p-data-sync-for-mobile","h":"#basic-calculations-for-bandwidth-multiplier","p":374},{"i":395,"t":"There are many enhancements that can be made and are desirable. Let's outline a few. Data sync clients. Examples of actual usage of data sync, with more interesting domain semantics. This also includes usage of sequence numbers and DAGs to know what content is missing and ought to be synced. Remote log. As alluded to above, this is necessary. It needs a more clear specification and solid proof of concepts. More efficient ways of syncing with large number of nodes. When the number of nodes goes up, the algorithmic complexity doesn't look great. This also touches on things such as ambient content discovery. More robust simulations and real-world deployments. Exisiting simulation is ad hoc, and there are many improvements that can be made to gain more confidence and identify issues. Additionally, better formal analysis. Example usage over multiple transports. Including things like sneakernet and meshnets. The described protocol is designed to work over unstructured, structured and private p2p networks. In some cases it can leverage differences in topology, such as multicast, or direct connections.","s":"Future work","u":"/rlog/p2p-data-sync-for-mobile","h":"#future-work","p":374},{"i":398,"t":"May 13, 2024 by Aaryamann 7 min read","s":"RLN-v3: Towards a Flexible and Cost-Efficient Implementation","u":"/rlog/page/2","h":"","p":396},{"i":400,"t":"May 3, 2024 by Aaryamann 5 min read","s":"Verifying RLN Proofs in Light Clients with Subtrees","u":"/rlog/page/2","h":"","p":396},{"i":402,"t":"November 7, 2023 by Aaryamann 7 min read","s":"Strengthening Anonymous DoS Prevention with Rate Limiting Nullifiers in Waku","u":"/rlog/page/2","h":"","p":396},{"i":404,"t":"November 6, 2023 by Umar Farooq 14 min read","s":"GossipSub Improvements: Evolution of Overlay Design and Message Dissemination in Unstructured P2P Networks","u":"/rlog/page/2","h":"","p":396},{"i":406,"t":"August 28, 2023 by Moudy 32 min read","s":"Nescience - A zkVM leveraging hiding properties","u":"/rlog/page/2","h":"","p":396},{"i":408,"t":"April 24, 2023 by Richard 5 min read","s":"Device Pairing in Js-waku and Go-waku","u":"/rlog/page/2","h":"","p":396},{"i":410,"t":"April 3, 2023 by Franck 6 min read","s":"The Future of Waku Network: Scaling, Incentivization, and Heterogeneity","u":"/rlog/page/2","h":"","p":396},{"i":412,"t":"November 8, 2022 by Franck 7 min read","s":"Waku for All Decentralized Applications and Infrastructures","u":"/rlog/page/2","h":"","p":396},{"i":414,"t":"November 4, 2022 by Oskar 19 min read","s":"Building Privacy-Protecting Infrastructure","u":"/rlog/page/2","h":"","p":396},{"i":416,"t":"July 22, 2022 by Daniel 17 min read","s":"Waku Privacy and Anonymity Analysis Part I: Definitions and Waku Relay","u":"/rlog/page/2","h":"","p":396},{"i":419,"t":"May 17, 2022 by s1fr0 22 min read","s":"Noise handshakes as key-exchange mechanism for Waku","u":"/rlog/page/3","h":"","p":417},{"i":421,"t":"May 9, 2022 by Daniel 18 min read","s":"Waku v2 Ambient Peer Discovery","u":"/rlog/page/3","h":"","p":417},{"i":423,"t":"April 12, 2022 by Hanno Cornelius 11 min read","s":"Introducing nwaku","u":"/rlog/page/3","h":"","p":417},{"i":425,"t":"December 3, 2021 by Circe 12 min read","s":"Opinion: Pseudo-ethics in the Surveillance Tech Industry","u":"/rlog/page/3","h":"","p":417},{"i":427,"t":"November 3, 2021 by Hanno Cornelius 10 min read","s":"Waku v1 vs Waku v2: Bandwidth Comparison","u":"/rlog/page/3","h":"","p":417},{"i":429,"t":"August 6, 2021 by Oskar 8 min read","s":"[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging","u":"/rlog/page/3","h":"","p":417},{"i":431,"t":"June 4, 2021 by Franck 7 min read","s":"Presenting JS-Waku: Waku v2 in the Browser","u":"/rlog/page/3","h":"","p":417},{"i":433,"t":"March 5, 2021 by Sanaz 21 min read","s":"Privacy-preserving p2p economic spam protection in Waku v2","u":"/rlog/page/3","h":"","p":417},{"i":435,"t":"November 10, 2020 by Oskar 10 min read","s":"[Talk] Vac, Waku v2 and Ethereum Messaging","u":"/rlog/page/3","h":"","p":417},{"i":437,"t":"September 28, 2020 by Oskar 8 min read","s":"Waku v2 Update","u":"/rlog/page/3","h":"","p":417},{"i":440,"t":"July 1, 2020 by Oskar 14 min read","s":"What's the Plan for Waku v2?","u":"/rlog/page/4","h":"","p":438},{"i":442,"t":"April 27, 2020 by Dean 6 min read","s":"Feasibility Study: Discv5","u":"/rlog/page/4","h":"","p":438},{"i":444,"t":"April 16, 2020 by Oskar 26 min read","s":"What Would a WeChat Replacement Need?","u":"/rlog/page/4","h":"","p":438},{"i":446,"t":"April 9, 2020 by Dean 9 min read","s":"From Kademlia to Discv5","u":"/rlog/page/4","h":"","p":438},{"i":448,"t":"February 14, 2020 by Oskar 6 min read","s":"Waku Update","u":"/rlog/page/4","h":"","p":438},{"i":450,"t":"February 7, 2020 by Dean 6 min read","s":"DNS Based Discovery","u":"/rlog/page/4","h":"","p":438},{"i":452,"t":"December 3, 2019 by Oskar 10 min read","s":"Fixing Whisper with Waku","u":"/rlog/page/4","h":"","p":438},{"i":454,"t":"November 8, 2019 by Oskar 8 min read","s":"Feasibility Study: Semaphore rate limiting through zkSNARKs","u":"/rlog/page/4","h":"","p":438},{"i":456,"t":"October 4, 2019 by Oskar 5 min read","s":"P2P Data Sync with a Remote Log","u":"/rlog/page/4","h":"","p":438},{"i":458,"t":"August 2, 2019 by Oskar 6 min read","s":"Vac - A Rough Overview","u":"/rlog/page/4","h":"","p":438},{"i":461,"t":"July 19, 2019 by Oskar 12 min read","s":"P2P Data Sync for Mobile","u":"/rlog/page/5","h":"","p":459},{"i":463,"t":"This article introduces MDSECheck method — a novel approach to checking square MDS matrices for unconditional security as the components of affine permutation layers of P-SP-networks.","s":"The MDSECheck method: choosing secure square MDS matrices for P-SP-networks","u":"/rlog/mdsecheck-method","h":"","p":462},{"i":465,"t":"Maximum distance separable (MDS) matrices play a significant role in algebraic coding theory and symmetric cryptography. In particular, square MDS matrices are commonly used in affine permutation layers of partial substitution-permutation networks (P-SPNs). These are widespread designs of the modern symmetric ciphers and hash functions. A classic example of the latter is Poseidon [1], a well-known hash function used in zk-SNARK proving systems. Square MDS matrices differ in terms of security that they are able to provide for P-SPNs. The use of some such matrices in certain P-SPNs may result in existence of infinitely long subspace trails of small period for the latter, which make them vulnerable to differential cryptanalysis [2]. Two methods for security checking of square MDS matrices for P-SPNs have been proposed in [2]. The first one, which is referred to as the three tests method in the rest of the article, is aimed at security checking for a specified structure of the substitution layer of a P-SPN. The second method, which is referred here as the sufficient test method, has been designed to determine whether a square MDS matrix satisfies a sufficient condition of being secure regardless of the structure of a P-SPN substitution layer, i.e. to check whether the matrix belongs to the class of square MDS matrices, which are referred to as unconditionally secure in the current article. This article aims to introduce MDSECheck method — a novel approach to checking square MDS matrices for unconditional security, which has already been implemented in the Rust programming language as the library crate [3]. The next sections explain the notions mentioned above, describe the MDSECheck method as well as its mathematical foundations, provide a brief overview of the MDSECheck library crate and outline possible future research directions.","s":"Introduction","u":"/rlog/mdsecheck-method","h":"#introduction","p":462},{"i":467,"t":"An mmm x nnn matrix MMM over a finite field is called MDS, if and only if for distinct nnn-dimensional column vectors v1v_1v1​ and v2v_2v2​ the column vectors v1 ∣ Mv1v_1 \\: | \\: M v_1v1​∣Mv1​ and v2 ∣ Mv2v_2 \\: | \\: M v_2v2​∣Mv2​, where ∣|∣ stands for vertical concatenation, do not coincide in nnn or more components. The set of all possible column vectors v ∣ Mvv \\: | \\: M vv∣Mv for some fixed matrix MMM is a systematic MDS code, i.e. a linear code, which contains input symbols on their original positions and achieves the Singleton bound. The latter property results in good error-correction capability. There are several equivalent definitions of MDS matrices, but the next one is especially useful for constructing them directly by means of algebraic methods. A matrix over a finite field is called MDS, if and only if all its square submatrices are nonsingular. The matrix entries and the matrix itself are also considered submatrices. One of the most efficient and straightforward methods to directly construct an MDS matrix is generating a Cauchy matrix [4]. Such an mmm x nnn matrix is defined using mmm-dimensional vector xxx and nnn-dimensional vector yyy, for which all entries in the concatenation of xxx and yyy are distinct. The entries of the Cauchy matrix are described by the formula Mi,j=1 / (xi−yj)M_{i, j} = 1 \\: / \\: (x_i - y_j)Mi,j​=1/(xi​−yj​). It is obvious that any submatrix of a Cauchy matrix is also a Cauchy matrix. The Cauchy determinant formula [5] implies that every square Cauchy matrix is nonsingular. Thus, Cauchy matrices satisfy the second definition of MDS matrices.","s":"MDS matrix: how to define and construct","u":"/rlog/mdsecheck-method","h":"#mds-matrix-how-to-define-and-construct","p":462},{"i":469,"t":"Describing SPNs in algebraic terms is convenient, so this approach has been chosen for this article. SPNs are designs of the symmetric cryptoprimitives, which operate on an internal state, which is represented as an nnn-dimensional vector over some finite field, and update this state iteratively by means of the round transformations described below. Each round begins with an optional update of the internal state by adding to its components some input data or extraction of some of these components as the output data. This optional step depends on the specific cryptoprimitive and the current round number. The next step is called the nonlinear substitution layer and lies in replacing the iii-th component of the internal state with Si(c)S_i(c)Si​(c) for each i∈[1..n]i \\in [1..n]i∈[1..n], where ccc is the component value and Si(x)S_i(x)Si​(x) is a nonlinear invertible function over the finite field. The function Si(x)S_i(x)Si​(x) is specific to the cryptoprimitive and called an S-Box. The final step, which is known as the affine permutation layer, replaces the internal state with MX+cM X + cMX+c, where XXX is the current internal state, MMM is a nonsingular square matrix and ccc is the vector of the round constants. The value of ccc is specific to the cryptoprimitive and the current round number, while MMM typically depends only on the cryptoprimitive. The data flow diagram for an SPN is given below. .................................. │ │ │ │ ▼ ▼ ▼ ▼ ┌────────────────────────────────┐ │ Optional addition / extraction │ <─────> Input / output └──┬────────┬────────┬────────┬──┘ ▼ ▼ ▼ ▼ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │S₁(x)│ │S₂(x)│ │ ... │ │Sₙ(x)│ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ ▼ ▼ ▼ ▼ ┌────────────────────────────────┐ │ Affine permutation │ └──┬────────┬────────┬────────┬──┘ ▼ ▼ ▼ ▼ ┌────────────────────────────────┐ │ Optional addition / extraction │ <─────> Input / output └──┬────────┬────────┬────────┬──┘ ▼ ▼ ▼ ▼ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │S₁(x)│ │S₂(x)│ │ ... │ │Sₙ(x)│ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ ▼ ▼ ▼ ▼ ┌────────────────────────────────┐ │ Affine permutation │ └──┬────────┬────────┬────────┬──┘ ▼ ▼ ▼ ▼ .................................. Partial SPNs are modifications of SPNs, where for certain rounds some S-Boxes are replaced with the identity functions to reduce computational efforts [2]. For example, the nonlinear substitution layers of the partial rounds of Poseidon update only the first internal state component [1]. In the case of P-SPNs, security considerations commonly demand to choose MMM as a square MDS matrix, because these matrices provide perfect diffusion property for the affine permutation layer [6]. Possessing this property means ensuring that any two nnn-dimensional internal states, which differ in exactly ttt components, are mapped by the affine permutation layer to two new internal states that differ in at least n−t+1n - t + 1n−t+1 components.","s":"Partial substitution-permutation networks","u":"/rlog/mdsecheck-method","h":"#partial-substitution-permutation-networks","p":462},{"i":471,"t":"Certain square MDS matrices should not be used in certain P-SPNs to avoid making them vulnerable to differential cryptanalysis, since it may exploit the existence of infinitely long subspace trails of small period for vulnerable P-SPNs. [2]. Such matrices are called insecure with respect to particular P-SPNs. An infinitely long subspace trail of period lll exists for a P-SPN, if and only if there is a proper subspace of differences of internal state vectors, such that if for a pair of initial internal states the difference belongs to this subspace, then the difference for the new internal states, which are obtained from the initial ones by means of the same lll-round transformation, also belongs to this subspace [2]. Two methods for checking square MDS matrices for suitability for P-SPNs in terms of existence of infinitely long subspace trails have been proposed in [2]. The three tests method is aimed at checking whether using a specified matrix for a P-SPN with a specified structure of the substitution layer leads to existence of infinitely long subspace trails of period ppp for this P-SPN for all ppp no larger than a given lll. The sufficient test method has been designed to determine whether a square MDS matrix satisfies a sufficient condition of non-existence of infinitely long subspace trails of period ppp for P-SPNs using this matrix for all ppp no larger than a specified lll. The sufficient test method is a direct consequence of Theorem 8 in [2] and consists in checking that the minimal polynomial of the ppp-th power of the tested matrix has maximum degree and is irreducible for all p∈[1..l]p \\in [1..l]p∈[1..l]. The aforesaid sufficient non-existence condition is satisfied by the matrix, if and only if all the checks yield positive results. It is convenient to define the unconditional P-SPN security level of the square MDS matrix as follows: this level is lll for the matrix MMM, if and only if the minimal polynomials of MMM, M2M^2M2, ........., MlM^lMl have maximum degree and are irreducible, but for Ml + 1M^{l \\: + \\: 1}Ml+1 the minimal polynomial does not have this property. Using this definition, the purpose of the sufficient test method can be described as checking whether the unconditional P-SPN security level of the specified matrix is no less than a given bound.","s":"Square MDS matrix security check in the context of P-SPNs","u":"/rlog/mdsecheck-method","h":"#square-mds-matrix-security-check-in-the-context-of-p-spns","p":462},{"i":473,"t":"The MDSECheck method, whose name is derived from the words \"MDS\", \"security\", \"elaborated\" and \"check\", has the same purpose as the sufficient test method, but achieves it differently. The differences of the first method from the latter and approaches to implementing them can be described as follows: Computation and verification of minimal polynomials of M2M^2M2, M3M^3M3, ........., MlM^lMl, where MMM is the tested nnn x nnn matrix over GF(q)GF(q)GF(q) and lll is the security level bound, has been replaced with checks for the corresponding powers of a root of the characteristic polynomial of MMM for non-presence in nontrivial subfields of GF(qn)GF(q^n)GF(qn). The non-presence check is performed without straightforward consideration of all nontrivial subfields of GF(qn)GF(q^n)GF(qn). The root is checked only for non-presence in the subfields GF(qn / p1)GF(q^{n \\: / \\: p_1})GF(qn/p1​), GF(qn / p2)GF(q^{n \\: / \\: p_2})GF(qn/p2​), ........., GF(qn / pd)GF(q^{n \\: / \\: p_d})GF(qn/pd​), where p1p_1p1​, p2p_2p2​, ........., pdp_dpd​ are all prime divisors of nnn. The non-presence check reuses some data computed during the checking for irreducibility the minimal polynomial of MMM, which in this case coincides with f(y)f(y)f(y) designating the characteristic polynomial of MMM. The values of yqn / pjmod f(y)y^{q^{n \\: / \\: p_j}} \\mod f(y)yqn/pj​modf(y) are saved for each j∈[1..d]j \\in [1..d]j∈[1..d] during the irreducibility check to replace exponentiations with sequential computations of (yi)qn / pjmod f(y)(y^i)^{q^{n \\: / \\: p_j}} \\mod f(y)(yi)qn/pj​modf(y) from (y(i − 1))qn / pjmod f(y)(y^{(i \\: - \\: 1)})^{q^{n \\: / \\: p_j}} \\mod f(y)(y(i−1))qn/pj​modf(y) as its product with yqn / pjmod f(y)y^{q^{n \\: / \\: p_j}} \\mod f(y)yqn/pj​modf(y). The check of the minimal polynomial of MMM for irreducibility and maximum degree is performed without unconditional computation of this polynomial. This computation has been replaced with the Krylov method fragment, which consists in building and solving only one system of linear equations over GF(q)GF(q)GF(q). If MMM has an irreducible minimal polynomial of maximum degree, then its coefficients are trivially determined from the system solution. If the system is degenerate, then the minimal polynomial of MMM does not have such properties. The correctness of the first distinctive feature can be proven as follows. Verifying that the minimal polynomial of a matrix is of maximum degree and irreducible is equivalent to verifying that the characteristic polynomial of this matrix is irreducible, because the minimal polynomial divides the characteristic one. Also, it is trivially proven that for a matrix with such a minimal polynomial it is equal to the characteristic polynomial. Thus, the required checks for the matrices M2M^2M2, M3M^3M3, ........., MlM^lMl can be done by checking their characteristic polynomials for irreducibility. Let MMM be nnn x nnn matrix over GF(q)GF(q)GF(q), whose minimal polynomial is of maximum degree and irreducible. The statements in the previous paragraph imply that f(y)f(y)f(y), which is the nnn-degree characteristic polynomial of MMM, is irreducible. Consider MMM over the extension field GF(qn)GF(q^n)GF(qn), which is the splitting field of f(y)f(y)f(y). Let α∈GF(qn)α \\in GF(q^n)α∈GF(qn) be a root of f(y)f(y)f(y). According to standard results from the Galois field theory, ααα, αqα^qαq, αq2α^{q^2}αq2, ........., αqn − 1α^{q^{n \\: - \\: 1}}αqn−1 are distinct roots of f(y)f(y)f(y) [7]. Thus, these powers of ααα are nnn distinct eigenvalues of MMM. Hence, due to matrix similarity properties, there is some matrix SSS such that SMS−1=DS M S^{-1} = DSMS−1=D, where DDD is the diagonal matrix, whose nonzero elements are ααα, αqα^qαq, αq2α^{q^2}αq2, ........., αqn − 1α^{q^{n \\: - \\: 1}}αqn−1. Therefore, SMiS−1=DiS M^i S^{-1} = D^iSMiS−1=Di, so the roots of the characteristic polynomial of MiM^iMi are αiα^iαi, (αq)i(α^q)^i(αq)i, (αq2)i(α^{q^2})^i(αq2)i, ........., (αqn − 1)i(α^{q^{n \\: - \\: 1}})^i(αqn−1)i. If the minimal polynomial of αiα^iαi has degree less than nnn, then the characteristic polynomial of MiM^iMi is divisible by this minimal polynomial, while αiα^iαi lies in some nontrivial subfield of GF(qn)GF(q^n)GF(qn). One of the fields isomorphic to this subfield is a residue class ring of polynomials modulo the minimal polynomial of αiα^iαi [7]. If the minimal polynomial of αiα^iαi is of degree nnn, then the characteristic polynomial of MiM^iMi equals this minimal polynomial and therefore is irreducible, while αiα^iαi does not lie in any nontrivial subfield of GF(qn)GF(q^n)GF(qn). In this case, 111, αiα^iαi, (αi)2(α^i)^2(αi)2, ........., (αi)n − 1(α^i)^{n \\: - \\: 1}(αi)n−1 are linearly independent as distinct roots of an irreducible polynomial over a finite field [7], so any field containing αiα^iαi has at least qnq^nqn elements and therefore cannot be a trivial subfield of GF(qn)GF(q^n)GF(qn). Thus, checking the characteristic polynomials of the matrices M2M^2M2, M3M^3M3, ........., MlM^lMl for irreducibility is equivalent to verifying that α2α^2α2, α3α^3α3, ........., αlα^lαl do not lie in any nontrivial subfield of GF(qn)GF(q^n)GF(qn). The last sentences of the two previous paragraphs imply the following: verifying that the minimal polynomials of M2M^2M2, M3M^3M3, ........., MlM^lMl are of maximum degree and irreducible can be performed by verifying that the corresponding powers of a root of the characteristic polynomial of the nnn x nnn matrix MMM over GF(q)GF(q)GF(q) do not belong to any nontrivial subfield of GF(qn)GF(q^n)GF(qn). ■\\blacksquare■ The approaches to implementing the first distinctive feature can be explained and proven to be correct as follows. Since GF(qw)GF(q^w)GF(qw) is a nontrivial subfield of GF(qu)GF(q^u)GF(qu) if and only if www divides uuu and w