<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.0.1">Jekyll</generator><link href="https://vac.dev/feed.xml" rel="self" type="application/atom+xml" /><link href="https://vac.dev/" rel="alternate" type="text/html" /><updated>2022-01-11T20:47:38+00:00</updated><id>https://vac.dev/feed.xml</id><title type="html">Vac</title><subtitle>Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging.</subtitle><entry><title type="html">Opinion: Pseudo-ethics in the Surveillance Tech Industry</title><link href="https://vac.dev/ethics-surveillance-tech" rel="alternate" type="text/html" title="Opinion: Pseudo-ethics in the Surveillance Tech Industry" /><published>2021-12-03T08:00:00+00:00</published><updated>2021-12-03T08:00:00+00:00</updated><id>https://vac.dev/ethics-surveillance-tech</id><content type="html" xml:base="https://vac.dev/ethics-surveillance-tech">&lt;p&gt;&lt;em&gt;This is an opinion piece by pseudonymous contributor, circe.&lt;/em&gt;&lt;/p&gt;

&lt;h2 id=&quot;preface&quot;&gt;Preface&lt;/h2&gt;

&lt;p&gt;The Vac team aims to provide a public good in the form of freely available, open source tools and protocols for decentralized communication.
As such, we value our independence and the usefulness of our protocols for a wide range of applications.
At the same time, we realize that all technical development, including ours, has a moral component.
As a diverse team we are guided by a shared devotion to the principles of human rights and liberty.
This explains why we place such a high premium on security, censorship-resistance and privacy -
a stance we &lt;a href=&quot;https://our.status.im/our-principles/&quot;&gt;share with the wider Status Network&lt;/a&gt;.
The post below takes a different approach from our usual more technical analyses,
by starting to peel back the curtain on the ethical shortfalls of the global surveillance tech industry.&lt;/p&gt;

&lt;h2 id=&quot;spotlight-on-an-industry&quot;&gt;Spotlight on an industry&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;https://www.apple.com/newsroom/2021/11/apple-sues-nso-group-to-curb-the-abuse-of-state-sponsored-spyware/&quot;&gt;Apple’s announcement&lt;/a&gt; of their lawsuit against Israel’s NSO Group
marks the latest in a series of recent setbacks for the surveillance tech company.
In early November, the &lt;a href=&quot;https://public-inspection.federalregister.gov/2021-24123.pdf&quot;&gt;United States blacklisted the firm&lt;/a&gt;,
citing concerns about the use of their spyware by foreign governments targeting civilians such as “journalists, businesspeople, activists” and more.
The company is already &lt;a href=&quot;https://www.reuters.com/article/us-facebook-cyber-whatsapp-nsogroup-idUSKBN1X82BE&quot;&gt;embroiled in a lawsuit with Whatsapp&lt;/a&gt;
over their exploit of the chat app’s video calling service to install malware on target devices.
NSO Group’s most infamous product, &lt;a href=&quot;https://forbiddenstories.org/case/the-pegasus-project/&quot;&gt;Pegasus&lt;/a&gt;, operates as a hidden exploit installed on victims’ mobile phones,
sometimes without even requiring as much as an unguarded click on a malicious link.
It has the potential to lay bare, and report to its owners, &lt;em&gt;everything&lt;/em&gt; within the reach of the infected device.
For most people this amounts to a significant portion of their private lives and thoughts.
Pegasus can read your private messages (even encrypted), collect your passwords, record calls, track your location and access your device’s microphone and camera.
No activity or application on an infected phone would be hidden.&lt;/p&gt;

&lt;p&gt;The latest controversies are perhaps less because of the novelty of the revelations -
the existence of Pegasus has been known to civil activists &lt;a href=&quot;https://www.bbc.com/news/technology-37192670&quot;&gt;since at least 2016&lt;/a&gt;.
Rather, the public was reminded again of the potential scope of surveillance tech
in the indiscriminate use of Pegasus on private citizens.
This has far-reaching implications for human freedoms worldwide.
Earlier this year, a &lt;a href=&quot;https://www.theguardian.com/world/2021/jul/18/revealed-leak-uncovers-global-abuse-of-cyber-surveillance-weapon-nso-group-pegasus&quot;&gt;leaked list of over 50,000 targets&lt;/a&gt;, or possible targets, of Pegasus included
the phone numbers of human rights advocates, independent journalists, lawyers and political activists.
This should have come as no surprise.
The type of autocratically inclined agents, and governments, who would venture to buy and use such invasive cyber-arms often target those they find politically inconvenient.
Pegasus, and similar technologies, simply extend the reach and capacity of such individuals and governments -
no border or distance, no political rank or social advantage, no sanctity of profession or regard for dignity,
provide any indemnity from becoming a victim.
Your best hope is to remain uninteresting enough to escape consideration.&lt;/p&gt;

&lt;p&gt;The NSO Group has, of course, denied allegations of culpability and questions the authenticity of the list.
At this stage, the latter is almost beside the point:
Amnesty International’s cybersecurity team, Security Lab, &lt;em&gt;did&lt;/em&gt; find &lt;a href=&quot;https://www.amnesty.org/en/latest/research/2021/07/forensic-methodology-report-how-to-catch-nso-groups-pegasus/#_ftn1&quot;&gt;forensic evidence of Pegasus&lt;/a&gt; on the phones of several volunteers whose numbers appeared on the original list,
including those of journalists and human rights activists.
(Security Lab has since opened up their &lt;a href=&quot;https://github.com/mvt-project/mvt&quot;&gt;infection finding tool&lt;/a&gt; to the public.)
French intelligence has similarly &lt;a href=&quot;https://www.theguardian.com/news/2021/aug/02/pegasus-spyware-found-on-journalists-phones-french-intelligence-confirms&quot;&gt;inspected and confirmed&lt;/a&gt; infection of at least three devices belonging to journalists.
The phones of several people who were close to the Saudi-American journalist, Jamal Khashoggi, were &lt;a href=&quot;https://www.bbc.com/news/world-57891506&quot;&gt;confirmed hacked&lt;/a&gt;
both before and after Khashoggi’s brutal murder at the Saudi embassy in Istanbul in 2018.
&lt;a href=&quot;https://www.theguardian.com/news/2021/sep/21/hungary-journalist-daniel-nemeth-phones-infected-with-nso-pegasus-spyware&quot;&gt;More reports&lt;/a&gt; of confirmed Pegasus hacks are still published with some regularity.
It is now an open secret that many authoritarian governments have bought Pegasus.
It’s not difficult to extrapolate from existing reports and such clients’ track records
what the potential injuries to human freedoms are that they can inflict with access to such a powerful cyberweapon.&lt;/p&gt;

&lt;h2 id=&quot;a-typical-response&quot;&gt;A typical response&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;https://www.theguardian.com/news/2021/jul/18/response-from-nso-and-governments&quot;&gt;NSO’s response&lt;/a&gt; to the allegations follows a textbook approach
of avoiding earnest ethical introspection on the manufacturing, and selling, of cyber-arms.
Firstly, shift ethical responsibility to a predetermined process, a list of checkboxes of your own making.
The Group, for example, claims to sell only to “vetted governments”, following a classification process
of which they have now &lt;a href=&quot;https://www.nsogroup.com/wp-content/uploads/2021/06/ReportBooklet.pdf&quot;&gt;published some procedural details&lt;/a&gt; but no tangible criteria.
The next step is to reaffirm continuously, and repetitively, your dedication to the &lt;em&gt;legal&lt;/em&gt; combat against crime,
&lt;a href=&quot;https://www.nsogroup.com/wp-content/uploads/2021/06/ReportBooklet.pdf&quot;&gt;“legitimate law enforcement agencies”&lt;/a&gt; (note the almost tautological phrasing),
adherence to international arms trade laws,
compliance clauses in customer contracts, etc.
Thirdly, having been absolved of any moral suspicions that might exist about product and process,
from conception to engineering to trade,
distance yourself from the consequences of its use in the world.
&lt;a href=&quot;https://www.theguardian.com/news/2021/jul/18/response-from-nso-and-governments&quot;&gt;“NSO does not operate its technology, does not collect, nor possesses, nor has any access to any kind of data of its customers.”&lt;/a&gt;
It is interesting that directly after this statement they claim with contradictory confidence that
their “technology was not associated in any way with the heinous murder of Jamal Khashoggi”.
The unapologetic tone seems hardly appropriate when the same document confirms that the Group had to
shut down customers’ systems due to “confirmed misuse” and have had to do so “multiple times” in the past.
Given all this, the response manages to evade any serious interrogation of the “vetting” process itself,
which forced the company to reject “approximately 15% of potential new opportunities for Pegasus” in one year.
Courageous.&lt;/p&gt;

&lt;p&gt;We have heard this all before.
There exists a multi-billion dollar industry of private companies and engineering firms &lt;a href=&quot;https://www.economist.com/business/2019/12/12/offering-software-for-snooping-to-governments-is-a-booming-business&quot;&gt;thriving on proceeds&lt;/a&gt; from
selling surveillance tools and cyber-arms to dubious agencies and foreign governments.
In turn, the most power-hungry and oppressive regimes often &lt;em&gt;rely&lt;/em&gt; on such technological innovations -
for which they lack the in-country engineering expertise -
to maintain control, suppress uprisings, intimidate opposing journalists, and track their citizens.
It’s a lucrative business opportunity, and resourceful companies have sprung up everywhere to supply this demand,
often in countries where citizens, including employees of the company, would be horrified if they were similarly subject to the oppressions of their own products.
When, in 2014, Italy’s &lt;em&gt;HackingTeam&lt;/em&gt; were pulsed by the United Nations about their (then alleged) selling of spyware to Sudan,
which would have been a contravention of the UN’s weapon export ban,
they simply replied that their product was not controlled as a weapon and therefore not subject to such scrutiny.
They remained within their legal bounds, technically.
Furthermore, they similarly shifted ethical responsibility to external standards of legitimacy,
claiming their &lt;a href=&quot;https://citizenlab.ca/2014/02/mapping-hacking-teams-untraceable-spyware/&quot;&gt;“software is not sold to governments that are blacklisted by the EU, the US, NATO, and similar international organizations”&lt;/a&gt;.
When the company themselves were &lt;a href=&quot;https://www.wired.com/2015/07/hacking-team-breach-shows-global-spying-firm-run-amok/&quot;&gt;hacked in 2015&lt;/a&gt;,
revelations (confirmations, that is) of widespread misuse by repressive governments were damaging enough to force them to disappear and rebrand as Memento Labs.
&lt;a href=&quot;https://www.mem3nt0.com/en/&quot;&gt;Their website&lt;/a&gt; boasts an impressive list of statutes, regulations, procedures, export controls and legal frameworks,
all of which the rebranded hackers proudly comply with.
Surely no further ethical scrutiny is necessary?&lt;/p&gt;

&lt;h2 id=&quot;ethics--the-law&quot;&gt;Ethics != the law&lt;/h2&gt;

&lt;h3 id=&quot;the-law-is-trailing-behind&quot;&gt;The law is trailing behind&lt;/h3&gt;

&lt;p&gt;Such recourse to the &lt;em&gt;legality&lt;/em&gt; of your action as ethical justification is moot for several reasons.
The first is glaringly obvious -
our laws are ill-equipped to address the implications of modern technology.
Legal systems are a cumbersome inheritance built over generations.
This is especially true of the statutes and regulations governing international trade, behind which these companies so often hide.
Our best legal systems are trailing miles behind the technology for which we seek guidelines.
Legislators are still struggling to make sense of technologies like face recognition,
the repercussions of smart devices acting “on their own” and biases in algorithms.
To claim you are performing ethical due diligence by resorting to an outdated and incomplete system of legal codes is disingenuous.&lt;/p&gt;

&lt;h3 id=&quot;the-law-depends-on-ethics&quot;&gt;The law depends on ethics&lt;/h3&gt;

&lt;p&gt;The second reason is more central to my argument,
and an important flaw in these sleight of hand justifications appearing from time to time in the media.
Ethics can in no way be confused as synonymous with legality or legitimacy.
These are incommensurable concepts.
In an ideal world, of course, the law is meant to track the minimum standards of ethical conduct in a society.
Laws are often drafted exactly from some ethical, and practical, impulse to minimize harmful conduct
and provide for corrective and punitive measures where transgressions do occur.
The law, however, has a much narrower scope than ethics.
It can be just or unjust.
In fact, it is in need of ethics to constantly reform.
Ethics and values are born out of collective self-reflection.
It develops in our conversation with ourselves and others about the type of society we strive for.
As such, an ethical worldview summarizes our deepest intuitions about how we should live and measure our impact on the world.
For this reason, ethics is primarily enforced by social and internal pressures, not legal boundaries -
our desire to do what &lt;em&gt;ought&lt;/em&gt; to be done, however we define that.
Ethics is therefore a much grander scheme than global legal systems
and the diplomatic frameworks that grants legitimacy to governments.
These are but one limited outflow of the human aspiration to form societies in accordance with our ideologies and ethics.&lt;/p&gt;

&lt;h3 id=&quot;international-law-is-vague-and-exploitable&quot;&gt;International law is vague and exploitable&lt;/h3&gt;

&lt;p&gt;Of course, the cyber-arms trade has a favorite recourse, &lt;em&gt;international&lt;/em&gt; law, which is even more limited.
Since such products are seldomly sold to governments and agencies within the country of production,
it enables a further distancing from consequences.
Many private surveillance companies are based in fairly liberal societies with (seemingly) strict emphases on human rights in their domestic laws.
International laws are much more complicated - for opportunists a synonym for “more grey areas in which to hide”.
Company conduct can now be governed, and excused, by a system that follows
the whims of autocrats with exploitative intent and vastly different ethical conceptions from the company’s purported aims.
International law, and the ways it is most often enforced by way of, say, UN-backed sanctions,
have long been shaped by the compromises of international diplomacy.
To be blunt: these laws are weak and subject to exactly the sort of narrow interests behind which mercenaries have always hidden.
The surveillance tech industry is no exception.&lt;/p&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;

&lt;p&gt;My point is simple:
selling cyber-arms with the potential to become vast tools of oppression to governments and bodies with blatant histories of human rights violations,
and all but the publicly announced intention to continue operating in this way,
is categorically unconscionable.
This seems obvious no matter what ethics system you argue from,
provided it harbors any consideration for human dignity and freedom.
It is a sign of poor moral discourse that such recourses to law and legitimacy are often considered synonymous with ethical justification.
“&lt;em&gt;I have acted within the bounds of law&lt;/em&gt;”, &lt;em&gt;“We supply only to legitimate law enforcement agencies”&lt;/em&gt;, etc. are no substitutes.
Ethical conduct requires an honest evaluation of an action against some conception of “the good”,
however you define that.
Too often the surveillance tech industry precisely sidesteps this question,
both in internal processes and external rationalisations to a concerned public.&lt;/p&gt;

&lt;p&gt;John Locke, he of the life-liberty-and-property, articulated the idea that government exists solely through the consent of the governed.
Towards the end of the 17th century, he wrote in his &lt;em&gt;Second Treatise on Civil Government&lt;/em&gt;,
“[w]henever legislators endeavor to take away,
and destroy the property of the people, or to reduce them to slavery under arbitrary power,
they put themselves in a state of war with the people, who are thereupon absolved from any further obedience”.
The inference is straightforward and humanist in essence:
legitimacy is not something that is conferred by governments and institutions.
Rather, they derive their legitimacy from us, their citizens, holding them to standards of ethics and societal ideals.
This legitimacy only remains in tact as long as this mandate is honored and continuously extended by a well-informed public.
This is the principle of informed consent on which all reciprocal ethics is based.&lt;/p&gt;

&lt;p&gt;The surveillance tech industry may well have nothing more or less noble in mind than profit-making within legal bounds
when developing and selling their products.
However, when such companies are revealed again and again to have supplied tools of gross human rights violations to known human rights violators,
they will do well to remember that ethics always &lt;em&gt;precedes&lt;/em&gt; requirements of legality and legitimacy.
It is a fallacy to take normative guidance from the concept of “legitimacy”
if the concept itself depends on such normative guidelines for definition.
Without examining the ethical standards by which institutions, governments, and laws, were created,
no value-judgements about their legitimacy can be made.
Hiding behind legal compliance as substitute for moral justification is not enough.
Targets of increasingly invasive governmental snooping are too often chosen precisely to suppress the mechanisms from which the legitimacy of such governments flow -
the consent of ordinary civilians.
Free and fair elections, free speech, free media, freedom of thought are all at risk.&lt;/p&gt;

&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;https://our.status.im/our-principles/&quot;&gt;Status Principles&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://public-inspection.federalregister.gov/2021-24123.pdf&quot;&gt;Federal Register: Addition of Certain Entities to the Entity List&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://forbiddenstories.org/case/the-pegasus-project/&quot;&gt;forbiddenstories.org: The Pegasus Project&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.theguardian.com/news/series/pegasus-project&quot;&gt;theguardian.com: The Pegasus Project&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.amnesty.org/en/latest/research/2021/07/forensic-methodology-report-how-to-catch-nso-groups-pegasus/#_ftn1&quot;&gt;amnesty.org Forensic Methodology Report: How to catch NSO Group’s Pegasus&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.apple.com/newsroom/2021/11/apple-sues-nso-group-to-curb-the-abuse-of-state-sponsored-spyware/&quot;&gt;Apple sues NSO Group to curb the abuse of state-sponsored spyware&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.bbc.com/news/technology-37192670&quot;&gt;bbc.com: Who are the hackers who cracked the iPhone?&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.bbc.com/news/world-57891506&quot;&gt;bbc.com: Pegasus: Who are the alleged victims of spyware targeting?&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://citizenlab.ca/2014/02/mapping-hacking-teams-untraceable-spyware/&quot;&gt;citizenlab.ca: Mapping Hacking Team’s “Untraceable” Spyware&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.economist.com/business/2019/12/12/offering-software-for-snooping-to-governments-is-a-booming-business&quot;&gt;economist.com: Offering software for snooping to governments is a booming business&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.mem3nt0.com/en/&quot;&gt;Memento Labs&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://github.com/mvt-project/mvt&quot;&gt;Mobile Verification Toolkit to identify compromised devices&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.nsogroup.com/wp-content/uploads/2021/06/ReportBooklet.pdf&quot;&gt;NSO Group: Transparency and Responsibility Report 2021&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.reuters.com/article/us-facebook-cyber-whatsapp-nsogroup-idUSKBN1X82BE&quot;&gt;reuters.com: WhatsApp sues Israel’s NSO for allegedly helping spies hack phones around the world&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.wired.com/2015/07/hacking-team-breach-shows-global-spying-firm-run-amok/&quot;&gt;wired.com: Hacking Team Breach Shows a Global Spying Firm Run Amok&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</content><author><name>circe</name></author><category term="research" /><summary type="html">This is an opinion piece by pseudonymous contributor, circe.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/vac.png" /><media:content medium="image" url="https://vac.dev/assets/img/vac.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Waku v1 vs Waku v2: Bandwidth Comparison</title><link href="https://vac.dev/waku-v1-v2-bandwidth-comparison" rel="alternate" type="text/html" title="Waku v1 vs Waku v2: Bandwidth Comparison" /><published>2021-11-03T08:00:00+00:00</published><updated>2021-11-03T08:00:00+00:00</updated><id>https://vac.dev/waku-v1-vs-waku-v2</id><content type="html" xml:base="https://vac.dev/waku-v1-v2-bandwidth-comparison">&lt;h2 id=&quot;background&quot;&gt;Background&lt;/h2&gt;

&lt;p&gt;The &lt;a href=&quot;https://vac.dev/waku-v2-plan&quot;&gt;original plan&lt;/a&gt; for Waku v2 suggested theoretical improvements in resource usage over Waku v1,
mainly as a result of the improved amplification factors provided by GossipSub.
In its turn, &lt;a href=&quot;https://vac.dev/fixing-whisper-with-waku&quot;&gt;Waku v1 proposed improvements&lt;/a&gt; over its predecessor, Whisper.&lt;/p&gt;

&lt;p&gt;Given that Waku v2 is aimed at resource restricted environments,
we are specifically interested in its scalability and resource usage characteristics.
However, the theoretical performance improvements of Waku v2 over Waku v1,
has never been properly benchmarked and tested.&lt;/p&gt;

&lt;p&gt;Although we’re working towards a full performance evaluation of Waku v2,
this would require significant planning and resources,
if it were to simulate “real world” conditions faithfully and measure bandwidth and resource usage across different network connections,
robustness against attacks/losses, message latencies, etc.
(There already exists a fairly comprehensive &lt;a href=&quot;https://research.protocol.ai/publications/gossipsub-v1.1-evaluation-report/vyzovitis2020.pdf&quot;&gt;evaluation of GossipSub v1.1&lt;/a&gt;,
on which &lt;a href=&quot;https://rfc.vac.dev/spec/11/&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;11/WAKU2-RELAY&lt;/code&gt;&lt;/a&gt; is based.)&lt;/p&gt;

&lt;p&gt;As a starting point,
this post contains a limited and local comparison of the &lt;em&gt;bandwidth&lt;/em&gt; profile (only) between Waku v1 and Waku v2.
It reuses and adapts existing network simulations for &lt;a href=&quot;https://github.com/status-im/nim-waku/blob/master/waku/v1/node/quicksim.nim&quot;&gt;Waku v1&lt;/a&gt; and &lt;a href=&quot;https://github.com/status-im/nim-waku/blob/master/waku/v2/node/quicksim2.nim&quot;&gt;Waku v2&lt;/a&gt;
and compares bandwidth usage for similar message propagation scenarios.&lt;/p&gt;

&lt;h2 id=&quot;theoretical-improvements-in-waku-v2&quot;&gt;Theoretical improvements in Waku v2&lt;/h2&gt;

&lt;p&gt;Messages are propagated in Waku v1 using &lt;a href=&quot;https://en.wikipedia.org/wiki/Flooding_(computer_networking)&quot;&gt;flood routing&lt;/a&gt;.
This means that every peer will forward every new incoming message to all its connected peers (except the one it received the message from).
This necessarily leads to unnecessary duplication (termed &lt;em&gt;amplification factor&lt;/em&gt;),
wasting bandwidth and resources.
What’s more, we expect this effect to worsen the larger the network becomes,
as each &lt;em&gt;connection&lt;/em&gt; will receive a copy of each message,
rather than a single copy per peer.&lt;/p&gt;

&lt;p&gt;Message routing in Waku v2 follows the &lt;code class=&quot;highlighter-rouge&quot;&gt;libp2p&lt;/code&gt; &lt;em&gt;GossipSub&lt;/em&gt; protocol,
which lowers amplification factors by only sending full message contents to a subset of connected peers.
As a Waku v2 network grows, each peer will limit its number of full-message (“mesh”) peerings -
&lt;code class=&quot;highlighter-rouge&quot;&gt;libp2p&lt;/code&gt; suggests a maximum of &lt;code class=&quot;highlighter-rouge&quot;&gt;12&lt;/code&gt; such connections per peer.
This allows much better scalability than a flood-routed network.
From time to time, a Waku v2 peer will send metadata about the messages it has seen to other peers (“gossip” peers).&lt;/p&gt;

&lt;p&gt;See &lt;a href=&quot;https://hackmd.io/@vac/main/%2FYYlZYBCURFyO_ZG1EiteWg#11WAKU2-RELAY-gossipsub&quot;&gt;this explainer&lt;/a&gt; for a more detailed discussion.&lt;/p&gt;

&lt;h2 id=&quot;methodology&quot;&gt;Methodology&lt;/h2&gt;

&lt;p&gt;The results below contain only some scenarios that provide an interesting contrast between Waku v1 and Waku v2.
For example, &lt;a href=&quot;https://en.wikipedia.org/wiki/Star_network&quot;&gt;star network topologies&lt;/a&gt; do not show a substantial difference between Waku v1 and Waku v2.
This is because each peer relies on a single connection to the central node for every message,
which barely requires any routing:
each connection receives a copy of every message for both Waku v1 and Waku v2.
Hybrid topologies similarly show only a difference between Waku v1 and Waku v2 for network segments with &lt;a href=&quot;https://en.wikipedia.org/wiki/Mesh_networking&quot;&gt;mesh-like connections&lt;/a&gt;,
where routing decisions need to be made.&lt;/p&gt;

&lt;p&gt;For this reason, the following approach applies to all iterations:&lt;/p&gt;
&lt;ol&gt;
  &lt;li&gt;Simulations are run &lt;strong&gt;locally&lt;/strong&gt;.
This limits the size of possible scenarios due to local resource constraints,
but is a way to quickly get an approximate comparison.&lt;/li&gt;
  &lt;li&gt;Nodes are treated as a &lt;strong&gt;blackbox&lt;/strong&gt; for which we only measure bandwidth,
using an external bandwidth monitoring tool.
In other words, we do not consider differences in the size of the envelope (for v1) or the message (for v2).&lt;/li&gt;
  &lt;li&gt;Messages are published at a rate of &lt;strong&gt;50 new messages per second&lt;/strong&gt; to each network,
except where explicitly stated otherwise.&lt;/li&gt;
  &lt;li&gt;Each message propagated in the network carries &lt;strong&gt;8 bytes&lt;/strong&gt; of random payload, which is &lt;strong&gt;encrypted&lt;/strong&gt;.
The same symmetric key cryptographic algorithm (with the same keys) are used in both Waku v1 and v2.&lt;/li&gt;
  &lt;li&gt;Traffic in each network is &lt;strong&gt;generated from 10 nodes&lt;/strong&gt; (randomly-selected) and published in a round-robin fashion to &lt;strong&gt;10 topics&lt;/strong&gt; (content topics for Waku v2).
In practice, we found no significant difference in &lt;em&gt;average&lt;/em&gt; bandwidth usage when tweaking these two parameters (the number of traffic generating nodes and the number of topics).&lt;/li&gt;
  &lt;li&gt;Peers are connected in a decentralized &lt;strong&gt;full mesh topology&lt;/strong&gt;,
i.e. each peer is connected to every other peer in the network.
Waku v1 is expected to flood all messages across all existing connections.
Waku v2 gossipsub will GRAFT some of these connections for full-message peerings,
with the rest being gossip-only peerings.&lt;/li&gt;
  &lt;li&gt;After running each iteration, we &lt;strong&gt;verify that messages propagated to all peers&lt;/strong&gt; (comparing the number of published messages to the metrics logged by each peer).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For Waku v1, nodes are configured as “full” nodes (i.e. with full bloom filter),
while Waku v2 nodes are &lt;code class=&quot;highlighter-rouge&quot;&gt;relay&lt;/code&gt; nodes, all subscribing and publishing to the same PubSub topic.&lt;/p&gt;

&lt;h2 id=&quot;network-size-comparison&quot;&gt;Network size comparison&lt;/h2&gt;

&lt;h3 id=&quot;iteration-1-10-nodes&quot;&gt;Iteration 1: 10 nodes&lt;/h3&gt;

&lt;p&gt;Let’s start with a small network of 10 nodes only and see how Waku v1 bandwidth usage compares to that of Waku v2.
At this small scale we don’t expect to see improved bandwidth usage in Waku v2 over Waku v1,
since all connections, for both Waku v1 and Waku v2, will be full-message connections.
The number of connections is low enough that Waku v2 nodes will likely GRAFT all connections to full-message peerings,
essentially flooding every message on every connection in a similar fashion to Waku v1.
If our expectations are confirmed, it helps validate our methodology,
showing that it gives more or less equivalent results between Waku v1 and Waku v2 networks.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-10-nodes.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Sure enough, the figure shows that in this small-scale setup,
Waku v1 actually has a lower per-peer bandwidth usage than Waku v2.
One reason for this may be the larger overall proportion of control messages in a gossipsub-routed network such as Waku v2.
These play a larger role when the total network traffic is comparatively low, as in this iteration.
Also note that the average bandwidth remains more or less constant as long as the rate of published messages remains stable.&lt;/p&gt;

&lt;h3 id=&quot;iteration-2-30-nodes&quot;&gt;Iteration 2: 30 nodes&lt;/h3&gt;

&lt;p&gt;Now, let’s run the same scenario for a larger network of highly-connected nodes, this time consisting of 30 nodes.
At this point, the Waku v2 nodes will start pruning some connections to limit the number of full-message peerings (to a maximum of &lt;code class=&quot;highlighter-rouge&quot;&gt;12&lt;/code&gt;),
while the Waku v1 nodes will continue flooding messages to all connected peers.
We therefore expect to see a somewhat improved bandwidth usage in Waku v2 over Waku v1.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-30-nodes.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Bandwidth usage in Waku v2 has increased only slightly from the smaller network of 10 nodes (hovering between 2000 and 3000 kbps).
This is because there are only a few more full-message peerings than before.
Compare this to the much higher increase in bandwidth usage for Waku v1, which now requires more than 4000 kbps on average.&lt;/p&gt;

&lt;h3 id=&quot;iteration-3-50-nodes&quot;&gt;Iteration 3: 50 nodes&lt;/h3&gt;

&lt;p&gt;For an even larger network of 50 highly connected nodes,
the divergence between Waku v1 and Waku v2 is even larger.
The following figure shows comparative average bandwidth usage for a throughput of 50 messages per second.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-50-nodes.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Average bandwidth usage (for the same message rate) has remained roughly the same for Waku v2 as it was for 30 nodes,
indicating that the number of full-message peerings per node has not increased.&lt;/p&gt;

&lt;h3 id=&quot;iteration-4-85-nodes&quot;&gt;Iteration 4: 85 nodes&lt;/h3&gt;

&lt;p&gt;We already see a clear trend in the bandwidth comparisons above,
so let’s confirm by running the test once more for a network of 85 nodes.
Due to local resource constraints, the effective throughput for Waku v1 falls to below 50 messages per second,
so the v1 results below have been normalized and are therefore approximate.
The local Waku v2 simulation maintains the message throughput rate without any problems.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-85-nodes.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;iteration-5-150-nodes&quot;&gt;Iteration 5: 150 nodes&lt;/h3&gt;

&lt;p&gt;Finally, we simulate message propagation in a network of 150 nodes.
Due to local resource constraints, we run this simulation at a lower rate -
35 messages per second -
and for a shorter amount of time.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-150-nodes.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Notice how the Waku v1 bandwidth usage is now more than 10 times worse than that of Waku v2.
This is to be expected, as each Waku v1 node will try to flood each new message to 149 other peers,
while the Waku v2 nodes limit their full-message peerings to no more than 12.&lt;/p&gt;

&lt;h3 id=&quot;discussion&quot;&gt;Discussion&lt;/h3&gt;

&lt;p&gt;Let’s summarize average bandwidth growth against network growth for a constant message propagation rate.
Since we are particularly interested in how Waku v1 compares to Waku v2 in terms of bandwidth usage,
the results are normalised to the Waku v2 average bandwidth usage for each network size.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-overall-network-size.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Extrapolation is a dangerous game,
but it’s safe to deduce that the divergence will only grow for even larger network topologies.
Although control signalling contributes more towards overall bandwidth for Waku v2 networks,
this effect becomes less noticeable for larger networks.
For network segments with more than ~18 densely connected nodes,
the advantage of using Waku v2 above Waku v1 becomes clear.&lt;/p&gt;

&lt;h2 id=&quot;network-traffic-comparison&quot;&gt;Network traffic comparison&lt;/h2&gt;

&lt;p&gt;The analysis above controls the average message rate while network size grows.
In reality, however, active users (and therefore message rates) are likely to grow in conjunction with the network.
This will have an effect on bandwidth for both Waku v1 and Waku v2, though not in equal measure.
Consider the impact of an increasing rate of messages in a network of constant size:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/waku1-vs-waku2/waku1-vs-waku2-overall-message-rate.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The &lt;em&gt;rate&lt;/em&gt; of increase in bandwidth for Waku v2 is slower than that for Waku v1 for a corresponding increase in message propagation rate.
In fact, for a network of 30 densely-connected nodes,
if the message propagation rate increases by 1 per second,
Waku v1 requires an increased average bandwidth of almost 70kbps at each node.
A similar traffic increase in Waku v2 requires on average 40kbps more bandwidth per peer, just over half that of Waku v1.&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;Waku v2 scales significantly better than Waku v1 in terms of average bandwidth usage&lt;/strong&gt;,
especially for densely connected networks.&lt;/li&gt;
  &lt;li&gt;E.g. for a network consisting of &lt;strong&gt;150&lt;/strong&gt; or more densely connected nodes,
Waku v2 provides more than &lt;strong&gt;10x&lt;/strong&gt; better average bandwidth usage rates than Waku v1.&lt;/li&gt;
  &lt;li&gt;As the network continues to scale, both in absolute terms (number of nodes) and in network traffic (message rates) the disparity between Waku v2 and Waku v1 becomes even larger.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;future-work&quot;&gt;Future work&lt;/h2&gt;

&lt;p&gt;Now that we’ve confirmed that Waku v2’s bandwidth improvements over its predecessor matches theory,
we can proceed to a more in-depth characterisation of Waku v2’s resource usage.
Some questions that we want to answer include:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;What proportion of Waku v2’s bandwidth usage is used to propagate &lt;em&gt;payload&lt;/em&gt; versus bandwidth spent on &lt;em&gt;control&lt;/em&gt; messaging to maintain the mesh?&lt;/li&gt;
  &lt;li&gt;To what extent is message latency (time until a message is delivered to its destination) affected by network size and message rate?&lt;/li&gt;
  &lt;li&gt;How &lt;em&gt;reliable&lt;/em&gt; is message delivery in Waku v2 for different network sizes and message rates?&lt;/li&gt;
  &lt;li&gt;What are the resource usage profiles of other Waku v2 protocols (e.g.&lt;a href=&quot;https://rfc.vac.dev/spec/12/&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;12/WAKU2-FILTER&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://rfc.vac.dev/spec/19/&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;19/WAKU2-LIGHTPUSH&lt;/code&gt;&lt;/a&gt;)?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Our aim is to get ever closer to a “real world” understanding of Waku v2’s performance characteristics,
identify and fix vulnerabilities
and continually improve the efficiency of our suite of protocols.&lt;/p&gt;

&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;https://research.protocol.ai/publications/gossipsub-v1.1-evaluation-report/vyzovitis2020.pdf&quot;&gt;Evaluation of GossipSub v1.1&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://vac.dev/fixing-whisper-with-waku&quot;&gt;Fixing Whisper with Waku&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://hackmd.io/@vac/main/%2FYYlZYBCURFyO_ZG1EiteWg#11WAKU2-RELAY-gossipsub&quot;&gt;GossipSub vs flood routing&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://www.techopedia.com/definition/13335/star-topology#:~:text=Star%20topology%20is%20a%20network,known%20as%20a%20star%20network.&quot;&gt;Network topologies: star&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Mesh_networking&quot;&gt;Network topologies: mesh&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://vac.dev/waku-v2-plan&quot;&gt;Waku v2 original plan&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</content><author><name>hanno</name></author><category term="research" /><summary type="html">Background</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/waku1-vs-waku2/waku1-vs-waku2-overall-network-size.png" /><media:content medium="image" url="https://vac.dev/assets/img/waku1-vs-waku2/waku1-vs-waku2-overall-network-size.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging</title><link href="https://vac.dev/waku-v2-ethereum-coscup" rel="alternate" type="text/html" title="[Talk at COSCUP] Vac, Waku v2 and Ethereum Messaging" /><published>2021-08-06T04:00:00+00:00</published><updated>2021-08-06T04:00:00+00:00</updated><id>https://vac.dev/coscup-waku-ethereum</id><content type="html" xml:base="https://vac.dev/waku-v2-ethereum-coscup">&lt;p&gt;&lt;em&gt;This is the English version of a talk originally given in Chinese at COSCUP in Taipei. There is a &lt;a href=&quot;https://www.youtube.com/watch?v=s0ATpQ4_XFc&quot;&gt;video recording&lt;/a&gt; with Chinese and English subtitles.&lt;/em&gt;&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;introduction&quot;&gt;Introduction&lt;/h2&gt;

&lt;p&gt;Hi everyone!&lt;/p&gt;

&lt;p&gt;Today I’ll talk to you about Waku v2. What it is, what problems it is solving,
and how it can be useful for things such as messaging in Ethereum. First, let me
start with some brief background.&lt;/p&gt;

&lt;h2 id=&quot;brief-history-and-background&quot;&gt;Brief history and background&lt;/h2&gt;

&lt;p&gt;Back when Ethereum got started, there used to be this concept of the “holy
trinity”. You had Ethereum for compute/consensus, Swarm for storage, and Whisper
for messaging. This is partly where the term Web3 comes from.&lt;/p&gt;

&lt;p&gt;Status started out as an app with the goal of being a window onto Ethereum and
a secure messenger. As one of the few, if not the only, apps using Whisper in
production, not to mention on a mobile phone, we quickly realized there were
problems with the underlying protocols and infrastructure. Protocols such as
Whisper weren’t quite ready for prime time yet when it came to things such as
scalability and working in the real world.&lt;/p&gt;

&lt;p&gt;As we started addressing some of these challenges, and moved from app
developement to focusing on protocols, research and infrastructure, we created
Vac. Vac is an r&amp;amp;d unit doing protocol research focused on creating modular p2p
messaging protocols for private, secure, censorship resistant communication.&lt;/p&gt;

&lt;p&gt;I won’t go into too much detail on the issues with Whisper, if you are
interested in this check out this talk
&lt;a href=&quot;https://www.youtube.com/watch?v=6lLT33tsJjs&quot;&gt;here&lt;/a&gt; or this
&lt;a href=&quot;https://vac.dev/fixing-whisper-with-waku&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In a nutshell, we forked Whisper to address immediate shortcomings and this
became Waku v1. Waku v2 is complete re-thought implementation from scratch on top
of libp2p. This will be the subject of today’s talk.&lt;/p&gt;

&lt;h2 id=&quot;waku-v2&quot;&gt;Waku v2&lt;/h2&gt;

&lt;h3 id=&quot;overview&quot;&gt;Overview&lt;/h3&gt;

&lt;p&gt;Waku v2 is a privacy-preserving peer-to-peer messaging protocol for resource
restricted devices. We can look at Waku v2 as several things:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Set of protocols&lt;/li&gt;
  &lt;li&gt;Set of implementations&lt;/li&gt;
  &lt;li&gt;Network of nodes&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Let’s first look at what the goals are.&lt;/p&gt;

&lt;h3 id=&quot;goals&quot;&gt;Goals&lt;/h3&gt;

&lt;p&gt;Waku v2 provides a PubSub based messaging protocol with the following
characteristics:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;strong&gt;Generalized messaging&lt;/strong&gt;. Applications that require a messaging protocol to
communicate human to human, machine to machine, or a mix.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Peer-to-peer&lt;/strong&gt;. For applications that require a p2p solution.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Resource restricted&lt;/strong&gt;. For example, running with limited bandwidth, being
mostly-offline, or in a browser.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Privacy&lt;/strong&gt;. Applications that have privacy requirements, such as pseudonymity,
metadata protection, etc.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And to provide these properties in a modular fashion, where applications can
choose their desired trade-offs.&lt;/p&gt;

&lt;h3 id=&quot;protocols&quot;&gt;Protocols&lt;/h3&gt;

&lt;p&gt;Waku v2 consists of several protocols. Here we highlight a few of the most
important ones:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;10/WAKU2 - main specification, details how all the pieces fit together&lt;/li&gt;
  &lt;li&gt;11/RELAY - thin layer on top of GossipSub for message dissemination&lt;/li&gt;
  &lt;li&gt;13/STORE - fetching of historical messages&lt;/li&gt;
  &lt;li&gt;14/MESSAGE - message payload&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is the recommended subset for a minimal Waku v2 client.&lt;/p&gt;

&lt;p&gt;In addition to this there are many other types of specifications at various
stages of maturity, such as: content based filtering, bridge mode to Waku v1,
JSON RPC API, zkSNARKS based spam protection with RLN, accounting and
settlements with SWAP, fault-tolerant store nodes, recommendations around topic
usage, and more.&lt;/p&gt;

&lt;p&gt;See https://rfc.vac.dev/ for a full overview.&lt;/p&gt;

&lt;h3 id=&quot;implementations&quot;&gt;Implementations&lt;/h3&gt;

&lt;p&gt;Waku v2 consists of multiple implementations. This allows for client diversity,
makes it easier to strengthen the protocols, and allow people to use Waku v2 in
different contexts.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;nim-waku - the reference client written in Nim, most full-featured.&lt;/li&gt;
  &lt;li&gt;js-waku - allow usage of Waku v2 from browsers, focus on interacting with dapps.&lt;/li&gt;
  &lt;li&gt;go-waku - subset of Waku v2 to ease integration into the Status app.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;testnet-huilong-and-dogfooding&quot;&gt;Testnet Huilong and dogfooding&lt;/h3&gt;

&lt;p&gt;In order to test the protocol we have setup a testnet across all implementations
called Huilong. Yes, that’s the Taipei subway station!&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/coscup-waku/huilong.jpg&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Among us core devs we have disabled the main #waku Discord channel used for
development, and people run their own node connected to this toy chat application.&lt;/p&gt;

&lt;p&gt;Feel free to join and say hi! Instructions can be found here:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md&quot;&gt;nim-waku chat&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://status-im.github.io/js-waku/&quot;&gt;js-waku chat&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://github.com/status-im/go-waku/tree/master/examples/chat2&quot;&gt;go-waku chat&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;research&quot;&gt;Research&lt;/h3&gt;

&lt;p&gt;While Waku v2 is being used today, we are actively researching improvements.
Since the design is modular, we can gracefully introduce new capabilities. Some
of these research areas are:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Privacy-preserving spam protection using zkSNARKs and RLN&lt;/li&gt;
  &lt;li&gt;Accounting and settlement of resource usage to incentivize nodes to provide services with SWAP&lt;/li&gt;
  &lt;li&gt;State synchronization for store protocol to make it easier to run a store node without perfect uptime&lt;/li&gt;
  &lt;li&gt;Better node discovery&lt;/li&gt;
  &lt;li&gt;More rigorous privacy analysis&lt;/li&gt;
  &lt;li&gt;Improving interaction with wallets and dapp&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;use-cases&quot;&gt;Use cases&lt;/h2&gt;

&lt;p&gt;Let’s look at where Waku v2 is and can be used.&lt;/p&gt;

&lt;h3 id=&quot;prelude-topics-in-waku-v2&quot;&gt;Prelude: Topics in Waku v2&lt;/h3&gt;

&lt;p&gt;To give some context, there are two different types of topics in Waku v2. One is
a PubSub topic, for routing. The other is a content topic, which is used for
content based filtering. Here’s an example of the default PubSub topic:&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;/waku/2/default-waku/proto&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;This is recommended as it increases privacy for participants and it is stored by
default, however this is up to the application.&lt;/p&gt;

&lt;p&gt;The second type of topic is a content topic, which is application specific. For
example, here’s the content topic used in our testnet:&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;/toychat/2/huilong/proto&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;For more on topics, see https://rfc.vac.dev/spec/23/&lt;/p&gt;

&lt;h3 id=&quot;status-app&quot;&gt;Status app&lt;/h3&gt;

&lt;p&gt;In the Status protocol, content topics - topics in Whisper/Waku v1 - are used for several things:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Contact code topic to discover X3DH bundles for perfect forward secrecy
    &lt;ul&gt;
      &lt;li&gt;Partitioned into N (currently 5000) content topics to balance privacy with efficiency&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Public chats correspond to hash of the plaintext name&lt;/li&gt;
  &lt;li&gt;Negotiated topic for 1:1 chat with DHKE derived content topic&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;See more here https://specs.status.im/spec/10&lt;/p&gt;

&lt;p&gt;Currently, Status app is in the process of migrating to and testing Waku v2.&lt;/p&gt;

&lt;h3 id=&quot;dappconnect-ethereum-messaging&quot;&gt;DappConnect: Ethereum messaging&lt;/h3&gt;

&lt;p&gt;It is easy to think of Waku as being for human messaging, since that’s how it is
primarily used in the Status app, but the goal is to be useful for generalized
messaging, which includes Machine-To-Machine (M2M) messaging.&lt;/p&gt;

&lt;p&gt;Recall the concept of the holy trinity with Ethereum/Swarm/Whisper and Web3 that
we mentioned in the beginning. Messaging can be used as a building block for
dapps, wallets, and users to communicate with each other. It can be used for
things such as:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Multisig and DAO vote transactions only needing one on-chain operation&lt;/li&gt;
  &lt;li&gt;Giving dapps ability to send push notifications to users&lt;/li&gt;
  &lt;li&gt;Giving users ability to directly respond to requests from dapps&lt;/li&gt;
  &lt;li&gt;Decentralized WalletConnect&lt;/li&gt;
  &lt;li&gt;Etc&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Basically anything that requires communication and doesn’t have to be on-chain.&lt;/p&gt;

&lt;h3 id=&quot;walletconnect-v2&quot;&gt;WalletConnect v2&lt;/h3&gt;

&lt;p&gt;WalletConnect is an open protocol for connecting dapps to wallets with a QR
code. Version 2 is using Waku v2 as a communication channel to do so in a
decentralized and private fashion.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/coscup-waku/walletconnect.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;See for more: https://docs.walletconnect.org/v/2.0/tech-spec&lt;/p&gt;

&lt;p&gt;WalletConnect v2 is currently in late alpha using Waku v2.&lt;/p&gt;

&lt;h3 id=&quot;more-examples&quot;&gt;More examples&lt;/h3&gt;

&lt;ul&gt;
  &lt;li&gt;Gasless voting and vote aggregation off-chain&lt;/li&gt;
  &lt;li&gt;Dapp games using Waku as player discovery mechanism&lt;/li&gt;
  &lt;li&gt;Send encrypted message to someone with an Ethereum key&lt;/li&gt;
  &lt;li&gt;
    &lt;Your dapp=&quot;&quot; here=&quot;&quot;&gt;

&lt;/Your&gt;
  &lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These are all things that are in progress / proof of concept stage.&lt;/p&gt;

&lt;h2 id=&quot;contribute&quot;&gt;Contribute&lt;/h2&gt;

&lt;p&gt;We’d love to see contributions of any form!&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;You can play with it here: &lt;a href=&quot;https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md&quot;&gt;nim-waku chat&lt;/a&gt; (/ &lt;a href=&quot;https://status-im.github.io/js-waku/&quot;&gt;js-waku browser chat&lt;/a&gt;)&lt;/li&gt;
  &lt;li&gt;Use Waku to build a dapp: &lt;a href=&quot;https://status-im.github.io/js-waku/docs/&quot;&gt;js-waku docs&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Contribute to code: &lt;a href=&quot;https://github.com/status-im/js-waku&quot;&gt;js-waku&lt;/a&gt; / &lt;a href=&quot;https://github.com/status-im/nim-waku&quot;&gt;nim-waku&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Contribute to specs: &lt;a href=&quot;https://github.com/vacp2p/rfc&quot;&gt;vacp2p/rfc&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;We are hiring: Wallet &amp;amp; Dapp Integration Developer, Distributed Systems Engineer, Protocol Engineer, Protocol Researcher - all &lt;a href=&quot;https://status.im/our_team/jobs.html&quot;&gt;job listings&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Join our new &lt;a href=&quot;https://discord.gg/bJCTqS5H&quot;&gt;Discord&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;

&lt;p&gt;In this talk we’ve gone over the original vision for Web3 and how Waku came to
be. We’ve also looked at what Waku v2 aims to do. We looked at its protocols,
implementations, the current testnet as well as briefly on some ongoing
research for Vac.&lt;/p&gt;

&lt;p&gt;We’ve also looked at some specific use cases for Waku. First we looked at how
Status uses it with different topics. Then we looked at how it can be useful for
messaging in Ethereum, including for things like WalletConnect.&lt;/p&gt;

&lt;p&gt;I hope this talk gives you a better idea of what Waku is, why it exists, and
that it inspires you to contribute, either to Waku itself or by using it in your
own project!&lt;/p&gt;</content><author><name>oskarth</name></author><category term="research" /><summary type="html">This is the English version of a talk originally given in Chinese at COSCUP in Taipei. There is a video recording with Chinese and English subtitles.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/coscup-waku/talk.png" /><media:content medium="image" url="https://vac.dev/assets/img/coscup-waku/talk.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Presenting JS-Waku: Waku v2 in the Browser</title><link href="https://vac.dev/presenting-js-waku" rel="alternate" type="text/html" title="Presenting JS-Waku: Waku v2 in the Browser" /><published>2021-06-04T04:00:00+00:00</published><updated>2021-06-04T04:00:00+00:00</updated><id>https://vac.dev/presenting-js-waku</id><content type="html" xml:base="https://vac.dev/presenting-js-waku">&lt;p&gt;For the past 3 months, we have been working on bringing Waku v2 to the browser.
Our aim is to empower dApps with Waku v2, and it led to the creation of a new library.
We believe now is good time to introduce it!&lt;/p&gt;

&lt;h2 id=&quot;waku-v2&quot;&gt;Waku v2&lt;/h2&gt;

&lt;p&gt;First, let’s review what Waku v2 is and what problem it is trying to solve.&lt;/p&gt;

&lt;p&gt;Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised
communications on resource restricted devices (i.e., mobile phones).&lt;/p&gt;

&lt;p&gt;The Status chat feature was initially built over Whisper.
However, Whisper has a number of caveats which makes it inefficient for mobile phones.
For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.&lt;/p&gt;

&lt;p&gt;To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced.
To further enable web and restricted resource environments, Waku v2 was created based on libp2p.
The migration of the Status chat feature to Waku v2 is currently in progress.&lt;/p&gt;

&lt;p&gt;We see the need of such solution in the broader Ethereum ecosystem, beyond Status.
This is why we are building Waku v2 as a decentralised communication platform for all to use and build on.
If you want to read more about Waku v2 and what it aims to achieve,
checkout &lt;a href=&quot;/waku-v2-plan&quot;&gt;What’s the Plan for Waku v2?&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Since last year, we have been busy defining and implementing Waku v2 protocols in &lt;a href=&quot;https://github.com/status-im/nim-waku&quot;&gt;nim-waku&lt;/a&gt;,
from which you can build &lt;a href=&quot;https://github.com/status-im/nim-waku#wakunode&quot;&gt;wakunode2&lt;/a&gt;.
Wakunode2 is an adaptive and modular Waku v2 node,
it allows users to run their own node and use the Waku v2 protocols they need.
The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.&lt;/p&gt;

&lt;h2 id=&quot;waku-v2-in-the-browser&quot;&gt;Waku v2 in the browser&lt;/h2&gt;

&lt;p&gt;We believe that dApps and wallets can benefit from the Waku network in several ways.
For some dApps, it makes sense to enable peer-to-peer communications.
For others, machine-to-machine communications would be a great asset.
For example, in the case of a DAO,
Waku could be used for gas-less voting.
Enabling the DAO to notify their users of a new vote,
and users to vote without interacting with the blockchain and spending gas.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/status-im/murmur&quot;&gt;Murmur&lt;/a&gt; was the first attempt to bring Whisper to the browser,
acting as a bridge between devp2p and libp2p.
Once Waku v2 was started and there was a native implementation on top of libp2p,
a &lt;a href=&quot;https://github.com/vacp2p/waku-web-chat&quot;&gt;chat POC&lt;/a&gt; was created to demonstrate the potential of Waku v2
in web environment.
It showed how using js-libp2p with few modifications enabled access to the Waku v2 network.
There was still some unresolved challenges.
For example, nim-waku only support TCP connections which are not supported by browser applications.
Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,
which in turn could connect to wakunode2 via TCP.&lt;/p&gt;

&lt;p&gt;However, to enable dApp and Wallet developers to easily integrate Waku in their product,
we need to give them a library that is easy to use and works out of the box:
introducing &lt;a href=&quot;https://github.com/status-im/js-waku&quot;&gt;JS-Waku&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network.
It is available right now on &lt;a href=&quot;https://www.npmjs.com/package/js-waku&quot;&gt;npm&lt;/a&gt;:&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;npm install js-waku&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.&lt;/p&gt;

&lt;p&gt;Key Waku v2 protocols are already available:
&lt;a href=&quot;https://rfc.vac.dev/spec/14/&quot;&gt;message&lt;/a&gt;, &lt;a href=&quot;https://rfc.vac.dev/spec/13/&quot;&gt;store&lt;/a&gt;, &lt;a href=&quot;https://rfc.vac.dev/spec/11/&quot;&gt;relay&lt;/a&gt; and &lt;a href=&quot;https://rfc.vac.dev/spec/19/&quot;&gt;light push&lt;/a&gt;,
enabling your dApp to:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Send and receive near-instant messages on the Waku network (relay),&lt;/li&gt;
  &lt;li&gt;Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),&lt;/li&gt;
  &lt;li&gt;Send messages with confirmations (light push).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;JS-Waku needs to operate in the same context from which Waku v2 was born:
a restricted environment were connectivity or uptime are not guaranteed;
JS-Waku brings Waku v2 to the browser.&lt;/p&gt;

&lt;h2 id=&quot;achievements-so-far&quot;&gt;Achievements so far&lt;/h2&gt;

&lt;p&gt;We focused the past month on developing a &lt;a href=&quot;https://status-im.github.io/js-waku/&quot;&gt;ReactJS Chat App&lt;/a&gt;.
The aim was to create enough building blocks in JS-Waku to enable this showcase web app that
we now &lt;a href=&quot;https://github.com/status-im/nim-waku/issues/399&quot;&gt;use for dogfooding&lt;/a&gt; purposes.&lt;/p&gt;

&lt;p&gt;Most of the effort was on getting familiar with the &lt;a href=&quot;https://github.com/libp2p/js-libp2p&quot;&gt;js-libp2p&lt;/a&gt; library
that we heavily rely on.
JS-Waku is the second implementation of Waku v2 protocol,
so a lot of effort on interoperability was needed.
For example, to ensure compatibility with the nim-waku reference implementation,
we run our &lt;a href=&quot;https://github.com/status-im/js-waku/blob/90c90dea11dfd1277f530cf5d683fb92992fe141/src/lib/waku_relay/index.spec.ts#L137&quot;&gt;tests against wakunode2&lt;/a&gt; as part of the CI.&lt;/p&gt;

&lt;p&gt;This interoperability effort helped solidify the current Waku v2 specifications:
By clarifying the usage of topics
(&lt;a href=&quot;https://github.com/vacp2p/rfc/issues/327&quot;&gt;#327&lt;/a&gt;, &lt;a href=&quot;https://github.com/vacp2p/rfc/pull/383&quot;&gt;#383&lt;/a&gt;),
fix discrepancies between specs and nim-waku
(&lt;a href=&quot;https://github.com/status-im/nim-waku/issues/418&quot;&gt;#418&lt;/a&gt;, &lt;a href=&quot;https://github.com/status-im/nim-waku/issues/419&quot;&gt;#419&lt;/a&gt;)
and fix small nim-waku &amp;amp; nim-libp2p bugs
(&lt;a href=&quot;https://github.com/status-im/nim-waku/issues/411&quot;&gt;#411&lt;/a&gt;, &lt;a href=&quot;https://github.com/status-im/nim-waku/issues/439&quot;&gt;#439&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.
A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.
Unfortunately websocket support is not yet available in &lt;a href=&quot;https://github.com/status-im/nim-libp2p/issues/407&quot;&gt;nim-libp2p&lt;/a&gt; so 
we ended up deploying &lt;a href=&quot;https://github.com/novnc/websockify&quot;&gt;websockify&lt;/a&gt; alongside wakunode2 instances.&lt;/p&gt;

&lt;p&gt;As we built the &lt;a href=&quot;https://github.com/status-im/js-waku/tree/main/examples/web-chat&quot;&gt;web chat app&lt;/a&gt;,
we were able to fine tune the API to provide a simple and succinct interface.
You can start a node, connect to other nodes and send a message in less than ten lines of code:&lt;/p&gt;

&lt;div class=&quot;language-javascript highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;Waku&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;dl&quot;&gt;'&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;js-waku&lt;/span&gt;&lt;span class=&quot;dl&quot;&gt;'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;

&lt;span class=&quot;kd&quot;&gt;const&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;waku&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;Waku&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;create&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;({});&lt;/span&gt;

&lt;span class=&quot;kd&quot;&gt;const&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;nodes&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;getStatusFleetNodes&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;Promise&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;all&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;nodes&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;map&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;addr&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&amp;gt;&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;waku&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;dial&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;addr&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)));&lt;/span&gt;

&lt;span class=&quot;kd&quot;&gt;const&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;msg&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;WakuMessage&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;fromUtf8String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;dl&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;Here is a message!&lt;/span&gt;&lt;span class=&quot;dl&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;dl&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/my-cool-app/1/my-use-case/proto&lt;/span&gt;&lt;span class=&quot;dl&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;waku&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;relay&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;send&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;msg&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We have also put a bounty at &lt;a href=&quot;https://0xhack.dev/&quot;&gt;0xHack&lt;/a&gt; for using JS-Waku
and running a &lt;a href=&quot;https://www.youtube.com/watch?v=l77j0VX75QE&quot;&gt;workshop&lt;/a&gt;.
We were thrilled to have a couple of hackers create new software using our libraries.
One of the projects aimed to create a decentralised, end-to-end encrypted messenger app,
similar to what the &lt;a href=&quot;https://rfc.vac.dev/spec/20/&quot;&gt;ETH-DM&lt;/a&gt; protocol aims to achieve.
Another project was a decentralised Twitter platform.
Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.&lt;/p&gt;

&lt;p&gt;As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network
while preserving this out of the box experience.&lt;/p&gt;

&lt;h2 id=&quot;whats-next&quot;&gt;What’s next?&lt;/h2&gt;

&lt;p&gt;Next, we are directing our attention towards &lt;a href=&quot;https://github.com/status-im/js-waku/issues/68&quot;&gt;Developer Experience&lt;/a&gt;.
We already have &lt;a href=&quot;https://www.npmjs.com/package/js-waku&quot;&gt;documentation&lt;/a&gt; available but we want to provide more:
&lt;a href=&quot;https://github.com/status-im/js-waku/issues/56&quot;&gt;Tutorials&lt;/a&gt;, various examples
and showing how &lt;a href=&quot;https://github.com/status-im/js-waku/issues/72&quot;&gt;JS-Waku can be used with Web3&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets.
We think JS-Waku builds a strong case for machine-to-machine (M2M) communications.
The first use cases we are looking into are dApp notifications:
Enabling dApp to notify their user directly in their wallets!
Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified
of events such as DAO voting.&lt;/p&gt;

&lt;p&gt;We already have some POC in the pipeline to enable voting and polling on the Waku network,
allowing users to save gas by &lt;strong&gt;not&lt;/strong&gt; broadcasting each individual vote on the blockchain.&lt;/p&gt;

&lt;p&gt;To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples
of signing, validating, encrypting and decrypting messages using Web3.
Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users’ Ethereum identity.&lt;/p&gt;

&lt;p&gt;As you can read, we have grand plans for JS-Waku and Waku v2.
There is a lot to do, and we would love some help so feel free to
check out the new role in our team:
&lt;a href=&quot;https://status.im/our_team/jobs.html?gh_jid=3157894&quot;&gt;js-waku: Wallet &amp;amp; Dapp Integration Developer&lt;/a&gt;.
We also have a number of &lt;a href=&quot;https://status.im/our_team/jobs.html&quot;&gt;positions&lt;/a&gt; open to work on Waku protocol and nim-waku.&lt;/p&gt;

&lt;p&gt;If you are as excited as us by JS-Waku, why not build a dApp with it?
You can find documentation on the &lt;a href=&quot;https://www.npmjs.com/package/js-waku&quot;&gt;npmjs page&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Whether you are a developer, you can come chat with us using &lt;a href=&quot;https://status-im.github.io/js-waku/&quot;&gt;WakuJS Web Chat&lt;/a&gt;
or &lt;a href=&quot;https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md&quot;&gt;chat2&lt;/a&gt;.
You can get support in #dappconnect-support on &lt;a href=&quot;https://discord.gg/j5pGbn7MHZ&quot;&gt;Vac Discord&lt;/a&gt; or &lt;a href=&quot;https://t.me/dappconnectsupport&quot;&gt;Telegram&lt;/a&gt;.
If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.&lt;/p&gt;</content><author><name>franck</name></author><category term="platform" /><summary type="html">For the past 3 months, we have been working on bringing Waku v2 to the browser. Our aim is to empower dApps with Waku v2, and it led to the creation of a new library. We believe now is good time to introduce it!</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/js-waku-gist.png" /><media:content medium="image" url="https://vac.dev/assets/img/js-waku-gist.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Privacy-preserving p2p economic spam protection in Waku v2</title><link href="https://vac.dev/rln-relay" rel="alternate" type="text/html" title="Privacy-preserving p2p economic spam protection in Waku v2" /><published>2021-03-05T04:00:00+00:00</published><updated>2021-03-05T04:00:00+00:00</updated><id>https://vac.dev/rln-relay</id><content type="html" xml:base="https://vac.dev/rln-relay">&lt;h1 id=&quot;introduction&quot;&gt;Introduction&lt;/h1&gt;
&lt;p&gt;This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol&lt;sup id=&quot;fnref:2&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:2&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;1&lt;/a&gt;&lt;/sup&gt; through Rate-Limiting Nullifiers&lt;sup id=&quot;fnref:3&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:3&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;2&lt;/a&gt;&lt;/sup&gt; &lt;sup id=&quot;fnref:4&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:4&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;3&lt;/a&gt;&lt;/sup&gt; or RLN for short.&lt;/p&gt;

&lt;p&gt;Let me give a little background about Waku(v2)&lt;sup id=&quot;fnref:1&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:1&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;4&lt;/a&gt;&lt;/sup&gt;. Waku is a privacy-preserving peer-to-peer (p2p) messaging protocol for resource-restricted devices. Being p2p means that Waku relies on &lt;strong&gt;No&lt;/strong&gt; central server. Instead, peers collaboratively deliver messages in the network. Waku uses GossipSub&lt;sup id=&quot;fnref:16&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:16&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;5&lt;/a&gt;&lt;/sup&gt; as the underlying routing protocol (as of the writeup of this post). At a high level, GossipSub is based on publisher-subscriber architecture. That is, &lt;em&gt;peers, congregate around topics they are interested in and can send messages to topics. Each message gets delivered to all peers subscribed to the topic&lt;/em&gt;. In GossipSub, a peer has a constant number of direct connections/neighbors. In order to publish a message, the author forwards its message to a subset of neighbors. The neighbors proceed similarly till the message gets propagated in the network of the subscribed peers. The message publishing and routing procedures are part of the Waku Relay&lt;sup id=&quot;fnref:17&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:17&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;6&lt;/a&gt;&lt;/sup&gt; protocol.&lt;/p&gt;
&lt;p align=&quot;center&quot;&gt;
    &lt;img src=&quot;../assets/img/rln-relay/rln-relay-overview.png&quot; width=&quot;200%&quot; /&gt;
    &lt;br /&gt;
    Figure 1: An overview of privacy-preserving p2p economic spam protection in Waku v2 RLN-Relay protocol.
&lt;/p&gt;

&lt;h2 id=&quot;what-do-we-mean-by-spamming&quot;&gt;What do we mean by spamming?&lt;/h2&gt;
&lt;p&gt;In centralized messaging systems, a spammer usually indicates an entity that uses the messaging system to send an unsolicited message (spam) to large numbers of recipients. However, in Waku with a p2p architecture, spam messages not only affect the recipients but also all the other peers involved in the routing process as they have to spend their computational power/bandwidth/storage capacity on processing spam messages. As such, we define a spammer as an entity that uses the messaging system to publish a large number of messages in a short amount of time. The messages issued in this way are called spam. In this definition, we disregard the intention of the spammer as well as the content of the message and the number of recipients.&lt;/p&gt;

&lt;h2 id=&quot;possible-solutions&quot;&gt;Possible Solutions&lt;/h2&gt;
&lt;p&gt;Has the spamming issue been addressed before? Of course yes! Here is an overview of the spam protection techniques with their trade-offs and use-cases. In this overview, we distinguish between protection techniques that are targeted for centralized messaging systems and those for p2p architectures.&lt;/p&gt;

&lt;h3 id=&quot;centralized-messaging-systems&quot;&gt;Centralized Messaging Systems&lt;/h3&gt;
&lt;p&gt;In traditional centralized messaging systems, spam usually signifies unsolicited messages sent in bulk or messages with malicious content like malware. Protection mechanisms include&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;authentication through some piece of personally identifiable information e.g., phone number&lt;/li&gt;
  &lt;li&gt;checksum-based filtering to protect against messages sent in bulk&lt;/li&gt;
  &lt;li&gt;challenge-response systems&lt;/li&gt;
  &lt;li&gt;content filtering on the server or via a proxy application&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These methods exploit the fact that the messaging system is centralized and a global view of the users’ activities is available based on which spamming patterns can be extracted and defeated accordingly.  Moreover, users are associated with an identifier e.g., a username which enables the server to profile each user e.g., to detect suspicious behavior like spamming. Such profiling possibility is against the user’s anonymity and privacy.&lt;/p&gt;

&lt;p&gt;Among the techniques enumerated above, authentication through phone numbers is a some-what economic-incentive measure as providing multiple valid phone numbers will be expensive for the attacker. Notice that while using an expensive authentication method can reduce the number of accounts owned by a single spammer, cannot address the spam issue entirely. This is because the spammer can still send bulk messages through one single account. For this approach to be effective, a centralized mediator is essential. That is why such a solution would not fit the p2p environments where no centralized control exists.&lt;/p&gt;

&lt;h3 id=&quot;p2p-systems&quot;&gt;P2P Systems&lt;/h3&gt;
&lt;p&gt;What about spam prevention in p2p messaging platforms? There are two techniques, namely &lt;em&gt;Proof of Work&lt;/em&gt;&lt;sup id=&quot;fnref:8&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:8&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;7&lt;/a&gt;&lt;/sup&gt;  deployed by Whisper&lt;sup id=&quot;fnref:9&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:9&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;8&lt;/a&gt;&lt;/sup&gt; and &lt;em&gt;Peer scoring&lt;/em&gt;&lt;sup id=&quot;fnref:6&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:6&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;9&lt;/a&gt;&lt;/sup&gt; method (namely reputation-based approach) adopted by LibP2P. However, each of these solutions has its own shortcomings for real-life use-cases as explained below.&lt;/p&gt;

&lt;h4 id=&quot;proof-of-work&quot;&gt;Proof of work&lt;/h4&gt;
&lt;p&gt;The idea behind the Proof Of Work i.e., POW&lt;sup id=&quot;fnref:8:1&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:8&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;7&lt;/a&gt;&lt;/sup&gt; is to make messaging a computationally costly operation hence lowering the messaging rate of &lt;strong&gt;all&lt;/strong&gt; the peers including the spammers. In specific, the message publisher has to solve a puzzle and the puzzle is to find a nonce such that the hash of the message concatenated with the nonce has at least z leading zeros. z is known as the difficulty of the puzzle. Since the hash function is one-way, peers have to brute-force to find a nonce. Hashing is a computationally-heavy operation so is the brute-force. While solving the puzzle is computationally expensive, it is comparatively cheap to verify the solution.&lt;/p&gt;

&lt;p&gt;POW is also used as the underlying mining algorithm in Ethereum and Bitcoin blockchain. There, the goal is to contain the mining speed and allow the decentralized network to come to a consensus, or agree on things like account balances and the order of transactions.&lt;/p&gt;

&lt;p&gt;While the use of POW makes perfect sense in Ethereum / Bitcoin blockchain, it shows practical issues in heterogeneous p2p messaging systems with resource-restricted peers. Some peers won’t be able to carry the designated computation and will be effectively excluded. Such exclusion showed to be practically an issue in applications like Status, which used to rely on POW for spam-protection, to the extent that the difficulty level had to be set close to zero.&lt;/p&gt;

&lt;h4 id=&quot;peer-scoring&quot;&gt;Peer Scoring&lt;/h4&gt;
&lt;p&gt;The peer scoring method&lt;sup id=&quot;fnref:6:1&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:6&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;9&lt;/a&gt;&lt;/sup&gt; that is utilized by libp2p is to limit the number of messages issued by a peer in connection to another peer. That is each peer monitors all the peers to which it is directly connected and adjusts their messaging quota i.e., to route or not route their messages depending on their past activities. For example, if a peer detects its neighbor is sending more than x messages per month, can drop its quota to z.x where z is less than one.  The shortcoming of this solution is that scoring is based on peers’ local observations and the concept of the score is defined in relation to one single peer. This leaves room for an attack where a spammer can make connections to k peers in the system and publishes k.(x-1)  messages by exploiting all of its k connections. Another attack scenario is through botnets consisting of a large number of e.g., a million bots. The attacker rents a botnet and inserts each of them as a legitimate peer to the network and each can publish x-1 messages per month&lt;sup id=&quot;fnref:7&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:7&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;10&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;

&lt;h4 id=&quot;economic-incentive-spam-protection&quot;&gt;Economic-Incentive Spam protection&lt;/h4&gt;
&lt;p&gt;Is this the end of our spam-protection journey? Shall we simply give up and leave spammers be? Certainly not!
Waku RLN-Relay gives us a p2p spam-protection method which:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;suits &lt;strong&gt;p2p&lt;/strong&gt; systems and does not rely on any central entity.&lt;/li&gt;
  &lt;li&gt;is &lt;strong&gt;efficient&lt;/strong&gt;  i.e., with no unreasonable computational, storage, memory, and bandwidth requirement! as such, it fits the network of &lt;strong&gt;heterogeneous&lt;/strong&gt; peers.&lt;/li&gt;
  &lt;li&gt;respects users &lt;strong&gt;privacy&lt;/strong&gt; unlike reputation-based and centralized methods.&lt;/li&gt;
  &lt;li&gt;deploys &lt;strong&gt;economic-incentives&lt;/strong&gt; to contain spammers’ activity. Namely, there is a financial sacrifice for those who want to spam the system. How? follow along …&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We devise a general rule to save everyone’s life and that is&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No one can publish more than M messages per epoch without being financially charged!&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;We set M to 1 for now, but this can be any arbitrary value. You may be thinking “This is too restrictive! Only one per epoch?”. Don’t worry, we set the epoch to a reasonable value so that it does not slow down the communication of innocent users but will make the life of spammers harder! Epoch here can be every second, as defined by UTC date-time +-20s.&lt;/p&gt;

&lt;p&gt;The remainder of this post is all about the story of how to enforce this limit on each user’s messaging rate as well as how to impose the financial cost when the limit gets violated.  This brings us to the Rate Limiting Nullifiers and how we integrate this technique into Waku v2 (in specific the Waku Relay protocol) to protect our valuable users against spammers.&lt;/p&gt;

&lt;h1 id=&quot;technical-terms&quot;&gt;Technical Terms&lt;/h1&gt;
&lt;p&gt;&lt;strong&gt;Zero-knowledge proof&lt;/strong&gt;: Zero-knowledge proof (ZKP)&lt;sup id=&quot;fnref:14&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:14&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;11&lt;/a&gt;&lt;/sup&gt; allows a &lt;em&gt;prover&lt;/em&gt; to show a &lt;em&gt;verifier&lt;/em&gt; that they know something, without revealing what that something is. This means you can do the trust-minimized computation that is also privacy-preserving. As a basic example, instead of showing your ID when going to a bar you simply give them proof that you are over 18, without showing the doorman your id. In this write-up, by ZKP we essentially mean zkSNARK&lt;sup id=&quot;fnref:15&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:15&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;12&lt;/a&gt;&lt;/sup&gt; which is one of the many types of ZKPs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Threshold Secret Sharing Scheme&lt;/strong&gt;: (m,n) Threshold secret-sharing is a method by which you can split a secret value s into n pieces in a way that the secret s can be reconstructed by having m pieces (m &amp;lt;= n). The economic-incentive spam protection utilizes a (2,n) secret sharing realized by Shamir Secret Sharing Scheme&lt;sup id=&quot;fnref:13&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:13&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;13&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;

&lt;h1 id=&quot;overview-economic-incentive-spam-protection-through-rate-limiting-nullifiers&quot;&gt;Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers&lt;/h1&gt;
&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;: We started the idea of economic-incentive spam protection more than a year ago and conducted a feasibility study to identify blockers and unknowns. The results are published in our prior &lt;a href=&quot;https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks&quot;&gt;post&lt;/a&gt;. Since then major progress has been made and the prior identified blockers that are listed below are now addressed. Kudos to &lt;a href=&quot;https://github.com/barryWhiteHat&quot;&gt;Barry WhiteHat&lt;/a&gt;, &lt;a href=&quot;https://github.com/kilic&quot;&gt;Onur Kilic&lt;/a&gt;, &lt;a href=&quot;https://github.com/weijiekoh/perpetualpowersoftau&quot;&gt;Koh Wei Jie&lt;/a&gt; for all of their hard work, research, and development which made this progress possible.&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;the proof time&lt;sup id=&quot;fnref:22&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:22&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;14&lt;/a&gt;&lt;/sup&gt; which was initially in the order of minutes ~10 mins and now is almost 0.5 seconds&lt;/li&gt;
  &lt;li&gt;the prover key size&lt;sup id=&quot;fnref:21&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:21&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;15&lt;/a&gt;&lt;/sup&gt; which was initially ~110MB and now is ~3.9MB&lt;/li&gt;
  &lt;li&gt;the lack of Shamir logic&lt;sup id=&quot;fnref:19&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:19&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;16&lt;/a&gt;&lt;/sup&gt; which is now implemented and part of the RLN repository&lt;sup id=&quot;fnref:4:1&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:4&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;3&lt;/a&gt;&lt;/sup&gt;&lt;/li&gt;
  &lt;li&gt;the concern regarding the potential multi-party computation for the trusted setup of zkSNARKs which got resolved&lt;sup id=&quot;fnref:20&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:20&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;17&lt;/a&gt;&lt;/sup&gt;&lt;/li&gt;
  &lt;li&gt;the lack of end-to-end integration that now we made it possible, have it implemented, and are going to present it in this post. New blockers are also sorted out during the e2e integration which we will discuss in the &lt;a href=&quot;#feasibility-and-open-issues&quot;&gt;Feasibility and Open Issues&lt;/a&gt; section.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Now that you have more context, let’s see how the final solution works. The fundamental point is to make it economically costly to send more than your share of messages and to do so in a privacy-preserving and e2e fashion. To do that we have the following components:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;1- &lt;strong&gt;Group&lt;/strong&gt;: We manage all the peers inside a large group (later we can split peers into smaller groups, but for now consider only one). The group management is done via a smart contract which is devised for this purpose and is deployed on the Ethereum blockchain.&lt;/li&gt;
  &lt;li&gt;2- &lt;strong&gt;Membership&lt;/strong&gt;: To be able to send messages and in specific for the published messages to get routed by all the peers, publishing peers have to register to the group. Membership involves setting up public and private key pairs (think of it as the username and password). The private key remains at the user side but the public key becomes a part of the group information on the contract (publicly available) and everyone has access to it. Public keys are not human-generated (like usernames) and instead they are random numbers, as such, they do not reveal any information about the owner (think of public keys as pseudonyms). Registration is mandatory for the users who want to publish a message, however, users who only want to listen to the messages are more than welcome and do not have to register in the group.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Membership fee&lt;/strong&gt;: Membership is not for free! each peer has to lock a certain amount of funds during the registration (this means peers have to have an Ethereum account with sufficient balance for this sake). This fund is safely stored on the contract and remains intact unless the peer attempts to break the rules and publish more than one message per epoch.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Zero-knowledge Proof of membership&lt;/strong&gt;: Do you want your message to get routed to its destination, fine, but you have to prove that you are a member of the group (sorry, no one can escape the registration phase!). Now, you may be thinking that should I attach my public key to my message to prove my membership? Absolutely Not! we said that our solution respects privacy! membership proofs are done in a zero-knowledge manner that is each message will carry cryptographic proof asserting that “the message is generated by one of the current members of the group”, so your identity remains private and your anonymity is preserved!&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Slashing through secret sharing&lt;/strong&gt;: Till now it does not seem like we can catch spammers, right? yes, you are right! now comes the exciting part, detecting spammers and slashing them. The core idea behind the slashing is that each publishing peer (not routing peers!) has to integrate a secret share of its private key inside the message. The secret share is deterministically computed over the private key and the current epoch. The content of this share is harmless for the peer’s privacy (it looks random) unless the peer attempts to publish more than one message in the same epoch hence disclosing more than one secret share of its private key. Indeed two distinct shares of the private key under the same epoch are enough to reconstruct the entire private key. Then what should you do with the recovered private key? hurry up! go to the contract and withdraw the private key and claim its fund and get rich!! Are you thinking what if spammers attach junk values instead of valid secret shares? Of course, that wouldn’t be cool! so, there is a zero-knowledge proof for this sake as well where the publishing peer has to prove that the secret shares are generated correctly.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A high-level overview of the economic spam protection is shown in Figure 1.&lt;/p&gt;

&lt;h1 id=&quot;flow&quot;&gt;Flow&lt;/h1&gt;
&lt;p&gt;In this section, we describe the flow of the economic-incentive spam detection mechanism from the viewpoint of a single peer. An overview of this flow is provided in Figure 3.&lt;/p&gt;

&lt;h2 id=&quot;setup-and-registration&quot;&gt;Setup and Registration&lt;/h2&gt;
&lt;p&gt;A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members’ public keys. An overview of registration is illustrated in Figure 2.&lt;/p&gt;

&lt;p&gt;For the registration, a peer creates a transaction that sends x amount of Ether to the contract. The peer who has the “private key” &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; associated with that deposit would be able to withdraw x Ether by providing valid proof. Note that  &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch.
The following relation holds between the &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;pk&lt;/code&gt; i.e., &lt;code class=&quot;highlighter-rouge&quot;&gt;pk = H(sk)&lt;/code&gt; where &lt;code class=&quot;highlighter-rouge&quot;&gt;H&lt;/code&gt; denotes a hash function.&lt;/p&gt;
&lt;p align=&quot;center&quot;&gt;
    &lt;img src=&quot;../assets/img/rln-relay/rln-relay.png&quot; /&gt;
    &lt;br /&gt;
    Figure 2: Registration.
&lt;/p&gt;

&lt;h2 id=&quot;maintaining-the-membership-merkle-tree&quot;&gt;Maintaining the membership Merkle Tree&lt;/h2&gt;
&lt;p&gt;The ZKP of membership that we mentioned before relies on the representation of the entire group as a &lt;a href=&quot;&quot;&gt;Merkle Tree&lt;/a&gt;. The tree construction and maintenance is delegated to the peers (the initial idea was to keep the tree on the chain as part of the contract, however, the cost associated with member deletion and insertion was high and unreasonable, please see &lt;a href=&quot;#Feasibility-and-Open-Issues&quot;&gt;Feasibility and Open Issues&lt;/a&gt; for more details). As such, each peer needs to build the tree locally and sync itself with the contract updates (peer insertion and deletion) to mirror them on its tree.
Two pieces of information of the tree are important as they enable peers to generate zero-knowledge proofs. One is the root of the tree and the other is the membership proof (or the authentication path). The tree root is public information whereas the membership proof is private data (or more precisely the index of the peer in the tree).&lt;/p&gt;

&lt;h2 id=&quot;publishing&quot;&gt;Publishing&lt;/h2&gt;
&lt;p&gt;In order to publish at a given epoch, each message must carry a proof i.e., a zero-knowledge proof signifying that the publishing peer is a  registered member, and has not exceeded the messaging rate at the given epoch.&lt;/p&gt;

&lt;p&gt;Recall that the enforcement of the messaging rate was through associating a secret shared version of the peer’s &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; into the message together with a ZKP that the secret shares are constructed correctly. As for the secret sharing part, the peer generates the following data:&lt;/p&gt;
&lt;ol&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;shareX&lt;/code&gt;&lt;/li&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;shareY&lt;/code&gt;&lt;/li&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The pair (&lt;code class=&quot;highlighter-rouge&quot;&gt;shareX&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;shareY&lt;/code&gt;)  is the secret shared version of &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; that are generated using Shamir secret sharing scheme. Having two such pairs for an identical &lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt; results in full disclosure of peer’s &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; and hence burning the associated deposit. Note that the &lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt; is a deterministic value derived from &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;epoch&lt;/code&gt; therefore any two messages issued by the same peer (i.e., using the same &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt;) for the same &lt;code class=&quot;highlighter-rouge&quot;&gt;epoch&lt;/code&gt; are guaranteed to have identical &lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt;s.&lt;/p&gt;

&lt;p&gt;Finally, the peer generates a zero-knowledge proof  &lt;code class=&quot;highlighter-rouge&quot;&gt;zkProof&lt;/code&gt;  asserting the membership of the peer in the group and the correctness of the attached secret share (&lt;code class=&quot;highlighter-rouge&quot;&gt;shareX&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;shareY&lt;/code&gt;) and the &lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt;. In order to generate a valid proof, the peer needs to have two private inputs i.e., its &lt;code class=&quot;highlighter-rouge&quot;&gt;sk&lt;/code&gt; and its authentication path. Other inputs are the tree root, epoch, and the content of the message.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Privacy Hint:&lt;/strong&gt; Note that the authentication path of each peer depends on the recent list of members (hence changes when new peers register or leave). As such, it is recommended (and necessary for privacy/anonymity) that the publisher updates her authentication path based on the latest status of the group and attempts the proof using the updated version.&lt;/p&gt;

&lt;p&gt;An overview of the publishing procedure is provided in Figure 3.&lt;/p&gt;

&lt;h2 id=&quot;routing&quot;&gt;Routing&lt;/h2&gt;
&lt;p&gt;Upon the receipt of a message, the routing peer needs to decide whether to route it or not. This decision relies on the following factors:
1) If the epoch value attached to the message has a non-reasonable gap with the routing peer’s current epoch then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs). 
2) The message MUST contain valid proof that gets verified by the routing peer. 
If the preceding checks are passed successfully, then the message is relayed. In case of an invalid proof, the message is dropped. If spamming is detected, the publishing peer gets slashed (see &lt;a href=&quot;#Spam-Detection-and-Slashing&quot;&gt;Spam Detection and Slashing&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;An overview of the routing procedure is provided in Figure 3.&lt;/p&gt;

&lt;h3 id=&quot;spam-detection-and-slashing&quot;&gt;Spam Detection and Slashing&lt;/h3&gt;
&lt;p&gt;In order to enable local spam detection and slashing, routing peers MUST record the &lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;shareX&lt;/code&gt;, and &lt;code class=&quot;highlighter-rouge&quot;&gt;shareY&lt;/code&gt; of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.&lt;/p&gt;
&lt;ol&gt;
  &lt;li&gt;The routing peer first verifies the &lt;code class=&quot;highlighter-rouge&quot;&gt;zkProof&lt;/code&gt; and drops the message if not verified.&lt;/li&gt;
  &lt;li&gt;Otherwise, it checks whether a message with an identical &lt;code class=&quot;highlighter-rouge&quot;&gt;nullifier&lt;/code&gt; has already been relayed.
    &lt;ul&gt;
      &lt;li&gt;a) If such message exists and its &lt;code class=&quot;highlighter-rouge&quot;&gt;shareX&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;shareY&lt;/code&gt; components are different from the incoming message, then slashing takes place (if the &lt;code class=&quot;highlighter-rouge&quot;&gt;shareX&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;shareY&lt;/code&gt; fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).&lt;/li&gt;
      &lt;li&gt;b) If none found, then the message gets relayed.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;An overview of the slashing procedure is provided in Figure 3.&lt;/p&gt;

&lt;p align=&quot;center&quot;&gt;
    &lt;img src=&quot;../assets/img/rln-relay/rln-message-verification.png&quot; /&gt;
    &lt;br /&gt;
    Figure 3: Publishing, Routing and Slashing workflow.
&lt;/p&gt;

&lt;h1 id=&quot;feasibility-and-open-issues&quot;&gt;Feasibility and Open Issues&lt;/h1&gt;
&lt;p&gt;We’ve come a long way since a year ago, blockers resolved, now we have implemented it end-to-end. We learned lot and could identify further issues and unknowns some of which are blocking getting to production. The summary of the identified issues  are presented below.&lt;/p&gt;

&lt;h2 id=&quot;storage-overhead-per-peer&quot;&gt;Storage overhead per peer&lt;/h2&gt;
&lt;p&gt;Currently, peers are supposed to maintain the entire tree locally and it imposes storage overhead which is linear in the size of the group (see this &lt;a href=&quot;https://github.com/vacp2p/research/issues/57&quot;&gt;issue&lt;/a&gt;&lt;sup id=&quot;fnref:11&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:11&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;18&lt;/a&gt;&lt;/sup&gt; for more details). One way to cope with this is to use the light-node and full-node paradigm in which only a subset of peers who are more resourceful retain the tree whereas the light nodes obtain the necessary information by interacting with the full nodes. Another way to approach this problem is through a more storage efficient method (as described in this research issue&lt;sup id=&quot;fnref:12&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:12&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;19&lt;/a&gt;&lt;/sup&gt;) where peers store a partial view of the tree instead of the entire tree. Keeping the partial view lowers the storage complexity to O(log(N)) where N is the size of the group. There are still unknown unknowns to this solution, as such, it must be studied further to become fully functional.&lt;/p&gt;

&lt;h2 id=&quot;cost-effective-way-of-member-insertion-and-deletion&quot;&gt;Cost-effective way of member insertion and deletion&lt;/h2&gt;
&lt;p&gt;Currently, the cost associated with RLN-Relay membership is around 30 USD&lt;sup id=&quot;fnref:10&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:10&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;20&lt;/a&gt;&lt;/sup&gt;. We aim at finding a more cost-effective approach. Please feel free to share with us your solution ideas in this regard in this &lt;a href=&quot;https://github.com/vacp2p/research/issues/56&quot;&gt;issue&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;exceeding-the-messaging-rate-via-multiple-registrations&quot;&gt;Exceeding the messaging rate via multiple registrations&lt;/h2&gt;
&lt;p&gt;While the economic-incentive solution has an economic incentive to discourage spamming, we should note that there is still &lt;strong&gt;expensive attack(s)&lt;/strong&gt;&lt;sup id=&quot;fnref:23&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:23&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;21&lt;/a&gt;&lt;/sup&gt; that a spammer can launch to break the messaging rate limit. That is, the attacker can pay for multiple legit registrations e.g., k, hence being able to publish k messages per epoch. We believe that the higher the membership fee is, the less probable would be such an attack, hence a stronger level of spam-protection can be achieved. Following this argument, the high fee associated with the membership (which we listed above as an open problem) can indeed be contributing to a better protection level.&lt;/p&gt;

&lt;h1 id=&quot;conclusion-and-future-steps&quot;&gt;Conclusion and Future Steps&lt;/h1&gt;
&lt;p&gt;As discussed in this post, Waku RLN Relay can achieve a privacy-preserving economic spam protection through rate-limiting nullifiers. The idea is to financially discourage peers from publishing more than one message per epoch. In specific, exceeding the messaging rate results in a financial charge. Those who violate this rule are called spammers and their messages are spam. The identification of spammers does not rely on any central entity. Also, the financial punishment of spammers is cryptographically guaranteed. 
In this solution, privacy is guaranteed since: 1) Peers do not have to disclose any piece of personally identifiable information in any phase i.e., neither in the registration nor in the messaging phase 2) Peers can prove that they have not exceeded the messaging rate in a zero-knowledge manner and without leaving any trace to their membership accounts. 
Furthermore, all the computations are light hence this solution fits the heterogenous p2p messaging system. Note that  the zero-knowledge proof parts are handled through zkSNARKs  and  the benchmarking result can be found in the RLN benchmark report&lt;sup id=&quot;fnref:5&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:5&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;22&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Future steps&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;We are still at the PoC level, and the development is in progress. As our future steps,&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;we would like to evaluate the running time associated with the Merkle tree operations. Indeed, the need to locally store Merkle tree on each peer was one of the unknowns discovered during this PoC and yet the concrete benchmarking result in this regard is not available.&lt;/li&gt;
  &lt;li&gt;We would also like to pursue our storage-efficient Merkle Tree maintenance solution in order to lower the storage overhead of peers.&lt;/li&gt;
  &lt;li&gt;In line with the storage optimization, the full-node light-node structure is another path to follow.&lt;/li&gt;
  &lt;li&gt;Another possible improvement is to replace the membership contract with a distributed group management scheme e.g., through distributed hash tables. This is to address possible performance issues that the interaction with the Ethereum blockchain may cause.  For example, the registration transactions are subject to delay as they have to be mined before being visible in the state of the membership contract. This means peers have to wait for some time before being able to publish any message.&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;acknowledgement&quot;&gt;Acknowledgement&lt;/h1&gt;
&lt;p&gt;Thanks to Onur Kılıç for his explanation and pointers and for assisting with development and runtime issues. Also thanks to Barry Whitehat for his time and insightful comments. Special thanks to Oskar Thoren for his constructive comments and his guides during the development of this PoC and the writeup of this post.&lt;/p&gt;

&lt;h1 id=&quot;references&quot;&gt;References&lt;/h1&gt;

&lt;div class=&quot;footnotes&quot; role=&quot;doc-endnotes&quot;&gt;
  &lt;ol&gt;
    &lt;li id=&quot;fn:2&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;RLN-Relay specification: https://rfc.vac.dev/spec/17/ &lt;a href=&quot;#fnref:2&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:3&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;RLN documentation: &lt;a href=&quot;https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both&quot;&gt;https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both&lt;/a&gt; &lt;a href=&quot;#fnref:3&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:4&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;RLN repositories: &lt;a href=&quot;https://github.com/kilic/RLN&quot;&gt;https://github.com/kilic/RLN&lt;/a&gt; and &lt;a href=&quot;https://github.com/kilic/rlnapp&quot;&gt;https://github.com/kilic/rlnapp&lt;/a&gt; &lt;a href=&quot;#fnref:4&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt; &lt;a href=&quot;#fnref:4:1&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:1&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Waku v2: https://rfc.vac.dev/spec/10/ &lt;a href=&quot;#fnref:1&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:16&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;GossipSub: &lt;a href=&quot;https://docs.libp2p.io/concepts/publish-subscribe/&quot;&gt;https://docs.libp2p.io/concepts/publish-subscribe/&lt;/a&gt; &lt;a href=&quot;#fnref:16&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:17&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Waku Relay: https://rfc.vac.dev/spec/11/ &lt;a href=&quot;#fnref:17&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:8&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Proof of work: &lt;a href=&quot;http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf&quot;&gt;http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf&lt;/a&gt; and &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf&quot;&gt;https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf&lt;/a&gt; &lt;a href=&quot;#fnref:8&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt; &lt;a href=&quot;#fnref:8:1&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:9&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;EIP-627 Whisper: https://eips.ethereum.org/EIPS/eip-627 &lt;a href=&quot;#fnref:9&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:6&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Peer Scoring: &lt;a href=&quot;https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring&quot;&gt;https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring&lt;/a&gt; &lt;a href=&quot;#fnref:6&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt; &lt;a href=&quot;#fnref:6:1&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:7&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Peer scoring security issues: &lt;a href=&quot;https://github.com/vacp2p/research/issues/44&quot;&gt;https://github.com/vacp2p/research/issues/44&lt;/a&gt; &lt;a href=&quot;#fnref:7&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:14&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Zero Knowledge Proof: &lt;a href=&quot;https://dl.acm.org/doi/abs/10.1145/3335741.3335750&quot;&gt;https://dl.acm.org/doi/abs/10.1145/3335741.3335750&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Zero-knowledge_proof&quot;&gt;https://en.wikipedia.org/wiki/Zero-knowledge_proof&lt;/a&gt; &lt;a href=&quot;#fnref:14&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:15&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;zkSNARKs: &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11&quot;&gt;https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11&lt;/a&gt; and &lt;a href=&quot;https://coinpare.io/whitepaper/zcash.pdf&quot;&gt;https://coinpare.io/whitepaper/zcash.pdf&lt;/a&gt; &lt;a href=&quot;#fnref:15&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:13&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Shamir Secret Sharing Scheme: &lt;a href=&quot;https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing&quot;&gt;https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing&lt;/a&gt; &lt;a href=&quot;#fnref:13&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:22&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;zkSNARKs proof time: &lt;a href=&quot;https://github.com/vacp2p/research/issues/7&quot;&gt;https://github.com/vacp2p/research/issues/7&lt;/a&gt; &lt;a href=&quot;#fnref:22&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:21&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Prover key size: &lt;a href=&quot;https://github.com/vacp2p/research/issues/8&quot;&gt;https://github.com/vacp2p/research/issues/8&lt;/a&gt; &lt;a href=&quot;#fnref:21&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:19&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;The lack of Shamir secret sharing in zkSNARKs: &lt;a href=&quot;https://github.com/vacp2p/research/issues/10&quot;&gt;https://github.com/vacp2p/research/issues/10&lt;/a&gt; &lt;a href=&quot;#fnref:19&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:20&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;The MPC required for zkSNARKs trusted setup: &lt;a href=&quot;https://github.com/vacp2p/research/issues/9&quot;&gt;https://github.com/vacp2p/research/issues/9&lt;/a&gt; &lt;a href=&quot;#fnref:20&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:11&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Storage overhead per peer: &lt;a href=&quot;https://github.com/vacp2p/research/issues/57&quot;&gt;https://github.com/vacp2p/research/issues/57&lt;/a&gt; &lt;a href=&quot;#fnref:11&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:12&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Storage-efficient Merkle Tree maintenance: &lt;a href=&quot;https://github.com/vacp2p/research/pull/54&quot;&gt;https://github.com/vacp2p/research/pull/54&lt;/a&gt; &lt;a href=&quot;#fnref:12&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:10&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Cost-effective way of member insertion and deletion: &lt;a href=&quot;https://github.com/vacp2p/research/issues/56&quot;&gt;https://github.com/vacp2p/research/issues/56&lt;/a&gt; &lt;a href=&quot;#fnref:10&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:23&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Attack on the messaging rate: &lt;a href=&quot;https://github.com/vacp2p/specs/issues/251&quot;&gt;https://github.com/vacp2p/specs/issues/251&lt;/a&gt; &lt;a href=&quot;#fnref:23&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:5&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;RLN Benchmark: &lt;a href=&quot;https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks&quot;&gt;https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks&lt;/a&gt; &lt;a href=&quot;#fnref:5&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
  &lt;/ol&gt;
&lt;/div&gt;</content><author><name>sanaztaheri</name></author><category term="research" /><summary type="html">Introduction This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol1 through Rate-Limiting Nullifiers2 3 or RLN for short. RLN-Relay specification: https://rfc.vac.dev/spec/17/ &amp;#8617; RLN documentation: https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both &amp;#8617; RLN repositories: https://github.com/kilic/RLN and https://github.com/kilic/rlnapp &amp;#8617;</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/rain.png" /><media:content medium="image" url="https://vac.dev/assets/img/rain.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">[Talk] Vac, Waku v2 and Ethereum Messaging</title><link href="https://vac.dev/waku-v2-ethereum-messaging" rel="alternate" type="text/html" title="[Talk] Vac, Waku v2 and Ethereum Messaging" /><published>2020-11-10T04:00:00+00:00</published><updated>2020-11-10T04:00:00+00:00</updated><id>https://vac.dev/waku-v2-ethereum-messaging</id><content type="html" xml:base="https://vac.dev/waku-v2-ethereum-messaging">&lt;p&gt;&lt;em&gt;The following post is a transcript of the talk given at the &lt;a href=&quot;https://www.meetup.com/Taipei-Ethereum-Meetup/events/274033344/&quot;&gt;Taipei Ethereum meetup, November 5&lt;/a&gt;. There is also a &lt;a href=&quot;https://www.youtube.com/watch?v=lUDy1MoeYnI&quot;&gt;video recording&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;0-introduction&quot;&gt;0. Introduction&lt;/h2&gt;

&lt;p&gt;Hi! My name is Oskar and I’m the protocol research lead at Vac. This talk will be divided into two parts. First I’ll talk about the journey from Whisper, to Waku v1 and now to Waku v2. Then I’ll talk about messaging in Ethereum. After this talk, you should have an idea of what Waku v2 is, the problems it is trying to solve, as well as where it can be useful for messaging in Ethereum.&lt;/p&gt;

&lt;h2 id=&quot;part-1---vac-and-the-journey-from-whisper-to-waku-v1-to-waku-v2&quot;&gt;PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2&lt;/h2&gt;

&lt;h2 id=&quot;1-vac-intro&quot;&gt;1. Vac intro&lt;/h2&gt;

&lt;p&gt;First, what is Vac? Vac grew out of our efforts Status to create a window on to Ethereum and secure messenger. Vac is modular protocol stack for p2p secure messaging, paying special attention to resource restricted devices, privacy and censorship resistance.&lt;/p&gt;

&lt;p&gt;Today we are going to talk mainly about Waku v2, which is the transport privacy / routing aspect of the Vac protocol stack. It sits “above” the p2p overlay, such as libp2p dealing with transports etc, and below a conversational security layer dealing with messaging encryption, such as using Double Ratchet etc.&lt;/p&gt;

&lt;h2 id=&quot;2-whisper-to-waku-v1&quot;&gt;2. Whisper to Waku v1&lt;/h2&gt;

&lt;p&gt;In the beginning, there was Whisper. Whisper was part of the holy trinity of Ethereum. You had Ethereum for consensus/computation, Whisper for messaging, and Swarm for storage.&lt;/p&gt;

&lt;p&gt;However, for various reasons, Whisper didn’t get the attention it deserved. Development dwindled, it promised too much and it suffered from many issues, such as being extremely inefficient and not being suitable for running on e.g. mobile phone. Despite this, Status used it in its app from around 2017 to 2019. As far as I know, it was one of very few, if not the only, production uses of Whisper.&lt;/p&gt;

&lt;p&gt;In an effort to solve some of its immediate problems, we forked Whisper into Waku and formalized it with a proper specification. This solved immediate bandwidth issues for light nodes, introduced rate limiting for better spam protection, improved historical message support, etc.&lt;/p&gt;

&lt;p&gt;If you are interested in this journey, checkout the &lt;a href=&quot;https://www.youtube.com/watch?v=6lLT33tsJjs&quot;&gt;EthCC talk Dean and I gave in Paris earlier this year&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Status upgraded to Waku v1 early 2020. What next?&lt;/p&gt;

&lt;h2 id=&quot;3-waku-v1-to-v2&quot;&gt;3. Waku v1 to v2&lt;/h2&gt;

&lt;p&gt;We were far from done. The changes we had made were quite incremental and done in order to get tangible improvements as quickly as possible. This meant we couldn’t address more fundamental issues related to full node routing scalability, running with libp2p for more transports, better security, better spam protection and incentivization.&lt;/p&gt;

&lt;p&gt;This kickstarted Waku v2 efforts, which is what we’ve been working on since July. This work was and is initally centered around a few pieces:&lt;/p&gt;

&lt;p&gt;(a) Moving to libp2p&lt;/p&gt;

&lt;p&gt;(b) Better routing&lt;/p&gt;

&lt;p&gt;(c) Accounting and user-run nodes&lt;/p&gt;

&lt;p&gt;The general theme was: making the Waku network more scalable and robust.&lt;/p&gt;

&lt;p&gt;We also did a scalability study to show at what point the network would run into issues, due to the inherent lack of routing that Whisper and Waku v1 provided.&lt;/p&gt;

&lt;p&gt;You can read more about this &lt;a href=&quot;https://vac.dev/waku-v2-plan&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;35-waku-v2---design-goals&quot;&gt;3.5 Waku v2 - Design goals&lt;/h2&gt;

&lt;p&gt;Taking a step back, what problem does Waku v2 attempt to solve compared to all the other solutions that exists out there? What type of applications should use it and why? We have the following design goals:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Generalized messaging&lt;/strong&gt;. Many applications requires some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Peer-to-peer&lt;/strong&gt;. These applications sometimes have requirements that make them suitable for peer-to-peer solutions.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Resource restricted&lt;/strong&gt;. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:&lt;/p&gt;

    &lt;ul&gt;
      &lt;li&gt;limited bandwidth, CPU, memory, disk, battery, etc&lt;/li&gt;
      &lt;li&gt;not being publicly connectable&lt;/li&gt;
      &lt;li&gt;only being intermittently connected; mostly-offline&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Privacy&lt;/strong&gt;. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;As well as to do so in a modular fashion. Meaning you can find a reasonable trade-off depending on your exact requirements. For example, you usually have to trade off some bandwidth to get metadata protection, and vice versa.&lt;/p&gt;

&lt;p&gt;The concept of designing for resource restricted devices also leads to the concept of adaptive nodes, where you have more of a continuum between full nodes and light nodes. For example, if you switch your phone from mobile data to WiFi you might be able to handle more bandwidth, and so on.&lt;/p&gt;

&lt;h2 id=&quot;4-waku-v2---breakdown&quot;&gt;4. Waku v2 - Breakdown&lt;/h2&gt;

&lt;p&gt;Where is Waku v2 at now, and how is it structured?&lt;/p&gt;

&lt;p&gt;It is running over libp2p and we had our second internal testnet last week or so. As a side note, we name our testnets after subway stations in Taipei, the first one being Nangang, and the most recent one being Dingpu.&lt;/p&gt;

&lt;p&gt;The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.&lt;/li&gt;
  &lt;li&gt;Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.&lt;/li&gt;
  &lt;li&gt;Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.&lt;/li&gt;
  &lt;li&gt;Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.&lt;/li&gt;
  &lt;li&gt;Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.&lt;/li&gt;
  &lt;li&gt;Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.&lt;/p&gt;

&lt;p&gt;You can read more about the breakdown in this &lt;a href=&quot;https://vac.dev/waku-v2-update&quot;&gt;update&lt;/a&gt; though some progress has been made since then, as well was in the &lt;a href=&quot;https://rfc.vac.dev/spec/10&quot;&gt;main Waku v2 spec&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;5-waku-v2---upcoming&quot;&gt;5. Waku v2 - Upcoming&lt;/h2&gt;

&lt;p&gt;What’s coming up next? There are a few things.&lt;/p&gt;

&lt;p&gt;For Status to use it in production, it needs to be integrated into the main app using the Nim Node API. The bridge also needs to be implemented and tested.&lt;/p&gt;

&lt;p&gt;For other users, we are currently overhauling the API to allow usage from a browser, e.g. To make this experience great, there are also a few underlying infrastructure things that we need in nim-libp2p, such as a more secure HTTP server in Nim, Websockets and WebRTC support.&lt;/p&gt;

&lt;p&gt;There are also some changes we made to at what level content encryption happens, and this needs to be made easier to use in the API. This means you can use a node without giving your keys to it, which is useful in some environments.&lt;/p&gt;

&lt;p&gt;More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Better scaling, by using topic sharding.&lt;/li&gt;
  &lt;li&gt;Accounting and user-run nodes, to account for and incentives full nodes.&lt;/li&gt;
  &lt;li&gt;Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.&lt;/li&gt;
  &lt;li&gt;Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As well as better support for Ethereum M2M Messaging. Which is what I’ll talk about next.&lt;/p&gt;

&lt;h2 id=&quot;part-2---ethereum-messaging&quot;&gt;PART 2 - ETHEREUM MESSAGING&lt;/h2&gt;

&lt;p&gt;A lot of what follows is inspired by exploratory work that John Lea has done at Status, previously Head of UX Architecture at Ubuntu.&lt;/p&gt;

&lt;h2 id=&quot;6-ethereum-messaging---why&quot;&gt;6. Ethereum Messaging - Why?&lt;/h2&gt;

&lt;p&gt;It is easy to think that Waku v2 is only for human to human messaging, since that’s how Waku is currently primarily used in the Status app. However, the goal is to be useful for generalized messaging, which includes other type of information as well as machine to machine messaging.&lt;/p&gt;

&lt;p&gt;What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Reducing on-chain transactions&lt;/li&gt;
  &lt;li&gt;Reduce latency for operations&lt;/li&gt;
  &lt;li&gt;Decentralize centrally coordinated services (like WalletConnect)&lt;/li&gt;
  &lt;li&gt;Improve UX of dapps&lt;/li&gt;
  &lt;li&gt;Broadcast live information&lt;/li&gt;
  &lt;li&gt;A message transport layer for state channels&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And so on.&lt;/p&gt;

&lt;h2 id=&quot;7-ethereum-messaging---why-cont&quot;&gt;7. Ethereum Messaging - Why? (Cont)&lt;/h2&gt;

&lt;p&gt;What are some examples of practical things Waku as used for Ethereum Messaging could solve?&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Multisig transfers only needing one on chain transaction&lt;/li&gt;
  &lt;li&gt;DAO votes only needing one one chain transaction&lt;/li&gt;
  &lt;li&gt;Giving dapps ability to direct push notifications to users&lt;/li&gt;
  &lt;li&gt;Giving users ability to directly respond to requests from daps&lt;/li&gt;
  &lt;li&gt;Decentralized Wallet Connect&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Etc.&lt;/p&gt;

&lt;h2 id=&quot;8-whats-needed-to-deliver-this&quot;&gt;8. What’s needed to deliver this?&lt;/h2&gt;

&lt;p&gt;We can break it down into our actors:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Decentralized M2M messaging system (Waku)&lt;/li&gt;
  &lt;li&gt;Native wallets (Argent, Metamask, Status, etc)&lt;/li&gt;
  &lt;li&gt;Dapps that benefit from M2M messaging&lt;/li&gt;
  &lt;li&gt;Users whose problems are being solved&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Each of these has a bunch of requirements in turn. The messaging system needs to be decentralized, scalable, robust, etc. Wallets need support for messaging layer, dapps need to integrate this, etc.&lt;/p&gt;

&lt;p&gt;This is a lot! Growing adoption is a challenge. There is a catch 22 in terms of justifying development efforts for wallets, when no dapps need it, and likewise for dapps when no wallets support Waku. In addition to this, there must be proven usage of Waku before it can be relied on, etc. How can we break this up into smaller pieces of work?&lt;/p&gt;

&lt;h2 id=&quot;9-breaking-up-the-problem-and-a-high-level-roadmap&quot;&gt;9. Breaking up the problem and a high level roadmap&lt;/h2&gt;

&lt;p&gt;We can start small. It doesn’t and need to be used for critical features first. A more hybrid approach can be taken where it acts more as nice-to-haves.&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Forking Whisper and solving scalablity, spam etc issues with it.
This is a work in progress. What we talked about in part 1.&lt;/li&gt;
  &lt;li&gt;Expose messaging API for Dapp developers.&lt;/li&gt;
  &lt;li&gt;Implement decentralized version of WalletConnect.
Currently wallets connect ot dapps with centralized service. Great UX.&lt;/li&gt;
  &lt;li&gt;Solve DAO/Multi-Sig coordination problem.
E.g. send message to wallet-derived key when it is time to sign a transaction.&lt;/li&gt;
  &lt;li&gt;Extend dapp-to-user and user-to-dapp communication to more dapps.
Use lessons learned and examples to drive adoptation for wallets/dapps.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And then build up from there.&lt;/p&gt;

&lt;h2 id=&quot;10-we-are-hiring&quot;&gt;10. We are hiring!&lt;/h2&gt;

&lt;p&gt;A lot of this will happen in Javascript and browsers, since that’s the primarily environment for a lot of wallets and dapps. We are currently hiring for a Waku JS Wallet integration lead to help push this effort further.&lt;/p&gt;

&lt;p&gt;Come talk to me after or &lt;a href=&quot;https://status.im/our_team/open_positions.html?gh_jid=2385338&quot;&gt;apply here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;That’s it! You can find us on Status, Telegram, vac.dev. I’m on twitter &lt;a href=&quot;https://twitter.com/oskarth&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Questions?&lt;/p&gt;

&lt;hr /&gt;</content><author><name>oskarth</name></author><category term="research" /><summary type="html">The following post is a transcript of the talk given at the Taipei Ethereum meetup, November 5. There is also a video recording.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/taipei_ethereum_meetup_slide.png" /><media:content medium="image" url="https://vac.dev/assets/img/taipei_ethereum_meetup_slide.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Waku v2 Update</title><link href="https://vac.dev/waku-v2-update" rel="alternate" type="text/html" title="Waku v2 Update" /><published>2020-09-28T04:00:00+00:00</published><updated>2020-09-28T04:00:00+00:00</updated><id>https://vac.dev/waku-v2-update</id><content type="html" xml:base="https://vac.dev/waku-v2-update">&lt;p&gt;It has been a while since the last post. It is time for an update on Waku v2. Aside from getting more familiar with libp2p (specifically nim-libp2p) and some vacation, what have we been up to? In this post we’ll talk about what we’ve gotten done since last time, and briefly talk about immediate next steps and future. But first, a recap.&lt;/p&gt;

&lt;h1 id=&quot;recap&quot;&gt;Recap&lt;/h1&gt;

&lt;p&gt;In the last post (&lt;a href=&quot;https://vac.dev/waku-v2-plan&quot;&gt;Waku v2 plan&lt;/a&gt;) we explained the rationale of Waku v2 - the current Waku network is fragile and doesn’t scale. To solve this, Waku v2 aims to reduce amplification factors and get more user run nodes. We broke the work down into three separate tracks.&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Track 1 - Move to libp2p&lt;/li&gt;
  &lt;li&gt;Track 2 - Better routing&lt;/li&gt;
  &lt;li&gt;Track 3 - Accounting and user-run nodes&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;As well as various rough components for each track. The primary initial focus is track 1. This means things like: moving to FloodSub, simplify the protocol, core integration, topic interest behavior, historical message caching, and Waku v1&amp;lt;&amp;gt;v2 bridge.&lt;/p&gt;

&lt;h1 id=&quot;current-state&quot;&gt;Current state&lt;/h1&gt;

&lt;p&gt;Let’s talk about the state of specs and our main implementation nim-waku. Then we’ll go over our recent testnet, Nangang, and finish off with a Web PoC.&lt;/p&gt;

&lt;h2 id=&quot;specs&quot;&gt;Specs&lt;/h2&gt;

&lt;p&gt;After some back and forth on how to best structure things, we ended up breaking down the specs into a few pieces. While Waku v2 is best thought of as a cohesive whole in terms of its capabilities, it is made up of several protocols. Here’s a list of the current specs and their status:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;https://rfc.vac.dev/spec/10/&quot;&gt;Main spec&lt;/a&gt; (draft)&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://rfc.vac.dev/spec/11/&quot;&gt;Relay protocol spec&lt;/a&gt; (draft)&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://rfc.vac.dev/spec/12&quot;&gt;Filter protocol spec&lt;/a&gt; (raw)&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://rfc.vac.dev/spec/13&quot;&gt;Store protocol spec&lt;/a&gt; (raw)&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://rfc.vac.dev/spec/15/&quot;&gt;Bridge spec&lt;/a&gt; (raw)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Raw means there is not yet an implementation that corresponds fully to the spec, and draft means there is an implementation that corresponds to the spec. In the interest of space, we won’t go into too much detail on the specs here except to note a few things:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;The relay spec is essentially a thin wrapper on top of PubSub/FloodSub/GossipSub&lt;/li&gt;
  &lt;li&gt;The filter protocol corresponds to previous light client mode in Waku v1&lt;/li&gt;
  &lt;li&gt;The store protocol corresponds to the previous mailserver construct in Waku v1&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The filter and store protocol allow for adaptive nodes, i.e. nodes that have various capabilities. For example, a node being mostly offline, or having limited bandwidth capacity. The bridge spec outlines how to bridge the Waku v1 and v2 networks.&lt;/p&gt;

&lt;h2 id=&quot;implementation&quot;&gt;Implementation&lt;/h2&gt;

&lt;p&gt;The main implementation we are working on is &lt;a href=&quot;https://github.com/status-im/nim-waku/&quot;&gt;nim-waku&lt;/a&gt;. This builds on top of libraries such as &lt;a href=&quot;https://github.com/status-im/nim-libp2p&quot;&gt;nim-libp2p&lt;/a&gt; and others that the &lt;a href=&quot;https://nimbus.team/&quot;&gt;Nimbus team&lt;/a&gt; have been working on as part of their Ethereum 2.0 client.&lt;/p&gt;

&lt;p&gt;Currently nim-waku implements the relay protocol, and is close to implementing filter and store protocol. It also exposes a &lt;a href=&quot;https://github.com/status-im/nim-waku/blob/master/docs/api/v2/node.md&quot;&gt;Nim Node API&lt;/a&gt; that allows libraries such as &lt;a href=&quot;https://github.com/status-im/status-nim&quot;&gt;nim-status&lt;/a&gt; to use it. Additionally, there is also a rudimentary JSON RPC API for command line scripting.&lt;/p&gt;

&lt;h2 id=&quot;nangang-testnet&quot;&gt;Nangang testnet&lt;/h2&gt;

&lt;p&gt;Last week we launched a very rudimentary internal testnet called Nangang. The goal was to test basic connectivity and make sure things work end to end. It didn’t have things like: client integration, encryption, bridging, multiple clients, store/filter protocol, or even a real interface. What it did do is allow Waku developers to “chat” via RPC calls and looking in the log output. Doing this meant we exposed and fixed a few blockers, such as connection issues, deployment, topic subscription management, protocol and node integration, and basic scripting/API usage. After this, we felt confident enough to upgrade the main and relay spec to “draft” status.&lt;/p&gt;

&lt;h2 id=&quot;waku-web-poc&quot;&gt;Waku Web PoC&lt;/h2&gt;

&lt;p&gt;As a bonus, we wanted to see what it’d take to get Waku running in a browser. This is a very powerful capability that enables a lot of use cases, and something that libp2p enables with its multiple transport support.&lt;/p&gt;

&lt;p&gt;Using the current stack, with nim-waku, would require quite a lot of ground work with WASM, WebRTC, Websockets support etc. Instead, we decided to take a shortcut and hack together a JS implementation called &lt;a href=&quot;https://github.com/vacp2p/waku-web-chat/&quot;&gt;Waku Web Chat&lt;/a&gt;. This quick hack wouldn’t be possible without the people behind &lt;a href=&quot;https://github.com/libp2p/js-libp2p-examples/&quot;&gt;js-libp2p-examples&lt;/a&gt; and &lt;a href=&quot;https://github.com/libp2p/js-libp2p&quot;&gt;js-libp2p&lt;/a&gt; and all its libraries. These are people like Jacob Heun, Vasco Santos, and Cayman Nava. Thanks!&lt;/p&gt;

&lt;p&gt;It consists of a brower implementation, a NodeJS implementation and a bootstrap server that acts as a signaling server for WebRTC. It is largely a bastardized version of GossipSub, and while it isn’t completely to spec, it does allow messages originating from a browser to eventually end up at a nim-waku node, and vice versa. Which is pretty cool.&lt;/p&gt;

&lt;h1 id=&quot;coming-up&quot;&gt;Coming up&lt;/h1&gt;

&lt;p&gt;Now that we know what the current state is, what is still missing? what are the next steps?&lt;/p&gt;

&lt;h2 id=&quot;things-that-are-missing&quot;&gt;Things that are missing&lt;/h2&gt;

&lt;p&gt;While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope:&lt;/p&gt;

&lt;p&gt;1) Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level.&lt;/p&gt;

&lt;p&gt;2) Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status (&lt;a href=&quot;https://github.com/status-im/status-nim&quot;&gt;status-nim&lt;/a&gt; has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings
us too…&lt;/p&gt;

&lt;p&gt;3) Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 (&lt;a href=&quot;https://github.com/vacp2p/specs/issues/158&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://github.com/vacp2p/specs/issues/181&quot;&gt;2&lt;/a&gt;). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc).&lt;/p&gt;

&lt;p&gt;4) Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented.&lt;/p&gt;

&lt;p&gt;5) General tightening up of specs and implementation.&lt;/p&gt;

&lt;p&gt;While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope.&lt;/p&gt;

&lt;h2 id=&quot;upcoming&quot;&gt;Upcoming&lt;/h2&gt;

&lt;p&gt;Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details.&lt;/p&gt;

&lt;p&gt;1) Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors.&lt;/p&gt;

&lt;p&gt;2) Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner.&lt;/p&gt;

&lt;p&gt;3) Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet
format can improve things like unlinkability.&lt;/p&gt;

&lt;p&gt;4) zkSnarks RLN for spam protection and incentivization. We studied this &lt;a href=&quot;https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks&quot;&gt;last year&lt;/a&gt; and recent developments have made this relevant to study again. Create an &lt;a href=&quot;https://github.com/vacp2p/specs/issues/189&quot;&gt;experimental spec/PoC&lt;/a&gt; as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this!&lt;/p&gt;

&lt;p&gt;5) Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we’d like to deliver on that.&lt;/p&gt;

&lt;p&gt;As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;If you are feeling adventurous and want to use early stage alpha software, check out the &lt;a href=&quot;https://github.com/status-im/nim-waku/tree/master/docs&quot;&gt;docs&lt;/a&gt;. If you want to read the specs, head over to &lt;a href=&quot;https://rfc.vac.dev/spec/10/&quot;&gt;Waku spec&lt;/a&gt;. If you want to talk with us, join us on &lt;a href=&quot;https://get.status.im/chat/public/vac&quot;&gt;Status&lt;/a&gt; or on &lt;a href=&quot;https://t.me/vacp2p&quot;&gt;Telegram&lt;/a&gt; (they are bridged).&lt;/p&gt;</content><author><name>oskarth</name></author><category term="research" /><summary type="html">It has been a while since the last post. It is time for an update on Waku v2. Aside from getting more familiar with libp2p (specifically nim-libp2p) and some vacation, what have we been up to? In this post we’ll talk about what we’ve gotten done since last time, and briefly talk about immediate next steps and future. But first, a recap.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/vac.png" /><media:content medium="image" url="https://vac.dev/assets/img/vac.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">What’s the Plan for Waku v2?</title><link href="https://vac.dev/waku-v2-plan" rel="alternate" type="text/html" title="What’s the Plan for Waku v2?" /><published>2020-07-01T04:00:00+00:00</published><updated>2020-07-01T04:00:00+00:00</updated><id>https://vac.dev/waku-v2-pitch</id><content type="html" xml:base="https://vac.dev/waku-v2-plan">&lt;p&gt;&lt;strong&gt;tldr: The Waku network is fragile and doesn’t scale. Here’s how to solve it.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.&lt;/em&gt;&lt;/p&gt;

&lt;h1 id=&quot;problem&quot;&gt;Problem&lt;/h1&gt;

&lt;p&gt;The Waku network is fragile and doesn’t scale.&lt;/p&gt;

&lt;p&gt;As &lt;a href=&quot;https://status.im&quot;&gt;Status&lt;/a&gt; is moving into a user-acquisition phase and is improving retention rates for users they need the infrastructure to keep up, specifically when it comes to messaging.&lt;/p&gt;

&lt;p&gt;Based on user acquisition models, the initial goal is to support 100k DAU in September, with demand growing from there.&lt;/p&gt;

&lt;p&gt;With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;**1. Connection limits&lt;/strong&gt;**. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;**2. Bandwidth as a bottleneck&lt;/strong&gt;**. We notice that memory usage appears to not be
the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;**3. Amplification factors&lt;/strong&gt;**. Reducing amplification factors with better routing, would have a high impact, but it is likely we’d need additional measures as well, such as topic sharding or similar. See Fig 8-13.&lt;/p&gt;

&lt;p&gt;Figure 1-5:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/status_scaling_model_fig1.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig2.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig3.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig4.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig5.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;See &lt;a href=&quot;https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0&quot;&gt;https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0&lt;/a&gt; for the full report.&lt;/p&gt;

&lt;p&gt;What we need to do is:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Reduce amplification factors&lt;/li&gt;
  &lt;li&gt;Get more user-run full nodes&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;keep increasing requirements for Internet speed for full nodes&lt;/li&gt;
  &lt;li&gt;are vulnerable to censorship and attacks&lt;/li&gt;
  &lt;li&gt;have to control the topology in an artifical manner to keep up with load&lt;/li&gt;
  &lt;li&gt;basically re-invent a traditional centralized client-server app with extra steps&lt;/li&gt;
  &lt;li&gt;deliberately ignore most of our principles&lt;/li&gt;
  &lt;li&gt;risk the network being shut down when we run out of cash&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;appetite&quot;&gt;Appetite&lt;/h1&gt;

&lt;p&gt;Our initial risk appetite for this is 6 weeks for a small team.&lt;/p&gt;

&lt;p&gt;The idea is that we want to make tangible progress towards the goal in a limited period of time, as opposed to getting bogged down in trying to find a theoretically perfect generalized solution. Fixed time, variable scope.&lt;/p&gt;

&lt;p&gt;It is likely some elements of a complete solution will be done separately. See later sections for that.&lt;/p&gt;

&lt;h1 id=&quot;solution&quot;&gt;Solution&lt;/h1&gt;

&lt;p&gt;There are two main parts of the solution. One is to reduce amplification factors, and the other is incentivization to get more user run full nodes with desktop, etc.&lt;/p&gt;

&lt;p&gt;What does a full node provide? It provides connectivity to the network, can act as a bandwidth “barrier” and be high or reasonably high availability. What this means right now is essentially topic interest and storing historical messages.&lt;/p&gt;

&lt;p&gt;The goal is here to improve the status quo, not get a perfect solution from the get go. All of this can be iterated on further, for stronger guarantees, as well as replaced by other new modules.&lt;/p&gt;

&lt;p&gt;Let’s first look at the baseline, and then go into some of the tracks and their phases. Track 1 is best done first, after which track 2 and 3 can be executed in parallel. Track 1 gives us more options for track 2 and 3. The work in track 1 is currently more well-defined, so it is likely the specifics of track 2 and 3 will get refined at a later stage.&lt;/p&gt;

&lt;h2 id=&quot;baseline&quot;&gt;Baseline&lt;/h2&gt;

&lt;p&gt;Here’s where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can’t easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/waku_v1_routing_small.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;track-1---move-to-libp2p&quot;&gt;Track 1 - Move to libp2p&lt;/h2&gt;

&lt;p&gt;Moving to PubSub over libp2p wouldn’t improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.&lt;/p&gt;

&lt;p&gt;See more on libp2p PubSub here: &lt;a href=&quot;https://docs.libp2p.io/concepts/publish-subscribe/&quot;&gt;https://docs.libp2p.io/concepts/publish-subscribe/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;As part of this move, there are a few individual pieces that are needed.&lt;/p&gt;

&lt;h3 id=&quot;1-floodsub&quot;&gt;1. FloodSub&lt;/h3&gt;

&lt;p&gt;This is essentially what Waku over libp2p would look like in its most basic form.&lt;/p&gt;

&lt;p&gt;One difference that is worth noting is that the app topics would &lt;strong&gt;not&lt;/strong&gt; be the same as Waku topics. Why? In Waku we currently don’t use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.&lt;/p&gt;

&lt;p&gt;Why can’t we use Waku topics for routing directly? PubSub over libp2p isn’t built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/waku_v2_routing_flood_small.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.&lt;/p&gt;

&lt;p&gt;Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: &lt;a href=&quot;https://github.com/status-im/nim-waku&quot;&gt;https://github.com/status-im/nim-waku&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;2-simplify-the-protocol&quot;&gt;2. Simplify the protocol&lt;/h3&gt;

&lt;p&gt;Due to Waku’s origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (&lt;a href=&quot;https://rfc.vac.dev/spec/6/&quot;&gt;https://rfc.vac.dev/spec/6/&lt;/a&gt;). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?&lt;/p&gt;

&lt;p&gt;Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.&lt;/p&gt;

&lt;p&gt;The essential characteristic that has to be maintained is that we don’t need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.&lt;/p&gt;

&lt;h3 id=&quot;3-core-integration&quot;&gt;3. Core integration&lt;/h3&gt;

&lt;p&gt;As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.&lt;/p&gt;

&lt;h3 id=&quot;4-topic-interest-behavior&quot;&gt;4. Topic interest behavior&lt;/h3&gt;

&lt;p&gt;While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (&lt;a href=&quot;https://vac.dev/fixing-whisper-with-waku&quot;&gt;https://vac.dev/fixing-whisper-with-waku&lt;/a&gt;). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.&lt;/p&gt;

&lt;h3 id=&quot;5-historical-message-caching&quot;&gt;5. Historical message caching&lt;/h3&gt;

&lt;p&gt;Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there’s no reason to reimplement a poor API if we don’t have to.&lt;/p&gt;

&lt;p&gt;Also see section below on adaptive nodes and capabilities.&lt;/p&gt;

&lt;h3 id=&quot;6-waku-v1--libp2p-bridge&quot;&gt;6. Waku v1 &amp;lt;&amp;gt; Libp2p bridge&lt;/h3&gt;

&lt;p&gt;To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.&lt;/p&gt;

&lt;h2 id=&quot;track-2---better-routing&quot;&gt;Track 2 - Better routing&lt;/h2&gt;

&lt;p&gt;This is where we improve the amplification factors.&lt;/p&gt;

&lt;h3 id=&quot;1-gossipsub&quot;&gt;1. GossipSub&lt;/h3&gt;

&lt;p&gt;This is a subprotocol of FloodSub in the libp2p world. Moving to GossipSub would allow traffic between full nodes to go from an amplification factor of ~25 to ~6. This basically creates a mesh of stable bidirectional connections, together with some gossiping capabilities outside of this view.&lt;/p&gt;

&lt;p&gt;Explaining how GossipSub works is out of scope of this document. It is implemented in nim-libp2p and used by Nimbus as part of Eth2. You can read the specs here in more detail if you are interested: &lt;a href=&quot;https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md&quot;&gt;https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md&lt;/a&gt; and &lt;a href=&quot;https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md&quot;&gt;https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/waku_v2_routing_gossip_small.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/status_scaling_model_fig8.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig9.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig10.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig11.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;While we technically could implement this over existing Waku, we’d have to re-implement it, and we’d lose out on all the other benefits libp2p would provide, as well as the ecosystem of people and projects working on improving the scalability and security of these protocols.&lt;/p&gt;

&lt;h3 id=&quot;2-topic-sharding&quot;&gt;2. Topic sharding&lt;/h3&gt;

&lt;p&gt;This one is slightly more speculative in terms of its ultimate impact. The basic idea is to split the application topic into N shards, say 10, and then each full node can choose which shards to listen to. This can reduce amplification factors by another factor of 10.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/waku_v2_routing_sharding_small.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;assets/img/status_scaling_model_fig12.png&quot; alt=&quot;&quot; /&gt;
&lt;img src=&quot;assets/img/status_scaling_model_fig13.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Note that this means a light node that listens to several topics would have to be connected to more full nodes to get connectivity. For a more exotic version of this, see &lt;a href=&quot;https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47&quot;&gt;https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is orthogonal from the choice of FloodSub or GossipSub, but due to GossipSub’s more dynamic nature it is likely best combined with it.&lt;/p&gt;

&lt;h3 id=&quot;3-other-factors&quot;&gt;3. Other factors&lt;/h3&gt;

&lt;p&gt;Not a primary focus, but worth a look. Looking at the scaling model, there might be other easy wins to improve overall bandwidth consumption between full nodes. For example, can we reduce envelope size by a significant factor?&lt;/p&gt;

&lt;h2 id=&quot;track-3---accounting-and-user-run-nodes&quot;&gt;Track 3 - Accounting and user-run nodes&lt;/h2&gt;

&lt;p&gt;This is where we make sure the network isn’t fragile, become a true p2p app, get our users excited and engaged, and allow us to scale the network without creating an even bigger cluster.&lt;/p&gt;

&lt;p&gt;To work in practice, this has a soft dependency on node discovery such as DNS based discovery (&lt;a href=&quot;https://eips.ethereum.org/EIPS/eip-1459&quot;&gt;https://eips.ethereum.org/EIPS/eip-1459&lt;/a&gt;) or Discovery v5 (&lt;a href=&quot;https://vac.dev/feasibility-discv5&quot;&gt;https://vac.dev/feasibility-discv5&lt;/a&gt;).&lt;/p&gt;

&lt;h3 id=&quot;1-adaptive-nodes-and-capabilities&quot;&gt;1. Adaptive nodes and capabilities&lt;/h3&gt;

&lt;p&gt;We want to make the gradation between light nodes, full nodes, storing (partial set of) historical messages, only acting for a specific shard, etc more flexible and explicit. This is required to identify and discover the nodes you want. See &lt;a href=&quot;https://github.com/vacp2p/specs/issues/87&quot;&gt;https://github.com/vacp2p/specs/issues/87&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Depending on how the other tracks come together, this design should allow for a desktop node to identify as a full relaying node for some some app topic shard, but also express waku topic interest and retrieve historical messages itself.&lt;/p&gt;

&lt;p&gt;E.g. Disc v5 can be used to supply node properties through ENR.&lt;/p&gt;

&lt;h3 id=&quot;2-accounting&quot;&gt;2. Accounting&lt;/h3&gt;

&lt;p&gt;This is based on a few principles:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Some nodes contribute a lot more than other nodes in the network&lt;/li&gt;
  &lt;li&gt;We can account for the difference in contribution in some fashion&lt;/li&gt;
  &lt;li&gt;We want to incentivize nodes to tell the true, and be incentivized not to lie&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Accounting here is a stepping stone, where accounting is the raw data upon which some settlement later occurs. It can have various forms of granularity. See &lt;a href=&quot;https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31&quot;&gt;https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31&lt;/a&gt; for discussion.&lt;/p&gt;

&lt;p&gt;We also note that in GossipSub, the mesh is bidrectional. Additionally, it doesn’t appears to be a high priority issue in terms of nodes misreporting. What is an issue is having people run full nodes in the first place. There are a few points to that. It has to be possible in the end-user UX, nodes have to be discovered, and it has to be profitable/visible that you are contributing. UX and discovery are out of scope for this work, whereas visibility/accounting is part of this scope. Settlement is a stretch goal here.&lt;/p&gt;

&lt;p&gt;The general shape of the solution is inspired by the Swarm model, where we do accounting separate from settlement. It doesn’t require any specific proofs, but nodes are incentivized to tell the truth in the following way:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Both full node and light node do accounting in a pairwise, local fashion&lt;/li&gt;
  &lt;li&gt;If a light node doesn’t ultimately pay or lie about reporting, they get disconnected (e.g.)&lt;/li&gt;
  &lt;li&gt;If a full node doesn’t provide its service the light node may pick another full node (e.g.)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;end to end delivery&lt;/li&gt;
  &lt;li&gt;online time&lt;/li&gt;
  &lt;li&gt;completeness of storage&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See &lt;a href=&quot;https://discuss.status.im/t/network-incentivisation-first-draft/1037&quot;&gt;https://discuss.status.im/t/network-incentivisation-first-draft/1037&lt;/a&gt; for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And &lt;a href=&quot;https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51&quot;&gt;https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51&lt;/a&gt; for using validity proofs and removing liveness requirement for settlement.&lt;/p&gt;

&lt;p&gt;All of this is optional at this stage, because our goal here is to improve the status quo for user run nodes. Accounting at this stage should be visible and correspond to the net benefit a node provides to another.&lt;/p&gt;

&lt;p&gt;As a concrete example: a light node has some topic interest and cares about historical messages on some topic. A full node communicates envelopes as they come in, communicates their high availability (online time) and stores/forward stored messages. Both nodes have this information, and if they agree settlement (initially just a mock message) can be sending a payment to an address at some time interval / over some defined volume. See future sections for how this can be improved upon.&lt;/p&gt;

&lt;p&gt;Also see below in section 4, using constructs such as eigentrust as a local reputation mechanism.&lt;/p&gt;

&lt;h3 id=&quot;3-relax-high-availability-requirement&quot;&gt;3. Relax high availability requirement&lt;/h3&gt;

&lt;p&gt;If we want desktop nodes to participate in the storing of historical messages, high availability is a problem. It is a problem for any node, especially if they lie about it, but assuming they are honest it is still an issue.&lt;/p&gt;

&lt;p&gt;By being connected to multiple nodes, we can get an overlapping online window. Then these can be combined together to get consistency. This is obviously experimental and would need to be tested before being deployed, but if it works it’d be very useful.&lt;/p&gt;

&lt;p&gt;Additionally or alternatively, instead of putting a high requirement on message availability, focus on detection of missing information. This likely requires re-thinking how we do data sync / replication.&lt;/p&gt;

&lt;h3 id=&quot;4-incentivize-light-and-full-nodes-to-tell-the-truth-policy-etc&quot;&gt;4. Incentivize light and full nodes to tell the truth (policy, etc)&lt;/h3&gt;

&lt;p&gt;In accounting phase it is largely assumed nodes are honest. What happens when they lie, and how do we incentivize them to be honest? In the case of Bittorrent this is done with tit-for-tat, however this is a different kind of relationship. What follows are some examples of how this can be done.&lt;/p&gt;

&lt;p&gt;For light nodes:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;if they don’t, they get disconnected&lt;/li&gt;
  &lt;li&gt;prepayment (especially to “high value” nodes)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For full nodes:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;multiple nodes reporting to agree, where truth becomes a shelling point&lt;/li&gt;
  &lt;li&gt;use eigentrust&lt;/li&gt;
  &lt;li&gt;staking for discovery visibility with slashing&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;5-settlement-poc&quot;&gt;5. Settlement PoC&lt;/h3&gt;

&lt;p&gt;Can be done after phase 2 if so desired. Basically integrate payments based on accounting and policy.&lt;/p&gt;

&lt;h1 id=&quot;out-of-scope&quot;&gt;Out of scope&lt;/h1&gt;

&lt;ol&gt;
  &lt;li&gt;We assume the Status Base model requirements are accurate.&lt;/li&gt;
  &lt;li&gt;We assume Core will improve retention rates.&lt;/li&gt;
  &lt;li&gt;We assume the Stimbus production team will enable integration of nim-waku.&lt;/li&gt;
  &lt;li&gt;We assume Discovery mechanisms such as DNS and Discovery v5 will be worked on separately.&lt;/li&gt;
  &lt;li&gt;We assume Core will, at some point, provide an UX for integrating payment of services.&lt;/li&gt;
  &lt;li&gt;We assume the desktop client is sufficiently usable.&lt;/li&gt;
  &lt;li&gt;We assume Core and Infra will investigate ways of improving MaxPeers.&lt;/li&gt;
&lt;/ol&gt;</content><author><name>oskarth</name></author><category term="research" /><summary type="html">tldr: The Waku network is fragile and doesn’t scale. Here’s how to solve it.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/status_scaling_model_fig4.png" /><media:content medium="image" url="https://vac.dev/assets/img/status_scaling_model_fig4.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Feasibility Study: Discv5</title><link href="https://vac.dev/feasibility-discv5" rel="alternate" type="text/html" title="Feasibility Study: Discv5" /><published>2020-04-27T04:00:00+00:00</published><updated>2020-04-27T04:00:00+00:00</updated><id>https://vac.dev/feasibility-discv5</id><content type="html" xml:base="https://vac.dev/feasibility-discv5">&lt;blockquote&gt;
  &lt;p&gt;Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This post serves as a more authoritative overview of the discv5 study, for a discussionary post providing more context make sure to check out the corresponding &lt;a href=&quot;https://discuss.status.im/t/discv5-feasibility-study/1632&quot;&gt;discuss post&lt;/a&gt;. Additionally, if you are unfamiliar with discv5, check out my previous write-up: &lt;a href=&quot;https://vac.dev/kademlia-to-discv5&quot;&gt;“From Kademlia to Discv5”&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;motivating-problem&quot;&gt;Motivating Problem&lt;/h2&gt;

&lt;p&gt;The discovery method currently used by &lt;a href=&quot;https://status.im&quot;&gt;Status&lt;/a&gt;, is made up of various components and grew over time to solve a mix of problems. We want to simplify this while maintaining some of the properties we currently have.&lt;/p&gt;

&lt;p&gt;Namely, we want to ensure censorship resistance to state-level adversaries. One of the issues Status had which caused us them add to their discovery method was the fact that addresses from providers like AWS and GCP were blocked both in Russia and China. Additionally, one of the main factors required is the ability to function on resource restricted devices.&lt;/p&gt;

&lt;p&gt;Considering we are talking about resource restricted devices, let’s look at the implications and what we need to consider:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;Battery consumption&lt;/strong&gt; - constant connections like websockets consume a lot of battery life.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;CPU usage&lt;/strong&gt; - certain discovery methods may be CPU incentive, slowing an app down and making it unusable.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Bandwidth consumption&lt;/strong&gt; - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Short connection windows&lt;/strong&gt; - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Not publicly connectable&lt;/strong&gt; - There is a good chance that most resource restricted devices are not publicly connectable.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes’ stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node.&lt;/p&gt;

&lt;p&gt;With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation.&lt;/p&gt;

&lt;p&gt;In order to answer our questions, we formulated the problem with a simple method for testing. The “needle in a haystack” problem was formulated to figure out how easily a specific node can be found within a given network. This issue was fully formulated in &lt;a href=&quot;https://github.com/vacp2p/research/issues/15&quot;&gt;vacp2p/research#15&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;overview&quot;&gt;Overview&lt;/h2&gt;

&lt;p&gt;The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.&lt;/li&gt;
  &lt;li&gt;We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through &lt;a href=&quot;https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement&quot;&gt;topic tables&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;feasbility&quot;&gt;Feasbility&lt;/h2&gt;

&lt;p&gt;To be able to investigate the feasibility of discv5, we used various methods including rough calculations which can be found in the &lt;a href=&quot;https://vac.dev/discv5-notebook/&quot;&gt;notebook&lt;/a&gt;, and a simulation isolated in &lt;a href=&quot;https://github.com/vacp2p/research/pull/19&quot;&gt;vacp2p/research#19&lt;/a&gt;.&lt;/p&gt;

&lt;h3 id=&quot;cpu--memory-usage&quot;&gt;CPU &amp;amp; Memory Usage&lt;/h3&gt;

&lt;p&gt;The experimental discv5 has already been used within Status, however what was noticed was that the CPU and memory usage was rather high. It therefore should be investiaged if this is still the case, and if it is, it should be isolated where this stems from. Additionally it is worth looking at whether or not this is the case with both the go and nim implementation.&lt;/p&gt;

&lt;p&gt;See details: &lt;a href=&quot;https://github.com/vacp2p/research/issues/31&quot;&gt;vacp2p/research#31&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;nat-on-cellular-data&quot;&gt;NAT on Cellular Data&lt;/h3&gt;

&lt;p&gt;If a peer is not publically connectable it can not participate in the DHT both ways. A lot of mobile phones are behind symmetric NATs which UDP hole-punching close to impossible. It should be investigated whether or not mobile phones will be able to participate both ways and if there are good methods for doing hole-punching.&lt;/p&gt;

&lt;p&gt;See details: &lt;a href=&quot;https://github.com/vacp2p/research/issues/29&quot;&gt;vacp2p/research#29&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;topic-tables&quot;&gt;Topic Tables&lt;/h3&gt;

&lt;p&gt;Topic Tables allow us the ability to efficiently find nodes given a specific topic. However, they are not implemented in the &lt;a href=&quot;https://github.com/status-im/nim-eth/&quot;&gt;status-im/nim-eth&lt;/a&gt; implementation nor are they fully finalized in the spec. These are important if the network grows past a size where the concentration of specific nodes is relatively low making them hard to find.&lt;/p&gt;

&lt;p&gt;See details: &lt;a href=&quot;https://github.com/vacp2p/research/issues/26&quot;&gt;vacp2p/research#26&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;finding-a-node&quot;&gt;Finding a node&lt;/h3&gt;

&lt;p&gt;It is important to note, that given a network is relatively small sized, eg 100-500 nodes, then finding a node given a specific address is relatively managable. Additionally, if the concentration of a specific capability in a network is reasonable, then finding a node advertising its capabilities using an ENR rather than the topic table is also managable. A reasonable concentration for example would be 10%, which would give us an 80% chance of getting a node with that capability in the first lookup request. This can be explored more using our &lt;a href=&quot;https://vac.dev/discv5-notebook/#Needle-in-a-haystack-with-ENR-records-indicating-capabilities&quot;&gt;discv5 notebook&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;results&quot;&gt;Results&lt;/h2&gt;

&lt;p&gt;Research has shown that finding a node in the DHT has a relatively low effect on bandwidth, both inbound and outbound. For example when trying to find a node in a network of 100 nodes, it would take roughly 5668 bytes total. Additionally if we assume 100ms latency per request it would range at ≈ 300ms latency, translating to 3 requests to find a specific node.&lt;/p&gt;

&lt;h2 id=&quot;general-thoughts&quot;&gt;General Thoughts&lt;/h2&gt;

&lt;p&gt;One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn’t too high, discv5 could probably be used as the other issues are more “features” than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.&lt;/p&gt;

&lt;h2 id=&quot;acknowledgements&quot;&gt;Acknowledgements&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;Oskar Thoren&lt;/li&gt;
  &lt;li&gt;Dmitry Shmatko&lt;/li&gt;
  &lt;li&gt;Kim De Mey&lt;/li&gt;
  &lt;li&gt;Corey Petty&lt;/li&gt;
&lt;/ul&gt;</content><author><name>dean</name></author><category term="research" /><summary type="html">Disclaimer: some of the numbers found in this write-up could be inaccurate. They are based on the current understanding of theoretical parts of the protocol itself by the author and are meant to provide a rough overview rather than bindable numbers.</summary></entry><entry><title type="html">What Would a WeChat Replacement Need?</title><link href="https://vac.dev/wechat-replacement-need" rel="alternate" type="text/html" title="What Would a WeChat Replacement Need?" /><published>2020-04-16T04:00:00+00:00</published><updated>2020-04-16T04:00:00+00:00</updated><id>https://vac.dev/wechat-replacement-need</id><content type="html" xml:base="https://vac.dev/wechat-replacement-need">&lt;p&gt;What would it take to replace WeChat? More specifically, what would a self-sovereign, private, censorship-resistant and open alternative look like? One that allows people to communicate, coordinate and transact freely.&lt;/p&gt;

&lt;h2 id=&quot;background&quot;&gt;Background&lt;/h2&gt;

&lt;h3 id=&quot;what-wechat-provides-to-the-end-user&quot;&gt;What WeChat provides to the end-user&lt;/h3&gt;

&lt;p&gt;Let’s first look at some of the things that WeChat providers. It is a lot:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;Messaging:&lt;/strong&gt; 1:1 and group chat. Text, as well as voice and video. Post gifs. Share location.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Group chat:&lt;/strong&gt; Limited to 500 people; above 100 people people need to verify with a bank account. Also has group video chat and QR code to join a group.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Timeline/Moments:&lt;/strong&gt; Post comments with attachments and have people like/comment on it.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Location Discovery:&lt;/strong&gt; See WeChat users that are nearby.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Profile:&lt;/strong&gt; Nickname and profile picture; can alias people.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;“Broadcast” messages:&lt;/strong&gt; Send one message to many contacts, up to 200 people (spam limited).&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Contacts:&lt;/strong&gt; Max 5000 contacts (people get around it with multiple accounts and sim cards).&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;App reach:&lt;/strong&gt; Many diferent web apps, extensions, native apps, etc. Scan QR code to access web app from phone.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Selective posting:&lt;/strong&gt; Decide who can view your posts and who can view your comments on other people’s post.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Transact:&lt;/strong&gt; Send money gifts through red envelopes.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Transact:&lt;/strong&gt; Use WeChat pay to transfer money to friends and businesses; linked account with Alipay that is connected to your bank account.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Services:&lt;/strong&gt; Find taxis and get notifications; book flights, train tickets, hotels etc.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Mini apps:&lt;/strong&gt; API for all kinds of apps that allow you to provide services etc.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Picture in picture:&lt;/strong&gt; allowing you to have a video call while using the app.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And much more. Not going to through it all in detail, and there are probably many things I don’t know about WeChat since I’m not a heavy user living in mainland China.&lt;/p&gt;

&lt;h3 id=&quot;how-wechat-works---a-toy-model&quot;&gt;How WeChat works - a toy model&lt;/h3&gt;

&lt;p&gt;This is an overly simplistic model of how WeChat works, but it is sufficient for our purposes. This general design applies to most traditional client-server apps today.&lt;/p&gt;

&lt;p&gt;To sign up for account you need a phone number or equivalent. To get access to some features you need to verify your identity further, for example with official ID and/or bank account.&lt;/p&gt;

&lt;p&gt;When you signup this creates an entry in the WeChat server, from now on treated as a black box. You authenticate with that box, and thats where you get your messages from. If you go online the app asks that box for messages you have received while you were offline. If you login from a different app your contacts and conversations are synced from that box.&lt;/p&gt;

&lt;p&gt;The box gives you an account, it deals with routing to your contacts, it stores messages and attachments and gives access to mini apps that people have uploaded. For transacting money, there is a partnership with a different company that has a different box which talks to your bank account.&lt;/p&gt;

&lt;p&gt;This is done in a such a way that they can support a billion users with the features above, no sweat.&lt;/p&gt;

&lt;p&gt;Whoever controls that box can sees who you are talking with and what the content of those messages are. There is no end to end encryption. If WeChat/Tencent disagrees with you for some reason they can ban you. This means you can’t interact with the box under that name anymore.&lt;/p&gt;

&lt;h2 id=&quot;what-do-we-want&quot;&gt;What do we want?&lt;/h2&gt;

&lt;p&gt;We want something that is self-sovereign, private, censorship-resistant and open that allows individuals and groups of people to communicate and transact freely. To explore what this means in more detail, without getting lost in the weeds, we provide the following list of properties. A lot of these are tied together, and some fall out of the other requirements. Some of them stand in slight opposition to each other.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Self-sovereignity identity&lt;/strong&gt;. Exercises authority within your own sphere. If you aren’t harming anyone, you should be able to have an account and communicate with other people.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pseudonymity, and ideally total anonymity&lt;/strong&gt;. Not having your identity tied to your real name (e.g. through phone number, bank account, ID, etc). This allows people to act more freely without being overly worried about censorship and coercion in the real world. While total anonymity is even more desirable - especially to break multiple hops to a true-name action - real-world constraints sometimes makes this more challenging.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Private and secure communication&lt;/strong&gt;. Your communication and who you transact with should be for your eyes only. This includes transactions (transfer of value) as a form of communication.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Censorship-resistance&lt;/strong&gt;. Not being able to easily censor individuals on the platform. Both at an individual, group and collective level. Not having single points of failure that allow service to be disrupted.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Decentralization&lt;/strong&gt;. Partly falls out of censorship-resistance and other properties. If infrastructure isn’t decentralized it means there’s a single point of failure that can be disrupted. This is more of a tool than a goal on its own, but it is an important tool.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Built for mass adoption&lt;/strong&gt;. Includes scalabiltiy, UX (latency, reliability, bandwidth consumption, UI etc), and allowing for people to stick around. One way of doing this is to allow users to discover people they want to talk to.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Scalability&lt;/strong&gt;. Infrastructure needs to support a lot of users to be a viabile alternative. Like, a billion of them (eventually).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fundamentals in place to support great user experience&lt;/strong&gt;. To be a viable alternative, aside from good UI and distribution, fundamentals such as latency, bandwidth usage, consistency etc must support great UX to be a viable alternative.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Works for resource restricted devices, including smartphones&lt;/strong&gt;. Most people will use a smartphone to use this. This means it has to work well on them and similar devices, without becoming a second-class citizen where we ignore properties such as censorship-resistance and privacy. Some concession to reality will be necessary due to additional constraints, which leads us to…&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Adaptive nodes&lt;/strong&gt;. Nodes will have different capabilities, and perhaps at different times. To maintain a lot of the properties described here it is desirable if as many participants as possible are first-class citizens. If a phone is switching from a limited data plan to a WiFi network or from battery to AC power it can do more useful work, and so on. Likewise for a laptop with a lot of free disk space and spare compute power, etc.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Sustainable&lt;/strong&gt;. If there’s no centralized, top down ad-driven model, this means all the infrastructure has to be sustainable somehow. Since these are individual entitites, this means it has to be paid for. While altruistic modes and similar can be used, this likely requires some form of incentivization scheme for useful services provided in the network. Related: free rider problem.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Spam resistant&lt;/strong&gt;. Relates to sustainability, scalability and built for mass adoption. Made more difficult by pseudonymous identity due to whitewashing attacks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Trust-minimized&lt;/strong&gt;. To know that properties are provided for and aren’t compromised, various ways of minimizing trust requirements are useful. This also related to mass adoption and social cohesion. Examples include: open and audited protocols, open source, reproducible builds, etc. This also relates to how mini apps are provided for, since we may not know their source but want to be able to use them anyway.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Open source&lt;/strong&gt;. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn’t been compromised, e.g. by uploading private data to a third party.&lt;/p&gt;

&lt;p&gt;Some of these are graded and a bit subtle, i.e.:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible.&lt;/li&gt;
  &lt;li&gt;Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience.&lt;/li&gt;
  &lt;li&gt;Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals.&lt;/li&gt;
  &lt;li&gt;“Discover people” and striving for “total anonymity” might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy.&lt;/li&gt;
  &lt;li&gt;Users often want &lt;em&gt;some&lt;/em&gt; form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;counterpoint-1&quot;&gt;Counterpoint 1&lt;/h3&gt;

&lt;p&gt;We could refute the above by saying that the design goals are undesirable. We want a system where people can censor others, and where everyone is tied to their real identity. Or we could say something like, freedom of speech is a general concept, and it doesn’t apply to Internet companies, even if they provide a vital service. You can survive without it and you should’ve read the terms of service. This roughly charactericizes the mainstream view.&lt;/p&gt;

&lt;p&gt;Additional factor here is the idea that a group of people know more about what’s good for you then you do, so they are protecting you.&lt;/p&gt;

&lt;h3 id=&quot;counterpoint-2&quot;&gt;Counterpoint 2&lt;/h3&gt;

&lt;p&gt;We could agree with all these design goals, but think they are too extreme in terms of their requirements. For example, we could operate as a non profit, take donations and volunteers, and then host the whole infrastructure ourselves. We could say we are in a friendly legislation, so we won’t be a single point of failure. Since we are working on this and maybe even our designs are open, you can trust us and we’ll provide service and infrastructure that gives you what you want without having to pay for it or solve all these complex decentralized computation and so on problems. If you don’t trust us for some reason, you shouldn’t use us regardless. Also, this is better than status quo. And we are more likely to survive by doing this, either by taking shortcuts or by being less ambituous in terms of scope.&lt;/p&gt;

&lt;h2 id=&quot;principal-components&quot;&gt;Principal components&lt;/h2&gt;

&lt;p&gt;There are many ways to skin a cat, but this is one way of breaking down the problem. We have a general direction with the properties listed above, together with some understanding of how WeChat works for the everday user. Now the question is, what infrastructure do we need to support this? How do we achieve the above properties, or at least get closer to them? We want to figure out the necessary building blocks, and one of doing this is to map out likely necessary components.&lt;/p&gt;

&lt;h3 id=&quot;background-ethereum-and-web3-stack&quot;&gt;Background: Ethereum and Web3 stack&lt;/h3&gt;

&lt;p&gt;It is worth noting that a lot of the required infrastructure has been developed, at least as concepts, in the original Ethereum / Web3 vision. In it there is Ethereum for consensus/compute/transact, storage through Swarm, and communication through Whisper. That said, the main focus has been on the Ethereum blockchain itself, and a lot of things have happened in the last 5y+ with respect to technology around privacy and scalabilty. It is worth revisiting things from a fresh point of view, with the WeChat alternative in mind as a clear use case.&lt;/p&gt;

&lt;h3 id=&quot;account---self-sovereign-identity-and-the-perils-of-phone-numbers&quot;&gt;Account - self-sovereign identity and the perils of phone numbers&lt;/h3&gt;

&lt;p&gt;Starting from the most basic: what is an account and how do you get one? With most internet services today, WeChat and almost all popular messaging apps included, you need to signup with some centralized authority. Usually you also have to verify this with some data that ties this account to you as an individual. E.g. by requiring a phone number, which in most jurisdictions &lt;sup id=&quot;fnref:1&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:1&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;1&lt;/a&gt;&lt;/sup&gt; means giving out your real ID. This also means you can be banned from using the service by a somewhat arbitrary process, with no due process.&lt;/p&gt;

&lt;p&gt;Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this.&lt;/p&gt;

&lt;p&gt;Justifications for requiring phone numbers are usually centered around three claims:
1) Avoiding spam
2) Tying your account to your real name, for various reasons
3) Using as a commonly shared identifier as a social network discovery mechanism&lt;/p&gt;

&lt;p&gt;Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on.&lt;/p&gt;

&lt;p&gt;In contrast, a self-sovereign system would allow you to “create an account” completely on your own. This can easily be done with public key cryptograpy, and it also paves the way for end-to-end encryption to make your messages private.&lt;/p&gt;

&lt;p&gt;The main issue with this that you need to get more creative about avoiding spam (e.g. through white washing attacks), and ideally there is some other form of social discovery mechanism.&lt;/p&gt;

&lt;p&gt;Just having a public key as an account isn’t enough though. If it goes through a central server, then nothing is stopping that server from arbitrarly blocking requests related to that public key. Of course, this also depends on how transparent such requests are. Fundamentally, lest we rely completely on goodwill, there needs to be multiple actors by which you can use the service. This naturally points to decentralization as a requirement. See counterpoint.&lt;/p&gt;

&lt;p&gt;Even so, if the system is closed source we don’t know what it is doing. Perhaps the app communicating is also uploading data to another place, or somehow making it possible to see who is who and act accordingly.&lt;/p&gt;

&lt;p&gt;You might notice that just one simple property, self-sovereign identity, leads to a slew of other requirements and properties. You might also notice that WeChat is far from alone in this, even if their identity requirements might be a bit stringent than, say, Telegram. Their control aspects are also a bit more extreme, at least for someone with western sensibilities &lt;sup id=&quot;fnref:2&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:2&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;

&lt;p&gt;Most user facing applications have similar issues, Google Apps/FB/Twitter etc. For popular tools that have this built in, we can look at git - which is truly decentralized and have keypair at the bottom. It is for a very specific technical domain, and even then people rely on Github. Key management is fairly difficult even for technical people, and for normal people even more so. Banks are generally far behind on this tech, relying on arcane procedures and special purpose hardware for 2FA. That’s another big issue.&lt;/p&gt;

&lt;p&gt;Let’s shift gears a bit and talk about some other functional requirements.&lt;/p&gt;

&lt;h3 id=&quot;routing---packets-from-a-to-b&quot;&gt;Routing - packets from A to B&lt;/h3&gt;

&lt;p&gt;In order to get a lot of the features WeChat provides, we need the ability to do three things: communicate, store data, and transact with people. We need a bit more than that, but let’s focus on this for now.&lt;/p&gt;

&lt;p&gt;To communicate with people, in the base case, we need to go from one phone to another phone that is separated by a large distance. This requires some form of routing. The most natural platform to build this on is the existing Internet, though not the only one. Most phones are resource restricted, and are only “on” for brief periods of time. This is needed to preserve battery and bandwidth. Additionally, Internet uses IPs as endpoints, which change as a phones move through space. NAT punching etc isn’t always perfect either. This means we need a way to get a message from one public key to another, and through some intermediate nodes. We can think of these nodes as a form of service network. Similar to how a power grid works, or phone lines, or collection of ISPs.&lt;/p&gt;

&lt;p&gt;One important property here is to ensure we don’t end up in a situation like the centralized capture scenario above, something we’ve seen with centralized ISPs &lt;sup id=&quot;fnref:3&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:3&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;3&lt;/a&gt;&lt;/sup&gt; &lt;sup id=&quot;fnref:4&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:4&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;4&lt;/a&gt;&lt;/sup&gt; where they can choose which traffic is good and which is bad. We want to allow the use of different service nodes, just like if a restaurant gives you food poisioning you can go to the one next door and then the first one goes out of business after a while. And the circle of life continues.&lt;/p&gt;

&lt;p&gt;We shouldn’t be naive though, and think that this is something nodes are likely to do for free. They need to be adequately compensated for their services, in some of incentivization scheme. That can either be monetary, or as in the case of Bittorrent, more of a barter situation where you use game theory to coordinate with strangers &lt;sup id=&quot;fnref:5&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:5&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;5&lt;/a&gt;&lt;/sup&gt;, and some form of reputation attached to it (for private trackers).&lt;/p&gt;

&lt;p&gt;There are many ways of doing routing, and we won’t go into too much technical detail here. Suffice to say is that you likely want both a structured and unstructured alternative, and that these comes with several trade-offs when it comes to efficiency, metadata protection, ability to incentivize, compatibility with existing topologies, and suitability for mobilephones (mostly offline, bandwidth restricted, not directly connectable). Expect more on this in a future article.&lt;/p&gt;

&lt;p&gt;Some of these considerations naturally leads us into the storage and transaction components.&lt;/p&gt;

&lt;h3 id=&quot;storage---available-and-persistant-for-later&quot;&gt;Storage - available and persistant for later&lt;/h3&gt;

&lt;p&gt;If mobile phones are mostly offline, we need some way to store these messages so they can be retrieved when online again. The same goes for various kinds attachments as well, and for when people are switching devices. A user might control their timeline, but in the WeChat case that timeline is stored on Tencent’s servers, and queried from there as well. This naturally needs to happen by some other service nodes. In the WeChat case, and for most IMs, the way these servers are paid for is through some indirect ad mechanism. The entity controlling these ads and so on is the same one as the one operating the servers for storage. A more direct model with different entities would see these services being compensated for their work.&lt;/p&gt;

&lt;p&gt;We also need storage for attachments, mini-apps, as well as a way of understanding the current state of consensus when it comes to the compute/transact module. In the WeChat case, this state is completely handled by the bank institution or one of their partners, such as Alibaba. When it comes to bearer instruments like cash, no state needs to be kept as that’s a direct exchange in the physical world. This isn’t directly compatible with transfering value over a distance.&lt;/p&gt;

&lt;p&gt;All of this state requires availability and persistance. It should be done in a trust minimized fashion and decentralized, which requires some form of incentivization for keeping data around. If it isn’t, you are relying on social cohesion which breaks down at very large scales.&lt;/p&gt;

&lt;p&gt;Since data will be spread out across multiple nodes, you need a way to sync data and transfer it in the network. As well as being able to add and query data from it. All of this requires a routing component.&lt;/p&gt;

&lt;p&gt;To make it more censorship resistant it might be better to keep it as a general-purpose store, i.e. individuals don’t need to know what they storing. Otherwise, you naturally end up in a situation where individual nodes can be pressured to not store certain content.&lt;/p&gt;

&lt;h3 id=&quot;messaging---from-me-to-you-to-all-of-us-not-them&quot;&gt;Messaging - from me to you to all of us (not them)&lt;/h3&gt;

&lt;p&gt;This builds on top of routing, but it has a slightly different focus. The goal is to allow for individuals and groups to communicate in a private, secure and censorship-resistant manner.&lt;/p&gt;

&lt;p&gt;It also needs to provide a decent interface to the end user, in terms of dealing seamlessly with offline messages, providing reliable and timely messaging.&lt;/p&gt;

&lt;p&gt;In order to get closer to the ideal of total anonymity, it is useful to be able to hide metadata of who is talking to whom. This applies to both normal communication as well as for transactions. Ideally, no one but the parties involved can see who is taking part in a conversation. This can be achieved through various techniques such as mixnets, anonymous credentials, private information retrieval, and so on. Many of these techniques have a fundamental trade-off with latency and bandwidth, something that is a big concern for mobilephones. Being able to do some form of tuning, in an adaptive node manner, depending on your threat model and current capabilities is useful here.&lt;/p&gt;

&lt;p&gt;The baseline here is pseudonymity, and having tools to allow individuals to “cut off” ties to their real world identity and transactions. People act different in different circles in the real world, and this should be mimicked online as well. Your company, family or government shouldn’t be able to know what exactly you use your paycheck for, and who you are talking to.&lt;/p&gt;

&lt;h3 id=&quot;compute---transact-contract-and-settle&quot;&gt;Compute - transact, contract and settle&lt;/h3&gt;

&lt;p&gt;The most immediate need here is transaction from A to B. Direct exchange. There is also a more indirect need for private lawmaking and contracting.&lt;/p&gt;

&lt;p&gt;We talked about routing and storage and how they likely need to be incentivized to work properly. How are they going to be compensated? While this could in theory work via existing banking system and so on, this would be rather heavy. It’d also very likely require tying your identifier to your legal name, something that goes against what we want to achieve. What we want is something that acts more as right-to-access, similar to the way cash functions in a society &lt;sup id=&quot;fnref:6&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:6&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;6&lt;/a&gt;&lt;/sup&gt;. I pay for a fruit with something that is valuable to you and then I’m on my way.&lt;/p&gt;

&lt;p&gt;While there might be other candidates, such as pre-paid debit cards and so on, this transaction mode pretty much requires a cryptocurrency component. The alternative is to do it on a reputation basis, which might work for small communities, due to social cohesion, but quickly detoriates for large ones &lt;sup id=&quot;fnref:7&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:7&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;7&lt;/a&gt;&lt;/sup&gt;. Ad hoc models like private Bittorrent trackers are centralized and easy to censor.&lt;/p&gt;

&lt;p&gt;Now, none of the existing cryptocurrency models are ideal. They also all suffer from lack of widespread use, and it is difficult to get onboarded to them in the first place. Transactions in Bitcoin are slow. Ethereum is faster and has more capabilities, but it still suffers from linking payments over time, which makes the privacy part of this more difficult. Zcash, Monero and similar are interesting, but also require more use. For Zcash, shielded transactions appear to only account for less than 2% of all transactions in 2019 &lt;sup id=&quot;fnref:8&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:8&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;8&lt;/a&gt;&lt;/sup&gt; &lt;sup id=&quot;fnref:9&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:9&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;9&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;

&lt;p&gt;Another dimension is what sets general purpose cryptocurrencies like Ethereum apart. Aside from just paying from A to B, you can encode rules about when something should be paid out and not. This is very useful for doing a form of private lawmaking, contracting, for setting up service agreements with these nodes. If there’s no trivial recourse as in the meatspace world, where you know someone’s name and you can sue them, you need a different kind of model.&lt;/p&gt;

&lt;p&gt;What makes something like Zcash interesting is that it works more like digital cash. Instead of leaving a public trail for everyone, where someone can see where you got the initial money from and then trace you across various usage, for Zcash every hop is privacy preserving.&lt;/p&gt;

&lt;p&gt;To fulfill the general goals of being censorship resistance and secure, it is also vital that the system being used stays online and can’t be easily disrupted. That points to disintermediation, as opposed to using gateways and exchanges. This is a case where something like cash, or gold, is more direct, since no one can censor this transaction without being physically present where this direct exchange is taking place. However, like before, this doesn’t work over distance.&lt;/p&gt;

&lt;h3 id=&quot;secure-chat---just-our-business&quot;&gt;Secure chat - just our business&lt;/h3&gt;

&lt;p&gt;Similar to the messaging module above. The distinction here is that we assume the network part has already taken place. Here we are interested in keeping the contents of messages private, so that means confidentiality/end-to-end encryption, integrity, authentication, as well as forward secrecy and plausible deniability. This means that even if there’s some actor that gets some private key material, or confiscated your phone, there is some level of…ephemerality to your conversations. Another issue here in terms of scalable private group chat.&lt;/p&gt;

&lt;h3 id=&quot;extensible-mini-apps&quot;&gt;Extensible mini apps&lt;/h3&gt;

&lt;p&gt;This relates to the compute and storage module above. Essentially we want to provide mini apps as in WeChat, but to do so in a way that is compatible with what we want to achieve more generally. This allows individuals and small businesses to create small tools for various purposes, and coordinate with strangers. E.g. booking a cab or getting an insurance, and so on.&lt;/p&gt;

&lt;p&gt;This has a higher dependency on the contracting/general computation aspect. I.e. often it isn’t only a transaction, but you might want to encode some specific rules here that strangers can abide by without having too high trust requirements. As a simple example: escrows.&lt;/p&gt;

&lt;p&gt;This also needs an open API that anyone can use. It should be properly secured, so using one doesn’t compromise the rest of the system it is operating in. To be censorship resistant it requires the routing and storage component to work properly.&lt;/p&gt;

&lt;h2 id=&quot;where-are-we-now&quot;&gt;Where are we now?&lt;/h2&gt;

&lt;p&gt;Let’s look back at some of desirable properties we set out in the beginning and see how close we are to building out the necessary components. Is it realistic at all or just a pipe dream? We’ll see that there are many building blocks in place, and there’s reason for hope.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Self-sovereignity identity&lt;/strong&gt;. Public key crypto and web of trust like constructs makes this possible.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pseudonymity, and ideally total anonymity&lt;/strong&gt;. Pseudonymity can largely be achieved with public key crypto and open systems that allow for permissionless participation. For transactions, pseudonymity exists in most cryptocurrencies. The challenge is linkage across time, especially when interfacing with other “legacy” system. There are stronger constructs that are actively being worked on and are promising here, such as mixnets (Nym), mixers (Wasabi Wallet, Tornado.Cash) and zero knowledge proofs (Zcash, Ethereum, Starkware). This area of applied research has exploded over the last few years.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Private and secure communication&lt;/strong&gt;. Signal has pioneered a lot of this, following OTR. Double Ratchet, X3DH. E2EE is minimum these days, and properties like PFS and PD are getting better. For metadata protection, you have Tor, with its faults, and more active research on mixnets and private information retrieval, etc.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Censorship-resistance&lt;/strong&gt;. This covers a lot of ground across the spectrum. You have technologies like Bittorrent, Bitcoin/Ethereum, Tor obfuscated transports, E2EE by default, partial mesh networks in production, abilit to move/replicate host machines more quickly have all made this more of a reality than it used to be. this easier. Of course, techniques such as deep packet inspection and internet shutdowns have increased.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Decentralization&lt;/strong&gt;. Cryptocurrencies, projects like libp2p and IPFS. Need to be mindful here of many projects that claim decentralization but are still vulnerable to single points of failures, such as relying on gateways.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Built for mass adoption&lt;/strong&gt;. This one is more subjective. There’s definitely a lot of work to be done here, both when it comes to fundamental performance, key management and things like social discoverability. Directionally these things are improving and becoming easier for the average person but there is a lot ot be done here.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Scalability&lt;/strong&gt;. With projects like Ethereum 2.0 and IPFS more and more resources are a being put into this, both at the consensus/compute layer as well as networking (gossip, scalable Kademlia) layer. Also various layer 2 solutions for transactions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fundamentals in place to support great user experience&lt;/strong&gt;. Similar to built for mass adoption. As scalability becomes more important, more applied research is being done in the p2p area to improve things like latency, bandwidth.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Works for resource restricted devices, including smartphones&lt;/strong&gt;. Work in progress and not enough focus here, generally an after thought. Also have stateless clients etc.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Adaptive nodes&lt;/strong&gt;. See above. With subprotocols and capabilities in Ethereum and libp2p, this is getting easier.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Sustainable&lt;/strong&gt;. Token economics is a thing. While a lot of it won’t stay around, there are many more projects working on making themselves dispensable. Being open source, having an engaged community and enabling users run their own infrastructure. Users as stakeholders.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Spam resistant&lt;/strong&gt;. Tricky problem if you want to be pseudonymous, but some signs of hope with incentivization mechanisms, zero knowledge based signaling, etc. Together with various forms of rate limiting and better controlling of topology and network amplification. And just generally being battle-tested by real world attacks, such as historical Ethereum DDoS attacks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Trust minimized&lt;/strong&gt;. Bitcoin. Zero knowledge provable computation. Open source. Reproducible builds. Signed binaries. Incentive compatible structures. Independent audits. Still a lot of work, but getting better.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Open source&lt;/strong&gt;. Big and only getting bigger. Including mainstream companies.&lt;/p&gt;

&lt;h2 id=&quot;whats-next&quot;&gt;What’s next?&lt;/h2&gt;

&lt;p&gt;We’ve look at what WeChat provides and what we’d like an alternative to look like. We’ve also seen a few principal modules that are necessary to achieve those goals. To achieve all of this is a daunting task, and one might call it overly ambitiuous. We’ve also seen how far we’ve come with some of the goals, and how a lot of the pieces are there, in one form or another. Then it is a question of putting them all together in the right mix.&lt;/p&gt;

&lt;p&gt;The good news is that a lot of people are working all these building blocks and thinking about these problems. Compared to a few years ago we’ve come quite far when it comes to p2p infrastructure, privacy, security, scalability, and general developer mass and mindshare. If you want to join us in building some of these building blocks, and assembling them, check out our forum.&lt;/p&gt;

&lt;p&gt;PS. We are &lt;a href=&quot;https://status.im/our_team/open_positions.html&quot;&gt;hiring protocol engineers&lt;/a&gt;. DS&lt;/p&gt;

&lt;h2 id=&quot;acknowledgements&quot;&gt;Acknowledgements&lt;/h2&gt;

&lt;p&gt;Corey, Dean, Jacek.&lt;/p&gt;

&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;

&lt;div class=&quot;footnotes&quot; role=&quot;doc-endnotes&quot;&gt;
  &lt;ol&gt;
    &lt;li id=&quot;fn:1&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws &lt;a href=&quot;#fnref:1&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:2&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/ &lt;a href=&quot;#fnref:2&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:3&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Net Neutrality: https://www.eff.org/issues/net-neutrality &lt;a href=&quot;#fnref:3&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:4&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/ &lt;a href=&quot;#fnref:4&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:5&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf &lt;a href=&quot;#fnref:5&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:6&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;The Case for Electronic Cash:  https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf &lt;a href=&quot;#fnref:6&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:7&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html &lt;a href=&quot;#fnref:7&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:8&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions &lt;a href=&quot;#fnref:8&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:9&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Shielded transactions usage (stats page 404s): https://z.cash/support/faq/ &lt;a href=&quot;#fnref:9&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
  &lt;/ol&gt;
&lt;/div&gt;</content><author><name>oskarth</name></author><category term="research" /><summary type="html">What would it take to replace WeChat? More specifically, what would a self-sovereign, private, censorship-resistant and open alternative look like? One that allows people to communicate, coordinate and transact freely.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://vac.dev/assets/img/tianstatue.jpg" /><media:content medium="image" url="https://vac.dev/assets/img/tianstatue.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>