diff --git a/rlog/2024-10-28-gsub-idontwant-perf-eval.mdx b/rlog/2024-10-28-gsub-idontwant-perf-eval.mdx new file mode 100644 index 00000000..894a31b6 --- /dev/null +++ b/rlog/2024-10-28-gsub-idontwant-perf-eval.mdx @@ -0,0 +1,130 @@ +--- +title: 'Libp2p GossipSub IDONTWANT Message Performance Impact' + + +date: 2024-10-28 12:00:00 +authors: farooq +published: true +slug: gsub-idontwant-perf-eval + +categories: research +discuss: https://forum.vac.dev/t/libp2p-gossipsub-idontwant-message-performance-impact/374 + + +toc_min_heading_level: 2 +toc_max_heading_level: 5 +--- + +This post provides quick insights into the IDONTWANT message performance and highlights minor tweaks that can further contribute to performance gains. + + +## Overview + +[IDONTWANT](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md?plain=1#L52) messages are introduced to curtail redundant transmissions without compromising resilience. +Cutting down on duplicates can potentially render two significant advantages: + + +1. Reducing bandwidth requirements + +2. Reducing message dissemination time (latency) + + +For IDONTWANTs to be effective, they must be received and processed by the sender before the sender starts relaying the respective message. + +[Duplicates investigation](https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921#arrival-time-of-duplicates-9) reveals that +the average time difference between the first message arrival and the first duplicate arrival is higher than the average round trip time in Ethereum's GossipSub network. + +This allows for timely IDONTWANT reception and canceling of many duplicate transmissions, +showing a potential for a significant drop in bandwidth utilization. +On the other hand, lowering message dissemination time is only possible by minimizing queuing delays at busy peers. + +## Experiments + +We conducted a series of experiments with different arrangements (changing heartbeat_interval and message size) +to precisely identify the impact of IDONTWANT messages on bandwidth utilization and message dissemination time. + +The experiments are performed on nim-libp2p using the [shadow simulator](https://github.com/vacp2p/dst-gossipsub-test-node/pull/4). +The peer bandwidth and link latency are uniformly set between 50-150 Mbps and 40-130 milliseconds in five stages. + +In all experiments, ten messages are transmitted to the network, i.e., +ten peers (publishers) are selected as the message transmitters. +Every publisher transmits exactly one message, +and inter-packet spacing (delay) is set to four seconds for each published message. +For a fair assessment, we ensure that the publishers are uniformly selected from each bandwidth class. + +At the start of each experiment, two additional messages are transmitted to increase the TCP $C_{wnd}$. +These messages are not included in latency computations. + +The simulation details are presented in the table below. + + +| **Parameter** | **Value** | **Parameter** | **Value** | +| ------------- | --------- | ------------- | --------- | +| Peers | 2000 | Publishers | 10 | +| Peer bandwidth | 50-150 Mbps | Link latency | 40-130 ms | + +| Message size | 1KB, 50KB, 500KB, 1MB | $D$ | 8 | + +| Heartbeat interval | 700ms, 1000ms, 1500ms | $D_{low}$ | 6 | + +| FloodPublish | False | $D_{high}$ | 12 | +| Gossip factor | 0.05 | Muxer | yamux | + + +## Findings + +We use bandwidth utilization and latency as evaluation metrics. +Bandwidth utilization represents total network-wide traffic (including gossip and other control messages). +Latency refers to network-wide message dissemination time. +The total number of IWANT requests and the number of message transmissions saved by IDONTWANT messages are also presented for detailed insights. + + +Experiments reveal that IDONTWANT messages yield a noticeable (up to 21%) drop in bandwidth utilization. +A higher drop is seen with a higher heartbeat interval. +Interestingly, a relatively low bandwidth reduction (12-20%) is seen for 1MB messages, +compared to 500KB messages (18-21%). + + +| ![Image 1](/img/idontwanttest/BW_700ms.png) | ![Image 2](/img/idontwanttest/BW_1000ms.png) | ![Image 3](/img/idontwanttest/BW_1500ms.png) | +|-----------------------------|-----------------------------|-----------------------------| + + +This is because downloading a large message may consume several hundred milliseconds. +During this time, a receiver will likely +[generate multiple IWANT requests](https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366) +for the same message, increasing bandwidth utilization. + +Moreover, a peer can generate +[IDONTWANTs only after it has finished downloading the message](https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281). +A longer download time will result in simultaneous reception of the same message from other mesh members. + + +| ![Image 1](/img/idontwanttest/IWANT_Requests.png) | ![Image 2](/img/idontwanttest/IDONTWANT_Saves.png) | +|-----------------------------|-----------------------------| + + +These IWANT requests mainly overwhelm early message receivers, +which can negatively impact message dissemination time on some occasions. +Therefore, a similar message dissemination time is seen with and without IDONTWANT messages. + + +| ![Image 1](/img/idontwanttest/Lat_700ms.png) | ![Image 2](/img/idontwanttest/Lat_1000ms.png) | ![Image 3](/img/idontwanttest/Lat_1500ms.png) | +|-----------------------------|-----------------------------|-----------------------------| + +Similar results are seen on our large-scale deployment runs +([running Waku nodes in Kubernetes](https://zealous-polka-dc7.notion.site/Nim-libp2p-v1-5-0-regression-testing-August-2024-25edba733c704ccaa411919555c5db1a)). + +Please feel free to join the discussion and leave feedback regarding this post in the +[VAC forum](https://forum.vac.dev/t/libp2p-gossipsub-idontwant-message-performance-impact/374). + + +## References + +- [GossipSub Specifications v1.2](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md) +- [GossipSub v1.2: IDONTWANT Control Message](https://github.com/libp2p/specs/pull/548) +- [Number Duplicate Messages in Ethereum’s Gossipsub Network](https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921) +- [IWANT Messages Impact on Latency ](https://forum.vac.dev/t/iwant-messages-may-have-negative-impact-on-message-dissemination-latency-for-large-messages/366) +- [Large Message Handling (IDONTWANT + IMReceiving)](https://forum.vac.dev/t/large-message-handling-idontwant-imreceiving/281) +- [IDONTWANT Message Impact Before/After Message Validation](https://forum.vac.dev/t/idontwant-message-impact/283) +- [GossipSub for Big Messages](https://hackmd.io/X1DoBHtYTtuGqYg0qK4zJw#2) +- [Regression Test Results: nim-libp2p](https://zealous-polka-dc7.notion.site/Nim-libp2p-v1-5-0-regression-testing-August-2024-25edba733c704ccaa411919555c5db1a) diff --git a/static/img/idontwanttest/BW_1000ms.png b/static/img/idontwanttest/BW_1000ms.png new file mode 100644 index 00000000..763f7ca6 Binary files /dev/null and b/static/img/idontwanttest/BW_1000ms.png differ diff --git a/static/img/idontwanttest/BW_1500ms.png b/static/img/idontwanttest/BW_1500ms.png new file mode 100644 index 00000000..043c653e Binary files /dev/null and b/static/img/idontwanttest/BW_1500ms.png differ diff --git a/static/img/idontwanttest/BW_700ms.png b/static/img/idontwanttest/BW_700ms.png new file mode 100644 index 00000000..06c3fa3d Binary files /dev/null and b/static/img/idontwanttest/BW_700ms.png differ diff --git a/static/img/idontwanttest/IDONTWANT_Saves.png b/static/img/idontwanttest/IDONTWANT_Saves.png new file mode 100644 index 00000000..772b8dec Binary files /dev/null and b/static/img/idontwanttest/IDONTWANT_Saves.png differ diff --git a/static/img/idontwanttest/IWANT_Requests.png b/static/img/idontwanttest/IWANT_Requests.png new file mode 100644 index 00000000..257b1688 Binary files /dev/null and b/static/img/idontwanttest/IWANT_Requests.png differ diff --git a/static/img/idontwanttest/Lat_1000ms.png b/static/img/idontwanttest/Lat_1000ms.png new file mode 100644 index 00000000..0af336bc Binary files /dev/null and b/static/img/idontwanttest/Lat_1000ms.png differ diff --git a/static/img/idontwanttest/Lat_1500ms.png b/static/img/idontwanttest/Lat_1500ms.png new file mode 100644 index 00000000..0f399407 Binary files /dev/null and b/static/img/idontwanttest/Lat_1500ms.png differ diff --git a/static/img/idontwanttest/Lat_700ms.png b/static/img/idontwanttest/Lat_700ms.png new file mode 100644 index 00000000..e98cc552 Binary files /dev/null and b/static/img/idontwanttest/Lat_700ms.png differ