From 4010ed8cb006f869ad56170d8e23c72ac16f88d1 Mon Sep 17 00:00:00 2001 From: Oskar Thoren Date: Tue, 1 Oct 2019 12:33:06 +0800 Subject: [PATCH] rewrite intro --- _posts/2019-10-04-remote-log.md | 49 ++++++++++++++++----------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/_posts/2019-10-04-remote-log.md b/_posts/2019-10-04-remote-log.md index 045b564f..ebfc7779 100644 --- a/_posts/2019-10-04-remote-log.md +++ b/_posts/2019-10-04-remote-log.md @@ -11,13 +11,29 @@ summary: A research log. Reliable and decentralized, pick two. image: /assets/img/remote_log.png --- -A big problem when doing p2p data sync over mobilephones is that most devices -are offline. With a naive approach, you quickly run into issues of 'ping-pong' -behavior. +A big problem when doing end-to-end data sync between mobile nodes is that most +devices are offline most of the time. With a naive approach, you quickly run +into issues of 'ping-pong' behavior, where messages have to be constantly +retransmitted. We saw some basic calculations of what this bandwidth multiplier +looks like in a [previous post](https://vac.dev/p2p-data-sync-for-mobile). -This problem was identified in the previous post on data sync (TODO: link). -Recall that some requirements weren't fully satisfied. In this post we outline -an extension that solves his. +While you could do some background processing, this is really draining the +battery, and on iOS these capabilities are limited. A better approach instead is +to loosen the constraint that two nodes need to be online at the same time. How +do we do this? There are two main approaches, one is the *store and forward +model*, and the other is a *remote log*. + +In the *store and forward* model, we use an intermediate node that forward +messages on behalf of the recipient. In the *remote log* model, you instead +replicate the data onto some decentralized storage, and have a mutable reference +to the latest state, similar to DNS. While both work, the latter is somewhat +more elegant and "pure", as it has less strict requirements of an individual +node's uptime. Both act as a highly-available cache to smoothen over +non-overlapping connection windows between endpoints. + +In this post we are going to describe how such a remote log schema could work. +Specifically, how it enhances p2p data sync and takes care of the [following +requirements](https://vac.dev/p2p-data-sync-for-mobile): > 3. MUST allow for mobile-friendly usage. By mobile-friendly we mean devices > that are resource restricted, mostly-offline and often changing network. @@ -27,27 +43,8 @@ an extension that solves his. > Swarm. These help with availability and latency of data for mostly-offline > devices. -We wrote the following: -> The problem above hints at the requirements 3 and 4 above. While we did get -> reliable syncing (requirement 1), it came at a big cost. - -> There are a few ways of getting around this issue. One is having a *store and -> forward* model, where some intermediary node picks up (encrypted) messages and -> forwards them to the recipient. This is what we have in production right now -> at Status. - -> Another, arguably more pure and robust, way is having a *remote log*, where -> the actual data is spread over some decentralized storage layer, and you have -> a mutable reference to find the latest messages, similar to DNS. - -> What they both have in common is that they act as a sort of highly-available -> cache to smooth over the non-overlapping connection windows between two -> endpoints. Neither of them are *required* to get reliable data transmission. - -In this post, we'll outline this remote log in a bit more detail. - -# Remote log +## Remote log A remote log is a replication of a local log. This means a node can read data from a node that is offline.