github/tlsn
1
1
Fork 0
mirror of https://github.com/tlsnotary/tlsn.git synced 2026-01-10 07:07:57 -05:00
Code Issues Packages Projects Releases Wiki Activity

feat: automated network benches (#457)

Browse Source 
* feat: automated network benches

* Update tlsn/benches/src/metrics.rs

Co-authored-by: dan <themighty1@users.noreply.github.com>

* remove explicit drops

* remove unnecessary sudo

---------

Co-authored-by: dan <themighty1@users.noreply.github.com>
This commit is contained in:
sinu.eth
2024-03-20 09:01:40 -08:00
committed by GitHub
parent 19e9c50f35
commit 9a081c6cbc
20 changed files with 835 additions and 452 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@@ -30,3 +30,6 @@ Cargo.lock
# logs
*.log
# metrics
*.csv

16
tlsn/benches/Cargo.toml
View File

@@ -21,19 +21,19 @@ tokio = { workspace = true, features = [
] }
tokio-util.workspace = true
tracing-subscriber = { workspace = true, features = ["env-filter"] }
anyhow = "1.0"
serde.workspace = true
toml = "0.8.11"
csv = "1.3.0"
[[bin]]
name = "setup_network"
path = "src/setup_network.rs"
[[bin]]
name = "cleanup_network"
path = "src/cleanup_network.rs"
name = "bench"
path = "bin/bench.rs"
[[bin]]
name = "prover"
path = "src/prover.rs"
path = "bin/prover.rs"
[[bin]]
name = "verifier"
path = "src/verifier.rs"
path = "bin/verifier.rs"

116
tlsn/benches/README.md
View File

@@ -4,118 +4,32 @@ This crate provides utilities for benchmarking protocol performance under variou
As the protocol is mostly IO bound, it's important to track how it performs in low bandwidth and/or high latency environments. To do this we set up temporary network namespaces and add virtual ethernet interfaces which we can control using the linux `tc` (Traffic Control) utility.
## Setup
## Configuration
To start we must create network namespaces for the prover and verifier, respectively.
See the `bench.toml` file for benchmark configurations.
## Preliminaries
To run the benchmarks you will need `iproute2` installed, eg:
```sh
ip netns add prover-ns
ip netns add verifier-ns
```
Then we create a pair of virtual ethernet interfaces and add them to their respective namespaces.
```sh
ip link add prover-veth type veth peer name verifier-veth
ip link set prover-veth netns prover-ns
ip link set verifier-veth netns verifier-ns
```
If successful you should be able to see each interface in its namespace. For example, to see the prover interface:
```sh
ip netns exec prover-ns ip link
```
Then, activate each interface (bring it up).
```sh
ip netns exec prover-ns ip link set prover-veth up
ip netns exec verifier-ns ip link set verifier-veth up
```
Next we'll assign IP addresses to each interface and set default routes:
```sh
ip netns exec prover-ns ip addr add 10.10.1.0/24 dev prover-veth
ip netns exec prover-ns ip route add default via 10.10.1.0 dev prover-veth
ip netns exec verifier-ns ip addr add 10.10.1.1/24 dev verifier-veth
ip netns exec verifier-ns ip route add default via 10.10.1.1 dev verifier-veth
```
Verify that everything worked by pinging between them:
```sh
ip netns exec prover-ns ping 10.10.1.1
```
## Clean up
For future reference, you can clean up this configuration as shown below.
First, delete the interface pair (this removes both):
```sh
ip netns exec prover-ns ip link delete prover-veth
```
Finally, delete each namespace:
```sh
ip netns del prover-ns
ip netns del verifier-ns
```
## Configuration binaries
Alternatively, instead of doing the above configuration manually, you can build the `setup_network` and `cleanup_network` binaries and execute them instead. Though they haven't been tested and you have to run them as root, so use at your own risk.
## Configuring network
To simulate different network conditions we use the linux utility `tc`. Typically, only the egress performance of an interface is configured. So we will configure the egress of both the prover and verifier to simulate the conditions we want.
### Adding rules
For example, to add both an egress bandwidth limit and delay to the prover we can do this:
```sh
ip netns exec prover-ns tc qdisc add dev prover-veth root handle 1: tbf rate 10mbit burst 1mbit latency 60s
ip netns exec prover-ns tc qdisc add dev prover-veth parent 1:1 handle 10: netem delay 50ms
```
The above command will chain a bandwidth filter with a delay filter. The bandwidth filter will cap prover "upload" at 10Mbps with 1Mbps bursts, and drops packets not sent within 60. The delay filter will cause all packets to wait 50ms before arriving at the verifier's network interface.
To simulate a prover with 10Mbps up and 100Mbps down @100ms latency with the verifier, one would also add the following filters to the verifier interface:
```sh
ip netns exec verifier-ns tc qdisc add dev verifier-veth root handle 1: tbf rate 100mbit burst 1mbit latency 60s
ip netns exec verifier-ns tc qdisc add dev verifier-veth parent 1:1 handle 10: netem delay 50ms
```
### Modifying rules
To modify a rule you have to delete the existing one and re-add a new one.
### Deleting rules
You can delete all rules on a device like so:
```sh
ip netns exec prover-ns tc qdisc del dev prover-veth root
sudo apt-get install iproute2 -y
```
## Running benches
In order to run a binary in another network namespace you need to run as root, and this won't place nice with cargo. The simplest way to run the bench is to first compile the binaries and run them directly.
Running the benches requires root privileges because they will set up virtual interfaces. The script is designed to fully clean up when the benches are done, but run them at your own risk.
Make sure you're in the `tlsn/benches/` directory, build the binaries then run the script:
```sh
cargo b --bin prover --release
cargo b --bin verifier --release
cargo build --release
sudo ./bench.sh
```
Run these separately:
## Metrics
After you run the benches you will see a `metrics.csv` file in the working directory. It will be owned by `root`, so you probably want to run
```sh
ip netns exec prover-ns ../target/release/prover
ip netns exec verifier-ns ../target/release/verifier
sudo chown $USER metrics.csv
```

10
tlsn/benches/bench.sh Executable file
View File

@@ -0,0 +1,10 @@
#! /bin/bash
# Check if we are running as root
if [ "$EUID" -ne 0 ]
then echo "This script must be run as root"
exit
fi
# Run the benchmark binary
../target/release/bench

39
tlsn/benches/bench.toml Normal file
View File

@@ -0,0 +1,39 @@
[[benches]]
name = "latency"
upload = 250
upload-delay = [10, 25, 50]
download = 250
download-delay = [10, 25, 50]
upload-size = 1024
download-size = 4096
defer-decryption = true
[[benches]]
name = "download_bandwidth"
upload = 250
upload-delay = 25
download = [10, 25, 50, 100, 250]
download-delay = 25
upload-size = 1024
download-size = 4096
defer-decryption = true
[[benches]]
name = "upload_bandwidth"
upload = [10, 25, 50, 100, 250]
upload-delay = 25
download = 250
download-delay = 25
upload-size = 1024
download-size = 4096
defer-decryption = [false, true]
[[benches]]
name = "download_volume"
upload = 250
upload-delay = 25
download = 250
download-delay = 25
upload-size = 1024
download-size = [1024, 4096, 16384, 65536]
defer-decryption = true

44
tlsn/benches/bin/bench.rs Normal file
View File

@@ -0,0 +1,44 @@
use std::process::Command;
use tlsn_benches::{clean_up, set_up};
fn main() {
let prover_path =
std::env::var("PROVER_PATH").unwrap_or_else(|_| "../target/release/prover".to_string());
let verifier_path =
std::env::var("VERIFIER_PATH").unwrap_or_else(|_| "../target/release/verifier".to_string());
if let Err(e) = set_up() {
println!("Error setting up: {}", e);
clean_up();
}
// Run prover and verifier binaries in parallel
let Ok(mut verifier) = Command::new("ip")
.arg("netns")
.arg("exec")
.arg("verifier-ns")
.arg(verifier_path)
.spawn()
else {
println!("Failed to start verifier");
return clean_up();
};
let Ok(mut prover) = Command::new("ip")
.arg("netns")
.arg("exec")
.arg("prover-ns")
.arg(prover_path)
.spawn()
else {
println!("Failed to start prover");
return clean_up();
};
// Wait for both to finish
_ = prover.wait();
_ = verifier.wait();
clean_up();
}

183
tlsn/benches/bin/prover.rs Normal file
View File

@@ -0,0 +1,183 @@
use std::{
io::Write,
sync::{
atomic::{AtomicU64, Ordering},
Arc,
},
time::Instant,
};
use anyhow::Context;
use futures::{AsyncReadExt, AsyncWriteExt};
use tlsn_benches::{
config::{BenchInstance, Config},
metrics::Metrics,
set_interface, PROVER_INTERFACE,
};
use tlsn_core::Direction;
use tlsn_server_fixture::{CA_CERT_DER, SERVER_DOMAIN};
use tokio::io::{AsyncRead, AsyncWrite};
use tokio_util::{
compat::TokioAsyncReadCompatExt,
io::{InspectReader, InspectWriter},
};
use tlsn_prover::tls::{Prover, ProverConfig};
use tracing_subscriber::{fmt::format::FmtSpan, EnvFilter};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let config_path = std::env::var("CFG").unwrap_or_else(|_| "bench.toml".to_string());
let config: Config = toml::from_str(
&std::fs::read_to_string(config_path).context("failed to read config file")?,
)
.context("failed to parse config")?;
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_span_events(FmtSpan::NEW | FmtSpan::CLOSE)
.init();
let ip = std::env::var("VERIFIER_IP").unwrap_or_else(|_| "10.10.1.1".to_string());
let port: u16 = std::env::var("VERIFIER_PORT")
.map(|port| port.parse().expect("port is valid u16"))
.unwrap_or(8000);
let verifier_host = (ip.as_str(), port);
let mut file = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open("metrics.csv")
.context("failed to open metrics file")?;
{
let mut metric_wtr = csv::Writer::from_writer(&mut file);
for bench in config.benches {
let instances = bench.flatten();
for instance in instances {
println!("{:?}", &instance);
let io = tokio::net::TcpStream::connect(verifier_host)
.await
.context("failed to open tcp connection")?;
metric_wtr.serialize(
run_instance(instance, io)
.await
.context("failed to run instance")?,
)?;
metric_wtr.flush()?;
}
}
}
file.flush()?;
Ok(())
}
async fn run_instance<S: AsyncWrite + AsyncRead + Send + Sync + Unpin + 'static>(
instance: BenchInstance,
io: S,
) -> anyhow::Result<Metrics> {
let uploaded = Arc::new(AtomicU64::new(0));
let downloaded = Arc::new(AtomicU64::new(0));
let io = InspectWriter::new(
InspectReader::new(io, {
let downloaded = downloaded.clone();
move |data| {
downloaded.fetch_add(data.len() as u64, Ordering::Relaxed);
}
}),
{
let uploaded = uploaded.clone();
move |data| {
uploaded.fetch_add(data.len() as u64, Ordering::Relaxed);
}
},
);
let BenchInstance {
name,
upload,
upload_delay,
download,
download_delay,
upload_size,
download_size,
defer_decryption,
} = instance.clone();
set_interface(PROVER_INTERFACE, upload, 1, upload_delay)?;
let (client_conn, server_conn) = tokio::io::duplex(2 << 16);
tokio::spawn(tlsn_server_fixture::bind(server_conn.compat()));
let start_time = Instant::now();
let prover = Prover::new(
ProverConfig::builder()
.id("test")
.server_dns(SERVER_DOMAIN)
.root_cert_store(root_store())
.max_sent_data(upload_size + 256)
.max_recv_data(download_size + 256)
.build()
.context("invalid prover config")?,
)
.setup(io.compat())
.await?;
let (mut mpc_tls_connection, prover_fut) = prover.connect(client_conn.compat()).await.unwrap();
let prover_ctrl = prover_fut.control();
let prover_task = tokio::spawn(prover_fut);
let request = format!(
"GET /bytes?size={} HTTP/1.1\r\nConnection: close\r\nData: {}\r\n\r\n",
download_size,
String::from_utf8(vec![0x42u8; upload_size]).unwrap(),
);
if defer_decryption {
prover_ctrl.defer_decryption().await?;
}
mpc_tls_connection.write_all(request.as_bytes()).await?;
mpc_tls_connection.close().await?;
let mut response = vec![];
mpc_tls_connection.read_to_end(&mut response).await?;
let mut prover = prover_task.await??.start_prove();
prover.reveal(0..prover.sent_transcript().data().len(), Direction::Sent)?;
prover.reveal(
0..prover.recv_transcript().data().len(),
Direction::Received,
)?;
prover.prove().await?;
prover.finalize().await?;
Ok(Metrics {
name,
upload,
upload_delay,
download,
download_delay,
upload_size,
download_size,
defer_decryption,
runtime: Instant::now().duration_since(start_time).as_secs(),
uploaded: uploaded.load(Ordering::SeqCst),
downloaded: downloaded.load(Ordering::SeqCst),
})
}
fn root_store() -> tls_core::anchors::RootCertStore {
let mut root_store = tls_core::anchors::RootCertStore::empty();
root_store
.add(&tls_core::key::Certificate(CA_CERT_DER.to_vec()))
.unwrap();
root_store
}

89
tlsn/benches/bin/verifier.rs Normal file
View File

@@ -0,0 +1,89 @@
use anyhow::Context;
use tls_core::verify::WebPkiVerifier;
use tlsn_benches::{
config::{BenchInstance, Config},
set_interface, VERIFIER_INTERFACE,
};
use tlsn_server_fixture::CA_CERT_DER;
use tokio::io::{AsyncRead, AsyncWrite};
use tokio_util::compat::TokioAsyncReadCompatExt;
use tlsn_verifier::tls::{Verifier, VerifierConfig};
use tracing_subscriber::{fmt::format::FmtSpan, EnvFilter};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let config_path = std::env::var("CFG").unwrap_or_else(|_| "bench.toml".to_string());
let config: Config = toml::from_str(
&std::fs::read_to_string(config_path).context("failed to read config file")?,
)
.context("failed to parse config")?;
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_span_events(FmtSpan::NEW | FmtSpan::CLOSE)
.init();
let ip = std::env::var("VERIFIER_IP").unwrap_or_else(|_| "10.10.1.1".to_string());
let port: u16 = std::env::var("VERIFIER_PORT")
.map(|port| port.parse().expect("port is valid u16"))
.unwrap_or(8000);
let host = (ip.as_str(), port);
let listener = tokio::net::TcpListener::bind(host)
.await
.context("failed to bind to port")?;
for bench in config.benches {
for instance in bench.flatten() {
let (io, _) = listener
.accept()
.await
.context("failed to accept connection")?;
run_instance(instance, io)
.await
.context("failed to run instance")?;
}
}
Ok(())
}
async fn run_instance<S: AsyncWrite + AsyncRead + Send + Sync + Unpin + 'static>(
instance: BenchInstance,
io: S,
) -> anyhow::Result<()> {
let BenchInstance {
download,
download_delay,
upload_size,
download_size,
..
} = instance;
set_interface(VERIFIER_INTERFACE, download, 1, download_delay)?;
let verifier = Verifier::new(
VerifierConfig::builder()
.id("test")
.cert_verifier(cert_verifier())
.max_sent_data(upload_size + 256)
.max_recv_data(download_size + 256)
.build()?,
);
_ = verifier.verify(io.compat()).await?;
println!("verifier done");
Ok(())
}
fn cert_verifier() -> WebPkiVerifier {
let mut root_store = tls_core::anchors::RootCertStore::empty();
root_store
.add(&tls_core::key::Certificate(CA_CERT_DER.to_vec()))
.unwrap();
WebPkiVerifier::new(root_store, None)
}

30
tlsn/benches/src/cleanup_network.rs
View File

@@ -1,30 +0,0 @@
// Clean up the network namespaces and interface pair created by setup_network.rs
use std::process::Command;
use tlsn_benches::*;
fn main() -> Result<(), std::io::Error> {
// Delete interface pair
Command::new("sudo")
.args(&[
"ip",
"netns",
"exec",
PROVER_NAMESPACE,
"ip",
"link",
"delete",
PROVER_INTERFACE,
])
.status()?;
// Delete namespaces
Command::new("sudo")
.args(&["ip", "netns", "del", PROVER_NAMESPACE])
.status()?;
Command::new("sudo")
.args(&["ip", "netns", "del", VERIFIER_NAMESPACE])
.status()?;
Ok(())
}

111
tlsn/benches/src/config.rs Normal file
View File

@@ -0,0 +1,111 @@
use serde::{Deserialize, Serialize};
#[derive(Deserialize)]
#[serde(untagged)]
pub enum Field<T> {
Single(T),
Multiple(Vec<T>),
}
#[derive(Deserialize)]
pub struct Config {
pub benches: Vec<Bench>,
}
#[derive(Deserialize)]
pub struct Bench {
pub name: String,
pub upload: Field<usize>,
#[serde(rename = "upload-delay")]
pub upload_delay: Field<usize>,
pub download: Field<usize>,
#[serde(rename = "download-delay")]
pub download_delay: Field<usize>,
#[serde(rename = "upload-size")]
pub upload_size: Field<usize>,
#[serde(rename = "download-size")]
pub download_size: Field<usize>,
#[serde(rename = "defer-decryption")]
pub defer_decryption: Field<bool>,
}
impl Bench {
/// Flattens the config into a list of instances
pub fn flatten(self) -> Vec<BenchInstance> {
let mut instances = vec![];
let upload = match self.upload {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
let upload_delay = match self.upload_delay {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
let download = match self.download {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
let download_latency = match self.download_delay {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
let upload_size = match self.upload_size {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
let download_size = match self.download_size {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
let defer_decryption = match self.defer_decryption {
Field::Single(u) => vec![u],
Field::Multiple(u) => u,
};
for u in upload {
for ul in &upload_delay {
for d in &download {
for dl in &download_latency {
for us in &upload_size {
for ds in &download_size {
for dd in &defer_decryption {
instances.push(BenchInstance {
name: self.name.clone(),
upload: u,
upload_delay: *ul,
download: *d,
download_delay: *dl,
upload_size: *us,
download_size: *ds,
defer_decryption: *dd,
});
}
}
}
}
}
}
}
instances
}
}
#[derive(Debug, Clone, Serialize)]
pub struct BenchInstance {
pub name: String,
pub upload: usize,
pub upload_delay: usize,
pub download: usize,
pub download_delay: usize,
pub upload_size: usize,
pub download_size: usize,
pub defer_decryption: bool,
}

249
tlsn/benches/src/lib.rs
View File

@@ -1,6 +1,255 @@
pub mod config;
pub mod metrics;
use std::{io, process::Command};
pub const PROVER_NAMESPACE: &str = "prover-ns";
pub const PROVER_INTERFACE: &str = "prover-veth";
pub const PROVER_SUBNET: &str = "10.10.1.0/24";
pub const VERIFIER_NAMESPACE: &str = "verifier-ns";
pub const VERIFIER_INTERFACE: &str = "verifier-veth";
pub const VERIFIER_SUBNET: &str = "10.10.1.1/24";
pub fn set_up() -> io::Result<()> {
// Create network namespaces
create_network_namespace(PROVER_NAMESPACE)?;
create_network_namespace(VERIFIER_NAMESPACE)?;
// Create veth pair and attach to namespaces
create_veth_pair(
PROVER_NAMESPACE,
PROVER_INTERFACE,
VERIFIER_NAMESPACE,
VERIFIER_INTERFACE,
)?;
// Set devices up
set_device_up(PROVER_NAMESPACE, PROVER_INTERFACE)?;
set_device_up(VERIFIER_NAMESPACE, VERIFIER_INTERFACE)?;
// Assign IPs
assign_ip_to_interface(PROVER_NAMESPACE, PROVER_INTERFACE, PROVER_SUBNET)?;
assign_ip_to_interface(VERIFIER_NAMESPACE, VERIFIER_INTERFACE, VERIFIER_SUBNET)?;
// Set default routes
set_default_route(
PROVER_NAMESPACE,
PROVER_INTERFACE,
PROVER_SUBNET.split('/').nth(0).unwrap(),
)?;
set_default_route(
VERIFIER_NAMESPACE,
VERIFIER_INTERFACE,
VERIFIER_SUBNET.split('/').nth(0).unwrap(),
)?;
Ok(())
}
pub fn clean_up() {
// Delete interface pair
if let Err(e) = Command::new("ip")
.args(&[
"netns",
"exec",
PROVER_NAMESPACE,
"ip",
"link",
"delete",
PROVER_INTERFACE,
])
.status()
{
println!("Error deleting interface {}: {}", PROVER_INTERFACE, e);
}
// Delete namespaces
if let Err(e) = Command::new("ip")
.args(&["netns", "del", PROVER_NAMESPACE])
.status()
{
println!("Error deleting namespace {}: {}", PROVER_NAMESPACE, e);
}
if let Err(e) = Command::new("ip")
.args(&["netns", "del", VERIFIER_NAMESPACE])
.status()
{
println!("Error deleting namespace {}: {}", VERIFIER_NAMESPACE, e);
}
}
/// Sets the interface parameters.
///
/// Must be run in the correct namespace.
///
/// # Arguments
///
/// * `egress` - The egress bandwidth in mbps.
/// * `burst` - The burst in mbps.
/// * `delay` - The delay in ms.
pub fn set_interface(interface: &str, egress: usize, burst: usize, delay: usize) -> io::Result<()> {
// Clear rules
_ = Command::new("tc")
.arg("qdisc")
.arg("del")
.arg("dev")
.arg(interface)
.arg("root")
.status();
// Egress
Command::new("tc")
.arg("qdisc")
.arg("add")
.arg("dev")
.arg(interface)
.arg("root")
.arg("handle")
.arg("1:")
.arg("tbf")
.arg("rate")
.arg(format!("{}mbit", egress))
.arg("burst")
.arg(format!("{}mbit", burst))
.arg("latency")
.arg("60s")
.status()?;
// Delay
Command::new("tc")
.arg("qdisc")
.arg("add")
.arg("dev")
.arg(interface)
.arg("parent")
.arg("1:1")
.arg("handle")
.arg("10:")
.arg("netem")
.arg("delay")
.arg(format!("{}ms", delay))
.status()?;
Ok(())
}
/// Create a network namespace with the given name if it does not already exist.
fn create_network_namespace(name: &str) -> io::Result<()> {
// Check if namespace already exists
if Command::new("ip")
.args(&["netns", "list"])
.output()?
.stdout
.windows(name.len())
.any(|ns| ns == name.as_bytes())
{
println!("Namespace {} already exists", name);
return Ok(());
} else {
println!("Creating namespace {}", name);
Command::new("ip").args(&["netns", "add", name]).status()?;
}
Ok(())
}
fn create_veth_pair(
left_namespace: &str,
left_interface: &str,
right_namespace: &str,
right_interface: &str,
) -> io::Result<()> {
// Check if interfaces are already present in namespaces
if is_interface_present_in_namespace(left_namespace, left_interface)?
|| is_interface_present_in_namespace(right_namespace, right_interface)?
{
println!("Virtual interface already exists.");
return Ok(());
}
// Create veth pair
Command::new("ip")
.args(&[
"link",
"add",
left_interface,
"type",
"veth",
"peer",
"name",
right_interface,
])
.status()?;
println!(
"Created veth pair {} and {}",
left_interface, right_interface
);
// Attach veth pair to namespaces
attach_interface_to_namespace(left_namespace, left_interface)?;
attach_interface_to_namespace(right_namespace, right_interface)?;
Ok(())
}
fn attach_interface_to_namespace(namespace: &str, interface: &str) -> io::Result<()> {
Command::new("ip")
.args(&["link", "set", interface, "netns", namespace])
.status()?;
println!("Attached {} to namespace {}", interface, namespace);
Ok(())
}
fn set_default_route(namespace: &str, interface: &str, ip: &str) -> io::Result<()> {
Command::new("ip")
.args(&[
"netns", "exec", namespace, "ip", "route", "add", "default", "via", ip, "dev",
interface,
])
.status()?;
println!(
"Set default route for namespace {} ip {} to {}",
namespace, ip, interface
);
Ok(())
}
fn is_interface_present_in_namespace(
namespace: &str,
interface: &str,
) -> Result<bool, std::io::Error> {
Ok(Command::new("ip")
.args(&[
"netns", "exec", namespace, "ip", "link", "list", "dev", interface,
])
.output()?
.stdout
.windows(interface.len())
.any(|ns| ns == interface.as_bytes()))
}
fn set_device_up(namespace: &str, interface: &str) -> io::Result<()> {
Command::new("ip")
.args(&[
"netns", "exec", namespace, "ip", "link", "set", interface, "up",
])
.status()?;
Ok(())
}
fn assign_ip_to_interface(namespace: &str, interface: &str, ip: &str) -> io::Result<()> {
Command::new("ip")
.args(&[
"netns", "exec", namespace, "ip", "addr", "add", ip, "dev", interface,
])
.status()?;
Ok(())
}

26
tlsn/benches/src/metrics.rs Normal file
View File

@@ -0,0 +1,26 @@
use serde::Serialize;
#[derive(Debug, Clone, Serialize)]
pub struct Metrics {
pub name: String,
/// Upload bandwidth in Mbps.
pub upload: usize,
/// Upload latency in ms.
pub upload_delay: usize,
/// Download bandwidth in Mbps.
pub download: usize,
/// Download latency in ms.
pub download_delay: usize,
/// Total bytes sent to the server.
pub upload_size: usize,
/// Total bytes received from the server.
pub download_size: usize,
/// Whether deferred decryption was used.
pub defer_decryption: bool,
/// The total runtime of the benchmark in seconds.
pub runtime: u64,
/// The total amount of data uploaded to the verifier in bytes.
pub uploaded: u64,
/// The total amount of data downloaded from the verifier in bytes.
pub downloaded: u64,
}

79
tlsn/benches/src/prover.rs
View File

@@ -1,79 +0,0 @@
use std::time::Instant;
use futures::{AsyncReadExt, AsyncWriteExt};
use tlsn_core::Direction;
use tlsn_server_fixture::{CA_CERT_DER, SERVER_DOMAIN};
use tokio_util::compat::TokioAsyncReadCompatExt;
use tlsn_prover::tls::{Prover, ProverConfig};
use tracing_subscriber::{fmt::format::FmtSpan, EnvFilter};
#[tokio::main]
async fn main() {
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_span_events(FmtSpan::NEW | FmtSpan::CLOSE)
.init();
let (client_conn, server_conn) = tokio::io::duplex(2 << 16);
let server_task = tokio::spawn(tlsn_server_fixture::bind(server_conn.compat()));
let ip = std::env::var("VERIFIER_IP").unwrap_or_else(|_| "10.10.1.1".to_string());
let port: u16 = std::env::var("VERIFIER_PORT")
.map(|port| port.parse().expect("port is valid u16"))
.unwrap_or(8000);
let verifier_host = (ip.as_str(), port);
let verifier_conn = tokio::net::TcpStream::connect(verifier_host).await.unwrap();
let mut root_store = tls_core::anchors::RootCertStore::empty();
root_store
.add(&tls_core::key::Certificate(CA_CERT_DER.to_vec()))
.unwrap();
let start_time = Instant::now();
let prover = Prover::new(
ProverConfig::builder()
.id("test")
.server_dns(SERVER_DOMAIN)
.root_cert_store(root_store)
.build()
.unwrap(),
)
.setup(verifier_conn.compat())
.await
.unwrap();
let (mut mpc_tls_connection, prover_fut) = prover.connect(client_conn.compat()).await.unwrap();
let prover_task = tokio::spawn(async { prover_fut.await.unwrap() });
mpc_tls_connection
.write_all(b"GET /formats/json?size=8 HTTP/1.1\r\nConnection: close\r\n\r\n")
.await
.unwrap();
mpc_tls_connection.close().await.unwrap();
let mut response = vec![0u8; 1024];
mpc_tls_connection.read_to_end(&mut response).await.unwrap();
server_task.await.unwrap();
let mut prover = prover_task.await.unwrap().start_prove();
prover
.reveal(0..prover.sent_transcript().data().len(), Direction::Sent)
.unwrap();
prover
.reveal(
0..prover.recv_transcript().data().len(),
Direction::Received,
)
.unwrap();
prover.prove().await.unwrap();
prover.finalize().await.unwrap();
println!(
"completed: {} seconds",
Instant::now().duration_since(start_time).as_secs()
);
}

171
tlsn/benches/src/setup_network.rs
View File

@@ -1,171 +0,0 @@
// Set up network namespaces and veth pairs for benchmarking
use std::process::Command;
use tlsn_benches::{
PROVER_INTERFACE, PROVER_NAMESPACE, PROVER_SUBNET, VERIFIER_INTERFACE, VERIFIER_NAMESPACE,
VERIFIER_SUBNET,
};
fn main() -> Result<(), std::io::Error> {
// Create network namespaces
create_network_namespace(PROVER_NAMESPACE)?;
create_network_namespace(VERIFIER_NAMESPACE)?;
// Create veth pair and attach to namespaces
create_veth_pair(
PROVER_NAMESPACE,
PROVER_INTERFACE,
VERIFIER_NAMESPACE,
VERIFIER_INTERFACE,
)?;
// Set devices up
set_device_up(PROVER_NAMESPACE, PROVER_INTERFACE)?;
set_device_up(VERIFIER_NAMESPACE, VERIFIER_INTERFACE)?;
// Assign IPs
assign_ip_to_interface(PROVER_NAMESPACE, PROVER_INTERFACE, PROVER_SUBNET)?;
assign_ip_to_interface(VERIFIER_NAMESPACE, VERIFIER_INTERFACE, VERIFIER_SUBNET)?;
// Set default routes
set_default_route(
PROVER_NAMESPACE,
PROVER_INTERFACE,
PROVER_SUBNET.split('/').nth(0).unwrap(),
)?;
set_default_route(
VERIFIER_NAMESPACE,
VERIFIER_INTERFACE,
VERIFIER_SUBNET.split('/').nth(0).unwrap(),
)?;
Ok(())
}
/// Create a network namespace with the given name if it does not already exist.
fn create_network_namespace(name: &str) -> Result<(), std::io::Error> {
// Check if namespace already exists
if Command::new("sudo")
.args(&["ip", "netns", "list"])
.output()?
.stdout
.windows(name.len())
.any(|ns| ns == name.as_bytes())
{
println!("Namespace {} already exists", name);
return Ok(());
} else {
println!("Creating namespace {}", name);
Command::new("sudo")
.args(&["ip", "netns", "add", name])
.status()?;
}
Ok(())
}
fn create_veth_pair(
left_namespace: &str,
left_interface: &str,
right_namespace: &str,
right_interface: &str,
) -> Result<(), std::io::Error> {
// Check if interfaces are already present in namespaces
if is_interface_present_in_namespace(left_namespace, left_interface)?
|| is_interface_present_in_namespace(right_namespace, right_interface)?
{
println!("Virtual interface already exists.");
return Ok(());
}
// Create veth pair
Command::new("sudo")
.args(&[
"ip",
"link",
"add",
left_interface,
"type",
"veth",
"peer",
"name",
right_interface,
])
.status()?;
println!(
"Created veth pair {} and {}",
left_interface, right_interface
);
// Attach veth pair to namespaces
attach_interface_to_namespace(left_namespace, left_interface)?;
attach_interface_to_namespace(right_namespace, right_interface)?;
Ok(())
}
fn attach_interface_to_namespace(namespace: &str, interface: &str) -> Result<(), std::io::Error> {
Command::new("sudo")
.args(&["ip", "link", "set", interface, "netns", namespace])
.status()?;
println!("Attached {} to namespace {}", interface, namespace);
Ok(())
}
fn set_default_route(namespace: &str, interface: &str, ip: &str) -> Result<(), std::io::Error> {
Command::new("sudo")
.args(&[
"ip", "netns", "exec", namespace, "ip", "route", "add", "default", "via", ip, "dev",
interface,
])
.status()?;
println!(
"Set default route for namespace {} ip {} to {}",
namespace, ip, interface
);
Ok(())
}
fn is_interface_present_in_namespace(
namespace: &str,
interface: &str,
) -> Result<bool, std::io::Error> {
Ok(Command::new("sudo")
.args(&[
"ip", "netns", "exec", namespace, "ip", "link", "list", "dev", interface,
])
.output()?
.stdout
.windows(interface.len())
.any(|ns| ns == interface.as_bytes()))
}
fn set_device_up(namespace: &str, interface: &str) -> Result<(), std::io::Error> {
Command::new("sudo")
.args(&[
"ip", "netns", "exec", namespace, "ip", "link", "set", interface, "up",
])
.status()?;
Ok(())
}
fn assign_ip_to_interface(
namespace: &str,
interface: &str,
ip: &str,
) -> Result<(), std::io::Error> {
Command::new("sudo")
.args(&[
"ip", "netns", "exec", namespace, "ip", "addr", "add", ip, "dev", interface,
])
.status()?;
Ok(())
}

43
tlsn/benches/src/verifier.rs
View File

@@ -1,43 +0,0 @@
use tls_core::verify::WebPkiVerifier;
use tlsn_server_fixture::CA_CERT_DER;
use tokio_util::compat::TokioAsyncReadCompatExt;
use tlsn_verifier::tls::{Verifier, VerifierConfig};
use tracing_subscriber::{fmt::format::FmtSpan, EnvFilter};
#[tokio::main]
async fn main() {
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_span_events(FmtSpan::NEW | FmtSpan::CLOSE)
.init();
let ip = std::env::var("VERIFIER_IP").unwrap_or_else(|_| "10.10.1.1".to_string());
let port: u16 = std::env::var("VERIFIER_PORT")
.map(|port| port.parse().expect("port is valid u16"))
.unwrap_or(8000);
let host = (ip.as_str(), port);
let mut root_store = tls_core::anchors::RootCertStore::empty();
root_store
.add(&tls_core::key::Certificate(CA_CERT_DER.to_vec()))
.unwrap();
let verifier = Verifier::new(
VerifierConfig::builder()
.id("test")
.cert_verifier(WebPkiVerifier::new(root_store, None))
.build()
.unwrap(),
);
let listener = tokio::net::TcpListener::bind(host).await.unwrap();
let (prover_conn, _) = listener.accept().await.unwrap();
println!("connected to prover");
verifier.verify(prover_conn.compat()).await.unwrap();
println!("success");
}

2
tlsn/tlsn-prover/src/tls/error.rs
View File

@@ -21,7 +21,7 @@ pub enum ProverError {
#[error(transparent)]
InvalidServerName(#[from] tls_core::dns::InvalidDnsNameError),
#[error("error occurred in MPC protocol: {0}")]
MpcError(Box<dyn Error + Send + 'static>),
MpcError(Box<dyn Error + Send + Sync + 'static>),
#[error("server did not send a close_notify")]
ServerNoCloseNotify,
#[error(transparent)]

1
tlsn/tlsn-server-fixture/Cargo.toml
View File

@@ -6,6 +6,7 @@ edition = "2021"
[dependencies]
async-rustls = "0.4.1"
axum = "0.6"
anyhow = "1.0"
futures.workspace = true
hyper.workspace = true
rustls = "0.21.7"

65
tlsn/tlsn-server-fixture/src/lib.rs
View File

@@ -1,13 +1,16 @@
use std::{collections::HashMap, sync::Arc};
use std::{
collections::HashMap,
sync::{Arc, Mutex},
};
use async_rustls::TlsAcceptor;
use axum::{
extract::Query,
extract::{Query, State},
response::{Html, Json},
routing::get,
Router,
};
use futures::{AsyncRead, AsyncWrite};
use futures::{channel::oneshot, AsyncRead, AsyncWrite};
use hyper::{body::Bytes, server::conn::Http, StatusCode};
use rustls::{Certificate, PrivateKey, ServerConfig};
@@ -22,16 +25,23 @@ pub static SERVER_KEY_DER: &[u8] = include_bytes!("tls/domain_key.der");
/// The domain name bound to the server certificate.
pub static SERVER_DOMAIN: &str = "test-server.io";
fn app() -> Router {
struct AppState {
shutdown: Option<oneshot::Sender<()>>,
}
fn app(state: AppState) -> Router {
Router::new()
.route("/", get(|| async { "Hello, World!" }))
.route("/bytes", get(bytes))
.route("/formats/json", get(json))
.route("/formats/html", get(html))
.with_state(Arc::new(Mutex::new(state)))
}
/// Bind the server to the given socket.
pub async fn bind<T: AsyncRead + AsyncWrite + Send + Unpin + 'static>(socket: T) {
pub async fn bind<T: AsyncRead + AsyncWrite + Send + Unpin + 'static>(
socket: T,
) -> anyhow::Result<()> {
let key = PrivateKey(SERVER_KEY_DER.to_vec());
let cert = Certificate(SERVER_CERT_DER.to_vec());
@@ -43,26 +53,42 @@ pub async fn bind<T: AsyncRead + AsyncWrite + Send + Unpin + 'static>(socket: T)
let acceptor = TlsAcceptor::from(Arc::new(config));
let conn = acceptor.accept(socket).await.unwrap();
let conn = acceptor.accept(socket).await?;
Http::new()
.http1_only(true)
.http1_keep_alive(false)
.serve_connection(conn.compat(), app())
.await
.unwrap();
let (sender, receiver) = oneshot::channel();
let state = AppState {
shutdown: Some(sender),
};
tokio::select! {
_ = Http::new()
.http1_only(true)
.http1_keep_alive(false)
.serve_connection(conn.compat(), app(state)) => {},
_ = receiver => {},
}
Ok(())
}
async fn bytes(Query(params): Query<HashMap<String, String>>) -> Result<Bytes, StatusCode> {
async fn bytes(
State(state): State<Arc<Mutex<AppState>>>,
Query(params): Query<HashMap<String, String>>,
) -> Result<Bytes, StatusCode> {
let size = params
.get("size")
.and_then(|size| size.parse::<usize>().ok())
.unwrap_or(1);
if params.get("shutdown").is_some() {
_ = state.lock().unwrap().shutdown.take().unwrap().send(());
}
Ok(Bytes::from(vec![0x42u8; size]))
}
async fn json(
State(state): State<Arc<Mutex<AppState>>>,
Query(params): Query<HashMap<String, String>>,
) -> Result<Json<&'static str>, StatusCode> {
let size = params
@@ -70,6 +96,10 @@ async fn json(
.and_then(|size| size.parse::<usize>().ok())
.unwrap_or(1);
if params.get("shutdown").is_some() {
_ = state.lock().unwrap().shutdown.take().unwrap().send(());
}
match size {
1 => Ok(Json(include_str!("data/1kb.json"))),
4 => Ok(Json(include_str!("data/4kb.json"))),
@@ -78,6 +108,13 @@ async fn json(
}
}
async fn html() -> Html<&'static str> {
async fn html(
State(state): State<Arc<Mutex<AppState>>>,
Query(params): Query<HashMap<String, String>>,
) -> Html<&'static str> {
if params.get("shutdown").is_some() {
_ = state.lock().unwrap().shutdown.take().unwrap().send(());
}
Html(include_str!("data/4kb.html"))
}

8
tlsn/tlsn-server-fixture/src/main.rs
View File

@@ -1,16 +1,16 @@
use std::env;
use std::{env, io};
use tlsn_server_fixture::bind;
use tokio::net::TcpListener;
use tokio_util::compat::TokioAsyncWriteCompatExt;
#[tokio::main]
async fn main() {
async fn main() -> io::Result<()> {
let port = env::var("PORT").unwrap_or_else(|_| "3000".to_string());
let listener = TcpListener::bind(&format!("0.0.0.0:{port}")).await.unwrap();
let listener = TcpListener::bind(&format!("0.0.0.0:{port}")).await?;
loop {
let (socket, _) = listener.accept().await.unwrap();
let (socket, _) = listener.accept().await?;
tokio::spawn(bind(socket.compat_write()));
}
}

2
tlsn/tlsn-verifier/src/tls/error.rs
View File

@@ -10,7 +10,7 @@ pub enum VerifierError {
#[error(transparent)]
MuxerError(#[from] utils_aio::mux::MuxerError),
#[error("error occurred in MPC protocol: {0}")]
MpcError(Box<dyn Error + Send + 'static>),
MpcError(Box<dyn Error + Send + Sync + 'static>),
#[error("Range exceeds transcript length")]
InvalidRange,
}