consensus-specs/specs/simpleserialize.md

SimpleSerialize (SSZ) Spec

Work In Progress

This is the work in progress document to describe simpleserialize, the current selected serialization method for Ethereum 2.0 using the Beacon Chain.

This document specifies the general information for serializing and deserializing objects and data types.

ToC

• About
• Terminology
• Constants
• Overview

• Serialize/Encode
  • int/uint: 8/16/24/32/64/256
  • Address
  • Hash32
  • Bytes
  • List
• Deserialize/Decode
  • int/uint: 8/16/24/32/64/256
  • Address
  • Hash32
  • Bytes
  • List

• Implementations

About

SimpleSerialize was first proposed by Vitalik Buterin as the serializaiton protocol for use in the Ethereum 2.0 Beacon Chain.

The core feature of ssz is the simplicity of the serialization with low overhead.

Terminology

Term	Definition
big	Big Endian
byte_order	Specifies endianness: Big Endian or Little Endian.
len	Length/Number of Bytes.
to_bytes	Convert to bytes. Should take parameters size and byte_order.
from_bytes	Convert form bytes to object. Should take bytes and byte_order.
value	The value to serialize.
rawbytes	Raw serialized bytes.

Constants

Constant	Value	Definition
LENGTH_BYTES	4	Number of bytes used for the length added before the serialized object.

Overview

Serialize/Encode

int/uint: 8/16/24/32/64/256

Convert directly to bytes the size of the int. (e.g. uint16 = 2 bytes)

All integers are serialized as big endian.

Check to perform	Code
Size is a byte integer	int_size % 8 == 0
Value is less than max	2**int_size > value

buffer_size = int_size / 8
return value.to_bytes(buffer_size, 'big')

Address

The address should already come as a hash/byte format. Ensure that length is 20.

Check to perform	Code
Length is correct (20)	len(value) == 20

assert( len(value) == 20 )
return value

Hash32

The hash32 should already be a 32 byte length serialized data format. The safety check ensures the 32 byte length is satisfied.

Check to perform	Code
Length is correct (32)	len(value) == 32

assert( len(value) == 32 )
return value

Bytes

For general byte type:

1. Get the length/number of bytes; Encode into a 4 byte integer.
2. Append the value to the length and return: [ length_bytes ] + [ value_bytes ]

byte_length = (len(value)).to_bytes(4, 'big')
return byte_length + value

List

For lists of values, get the length of the list and then serialize the value of each item in the list:

1. For each item in list:
   1. serialize.
   2. append to string.
2. Get size of serialized string. Encode into a 4 byte integer.

serialized_list_string = ''

for item in value:
   serialized_list_string += serialize(item)

serialized_len = len(serialized_list_string)

return serialized_len + serialized_list_string

Deserialize/Decode

The decoding requires knowledge of the type of the item to be decoded. When performing decoding on an entire serialized string, it also requires knowledge of what order the objects have been serialized in.

Note: Each return will provide deserialized_object, new_index keeping track of the new index.

At each step, the following checks should be made:

Check Type	Check
Ensure sufficient length	length(rawbytes) > current_index + deserialize_length

int/uint: 8/16/24/32/64/256

Convert directly from bytes into integer utilising the number of bytes the same size as the integer length. (e.g. uint16 == 2 bytes)

All integers are interpreted as big endian.

byte_length = int_size / 8
new_index = current_index + int_size
return int.from_bytes(rawbytes[current_index:current_index+int_size], 'big'), new_index

Address

Return the 20 bytes.

new_index = current_index + 20
return rawbytes[current_index:current_index+20], new_index

Hash32

Return the 32 bytes.

new_index = current_index + 32
return rawbytes[current_index:current_index+32], new_index

Bytes

Get the length of the bytes, return the bytes.

bytes_length = int.from_bytes(rawbytes[current_index:current_index+4], 'big')
new_index = current_index + 4 + bytes_lenth
return rawbytes[current_index+4:current_index+4+bytes_length], new_index

List

Deserailize each object in the list.

1. Get the length of the serialized list.
2. Loop through deseralizing each item in the list until you reach the entire length of the list.

Check type	code
rawbytes has enough left for length	len(rawbytes) > current_index + 4

total_length = int.from_bytes(rawbytes[current_index:current_index+4], 'big')
new_index = current_index + 4 + total_length
item_index = current_index + 4
deserialized_list = []

while item_index < new_index:
   object, item_index = deserialize(rawbytes, item_index, item_type)
   deserialized_list.append(object)

return deserialized_list, new_index

Implementations

Language	Implementation	Description
Python	https://github.com/ethereum/beacon_chain/blob/master/ssz/ssz.py	Beacon chain reference implementation written in Python.
Rust	https://github.com/sigp/lighthouse/tree/master/ssz	Lighthouse (Rust Ethereum 2.0 Node) maintained SimpleSerialize.