consensus-specs/specs/simpleserialize.md

SimpleSerialize (SSZ) Spec

Work In Progress

This is the work in progress document to describe simpleserialize, the current selected serialization method for Ethereum 2.0 using the Beacon Chain.

This document specifies the general information for serializing and deserializing objects and data types.

ToC

• About
• Terminology
• Constants
• Overview
  • Serialize/Encode
    • uint: 8/16/24/32/64/256
    • Address
    • Hash
      • Hash32
      • Hash96
      • Hash97
    • Bytes
    • List
  • Deserialize/Decode
    • uint: 8/16/24/32/64/256
    • Address
    • Hash
      • Hash32
      • Hash96
      • Hash97
    • Bytes
    • List
• Implementations

About

SimpleSerialize was first proposed by Vitalik Buterin as the serialization protocol for use in the Ethereum 2.0 Beacon Chain.

The core feature of ssz is the simplicity of the serialization with low overhead.

Terminology

Term	Definition
big	Big Endian
byte_order	Specifies endianness: Big Endian or Little Endian.
len	Length/Number of Bytes.
to_bytes	Convert to bytes. Should take parameters size and byte_order.
from_bytes	Convert form bytes to object. Should take bytes and byte_order.
value	The value to serialize.
rawbytes	Raw serialized bytes.

Constants

Constant	Value	Definition
LENGTH_BYTES	4	Number of bytes used for the length added before the serialized object.

Overview

Serialize/Encode

uint: 8/16/24/32/64/256

Convert directly to bytes the size of the int. (e.g. uint16 = 2 bytes)

All integers are serialized as big endian.

Check to perform	Code
Size is a byte integer	int_size % 8 == 0
Value is less than max	2**int_size > value

buffer_size = int_size / 8
return value.to_bytes(buffer_size, 'big')

Address

The address should already come as a hash/byte format. Ensure that length is 20.

Check to perform	Code
Length is correct (20)	len(value) == 20

assert( len(value) == 20 )
return value

Hash

Hash Type	Usage
hash32	Hash size of keccak or blake2b[0.. < 32].
hash96	BLS Public Key Size.
hash97	BLS Public Key Size with recovery bit.

Checks to perform	Code
Length is correct (32) if hash32	len(value) == 32
Length is correct (96) if hash96	len(value) == 96
Length is correct (97) if hash97	len(value) == 97

Example all together

if (type(value) == 'hash32'):
   assert(len(value) == 32)
elif (type(value) == 'hash96'):
   assert(len(value) == 96)
elif (type(value) == 'hash97'):
   assert(len(value) == 97)
else:
   raise TypeError('Invalid hash type supplied')

return value

Hash32

Ensure 32 byte length and return the bytes.

assert(len(value) == 32)
return value

Hash96

Ensure 96 byte length and return the bytes.

assert(len(value) == 96)
return value

Hash97

Ensure 97 byte length and return the bytes.

assert(len(value) == 97)
return value

Bytes

For general byte type:

1. Get the length/number of bytes; Encode into a 4 byte integer.
2. Append the value to the length and return: [ length_bytes ] + [ value_bytes ]

Check to perform	Code
Length of bytes can fit into 4 bytes	len(value) < 2**32

byte_length = (len(value)).to_bytes(LENGTH_BYTES, 'big')
return byte_length + value

List

For lists of values, get the length of the list and then serialize the value of each item in the list:

1. For each item in list:
   1. serialize.
   2. append to string.
2. Get size of serialized string. Encode into a 4 byte integer.

serialized_list_string = ''

for item in value:
   serialized_list_string += serialize(item)

serialized_len = (len(serialized_list_string).to_bytes(LENGTH_BYTES, 'big'))

return serialized_len + serialized_list_string

Deserialize/Decode

The decoding requires knowledge of the type of the item to be decoded. When performing decoding on an entire serialized string, it also requires knowledge of what order the objects have been serialized in.

Note: Each return will provide deserialized_object, new_index keeping track of the new index.

At each step, the following checks should be made:

Check Type	Check
Ensure sufficient length	length(rawbytes) > current_index + deserialize_length

uint: 8/16/24/32/64/256

Convert directly from bytes into integer utilising the number of bytes the same size as the integer length. (e.g. uint16 == 2 bytes)

All integers are interpreted as big endian.

byte_length = int_size / 8
new_index = current_index + int_size
return int.from_bytes(rawbytes[current_index:current_index+int_size], 'big'), new_index

Address

Return the 20 bytes.

new_index = current_index + 20
return rawbytes[current_index:current_index+20], new_index

Hash

Hash32

Return the 32 bytes.

new_index = current_index + 32
return rawbytes[current_index:current_index+32], new_index

Hash96

Return the 96 bytes.

new_index = current_index + 96
return rawbytes[current_index:current_index+96], new_index

Hash97

Return the 97 bytes.

new_index = current_index + 97
return rawbytes[current_index:current_index+97], new_index

Bytes

Get the length of the bytes, return the bytes.

bytes_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'big')
new_index = current_index + LENGTH_BYTES + bytes_lenth
return rawbytes[current_index + LENGTH_BYTES:current_index+ LENGTH_BYTES +bytes_length], new_index

List

Deserialize each object in the list.

1. Get the length of the serialized list.
2. Loop through deserializing each item in the list until you reach the entire length of the list.

Check type	code
rawbytes has enough left for length	len(rawbytes) > current_index + 4

total_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'big')
new_index = current_index + LENGTH_BYTES + total_length
item_index = current_index + LENGTH_BYTES
deserialized_list = []

while item_index < new_index:
   object, item_index = deserialize(rawbytes, item_index, item_type)
   deserialized_list.append(object)

return deserialized_list, new_index

Implementations

Language	Implementation	Description
Python	https://github.com/ethereum/beacon_chain/blob/master/ssz/ssz.py	Beacon chain reference implementation written in Python.
Rust	https://github.com/sigp/lighthouse/tree/master/ssz	Lighthouse (Rust Ethereum 2.0 Node) maintained SSZ.
Nim	https://github.com/status-im/nim-beacon-chain/blob/master/beacon_chain/ssz.nim	Nim Implementation maintained SSZ.
Rust	https://github.com/paritytech/shasper/tree/master/util/ssz	Shasper implementation of SSZ maintained by ParityTech.