From 6ac5608d0be01a8eb40440bcb2030e3439af0557 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Tue, 22 Jan 2019 01:21:22 +0800 Subject: [PATCH 1/8] Explicit check bytes end --- specs/simple-serialize.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 8630c47c6..9774a9ecd 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -219,14 +219,22 @@ The decoding requires knowledge of the type of the item to be decoded. When performing decoding on an entire serialized string, it also requires knowledge of the order in which the objects have been serialized. -Note: Each return will provide ``deserialized_object, new_index`` keeping track -of the new index. +Note: Each return will provide: +- `deserialized_object` +- `new_index` At each step, the following checks should be made: | Check to perform | Check | |:-------------------------|:-----------------------------------------------------------| -| Ensure sufficient length | ``length(rawbytes) >= current_index + deserialize_length`` | +| Ensure sufficient length | ``len(rawbytes) >= current_index + deserialize_length`` | + +At the final step, the following checks should be made: + +| Check to perform | Check | +|:-------------------------|:-------------------------------------| +| Ensure no extra length | `new_index == len(rawbytes)` | + #### uint @@ -326,6 +334,7 @@ Instantiate a container with the full set of deserialized data, matching each me | rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | + To deserialize: 1. Get the list of the container's fields. From 5bb02a9d09cd2fe8a5358d1854bb1c9c888bf798 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Thu, 24 Jan 2019 15:53:52 +0800 Subject: [PATCH 2/8] Update specs/simple-serialize.md Co-Authored-By: ChihChengLiang --- specs/simple-serialize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 9774a9ecd..60b436e6d 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -331,7 +331,7 @@ Instantiate a container with the full set of deserialized data, matching each me | Check to perform | code | |:------------------------------------------|:----------------------------------------------------------------| -| rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | +| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | From c2112f0bfcf122856105d0690cccef9156dfc484 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 15:57:50 +0800 Subject: [PATCH 3/8] PR feedback: remove unnecessary newline --- specs/simple-serialize.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 60b436e6d..524a010d3 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -331,10 +331,9 @@ Instantiate a container with the full set of deserialized data, matching each me | Check to perform | code | |:------------------------------------------|:----------------------------------------------------------------| -| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | +| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | - To deserialize: 1. Get the list of the container's fields. From 88ffae633527a02c6de68df9c40b0e98615ffc83 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:08:56 +0800 Subject: [PATCH 4/8] define deserialized_object and new_index --- specs/simple-serialize.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 524a010d3..e0528b562 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -54,6 +54,9 @@ overhead. | `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byte_order``. | | `value` | The value to serialize. | | `rawbytes` | Raw serialized bytes. | +| `deserialized_object` | The deserialized data in the data structure of your programming language. | +| `new_index` | An index to keep track the latest position where the `rawbytes` have been deserialized. | + ## Constants From 14432e91a3fd0f66806faccc533c384c0fb5eb63 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:11:04 +0800 Subject: [PATCH 5/8] add code block to variable in the table --- specs/simple-serialize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index e0528b562..70245aebe 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -306,7 +306,7 @@ entire length of the list. | Check to perform | code | |:------------------------------------------|:----------------------------------------------------------------| -| rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | +| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | ```python From d41215aeececf065befefd2b99830a434ba58d01 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:11:45 +0800 Subject: [PATCH 6/8] rename Terminology to Variables and Functions --- specs/simple-serialize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 70245aebe..f69544ffa 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -43,7 +43,7 @@ protocol for use in the Ethereum 2.0 Beacon Chain. The core feature of `ssz` is the simplicity of the serialization with low overhead. -## Terminology +## Variables and Functions | Term | Definition | |:-------------|:-----------------------------------------------------------------------------------------------| From 5dfa4e005b2d9e07e684cecc027c70407463bda0 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:12:43 +0800 Subject: [PATCH 7/8] rename byte_order to byteorder --- specs/simple-serialize.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index f69544ffa..5edaa6e10 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -48,10 +48,10 @@ overhead. | Term | Definition | |:-------------|:-----------------------------------------------------------------------------------------------| | `little` | Little endian. | -| `byte_order` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. | +| `byteorder` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. | | `len` | Length/number of bytes. | -| `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byte_order``. | -| `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byte_order``. | +| `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byteorder``. | +| `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byteorder``. | | `value` | The value to serialize. | | `rawbytes` | Raw serialized bytes. | | `deserialized_object` | The deserialized data in the data structure of your programming language. | From 45c064a2d6cbba2fb284b0e8fae7d83742dab31c Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:24:05 +0800 Subject: [PATCH 8/8] remove all unnecessary newline --- specs/simple-serialize.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 5edaa6e10..150c71b62 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -57,7 +57,6 @@ overhead. | `deserialized_object` | The deserialized data in the data structure of your programming language. | | `new_index` | An index to keep track the latest position where the `rawbytes` have been deserialized. | - ## Constants | Constant | Value | Definition | @@ -75,7 +74,6 @@ overhead. |:---------:|:-----------------------------------------------------------| | `uintN` | Type of `N` bits unsigned integer, where ``N % 8 == 0``. | - Convert directly to bytes the size of the int. (e.g. ``uint16 = 2 bytes``) All integers are serialized as **little endian**. @@ -145,7 +143,6 @@ Lists are a collection of elements of the same homogeneous type. |:--------------------------------------------|:----------------------------| | Length of serialized list fits into 4 bytes | ``len(serialized) < 2**32`` | - 1. Get the number of raw bytes to serialize: it is ``len(list) * sizeof(element)``. * Encode that as a `4-byte` **little endian** `uint32`. 2. Append the elements in a packed manner. @@ -174,7 +171,6 @@ A container represents a heterogenous, associative collection of key-value pairs To serialize a container, obtain the list of its field's names in the specified order. For each field name in this list, obtain the corresponding value and serialize it. Tightly pack the complete set of serialized values in the same order as the field names into a buffer. Calculate the size of this buffer of serialized bytes and encode as a `4-byte` **little endian** `uint32`. Prepend the encoded length to the buffer. The result of this concatenation is the final serialized value of the container. - | Check to perform | Code | |:--------------------------------------------|:----------------------------| | Length of serialized fields fits into 4 bytes | ``len(serialized) < 2**32`` | @@ -238,7 +234,6 @@ At the final step, the following checks should be made: |:-------------------------|:-------------------------------------| | Ensure no extra length | `new_index == len(rawbytes)` | - #### uint Convert directly from bytes into integer utilising the number of bytes the same @@ -453,6 +448,5 @@ return hash(b''.join([hash_tree_root(getattr(x, field)) for field in value.field | Go | [ https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz ](https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz) | Go implementation of SSZ mantained by Prysmatic Labs | | Swift | [ https://github.com/yeeth/SimpleSerialize.swift ](https://github.com/yeeth/SimpleSerialize.swift) | Swift implementation maintained SSZ | - ## Copyright Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).