Retry logic for getBlobsV2 in peerDAS (#15520)

* PeerDAS: Implement sync

* Fix Potuz's comment.

* Fix Potuz's comment.

* Fix Potuz's comment.

* Fix Potuz's comment.

* Fix Potuz's comment.

* Implement `TestFetchDataColumnSidecarsFromPeers`.

* Implement `TestSelectPeers`.

* Fix James' comment.

* Fix flakiness in `TestSelectPeers`.

* Revert "Fix Potuz's comment."

This reverts commit c45230b455.

* Revert "Fix James' comment."

This reverts commit a3f919205a.

* `selectPeers`: Avoid map with key but empty value.

* Fix Potuz's comment.

* Add DataColumnStorage and SubscribeAllDataSubnets flag.

* getBlobsV2: retry if reconstruction isnt successful

* test: engine client and sync package, metrics

* lint: fmt and log capitalisation

* lint: return error when it is not nil

* config: make retry interval configurable

* sidecar: recover function and different context for retrying

* lint: remove unused field

* beacon: default retry interval

* reconstruct: load once, correctly deliver the result to all waiting goroutines

* reconstruct: simplify multi goroutine case and avoid race condition

* engine: remove isDataAlreadyAvailable function

* sync: no goroutine, getblobsv2 in absence of block as well, wrap error

* exec: hardcode retry interval

* da: non blocking checks

* sync: remove unwanted checks

* execution: fix test

* execution: retry atomicity test

* da: updated IsDataAvailable

* sync: remove unwanted tests

* bazel: bazel run //:gazelle -- fix

* blockchain: fix CustodyGroupCount return

* lint: formatting

* lint: lint and use unused metrics

* execution: retry logic inside ReconstructDataColumnSidecars itself

* lint: format

* execution: ensure the retry actually happens when it needs to

* execution: ensure single responsibility, execution should not do DA check

* sync: don't call ReconstructDataColumnSidecars if not required

* blockchain: move IsDataAvailable interface to blockchain package

* execution: make reconstructSingleflight part of the service struct

* blockchain: cleaner DA check

* lint: formatting and remove confusing comment

* sync: fix lint, test and add extra test for when data is actually not available

* sync: new appropriate mock service

* execution: edge case - delete activeRetries on success

* execution: use service context instead of function's for retry

* blockchain: get variable samplesPerSlot only when required

* remove redundant function and fix name

* fix test

* fix more tests

* put samplesPerSlot at appropriate place

* tidy up IsDataAvailable

* correct bad merge

* fix bad merge

* remove redundant flag option

* refactor to deduplicate sidecar construction code

* - Add godocs
- Rename some functions to be closer to the spec
- Add err in return of commitments

* Replace mutating public method (but only internally used) `Populate` but private not mutating method `extract`.

* Implement a unique `processDataColumnSidecarsFromExecution` instead 2 separate functions from block and from sidecar.

* `ReceiveBlock`: Wrap errors.

* Remove useless tests.

* `ConstructionPopulator`: Add tests.

* Fix tests

* Move functions to be consistent with blobs.

* `fetchCellsAndProofsFromExecution`: Avoid useless flattening.

* `processDataColumnSidecarsFromExecution`: Stop using DB cache.

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>

specrefs/.ethspecify.yml

@@ -283,6 +283,7 @@ exceptions:
     - compute_fork_digest#fulu
     - compute_matrix#fulu
     - get_blob_parameters#fulu
+    - get_data_column_sidecars_from_block#fulu
     - get_data_column_sidecars_from_column_sidecar#fulu
     - get_extended_sample_count#fulu
     - recover_matrix#fulu

specrefs/functions.yml

@@ -2076,7 +2076,7 @@
 
 - name: get_data_column_sidecars
   sources:
-    - file: beacon-chain/core/peerdas/das_core.go
+    - file: beacon-chain/core/peerdas/validator.go
       search: func DataColumnSidecars(
   spec: |
     <spec fn="get_data_column_sidecars" fork="fulu" hash="317fc596">
@@ -2113,60 +2113,6 @@
         return sidecars
     </spec>
 
-- name: get_data_column_sidecars_from_block
-  sources:
-    - file: beacon-chain/core/peerdas/das_core.go
-      search: func dataColumnsSidecars(
-  spec: |
-    <spec fn="get_data_column_sidecars_from_block" fork="fulu" hash="02ffae23">
-    def get_data_column_sidecars_from_block(
-        signed_block: SignedBeaconBlock,
-        cells_and_kzg_proofs: Sequence[
-            Tuple[Vector[Cell, CELLS_PER_EXT_BLOB], Vector[KZGProof, CELLS_PER_EXT_BLOB]]
-        ],
-    ) -> Sequence[DataColumnSidecar]:
-        """
-        Given a signed block and the cells/proofs associated with each blob in the
-        block, assemble the sidecars which can be distributed to peers.
-        """
-        blob_kzg_commitments = signed_block.message.body.blob_kzg_commitments
-        signed_block_header = compute_signed_block_header(signed_block)
-        kzg_commitments_inclusion_proof = compute_merkle_proof(
-            signed_block.message.body,
-            get_generalized_index(BeaconBlockBody, "blob_kzg_commitments"),
-        )
-        return get_data_column_sidecars(
-            signed_block_header,
-            blob_kzg_commitments,
-            kzg_commitments_inclusion_proof,
-            cells_and_kzg_proofs,
-        )
-    </spec>
-
-- name: get_data_column_sidecars_from_column_sidecar
-  sources: []
-  spec: |
-    <spec fn="get_data_column_sidecars_from_column_sidecar" fork="fulu" hash="4304cdec">
-    def get_data_column_sidecars_from_column_sidecar(
-        sidecar: DataColumnSidecar,
-        cells_and_kzg_proofs: Sequence[
-            Tuple[Vector[Cell, CELLS_PER_EXT_BLOB], Vector[KZGProof, CELLS_PER_EXT_BLOB]]
-        ],
-    ) -> Sequence[DataColumnSidecar]:
-        """
-        Given a DataColumnSidecar and the cells/proofs associated with each blob corresponding
-        to the commitments it contains, assemble all sidecars for distribution to peers.
-        """
-        assert len(cells_and_kzg_proofs) == len(sidecar.kzg_commitments)
-
-        return get_data_column_sidecars(
-            sidecar.signed_block_header,
-            sidecar.kzg_commitments,
-            sidecar.kzg_commitments_inclusion_proof,
-            cells_and_kzg_proofs,
-        )
-    </spec>
-
 - name: get_domain
   sources:
     - file: beacon-chain/core/signing/domain.go