Files
linea-besu/errorprone-checks
Ratan (Rai) Sur a25d3f17e4 Introduce Parallel Marking for Pruning (#1219)
The parallel task production is by sub-trie, so calling `visitAll` on a 
root node will eventually spawn up to 16 tasks (for a hexary trie).

If we marked each sub-trie in its own thread, with no common queue of 
tasks, our mark speed would be limited by the sub-trie with the maximum 
number of nodes. In practice for the Ethereum mainnet, we see a large 
imbalance in sub-trie size so without a common task pool the time in 
which there is only 1 thread left marking its big sub-trie would be 
substantial.

If we were to leave all threads to produce mark tasks before starting 
to mark, we would run out of memory quickly.

If we were to have a constant number of threads producing the mark 
tasks with the others consuming them, we would have to optimize the 
production/consumption balance.

To get the best of both worlds, the marking executor has a 
ThreadPoolExecutor.CallerRunsPolicy which causes the producing tasks to 
essentially consume their own mark task immediately when the task queue 
is full. The resulting behavior is threads that mark their own sub-trie 
until they finish that sub-trie, at which point they switch to marking 
the sub-trie tasks produced by another thread.

Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
2020-10-23 13:14:43 -04:00
..

The creation of custom errorprone checkers was largely derived from:

To allow for debugging from within intellij, the following must be added to the VM args in the run/debug configuration (this assumes your gradle cache is at the default location under your home): -Xbootclasspath/p:${HOME}/.gradle/caches/./modules-2/files-2.1/com.google.errorprone/javac/9+181-r4173-1/bdf4c0aa7d540ee1f7bf14d47447aea4bbf450c5/javac-9+181-r4173-1.jar