Rendering those line numbers in the gutter isn't useful, and it puts
unneeded pressure to the DOM. In the process of changing
`updateLineNumbersState`, we have also refactored it to stop relying on
row ranges being contiguous. This allows that code path to be:
1. Less error-prone, because we were trying to access rows that weren't
actually rendered, thus potentially throwing errors when measuring
non-contiguous screen rows that weren't visible.
2. Tighter, because we can just iterate over each screen row and ask for
its soft-wrap descriptor.
...because the only possible scenario when a logical position in a text
node cannot be found is when the requested pixel position is exactly at
the end of the node.
This fixes an annoying problem that prevented the state of marker layers
from being saved when the window was reloaded either via `Cmd+R` in
DevTools or via `Ctrl+Option+Cmd+L` in Atom.
The issue was that we were *always* scheduling `saveState` on an idle
callback: `window.onbeforeunload`, however, doesn't wait for that event
before closing the window, and thus that state was never saved in those
situations.
The solution is to use idle callbacks only during the critical code path
(i.e. on mousedown and keydown), but save it synchronously otherwise.
Saving something to IndexedDB is actually asynchronous too, but it seems
like Chrome fulfills `put` requests that get executed right during
`onbeforeunload`.
Sometimes, when performing an edit, a change on some row can cause
another row's tokenization to be affected: the classic example is
opening a multi-line comment on a line, thereby causing subsequent lines
to become commented out without changing the buffer's contents at those
locations. We call this technique "spill detection".
Since the amount of affected lines can grow quite large, Atom tokenizes
synchronously only those lines where the edit occurred, triggering
background (i.e. `setInterval`) tokenization for all the other lines
that need to be refreshed because of a "spill".
As predictable, this approach causes a temporary inconsistency in the
stored tokenized lines. In particular, suppose we had two tokenized
lines, and that there's an open tag in the middle of the first one which
closes on the second one. If we perform an edit that causes that tag to
be deleted, when reading the second tokenized line we now have a
dangling close tag.
This didn't matter much in the `DisplayBuffer` version, because for each
line we reopened all the tags found in the stored `openScopes` property,
and closed all the tags starting on such line right at the end of it.
In the `DisplayLayer` world, however, we don't read tags from each
tokenized line, but we let `TokenizedBufferIterator` report tag
boundaries and their respective location: since this is an
iterator-based approach, we were not reading `openScopes` for each
`TokenizedLine`, making the dangling close tag example showed above
evident (e.g. close and open tags didn't match anymore, and exceptions
were being thrown all over the place).
To solve this issue I have considered several approaches:
1. Recompute all the lines where a spill occurs synchronously when the
buffer changes. For large files, this can be pretty onerous, and we
don't want to regress in terms of performance.
2. Let `TokenizedBuffer.tokenizedLineForRow(bufferRow)` recompute
potential invalid lines lazily (starting from the first invalid line,
down to the requested buffer row). When editing the first lines of a
long file and causing a spill to occur, Atom (or any other package,
for that matter) could request a line down in the file, causing this
method to recompute lots and lots of lines.
3. Let `DisplayLayer` deal with closing an un-opened tag. This is nice
because we already keep track of containing tags there. However, it
also feels like the wrong spot where to put this logic, as display
layers shouldn't deal with grammar-related stuff.
4. Keep track of containing tags in `TokenizedBufferIterator`, and
report a boundary at the end of the line when the subsequent one's
`openScopes` property doesn't match the `containingTags` that the
iterator has been keeping track of.
Of all these solutions I've chosen 4), because it's the most performant
and clean in terms of code.
