github/coffeescript
1
1
Fork 0
mirror of https://github.com/jashkenas/coffeescript.git synced 2026-01-15 01:38:13 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
cbf035fca9f5beebe4a90794eb061fe75cfd9601
coffeescript/docs/v2/annotated-source/rewriter.html
Geoffrey Booth 704924c3cd 2.0.0-beta3 (#4594) 
* Don’t confuse the syntax highlighter

* Comment Assign::compilePatternMatch a bit

* Assignment expressions in conditionals are a bad practice

* Rename `wrapInBraces` to `wrapInParentheses`, to set the stage for future `wrapInBraces` that uses `{` and `wrapInBrackets` that uses `[`

* Correct comment

* object destructuring

* Allow custom position of the rest element.

* Output simple array destructuring assignments to ES2015

* Output simple object destructured assignments to ES2015

* Compile shorthand object properties to ES2015 shorthand properties

This dramatically improves the appearance of destructured imports.

* Don’t confuse the syntax highlighter

* Comment Assign::compilePatternMatch a bit

* Assignment expressions in conditionals are a bad practice

* Rename `wrapInBraces` to `wrapInParentheses`, to set the stage for future `wrapInBraces` that uses `{` and `wrapInBrackets` that uses `[`

* object destructuring

* Allow custom position of the rest element.

* rest element in object destructuring

* rest element in object destructuring

* fix string interpolation

* merging

* fixing splats in object literal

* Rest element in parameter destructuring

* merging with CS2

* merged with CS2

* Add support for the object spread initializer. https://github.com/tc39/proposal-object-rest-spread/blob/master/Spread.md

* Fix misspellings, trailing whitespace, other minor details

* merging with beta2

* refactor object spread properties

* small fix

* - Fixed object spread function parameters.
- Clean up "Assign" and moved all logic for object rest properties in single method (compileObjectDestruct).
- Add helper function "objectWithoutKeys" to the "UTILITIES" for use with object rest properties,
  e.g. {a, b, r...} = obj => {a, b} = obj, r = objectWithoutKeys(...)
- Clean up "Obj" and moved all logic for object spread properties in single method (compileSpread).
- Clean up "Code".
- Add method "hasSplat" to "Obj" and "Value" for checking if Obj contains the splat.
- Enable placing spread syntax triple dots on either right or left, per #85 (https://github.com/coffeescript6/discuss/issues/85)

* Fixed typos

* Remove unused code

* Removed dots (e.g. splat) on the left side from the grammar

* Initial release for deep spread properties, e.g. obj2 = {obj.b..., a: 1} or {obj[b][c]..., d: 7}
Tests need to be prepared!

* 1. Object literal spread properties

Object literals:
- obj = { {b:{c:{d:1}}}..., a:1 }

Parenthetical:
- obj = { ( body ), a:1 }
- obj = { ( body )..., a:1 }

Invocation:
- obj = { ( (args) -> ... )(params), a:1 }
- obj = { ( (args) -> ... )(params)..., a:1 }
- obj = { foo(), a:1 }
- obj = { foo()..., a:1 }

2. Refactor, cleanup & other optimizations.

* Merged with 2.0

* Cleanup

* Some more cleanup.

* Fixed error with freeVariable and object destructuring.

* Fixed errors with object spread properties.

* Improvements, fixed errors.

* Minor improvement.

* Minor improvements.

* Typo.

* Remove unnecessary whitespace.

* Remove unnecessary whitespace.

* Changed few "assertErrorFormat" tests since parentheses are now allowed in the Obj.

* Whitespace cleanup

* Comments cleanup

* fix destructured obj param declarations

* refine fix; add test

* Refactor function args ({a, b...})

* Additional tests for object destructuring in function argument.

* Minor improvement for object destructuring variable declaration.

* refactor function args ({a, b...}) and ({a, b...} = {}); Obj And Param cleanup

* fix comment

* Fix object destructuring variable declaration.

* more tests with default values

* fix typo

* Fixed default values in object destructuring.

* small fix

* Babel’s tests for object rest spread

* Style: spaces after colons in object declarations

* Cleanup comments

* Simplify Babel tests

* Fix comments

* Fix destructuring with splats in multiple objects

* Add test for default values in detsructuring assignment with splats

* Handle default values when assigning to object splats

* Rewrite traverseRest to fix handling of dynamic keys

* Fix double parens around destructuring with splats

* Update compileObjectDestruct comments

* Improve formatting of top-level destructures with splats and tidy parens

* Added a bigger destructuring-with-defaults test and fixed a bug

* Refactor destructuring grammar to allow additional forms

* Add a missing case to ObjSpreadExpr

* These tests shouldn’t run in the browser

* Fix test.html

* Fix docs scroll position getting screwed up by CodeMirror initialization

* Breaking change documentation about => (fixes #4593)

* Spread/rest syntax documentation

* Documentation about bound class methods

* 2.0.0-beta3 changelog

* Add note about ‘lib’

* Fix accidentally converting this to tabs

* Bump version to 2.0.0-beta3

* Update annotated source and test.html
2017-06-30 09:58:05 -07:00

1296 lines
70 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html>
<head>
<title>rewriter.coffee</title>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, target-densitydpi=160dpi, initial-scale=1.0; maximum-scale=1.0; user-scalable=0;">
<link rel="stylesheet" media="all" href="docco.css" />
</head>
<body>
<div id="container">
<div id="background"></div>
<ul id="jump_to">
<li>
<a class="large" href="javascript:void(0);">Jump To &hellip;</a>
<a class="small" href="javascript:void(0);">+</a>
<div id="jump_wrapper">
<div id="jump_page_wrapper">
<div id="jump_page">
<a class="source" href="browser.html">
browser.coffee
</a>
<a class="source" href="cake.html">
cake.coffee
</a>
<a class="source" href="coffeescript.html">
coffeescript.coffee
</a>
<a class="source" href="command.html">
command.coffee
</a>
<a class="source" href="grammar.html">
grammar.coffee
</a>
<a class="source" href="helpers.html">
helpers.coffee
</a>
<a class="source" href="index.html">
index.coffee
</a>
<a class="source" href="lexer.html">
lexer.coffee
</a>
<a class="source" href="nodes.html">
nodes.coffee
</a>
<a class="source" href="optparse.html">
optparse.coffee
</a>
<a class="source" href="register.html">
register.coffee
</a>
<a class="source" href="repl.html">
repl.coffee
</a>
<a class="source" href="rewriter.html">
rewriter.coffee
</a>
<a class="source" href="scope.html">
scope.litcoffee
</a>
<a class="source" href="sourcemap.html">
sourcemap.litcoffee
</a>
</div>
</div>
</li>
</ul>
<ul class="sections">
<li id="title">
<div class="annotation">
<h1>rewriter.coffee</h1>
</div>
</li>
<li id="section-1">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-1">&#182;</a>
</div>
<p>The CoffeeScript language has a good deal of optional syntax, implicit syntax,
and shorthand syntax. This can greatly complicate a grammar and bloat
the resulting parse table. Instead of making the parser handle it all, we take
a series of passes over the token stream, using this <strong>Rewriter</strong> to convert
shorthand into the unambiguous long form, add implicit indentation and
parentheses, and generally clean things up.</p>
</div>
<div class="content"><div class='highlight'><pre>
{throwSyntaxError} = <span class="hljs-built_in">require</span> <span class="hljs-string">'./helpers'</span></pre></div></div>
</li>
<li id="section-2">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-2">&#182;</a>
</div>
<p>Create a generated token: one that exists due to a use of implicit syntax.</p>
</div>
<div class="content"><div class='highlight'><pre><span class="hljs-function"><span class="hljs-title">generate</span> = <span class="hljs-params">(tag, value, origin)</span> -&gt;</span>
tok = [tag, value]
tok.generated = <span class="hljs-literal">yes</span>
tok.origin = origin <span class="hljs-keyword">if</span> origin
tok</pre></div></div>
</li>
<li id="section-3">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-3">&#182;</a>
</div>
<p>The <strong>Rewriter</strong> class is used by the <a href="lexer.html">Lexer</a>, directly against
its internal array of tokens.</p>
</div>
<div class="content"><div class='highlight'><pre>exports.Rewriter = <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Rewriter</span></span></pre></div></div>
</li>
<li id="section-4">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-4">&#182;</a>
</div>
<p>Rewrite the token stream in multiple passes, one logical filter at
a time. This could certainly be changed into a single pass through the
stream, with a big ol’ efficient switch, but it’s much nicer to work with
like this. The order of these passes matters – indentation must be
corrected before implicit parentheses can be wrapped around blocks of code.</p>
</div>
<div class="content"><div class='highlight'><pre> rewrite: <span class="hljs-function"><span class="hljs-params">(@tokens)</span> -&gt;</span></pre></div></div>
</li>
<li id="section-5">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-5">&#182;</a>
</div>
<p>Helpful snippet for debugging:
console.log (t[0] + ‘/‘ + t[1] for t in @tokens).join ‘ ‘</p>
</div>
<div class="content"><div class='highlight'><pre> @removeLeadingNewlines()
@closeOpenCalls()
@closeOpenIndexes()
@normalizeLines()
@tagPostfixConditionals()
@addImplicitBracesAndParens()
@addLocationDataToGeneratedTokens()
@enforceValidCSXAttributes()
@fixOutdentLocationData()
@tokens</pre></div></div>
</li>
<li id="section-6">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-6">&#182;</a>
</div>
<p>Rewrite the token stream, looking one token ahead and behind.
Allow the return value of the block to tell us how many tokens to move
forwards (or backwards) in the stream, to make sure we don’t miss anything
as tokens are inserted and removed, and the stream changes length under
our feet.</p>
</div>
<div class="content"><div class='highlight'><pre> scanTokens: <span class="hljs-function"><span class="hljs-params">(block)</span> -&gt;</span>
{tokens} = <span class="hljs-keyword">this</span>
i = <span class="hljs-number">0</span>
i += block.call <span class="hljs-keyword">this</span>, token, i, tokens <span class="hljs-keyword">while</span> token = tokens[i]
<span class="hljs-literal">true</span>
detectEnd: <span class="hljs-function"><span class="hljs-params">(i, condition, action, opts = {})</span> -&gt;</span>
{tokens} = <span class="hljs-keyword">this</span>
levels = <span class="hljs-number">0</span>
<span class="hljs-keyword">while</span> token = tokens[i]
<span class="hljs-keyword">return</span> action.call <span class="hljs-keyword">this</span>, token, i <span class="hljs-keyword">if</span> levels <span class="hljs-keyword">is</span> <span class="hljs-number">0</span> <span class="hljs-keyword">and</span> condition.call <span class="hljs-keyword">this</span>, token, i
<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> EXPRESSION_START
levels += <span class="hljs-number">1</span>
<span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> EXPRESSION_END
levels -= <span class="hljs-number">1</span>
<span class="hljs-keyword">if</span> levels &lt; <span class="hljs-number">0</span>
<span class="hljs-keyword">return</span> <span class="hljs-keyword">if</span> opts.returnOnNegativeLevel
<span class="hljs-keyword">return</span> action.call <span class="hljs-keyword">this</span>, token, i
i += <span class="hljs-number">1</span>
i - <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-7">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-7">&#182;</a>
</div>
<p>Leading newlines would introduce an ambiguity in the grammar, so we
dispatch them here.</p>
</div>
<div class="content"><div class='highlight'><pre> removeLeadingNewlines: <span class="hljs-function">-&gt;</span>
<span class="hljs-keyword">break</span> <span class="hljs-keyword">for</span> [tag], i <span class="hljs-keyword">in</span> @tokens <span class="hljs-keyword">when</span> tag <span class="hljs-keyword">isnt</span> <span class="hljs-string">'TERMINATOR'</span>
@tokens.splice <span class="hljs-number">0</span>, i <span class="hljs-keyword">if</span> i</pre></div></div>
</li>
<li id="section-8">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-8">&#182;</a>
</div>
<p>The lexer has tagged the opening parenthesis of a method call. Match it with
its paired close.</p>
</div>
<div class="content"><div class='highlight'><pre> closeOpenCalls: <span class="hljs-function">-&gt;</span>
<span class="hljs-function"> <span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">')'</span>, <span class="hljs-string">'CALL_END'</span>]
<span class="hljs-function">
<span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
token[<span class="hljs-number">0</span>] = <span class="hljs-string">'CALL_END'</span>
@scanTokens (token, i) -&gt;
@detectEnd i + <span class="hljs-number">1</span>, condition, action <span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'CALL_START'</span>
<span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-9">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-9">&#182;</a>
</div>
<p>The lexer has tagged the opening bracket of an indexing operation call.
Match it with its paired close.</p>
</div>
<div class="content"><div class='highlight'><pre> closeOpenIndexes: <span class="hljs-function">-&gt;</span>
<span class="hljs-function"> <span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">']'</span>, <span class="hljs-string">'INDEX_END'</span>]
<span class="hljs-function">
<span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
token[<span class="hljs-number">0</span>] = <span class="hljs-string">'INDEX_END'</span>
@scanTokens (token, i) -&gt;
@detectEnd i + <span class="hljs-number">1</span>, condition, action <span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'INDEX_START'</span>
<span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-10">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-10">&#182;</a>
</div>
<p>Match tags in token stream starting at <code>i</code> with <code>pattern</code>, skipping ‘HERECOMMENT’s.
<code>pattern</code> may consist of strings (equality), an array of strings (one of)
or null (wildcard). Returns the index of the match or -1 if no match.</p>
</div>
<div class="content"><div class='highlight'><pre> indexOfTag: <span class="hljs-function"><span class="hljs-params">(i, pattern...)</span> -&gt;</span>
fuzz = <span class="hljs-number">0</span>
<span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> [<span class="hljs-number">0</span> ... pattern.length]
fuzz += <span class="hljs-number">2</span> <span class="hljs-keyword">while</span> @tag(i + j + fuzz) <span class="hljs-keyword">is</span> <span class="hljs-string">'HERECOMMENT'</span>
<span class="hljs-keyword">continue</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> pattern[j]?
pattern[j] = [pattern[j]] <span class="hljs-keyword">if</span> <span class="hljs-keyword">typeof</span> pattern[j] <span class="hljs-keyword">is</span> <span class="hljs-string">'string'</span>
<span class="hljs-keyword">return</span> <span class="hljs-number">-1</span> <span class="hljs-keyword">if</span> @tag(i + j + fuzz) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> pattern[j]
i + j + fuzz - <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-11">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-11">&#182;</a>
</div>
<p>Returns <code>yes</code> if standing in front of something looking like
<code>@&lt;x&gt;:</code>, <code>&lt;x&gt;:</code> or <code>&lt;EXPRESSION_START&gt;&lt;x&gt;...&lt;EXPRESSION_END&gt;:</code>,
skipping over ‘HERECOMMENT’s.</p>
</div>
<div class="content"><div class='highlight'><pre> looksObjectish: <span class="hljs-function"><span class="hljs-params">(j)</span> -&gt;</span>
<span class="hljs-keyword">return</span> <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> @indexOfTag(j, <span class="hljs-string">'@'</span>, <span class="hljs-literal">null</span>, <span class="hljs-string">':'</span>) &gt; <span class="hljs-number">-1</span> <span class="hljs-keyword">or</span> @indexOfTag(j, <span class="hljs-literal">null</span>, <span class="hljs-string">':'</span>) &gt; <span class="hljs-number">-1</span>
index = @indexOfTag(j, EXPRESSION_START)
<span class="hljs-keyword">if</span> index &gt; <span class="hljs-number">-1</span>
end = <span class="hljs-literal">null</span>
@detectEnd index + <span class="hljs-number">1</span>, <span class="hljs-function">(<span class="hljs-params">(token)</span> -&gt;</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> EXPRESSION_END), <span class="hljs-function">(<span class="hljs-params">(token, i)</span> -&gt;</span> end = i)
<span class="hljs-keyword">return</span> <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> @tag(end + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">':'</span>
<span class="hljs-literal">no</span></pre></div></div>
</li>
<li id="section-12">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-12">&#182;</a>
</div>
<p>Returns <code>yes</code> if current line of tokens contain an element of tags on same
expression level. Stop searching at LINEBREAKS or explicit start of
containing balanced expression.</p>
</div>
<div class="content"><div class='highlight'><pre> findTagsBackwards: <span class="hljs-function"><span class="hljs-params">(i, tags)</span> -&gt;</span>
backStack = []
<span class="hljs-keyword">while</span> i &gt;= <span class="hljs-number">0</span> <span class="hljs-keyword">and</span> (backStack.length <span class="hljs-keyword">or</span>
@tag(i) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> tags <span class="hljs-keyword">and</span>
(@tag(i) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> EXPRESSION_START <span class="hljs-keyword">or</span> @tokens[i].generated) <span class="hljs-keyword">and</span>
@tag(i) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> LINEBREAKS)
backStack.push @tag(i) <span class="hljs-keyword">if</span> @tag(i) <span class="hljs-keyword">in</span> EXPRESSION_END
backStack.pop() <span class="hljs-keyword">if</span> @tag(i) <span class="hljs-keyword">in</span> EXPRESSION_START <span class="hljs-keyword">and</span> backStack.length
i -= <span class="hljs-number">1</span>
@tag(i) <span class="hljs-keyword">in</span> tags</pre></div></div>
</li>
<li id="section-13">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-13">&#182;</a>
</div>
<p>Look for signs of implicit calls and objects in the token stream and
add them.</p>
</div>
<div class="content"><div class='highlight'><pre> addImplicitBracesAndParens: <span class="hljs-function">-&gt;</span></pre></div></div>
</li>
<li id="section-14">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-14">&#182;</a>
</div>
<p>Track current balancing depth (both implicit and explicit) on stack.</p>
</div>
<div class="content"><div class='highlight'><pre> stack = []
start = <span class="hljs-literal">null</span>
@scanTokens (token, i, tokens) -&gt;
[tag] = token
[prevTag] = prevToken = <span class="hljs-keyword">if</span> i &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> tokens[i - <span class="hljs-number">1</span>] <span class="hljs-keyword">else</span> []
[nextTag] = nextToken = <span class="hljs-keyword">if</span> i &lt; tokens.length - <span class="hljs-number">1</span> <span class="hljs-keyword">then</span> tokens[i + <span class="hljs-number">1</span>] <span class="hljs-keyword">else</span> []
<span class="hljs-function"> <span class="hljs-title">stackTop</span> = -&gt;</span> stack[stack.length - <span class="hljs-number">1</span>]
startIdx = i</pre></div></div>
</li>
<li id="section-15">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-15">&#182;</a>
</div>
<p>Helper function, used for keeping track of the number of tokens consumed
and spliced, when returning for getting a new token.</p>
</div>
<div class="content"><div class='highlight'><pre><span class="hljs-function"> <span class="hljs-title">forward</span> = <span class="hljs-params">(n)</span> -&gt;</span> i - startIdx + n</pre></div></div>
</li>
<li id="section-16">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-16">&#182;</a>
</div>
<p>Helper functions</p>
</div>
<div class="content"><div class='highlight'><pre><span class="hljs-function"> <span class="hljs-title">isImplicit</span> = <span class="hljs-params">(stackItem)</span> -&gt;</span> stackItem?[<span class="hljs-number">2</span>]?.ours
<span class="hljs-function"> <span class="hljs-title">isImplicitObject</span> = <span class="hljs-params">(stackItem)</span> -&gt;</span> isImplicit(stackItem) <span class="hljs-keyword">and</span> stackItem?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span>
<span class="hljs-function"> <span class="hljs-title">isImplicitCall</span> = <span class="hljs-params">(stackItem)</span> -&gt;</span> isImplicit(stackItem) <span class="hljs-keyword">and</span> stackItem?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'('</span>
<span class="hljs-function"> <span class="hljs-title">inImplicit</span> = -&gt;</span> isImplicit stackTop()
<span class="hljs-function"> <span class="hljs-title">inImplicitCall</span> = -&gt;</span> isImplicitCall stackTop()
<span class="hljs-function"> <span class="hljs-title">inImplicitObject</span> = -&gt;</span> isImplicitObject stackTop()</pre></div></div>
</li>
<li id="section-17">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-17">&#182;</a>
</div>
<p>Unclosed control statement inside implicit parens (like
class declaration or if-conditionals)</p>
</div>
<div class="content"><div class='highlight'><pre><span class="hljs-function"> <span class="hljs-title">inImplicitControl</span> = -&gt;</span> inImplicit() <span class="hljs-keyword">and</span> stackTop()?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'CONTROL'</span>
<span class="hljs-function">
<span class="hljs-title">startImplicitCall</span> = <span class="hljs-params">(idx)</span> -&gt;</span>
stack.push [<span class="hljs-string">'('</span>, idx, ours: <span class="hljs-literal">yes</span>]
tokens.splice idx, <span class="hljs-number">0</span>, generate <span class="hljs-string">'CALL_START'</span>, <span class="hljs-string">'('</span>
<span class="hljs-function">
<span class="hljs-title">endImplicitCall</span> = -&gt;</span>
stack.pop()
tokens.splice i, <span class="hljs-number">0</span>, generate <span class="hljs-string">'CALL_END'</span>, <span class="hljs-string">')'</span>, [<span class="hljs-string">''</span>, <span class="hljs-string">'end of input'</span>, token[<span class="hljs-number">2</span>]]
i += <span class="hljs-number">1</span>
<span class="hljs-function">
<span class="hljs-title">startImplicitObject</span> = <span class="hljs-params">(idx, startsLine = <span class="hljs-literal">yes</span>)</span> -&gt;</span>
stack.push [<span class="hljs-string">'{'</span>, idx, sameLine: <span class="hljs-literal">yes</span>, startsLine: startsLine, ours: <span class="hljs-literal">yes</span>]
val = <span class="hljs-keyword">new</span> String <span class="hljs-string">'{'</span>
val.generated = <span class="hljs-literal">yes</span>
tokens.splice idx, <span class="hljs-number">0</span>, generate <span class="hljs-string">'{'</span>, val, token
<span class="hljs-function">
<span class="hljs-title">endImplicitObject</span> = <span class="hljs-params">(j)</span> -&gt;</span>
j = j ? i
stack.pop()
tokens.splice j, <span class="hljs-number">0</span>, generate <span class="hljs-string">'}'</span>, <span class="hljs-string">'}'</span>, token
i += <span class="hljs-number">1</span>
<span class="hljs-function">
<span class="hljs-title">implicitObjectContinues</span> = <span class="hljs-params">(j)</span> =&gt;</span>
nextTerminatorIdx = <span class="hljs-literal">null</span>
@detectEnd j,
<span class="hljs-function"><span class="hljs-params">(token)</span> -&gt;</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span>
(token, i) -&gt; nextTerminatorIdx = i
returnOnNegativeLevel: <span class="hljs-literal">yes</span>
<span class="hljs-keyword">return</span> <span class="hljs-literal">no</span> <span class="hljs-keyword">unless</span> nextTerminatorIdx?
@looksObjectish nextTerminatorIdx + <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-18">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-18">&#182;</a>
</div>
<p>Don’t end an implicit call/object on next indent if any of these are in an argument/value</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> (
(inImplicitCall() <span class="hljs-keyword">or</span> inImplicitObject()) <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">in</span> CONTROL_IN_IMPLICIT <span class="hljs-keyword">or</span>
inImplicitObject() <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">is</span> <span class="hljs-string">':'</span> <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'FOR'</span>
)
stack.push [<span class="hljs-string">'CONTROL'</span>, i, ours: <span class="hljs-literal">yes</span>]
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span> inImplicit()</pre></div></div>
</li>
<li id="section-19">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-19">&#182;</a>
</div>
<p>An <code>INDENT</code> closes an implicit call unless</p>
<ol>
<li>We have seen a <code>CONTROL</code> argument on the line.</li>
<li>The last token before the indent is part of the list below</li>
</ol>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> prevTag <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> [<span class="hljs-string">'=&gt;'</span>, <span class="hljs-string">'-&gt;'</span>, <span class="hljs-string">'['</span>, <span class="hljs-string">'('</span>, <span class="hljs-string">','</span>, <span class="hljs-string">'{'</span>, <span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'='</span>]
<span class="hljs-keyword">while</span> inImplicitCall() <span class="hljs-keyword">or</span> inImplicitObject() <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">':'</span>
<span class="hljs-keyword">if</span> inImplicitCall()
endImplicitCall()
<span class="hljs-keyword">else</span>
endImplicitObject()
stack.pop() <span class="hljs-keyword">if</span> inImplicitControl()
stack.push [tag, i]
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)</pre></div></div>
</li>
<li id="section-20">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-20">&#182;</a>
</div>
<p>Straightforward start of explicit expression</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> EXPRESSION_START
stack.push [tag, i]
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)</pre></div></div>
</li>
<li id="section-21">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-21">&#182;</a>
</div>
<p>Close all implicit expressions inside of explicitly closed expressions.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> EXPRESSION_END
<span class="hljs-keyword">while</span> inImplicit()
<span class="hljs-keyword">if</span> inImplicitCall()
endImplicitCall()
<span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> inImplicitObject()
endImplicitObject()
<span class="hljs-keyword">else</span>
stack.pop()
start = stack.pop()</pre></div></div>
</li>
<li id="section-22">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-22">&#182;</a>
</div>
<p>Recognize standard implicit calls like
f a, f() b, f? c, h[0] d etc.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> (tag <span class="hljs-keyword">in</span> IMPLICIT_FUNC <span class="hljs-keyword">and</span> token.spaced <span class="hljs-keyword">or</span>
tag <span class="hljs-keyword">is</span> <span class="hljs-string">'?'</span> <span class="hljs-keyword">and</span> i &gt; <span class="hljs-number">0</span> <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> tokens[i - <span class="hljs-number">1</span>].spaced) <span class="hljs-keyword">and</span>
(nextTag <span class="hljs-keyword">in</span> IMPLICIT_CALL <span class="hljs-keyword">or</span>
nextTag <span class="hljs-keyword">in</span> IMPLICIT_UNSPACED_CALL <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> nextToken.spaced <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> nextToken.newLine)
tag = token[<span class="hljs-number">0</span>] = <span class="hljs-string">'FUNC_EXIST'</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'?'</span>
startImplicitCall i + <span class="hljs-number">1</span>
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">2</span>)</pre></div></div>
</li>
<li id="section-23">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-23">&#182;</a>
</div>
<p>Implicit call taking an implicit indented object as first argument.</p>
<pre><code>f
a: b
c: d
</code></pre><p>Don’t accept implicit calls of this type, when on the same line
as the control structures below as that may misinterpret constructs like:</p>
<pre><code><span class="hljs-keyword">if</span> f
a: <span class="hljs-number">1</span>
</code></pre><p>as</p>
<pre><code><span class="hljs-keyword">if</span> f(a: <span class="hljs-number">1</span>)
</code></pre><p>which is probably always unintended.
Furthermore don’t allow this in literal arrays, as
that creates grammatical ambiguities.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> IMPLICIT_FUNC <span class="hljs-keyword">and</span>
@indexOfTag(i + <span class="hljs-number">1</span>, <span class="hljs-string">'INDENT'</span>) &gt; <span class="hljs-number">-1</span> <span class="hljs-keyword">and</span> @looksObjectish(i + <span class="hljs-number">2</span>) <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> @findTagsBackwards(i, [<span class="hljs-string">'CLASS'</span>, <span class="hljs-string">'EXTENDS'</span>, <span class="hljs-string">'IF'</span>, <span class="hljs-string">'CATCH'</span>,
<span class="hljs-string">'SWITCH'</span>, <span class="hljs-string">'LEADING_WHEN'</span>, <span class="hljs-string">'FOR'</span>, <span class="hljs-string">'WHILE'</span>, <span class="hljs-string">'UNTIL'</span>])
startImplicitCall i + <span class="hljs-number">1</span>
stack.push [<span class="hljs-string">'INDENT'</span>, i + <span class="hljs-number">2</span>]
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">3</span>)</pre></div></div>
</li>
<li id="section-24">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-24">&#182;</a>
</div>
<p>Implicit objects start here</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">':'</span></pre></div></div>
</li>
<li id="section-25">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-25">&#182;</a>
</div>
<p>Go back to the (implicit) start of the object</p>
</div>
<div class="content"><div class='highlight'><pre> s = <span class="hljs-keyword">switch</span>
<span class="hljs-keyword">when</span> @tag(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> EXPRESSION_END <span class="hljs-keyword">then</span> start[<span class="hljs-number">1</span>]
<span class="hljs-keyword">when</span> @tag(i - <span class="hljs-number">2</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'@'</span> <span class="hljs-keyword">then</span> i - <span class="hljs-number">2</span>
<span class="hljs-keyword">else</span> i - <span class="hljs-number">1</span>
s -= <span class="hljs-number">2</span> <span class="hljs-keyword">while</span> @tag(s - <span class="hljs-number">2</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'HERECOMMENT'</span>
startsLine = s <span class="hljs-keyword">is</span> <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> @tag(s - <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> LINEBREAKS <span class="hljs-keyword">or</span> tokens[s - <span class="hljs-number">1</span>].newLine</pre></div></div>
</li>
<li id="section-26">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-26">&#182;</a>
</div>
<p>Are we just continuing an already declared object?</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> stackTop()
[stackTag, stackIdx] = stackTop()
<span class="hljs-keyword">if</span> (stackTag <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span> <span class="hljs-keyword">or</span> stackTag <span class="hljs-keyword">is</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span> @tag(stackIdx - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span>) <span class="hljs-keyword">and</span>
(startsLine <span class="hljs-keyword">or</span> @tag(s - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">or</span> @tag(s - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span>)
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)
startImplicitObject(s, !!startsLine)
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">2</span>)</pre></div></div>
</li>
<li id="section-27">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-27">&#182;</a>
</div>
<p>End implicit calls when chaining method calls
like e.g.:</p>
<pre><code>f -&gt;
a
.g b, <span class="hljs-function">-&gt;</span>
c
.h a
</code></pre><p>and also</p>
<pre><code>f a
.g b
.h a
</code></pre>
</div>
</li>
<li id="section-28">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-28">&#182;</a>
</div>
<p>Mark all enclosing objects as not sameLine</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> LINEBREAKS
<span class="hljs-keyword">for</span> stackItem <span class="hljs-keyword">in</span> stack <span class="hljs-keyword">by</span> <span class="hljs-number">-1</span> <span class="hljs-keyword">when</span> isImplicitObject stackItem
stackItem[<span class="hljs-number">2</span>].sameLine = <span class="hljs-literal">no</span>
newLine = prevTag <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">or</span> prevToken.newLine
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> IMPLICIT_END <span class="hljs-keyword">or</span> tag <span class="hljs-keyword">in</span> CALL_CLOSERS <span class="hljs-keyword">and</span> newLine
<span class="hljs-keyword">while</span> inImplicit()
[stackTag, stackIdx, {sameLine, startsLine}] = stackTop()</pre></div></div>
</li>
<li id="section-29">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-29">&#182;</a>
</div>
<p>Close implicit calls when reached end of argument list</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> inImplicitCall() <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">','</span>
endImplicitCall()</pre></div></div>
</li>
<li id="section-30">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-30">&#182;</a>
</div>
<p>Close implicit objects such as:
return a: 1, b: 2 unless true</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> inImplicitObject() <span class="hljs-keyword">and</span> sameLine <span class="hljs-keyword">and</span>
tag <span class="hljs-keyword">isnt</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">':'</span> <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> (tag <span class="hljs-keyword">in</span> [<span class="hljs-string">'POST_IF'</span>, <span class="hljs-string">'FOR'</span>, <span class="hljs-string">'WHILE'</span>, <span class="hljs-string">'UNTIL'</span>] <span class="hljs-keyword">and</span> startsLine <span class="hljs-keyword">and</span> implicitObjectContinues(i + <span class="hljs-number">1</span>))
endImplicitObject()</pre></div></div>
</li>
<li id="section-31">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-31">&#182;</a>
</div>
<p>Close implicit objects when at end of line, line didn’t end with a comma
and the implicit object didn’t start the line or the next line doesn’t look like
the continuation of an object.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> inImplicitObject() <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">','</span> <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> (startsLine <span class="hljs-keyword">and</span> @looksObjectish(i + <span class="hljs-number">1</span>))
<span class="hljs-keyword">return</span> forward <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> nextTag <span class="hljs-keyword">is</span> <span class="hljs-string">'HERECOMMENT'</span>
endImplicitObject()
<span class="hljs-keyword">else</span>
<span class="hljs-keyword">break</span></pre></div></div>
</li>
<li id="section-32">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-32">&#182;</a>
</div>
<p>Close implicit object if comma is the last character
and what comes after doesn’t look like it belongs.
This is used for trailing commas and calls, like:</p>
<pre><code>x =
a: b,
c: d,
e = <span class="hljs-number">2</span>
</code></pre><p>and</p>
<pre><code>f a, b: c, d: e, f, g: h: i, j
</code></pre>
</div>
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> @looksObjectish(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">and</span> inImplicitObject() <span class="hljs-keyword">and</span>
(nextTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">or</span> <span class="hljs-keyword">not</span> @looksObjectish(i + <span class="hljs-number">2</span>))</pre></div></div>
</li>
<li id="section-33">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-33">&#182;</a>
</div>
<p>When nextTag is OUTDENT the comma is insignificant and
should just be ignored so embed it in the implicit object.</p>
<p>When it isn’t the comma go on to play a role in a call or
array further up the stack, so give it a chance.</p>
</div>
<div class="content"><div class='highlight'><pre>
offset = <span class="hljs-keyword">if</span> nextTag <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">then</span> <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>
<span class="hljs-keyword">while</span> inImplicitObject()
endImplicitObject i + offset
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)</pre></div></div>
</li>
<li id="section-34">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-34">&#182;</a>
</div>
<p>Make sure only strings and wrapped expressions are used in CSX attributes</p>
</div>
<div class="content"><div class='highlight'><pre> enforceValidCSXAttributes: <span class="hljs-function">-&gt;</span>
@scanTokens (token, i, tokens) -&gt;
<span class="hljs-keyword">if</span> token.csxColon
next = tokens[i + <span class="hljs-number">1</span>]
<span class="hljs-keyword">if</span> next[<span class="hljs-number">0</span>] <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> [<span class="hljs-string">'STRING_START'</span>, <span class="hljs-string">'STRING'</span>, <span class="hljs-string">'('</span>]
throwSyntaxError <span class="hljs-string">'expected wrapped or quoted CSX attribute'</span>, next[<span class="hljs-number">2</span>]
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-35">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-35">&#182;</a>
</div>
<p>Add location data to all tokens generated by the rewriter.</p>
</div>
<div class="content"><div class='highlight'><pre> addLocationDataToGeneratedTokens: <span class="hljs-function">-&gt;</span>
@scanTokens (token, i, tokens) -&gt;
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> token[<span class="hljs-number">2</span>]
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">unless</span> token.generated <span class="hljs-keyword">or</span> token.explicit
<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span> <span class="hljs-keyword">and</span> nextLocation=tokens[i + <span class="hljs-number">1</span>]?[<span class="hljs-number">2</span>]
{first_line: line, first_column: column} = nextLocation
<span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prevLocation = tokens[i - <span class="hljs-number">1</span>]?[<span class="hljs-number">2</span>]
{last_line: line, last_column: column} = prevLocation
<span class="hljs-keyword">else</span>
line = column = <span class="hljs-number">0</span>
token[<span class="hljs-number">2</span>] =
first_line: line
first_column: column
last_line: line
last_column: column
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-36">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-36">&#182;</a>
</div>
<p>OUTDENT tokens should always be positioned at the last character of the
previous token, so that AST nodes ending in an OUTDENT token end up with a
location corresponding to the last “real” token under the node.</p>
</div>
<div class="content"><div class='highlight'><pre> fixOutdentLocationData: <span class="hljs-function">-&gt;</span>
@scanTokens (token, i, tokens) -&gt;
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">unless</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">or</span>
(token.generated <span class="hljs-keyword">and</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'CALL_END'</span>) <span class="hljs-keyword">or</span>
(token.generated <span class="hljs-keyword">and</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'}'</span>)
prevLocationData = tokens[i - <span class="hljs-number">1</span>][<span class="hljs-number">2</span>]
token[<span class="hljs-number">2</span>] =
first_line: prevLocationData.last_line
first_column: prevLocationData.last_column
last_line: prevLocationData.last_line
last_column: prevLocationData.last_column
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-37">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-37">&#182;</a>
</div>
<p>Because our grammar is LALR(1), it can’t handle some single-line
expressions that lack ending delimiters. The <strong>Rewriter</strong> adds the implicit
blocks, so it doesn’t need to. To keep the grammar clean and tidy, trailing
newlines within expressions are removed and the indentation tokens of empty
blocks are added.</p>
</div>
<div class="content"><div class='highlight'><pre> normalizeLines: <span class="hljs-function">-&gt;</span>
starter = indent = outdent = <span class="hljs-literal">null</span>
<span class="hljs-function">
<span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
token[<span class="hljs-number">1</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">';'</span> <span class="hljs-keyword">and</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> SINGLE_CLOSERS <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> (token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">and</span> @tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> EXPRESSION_CLOSE) <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> (token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'ELSE'</span> <span class="hljs-keyword">and</span> starter <span class="hljs-keyword">isnt</span> <span class="hljs-string">'THEN'</span>) <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> (token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'FINALLY'</span>] <span class="hljs-keyword">and</span> starter <span class="hljs-keyword">in</span> [<span class="hljs-string">'-&gt;'</span>, <span class="hljs-string">'=&gt;'</span>]) <span class="hljs-keyword">or</span>
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> CALL_CLOSERS <span class="hljs-keyword">and</span>
(@tokens[i - <span class="hljs-number">1</span>].newLine <span class="hljs-keyword">or</span> @tokens[i - <span class="hljs-number">1</span>][<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span>)
<span class="hljs-function">
<span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
@tokens.splice (<span class="hljs-keyword">if</span> @tag(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">then</span> i - <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> i), <span class="hljs-number">0</span>, outdent
@scanTokens (token, i, tokens) -&gt;
[tag] = token
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span>
<span class="hljs-keyword">if</span> @tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'ELSE'</span> <span class="hljs-keyword">and</span> @tag(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">isnt</span> <span class="hljs-string">'OUTDENT'</span>
tokens.splice i, <span class="hljs-number">1</span>, @indentation()...
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
<span class="hljs-keyword">if</span> @tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> EXPRESSION_CLOSE
tokens.splice i, <span class="hljs-number">1</span>
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span>
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'CATCH'</span>
<span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> [<span class="hljs-number">1.</span><span class="hljs-number">.2</span>] <span class="hljs-keyword">when</span> @tag(i + j) <span class="hljs-keyword">in</span> [<span class="hljs-string">'OUTDENT'</span>, <span class="hljs-string">'TERMINATOR'</span>, <span class="hljs-string">'FINALLY'</span>]
tokens.splice i + j, <span class="hljs-number">0</span>, @indentation()...
<span class="hljs-keyword">return</span> <span class="hljs-number">2</span> + j
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> [<span class="hljs-string">'-&gt;'</span>, <span class="hljs-string">'=&gt;'</span>] <span class="hljs-keyword">and</span> (@tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">or</span> @tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'.'</span> <span class="hljs-keyword">and</span> token.newLine)
[indent, outdent] = @indentation tokens[i]
tokens.splice i + <span class="hljs-number">1</span>, <span class="hljs-number">0</span>, indent, outdent
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> SINGLE_LINERS <span class="hljs-keyword">and</span> @tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">isnt</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span>
<span class="hljs-keyword">not</span> (tag <span class="hljs-keyword">is</span> <span class="hljs-string">'ELSE'</span> <span class="hljs-keyword">and</span> @tag(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'IF'</span>)
starter = tag
[indent, outdent] = @indentation tokens[i]
indent.fromThen = <span class="hljs-literal">true</span> <span class="hljs-keyword">if</span> starter <span class="hljs-keyword">is</span> <span class="hljs-string">'THEN'</span>
tokens.splice i + <span class="hljs-number">1</span>, <span class="hljs-number">0</span>, indent
@detectEnd i + <span class="hljs-number">2</span>, condition, action
tokens.splice i, <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'THEN'</span>
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-38">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-38">&#182;</a>
</div>
<p>Tag postfix conditionals as such, so that we can parse them with a
different precedence.</p>
</div>
<div class="content"><div class='highlight'><pre> tagPostfixConditionals: <span class="hljs-function">-&gt;</span>
original = <span class="hljs-literal">null</span>
<span class="hljs-function">
<span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
[tag] = token
[prevTag] = @tokens[i - <span class="hljs-number">1</span>]
tag <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">or</span> (tag <span class="hljs-keyword">is</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> SINGLE_LINERS)
<span class="hljs-function">
<span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -&gt;</span>
<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">or</span> (token.generated <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> token.fromThen)
original[<span class="hljs-number">0</span>] = <span class="hljs-string">'POST_'</span> + original[<span class="hljs-number">0</span>]
@scanTokens (token, i) -&gt;
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">unless</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'IF'</span>
original = token
@detectEnd i + <span class="hljs-number">1</span>, condition, action
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
</li>
<li id="section-39">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-39">&#182;</a>
</div>
<p>Generate the indentation tokens, based on another token on the same line.</p>
</div>
<div class="content"><div class='highlight'><pre> indentation: <span class="hljs-function"><span class="hljs-params">(origin)</span> -&gt;</span>
indent = [<span class="hljs-string">'INDENT'</span>, <span class="hljs-number">2</span>]
outdent = [<span class="hljs-string">'OUTDENT'</span>, <span class="hljs-number">2</span>]
<span class="hljs-keyword">if</span> origin
indent.generated = outdent.generated = <span class="hljs-literal">yes</span>
indent.origin = outdent.origin = origin
<span class="hljs-keyword">else</span>
indent.explicit = outdent.explicit = <span class="hljs-literal">yes</span>
[indent, outdent]
generate: generate</pre></div></div>
</li>
<li id="section-40">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-40">&#182;</a>
</div>
<p>Look up a tag by token index.</p>
</div>
<div class="content"><div class='highlight'><pre> tag: <span class="hljs-function"><span class="hljs-params">(i)</span> -&gt;</span> @tokens[i]?[<span class="hljs-number">0</span>]</pre></div></div>
</li>
<li id="section-41">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-41">&#182;</a>
</div>
<h2 id="constants">Constants</h2>
</div>
</li>
<li id="section-42">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-42">&#182;</a>
</div>
</div>
</li>
<li id="section-43">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-43">&#182;</a>
</div>
<p>List of the token pairs that must be balanced.</p>
</div>
<div class="content"><div class='highlight'><pre>BALANCED_PAIRS = [
[<span class="hljs-string">'('</span>, <span class="hljs-string">')'</span>]
[<span class="hljs-string">'['</span>, <span class="hljs-string">']'</span>]
[<span class="hljs-string">'{'</span>, <span class="hljs-string">'}'</span>]
[<span class="hljs-string">'INDENT'</span>, <span class="hljs-string">'OUTDENT'</span>],
[<span class="hljs-string">'CALL_START'</span>, <span class="hljs-string">'CALL_END'</span>]
[<span class="hljs-string">'PARAM_START'</span>, <span class="hljs-string">'PARAM_END'</span>]
[<span class="hljs-string">'INDEX_START'</span>, <span class="hljs-string">'INDEX_END'</span>]
[<span class="hljs-string">'STRING_START'</span>, <span class="hljs-string">'STRING_END'</span>]
[<span class="hljs-string">'REGEX_START'</span>, <span class="hljs-string">'REGEX_END'</span>]
]</pre></div></div>
</li>
<li id="section-44">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-44">&#182;</a>
</div>
<p>The inverse mappings of <code>BALANCED_PAIRS</code> we’re trying to fix up, so we can
look things up from either end.</p>
</div>
<div class="content"><div class='highlight'><pre>exports.INVERSES = INVERSES = {}</pre></div></div>
</li>
<li id="section-45">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-45">&#182;</a>
</div>
<p>The tokens that signal the start/end of a balanced pair.</p>
</div>
<div class="content"><div class='highlight'><pre>EXPRESSION_START = []
EXPRESSION_END = []
<span class="hljs-keyword">for</span> [left, rite] <span class="hljs-keyword">in</span> BALANCED_PAIRS
EXPRESSION_START.push INVERSES[rite] = left
EXPRESSION_END .push INVERSES[left] = rite</pre></div></div>
</li>
<li id="section-46">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-46">&#182;</a>
</div>
<p>Tokens that indicate the close of a clause of an expression.</p>
</div>
<div class="content"><div class='highlight'><pre>EXPRESSION_CLOSE = [<span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'THEN'</span>, <span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'FINALLY'</span>].concat EXPRESSION_END</pre></div></div>
</li>
<li id="section-47">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-47">&#182;</a>
</div>
<p>Tokens that, if followed by an <code>IMPLICIT_CALL</code>, indicate a function invocation.</p>
</div>
<div class="content"><div class='highlight'><pre>IMPLICIT_FUNC = [<span class="hljs-string">'IDENTIFIER'</span>, <span class="hljs-string">'PROPERTY'</span>, <span class="hljs-string">'SUPER'</span>, <span class="hljs-string">')'</span>, <span class="hljs-string">'CALL_END'</span>, <span class="hljs-string">']'</span>, <span class="hljs-string">'INDEX_END'</span>, <span class="hljs-string">'@'</span>, <span class="hljs-string">'THIS'</span>]</pre></div></div>
</li>
<li id="section-48">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-48">&#182;</a>
</div>
<p>If preceded by an <code>IMPLICIT_FUNC</code>, indicates a function invocation.</p>
</div>
<div class="content"><div class='highlight'><pre>IMPLICIT_CALL = [
<span class="hljs-string">'IDENTIFIER'</span>, <span class="hljs-string">'CSX_TAG'</span>, <span class="hljs-string">'PROPERTY'</span>, <span class="hljs-string">'NUMBER'</span>, <span class="hljs-string">'INFINITY'</span>, <span class="hljs-string">'NAN'</span>
<span class="hljs-string">'STRING'</span>, <span class="hljs-string">'STRING_START'</span>, <span class="hljs-string">'REGEX'</span>, <span class="hljs-string">'REGEX_START'</span>, <span class="hljs-string">'JS'</span>
<span class="hljs-string">'NEW'</span>, <span class="hljs-string">'PARAM_START'</span>, <span class="hljs-string">'CLASS'</span>, <span class="hljs-string">'IF'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'SWITCH'</span>, <span class="hljs-string">'THIS'</span>
<span class="hljs-string">'UNDEFINED'</span>, <span class="hljs-string">'NULL'</span>, <span class="hljs-string">'BOOL'</span>
<span class="hljs-string">'UNARY'</span>, <span class="hljs-string">'YIELD'</span>, <span class="hljs-string">'AWAIT'</span>, <span class="hljs-string">'UNARY_MATH'</span>, <span class="hljs-string">'SUPER'</span>, <span class="hljs-string">'THROW'</span>
<span class="hljs-string">'@'</span>, <span class="hljs-string">'-&gt;'</span>, <span class="hljs-string">'=&gt;'</span>, <span class="hljs-string">'['</span>, <span class="hljs-string">'('</span>, <span class="hljs-string">'{'</span>, <span class="hljs-string">'--'</span>, <span class="hljs-string">'++'</span>
]
IMPLICIT_UNSPACED_CALL = [<span class="hljs-string">'+'</span>, <span class="hljs-string">'-'</span>]</pre></div></div>
</li>
<li id="section-49">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-49">&#182;</a>
</div>
<p>Tokens that always mark the end of an implicit call for single-liners.</p>
</div>
<div class="content"><div class='highlight'><pre>IMPLICIT_END = [<span class="hljs-string">'POST_IF'</span>, <span class="hljs-string">'FOR'</span>, <span class="hljs-string">'WHILE'</span>, <span class="hljs-string">'UNTIL'</span>, <span class="hljs-string">'WHEN'</span>, <span class="hljs-string">'BY'</span>,
<span class="hljs-string">'LOOP'</span>, <span class="hljs-string">'TERMINATOR'</span>]</pre></div></div>
</li>
<li id="section-50">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-50">&#182;</a>
</div>
<p>Single-line flavors of block expressions that have unclosed endings.
The grammar can’t disambiguate them, so we insert the implicit indentation.</p>
</div>
<div class="content"><div class='highlight'><pre>SINGLE_LINERS = [<span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'-&gt;'</span>, <span class="hljs-string">'=&gt;'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'FINALLY'</span>, <span class="hljs-string">'THEN'</span>]
SINGLE_CLOSERS = [<span class="hljs-string">'TERMINATOR'</span>, <span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'FINALLY'</span>, <span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'OUTDENT'</span>, <span class="hljs-string">'LEADING_WHEN'</span>]</pre></div></div>
</li>
<li id="section-51">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-51">&#182;</a>
</div>
<p>Tokens that end a line.</p>
</div>
<div class="content"><div class='highlight'><pre>LINEBREAKS = [<span class="hljs-string">'TERMINATOR'</span>, <span class="hljs-string">'INDENT'</span>, <span class="hljs-string">'OUTDENT'</span>]</pre></div></div>
</li>
<li id="section-52">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-52">&#182;</a>
</div>
<p>Tokens that close open calls when they follow a newline.</p>
</div>
<div class="content"><div class='highlight'><pre>CALL_CLOSERS = [<span class="hljs-string">'.'</span>, <span class="hljs-string">'?.'</span>, <span class="hljs-string">'::'</span>, <span class="hljs-string">'?::'</span>]</pre></div></div>
</li>
<li id="section-53">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-53">&#182;</a>
</div>
<p>Tokens that prevent a subsequent indent from ending implicit calls/objects</p>
</div>
<div class="content"><div class='highlight'><pre>CONTROL_IN_IMPLICIT = [<span class="hljs-string">'IF'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'FINALLY'</span>, <span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'CLASS'</span>, <span class="hljs-string">'SWITCH'</span>]</pre></div></div>
</li>
</ul>
</div>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink