* Un-prefer global (#4543)
* 1.12.6 changelog; update NPM module in documentation to be `coffeescript` instead of `coffee-script`; update installation to add note about global vs local `coffee` command
* Update packages
* Updated output
* Simplify changelog
* Use Markdown-It instead of Marked for generating the docs; update package versions
* Fix links to v2 docs
* Bump version to 1.12.5; update changelog and compiled docs output
* Update compiled output for 1.12.5
* Improve styling for tables
This is an upstream port of https://github.com/decaffeinate/coffeescript/pull/10
See that PR for links to the issues that this fixes.
Just like OUTDENT and CALL_END tokens, close-curly-brace tokens can be generated
without having a real location, and if that position overlaps with a later
token, it can cause the AST to have bad location data. Just like the other two
token types, we now give `}` tokens the position of the previous real token,
which makes all AST nodes have reasonable locations.
Fixes https://github.com/decaffeinate/decaffeinate/issues/446
In addition to OUTDENT tokens, CALL_END tokens can also be virtual tokens
without a real location, and sometimes they end up with a location that's
incorrect.
This commit adds another post-processing step after normal lexing that sets the
locationData on all OUTDENT tokens to be at the last character of the previous
token. This does feel like a little bit of a hack. Ideally the location data
would be set correctly in the first place and not in a post-processing step, but
I tried that and some temporary intermediate tokens were causing problems, so I
decided to set the location data once those intermediate tokens were removed.
Also, having this as a separate processing step makes it more robust and
isolated.
This fixes the problem in https://github.com/decaffeinate/decaffeinate/issues/371 .
In that issue, the CoffeeScript tokens had three OUTDENT tokens in a row, and
the last two overlapped with the `]`. Since at least one of those OUTDENT tokens
was considered part of the function body, the function expression had an ending
position just after the end of the `]`.
OUTDENT tokens are sort of a weird case in the lexer anyway, since they often
don't correspond to an actual location in the source code. It seems like the
code in `lexer.coffee` makes an attempt at finding a good place for them, but in
some cases, it has a bad result. This seems hard to avoid in the general case.
For example, in this code:
```coffee
[->
a]
```
There must be an OUTDENT between the `a` and the `]`, but CoffeeScript tokens
have an inclusive start and end, so they must always be at least one character
wide (I think). In this case, the lexer was choosing the `]` as the location,
and the parser ended up generating correct location data, I believe because
it ignores the outermost INDENT and OUTDENT tokens. However, with multiple
OUTDENT tokens in a row, the parser ends up producing location data that is
wrong.
It seems to me like there isn't a solid answer to "what location do OUTDENT
tokens have", since it hasn't mattered much, but for this commit, I'm defining
it: they always have the location of the last character of the previous token.
This should hopefully be fairly safe because tokens are still in the same order
relative to each other. Also, it's worth noting that this makes the start
location for OUTDENT tokens awkward. However, OUTDENT tokens are always used to
mark the end of something, so their `last_line` and `last_column` values are
always what matter when determining AST node bounds, so it is most important for
those to be correct.
- Split out a PROPERTY token from the IDENTIFIER token.
- Split out Property from the Identifier in the grammar.
- Split out PropertyLiteral from IdentifierLiteral.
Previously, the parser created `Literal` nodes for many things. This resulted in
information loss. Instead of being able to check the node type, we had to use
regexes to tell the different types of `Literal`s apart. That was a bit like
parsing literals twice: Once in the lexer, and once (or more) in the compiler.
It also caused problems, such as `` `this` `` and `this` being indistinguishable
(fixes#2009).
Instead returning `new Literal` in the grammar, subtypes of it are now returned
instead, such as `NumberLiteral`, `StringLiteral` and `IdentifierLiteral`. `new
Literal` by itself is only used to represent code chunks that fit no category.
(While mentioning `NumberLiteral`, there's also `InfinityLiteral` now, which is
a subtype of `NumberLiteral`.)
`StringWithInterpolations` has been added as a subtype of `Parens`, and
`RegexWithInterpolations` as a subtype of `Call`. This makes it easier for other
programs to make use of CoffeeScript's "AST" (nodes). For example, it is now
possible to distinguish between `"a #{b} c"` and `"a " + b + " c"`. Fixes#4192.
`SuperCall` has been added as a subtype of `Call`.
Note, though, that some information is still lost, especially in the lexer. For
example, there is no way to distinguish a heredoc from a regular string, or a
heregex without interpolations from a regular regex. Binary and octal number
literals are indistinguishable from hexadecimal literals.
After the new subtypes were added, they were taken advantage of, removing most
regexes in nodes.coffee. `SIMPLENUM` (which matches non-hex integers) had to be
kept, though, because such numbers need special handling in JavaScript (for
example in `1..toString()`).
An especially nice hack to get rid of was using `new String()` for the token
value for reserved identifiers (to be able to set a property on them which could
survive through the parser). Now it's a good old regular string.
In range literals, slices, splices and for loop steps when number literals
are involved, CoffeeScript can do some optimizations, such as precomputing the
value of, say, `5 - 3` (outputting `2` instead of `5 - 3` literally). As a side
bonus, this now also works with hexadecimal number literals, such as `0x02`.
Finally, this also improves the output of `coffee --nodes`:
# Before:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value
Bool
Block
Value
Parens
Block
Op +
Value """"
Value
Parens
Block
Value "a" "break"
# After:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value BooleanLiteral: true
Block
Value
StringWithInterpolations
Block
Op +
Value StringLiteral: ""
Value
Parens
Block
Value IdentifierLiteral: a
StatementLiteral: break
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes#3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes#1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
Any variables generated by CoffeeScript are now made sure to be named to
something not present in the source code being compiled. This way you can no
longer interfere with them, either on purpose or by mistake. (#1500, #1574)
For example, `({a}, _arg) ->` now compiles correctly. (#1574)
As opposed to the somewhat complex implementations discussed in #1500, this
commit takes a very simple approach by saving all used variables names using a
single pass over the token stream. Any generated variables are then made sure
not to exist in that list.
`(@a) -> a` used to be equivalent to `(@a) -> @a`, but now throws a runtime
`ReferenceError` instead (unless `a` exists in an upper scope of course). (#3318)
`(@a) ->` used to compile to `(function(a) { this.a = a; })`. Now it compiles to
`(function(_at_a) { this.a = _at_a; })`. (But you cannot access `_at_a` either,
of course.)
Because of the above, `(@a, a) ->` is now valid; `@a` and `a` are not duplicate
parameters.
Duplicate this-parameters with a reserved word, such as `(@case, @case) ->`,
used to compile but now throws, just like regular duplicate parameters.
