This is an upstream port for the patch https://github.com/decaffeinate/coffeescript/pull/8
See https://github.com/decaffeinate/decaffeinate/issues/291 for the bug that this fixed.
For the most part, CoffeeScript and JavaScript have the same precedence rules,
but in some cases, the intermediate AST format didn't represent the actual
evaluation order. For example, in the expression `a or b and c`, the `and` is
evaluated first, but the parser treated the two operators with equal precedence.
This was still correct end-to-end because CoffeeScript simply emitted the result
without parens, but any intermediate tools using the CoffeeScript parser could
become confused.
Here are the JS operator precedence rules:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence
For the most part, CoffeeScript already follows these. `COMPARE` operators
already behave differently due to chained comparisons, so I think we don't need
to worry about following JS precedence for those. So I think the only case where
it was behaving differently in an important way was for the binary/bitwise
operators that are being changed here.
As part of this change, I also introduced a new token tag, `BIN?`, for the
binary form of the `?` operator.
Fixes https://github.com/decaffeinate/decaffeinate/issues/446
In addition to OUTDENT tokens, CALL_END tokens can also be virtual tokens
without a real location, and sometimes they end up with a location that's
incorrect.
This commit adds another post-processing step after normal lexing that sets the
locationData on all OUTDENT tokens to be at the last character of the previous
token. This does feel like a little bit of a hack. Ideally the location data
would be set correctly in the first place and not in a post-processing step, but
I tried that and some temporary intermediate tokens were causing problems, so I
decided to set the location data once those intermediate tokens were removed.
Also, having this as a separate processing step makes it more robust and
isolated.
This fixes the problem in https://github.com/decaffeinate/decaffeinate/issues/371 .
In that issue, the CoffeeScript tokens had three OUTDENT tokens in a row, and
the last two overlapped with the `]`. Since at least one of those OUTDENT tokens
was considered part of the function body, the function expression had an ending
position just after the end of the `]`.
OUTDENT tokens are sort of a weird case in the lexer anyway, since they often
don't correspond to an actual location in the source code. It seems like the
code in `lexer.coffee` makes an attempt at finding a good place for them, but in
some cases, it has a bad result. This seems hard to avoid in the general case.
For example, in this code:
```coffee
[->
a]
```
There must be an OUTDENT between the `a` and the `]`, but CoffeeScript tokens
have an inclusive start and end, so they must always be at least one character
wide (I think). In this case, the lexer was choosing the `]` as the location,
and the parser ended up generating correct location data, I believe because
it ignores the outermost INDENT and OUTDENT tokens. However, with multiple
OUTDENT tokens in a row, the parser ends up producing location data that is
wrong.
It seems to me like there isn't a solid answer to "what location do OUTDENT
tokens have", since it hasn't mattered much, but for this commit, I'm defining
it: they always have the location of the last character of the previous token.
This should hopefully be fairly safe because tokens are still in the same order
relative to each other. Also, it's worth noting that this makes the start
location for OUTDENT tokens awkward. However, OUTDENT tokens are always used to
mark the end of something, so their `last_line` and `last_column` values are
always what matter when determining AST node bounds, so it is most important for
those to be correct.
`"""` (and `"`) strings are lexed into an array of tokens, consisting of
strings and interpolations. Previously, the minimum indententation
inside `"""` strings was stripped from the beginning of _all_ of those
string tokens. Usually, the indentation is longer than any other
sequence of spaces in a `"""` string, so the problem didn't occur in
most cases. This commit makes sure to only strip indentation after
newlines.
Fixes#4314.
`isLiteralArguments` mistakenly looked at `Literal`s instead of
`IdentifierLiteral`s.
This also gets rid of the ugly `.asKey` hack in nodes.coffee.
Fixes#4320.
This prevents mixing spaces and tabs in the same ‘block’ of code.
Mixing is still allowed if each line uses the same mix and if the indentation level returns to 0.
This breaks the literate coffeescript test that mixes spaces and tabs.
This pull request adds support for ES2015 modules, by recognizing `import` and `export` statements. The following syntaxes are supported, based on the MDN [import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [export](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export) pages:
```js
import "module-name"
import defaultMember from "module-name"
import * as name from "module-name"
import { } from "module-name"
import { member } from "module-name"
import { member as alias } from "module-name"
import { member1, member2 as alias2, … } from "module-name"
import defaultMember, * as name from "module-name"
import defaultMember, { … } from "module-name"
export default expression
export class name
export { }
export { name }
export { name as exportedName }
export { name as default }
export { name1, name2 as exportedName2, name3 as default, … }
export * from "module-name"
export { … } from "module-name"
```
As a subsitute for ECMAScript’s `export var name = …` and `export function name {}`, CoffeeScript also supports:
```js
export name = …
```
CoffeeScript also supports optional commas within `{ … }`.
This PR converts the supported `import` and `export` statements into ES2015 `import` and `export` statements; it **does not resolve the modules**. So any CoffeeScript with `import` or `export` statements will be output as ES2015, and will need to be transpiled by another tool such as Babel before it can be used in a browser. We will need to add a warning to the documentation explaining this.
This should be fully backwards-compatible, as `import` and `export` were previously reserved keywords. No flags are used.
There are extensive tests included, though because no current JavaScript runtime supports `import` or `export`, the tests compare strings of what the compiled CoffeeScript output is against what the expected ES2015 should be. I also conducted two more elaborate tests:
* I forked the [ember-piqu](https://github.com/pauc/piqu-ember) project, which was an Ember CLI app that used ember-cli-coffeescript and [ember-cli-coffees6](https://github.com/alexspeller/ember-cli-coffees6) (which adds “support” for `import`/`export` by wrapping such statements in backticks before passing the result to the CoffeeScript compiler). I removed `ember-cli-coffees6` and replaced the CoffeeScript compiler used in the build chain with this code, and the app built without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-piqu)
* I also forked the [CoffeeScript version of Meteor’s Todos example app](https://github.com/meteor/todos/tree/coffeescript), and replaced all of its `require` statements with the `import` and `export` statements from the original ES2015 version of the app on its `master` branch. I then updated the `coffeescript` Meteor package in the app to use this new code, and again the app builds without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-meteor-todos)
The discussion history for this work started [here](https://github.com/jashkenas/coffeescript/pull/4160) and continued [here](https://github.com/GeoffreyBooth/coffeescript/pull/2). @lydell provided guidance, and @JimPanic and @rattrayalex contributed essential code.
In for example `for` loops, a variable called `i` is generated (for the
loop index). If that name is unavailable, `j` is used instead, then `k`,
`l`, etc. all the way to `z`. Then, `aa`, `ab`, `ac` etc. are used.
This meant that, eventually, `do` would be used, but that's not a valid
variable name since `do` is a JavaScript keyword.
This logic was also inefficiently implemented. For example, going from
`aa` to `ab` or from `az` to `ba` required lots of loop iterations.
This commit changes the variable naming convention. Now, `i`, `j`, `k`,
etc. to `z` are used like before. Then comes `i1`, `j1`, `k1`, etc. Then
`i2`, `j2`, `k2` and so on. This is simpler, efficient and easier to
understand. `i1` is more obvious to be a loop index than `aa`.
Fixes#4267.
This should have been done in commit 841b3cd2, but I forgot to. Since
that commit, `SourceMap::generate` returns an object instead of
`JSON.stringify()` of that object, but the tests still compared strings.
Fixes#4269.
Note: `SourceMap::generate` is only used internally, so its change in
return type is not a breaking change. The "public API" is unchanged.
- Show the same type of error message for compound assignment as for `=`
assignment when the LHS is invalid.
- Show the same type of error message when trying to assign to a CoffeeScript
keyword as when trying to assign to a JavaScript keyword.
- Now longer treat `&& =` as `&&=`. The same goes for `and=`, `||=` and `or=`.
- Unify the error message to: `<optional type> '<value>' can't be assigned`.
Previously, the parser created `Literal` nodes for many things. This resulted in
information loss. Instead of being able to check the node type, we had to use
regexes to tell the different types of `Literal`s apart. That was a bit like
parsing literals twice: Once in the lexer, and once (or more) in the compiler.
It also caused problems, such as `` `this` `` and `this` being indistinguishable
(fixes#2009).
Instead returning `new Literal` in the grammar, subtypes of it are now returned
instead, such as `NumberLiteral`, `StringLiteral` and `IdentifierLiteral`. `new
Literal` by itself is only used to represent code chunks that fit no category.
(While mentioning `NumberLiteral`, there's also `InfinityLiteral` now, which is
a subtype of `NumberLiteral`.)
`StringWithInterpolations` has been added as a subtype of `Parens`, and
`RegexWithInterpolations` as a subtype of `Call`. This makes it easier for other
programs to make use of CoffeeScript's "AST" (nodes). For example, it is now
possible to distinguish between `"a #{b} c"` and `"a " + b + " c"`. Fixes#4192.
`SuperCall` has been added as a subtype of `Call`.
Note, though, that some information is still lost, especially in the lexer. For
example, there is no way to distinguish a heredoc from a regular string, or a
heregex without interpolations from a regular regex. Binary and octal number
literals are indistinguishable from hexadecimal literals.
After the new subtypes were added, they were taken advantage of, removing most
regexes in nodes.coffee. `SIMPLENUM` (which matches non-hex integers) had to be
kept, though, because such numbers need special handling in JavaScript (for
example in `1..toString()`).
An especially nice hack to get rid of was using `new String()` for the token
value for reserved identifiers (to be able to set a property on them which could
survive through the parser). Now it's a good old regular string.
In range literals, slices, splices and for loop steps when number literals
are involved, CoffeeScript can do some optimizations, such as precomputing the
value of, say, `5 - 3` (outputting `2` instead of `5 - 3` literally). As a side
bonus, this now also works with hexadecimal number literals, such as `0x02`.
Finally, this also improves the output of `coffee --nodes`:
# Before:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value
Bool
Block
Value
Parens
Block
Op +
Value """"
Value
Parens
Block
Value "a" "break"
# After:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value BooleanLiteral: true
Block
Value
StringWithInterpolations
Block
Op +
Value StringLiteral: ""
Value
Parens
Block
Value IdentifierLiteral: a
StatementLiteral: break
`({a = 1}) ->` and `({a: b}) ->` worked, but not the combination of the two:
`({a: b = 1}) ->`. That destrucuring worked for normal assignments, though:
`{a: b = 1} = c`. This commit fixes the param case.
This breaks compatibility with
->
yield for i in [1..3]
i * 2
and
->
yield
i * 2
yield's behaviour now mirrors that of return in that it can be used stand alone as well as with expressions. Thus, it currently also inherits the above limitations.
This let's you do things like:
fullName = ({first = 'John', last = 'Doe'}) -> "#{first} #{last}"
Note: CoffeeScrits treats `undefined` and `null` the same, and that's true in
the case of destructuring defaults as well, as opposed to ES2015 which only uses
the default value if the target is `undefined`. A similar ES2015 difference
already exists for function parameter defaults. It is important for CoffeeScript
to be consistent with itself.
fullName2 = (first = 'John', last = 'Doe') -> "#{first} #{last}"
assert fullName('Bob', null) is fullName2(first: 'Bob', last: null)
Fixes#1558, #3288 and #4005.
Let me know if there's something I should be doing differently as this is my first contribution to coffeescript.
I fixed the reported issue where a generated variable could clash with a user-defined one in a try/catch block.
I added a test for a few scenarios with different variable names for a try/catch, to confirm the fix and avoid regressions.
