* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
* Try to detect when `from` in a `for` loop declaration is an identifier, not a keyword
* Handle destructured arrays
* from as a destructured, aliased object variable name in a for loop declaration
* Output interpolated strings as template literals.
* Update comments to use Markdown, so the annotated source reads well; remove TODOs per @lydell’s explanations
* Support JavaScript code blocks set apart by triple backticks (``` ... ```)
* Add test for escaped backticks
* Remove TODOs for things we’re never going to support
* Convert escaped backticks to backticks; update tests
* Block inline JavaScript can end with an escaped backtick character
* Updated JavaScript token regexes per @lydell
* In JavaScript blocks, escape backslashes when they immediately precede backticks; additional tests
* Test that we don’t break backslash escaping in JavaScript literals
* Add initial support for template literals with no
interpolation
* Change ‘unexpected string’ error message tests to
use number not identifier prefix.
Identifer prefixes are now valid as tagged
template literals
* Test tagged template literals for non-interpolated
strings and tag function.
* Tagged template literals work for pure Strings.
Pull tagged template definition up to Invocation
level in grammar, enabling chained invocation calls.
We can view a tagged template is a special form
of function call.
* Readying for StringWithInterpolations work.
* Tweaks.
* Fix style
* Pass StringWithInterpolations parameter straight
into Call constructor.
StringWithInterpolations will be output as
template literal, so already in correct form for
outputting tagged template literal.
* Strip down compileNode for StringWithInterpolations
* Done StringLiteral case for interpolated Strings
* Remove need for TemplateLiteral
* Simplify code.
* Small code tidy
* Interpolated strings now outputting as template literals.
Still needs comprehensive testing.
* Move error message tests into error_messages.coffee; remove test that is testing for a Node runtime error
* Split up tests that were testing multiple things per test, so that each test tests only one thing
* Edge cases: tagged template literals containing interpolated strings or even internal tagged template literals
* Make more concise, more idiomatic style
* Pull back extreme indentation
* Restore and fix commented-out tests
* Edge case: tagged template literal with empty string
* Only use new ES2015 interpolated string syntax if we’re inside a tagged template literal; this keeps this PR safe to merge into CoffeeScript 1.x. Remove the code from this commit to make all interpolated strings use ES2015 syntax, for CoffeeScript 2.
* Compiler now _doesn’t_ use template literals.
* Expand tagged template literal tests
* Move ‘Unexpected string’ error message tests into
tagged template literal section.
‘Unexpected string’ is not reported in these test
scenarios anymore. Instead, we error that the
prefixing literal is not a function.
* Don’t unwrap StringWithInterpolations.
Saw bug with program consisting of “#{2}” not
compiling with template literals. Root cause was
that Block.compileNode was unwrapping interpolated
string and so didn’t use compileNode logic at
StringWithInterpolations level.
* No need to bracket interpolated strings any more.
When interpolated string looks like `hello ${2}`,
no extract brackets are needed, as the `s mark the
beginning and end.
* Show html templating with tagged template literals
* Multiline should match multiline
* Comment out unnecessary `unwrap`, which is only needed for CoffeeScript 2 all-ES2015 syntax output
* Added support for for-from loop, see #3832
* for-from: remove extra newline and add support for ranges
* for-from: tidy up the lexer
* for-from: add support for patterns
* for-from: fix bad alignment
* for-from: add two more tests
* for-from: fix test "for-from loops over generators"
See explanation here: https://github.com/jashkenas/coffeescript/pull/4306#issuecomment-257066877
* for-from: delete leftover console.log
* Refactor the big `if` block in the lexer to be as minimal a change from `master` as we can get away with
* Cleanup to make more idiomatic, remove trailing whitespace, minor performance improvements
* for-from: move code from one file to another
* for-from: clean up whitespace
* for-from: lexer bikeshedding
* Move "own is not supported in for-from loops" test into error_messages.coffee; improve error message so that "own" is underlined
* Revert unnecessary changes, to minimize the lines of code modified by this PR
Rather than compiling splats to arrays built using `Array#concat`, splats
are now compiled directly to ES2015 splats, e.g.
f foo, arguments..., bar
[ foo, arguments..., bar ]
Which used to be compiled to:
f.apply(null, [foo].concat(slice.call(arguments), [bar]));
[foo].concat(slice.call(arguments), [bar]);
Is now compiled to:
f(foo, ...arguments, bar);
[ foo, ...arguments, bar ];
* changed jison acceptable versions
* added await support
* wrong function bug fix
* added tests for async/await
* invalid to have await, yield(from) in same function
* changed error handling and tests
* bug fix
* made error handling test more rigorous
* consolidated harmony test files
* added async constructor support and tests
* removed .orig files
* Fixed browser testing issue
* Minor cleanup
* Async test-suite and Cake support, simplified/removed funky tests
* Skip async/await tests when not supported in runtime
* cleanup
* Replaced polyfill with native JS async/await
* Oops
* Make 'async' reserved word
* Remove all async polyfills
* fix merge conflict
* make async testing opt-in
* restore test, remove confusing polyfill language
* Revert changes to test runners
* Only run async tests where async/await is supported (Node 7+ with --harmony, for now)
* remove 'async' from JS reserved words
* The async tests should use their own special async-capable version of `global.test`, which is only loaded for the async tests and only loaded by async-capable environments
* Reverting rename of `async`, it’s not a reserved word so there’s no longer a need for this change
* async test refactoring and additions
* oops
* sync
* better error reporting for `await`
* more stuff geoffrey wants
* fixed litcoffee tests
* change test title
This is an upstream port of https://github.com/decaffeinate/coffeescript/pull/10
See that PR for links to the issues that this fixes.
Just like OUTDENT and CALL_END tokens, close-curly-brace tokens can be generated
without having a real location, and if that position overlaps with a later
token, it can cause the AST to have bad location data. Just like the other two
token types, we now give `}` tokens the position of the previous real token,
which makes all AST nodes have reasonable locations.
* Eliminate wrapper around “bound” (arrow) functions; output `=>` for such functions
* Remove irrelevant (and breaking) tests
* Minor cleanup
* When a function parameter is a splat (i.e., it uses the ES2015 rest parameter syntax) output that parameter as ES2015
* Rearrange function parameters when one of the parameters is a splat and isn’t the last parameter (very WIP)
* Handle params like `@param`, adding assignment expressions for them when they appear; ensure splat parameter is last
* Add parameter names (not a text like `'\nValue IdentifierLiteral: a'`) to the scope, so that parameters can’t be deleted; move body-related lines together; more explanation of what’s going on
* For parameters with a default value, correctly add the parameter name to the function scope
* Handle expansions in function parameters: when an expansion is found, set the parameters to only be the original parameters left of the expansion, then an `...args` parameter; and in the function body define variables for the parameters to the right of the expansion, including setting default values
* Handle splat parameters the same way we handle expansions: if a splat parameter is found, it becomes the last parameter in the function definition, and all following parameters get declared in the function body. Fix the splat/rest parameter values after the post-splat parameters have been extracted from it. Clean up `Code.compileNode` so that we loop through the parameters only once, and we create all expressions using calls like `new IdentifierLiteral` rather than `@makeCode`.
* Fix parameter name when a parameter is a splat attached to `this` (e.g. `@param...`)
* Rather than assigning post-splat parameters based on index, use slice; passes test “Functions with splats being called with too few arguments”
* Dial back our w00t indentation
* Better parsing of parameter names (WIP)
* Refactor processing of splat/expansion parameters
* Fix assignment of default parameters for parameters that come after a splat
* Better check for whether a param is attached to `this`
* More understandable variable names
* For parameters after a splat or expansion, assign them similar to the 1.x destructuring method of using `arguments`, except only concern ourselves with the post-splat parameters instead of all parameters; and use the splat/expansion parameter name, since `arguments` in ES fat arrow functions refers to the parent function’s `arguments` rather than the fat arrow function’s arguments/parameters
* Don’t add unnamed parameters (like `[]` as a parameter) to the function scope
* Disallow multiple splat/expansion parameters in function definitions; disallow lone expansion parameters
* Fix `this` params not getting assigned if the parameter is after a splat parameter
* Allow names of function parameters attached to `this` to be reserved words
* Always add a statement to the function body defining a variable with its default value, if it has one, if the variable `== null`; this covers the case when ES doesn’t apply the default value when `null` is passed in as a value, but CoffeeScript expects `null` and `undefined` to act interchangeably
* Aftermath of having both `undefined` and `null` trigger the use of default values for parameters with default values
* More careful parsing of destructured parameters
* Fall back to processing destructured parameters in the function body, to account for `this` or default values within destructured objects
* Clean up comments
* Restore new bare function test, minus the arrow function part of it
* Test that bound/arrow functions aren’t overwriting the `arguments` object, which should refer to the parent scope’s `arguments` (like `this`)
* Follow ES2015 spec for parameter default values: `null` gets assigned as as `null`, not the default value
* Mimic ES default parameters behavior for parameters after a splat or expansion parameter
* Bound functions cannot be generators: remove no-longer-relevant test, add check to throw error if `yield` appears inside a bound (arrow) function
* Error for bound generator functions should underline the `yield`
* Add missing compiled files
* Compile computed properties to ES2015 equivalent
This is a fairly small change that simplifies the code generation for computed
properties as they're now generated in the object initializer like regular
properties.
* failing test case
* add markdown parser for literate coffeescript
this should help to handle ligitimate markdown with indentation correctly
* Update generated code
* Update package.json
* Add `marked` dependency to browser version of CoffeeScript
* Update invertLiterate to use a randomly-generated token that we check for uniqueness, rather than a magic number that we hope might not occur in the code
* Fix typos
This is an upstream port of https://github.com/decaffeinate/coffeescript/pull/9
The existing logic for computing the end location of a string was to take the
end of the string contents, then add the delimiter length to last_column. For
example, `"""abc"""` would have an end position three characters after the `c`.
However, if a string ended in a newline, then the end location for the string
contents would be one line above the end location for the string, so the proper
fix is to move the end location to the next line, not just to shift it to the
right.
This avoids a bug where the location data would sometimes reference a
non-existent location (one past the end of its line). It fixes the AST location
data, although as far as I know, it never has caused correctness issues in the
CoffeeScript output.
* Node 6 deprecated `new Buffer` in favor of `Buffer.from` and `Buffer.alloc`; update our calls, and set the required version of Node to be >= 6.9.0, and set this to be 2.0.0-alpha for now
* Bump to Node version 6.9.1
This is an upstream port for the patch https://github.com/decaffeinate/coffeescript/pull/8
See https://github.com/decaffeinate/decaffeinate/issues/291 for the bug that this fixed.
For the most part, CoffeeScript and JavaScript have the same precedence rules,
but in some cases, the intermediate AST format didn't represent the actual
evaluation order. For example, in the expression `a or b and c`, the `and` is
evaluated first, but the parser treated the two operators with equal precedence.
This was still correct end-to-end because CoffeeScript simply emitted the result
without parens, but any intermediate tools using the CoffeeScript parser could
become confused.
Here are the JS operator precedence rules:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence
For the most part, CoffeeScript already follows these. `COMPARE` operators
already behave differently due to chained comparisons, so I think we don't need
to worry about following JS precedence for those. So I think the only case where
it was behaving differently in an important way was for the binary/bitwise
operators that are being changed here.
As part of this change, I also introduced a new token tag, `BIN?`, for the
binary form of the `?` operator.
Fixes https://github.com/decaffeinate/decaffeinate/issues/446
In addition to OUTDENT tokens, CALL_END tokens can also be virtual tokens
without a real location, and sometimes they end up with a location that's
incorrect.
This commit adds another post-processing step after normal lexing that sets the
locationData on all OUTDENT tokens to be at the last character of the previous
token. This does feel like a little bit of a hack. Ideally the location data
would be set correctly in the first place and not in a post-processing step, but
I tried that and some temporary intermediate tokens were causing problems, so I
decided to set the location data once those intermediate tokens were removed.
Also, having this as a separate processing step makes it more robust and
isolated.
This fixes the problem in https://github.com/decaffeinate/decaffeinate/issues/371 .
In that issue, the CoffeeScript tokens had three OUTDENT tokens in a row, and
the last two overlapped with the `]`. Since at least one of those OUTDENT tokens
was considered part of the function body, the function expression had an ending
position just after the end of the `]`.
OUTDENT tokens are sort of a weird case in the lexer anyway, since they often
don't correspond to an actual location in the source code. It seems like the
code in `lexer.coffee` makes an attempt at finding a good place for them, but in
some cases, it has a bad result. This seems hard to avoid in the general case.
For example, in this code:
```coffee
[->
a]
```
There must be an OUTDENT between the `a` and the `]`, but CoffeeScript tokens
have an inclusive start and end, so they must always be at least one character
wide (I think). In this case, the lexer was choosing the `]` as the location,
and the parser ended up generating correct location data, I believe because
it ignores the outermost INDENT and OUTDENT tokens. However, with multiple
OUTDENT tokens in a row, the parser ends up producing location data that is
wrong.
It seems to me like there isn't a solid answer to "what location do OUTDENT
tokens have", since it hasn't mattered much, but for this commit, I'm defining
it: they always have the location of the last character of the previous token.
This should hopefully be fairly safe because tokens are still in the same order
relative to each other. Also, it's worth noting that this makes the start
location for OUTDENT tokens awkward. However, OUTDENT tokens are always used to
mark the end of something, so their `last_line` and `last_column` values are
always what matter when determining AST node bounds, so it is most important for
those to be correct.
`"""` (and `"`) strings are lexed into an array of tokens, consisting of
strings and interpolations. Previously, the minimum indententation
inside `"""` strings was stripped from the beginning of _all_ of those
string tokens. Usually, the indentation is longer than any other
sequence of spaces in a `"""` string, so the problem didn't occur in
most cases. This commit makes sure to only strip indentation after
newlines.
Fixes#4314.
Very large decimal number literals, binary number literals and octal
literals are lexed into an INFINITY token (instead of a NUMBER token)
and compiled into `2e308`. That is is supposed to be the case for very
large hexdecimal dumber literals as well, but previously wasn't.
Before:
$ node -p 'require("./").tokens(`0x${Array(256 + 1).join("f")}`)[0][0]'
NUMBER
After:
$ node -p 'require("./").tokens(`0x${Array(256 + 1).join("f")}`)[0][0]'
INFINITY
This commit also cleans up `numberToken` in lexer.coffee a bit.
`isLiteralArguments` mistakenly looked at `Literal`s instead of
`IdentifierLiteral`s.
This also gets rid of the ugly `.asKey` hack in nodes.coffee.
Fixes#4320.
