- Add missing await when reading file on import-scanner and also wait when ensuring the packages dependencies are loading.
- The "create" command is running now!
- As the getters are asynchronous for some file operations (i.e file.data, file.hash and file.sourceMap), we need to set those values in a different way.
Remove deprecated code from before Meteor 1.0 that was documented via XXX COMPAT in comments. Move deprecated packages into the deprecated folder (deps, livedata, srp).
* Avoid modifying unibuild.watchSet in PackageSourceBatch._watchOutputFiles.
Should fix#10736 by preventing old hashes from remaining in
unibuild.watchSet, which was sometimes causing isUpToDate to fail
immediately upon restart.
* Regression test for issue #10736.
The previous implementation simply avoided calling watchSet.addFile for
potentially unused files, trusting that addFile would be called later if
the file was eventually used. However, this strategy left the contents of
watchSet.files incomplete for tasks such as IsopackCache._checkUpToDate,
which require full information about all files, even the ones that might
not be used by the bundle. The new strategy maintains metadata about
potentially unused files in a separate data structure, which will be
merged/cloned/serialized/deserialized along with other WatchSet data.
Unfortunately, this conversion triggered an error due to one of the
shortcomings of the Babel implementation of TypeScript:
SyntaxError: /tools/tool-env/profile.ts: Namespace not marked type-only declare. Non-declarative namespaces are only supported experimentally in Babel. To enable and review caveats see: https://babeljs.io/docs/en/babel-plugin-transform-typescript
278 | }
279 |
> 280 | export namespace Profile {
| ^
281 | export let enabled = !! process.env.METEOR_PROFILE;
282 |
283 | export function time<TResult>(bucket: string, f: () => TResult) {
at File.buildCodeFrameError (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/core/lib/transformation/file/file.js:261:12)
at transpileNamespace (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/plugin-transform-typescript/lib/namespace.js:25:25)
at PluginPass.TSModuleDeclaration (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/plugin-transform-typescript/lib/index.js:271:32)
at newFn (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/visitors.js:193:21)
at NodePath._call (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/path/context.js:53:20)
at NodePath.call (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/path/context.js:40:17)
at NodePath.visit (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/path/context.js:88:12)
at TraversalContext.visitQueue (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/context.js:118:16)
at TraversalContext.visitMultiple (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/context.js:85:17)
at TraversalContext.visit (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/context.js:144:19)
In PR #10585, I tried enabling full compiler plugin processing for any
modules imported from `node_modules` that were not otherwise handled by
compiler plugins, but doing that for all packages was just too slow, not
to mention potentially dangerous for modules whose code cannot be safely
recompiled by Babel.
The primary reasons for wanting to recompile node_modules are
1. to enable "native" ECMAScript `import`/`export` syntax (given that
Node.js still does not fully support ESM syntax yet), and
2. to compile standard syntax like `const`, `let`, classes, and arrow
functions for legacy browsers.
The first goal can be achieved by compiling unanticipated `node_modules`
code with Reify, which is much faster than Babel, in part because Reify
can avoid doing any parsing when the source contains no `import` or
`export` identifiers. This compilation is also cached on disk, so its
impact should only be felt during cold builds after a `meteor reset`.
The second goal can be accomplished using the new `package.json`
configuration option `meteor.nodeModules.recompile`, which causes the
recompiled packages to be handled by the normal compiler plugins system,
so that the `ImportScanner`'s fallback compilation is not necessary.
Before this option was introduced, this recompilation behavior could be
achieved by symlinking application directories into `node_modules`, but
that always felt like an esoteric hack.
Instead of merely supporting ECMAScript module syntax via Reify, we should
really be compiling unanticipated modules (typically within node_modules)
using the same logic that the rest of the application uses.
Note: this processing applies only to .js files for now, since that's what
the ImportScanner works with.
Should help with #10563.
Instead of compiling ESM syntax in node_modules using compiler plugins,
the ImportScanner can provide "native" support for ESM syntax by using
Reify to quickly compile just the import/export syntax in any imported
modules that were not already handled by compiler plugins.
Since this code runs every time the app is built, it should not matter
which version of Meteor was used to publish a package. Compared to the
previous implementation based on PackageSource#_findSources and unibuild
JSON files (#10545), this implementation should have far fewer
compatibility concerns, as well as being faster thanks to not processing
or compiling modules until the ImportScanner determines that they are
actually imported.
Though the number of files that get compiled by this system should be
relatively small for now, to maintain good performance, the results of the
compilation are cached on disk and in memory.
Fixes#10393.
Bumping compiler.BUILT_BY and LINKER_CACHE_SALT because
PR #10414 changes the behavior of the build system in a subtle way that
does not automatically trigger recompilation.
Hashes have a number of overlapping but not entirely redundant or
equivalent purposes within the build system.
Hashes of source code are important because they can be computed before
compilation and processing, and thus are useful as keys for caching that
expensive work. Source hashes remain useful even after compilation, as a
way of reflecting the contributions of source-code-sensitive assets like
source maps.
However, source hashes do not tell the whole story, and using them as
cache keys can be risky if the work that's being cached depends on
generated code rather than source code, as we recently discovered with the
findImportedModuleIdentifiers function. The preliminary fix for that
problem (#10330) was to cache findImportedModuleIdentifiers using a hash
of the generated code rather than the source hash.
PR #10330 swung a bit too far in the direction of ignoring source hashes
and considering only hashes of generated code. For example, the URLs of
source maps share the hash of the corresponding resource, but source maps
can change (because of superficial changes in the source code) without
changing the generated code of the resource. Ignoring the source hash when
computing source map URLs resulted in stale source maps with incorrect
line numbers.
A better solution seems to be to propagate the source hash (along with any
hashes of intermediate generated artifacts) all the way through bundling,
so that the final hash of any static resource reflects all information
that could/should change the behavior of that static resource, including
its source map, which embeds the exact source code of all contributing
files in the sourcesContent property. At every step of the way, we merge
all the input hashes into a single hash, so we don't have to keep juggling
multiple hashes, thankfully.
Sub-Resource Integrity (SRI) hashes still need to be computed from just
the final contents of a given asset, so that the browser can verify those
contents without knowing anything about the Meteor build system, but
that's handled separately.
