Commit Graph

7138 Commits

Author SHA1 Message Date
Ben Newman
419ff7ce92 Merge pull request #10399 from zodern/windows-build-performance
Windows build performance
2019-01-31 12:02:41 -05:00
zodern
ead0a6de1f Fix indentation 2019-01-17 16:12:15 -06:00
zodern
31dfb0cc06 Remove timeout for starting delayed build 2019-01-17 16:09:21 -06:00
zodern
f9d17aaf0f Fix delaying render 2019-01-16 22:17:28 -06:00
zodern
285e1b50c3 Clean up code 2019-01-16 22:16:51 -06:00
zodern
65bc2b0100 Fix reusing symlinks from initial build 2019-01-16 20:21:00 -06:00
Ben Newman
a89b34c4c2 Remove hash of minimum modern browser versions from client hash.
This partially reverts commit 99b79dc00f,
which was added as part of PR #10055 in an effort to trigger hot reloads
on the client when/if the definition of a "modern" browser happened to
change, due to server code calling setMinimumBrowserVersions. Although
changes in the minimum modern browser versions are pretty rare, it seemed
important to incorporate this information into the client hash, because
code sent to the client tends to be dramatically different depending on
whether the client is considered modern.

However, this change was made without updating the corresponding version
calculations in CordovaBuilder#appendVersion in tools/cordova/builder.js,
so the versions in program.json for Cordova apps disagreed with the
versions served in manifest.json by the web server, leading to the
problems described by @lorensr in this cordova-plugin-meteor-webapp issue:
https://github.com/meteor/cordova-plugin-meteor-webapp/issues/69

It would be nice to include the minimum versions hash in program.json for
Cordova builds, but unfortunately these versions are not known at build
time, because they are determined by calls to setMinimumBrowserVersions
during server startup. In other words, if we wanted to access that
information during Cordova builds, we would have to start the web server
and run all server-side application initialization code just to find out
if setMinimumBrowserVersions was called anywhere.

In the future, we could consider including the minimum versions hash in
manifest.json, so cordova-plugin-meteor-webapp could compare the current
version to the new version whenever it fetches manifest.json. However, I
think simply removing the minimum versions hash from the client version
calculation is a fine solution in the meantime. If a developer needs to
trigger a hot reload because they changed their minimum modern versions,
they should just be sure to change their client code at the same time.
Any change that would normally trigger a client reload will work.
2019-01-14 20:41:16 -05:00
Ben Newman
5e26ebb6a9 Update cordova-{android,ios} to latest versions.
This version of cordova-android includes the PR that previously required
us to fork the package: https://github.com/apache/cordova-android/pull/417

The cordova-ios update is just 4.5.4 => 4.5.5, so hopefully entirely
backwards compatible. :crossed-fingers:
2019-01-14 15:58:10 -05:00
Ben Newman
cdb5854662 Merge branch 'devel' into release-1.8.1 2019-01-14 10:42:51 -05:00
Ben Newman
20da99c219 Do not treat client and server directories specially in packages. (#10414)
Fixes #10393.

Bumping compiler.BUILT_BY and LINKER_CACHE_SALT because
PR #10414 changes the behavior of the build system in a subtle way that
does not automatically trigger recompilation.
2019-01-11 16:52:23 -05:00
Ben Newman
f4485fe807 Use build.json to opt out of incompatible Xcode 10 build system.
https://github.com/apache/cordova-ios/issues/407
2019-01-11 12:14:51 -05:00
Ben Newman
991ff6d620 Merge branch 'devel' into release-1.8.1 2019-01-10 11:32:02 -05:00
Ben Newman
dea96ecac6 Fix #10409 by ignoring self-referential browser aliases in package.json. 2019-01-10 11:31:49 -05:00
Ben Newman
e25415927f Merge branch 'devel' into release-1.8.1 2019-01-06 19:10:45 -05:00
Ben Newman
fb2146cb3b Stop excluding test modules when meteor.testModule found in package.json. (#10402)
New Meteor apps have the following meteor.testModule in their package.json
files by default

  "meteor": {
    "testModule": "tests/main.js"
  }

When meteor.testModule is defined, it determines the test entry point when
running the `meteor test` command, ignoring legacy file naming conventions
like *.tests.js or *.app-tests.js.

The package-source.js code changed by this commit was incorrect because it
ignored those specially-named test files even when running tests, which
was a problem if the meteor.testModule tried to import them explicitly,
because they would not be properly compiled.

If you're using meteor.testModule, the distinction between `meteor test`
and `meteor test --full-app` matters a bit less, since the test entry
point will be the same for both modes, though you can still check
Meteor.isTest and Meteor.isAppTest at runtime to control test behavior.
2019-01-06 15:02:48 -05:00
Ben Newman
5d88d9a1a4 Improve stack traces for self-test failures. 2019-01-05 16:54:38 -05:00
Ben Newman
aaeb2a7c2c Pass --enableFreeMonitoring off to Mongo 4 shell, except on Linux. 2019-01-05 16:27:20 -05:00
Ben Newman
c7441e68c6 Ensure consistent LANG-related environment variables in findMongoPids.
For some reason, without all three of these environment variables set
(LANG, LC_ALL, and LANGUAGE), the STDOUT returned from the child process
in findMongoPids contained ?? in place of non-ASCII unicode characters,
which was causing the self-test of Mongo shell in a unicode application
directory to fail.

This implementation defaults all three environment variables to
process.env.LANG if it was defined, or "en_US.UTF-8" otherwise.
2019-01-05 15:05:34 -05:00
Ben Newman
bcf6bf2cfc Wait until test app is running before starting Mongo shell. 2019-01-05 15:04:30 -05:00
Ben Newman
61b2ac3878 Revert "Use --enableFreeMonitoring off instead of --quiet for Mongo shell."
This reverts commit 7055780697.

While this flag works as expected on MacOS, it appears not to be supported
on Linux, even using MongoDB 4.0.5 (the latest version):
https://jira.mongodb.org/browse/SERVER-38862
2019-01-05 13:17:33 -05:00
Ben Newman
7055780697 Use --enableFreeMonitoring off instead of --quiet for Mongo shell.
Previously: 7f7a987251

cc @klaussner @mitar
2019-01-05 12:45:24 -05:00
Ben Newman
6c87f68116 Fix mongo tests by adding a package.json to standard-app. 2019-01-05 12:45:22 -05:00
zodern
ca3d3911a9 Clean up watch code 2019-01-05 07:55:02 -06:00
zodern
a985bf4b22 Throttle rendering progress in updateProgress 2019-01-04 22:55:52 -06:00
zodern
70772dcae7 Reduce duplicate fs calls in watcher
Most directories are in the WatchSet at least twice, and the directory is read and each item is stat each time. In addition, when the watcher is not created by isUpToDate, each item in watchSet.directories and watchSet.files is checked twice.

With these changes, isUpToDate finishes in less than 1/2 the time on Windows, and creating a watcher takes around 1/4 the time.
2019-01-04 22:47:08 -06:00
zodern
5c86db22a7 Fix files.readdir and files.realpath not being cached
enableCache was called before readdir and realpath was added to "files".
2019-01-04 18:40:39 -06:00
Ben Newman
4859f426dd Adjust timeouts to help Mongo self-tests pass reliably. 2019-01-04 14:13:25 -05:00
zodern
e04c87adfe Clean up old symlinks 2019-01-01 12:08:35 -06:00
zodern
21f976d6f0 Fix delay between server starting and showing "=> Server restarted"
Creating the watcher can take up to 12+ seconds in small - medium apps, and uses sync fs calls.
The server would start right away, but the tool process wouldn't know about it until the watcher finished setting up. Also, the proxy doesn't forward requests until "=> Server restarted" is shown.
A new async option is added to Watcher which prevents it from blocking the event loop too long.
Also, the watcher and legacy bundle are only created after the server has started, or 3 seconds has passed.
2019-01-01 11:47:20 -06:00
zodern
2ae55e8664 Fix caching meteorNpm.isPortable on windows
The path was in the wrong format, so .meteor-portable.json was never saved.
2018-12-31 22:03:15 -06:00
zodern
92f84293b5 Add caching to _copyDirectory
This is most noticible in copyNodeModulesDirectory, which is called many times during each server build.
2018-12-31 22:01:36 -06:00
zodern
21e90bf18d Use in place builds on windows for the server
Since the server is always stopped when building the server, it should be safe to overwrite it's files.
2018-12-31 21:49:53 -06:00
zodern
99f969b930 Do not calculate rebuild dirs in "meteor run"
It is never used in the bundles created by "meteor run", and adds 0.5 - 1.5 seconds per build for smaller apps.
2018-12-31 21:46:52 -06:00
zodern
e14ce5b908 Do not atomicallyRewriteFile when not building in place 2018-12-31 21:38:26 -06:00
zodern
9c3385f542 Normalize paths before comparing them in symlinkWithOverwrite
In windows, source is in posix format and missing a trailing slash, causing the symlink to always be recreated.
2018-12-31 21:29:19 -06:00
zodern
c2ef3776a8 Delete garbage directory async 2018-12-31 21:27:20 -06:00
Ben Newman
905941fc2d Merge branch 'devel' into release-1.8.1 2018-11-23 12:48:04 -05:00
Ben Newman
0f5bed4707 Avoid passing lots of ...args to watch.sha1. 2018-11-20 17:47:49 -05:00
Ben Newman
346d512b13 Propagate input hashes all the way through bundling.
Hashes have a number of overlapping but not entirely redundant or
equivalent purposes within the build system.

Hashes of source code are important because they can be computed before
compilation and processing, and thus are useful as keys for caching that
expensive work. Source hashes remain useful even after compilation, as a
way of reflecting the contributions of source-code-sensitive assets like
source maps.

However, source hashes do not tell the whole story, and using them as
cache keys can be risky if the work that's being cached depends on
generated code rather than source code, as we recently discovered with the
findImportedModuleIdentifiers function. The preliminary fix for that
problem (#10330) was to cache findImportedModuleIdentifiers using a hash
of the generated code rather than the source hash.

PR #10330 swung a bit too far in the direction of ignoring source hashes
and considering only hashes of generated code. For example, the URLs of
source maps share the hash of the corresponding resource, but source maps
can change (because of superficial changes in the source code) without
changing the generated code of the resource. Ignoring the source hash when
computing source map URLs resulted in stale source maps with incorrect
line numbers.

A better solution seems to be to propagate the source hash (along with any
hashes of intermediate generated artifacts) all the way through bundling,
so that the final hash of any static resource reflects all information
that could/should change the behavior of that static resource, including
its source map, which embeds the exact source code of all contributing
files in the sourcesContent property. At every step of the way, we merge
all the input hashes into a single hash, so we don't have to keep juggling
multiple hashes, thankfully.

Sub-Resource Integrity (SRI) hashes still need to be computed from just
the final contents of a given asset, so that the browser can verify those
contents without knowing anything about the Meteor build system, but
that's handled separately.
2018-11-20 11:19:42 -05:00
Ben Newman
7ce3ca29fd Avoid computing servePath from undefined targetPath.
https://github.com/meteor/meteor/issues/10337#issuecomment-439674497
2018-11-19 12:40:33 -05:00
Ben Newman
b55806f931 Relax precondition in ResourceSlot#addJavaScript.
https://github.com/meteor/meteor/issues/10337#issuecomment-439638590
2018-11-17 15:26:05 -05:00
Ben Newman
c67d40f7f1 Merge branch 'devel' into release-1.8.1 2018-11-15 15:56:54 -05:00
Ben Newman
5e7e809cd1 Implement ResourceSlot#_addDirectlyToJsOutputResources to fix #10337. (#10338) 2018-11-15 13:50:48 -05:00
Ben Newman
014dfab1ac Implement ResourceSlot#_addDirectlyToJsOutputResources to fix #10337. (#10338) 2018-11-15 11:10:47 -06:00
Ben Newman
d19ac6a5dd Merge branch 'devel' into release-1.8.1 2018-11-12 20:52:26 -05:00
Ben Newman
4294414506 Update meteor-babel to version 7.1.5. 2018-11-12 19:35:47 -05:00
Ben Newman
44e713f046 Ensure file.hash is always computed from sha1(file.data).
With the introduction of lazy compilation in Meteor 1.8, calling

  inputFile.addJavaScript({
    ...
    hash: inputFile.getSourceHash(),
    ...
  }, function () {
    return compiler.processFilesForTarget(inputFile);
  });

becomes problematic, since inputFile.getSourceHash() is usually different
from compiler.processFilesForTarget(inputFile).hash, because the latter is
computed from the compiled code, whereas the former is computed from the
source code.

For example, when we use file.hash to cache imported module identifiers in
ImportScanner#_findImportedModuleIdentifiers, we really need to be using
the hash of the compiled code, since a single source module can be
compiled in different ways. If we cache based on the source hash, there's
a risk of reusing the scanned imports from the web.browser version for the
web.browser.legacy version, which can lead to all sorts of problems that
are only apparent in legacy browsers.

The quick fix is easy enough: BabelCompiler can simply stop including a
hash in the eager options to inputFile.addJavaScript. This fix can be
published as a minor update to the babel-compiler and ecmascript packages.

The remaining changes in this commit add another layer of defense against
this problem, by ignoring any hash options provided by compiler plugins,
in favor of simply computing the hash from the compiled data buffer.
These additional changes will become available in the next release of
Meteor (likely 1.8.1).
2018-11-12 18:57:06 -05:00
Ben Newman
3a18f67ac2 Rely on native Node.js Promise implementation.
There's no reason to use a Promise polyfill in Node.js at this point.
2018-11-03 13:58:48 -04:00
Christian Klaussner
0e9660097d Update self-tests for mongo-free autoupdate 2018-11-03 13:32:01 -04:00
Ben Newman
c96278700d Fix compiler plugins self-tests by waiting for lazy compilation to finish. 2018-10-31 10:41:31 -04:00