With Meteor 1.6 / Node 8, I noticed _buildLocalPackages taking multiple
seconds on initial server startup and restart, and the problem seems to be
that we call the global.gc function too often. This wasn't a problem in
previous versions of Node, as far as I know, but it makes sense to heed
the comment in tools/utils/gc.js, now that it matters.
This restores the behavior of 8c70716954 by
default, with the option of disabling the prioritized file watching system
by setting METEOR_WATCH_PRIORITIZE_CHANGED explicitly to "false".
The self-tests where the environment variable is explicitly set form a
nice to-do list of tests that should be improved to be more robust to cope
with differences in file watcher timing.
Helps with #8648 and similar issues.
Although I think 8c70716954 is a good idea
in practice, it altered the timing of self-tests enough to cause a number
of failures, so for now that behavior will be gated behind the environment
variable METEOR_WATCH_PRIORITIZE_CHANGED.
This drastically reduces the number of open file descriptors by not
preemptively acquiring a file descriptor for every watched file.
The downside is that the first time you edit a particular file, you may
have to wait up to DEFAULT_POLLING_INTERVAL milliseconds before Meteor
notices the change. The upside is that changes will be detected
instantaneously for that file in the future, even after restart.
If holding open too many file descriptors is indeed a contributing factor
to issues such as #8648, then this change should go a long way towards
mitigating those problems.
cc @abernix @hwillson
The root of the problem was that the es5-ext npm package contains
directories called '#', e.g.
https://github.com/medikoo/es5-ext/tree/master/array/%23
These directory names were being sanitized to '' and thus ignored when
reserving paths in the Builder, which led to reservation conflicts later.
This commit fixes the problem in three different and independently
sufficient ways:
* Use files.mkdir_p instead of files.mkdir when creating parent
directories of written files.
* Replace illegal characters in sanitized paths with '_' instead of ''.
* Allow '#' in sanitized paths (only needs to be escaped in the shell, not
actually forbidden in paths).
1a036553 in 1.4.4.2 expanded on the HTTP error checking added by 30aec9f in
1.4.2. Neither of these changes were aware that discoverGalaxy invokes
httpHelpers.request with json:true, resulting in a `body` that is a parsed JSON
object rather than a string or Buffer. Before 1.4.4.2, this had no consequences
because body.length is undefined and `undefined < 90` is false, but the change
to Buffer.byteLength actually made the condition true.
It's safe to not check length in the JSON case because a truncated JSON object
is not legal JSON (unless the truncation just drops trailing whitespace, in
white case that's OK).
I check for both string and Buffer because some calls to this function pass in
an encoding option. Buffer.byteLength works with both types.
This method appears to be causing large spikes in memory consumption on
Circle CI during the `meteor --get-ready` preparation step, which often
leads to the test process being killed.
Also added a call in IsopackCache#_loadLocalPackage for good measure.
We're now calling requestGarbageCollection pretty frequently when
we run Node with --expose-gc, but that currently only happens during
Circle CI tests, so I don't think we need to implement the improvements
suggested in tools/utils/gc.js, yet.
Previously: 35f488e140, f6df21ff1e
To deal with individual flaky tests, we often just re-run the entire test
suite, which feels like an enormous waste of shared computing resources.
This change automatically re-runs individual failed tests as many as two
more times, and considers the test successful if any of those attempts
succeeds.
cc @abernix @hwillson et al.
Windows began suffering from cryptic ENOTEMPTY and EPERM errors between
1.5-beta.20 and 1.5-rc.0. After a tricky `git bisect` adventure, I tracked
the problem down to my commit b6694b2f5d,
which caused dynamic modules to be written more than once by the bundler.
Though I don't understand exactly why Windows complained in this way, I'm
glad that it did, because otherwise this mistake would merely have been a
performance problem, and might not have been noticed before the release.
No longer using a RegExp when we know what the old file wildcard path
should be, and no longer using Fiber when we can just use a Promise
callback (since all Promise callbacks run in a Fiber).
