Note that the value of the "main" property must include the ".node"
extension explicitly, because I have yet to encounter an npm package with
a native "main" module that omits the ".node" extension, and I am loathe
to start calling files.stat(id + ".node") uselessly for every single
missing module.
Fixes#7947.
Meteor 1.4.2 has been mostly focused on improving rebuild times, but what
ultimately matters is the time from changing a file to being able to make
requests against the restarted server, and any code that runs before
server startup should be considered part of that critical path.
If you have a lot of packages, and they do non-trivial work on startup,
this commit should give you much better insight into where time is spent.
Credit to @stubailo for pushing me to provide Meteor developers with
better profiling tools, and thanks to @veered for this specific idea.
Note: because tools/tool-env/profile.js is now loaded as-is by boot.js,
certain ECMAScript features are off-limits, e.g. ...rest params.
This test has been the primary source of test suite unreliability in
recent weeks. By marking it "slow" I'm effectively disabling it on
CircleCI, which I think is acceptable because (1) it is, in fact, quite
slow; (2) it passes locally on Linux; and (3) it depends on the timing of
file change notifications, which is a recipe for test flakiness due to the
general non-determinism of file watching, but not necessarily a sign of a
broken developer experience.
Like other "slow" tests, I will keep running this test locally via
meteor self-test --slow 'javascript hot code push'
as part of the manual QA testing that we do.
It's a shame that Pathwatcher issues this warning using console.error,
without taking any verbosity options into account:
https://github.com/atom/node-pathwatcher/blob/7ef76e5dfd/src/main.coffee#L53
Fortunately, I believe I've identified the underlying reason why this
happens, which may help resolve the following issue:
https://github.com/atom/node-pathwatcher/issues/98
If all goes well, I'll submit an upstream pull request.
I've also reinstated an old file watching test that I mistakenly removed
when I attempted to switch to chokidar instead of pathwatcher.
This is a bit different from the previous strategy of invalidating
optimistic functions for specific npm package names.
Now, whenever we make changes to the contents of a specific node_modules
directory, or whenever the developer independently modifies an app's
node_modules directory, all optimistic results derived from paths
contained within that node_modules directory will be marked as dirty, and
thus may need to be recomputed.
This strategy prioritizes starting fewer watchers (just one per
node_modules directory) while still allowing npm packages to be added or
removed while the app is running:
https://github.com/meteor/meteor/pull/7668#issuecomment-255120373
The drawback is that changes within subdirectories of node_modules will
not be detected until the server is fully restarted, but that seems like
an acceptable tradeoff, since npm packages change much less often than
application code.
This reverts commit b69716d778.
Now that we have a better system for invalidating optimistic results
derived from node_modules paths, these optimizations are safe (and yes
fourseven:scss rebuilds successfully).
This fixes a bug that prevents fourseven:scss from properly rebuilding,
because the new .../node_modules/node-sass/vendor/<platform>/binding.node
file is not found by Builder#copyDirectory, because the cached results of
optimisticReaddir are the same as before the rebuild.
Unfortunately, this change introduces a small performance regression
(hundreds of milliseconds at worst), because these files.* methods are
called many times.
I think we can continue using optimistic functions here if we are more
careful about invalidating their results, especially after calling
meteorNpm.rebuildIfNonPortable, but I'll save that for a future commit.
When a download aborts prematurely, the status code is often 200 OK, even
though we probably should not proceed with any further processing of the
downloaded information.
This silent failure leads to problems like the dreaded "Error: ENOENT: no
such file or directory, open... os.json" (#7806 and others), which were
hard to diagnose properly because the failure occurred only later, when
extracting a buffer that downloaded incompletely.
The getUrlWithResuming helper should be able to retry after this error is
thrown, which will result in a more helpful warning, even if in the most
common case, i.e. MaxCDN failure, it will never actually succeed.
Note that this change will not help until Meteor 1.4.2 is officially
released and becomes the implementation used to download later releases.
Mitigates #7806.
File#computeAssignedVariables is one of the most expensive methods called
during initial startup. This change significantly reduces the number of
times it needs to call through to findAssignedGlobals, which saves quite a
bit of parsing time. The exact savings are hard to quantify, of course,
because they depend entirely on how many modules you have in your app's
node_modules directory.
Scanning for global variable assignments is only really useful in Meteor
packages, where we sometimes need to intercept assignments for the
purposes of api.export, though this is increasingly unnecessary now that
you can (and should) just import values from node_modules.
In the app, the only possible value of intercepting global variable
assignments was to prevent polluting the global scope, but we don't even
create a private scope for the app when useGlobalNamespace is true, so
there really was no point to scanning app node_modules.
This is a refinement of my previous commit 56c041c858.
The Meteor babel-runtime package does not provide substitutes for
babel-runtime/core-js/* modules, so we should not be discarding them from
the client bundle.
Fixes#7930.
If a test process does not explicitly call process.exit, pathwatcher
watchers may keep it alive indefinitely (either that, or there's a bug
with the persistent:false option to fs.watchFile).
This accidental immortality can be prevented by explicitly closing all
watchers when we no longer have any interest in file change notifications.
This is a refactor/finishing-move of @c9s's original PR meteor/meteor#7586.
This maintains backward compatibility with PACKAGE_DIRS using the original separator that it expected (':'). Users of the new METEOR_PACKAGE_DIRS variable should use the correct path separator for their platform (`:` on BSD/Linux and `;` on Windows).
Fixesmeteor/meteor#7585Fixesmeteor/meteor#4204
In particular, optimisticReaddir was getting called before relevant dirty
callbacks were firing, and thus failing to notice actual changes on the
file system.
In general, the tools/fs/watch.js code is the one place where we really
need an up-to-date view of the file system. Put another way, if optimistic
functions worked perfectly, we wouldn't need to rely so much on WatchSet
logic, but for now it's a balance of equally important strategies, and we
shouldn't be compromising one by intermingling it with the other.
This will make it easier to use tools like https://yarnpkg.com/ with the
right version of Node, etc.
With this commit, here's all you have to do:
meteor npm install -g yarnpkg
Then test that it works:
meteor yarn info
Note that any commands registered by Meteor itself will not be honored.
This partially reverts commit 96c95629eb.
When running `meteor update`, we call writeReleaseFileAndDevBundleLink
immediately before reading .meteor/release, so there's no time for a file
change notification to invalidate the cached contents of that file.
In the future, we could perhaps call optimisticReadFile.dirty(path) as a
consequence of files.writeFile(path, ...), but that may be tricky.
Judging from the variety and extent of test failures, switching to
chokidar.watch was too drastic a change for this late-beta stage of the
release cycle.
The problem with pathwatcher.watch was that watches don't survive the
deletion of the watched file, because (like fs.watch) it watches files
based on inodes, not paths. This problem can be solved in a relatively
narrow way, by attempting to rewatch the file after any "delete" or
"rename" events.
Especially on Linux, chokidar.watch tends to throw ENOSPC errors when too
many files are being watched.
I removed this logic earlier because I thought chokidar could handle such
failures by itself, but that was wishful thinking.
Healthy competition among fs.watch wrappers appears to have produced a
clear winner: https://www.npmjs.com/package/chokidar
This wrapper is better for Meteor than pathwatcher was, because it can
watch directory trees recursively, and it has no trouble watching
nonexistent file paths, whereas pathwatcher would throw an exception.
As of optimism@0.3.0, the value of `this` within subscribe functions is no
longer the optimistic wrapper function object (instead: null or global, as
if called with no receiver object), so we need to retain a reference to
the optimistic function so that the watcher can call .dirty(...args).
