This information is useful when you need a unique identifier for the
current version of the application (and you're using Git).
If the current Git HEAD revision can't be found for any reason, the
gitRevision property simply will not appear in star.json or
__meteor_runtime_config__.
Most directories are in the WatchSet at least twice, and the directory is read and each item is stat each time. In addition, when the watcher is not created by isUpToDate, each item in watchSet.directories and watchSet.files is checked twice.
With these changes, isUpToDate finishes in less than 1/2 the time on Windows, and creating a watcher takes around 1/4 the time.
Creating the watcher can take up to 12+ seconds in small - medium apps, and uses sync fs calls.
The server would start right away, but the tool process wouldn't know about it until the watcher finished setting up. Also, the proxy doesn't forward requests until "=> Server restarted" is shown.
A new async option is added to Watcher which prevents it from blocking the event loop too long.
Also, the watcher and legacy bundle are only created after the server has started, or 3 seconds has passed.
Follow-up to https://github.com/meteor/meteor/pull/9933.
As recommended by @abernix, the sha1 hash of every file is now computed
from the file's sha512 hash, so we don't have to hash the entire contents
of the file twice with two different algorithms.
Other changes/improvements:
* Invalidate the hashes when/if `File#setContents` is called.
* Ignore `options.hash` and just compute hashes from actual file contents.
Disagreement here would be worse than any performance benefits from
precomputing the hash.
When we copy node_modules directories during rebuilds, we try to create
symlinks instead of doing a full recursive copy. Very often, however, the
symlinks already exist, and they point to the right location already, so
we can avoid recreating them for even better performance.
Another optimization identified by using the qualia:profile package
created by @veered: https://atmospherejs.com/qualia/profile
Previously, the async fallback was only used in case of ENOTEMPTY errors,
but Windows appears to throw temporary EPERM errors as well. If the errors
are actually robust/non-temporary, the async version will fail, too.
Should help with #9540.
* Prevent Tool crash when settings file contains BOM
The use of a byte order mark in a `--settings` file
crashes the Meteor Tool, when it attempts to JSON parse
the loaded settings data. Since a BOM is not required
(or recommended) when using UTF-8, this commit removes
it from the loaded settings data, before it is parsed.
Fixes#5180.
* Add PR link to History.md
* Update History.md
A while back, for performance reasons, we disabled yielding for all
files.* operations unless METEOR_DISABLE_FS_FIBERS was set to false.
This was safe for almost all files.* operations, because most of them have
a synchronous fs.*Sync version available.
For a more complicated operation like files.rm_recursive, however, there
is no synchronous or asynchronous counterpart in the fs.* namespace, so
the safety of disabling fibers is not guaranteed.
Lately, files.rm_recursive has become a major source of uncaught ENOTEMPTY
errors on Windows, because rimraf.sync fails with that error, and we don't
give files.rm_recursive_async a chance to delete the directory in a more
persistent, forgiving manner.
The only reason we haven't been falling back to files.rm_recursive_async
is that YIELD_ALLOWED is false by default, so canYield() returns false.
This commit distinguishes between canYield() and mayYield(), and uses
canYield() in files.rm_recursive to determine whether it is technically
safe to yield, regardless of YIELD_ALLOWED.
Anyone who ever asked "Can I go to the bathroom?" in elementary school,
only to be mercilessly rebuked with "I don't know, CAN YOU?" should
understand the difference between these two functions.
Whatever benefits this may have had, a general purpose function like
files.extractTarGz is definitely not the place for such an aggressive
safety measure, and the performance penalty is significant, especially on
Windows.
Native file watching is notoriously unreliable on several Windows file
systems (e.g. NTFS, since network file systems have trouble supporting
change notifications).
However, disabling native file watching for all Windows developers was
probably a step too far, since it *could* work just fine, and we still
have the fs.watchFile-based safety net, which no longer hogs idle CPU
cycles for unchanged files.
You can explicitly disable native file watching and use polling instead by
setting METEOR_WATCH_FORCE_POLLING to a truthy value.
Should help with #9175.
Meteor attempts to use native file watchers to detect changes as soon as
possible, but we also employ an fs.watchFile-based safety net to detect
changes by polling, which adds reliability on platforms with poor (or
nonexistent) support for native file watching.
However, fs.watchFile tends to consume more idle CPU cycles when many
files are watched, so we use a relatively long (5000ms) polling interval
to watch files that have not yet been changed. After we detect the first
change to a file, we promote it to a much shorter (500ms) interval and
attempt to start a native file watcher.
Even if we were able to create a native file watcher using the pathwatcher
library, it may not fire notifications reliably on some file systems, so
this commit keeps polling changed files at the higher frequency, rather
than lowering the polling interval back to 5000ms.
The number of files the developer has changed by hand should never come
close to the total number of files watched by Meteor, so keeping them at
the elevated polling interval should pose no problem for idle CPU (#9175).
These errors are especially harmful because they cause files.rename to
fall back to copying rather than atomically renaming, which is both much
slower and not even remotely atomic.
Calling unwatchFile may result in stopping the watcher before watchFile is
called, which then restarts it. This temporary stoppage appears to cause
change events to be missed sometimes. In particular, preventing this
stop/start with the acrobatics in this commit seems to fix recent
compiler-plugins.js test failures.
Ever since Meteor 1.3 first introduced a module system based on something
other than `Npm.require`, we've continued throwing missing module
exceptions that refer to `Npm.depends` and/or `Npm.require`, even if the
developer called `require` or used an `import` declaration. This commit
fixes that, so that all missing module exceptions look like 'Cannot find
module "module/name"'.
I also noticed recently that `Npm.require` is capable of returning modules
installed in `node_modules` directories completely outside the app, which
is bad news for development/production reproducibility. Fixed that too.
CC @hwillson who has spoken of deprecating `Npm.require` entirely, and
just using `require` everywhere, instead.
This is performance-sensitive code, and I think this extra (uncached) call
to files.statOrNull could be contributing to test timeouts on Circle CI.
Follow-up to #9030.
As outlined in #3854, if `meteor --settings` points to a
directory instead of a file, an application gets caught up
in an infinite rebuild loop. Since it was assumed only a
file would be configured via `--settings`, a file watcher
is created for the specified directory, and that watcher
is then triggered ad infinitum.
These changes prevent a file watcher from being created
on a `--settings` directory.
Should fix#8988 and #8942.
Now that #8866 is the default behavior, it can take up to 5000ms for
changes to files modified during the build process to be noticed.
Before #8866, when we called e.g. files.writeFile(path), a native file
watcher would notice the change immediately, almost always before the
build process read the file again. This was definitely racy, but we were
getting away with it consistently... until #8866.
I was able to reproduce the problem in #8988 by running
echo some-local-package-name >> .meteor/packages
in an app with a local package of the given name. After debugging the
endless rebuild cycle, I found that .meteor/versions was being rewritten
by files.writeFile during the build process, but the file watching system
was not noticing the change in time to prevent watch.isUpToDate from
returning true. The change was finally detected when restarting the
Watcher responsible for .meteor/versions, which of course triggered
another rebuild, so the same problem kept happening again and again.
When checking the `entriesByIno` Map to see if an `entry` already exists for
the specified inode, also check to make sure the found `entry` is only
re-used if the current file watcher path matches the returned path. This
makes sure new file watchers are created for moved files (so files with the
same inode), instead of attempting to re-use a file watcher that's watching
an invalid path.
