This was dangerous because source is often a path relative to the old
target file, whereas files.stat was interpreting source as a path relative
to process.cwd().
Fixes#9203.
A while back, for performance reasons, we disabled yielding for all
files.* operations unless METEOR_DISABLE_FS_FIBERS was set to false.
This was safe for almost all files.* operations, because most of them have
a synchronous fs.*Sync version available.
For a more complicated operation like files.rm_recursive, however, there
is no synchronous or asynchronous counterpart in the fs.* namespace, so
the safety of disabling fibers is not guaranteed.
Lately, files.rm_recursive has become a major source of uncaught ENOTEMPTY
errors on Windows, because rimraf.sync fails with that error, and we don't
give files.rm_recursive_async a chance to delete the directory in a more
persistent, forgiving manner.
The only reason we haven't been falling back to files.rm_recursive_async
is that YIELD_ALLOWED is false by default, so canYield() returns false.
This commit distinguishes between canYield() and mayYield(), and uses
canYield() in files.rm_recursive to determine whether it is technically
safe to yield, regardless of YIELD_ALLOWED.
Anyone who ever asked "Can I go to the bathroom?" in elementary school,
only to be mercilessly rebuked with "I don't know, CAN YOU?" should
understand the difference between these two functions.
This means running `meteor update` on Windows with an existing (pre-1.6)
installation of Meteor will install a 32-bit version of Meteor 1.6, even
if the underlying machine architecture could handle a 64-bit version.
To get a native 64-bit version of Meteor, Windows developers should
reinstall Meteor 1.6 from scratch using the new and improved Windows
installer (👋 @abernix), rather than running `meteor update`.
If we really want to make `meteor update` just work ✨, we can
revisit this change in Meteor 1.6.1.
Whatever benefits this may have had, a general purpose function like
files.extractTarGz is definitely not the place for such an aggressive
safety measure, and the performance penalty is significant, especially on
Windows.
Though this seems wasteful, it can happen if a SpringboardToLatestRelease
exception is thrown, and it definitely should spawn a new process rather
than returning normally from the springboard function.
Since we can't change the springboarding logic of existing Meteor
releases, this decision has to be made by the springboarded-to Meteor 1.6
release, and may result in a second springboarding.
Once Windows developers update to a 64-bit version of Meteor 1.6, they may
still wish to run apps using older versions of Meteor where only 32-bit
builds of meteor-tool are available. This commit makes that possible.
This commit reverts much of the work @hwillson had put in place for
meteor/meteor#9173, which made it possible for 32-bit and 64-bit
Mongo versions to exist in harmony within the same dev-bundle. That
hard work was not in vein though as it offered invaluable research.
Ultimately, this showed that a more aggressive approach would be ideal,
even if the proposed option would have worked great in the short-term.
In the wake of the news that Mongo would no longer be supporting 32-bit
versions, these changes are important so 32-bit users of Meteor can
continue to have a functioning Mongo binary in development, while still
allowing Meteor to ship newer Mongo (e.g. 3.4+) binaries for 64-bit
users. This is particularly relevant for Windows users, who have
previously only had 32-bit Meteor builds and represented a majority of
"32-bit" development, despite the fact most of their hosts supported
64-bit. During another time in Meteor's life, this made sense.
This commit takes improved functionality to the next level (and makes
the aforementioned changes obsolete) by introducing support for building
and shipping Meteor for Windows in a 64-bit flavor (in addition to
32-bit), which will hopefully solve a number of performance matters on
Windows by using binaries which are pre-compiled for 64-bit, rather than
forcing the Windows kernel to deal with 32-bit binaries. While Windows
has shown it's quite capable of dealing with such a situation, it seems
more and more clear (given improvements in underlying dependencies) that
performance gains could be had by freeing Windows of its 32-bit work.
This commit also further perpetuates the "archinfo" plot-line with more
story (about Windows) and various spelling corrections.
These changes introduce dual Mongo support into the Meteor
Tool. 32-bit Mongo (3.2.15) will be used by Meteor when the
Tool is run on a 32-bit OS (32-bit Linux and Windows). 64-bit
Mongo (3.4.9) will be used when the Tool is run on a 64-bit
OS (64-bit Linux, Windows and macOS).
Fixes https://github.com/meteor/meteor-feature-requests/issues/129.
The syntax of these flags is the same as the equivalent Node.js options:
https://nodejs.org/en/docs/inspector/#command-line-options
When no port value is provided, the default is 9229.
Two notable differences:
* The flags affect the server process spawned by the parent build
process, rather than affecting the build process itself.
* The --inspect-brk flag causes the server process to pause just after
server code has loaded but before it begins to execute. This timing is
more useful than the Node.js --inspect-brk behavior, which is to pause
on the first instruction executed by the process, since that is too
early to set any useful breakpoints.
Implements https://github.com/meteor/meteor-feature-requests/issues/194.
Native file watching is notoriously unreliable on several Windows file
systems (e.g. NTFS, since network file systems have trouble supporting
change notifications).
However, disabling native file watching for all Windows developers was
probably a step too far, since it *could* work just fine, and we still
have the fs.watchFile-based safety net, which no longer hogs idle CPU
cycles for unchanged files.
You can explicitly disable native file watching and use polling instead by
setting METEOR_WATCH_FORCE_POLLING to a truthy value.
Should help with #9175.
Meteor attempts to use native file watchers to detect changes as soon as
possible, but we also employ an fs.watchFile-based safety net to detect
changes by polling, which adds reliability on platforms with poor (or
nonexistent) support for native file watching.
However, fs.watchFile tends to consume more idle CPU cycles when many
files are watched, so we use a relatively long (5000ms) polling interval
to watch files that have not yet been changed. After we detect the first
change to a file, we promote it to a much shorter (500ms) interval and
attempt to start a native file watcher.
Even if we were able to create a native file watcher using the pathwatcher
library, it may not fire notifications reliably on some file systems, so
this commit keeps polling changed files at the higher frequency, rather
than lowering the polling interval back to 5000ms.
The number of files the developer has changed by hand should never come
close to the total number of files watched by Meteor, so keeping them at
the elevated polling interval should pose no problem for idle CPU (#9175).
These errors are especially harmful because they cause files.rename to
fall back to copying rather than atomically renaming, which is both much
slower and not even remotely atomic.
This reverts commit b9f0a54b39.
Though probably a good idea for the future, this change was not really
necessary for Meteor 1.6, and probably too risky for a release candidate.
The `installPath` property was always essentially an absolute module
identifier that was simply missing the leading '/' character, so this
commit acknowledges that role by renaming the property to `absModuleId`
and adding the leading slash.
Previously, if more than one module in a package tried and failed to
import the same identifier, we would record information about only the
last failed import.
This was good enough for later attempting to resolve the failed import in
other packges or the application's `node_modules` directory (a concept
known as "peer dependencies"), but it sometimes discarded information
about whether the failed imports were dynamic. In particular, if the last
recorded failed import was a dynamic import, it could accidentally render
the entire peer dependency tree dynamic.
Although it's a bit more complicated than what we did before, I believe
the simplest solution is for the ImportScanner to maintain a mapping from
failed identifiers to lists of import information objects, rather than a
single object, so that no information is lost.
The `meteor debug` command behaves like Node's `--inspect-brk` flag, in
that it attempts to pause the server before executing any server code.
However, simply passing the `--inspect-brk` flag to Node causes execution
to pause on the very first line of code, which is not good for setting any
breakpoints, because no server code has actually loaded yet.
Instead, the `meteor debug` command uses Node's `--inspect` flag to enable
debugging without an initial pause, then manually pauses at an appropriate
moment during server startup. Ideally, the pause should last until an
inspector client has been attached to the process, at which point the
developer has a chance to set any desired breakpoints, then clicks the
continue button to proceed with server startup.
The most difficult part of this process is detecting when the inspector
client has attached. Previously, the parent process listened for the child
process to print a "Debugger attached" message to STDERR, which happens as
a result of this `fprintf` call in Native C++:
7cff6e80bf/src/inspector_io.cc (L396)
However, this message was not printed in some cases, especially on Windows
(#9165), and required inter-process communication even in the ideal case.
All of that logic is gone now, thanks to this commit.
This commit takes advantage of a difference in behavior of the `debugger`
keyword depending on whether or not an inspector client is attached. When
no client is attached, the `debugger` keyword is a no-op that takes no
time (or very little time) to execute. Once a client has attached, the
`debugger` keyword triggers a breakpoint that lasts until the developer
explicitly continues execution through the client UI. Needless to say,
this makes the `debugger` keyword take longer than a no-op.
Because the `debugger` keyword does nothing until a client connects, we
can safely poll a `pause` function containing a `debugger` keyword at a
frequent interval (say, every 500ms). Once a client connects, the
`debugger` keyword will become active, pausing the server at exactly the
point we hoped. The difference is easy to detect by timing the `pause()`
function call. Once the `debugger` keyword becomes active, we stop polling
and allow server startup to continue.
Elegant!
Fixes#9165.
By my calculations, the sum of the sizes of the individual isopackets was
152MB, and the size of the combined isopacket is now just 36MB. That
remarkable difference goes to show how much duplication of transitive
dependencies was happening before this change.
That's a savings of 116MB for the (uncompressed) size of the meteor-tool
package. In Meteor 1.5.x, the meteor-tool package is about 544MB, but in
Meteor 1.6 it's considerably smaller: 373MB. In other words, this change
should reduce those sizes to 428MB (-21%) and 257MB (-31%), respectively.
