This implements a first generation of Windows CI testing. Presently,
this only runs valuable, hand-picked tests which have been known to work
in the past, and whose failure would indicate a critical problem.
A test which isn't passing doesn't mean that the feature being tested is
not working. For example, the 'create' test fails ostentatiously,
though the 'meteor create' command certainly works in practice. This
points to problems some compatibility problems with the 'self-test'
harness itself, some of which I'm aware of. Though, it likely will
highlight some legitimate problems which Windows users experience too.
There are a number of additional tests which should be enabled which
likely pass already, and many more which are failing and we should fix.
Additional tests can be added to the scripts/windows/appveyor/test.ps1
file as they've been deemed working.
Altogether, this will take extensive work to achieve the same level of
coverage our Unix test suite enjoy, but we've got to start somewhere!
cc @benjamn
This was dangerous because source is often a path relative to the old
target file, whereas files.stat was interpreting source as a path relative
to process.cwd().
Fixes#9203.
A while back, for performance reasons, we disabled yielding for all
files.* operations unless METEOR_DISABLE_FS_FIBERS was set to false.
This was safe for almost all files.* operations, because most of them have
a synchronous fs.*Sync version available.
For a more complicated operation like files.rm_recursive, however, there
is no synchronous or asynchronous counterpart in the fs.* namespace, so
the safety of disabling fibers is not guaranteed.
Lately, files.rm_recursive has become a major source of uncaught ENOTEMPTY
errors on Windows, because rimraf.sync fails with that error, and we don't
give files.rm_recursive_async a chance to delete the directory in a more
persistent, forgiving manner.
The only reason we haven't been falling back to files.rm_recursive_async
is that YIELD_ALLOWED is false by default, so canYield() returns false.
This commit distinguishes between canYield() and mayYield(), and uses
canYield() in files.rm_recursive to determine whether it is technically
safe to yield, regardless of YIELD_ALLOWED.
Anyone who ever asked "Can I go to the bathroom?" in elementary school,
only to be mercilessly rebuked with "I don't know, CAN YOU?" should
understand the difference between these two functions.
This means running `meteor update` on Windows with an existing (pre-1.6)
installation of Meteor will install a 32-bit version of Meteor 1.6, even
if the underlying machine architecture could handle a 64-bit version.
To get a native 64-bit version of Meteor, Windows developers should
reinstall Meteor 1.6 from scratch using the new and improved Windows
installer (👋 @abernix), rather than running `meteor update`.
If we really want to make `meteor update` just work ✨, we can
revisit this change in Meteor 1.6.1.
Whatever benefits this may have had, a general purpose function like
files.extractTarGz is definitely not the place for such an aggressive
safety measure, and the performance penalty is significant, especially on
Windows.
Though this seems wasteful, it can happen if a SpringboardToLatestRelease
exception is thrown, and it definitely should spawn a new process rather
than returning normally from the springboard function.
Since we can't change the springboarding logic of existing Meteor
releases, this decision has to be made by the springboarded-to Meteor 1.6
release, and may result in a second springboarding.
Once Windows developers update to a 64-bit version of Meteor 1.6, they may
still wish to run apps using older versions of Meteor where only 32-bit
builds of meteor-tool are available. This commit makes that possible.
The Generate Dev Bundle process no longer requires any modifications
to the $PATH, preventing environmental artifacts which pile up when
running the script over and over again.
I thought it was nifty to use this Cmd-let from PowerShell, to directly
parse the JSON output from Node's `process.release`, but as it turns
out, that wasn't supported by the version of PowerShell on our Jenkins
server. Alas!
This is a re-write of the generate-dev-bundle.ps1 script, which occurred
during debugging of an unrelated concern of the (new) 64-bit Windows
build on our Jenkins server. Ultimately, I'm afraid this script doesn't
solve the problem I originally set out to fix, which was a Windows
long-file path concern.
The hunch behind that thinking was that the use of npm@1 to install
npm@5 could be causing problems, since npm@1 had no concept of nested
node_modules directories. We had used npm@1 because Node.js
for Windows hasn't always offered (via nodejs.org/dist/) versions
including npm which we could use to install our own requirements.
Happily, that is no longer the case!
While this script now deals with long paths much more gracefully itself,
I'm not sure it completely quelled the long-path issue, and there are
still some directory trees which are longer than I think they should be.
The warnings I was seeing may not have harmed the actual bundle and were
more problematic for this build script itself when it tried to deal with
the aftermath of all those files, since native Windows commands struggle
to deal with long file paths (when cleaning up, etc.).
In the end, this script does have performance enhancements though! For
starters, it's nearly twice as fast at finishing. Most of this was
gained by avoiding back-and-forth moving of large file structures,
opting instead to directly install into the targets when possible.
It also ensures that the npm build cache is not bundled, which started
occurring since our modification of the $HOME and $USERPROFILE variables
led npm@5 to think the npm cache was in the root of the bundle.
Additionally, it no longer modifies the $PATH, in any way, during the
build. This became particularly helpful during testing when I found that
PowerShell maintained that $PATH in the environment of the host shell.
I'd like to say it increases readability of the script, which had
become a bit of a patchwork quilt, but that's YTBD and YMMV.
This is my first "complete" PowerShell script myself so it probably
still leaves some things to desired, formatting wise. Functionality-
wise, I hope it's improved.
The Windows 64-bit support is believed to be important enough to be
included in Meteor 1.6 since it fixes Meteor 1.6 issues on Windows.
While the original reason for 64-bit may have been Mongo, we'll leave
this as the 3.2.15 (but 64-bit!) version for now.
This commit reverts much of the work @hwillson had put in place for
meteor/meteor#9173, which made it possible for 32-bit and 64-bit
Mongo versions to exist in harmony within the same dev-bundle. That
hard work was not in vein though as it offered invaluable research.
Ultimately, this showed that a more aggressive approach would be ideal,
even if the proposed option would have worked great in the short-term.
In the wake of the news that Mongo would no longer be supporting 32-bit
versions, these changes are important so 32-bit users of Meteor can
continue to have a functioning Mongo binary in development, while still
allowing Meteor to ship newer Mongo (e.g. 3.4+) binaries for 64-bit
users. This is particularly relevant for Windows users, who have
previously only had 32-bit Meteor builds and represented a majority of
"32-bit" development, despite the fact most of their hosts supported
64-bit. During another time in Meteor's life, this made sense.
This commit takes improved functionality to the next level (and makes
the aforementioned changes obsolete) by introducing support for building
and shipping Meteor for Windows in a 64-bit flavor (in addition to
32-bit), which will hopefully solve a number of performance matters on
Windows by using binaries which are pre-compiled for 64-bit, rather than
forcing the Windows kernel to deal with 32-bit binaries. While Windows
has shown it's quite capable of dealing with such a situation, it seems
more and more clear (given improvements in underlying dependencies) that
performance gains could be had by freeing Windows of its 32-bit work.
This commit also further perpetuates the "archinfo" plot-line with more
story (about Windows) and various spelling corrections.
If any part of the dev bundle generation fails, the script should stop.
I was encountering a barely-noticed failure in the Python extraction
when the download tarball was 404-ing (due to my own fault), but the
script proceeded as if the failure hadn't happened, resulting in a
broken dev bundle.
These changes introduce dual Mongo support into the Meteor
Tool. 32-bit Mongo (3.2.15) will be used by Meteor when the
Tool is run on a 32-bit OS (32-bit Linux and Windows). 64-bit
Mongo (3.4.9) will be used when the Tool is run on a 64-bit
OS (64-bit Linux, Windows and macOS).
Fixes https://github.com/meteor/meteor-feature-requests/issues/129.
