While running the tests three times might make a lot of sence in CI
environments, it certainly slows things down in development, especially
when you've failed a test on purpose, but still have to wait for it to
run three times to get the output.
This not only implements the `--retries` flag, but also activates it in
both CircleCI and AppVeyor test suites.
While this was necessary in previous versions of Meteor, as of 1.6 the
`self-test` tooling will automatically install the dependencies it needs
at runtime.
Not only does this allow the installation to be avoided when tests using
those facilities aren't invoked, it also allows us to reliably install
the same version of these npms, rather than falling back to the
`latest` npm tag on each CI run.
This code had been used to compile the previous generation of
the Windows installer, `InstallMeteor.exe`, however it has been
replaced with the `meteor` Chocolatey package.
Refs: https://github.com/meteor/meteor-chocolatey-installer.
Currently, three 'dynamic import' tests exist in the self-test suite:
* dynamic import(...) in development
* dynamic import(...) in production
* dynamic import(...) with cache
These tests take quite some time to run; each test takes ~5 minutes.
While it's great to test them in a faster environment (Unix), which
Meteor will continue to do, the overall value of the three separate
tests is not as great in a slower testing environment, like Windows.
Even with $ErrorActionPreference, PowerShell won't automatically fail
when an external command fails with an error code. This explicitly
checks those exit codes and throws an error when that occurs.
Hopefully prevents false successes like that shown in this AppVeyor
test run: https://goo.gl/xxRsF9.
Though it was thought to be reliable when running through 'self-test' on
Windows, it's yet to be seen how reliable.
The worst thing that could come of adding Windows testing would be that
we have test failures again after such a string of green checkmarks and
confidence.
Also, reordered.
This implements a first generation of Windows CI testing. Presently,
this only runs valuable, hand-picked tests which have been known to work
in the past, and whose failure would indicate a critical problem.
A test which isn't passing doesn't mean that the feature being tested is
not working. For example, the 'create' test fails ostentatiously,
though the 'meteor create' command certainly works in practice. This
points to problems some compatibility problems with the 'self-test'
harness itself, some of which I'm aware of. Though, it likely will
highlight some legitimate problems which Windows users experience too.
There are a number of additional tests which should be enabled which
likely pass already, and many more which are failing and we should fix.
Additional tests can be added to the scripts/windows/appveyor/test.ps1
file as they've been deemed working.
Altogether, this will take extensive work to achieve the same level of
coverage our Unix test suite enjoy, but we've got to start somewhere!
cc @benjamn
The Generate Dev Bundle process no longer requires any modifications
to the $PATH, preventing environmental artifacts which pile up when
running the script over and over again.
This is a re-write of the generate-dev-bundle.ps1 script, which occurred
during debugging of an unrelated concern of the (new) 64-bit Windows
build on our Jenkins server. Ultimately, I'm afraid this script doesn't
solve the problem I originally set out to fix, which was a Windows
long-file path concern.
The hunch behind that thinking was that the use of npm@1 to install
npm@5 could be causing problems, since npm@1 had no concept of nested
node_modules directories. We had used npm@1 because Node.js
for Windows hasn't always offered (via nodejs.org/dist/) versions
including npm which we could use to install our own requirements.
Happily, that is no longer the case!
While this script now deals with long paths much more gracefully itself,
I'm not sure it completely quelled the long-path issue, and there are
still some directory trees which are longer than I think they should be.
The warnings I was seeing may not have harmed the actual bundle and were
more problematic for this build script itself when it tried to deal with
the aftermath of all those files, since native Windows commands struggle
to deal with long file paths (when cleaning up, etc.).
In the end, this script does have performance enhancements though! For
starters, it's nearly twice as fast at finishing. Most of this was
gained by avoiding back-and-forth moving of large file structures,
opting instead to directly install into the targets when possible.
It also ensures that the npm build cache is not bundled, which started
occurring since our modification of the $HOME and $USERPROFILE variables
led npm@5 to think the npm cache was in the root of the bundle.
Additionally, it no longer modifies the $PATH, in any way, during the
build. This became particularly helpful during testing when I found that
PowerShell maintained that $PATH in the environment of the host shell.
I'd like to say it increases readability of the script, which had
become a bit of a patchwork quilt, but that's YTBD and YMMV.
This is my first "complete" PowerShell script myself so it probably
still leaves some things to desired, formatting wise. Functionality-
wise, I hope it's improved.
This commit reverts much of the work @hwillson had put in place for
meteor/meteor#9173, which made it possible for 32-bit and 64-bit
Mongo versions to exist in harmony within the same dev-bundle. That
hard work was not in vein though as it offered invaluable research.
Ultimately, this showed that a more aggressive approach would be ideal,
even if the proposed option would have worked great in the short-term.
In the wake of the news that Mongo would no longer be supporting 32-bit
versions, these changes are important so 32-bit users of Meteor can
continue to have a functioning Mongo binary in development, while still
allowing Meteor to ship newer Mongo (e.g. 3.4+) binaries for 64-bit
users. This is particularly relevant for Windows users, who have
previously only had 32-bit Meteor builds and represented a majority of
"32-bit" development, despite the fact most of their hosts supported
64-bit. During another time in Meteor's life, this made sense.
This commit takes improved functionality to the next level (and makes
the aforementioned changes obsolete) by introducing support for building
and shipping Meteor for Windows in a 64-bit flavor (in addition to
32-bit), which will hopefully solve a number of performance matters on
Windows by using binaries which are pre-compiled for 64-bit, rather than
forcing the Windows kernel to deal with 32-bit binaries. While Windows
has shown it's quite capable of dealing with such a situation, it seems
more and more clear (given improvements in underlying dependencies) that
performance gains could be had by freeing Windows of its 32-bit work.
This commit also further perpetuates the "archinfo" plot-line with more
story (about Windows) and various spelling corrections.
This is in an attempt to resolve a situation which could occur when the
extraction of the dev_bundle.tar fails with an EOF error, but the tool
continues as if the file was fine. Ultimately the, error preference
set in the PS1 script is only observed in the case of PowerShell
ErrorLevel and not just an error condition encountered in a command.
The SAVE_DEV_BUNDLE_TARBALL environment variable is already supported in the Unix version and this mimics that functionality in the Windows script. Use of this flag makes debugging much less painful, especially when switching branches (for example, during a `git bisect`) as the dev_bundle can be cached locally.
I also replaced a couple un-related uses of `echo` with `Write-Host` which is the preferred way to output messages in PowerShell as it supports more options.
Fixes: developer experience
Even though I think the fight is a bit futile, it is considered best practice to have newlines at the end of all files. At the request of meteor/meteor#7786, this commit adds newlines to the end of the new app skeleton `.gitignore` file. These skel files are used when generating new meteor apps so this avoids perpetuation of this problem into futrue generations. ;)
In an effort to beef up this PR, I also took the liberty of fixing the few other Meteor files which were lacking newlines (all `.gitignore` files)
Closesmeteor/meteor#7786
Turns out there was unused code that dealt with strings
that may or may not have been initialized, and treated
them as multi-byte strings even though they were unicode
strings.
I don't actually understand why this would cause a failure
only on Windows Server 2012, but this change definitely fixed
the problem, and the code removed was unnecessary and wrong anyways.
While at it, improved error reporting in case we fail to
get the link to the bootstrap tarball, or download it.
Also fixed a bug that broke login/signup (due to passing a
char* instead of a wchar_t*)
Uses the https://packages.meteor.com/bootstrap-link
URL to generate a link to the windows bootstrap tarball.
With this change we no longer have to build a new installer
for every release.
This API actually lets us download the bootstrap tarball from cloudfront
on Windows Server, /and/ correctly reads the proxy settings from Internet
Explorer.
This is a step on the way to building an installer that always
fetches the latest release of Meteor, so that we don't need to
build a new installer for every release.
TODO:
* Expose a new endpoint on https://package.meteor.com with the latest
release, and consume it in the installer
* Use libcurl to fetch the installer, so that we can bypass the TLS
settings on Windows Server which by default don't let you fetch
HTTPS assets that use TLS 1.2 (as do ours)
* Automatically detect and use the proxy settings configured in
Internet Explorer, eg
http://stackoverflow.com/questions/202547/how-do-i-find-out-the-browsers-proxy-settings
This duplicates logic we had for Mac/Linux that lets you
set a USE_TEST_DEV_BUNDLE_SERVER environment variable
to try running a newly built dev bundle before publishing
it.
