To deal with individual flaky tests, we often just re-run the entire test
suite, which feels like an enormous waste of shared computing resources.
This change automatically re-runs individual failed tests as many as two
more times, and considers the test successful if any of those attempts
succeeds.
cc @abernix @hwillson et al.
Now that the blaze submodule is mounted inside the packages/non-core/
directory, it appears that we no longer have to mention it explicitly in
the localPackageSearchDirs array.
This will make it much easier to fix bugs and make improvements going
forward, since they won't have to wait for the next release of Meteor.
One functional change: when the parent process exits, it no longer forces
all connected shell clients to disconnect, which is actually a more
convenient behavior, because it gives the clients a chance to reconnect
when/if the server starts up again, and it's easy enough to kill the
clients if that's what you want.
Saves 50MB from the extracted size of the dev_bundle/lib/node_modules
directory, previously 262MB. The entire dev bundle is 332MB extracted, for
perspective.
More importantly, since the `phantomjs` binary is not very compressible,
this change reduces the *compressed* size of the dev bundle from 71MB to
54MB, a 24% savings.
Closes#6905.
This never should have worked in Node v0.10.x, but with the v0.10.45
upgrade I stopped using npm@3 to build the Mac/Linux dev bundles (see
commit 916873ffe8f8ab6e922adfa68b8b8ab9c87db90d), which means we no longer
hoist the "process" npm package to the top level of the dev bundle
(dev_bundle/lib/node_modules/process), so now we're facing the expected
consequences of this mistake.
"universe" was an old attempt to allow you to run various MDG
servers (Meteor Developer Accounts, the package server, an old version
of the Galaxy deploy server, etc) on localhost and configure the tool to
talk to it not via a bunch of environment variables but via a file
called "universe" at the root of your checkout. Nobody uses this (and
most of the URLs have environment variables for them anyway). Simplify
the code by removing this entirely.
Also remove some more dead code, and a test that claims it only runs if
you have universe set up.
Mainly because the dev bundle is so big, it took upwards on 5 mins (on my MBP w/ SSD) to copy the warehouse packages into the sandbox. AFAICT there is no reason not to symlink it if we can.
Because longjohn was collecting frames across runs of self test in the loop, we were actually seeing frame lists that applied to more than one test.
This wasn't actually causing problems except in our frame parsing tests, but you can imagine that it would be quite unhelpful.
UX Changes:
- Added a `--galaxy` option to selftest to run tests against galaxy.
Self-test will NOT run those tests without this option. Self-test will not run any
other tests WITH this option.
The motivation here is two-fold:
- We want to make it easy to use self-test to test Galaxy without running a bunch
of extra tests. At least, we want this right now, while Galaxy is still in development.
- Galaxy is currently pretty unstable&slow; we don't want to run our regular test suite against
Galaxy all the time and slow down the tool development process. Additionally, the environment
variables required by Galaxy are a pain to set.
This is a TEMPORARY situation. Eventually, we will want to either merge the two tests together
(especially once we rotate out Mother...) or we will have a separate unrelated Galaxy test suite
and move a lot of this stuff here. With time, it will also become more obvious what a better default
(run Galaxy tests always/run Galaxy tests never/run non-Galaxy-specific tests against Galaxy) is.
The changes should be pretty self-contained.
- Added some number of environment variables to use with --galaxy tests. This is a bit complicated
and will be documented in
https://mdg.hackpad.com/GalaxyECS-Admin-tasks-RTXJ5YW8pDv#:h=Testing-Galaxy-with-selftest.
Walkthrough:
- galaxy-utls.js contains basic utils for running Galaxy tests. Use environment variables, etc.
- galaxy.js contains the acutal test
- simple-app is an app that responds to HTTP requests and serves some html that we can test to be running
- minor changes to config.js to allow overriding of the domain name
- minor changes to auth.js to allow us to automatically login as the (new) test user and use that
login to start a DDP collection.
- move a function out of deploy-settings.js into test-utils.js
- some minor additions to test-utils.js
Reviewed in https://github.com/meteor/meteor/pull/4997.
