mirror of
https://github.com/meteor/meteor.git
synced 2026-05-02 03:01:46 -04:00
540dc00230
* Add a self-test skip option Meteor's CI infrastructure is configured to exclude certain `self-test`'s on each run. These excludes are specified in each CI environment's config file, and included when running `meteor self-test`. Developers running `meteor self-test` locally however are not using these excludes by default, so developer's have to manually look up the current exclude list from one of the CI configs, then add these excludes to their own `meteor self-test` call manually. This commit adds a new `skip` option to Meteor's `self-test` system, that can be used to skip adding/running a defined `self-test` (similar in concept to Mocha's `skip` feature). This provides a way to skip the running of older `self-test`'s that are no longer needed, but allows them to be preserved in the `self-test` suite, for future reference. With this functionality in place, and the older test suites updated to use it, Meteor's base CI excludes no longer need to be maintained in their respective config files. The excludes are all managed at the source (the test definition), and can be leveraged by anyone/anything calling `meteor self-test`. * Log message describing skipped test * Add manually-ignored count to self-test summary * Small comment correction * History.md entry with PR link
Tools testing
Running end-to-end tests happens through the Self-Test. To run the tests:
./meteor self-test <regexp>
A very-very useful environment variable to set, in case you are running on a slow machine:
# set the multiplier for time-outs
set TIMEOUT_SCALE_FACTOR=3
Writing tests
All tests are currently stored at /tools/tests/, each JS file can register a
self-test. Example:
selftest.define("mongo failover", [/* tags */], function () {
var s = new Sandbox();
s.set('METEOR_TEST_MULTIPLE_MONGOD_REPLSET', 't');
s.createApp("failover-test", "failover-test");
s.cd("failover-test");
var run = s.run("--once", "--raw-logs");
run.waitSecs(120);
run.match("SUCCESS\n");
run.expectEnd();
run.expectExit(0);
});
The example above demonstrates how to define a test, create a Sandbox, create an app from a template and run the Meteor commands.
Templates for apps and packages are kept in /tools/tests, too.
Testing with Phantom/Browserstack
The sandbox has a testWithAllClients method that runs the clients like Phantom
or Browserstack pointed to the page of the app (localhost:3000 by default).
Tags
Tags are arbitrary. To make tags do anything, you should edit the selftest.js
code.
Examples of some tags that exist today:
slow- the test is skipped, unless the--slowflag is passedwindows- the test is not run unless on Windowsnet- the test is talking to external Internet services, thus requires an Internet connection to run
There are others.
Self-test gotchas
- The docs for self-test is reading the code of self-test
run.forbid(regexp)forbids the regexp from the entire output, not from the point it was called. It happens, because the output is matched asynchronously.