Commit Graph

28 Commits

Author SHA1 Message Date
Ben Newman
eee519ad58 Reduce default TIMEOUT_SCALE_FACTOR to hasten long test failures. 2017-06-28 13:32:12 -04:00
Ben Newman
fdd12e15e9 Stop clearing the npm cache unnecessarily before tests.
Based on this warning:

npm ERR! As of npm@5, the npm cache self-heals from corruption issues and
npm ERR! data extracted from the cache is guaranteed to be valid. If you
npm ERR! want to make sure everything is consistent, use 'npm cache
npm ERR! verify' instead.
npm ERR!
npm ERR! If you're sure you want to delete the entire cache, rerun this
npm ERR! command with --force.
2017-06-27 19:22:04 -04:00
Ben Newman
c076276a11 Merge branch 'devel' into release-1.4.3 2016-12-07 14:11:52 -05:00
Ben Newman
5d0da71e27 Use process.env.METEOR_HEADLESS in CI tests.
This should also fix a bug where Console.isHeadless() was returning false
in tests.

Follow-up to #8037.
2016-12-06 12:47:05 -05:00
Jesse Rosenberger
1c62dcc8a5 Remove skipped CI tests which don't exist anymore
Removing thest tests which aren't defined anywhere anymore but the exceptions to skip them are still in place:

* ^mongo - logged (in|out)
* ^logs - logged (in|out)
* ^can't publish package with colons
2016-12-02 16:13:15 +02:00
Jesse Rosenberger
8c7232c0f7 Track error codes from all self-test instances and fail accordingly
Since multiple self-tests could run in a series now, this makes sure that when any of them fails, the error is propagated to CircleCI but still lets them all finish.
2016-12-01 19:12:59 +02:00
Jesse Rosenberger
b2e80f5cbc Revert "Reformat the pre-defined tests to not be multiple lines."
This reverts commit 7df3ccaa2195af2628f20752104b331997137b73 which introduced a change which required `bash`.

I don't want to force `bash` for a test runner.
2016-12-01 19:12:59 +02:00
Jesse Rosenberger
6701357c41 Reformat the pre-defined tests to not be multiple lines.
This might be overboard as it adds some additional code but the regex off the screen was midly unpleasant.

One additional benefit to this is that you could have a list of excluded tests stored in a separate file and use:

```
ADDL_SELF_TEST_EXCLUDE="$(cat addl_tests.txt)"
```

(note the importance of double quotes to preserve newlines)
2016-12-01 19:12:58 +02:00
Jesse Rosenberger
b0c6804b63 Allow changes to some CI settings which may vary by environment.
This permits changing TIMEOUT_SCALE_FACTOR using a variable instead of hard-coding it in the script.

Also adds ADDL_SELF_TEST_EXCLUDE to define environment specific tests which must be skipped.

For example, I run my preliminary (personal) Meteor tests on SemaphoreCI, but there is a particular test that just _will not work_.  This allows me to override particulary problematic tests without sed-ing the script.
2016-12-01 19:12:58 +02:00
Jesse Rosenberger
054796f35e Make CI test runner more defensive and a bit more flexible.
This changes the CI script to make sure it runs all tests, regardless of the environment configuration while still allowing flexibility in the size of the container parallelism.

While not having any effect on the Meteor configuration this will now run all 8 test groups across a fewer number of containers, if necessary.

This fixes issues where false-positives are achieved on contributor PRs, like seen on meteor/meteor#7963.  I explained here:

https://github.com/meteor/meteor/pull/7963#issuecomment-264188162

This change will also allow the script to run tests in their entirety if the parallelism configuration is missing, such as when run from a local dev box.
2016-12-01 19:12:54 +02:00
Ben Newman
ad69d59fa7 Clear dev_bundle/.npm before tests to ensure consistent results. 2016-09-26 20:39:15 -04:00
Ben Newman
ef02320316 Initialize and update submodules in scripts/ci.sh. 2016-09-09 17:16:24 -04:00
Ben Newman
f6a52788f0 Remove BrowserStack from dev bundle. 2016-08-15 14:23:09 -04:00
Ben Newman
bae5983d59 Merge branch 'devel' into release-1.3.3 2016-05-27 00:02:48 -04:00
Tom Coleman
3b0ae7a903 Rebalanced tests 2016-05-25 17:03:05 -07:00
Tom Coleman
c9bf64250e Distribute custom warehouse tests to a single container 2016-05-24 16:19:00 -07:00
Ben Newman
6e1346a74b Remove phantomjs-prebuilt from the dev bundle.
Saves 50MB from the extracted size of the dev_bundle/lib/node_modules
directory, previously 262MB. The entire dev bundle is 332MB extracted, for
perspective.

More importantly, since the `phantomjs` binary is not very compressible,
this change reduces the *compressed* size of the dev bundle from 71MB to
54MB, a 24% savings.

Closes #6905.
2016-05-17 11:50:48 -04:00
Jesse Rosenberger
ed679d9fb7 Increase circleci timeout and disable spinner for CI tests 2016-05-16 09:01:01 +03:00
Ben Newman
a9d95b2a45 Reenable the spinner for now, so that CircleCI tests won't time out. 2016-05-09 20:11:54 -04:00
Ben Newman
eaab9e0dc3 Split scripts/ci.sh over 8 containers instead of 4. 2016-05-09 18:00:32 -04:00
Ben Newman
b3bcdbafda Disable progress spinner when running headless CI tests. 2016-05-09 18:00:32 -04:00
Sashko Stubailo
05d09edd12 Change parallelism to 4 for tests 2016-04-14 14:55:51 -07:00
Evan You
522834ca83 fix exclude pattern 2015-11-05 18:23:18 -05:00
Evan You
5b0be1b43c add excluded self-tests for CI 2015-11-05 17:23:32 -05:00
Evan You
40b7b4e3e9 adjust ci script for more parallel builds 2015-09-30 15:36:45 -04:00
Evan You
38928e5577 fix ci exlude patterns 2015-09-27 11:35:20 -04:00
Evan You
579618f947 move ci exclude patterns into the script so that they are available in pull request builds. 2015-09-27 11:10:50 -04:00
Evan You
8033ea03c7 setup circle ci 2015-09-25 15:25:11 -04:00