Commit Graph

6314 Commits

Author SHA1 Message Date
Ben Newman
269e10caa6 Remove support for pkg.module in client bundles.
This partially reverts commit d7cae3c939.

This functionality was problematic without providing any clear benefits to
Meteor developers, as reported by @fermuch and @markoshust:
https://github.com/meteor/meteor/pull/8327#issuecomment-298723203
2017-05-04 16:09:45 -04:00
Ben Newman
9c7778da36 Let the install npm package implement Module.prototype.prefetch.
Now anyone can define meteorInstall.fetch however they see fit, and the
install.js implementation will handle everything else.

This separation of concerns leads to significantly less code, too.
2017-05-01 23:55:00 -04:00
Ben Newman
d7cae3c939 Support pkg.module in client bundles, even for node_modules.
As proposed here: https://github.com/rollup/rollup/wiki/pkg.module

By supporting ECMAScript module entry points for npm packages in Meteor
1.5, we will be well-positioned to do more effective import/export-based
tree shaking in future versions of Meteor.

We can't do the same thing on the server because we can't change how
native Node resolves package entry points based on the "main" field of the
package.json module.

On the other hand, all npm packages have to work in Node using the "main"
field, and client bundles stand to benefit the most from tree shaking, so
this client/server difference should not be problematic.

Note that the "jsnext:main" property is also supported as a legacy synonym
for "module".
2017-04-27 17:34:34 -04:00
Ben Newman
b023f5c67a Merge branch 'release-1.4.4.2' into release-1.5 2017-04-27 17:16:05 -04:00
Bartosz Wojtkowiak
5a4d5035c3 Implement CORDOVA_COMPATIBILITY_VERSION_IOS/ANDROID and EXCLUDE (#8581)
* Implement CORDOVA_COMPATIBILITY_VERSION_EXCLUDE and CORDOVA_COMPATIBILITY_VERSION_IOS/ANDROID

CORDOVA_COMPATIBILITY_VERSION_IOS or CORDOVA_COMPATIBILITY_VERSION_ANDROID allows to override compatibility version for a specified platform.

CORDOVA_COMPATIBILITY_VERSION_EXCLUDE provides a way of excluding a certain plugin from compatibility version calculation. You can pass several plugin names with ';'. For example: `CORDOVA_COMPATIBILITY_VERSION_EXCLUDE='cordova-plugin-crosswalk-webview;cordova-plugin-device'`

* Changes after review
2017-04-26 13:23:22 -07:00
Ben Newman
1a036553c1 Fix faulty content-length comparison in utils/http-helpers.js.
Since body is a string, body.length is not necessarily the number of bytes
in the response body.
2017-04-25 16:11:29 -07:00
Jesse Rosenberger
99fffd8670 Fix disappearance of debug banner when running meteor debug.
This was broken due to a change in Node.js which enabled support for
using `--debug` in combination with IPv6 addresses.
2017-04-25 15:49:52 -07:00
Jesse Rosenberger
94f8c9d7ce Remove experimental trailing comma to avoid problems with jsdoc.
The comma in question: a trailing comma a rest-parameter, within a
function argument's parameter de-structuring:

	function a({
	  a = false,
	  ...b,
	}) {
	  // ...
	}

Espree, the parser used by `jsdoc` (used in Meteor docs) previously
allowed this with `experimentalObjectRestSpread` enabled but now throws
an error with the addition of 652990a7bf.

It might have been argued at one point that the trailing-comma could
allow for the easy, future addition of another parameter, ala:

	function a({
	  a = false,
	  ...b,
	  c = true,
	}) {
	  // ...
	}

Having the rest-parameter in the last position is certainly more clear
(it is the "rest", after-all, and there can be only one) but I'm not
going to have that discussion at the cost of docs not deploying!
2017-04-25 15:36:10 -07:00
Ben Newman
d9264e3b4b Better API for module.prefetch(id). 2017-04-25 10:26:24 -04:00
Ben Newman
5477aeac3f Support module.prefetch(id) to fetch but not evaluate dynamic modules.
Generally, module.prefetch(id) will not throw even if the fetched module
is missing. If you need to know whether module.prefetch(id) succeeded,
simply await the result of the promise, which will be null on success, or
an Error object if the module could not be imported.
2017-04-24 16:40:58 -04:00
Ben Newman
f4fc2a904d Use final hashes of dynamic modules for browser caching.
Previously, the linker included a "version" hash in each stub entry
corresponding to a dynamic module in the meteorInstall bundle. These stub
entries also contain information about any module identifiers ("deps")
imported by the module, and various other metadata.

This hash was derived from the source code of the dynamic module, instead
of its fully-processed generated code, which created a small risk that the
source hash would remain the same when anything changed in the post-linker
processing logic.

This new implementation uses the same hashes found in program.json
manifest files, which more reliably reflect changes in the actual final
contents of the modules.

Because these hashes are not known at link time, they are now injected as
one blob into the meteor/dynamic-import/dynamic-versions.js module, rather
than appearing in the meteorInstall bundles for their containing packages.
2017-04-21 16:15:09 -04:00
Ben Newman
b057415048 Merge branch 'devel' into release-1.5 2017-04-20 16:46:03 -04:00
Ben Newman
4122e6e7f2 Minor cleanup to make import chain test more robust. 2017-04-20 16:12:08 -04:00
Ben Newman
1c041237d9 Fix faulty content-length comparison in utils/http-helpers.js.
Since body is a string, body.length is not necessarily the number of bytes
in the response body.
2017-04-20 16:12:08 -04:00
Ben Newman
6f75eae135 Remove automatic Buffer polyfill.
This polyfill is unnecessary in Node, and added a whopping 22KB to the
minified client bundle. If you really need the Buffer API on the client,
you can get it from require("buffer").Buffer.
2017-04-19 17:41:11 -04:00
Jesse Rosenberger
27b2a04f6f Remove experimental trailing comma to avoid problems with jsdoc.
The comma in question: a trailing comma a rest-parameter, within a
function argument's parameter de-structuring:

	function a({
	  a = false,
	  ...b,
	}) {
	  // ...
	}

Espree, the parser used by `jsdoc` (used in Meteor docs) previously
allowed this with `experimentalObjectRestSpread` enabled but now throws
an error with the addition of 652990a7bf.

It might have been argued at one point that the trailing-comma could
allow for the easy, future addition of another parameter, ala:

	function a({
	  a = false,
	  ...b,
	  c = true,
	}) {
	  // ...
	}

Having the rest-parameter in the last position is certainly more clear
(it is the "rest", after-all, and there can be only one) but I'm not
going to have that discussion at the cost of docs not deploying!
2017-04-12 09:59:09 -07:00
Ben Newman
1a120a6f4d Merge branch 'devel' into release-1.5 2017-04-08 11:36:56 -04:00
Ben Newman
6055b49f8d Wait on removal of garbage directories.
This would have been a nice optimization if it had worked, but (in
addition to leaving garbage directories lying around sometimes if the
process was killed), it appears to have some unintended consequences for
meteorNpm.rebuildIfNonPortable and related functions, since the garbage
directories are easily confused for npm package directories.

Example stack trace:

  Error: ENOENT: no such file or directory, open '/home/travis/build/meteor/galaxy-server/node_modules/heapdump-garbage-1c2jqib/package.json'
      at Error (native)
   => awaited here:
      at Promise.await (/home/travis/.meteor/packages/less/.2.7.9.9fh5c1++os+web.browser+web.cordova/plugin.compileLessBatch.os/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:39:12)
      at copyFileHelper (/tools/fs/files.js:633:6)
      at Object.files.cp_r (/tools/fs/files.js:532:7)
      at /tools/isobuild/meteor-npm.js:393:11
      at Array.forEach (native)
      at copyNpmPackageWithSymlinkedNodeModules (/tools/isobuild/meteor-npm.js:386:29)
      at /tools/isobuild/meteor-npm.js:325:5
      at Array.forEach (native)
      at Object.rebuildIfNonPortable (/tools/isobuild/meteor-npm.js:315:17)
      at NodeModulesDirectory.rebuildIfNonPortable (/tools/isobuild/bundler.js:273:22)
      at /tools/isobuild/compiler.js:650:13
2017-04-07 16:21:43 -04:00
Ben Newman
43d339d087 Merge branch 'devel' into release-1.5 2017-04-07 12:45:29 -04:00
Jesse Rosenberger
948ec2b05f Merge branch 'master' into devel 2017-04-07 18:50:17 +03:00
Jesse Rosenberger
ae35f13ed2 Use new rm_recursive_async to cleanup garbage directory w/o waiting. 2017-04-06 20:18:38 +03:00
Jesse Rosenberger
0cf0fb12c3 Remove catch-ing of EXDEV from Builder#complete.
This logic is now handled in `files.renameDirAlmostAtomically`.
2017-04-06 20:03:35 +03:00
Jesse Rosenberger
960e78c250 Sacrifice atomicity in the event that the file-system won't allow it.
/cc @benjamn
2017-04-06 19:56:17 +03:00
Jesse Rosenberger
e9a2f6d22c Just Profile the *Atomically functions directly.
...Versus reassigning them immediately after.

 (Whitespace-only view is recommended, due to indentation change)
2017-04-06 19:30:06 +03:00
Jesse Rosenberger
af1bc232d6 Hoist "Can we Fiber?" logic up to a helper function.
This also makes it possible to disable it in places where we were not
checking against the `METEOR_DISABLE_FS_FIBERS` env. variable, like in
`rm_recursive`.
2017-04-06 17:17:56 +03:00
Jesse Rosenberger
7090129f89 Re-implement renameDirAlmostAtomically in Builder#complete.
The `rename` is not sufficient on its own since it won't pave the way
for the new directory by ensuring that it's empty first.

`renameDirAlmostAtomically` will do that.
2017-04-06 17:17:55 +03:00
Jesse Rosenberger
523c7ce9d4 If EXDEV is encountered, resort to a delete and cp_r.
This operation does not have to be done atomically and this should allow
cross-device copies to be made if necessary.
2017-04-06 06:54:37 +03:00
Jesse Rosenberger
9e0f61c4b3 Revert "Use fs.move() from fs-extra to fix EXDEV cross device error in docker builds. (#8491)"
Unfortunately, `fs-extra` is still not as perfect at handling various
file system conditions as would be ideal.  It seemed sensical to try and
use a library like this however, it turns out that the Meteor suite
of file system functions stands up best on Windows, which is where I
encountered most problems.

For example, `fs-extra` still tries to create symlinks as an unprivileged
user – a forbidden task on Windows unless running as Administrator.

In addition, I ran into a constant stream of other errors: `ENOTEMPTY`,
`EBUSY`, `EEXIST` – all for various reasons.

My current recommendation is that we remove `fs-extra` and replace the
`Builder#complete` `renameDirAlmostAtomically` call (which does not
absolutely _have_ to be done atomically) with a `try`/`catch` which
resorts to a basic copy if `err.code === 'EXDEV'`.  All other
functionality stays the same.

This reverts commits:

* d49f3e2704
* 3257bafc84
* 74cb8ebdc2
* 5bbdcc9baa
* 6a0767bbac
2017-04-06 06:50:03 +03:00
Jesse Rosenberger
6a0767bbac Don't reassign, just Profile the *Atomically functions directly. 2017-04-05 19:04:22 +03:00
Jesse Rosenberger
5bbdcc9baa Change order that functions are defined so move is wrapped correctly. 2017-04-05 18:56:05 +03:00
Jesse Rosenberger
27819a164b Update installNpmModule to properly capture newer npm error formats. (#8564)
Previously, we captured and displayed shorter error messages for the
more complicated and unnecessarily verbose messages which npm produced.

While updating npm to 4.4.4, I observed the changelog for 4.4.0
indicated it would now produce less verbose messages.  In searching for
possible Meteor conflicts with this, I discovered that
`installNpmModule` had already regressed on providing pretty messages.

This fixes those messages to be parsed properly and adds tests which
ensure if npm changes again that we can capture them.

Follows-up on: https://github.com/meteor/meteor/pull/8562
2017-04-05 11:46:29 -04:00
Jesse Rosenberger
74cb8ebdc2 Improve fs-extras move calls for Windows platform. (#8560)
* Improve `fs-extra.move` calls for Windows platform.

This is a follow-up to meteor/meteor#8491 which worked properly on Unix
platforms, but failed in a variety of ways on Windows due to its lack
of Fiber-awareness and desire to create symlinks as unprivileged users
(something not always possible on Windows).

The Fiber issue was observed when trying to remove "src" directories
within the `move` function (which tries a variety of OS/OS/arch-specific
techniques to accomplish its goal) after they had been copied to "dest".
On Windows, this resulted in `EDIRNOTEMPTY` errors since Windows appears
to temporarily cache the file-handle or doesn't release the file-handle
until the next tick.

The symlink issue will hopefully improve in an upcoming release of
Windows (Creator Edition) when Microsoft makes it possible to create
symlinks as an unprivileged user, however it will still require enabling
"Developer" mode in Windows settings.  This implements the same catch
which was already in place for `fs.rename` on the `fs.move` provided by
`fs-extra`.

Performance gains were the same in tests comparing before and after
these changes.

Relates to:
https://github.com/meteor/meteor/issues/8558#issuecomment-291194385

* A few code-cleanups to my original commit.
2017-04-05 12:58:32 +03:00
Ben Newman
337a61c426 Disable dynamic module fetching in Cordova apps.
Instead of fetching a bundle of JavaScript from the server, Cordova apps
include the complete bundle in the installed app, so there's no compelling
reason to allow dynamic module fetching.

The dynamic import(...) function will still work on Cordova, of course.
It will just never have to fetch anything from the server.
2017-04-04 19:09:26 -04:00
Jesse Rosenberger
cba8c63340 Update node-gyp and node-pre-gyp packages.
* `node-gyp` - Adds support for Visual Studio 2017.
  - https://github.com/nodejs/node-gyp/blob/master/CHANGELOG.md#v360-2017-03-16
* `node-pre-gyp` - Nothing notable.
  - https://github.com/mapbox/node-pre-gyp/blob/master/CHANGELOG.md#0634
2017-04-04 19:42:03 +03:00
Jesse Rosenberger
8bdd0142cd Disable display of update msg about npm itself, since it's bundled.
As of npm 4.4.0 this is necessary as it will now self-check once per day
for updates.  Meteor pre-bundles the version of npm though so this
message will be confusing to users of the `meteor` tool.

https://github.com/npm/npm/releases/tag/v4.4.0
2017-04-04 19:37:20 +03:00
Ben Newman
87d0a33067 Get Cordova plugin test to pass by adding time. 2017-04-03 18:43:54 -04:00
Ben Newman
954efa7f5a Support Google Sign-In in google-oauth package. (#8549)
* Support Google Sign-In in google-oauth package.

Addresses #8253.

* Use Meteor.startup instead of listening for deviceready event.

* Fix mobile-config.js typo.

* Bump accounts-google and google-oauth package versions.

I'm only bumping the patch versions, even though the recent changes to
these packages may seem significant, for two reasons:

1. Bumping the minor versions would force Meteor 1.4.3 developers to
   upgrade to Meteor 1.4.4 if they wanted to use these changes.

2. The accounts-google and google-oauth packages without these changes
   will stop working completely in two weeks, which is much worse than the
   risks of upgrading.
2017-04-03 17:07:55 -04:00
Jesse Rosenberger
d24cd94f20 Merge branch 'release-1.4.3.x' of https://github.com/meteor/meteor into release-1.4.3.x 2017-03-30 02:15:20 +03:00
Ben Newman
fcbd398ab7 Appending module identifier extensions should not match directories.
This logic needed to change not only in meteor/tools/isobuild/resolver.js
but also in the runtime module system:
cfcc422725

Fixes #8539.
2017-03-29 18:50:41 -04:00
Jesse Rosenberger
3257bafc84 Change wrapFsFunc for "move" to acknowledge _both_ arguments as paths.
The `wrapFsFunc` function accepts an array of indexes indicating which
arguments are paths.  This is particularly important on Windows, due to
the path-conversation which takes place on those strings.

The docs say:
> Indices of arguments that have paths, these arguments will be
> converted to the correct OS slashes

This follows up on the change made from meteor/meteor#8491 which failed
in our release pipeline when publishing for the Windows architecture
for `1.4.4-rc.3`.
2017-03-30 01:46:42 +03:00
Jeremy Shimko
d49f3e2704 Use fs.move() from fs-extra to fix EXDEV cross device error in docker builds. (#8491)
Use fs.move to fix EXDEV cross device error in docker builds.

Fixes #7852.
2017-03-28 15:53:22 +03:00
Ben Newman
5cf603ce82 Merge branch 'release-1.4.3.x' into release-1.5 2017-03-27 15:37:11 -04:00
Ben Newman
10127cea1f Fix modules test app now that exports.__esModule is non-enumerable.
3973bbad7f
2017-03-25 12:00:18 -04:00
Ben Newman
0eb52a1fd8 Upgrade meteor-babel to v0.19.1 and reify to v0.6.6. 2017-03-25 11:12:52 -04:00
Jesse Rosenberger
6bbd432081 Be more understanding of Windows' filesystem limitations.
Presently, the renaming of directories that are in-use will fail on
Windows.  This is already compensated for when `process.platform` is
set to `win32`.  However, within BashOnWindows/WSL (Windows Subsystem
for Linux), `process.platform` is equal to `linux`, though the
underlying filesystem is still the same.

Microsoft has stated that it is unlikely that they will remove
`Microsoft` from the `os.release()` value so we check for that.
2017-03-22 20:35:43 +02:00
Jesse Rosenberger
d9c3f80dbf Be more understanding of Windows' filesystem limitations.
Presently, the renaming of directories that are in-use will fail on
Windows.  This is already compensated for when `process.platform` is
set to `win32`.  However, within BashOnWindows/WSL (Windows Subsystem
for Linux), `process.platform` is equal to `linux`, though the
underlying filesystem is still the same.

Microsoft has stated that it is unlikely that they will remove
`Microsoft` from the `os.release()` value so we check for that.
2017-03-22 19:13:01 +02:00
Jeremy Shimko
20baddf7b7 Use fs.move() from fs-extra to fix EXDEV cross device error in docker builds. (#8491)
Use fs.move to fix EXDEV cross device error in docker builds.

Fixes #7852.
2017-03-22 12:54:07 -04:00
Jesse Rosenberger
41cef97664 Automatically trim DEPLOY_HOSTNAME of leading/trailing spaces. (#8508)
This simple fix prevents the disappointment of trying to deploy your app
but failing because there's a space on the end of the `DEPLOY_HOSTNAME`
environment variable.

`process.env` always contains string values and assigning a property on
`process.env` implicitly converts the value to a string so it should not
be necessary to check if `typeof` is a `string`.

Fixes Dev Experience.
2017-03-22 12:50:28 -04:00
Ben Newman
d0f179b2f5 Merge pull request #8502 from hwillson/issue-6945
Updated the builder whitelist to allow "@" in filenames.
2017-03-22 12:50:28 -04:00
Jesse Rosenberger
6621a63025 Automatically trim DEPLOY_HOSTNAME of leading/trailing spaces. (#8508)
This simple fix prevents the disappointment of trying to deploy your app
but failing because there's a space on the end of the `DEPLOY_HOSTNAME`
environment variable.

`process.env` always contains string values and assigning a property on
`process.env` implicitly converts the value to a string so it should not
be necessary to check if `typeof` is a `string`.

Fixes Dev Experience.
2017-03-22 12:01:53 -04:00