Summary:
What I'm looking for here is: attention to the few parts that aren't just
deletions (eg in auth.js), and thoughts about if removing any of these things
might break systems I haven't thought about.
Test Plan: test-packages, self-test (which mostly passes)
Reviewers: ekatek
Differential Revision: https://phabricator.meteor.io/D18
Contains:
- method to aggregate exports for a package in packageSource (exports are per-architecture).
- get this data from packageSource in PackageQuery for ‘meteor show’. Don’t store it in the
local catalog — while it is not a particularly expensive operation, it is still more expensive
than a simple lookup. We really do care about minimizing any sort of computation when we
are initializing packages, since we want the tool to be fast.
- display the data in ‘meteor show’. It makes sense to line wrap this with the ‘Exports:’ label as a
bulletPoint (just look at the test to see an example where this improves user experience). Since we
are doing that, we might as well use that bulletPoint functionality on the other labels as well.
- There is also a test. Run ‘meteor self-test show’ to test, or run ‘meteor show’ on a local package
with exports.
The Troposphere counterpoint to this is: meteor/troposphere#5
This also un-breaks soft refresh for troposphere packages:
previousIsopack was accidentally being set to an
{isopack,pluginProviderPackageMap} object, and so the "can we reuse the
previous one" check was never actually passing.
This will be useful when we want to be smart with windows file paths later
Also, all of the file calls are asynchronous with fibers now, which comes with
many benefits.
This is a combination of 23 commits. Original messages:
Wrap a large number of fs calls inside files.*
Convert a few more fs calls to files.*
More moving fs.* to files
Implement read/write streams and open/read/close
Get rid of fs from auth.js
Remove fs and unused imports from catalog-local and catalog-remote
Remove unused imports from catalog.js
Replace a whole lot of fs calls
Fix error
Migrate a lot more fs. calls to files.
Add a temporary symlink method
Convert old test to files.*
Use files.pathX instead of path.x everywhere
Replace path.x to files.pathX in tests
Small fixes to files.js and one rename
Make cleanup run in a fiber
Make wrapping functions take function name in case we need it
Add some timeouts and stuff to HCP tests
wrapFsFunc also makes a sync version of the function
Sometimes you just don't want to yield!
Make sure JsImage readFromDisk doesn't yield
Remove unused imports from npm test
Change order of test now that some things don't yield
Fix missing files import, and add a debug error printout
Specifically, we only compile them if there's a cordova platform in the
current project, or if we are publishing the package.
(Ideally, we wouldn't require every published package to have
web.browser and web.cordova unibuilds --- we'd just publish a 'web'
unibuild unless there's actually a difference between the two. But we
are not there yet.)
This adds an extra flag to isopack-buildinfo.json, so that we know to
rebuild all the isopacks when we add the first cordova platform (or
remove the last cordova platform).
The implementation around publish is a little clunky; if you're in a
non-Cordova app and run meteor publish, it will rebuild all the packages
with web.cordova, and the next time you prepare the app it will rebuild
them again without it. It does work though.
Fixes#3274.
This will refresh the status bar more often and make the build tool more
responsive to signals.
Now that yield uses setImmediate, this has minimal effect on
performance.
The extra legibility is not as useful for code that is always part of
core as it is for user's packages, and it actually has a notable (25%)
effect on isopacket load time.
Fixes#3218.
It wasn't really being used for anything anymore except complaining
about api.versionsFrom being used during isopacket builds, which is now
implemented in a simpler way.
It was 'name' in unibuilds, and both 'self.archName' and 'name:' option
in SourceArch. And for some reason we stopped writing it to isopack.json
at some point. Now we'll just call it 'kind' to make it clear it is
neither a name nor an arch. (I think this used to be how we
differentiated main from tests, but that doesn't exist any more.)
This fixes a bug introduced on this branch where weak dependencies might
get ignored if the weakly-depended package was just recompiled.
This gets rid of a call to determineBuildTimeDependencies, which calls
the constraint solver. This is step one of getting rid of
determineBuildTimeDependencies.
Note that this changes the build order logic a little.
In 0.8.3 and earlier we always built *all* the packages we *directly*
depended on before building ourselves. And this led to building the
transitive closure, or at least up until we hit pre-built (release)
packages, which couldn't depend on non-release packages anyway.
In 0.9.0 we only built packages we depend on that have plugins, and the
complete transitive closure of plugins within ourself. This required
the heavierweight constraint solver call, or at least futzing with the
catalog. I think the "don't build direct dependencies without plugins"
isn't a necessary optimization and it's canceled out by requiring you to
run the constraint solver. The transitive closure issue is real, but
before merging this branch we'll move the _build logic out of
LocalCatalog.
We no longer include versions in the interface. The version was only
used to do some error checking, and the error message in question hasn't
shown up in any bugs from users.
Notably, we were NOT using the version to trigger downloads (that
happens in a different place), so it's not super useful to talk about
multiple versions at all.
This was an undocumented and entirely unused feature (only two dummy
packages on the package server have this set to a non-default value).
No attempt is being made to remove the field from existing isopacks or
catalog entries. To continue to support existing clients, the package
server has been modified to ignore any provided
earliestCompatibleVersion and instead always write the default ECV to
the catalog.
You can only request a named set of packages, not a random assortment.
In future commits, we will pre-build these packages into JsImages and
load them from that. Building packages for uniload will eventually not
involve the .build.foo directories at all. (All saved packages will be
built in app context, eventually.)
Summary:
The `Npm.strip` method makes up for packages that have missing or
incomplete .npmignore files by telling the bundler to strip out certain
unnecessary files and/or directories during `meteor build`.
The `discards` parameter should be an object whose keys are NPM
package names and whose values are arrays of strings (or regular
expressions) that match paths in that package's directory that should be
stripped before installation. For example:
Npm.strip({
connect: [/*\.wmv$/],
useragent: ["tests/"]
});
This means (1) "remove any files with the `.wmv` extension from the
'connect' package directory" and (2) "remove the 'tests' directory from
the 'useragent' package directory."
The values can also be objects, in which case the keys of the nested
objects will match nested dependency names. For example,
Npm.strip({
connect: {
multiparty: ["test/"]
}
});
will discard the "test" directory from the "multiparty" package that is
depended upon by the "connect" package.
If you need to discard files from both "connect" and "multiparty", here's
a little trick you can use:
Npm.strip({
connect: {
connect: ["huge.wmv"],
multiparty: ["test/"]
}
});
This makes intuitive sense because requiring the "connect" package from
the "connect" package always returns the package itself.
Test Plan:
Run `meteor rebuild <package>` for the packages whose package.js files I
modified, and verify that the `Npm.strip`ped paths are absent from the
generated bundle.
Reviewers: glasser, nim, emily
Differential Revision: https://phabricator.meteor.com/D865
(still outstanding: changes to package publication workflow)
A package marked debugOnly in the package source is not to be bundled in production.
Moreover, if a package/app depends on a debugOnly package, that entire tree should
not be bundled. (But we should take it into account when figuring out versions!)
Does the following:
- In the catalog, we have a function that takes in a set of versions and a set of original
constraints and traverses it, recursively, to build a subset of versions that we *should*
bundle, and the corresponding subset of versions that we shouldn't (because they are either
debugOnly themselves or pulled in by debugOnly packages). (We do this in the catalog because
it is an addon onto the results of the constraint solver, tied deeply into our representation
of data)
- In the packageLoader, we keep track packages & versions that we should bundle, and also,
packages that we should exclude. We do this in the package-loader because, essentially, that's the
object that we use to keep the results of the constraint-solver, and we already propagate it to all
functions that care about it. (Possibly we should subsequently rename it later).
- In the compiler, when we figure out buildTimeDependencies, we ask if we need to bundle debug
builds. If not, we filter them out (see above). Also, when we actually build together unibuilds,
we don't touch the ones that the packageloader tells us to exclude (which ensures that they don't make
it into the final product).
- In the project, we keep track of whether this project is building in debug mode. That's because the project
is where we keep the state of our curent project that we are building, and if we are ever in the state of
building multiple things, then that's the code that we would need to touch (see also that we make a similar
assumption when solving constraints).
- Adds the option to keepthe project debug-build-free and calls it in commands when approporiate.
