This also un-breaks soft refresh for troposphere packages:
previousIsopack was accidentally being set to an
{isopack,pluginProviderPackageMap} object, and so the "can we reuse the
previous one" check was never actually passing.
This will be useful when we want to be smart with windows file paths later
Also, all of the file calls are asynchronous with fibers now, which comes with
many benefits.
This is a combination of 23 commits. Original messages:
Wrap a large number of fs calls inside files.*
Convert a few more fs calls to files.*
More moving fs.* to files
Implement read/write streams and open/read/close
Get rid of fs from auth.js
Remove fs and unused imports from catalog-local and catalog-remote
Remove unused imports from catalog.js
Replace a whole lot of fs calls
Fix error
Migrate a lot more fs. calls to files.
Add a temporary symlink method
Convert old test to files.*
Use files.pathX instead of path.x everywhere
Replace path.x to files.pathX in tests
Small fixes to files.js and one rename
Make cleanup run in a fiber
Make wrapping functions take function name in case we need it
Add some timeouts and stuff to HCP tests
wrapFsFunc also makes a sync version of the function
Sometimes you just don't want to yield!
Make sure JsImage readFromDisk doesn't yield
Remove unused imports from npm test
Change order of test now that some things don't yield
Fix missing files import, and add a debug error printout
Specifically, we only compile them if there's a cordova platform in the
current project, or if we are publishing the package.
(Ideally, we wouldn't require every published package to have
web.browser and web.cordova unibuilds --- we'd just publish a 'web'
unibuild unless there's actually a difference between the two. But we
are not there yet.)
This adds an extra flag to isopack-buildinfo.json, so that we know to
rebuild all the isopacks when we add the first cordova platform (or
remove the last cordova platform).
The implementation around publish is a little clunky; if you're in a
non-Cordova app and run meteor publish, it will rebuild all the packages
with web.cordova, and the next time you prepare the app it will rebuild
them again without it. It does work though.
Fixes#3274.
Includes the following changes to Console.js:
- Console.info, Console.warn, Console.debug and Console.error now automatically
line-wrap the output to 80 characters, or the width of the terminal screen (if
known). This is in line with our current style guide on how things should be wrapped!
- Sometimes, there are parts of text that we don't want to line-wrap. For example, if we are
telling the user to run 'meteor long-command --with --options' we don't want to
have a newline in the middle of that! Wrap those commands in Console.command, like
this:
Console.info("something and then run", Console.command(command), "and then");
This also makes them bold if chalk is on, as a nice bonus. So, if we ever turn
chalk back on, the bolding of commands will be more consistent.
- Sometimes, there is bulkier output that we don't want to format at all, including
line-wrapping: log snippets, stack traces, JSON output, etc. In that case, we can use
Console.rawInfo, Console.rawError, Console.rawWarn and Console.rawDebug. Don't use
Console.command inside the raw* functions! It won't be processed (at all).
- There are fancier things that we can do, other than just simply wrapping things.
We can indent:
" Start here and then when wrapping
continue over here".
We frequently do this for commands, for example. In the past, we did this manually --
but we can't do this for long messages that might get wrapped, and anyway, it is
good to codify this instead of counting spaces. Allows us to be better about consistency,
for example.
- We can also add a bulletPoint, which is a small notice in the beginning that looks like
this:
" => Start here and then when wrapping
continue below the bulletPoint".
Since it is a elss intuitive option, I have wrapped most of the time that we use a
bulletPoint into helper functions on the Console.js.
- Some common bulletpoints that we use are:
ASCII Checkboxes (Console.success)
ASCII X-s (Console.failWarn and Console.failInfo)
=> (Console.arrowError, Console.arrowWarn, Console.arrowInfo)
WARNING (Console.labelWarn)
The => are sometimes indented, so they take an optional indent argument, showing how
many spaces to indent by.
The wrapper interface would be less complicated, if there was a more unified conceit behind our
terminal messages. If there is one, it is not documented. My hope is that, in many cases,
moving these to Console will make it easier for someone with great product sense to
clean up our terminal messages. It will also make it easier to write such messages, since
it will be easier to follow an accepted standard.
In the codebase outside of Console:
- Went through and looked at our use of Console.error/info/etc, replacing with rawError/etc
whenever approporiate.
- Went through and modified most of 'stdout' and 'stderr' calls to use the new functions.
I made an exception for stuff that doesn't want a new line at the end, or otherwise does
weird things (ex: print user logs directly), on the basis that, at this juncture, it is
better to be safe than to be sorry.
- Long messages no longer need to break the code style guide by ignoring indentation rules.
Fixed that where approporiate.
- Fixed the tests! A number of our stock messages are actually longer than 80 chars.
- Personal favourite: The Android license agreement is now line wrapped! Much better experience.
- There is some more work to do on:
- longform help (currently comes with built-in linebreaks, would have to change the entire
mechanism for how that works)
- Buildmessage sometimes has headers that start with =>, but they are short. I didn't want to
pass wrapper options all the way to main.captureOrExit before merging the rest of this and
making sure that we like it. Since these messages are fairly short, I don't think that's
likely to be a serious problem.
I hope that this makes life easier for us in the future! No more counting chars, no more breaking
the style guide. Better experience for users with wider terminals (or even shorter terminals!).
Let's give this a try.
It wasn't really being used for anything anymore except complaining
about api.versionsFrom being used during isopacket builds, which is now
implemented in a simpler way.
We need to load the transitive closure of the plugin in order to compile
the package with the plugin. But we weren't following the transitive
closure when loading prebuilt packages. This fixes that.
It was 'name' in unibuilds, and both 'self.archName' and 'name:' option
in SourceArch. And for some reason we stopped writing it to isopack.json
at some point. Now we'll just call it 'kind' to make it clear it is
neither a name nor an arch. (I think this used to be how we
differentiated main from tests, but that doesn't exist any more.)
This fixes a bug introduced on this branch where weak dependencies might
get ignored if the weakly-depended package was just recompiled.
As far as I understand, cordovaDependencies exist in these places:
- On an entire package (not an individual unibuild), as stored in the
main JSON blob (isopack.json)
- On a target (combination of many packages), as stored in the star
This commit removes some stray code where they show up in other places:
- A field (which is never written) is read from the individual unibuild
JSON file and passed to the Unibuild constructor, which does not
expect it.
- A field (which is never read) is written to the buildinfo.json file.
Also, an assignment to Isopack.cordovaDependencies is pulled out of a
loop.
This was an undocumented and entirely unused feature (only two dummy
packages on the package server have this set to a non-default value).
No attempt is being made to remove the field from existing isopacks or
catalog entries. To continue to support existing clients, the package
server has been modified to ignore any provided
earliestCompatibleVersion and instead always write the default ECV to
the catalog.
There is no longer a uniload catalog when running from a release, since
the release contains whole isopackets (programs) not
isopacks (packages).
We only need a uniload-specific catalog when we're actually rebuilding
isopackets (not in order to load them), so we now have it as a local
variable in the two places that build isopackets.
The deleted code in package-loader.js was specific to the prebuilt
uniload package which no longer exists.
It should only need a capture if you need to include
buildinfo. Specifically, the use in tropohouse for package merging
should not be able to fail with a buildmessage.
Failures to statically compile packages for the tool now just crash the
tool, just like uniload failures. This should never happen unless you're
working from a checkout and have broken a core package.
Summary:
The `Npm.strip` method makes up for packages that have missing or
incomplete .npmignore files by telling the bundler to strip out certain
unnecessary files and/or directories during `meteor build`.
The `discards` parameter should be an object whose keys are NPM
package names and whose values are arrays of strings (or regular
expressions) that match paths in that package's directory that should be
stripped before installation. For example:
Npm.strip({
connect: [/*\.wmv$/],
useragent: ["tests/"]
});
This means (1) "remove any files with the `.wmv` extension from the
'connect' package directory" and (2) "remove the 'tests' directory from
the 'useragent' package directory."
The values can also be objects, in which case the keys of the nested
objects will match nested dependency names. For example,
Npm.strip({
connect: {
multiparty: ["test/"]
}
});
will discard the "test" directory from the "multiparty" package that is
depended upon by the "connect" package.
If you need to discard files from both "connect" and "multiparty", here's
a little trick you can use:
Npm.strip({
connect: {
connect: ["huge.wmv"],
multiparty: ["test/"]
}
});
This makes intuitive sense because requiring the "connect" package from
the "connect" package always returns the package itself.
Test Plan:
Run `meteor rebuild <package>` for the packages whose package.js files I
modified, and verify that the `Npm.strip`ped paths are absent from the
generated bundle.
Reviewers: glasser, nim, emily
Differential Revision: https://phabricator.meteor.com/D865
