This commit is based on the following design document:
https://mdg.hackpad.com/Creating-and-Updating-Docs-0ZyyDcSZDxp,
and some other stuff from here: https://mdg.hackpad.com/Meteor-Long-Description-wGZ1vIOwVlF
and was code reviewed here: https://github.com/meteor/meteor/pull/3375
It does the following:
- Allow the user to specify package documentation in Package.Describe.
We will take the README.md file by default, to make the transition easier.
Users can specify ‘documentation: null’ to not submit a README.md
- From that documentation, extract the section between the first and second header
to use as the long form description for the package.
- Upload the documentation to the server at publish-time. Allow metadata changes with ‘publish —update’.
- Change the default package skeleton to include the README.md file.
Also, changes the skeleton to have fewer useless placeholders in Package.describe values.
- Fix a minor bug where Git did not show up when running ‘meteor show’ on local packages.
A note on ‘documentation: null’ and blank documentation — we don’t let maintainers upload
blank README.md files, because we want to encourage people to fill them out. (Instead,
we allow a ‘documentation: null’ as an override) This is a UX issue! It is not a technical thing.
There is more discussion and code review in: https://github.com/meteor/meteor/pull/3375
Contains:
- method to aggregate exports for a package in packageSource (exports are per-architecture).
- get this data from packageSource in PackageQuery for ‘meteor show’. Don’t store it in the
local catalog — while it is not a particularly expensive operation, it is still more expensive
than a simple lookup. We really do care about minimizing any sort of computation when we
are initializing packages, since we want the tool to be fast.
- display the data in ‘meteor show’. It makes sense to line wrap this with the ‘Exports:’ label as a
bulletPoint (just look at the test to see an example where this improves user experience). Since we
are doing that, we might as well use that bulletPoint functionality on the other labels as well.
- There is also a test. Run ‘meteor self-test show’ to test, or run ‘meteor show’ on a local package
with exports.
The Troposphere counterpoint to this is: meteor/troposphere#5
Specifically, this means that errors during publish-release
--from-checkout can include the package name, since buildmessage errors
only mention the innermost job name. (It's more important for
publish-release's errors to include the package name than for publish,
since for publish you presumably know what package you're publishing
anyway.)
Fixes#3323.
This also un-breaks soft refresh for troposphere packages:
previousIsopack was accidentally being set to an
{isopack,pluginProviderPackageMap} object, and so the "can we reuse the
previous one" check was never actually passing.
This will be useful when we want to be smart with windows file paths later
Also, all of the file calls are asynchronous with fibers now, which comes with
many benefits.
This is a combination of 23 commits. Original messages:
Wrap a large number of fs calls inside files.*
Convert a few more fs calls to files.*
More moving fs.* to files
Implement read/write streams and open/read/close
Get rid of fs from auth.js
Remove fs and unused imports from catalog-local and catalog-remote
Remove unused imports from catalog.js
Replace a whole lot of fs calls
Fix error
Migrate a lot more fs. calls to files.
Add a temporary symlink method
Convert old test to files.*
Use files.pathX instead of path.x everywhere
Replace path.x to files.pathX in tests
Small fixes to files.js and one rename
Make cleanup run in a fiber
Make wrapping functions take function name in case we need it
Add some timeouts and stuff to HCP tests
wrapFsFunc also makes a sync version of the function
Sometimes you just don't want to yield!
Make sure JsImage readFromDisk doesn't yield
Remove unused imports from npm test
Change order of test now that some things don't yield
Fix missing files import, and add a debug error printout
It's still in unmigrated commands.
Remove tropohouse's catalog. It was there to stop you from downloading
local packages but downloadPackagesMissingFromMap does that based on the
PackageMap now.
This was an undocumented and entirely unused feature (only two dummy
packages on the package server have this set to a non-default value).
No attempt is being made to remove the field from existing isopacks or
catalog entries. To continue to support existing clients, the package
server has been modified to ignore any provided
earliestCompatibleVersion and instead always write the default ECV to
the catalog.
This commit does the following:
- Introduces the get-machine command. This command contacts the build farm server
gets back a machine reservation and then opens a secure shell to the machine (Alternatively,
you can ask for a json). This also involved factoring out some commands to deal with authenticated
ddp from package-client into a more general auth-client.
- No longer publish binary builds in publish or publish-release; instead give the user a warning
to run get-machine and then publish-for-arch. Someone could ignore this: --existing-version and
publish-for-arch both publish binary builds, but you need to be at least somewhat familiar with
what you are doing to run them. Hopefully, you are running them from a certified build machine, but
if you are not, then, well, it is your package.
Stuff remaining:
- We are going to have a url to external documentation, but I haven't written it yet.
- We are currently talking to the test-build server, instead of the build server, so mac doesn't
work.
(Neither of those changes require significant tool changes)
When publishing a core package, you’re allowed to omit version numbers in package.js. With this change, we determine the correct versions of all the dependent packages based on the current checkout, and we send them to troposphere (instead of core packages having “null” constraints on their dependencies).
We throw an error if you have any missing version constraints on your package dependencies and you are not using versionsFrom or publishing a core package from a checkout.
We’ve already fixed this (no constraints on core package deps) retroactively in troposphere.
It speeds up the constraint solver by orders of magnitude when publishing a package.
Rewrote parts of the update server package data unit test to compare a pre-sync and a
post-sync catalog, rather than the output of package client's attempts to contact the server.
This is because in the new world, there is no accurate output, and instead, the function
modifies the catalog in place. Additionally, removed the old function that read from
data.json, since it was not used anymore, and cleaned up some comments and return values in
package-client. We no longer claim to return the contents of data.json, instead we return
an object that signals if we should reset the entire catalog, and/or if our connection to the
server flat-out failed. I am not sure that this is the best abstraction for this piece of code
(why does package-client modify the catalog, but not reset it, for example? Since resetting
has consequences, in the ideal world, the package-client would only have the logic to get data
from the server and it would be up to the catalog to figure out how to insert it into sql lite,
I think, maybe. Regardless, now is not the time to do that refactoring.)
The test is a little odd in the following ways. First, comparing every record ever published is
something that is already significantly harder than it used to be, and will get only harder from
there. (However, the test claims to check that no data has been erased, so we need to check it).
We check the vague existence of most records, on the theory that it is unlikely that
we only got a portion of the record in, rather than the entire thing. Second, it doesn't check the
actual contents on disk, because I wasn't sure about writing another interface to sqllite this
late in the game. There are some ways to get around this -- we could be sneaky and submit a non-blank
syncToken in some way (faking a previous sync), so we only get the packages touched since (time X).
Usually, that might violate some internal consistency, but we only care about the contents at this stage.
Second, we should probably write some method on the catalogs to compare themselves instead of making
and querying a copy.For now, though, I think that this is sufficiently expedient.
Also, the test publishes 5 packages. That's a lot of packages, so I marked it as slow.
