Commit Graph

12019 Commits

Author SHA1 Message Date
David Glasser
bf619f2e8b be more consistent about naming imports 2014-11-25 09:06:13 -08:00
David Glasser
738a4a00b8 Escape package names in IsopackCache paths 2014-11-25 09:06:13 -08:00
David Glasser
7b529ba9c9 Add a note to get rid of catalog.complete 2014-11-25 09:06:13 -08:00
David Glasser
e737ef4cfe download correct arch of packages in bundle 2014-11-25 09:06:13 -08:00
David Glasser
1ea3a5c4a3 Revert "Prepare ProjectContext automatically in main"
This reverts commit 11f820a7dd38b6ce36fe06bcbf5131541d1e2756.

Better to have a little more control by doing this manually.
2014-11-25 09:06:12 -08:00
David Glasser
a1268b8d6b Just build all dependencies before a package
The old code failed because versioned packages (with or without plugins)
weren't loaded before compileUnibuild, and the eachUsedUnibuild call
inside it threw. We could load plugin'd versioned packages, but I also
realized that the old getPackagesToBuildFirst didn't find you plugins
that were pulled in via implies. This way is simpler.
2014-11-25 09:06:12 -08:00
David Glasser
f956f3b635 drop unused code 2014-11-25 09:06:12 -08:00
David Glasser
17f5cf6273 'meteor bundle' works with ProjectContext!
- the 'programs' subdirectory is no longer supported

- includeDebug is now an option to bundler.bundle, not global state

- some cordova-specific stuff has been disabled

- lots of other commands are presumably entirely broken
2014-11-25 09:06:12 -08:00
David Glasser
14db1041ce Prepare ProjectContext automatically in main 2014-11-25 09:06:12 -08:00
David Glasser
4a4db617d3 properly display download errors as buildmessage 2014-11-25 09:06:12 -08:00
David Glasser
431004c089 Remove pluginProviderPackageDirs
This has been replaced by the pluginProviderPackageMap which is built at
a different level.
2014-11-25 09:06:11 -08:00
David Glasser
ebf881894b buildinfo file should contain all pkgs in plugin 2014-11-25 09:06:11 -08:00
David Glasser
3983cabb78 Deal better with build failures 2014-11-25 09:06:11 -08:00
David Glasser
a3e5e1b40d drop support code for "build identifiers" 2014-11-25 09:06:11 -08:00
David Glasser
8aefc5b46f Use IsopackCache to build isopackets 2014-11-25 09:06:11 -08:00
David Glasser
16d8a8e301 Read isopack-buildinfo.json; kill buildinfo.json 2014-11-25 09:06:11 -08:00
David Glasser
ab4e3abc7b The cache actually caches! 2014-11-25 09:06:11 -08:00
David Glasser
63a3ec64c7 Don't need to build deps of versioned packages 2014-11-25 09:06:10 -08:00
David Glasser
db66d8d2b6 remove some logs 2014-11-25 09:06:10 -08:00
David Glasser
7152975316 add comment about build identifier 2014-11-25 09:06:10 -08:00
David Glasser
4a516fef87 unibuilds need to be rebuilt if their plugins are
Basically, this branch is moving back to the pre-0.9.0 model where you
tell if a dependent package has changed by explicitly putting its
relevant files in your watchset, instead of by including a "build
identifier" which requires a relatively complex calculation.

Notably, let's say you had package P with a plugin, and a package Q that
depends on it.  If a file in P that's neither part of the plugin nor the
package.js changes, you don't have to rebuild Q... but it did change the
0.9.0 "build identifier", which caused an unnecessary rebuild!
2014-11-25 09:06:10 -08:00
David Glasser
685df046d0 write out isopack-buildinfo.json
our replacement for buildinfo.json

it's not right yet: unibuilds don't merge in the plugin watchset.
2014-11-25 09:06:10 -08:00
David Glasser
090f131bc1 Cache isopacks to disk, uselessly.
Uselessly because we don't include any build info, so how would you know
when to rebuild?
2014-11-25 09:06:10 -08:00
David Glasser
23f5e354d0 I can compile everything! 2014-11-25 09:06:09 -08:00
David Glasser
d6cca3a89e First steps towards an IsopackCache.
It comes up with the right order to build and load packages, and loads
prebuilt packages. It does not yet build packages.
2014-11-25 09:06:09 -08:00
David Glasser
e660b2d9d6 ProjectContext: download missing packages
Note that unlike the previous implementation, we show a 'downloading'
progress bar iff we actually download any packages.  (In 1.0 we show a
'downloading' message while just deciding if we need to download
anything; on devel right now we never show 'downloading' since that was
confusing.)
2014-11-25 09:06:09 -08:00
David Glasser
964174249d run constraint solver and make a package map 2014-11-25 09:06:09 -08:00
David Glasser
f7ee343dcb Depend on local packages versions 2014-11-25 09:06:09 -08:00
David Glasser
8d7139e624 Start on a new lightweight ProjectContext
This one won't be a singleton.  Eventually project.js will be moved into
here.

Add a 'meteor prep' command which exercises new codepaths. It will be
deleted eventually.

So far, it converts .meteor/packages directly into the format that the
constraint solver wants.  (Unlike the existing code which has several
intermediate formats.)
2014-11-25 09:06:09 -08:00
David Glasser
da794ab528 PackageCache: Remove on-disk cache and soft reload
On-disk cache will be replaced by a new class.

Soft refresh was an optimization that wasn't actually very optimized in
0.9.0; it used to only have to look at an in-memory WatchSet and check
those files, but in 0.9.0 it also re-read the whole isopack. Will add
this optimization back in if necessary and safe later.
2014-11-25 09:06:08 -08:00
David Glasser
c3cd93c989 Get rid of a few random spots that refer to .build. 2014-11-25 09:06:08 -08:00
David Glasser
46bb59c687 Stop using the .build.p cache in LocalCatalog 2014-11-25 09:06:08 -08:00
David Glasser
261a93b3dd Remove 'meteor rebuild' and rebuildLocalPackages
This is the first step in removing all references to
PACKAGESOURCE/.build.packagename caches. We will then implement a new
package cache from the ground up.
2014-11-25 09:06:08 -08:00
David Glasser
0166cef1f0 eliminate compiler.getBuildOrderConstraints
This gets rid of a call to determineBuildTimeDependencies, which calls
the constraint solver. This is step one of getting rid of
determineBuildTimeDependencies.

Note that this changes the build order logic a little.

In 0.8.3 and earlier we always built *all* the packages we *directly*
depended on before building ourselves. And this led to building the
transitive closure, or at least up until we hit pre-built (release)
packages, which couldn't depend on non-release packages anyway.

In 0.9.0 we only built packages we depend on that have plugins, and the
complete transitive closure of plugins within ourself.  This required
the heavierweight constraint solver call, or at least futzing with the
catalog. I think the "don't build direct dependencies without plugins"
isn't a necessary optimization and it's canceled out by requiring you to
run the constraint solver.  The transitive closure issue is real, but
before merging this branch we'll move the _build logic out of
LocalCatalog.
2014-11-25 09:06:08 -08:00
David Glasser
1f31c22435 simplify compiler.getBuildOrderConstraints
We no longer include versions in the interface. The version was only
used to do some error checking, and the error message in question hasn't
shown up in any bugs from users.

Notably, we were NOT using the version to trigger downloads (that
happens in a different place), so it's not super useful to talk about
multiple versions at all.
2014-11-25 09:06:08 -08:00
David Greenspan
b132dcb677 Minor changes to Resolver
There were too many things called `constraints` — ConstraintsList objects, arrays and maps of Constraint strings or objects… Also “alternatives” is already taken, so use “disjunction.”  Consider propagating this rename upstream to package-version-parser, since it’s kind of weird that “constraints” is implicitly a disjunction (rather than, say, a conjunction).

Don’t call a constraint string a version string.
2014-11-24 17:16:33 -08:00
David Greenspan
29fb67931b Clean up logic around pre-releases in solver
When it comes to pre-releases, Constraints aren’t “context-free.”  That’s ok for now, but at least hoist the pre-release check out of the innermost loop.  Ideally we’d hoist it higher, but I can see that it’s pretty convenient this way.

Should be identical in behavior to before (differing maybe in some super edge case, but I doubt it).
2014-11-24 15:28:06 -08:00
David Greenspan
fee326899a Remove unreachable case in Constraint#isSatisfied 2014-11-24 14:25:48 -08:00
David Greenspan
9a4428bf6e Don’t support #arch when parsing a constraint 2014-11-22 17:29:54 -08:00
David Glasser
d4209aff02 Merge branch 'constraint-changes-from-isopack-cache' into devel 2014-11-22 14:21:06 -08:00
David Glasser
fb6b39d956 explicitly search for non-weak dep
dep.references ought to never be empty, but if it is we should consider
that to not be a dependency!
2014-11-22 14:20:57 -08:00
David Glasser
cec75e527c constraint edges are by package, not unibuild
Consider the following situation:

  - app uses package P
  - Every version of P contains `api.use('s', 'server')`
  - Every version of S contains `api.use('c', 'client')`
  - There's nothing else around using S or C

When we bundle our app, we will not end up putting any unibuilds from C
into the bundle. That's fine.

The previous version of the constraint solver understood this, and so C
wasn't even part of the constraint solver solution.

HOWEVER, even though C does not contribute any unibuilds to the app
bundle, we still need to compile C in order to compile S. That's because
our current implementation never compiles only part of an isopack, even
if only part of the isopack will be needed for the app.

The structured project initialization done via ProjectContext will thus
decide to not build or load C, which will make it fail to compile S when
it gets around to compiling the client unibuilds in S.

We could make the model more complex by making it possible to compile
only part of S.

Instead, we'll make the Meteor package model simpler.  Constraints, as
far as the constraint solver is concerned, are now at a package level.
So in this case, "C" will actually be part of the project (will end up
in .meteor/versions, etc) even though it will continue to not provide
any part of any of the bundled client or server programs.

This means that nodes in the constraint solver's graph will now just be
package versions, not unibuild versions.  That's already the language
that the constraint solver spoke in as far as its inputs, outputs, and
error messages were concerned!

An example of an app that couldn't be built on the isopack-cache branch
before this commit and can be now is
  https://github.com/glasser/precise-constraints-example
(It can be built with 1.0, but only by compiling a version of `c` that
isn't part of .meteor/versions!)

Note that this also means that .meteor/versions expresses enough to
allow us to implement a simpler constraint check that doesn't need to do
the full tree walk of the constraint solver.  Such a checker would be
implemented as:

  - gather root dependencies and constraints (project constraints,
    release constraints, etc)
  - add the dependencies and constraints from all versions mentioned
    in .meteor/versions
  - see if the choices made in .meteor/versions satisfy these
    dependencies and constraints

This algorithm did NOT work previously, because you couldn't just look
at the constraints coming from `s@0.0.0` and say "they're satifisfied"
because you had to know to "ignore" the constraint on c#web.browser
because s#web.browser is not part of the app, which is not a fact that
actually got recorded in .meteor/versions.

(This commit does not implement this simpler constraint check, though.)
2014-11-22 14:20:39 -08:00
David Glasser
db90bd6944 os was later changed to plugin 2014-11-22 14:20:10 -08:00
David Glasser
7d1cb122fe constraint solver should know about plugin deps 2014-11-22 14:19:47 -08:00
David Glasser
a1237ba1a8 fix typo 2014-11-22 11:17:21 -08:00
Ben Newman
f15f56fff7 Upgrade pathwatcher to prevent tests from hanging.
I fixed https://github.com/atom/node-pathwatcher/issues/47 with
https://github.com/atom/node-pathwatcher/pull/48, and @kevinsawicki
published my changes to NPM as pathwatcher@2.3.3.

These changes make it so pathwatcher no longer keeps our test processes
alive when they would otherwise naturally exit.
2014-11-21 17:37:15 -08:00
David Greenspan
b1ba8c3a64 Fix comment in resolver 2014-11-21 12:43:01 -08:00
David Glasser
e281442835 Fix meteor update --patch.
Broken in d90511128b which is in 1.0 but not 0.9.4. So we should
not create any explicitly-marked-as-patch releases against 1.0.
2014-11-18 14:28:28 -08:00
David Glasser
f404f28275 Use simpler query for getSortedVersions
The old one read and parsed JSON blobs from sqlite instead of just using
pre-existing columns.
2014-11-18 14:02:07 -08:00
Ben Newman
88477358ac Add node-pathwatcher NPM package to dev bundle. 2014-11-17 20:27:29 -05:00