Commit Graph

38 Commits

Author SHA1 Message Date
David Glasser
399681e04f move sha1 from Builder to watch
since watch actually uses it and Builder doesn't
2014-11-12 17:13:41 -08:00
Ben Newman
fe487ef0a3 Further simplify NpmDiscards to perform no additional disk I/O. 2014-10-21 12:19:59 -04:00
Ben Newman
fd3b2b0296 Implement Npm.strip, for use in package.js.
Summary:
The `Npm.strip` method makes up for packages that have missing or
incomplete .npmignore files by telling the bundler to strip out certain
unnecessary files and/or directories during `meteor build`.

The `discards` parameter should be an object whose keys are NPM
package names and whose values are arrays of strings (or regular
expressions) that match paths in that package's directory that should be
stripped before installation. For example:

  Npm.strip({
    connect: [/*\.wmv$/],
    useragent: ["tests/"]
  });

This means (1) "remove any files with the `.wmv` extension from the
'connect' package directory" and (2) "remove the 'tests' directory from
the 'useragent' package directory."

The values can also be objects, in which case the keys of the nested
objects will match nested dependency names. For example,

  Npm.strip({
    connect: {
      multiparty: ["test/"]
    }
  });

will discard the "test" directory from the "multiparty" package that is
depended upon by the "connect" package.

If you need to discard files from both "connect" and "multiparty", here's
a little trick you can use:

  Npm.strip({
    connect: {
      connect: ["huge.wmv"],
      multiparty: ["test/"]
    }
  });

This makes intuitive sense because requiring the "connect" package from
the "connect" package always returns the package itself.

Test Plan:
Run `meteor rebuild <package>` for the packages whose package.js files I
modified, and verify that the `Npm.strip`ped paths are absent from the
generated bundle.

Reviewers: glasser, nim, emily

Differential Revision: https://phabricator.meteor.com/D865
2014-10-17 18:24:31 -04:00
David Glasser
ecdc772f28 Write unipackages and bundles read-only
This will mean that the tropohouse's unipackages are read-only
again (they used to be, due to extractTarGz's read-only nature, before
cross-platform merging was implemented)
2014-08-06 21:23:18 -07:00
David Glasser
4b7601aa92 builder: remove unused 'append' option 2014-08-06 20:44:41 -07:00
David Glasser
ddba5d4a3f Never copy dev bundle node_modules into bundle
symlink as a special case for runner only

future commits on this branch will add a
package.json/npm-shrinkwrap.json that can be used by "meteor bundle"
users
2014-07-01 14:09:44 -07:00
ekatek
bb77100672 DO NOT remove the : character when sanitizing paths. That breaks NPM for packages 2014-06-16 21:03:07 -07:00
David Glasser
9bc1589a1c builder.copyDirectory: copy symlinks as symlink
otherwise our new change to have tool/node_modules be a symlink breaks
2014-06-16 14:49:13 -07:00
David Glasser
7beedc48c7 use streamy copy in builder too 2014-06-13 16:24:41 -07:00
David Glasser
1fc5280e15 finish includeTool implementation 2014-04-28 16:45:44 -07:00
David Glasser
0d7de14f2c first stab at including tool in unipackage 2014-04-25 18:37:19 -07:00
David Glasser
d4f95d2c0c write node modules correctly for fat packages
also, fix generateFilename("x", {directory: true}) to actually generate
a unique x.
2014-03-06 23:04:10 -08:00
James Hamlin
be0bce89c1 Maintain file modes when copying a directory.
A package may depend on some files in its dependencies being executable,
so builder ought to respect the modes of source files when copying into
a bundle.
2014-02-14 00:24:49 -08:00
Naomi Seyfer
d91110bd9c Merge branch 'devel' into sso
Conflicts:
	packages/livedata/livedata_server.js
2014-02-04 14:04:30 -08:00
Nick Martin
d1331e9c7a Reuse sha1 hashes calculated by the file watcher.
The file watcher (watch.js) calculates the sha1 hash of source files
to see if they've changed.  For static assets in `public`, the bundler
then calculates the same hash again for the client manifest.

This commit adds a small optimization to return the hash calculated by
the file watcher, so that it can be reused by the bundler.

There are more optimizations that probably could be done to avoid
other unnecessary sha1 hash recalculations, but they'd likely need
larger architectural changes.
2014-02-04 01:06:59 -08:00
David Glasser
990450c2a6 Backport some comment-punctuation changes
From tool-refactoring to sso.

Makes the tool-refactoring/sso diff a little smaller (including removing
some files from it entirely) and easier to review.  Only took about five
minutes to prepare, I swear this isn't a total waste of time :)
2014-01-30 21:08:56 -05:00
David Glasser
1b82376b4b in theory, this might work 2013-07-31 22:56:57 -07:00
David Glasser
f84fc21e8f checkpoint for initFromAppDir rewrite 2013-07-31 22:56:38 -07:00
David Glasser
ea22b3ab02 Checkpoint for actually using WatchSets.
Have not yet touched initFromAppDir.
2013-07-31 22:56:38 -07:00
David Glasser
3f5edbfae9 remove dead builder.copyDirectory({depend:true}) code 2013-07-31 22:56:38 -07:00
David Glasser
9538bec685 Replace "asset directory" concept with manifest of assets.
We were partway here already: the client served assets from the manifest instead
of from a static directory (since 5b8e1c1), and we already generated the list of
assets in the slice JSON file. But on the server, we ignored that list and
re-walked the asset directory at bundle time.

Now, the idea of asset directory is solely a part of initFromAppDir.

This also fixes a bug where assets that weren't associated with on-disk files
wouldn't work properly if Asset.get* is called in a package loaded with
unipackage.load. Not really sure how dev-bundle-fetcher even worked...

Also fixes a bug in builder where generated filenames didn't actually avoid
duplicate files.

This does not bump BUILT_BY because the previous commit did, and this commit
will not be pushed without the previous commit.
2013-07-17 00:01:06 -07:00
David Glasser
f786fd6fa5 Write (but don't use) sourceMaps (but not sources) for client programs.
Add builder.writeToGeneratedFilename helper and use it a lot.
2013-07-12 10:11:29 -07:00
David Glasser
d05e83d958 paths in program.json should not start with /. 2013-07-12 10:11:29 -07:00
David Glasser
9bafe2f881 builder: create ".build" slightly more atomically.
Now there never is a moment where ".build" contains a fraction of a unipackage.
2013-06-14 14:04:55 -07:00
David Glasser
f87a0ce09e Clean up commented-out code. 2013-06-10 22:44:16 -07:00
David Glasser
e838349cca Add missing var. 2013-06-10 22:21:07 -07:00
David Glasser
a578cb0c75 Fix symlinking bug revealed by parent commit.
With the precedence bug fixed, more directories are placed into
usedAsFile... enough to break nodeModulesMode=symlink. Fortunately bundler-test
did catch this.

The somewhat hacky fix is to look carefully for reserved empty directories and
replace them with symlinks. This may not be 100% correct; see the XXX comment.
2013-06-10 22:17:37 -07:00
David Glasser
b9e96ab2b7 Fix operator precedence bugs. 2013-06-10 20:55:37 -07:00
Geoff Schmidt
524daf4379 minor builder path sanitization tweaks 2013-05-13 11:57:48 -07:00
Geoff Schmidt
a780f9f427 rv commit of stale copy of builder.js 2013-05-13 11:56:51 -07:00
Geoff Schmidt
02e1b3b1c7 Make 'meteor --help' startup >3x faster, with the help of a require() tree profiler. The speedup is entirely due to lazy-loading third party modules. 2013-05-13 11:56:51 -07:00
Geoff Schmidt
9dd512e769 Remove builder.copyFile (it actually duplicates the file option to builder.write) 2013-05-13 11:54:20 -07:00
Geoff Schmidt
607e637b22 Generate dev_bundle trampoline scripts in star balls. Nothing uses them yet. 2013-05-13 11:53:51 -07:00
Geoff Schmidt
3abacc44f1 move server config info out of program.json into a separate config.json 2013-05-13 11:53:51 -07:00
Geoff Schmidt
54ca2a4463 Fix typos in bundler that prevented normal bundling 2013-05-13 11:51:53 -07:00
Geoff Schmidt
2c1b297dc1 Facility for including unipackages from non-Meteor-built command-line nodejs programs (eg, 'meteor'). Does not fully duplicate the server environment (specifically, does not provide a HTTP server) so livedata does not yet load. 2013-05-13 11:51:53 -07:00
Geoff Schmidt
503a84a3aa Compile and save packages to Unipackage format for faster startup. Move npm processing to package build time. Let npm dependencies vary per-slice (eg, don't include npm packages used only by tests in non-test builds) -- implemented on disk but not yet exposed via API. 2013-05-13 11:51:53 -07:00
Geoff Schmidt
e26732d3dd Factor out bundle file-handling logic into new Builder class 2013-05-13 11:51:52 -07:00