The previous implementation simply avoided calling watchSet.addFile for
potentially unused files, trusting that addFile would be called later if
the file was eventually used. However, this strategy left the contents of
watchSet.files incomplete for tasks such as IsopackCache._checkUpToDate,
which require full information about all files, even the ones that might
not be used by the bundle. The new strategy maintains metadata about
potentially unused files in a separate data structure, which will be
merged/cloned/serialized/deserialized along with other WatchSet data.
Unfortunately, this conversion triggered an error due to one of the
shortcomings of the Babel implementation of TypeScript:
SyntaxError: /tools/tool-env/profile.ts: Namespace not marked type-only declare. Non-declarative namespaces are only supported experimentally in Babel. To enable and review caveats see: https://babeljs.io/docs/en/babel-plugin-transform-typescript
278 | }
279 |
> 280 | export namespace Profile {
| ^
281 | export let enabled = !! process.env.METEOR_PROFILE;
282 |
283 | export function time<TResult>(bucket: string, f: () => TResult) {
at File.buildCodeFrameError (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/core/lib/transformation/file/file.js:261:12)
at transpileNamespace (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/plugin-transform-typescript/lib/namespace.js:25:25)
at PluginPass.TSModuleDeclaration (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/plugin-transform-typescript/lib/index.js:271:32)
at newFn (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/visitors.js:193:21)
at NodePath._call (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/path/context.js:53:20)
at NodePath.call (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/path/context.js:40:17)
at NodePath.visit (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/path/context.js:88:12)
at TraversalContext.visitQueue (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/context.js:118:16)
at TraversalContext.visitMultiple (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/context.js:85:17)
at TraversalContext.visit (/Users/ben/meteor/dev_bundle/lib/node_modules/@babel/traverse/lib/context.js:144:19)
Using the `isMaster` command instead of checking the replica set member state is more reliable because a node can be a primary without being writable.
Fixes#10381 and #9026.
For some reason, without all three of these environment variables set
(LANG, LC_ALL, and LANGUAGE), the STDOUT returned from the child process
in findMongoPids contained ?? in place of non-ASCII unicode characters,
which was causing the self-test of Mongo shell in a unicode application
directory to fail.
This implementation defaults all three environment variables to
process.env.LANG if it was defined, or "en_US.UTF-8" otherwise.
Creating the watcher can take up to 12+ seconds in small - medium apps, and uses sync fs calls.
The server would start right away, but the tool process wouldn't know about it until the watcher finished setting up. Also, the proxy doesn't forward requests until "=> Server restarted" is shown.
A new async option is added to Watcher which prevents it from blocking the event loop too long.
Also, the watcher and legacy bundle are only created after the server has started, or 3 seconds has passed.
Fixes#10073, per
https://github.com/meteor/meteor/issues/10073#issuecomment-405290391
While thinking about this bug, I realized that sending IPC messages to
specific packages in the server process was much less flexible than
sending messages based on an arbitrary topic string, since the topic
string approach allows both `autoupdate` and `dynamic-import` to listen
for the same message.
The topic string approach calls for a listener interface like
`onMessage(topic, callback)`, which elegantly replaces the previous
approach of requiring packages to export a single `onMessage` function.
However, because the `meteor` package does not have access to the module
system, implementing the `onMessage` listener interface in the `meteor`
package would have required exposing an API like `Meteor.onMessage(topic,
callback)`, which has an unpleasant global smell to it. Instead, the
`onMessage` function should be explicitly imported (using the module
system) from a less-generically-named package.
Since I knew I was going to have to move the message dispatch logic out of
the `meteor` package, I decided to create a new package called
`inter-process-messaging` to implement the parent/child components of the
IPC system.
https://github.com/meteor/meteor/pull/10055#discussion_r201855997
As I explained in this comment, Package._on(packageName, callback) was a
bad API because it never called the callback if the package was not
installed, which caused any app not using the autoupdate package to get
stuck trying to communicate with the autoupdate package.
Instead of having every message consumer listen to every message and act
on the ones that seem relevant to its interests, we now have a single
process.on("message", callback) hook that can dispatch messages to
different Meteor packages running in the server process.
Receiving packages should export an onMessage function. The onMessage
function may be async, and its result will be delivered back to the build
process as the result of the sendMessage Promise.
Most importantly, we no longer return a copy of previousBuilders from
bundler.bundle, but simply modify the input object over time. The copying
approach was nice in theory, but incompatible with delaying the bundling
of certain architectures (e.g. web.browser.legacy) until some time after
bundler.bundle returns.
* Update MongoDB driver to 3.0.5
* Use `MongoClient` instead of `Db` in Meteor Tool
* Update `mongo-livedata` test for new MongoDB driver version
* Consider `BulkWriteError` when checking MongoDB errors
* Introduce MONGO_BIND_IP override
Introduces an environment variable (MONGO_BIND_IP) for overriding of meteor's mongod's `bind_ip` option to allow docker containers to bind on `0.0.0.0` to be able to expose mongod processs via docker's `-p 3001:3001` port bindings.
Related issue: https://serverfault.com/questions/758225/cannot-connect-to-mongodb-in-docker
Partially Related PR: https://github.com/meteor/meteor/pull/469
* Use METEOR_MONGO_BIND_IP over ambiguous MONGO_BIND_IP
Change request to use METEOR_MONGO_BIND_IP over ambiguous MONGO_BIND_IP
This reverts commit dfc0702558.
We've seen some odd test failures (e.g. `passwords - tokens`) and trouble
updating from 1.6.1 to 1.6.2-beta.3 (easily solved with `meteor reset`,
but also worth investigating), so I think we should keep working on the
Mongo 3.6 upgrade in a PR, rather than running all of our tests against
a devel branch that includes Mongo 3.6.
cc @abernix @hwillson
Update the Meteor Tool to use Mongo 3.6.2 for 64-bit OS'
and Mongo 3.2.18 for 32-bit OS'. A few important mentions:
- As of Mongo 3.6, all Mongo binary downloads include
SSL - there is no longer a non-SSL based download bundle
(so it's a bit bigger, but that shouldn't be an issue).
- Using the `--nojournal` option with WiredTiger based
replica sets is no longer supported (see
https://jira.mongodb.org/browse/SERVER-30760). The
`--nojournal` flag was added in
bcfe072d52
to help reduce the amount of disk space used by Mongo,
but since this option is no longer supported, we'll
have to live with the extra disk space usage.
- Add PR link to History.md
This commit reverts much of the work @hwillson had put in place for
meteor/meteor#9173, which made it possible for 32-bit and 64-bit
Mongo versions to exist in harmony within the same dev-bundle. That
hard work was not in vein though as it offered invaluable research.
Ultimately, this showed that a more aggressive approach would be ideal,
even if the proposed option would have worked great in the short-term.
In the wake of the news that Mongo would no longer be supporting 32-bit
versions, these changes are important so 32-bit users of Meteor can
continue to have a functioning Mongo binary in development, while still
allowing Meteor to ship newer Mongo (e.g. 3.4+) binaries for 64-bit
users. This is particularly relevant for Windows users, who have
previously only had 32-bit Meteor builds and represented a majority of
"32-bit" development, despite the fact most of their hosts supported
64-bit. During another time in Meteor's life, this made sense.
This commit takes improved functionality to the next level (and makes
the aforementioned changes obsolete) by introducing support for building
and shipping Meteor for Windows in a 64-bit flavor (in addition to
32-bit), which will hopefully solve a number of performance matters on
Windows by using binaries which are pre-compiled for 64-bit, rather than
forcing the Windows kernel to deal with 32-bit binaries. While Windows
has shown it's quite capable of dealing with such a situation, it seems
more and more clear (given improvements in underlying dependencies) that
performance gains could be had by freeing Windows of its 32-bit work.
This commit also further perpetuates the "archinfo" plot-line with more
story (about Windows) and various spelling corrections.
These changes introduce dual Mongo support into the Meteor
Tool. 32-bit Mongo (3.2.15) will be used by Meteor when the
Tool is run on a 32-bit OS (32-bit Linux and Windows). 64-bit
Mongo (3.4.9) will be used when the Tool is run on a 64-bit
OS (64-bit Linux, Windows and macOS).
Fixes https://github.com/meteor/meteor-feature-requests/issues/129.
The syntax of these flags is the same as the equivalent Node.js options:
https://nodejs.org/en/docs/inspector/#command-line-options
When no port value is provided, the default is 9229.
Two notable differences:
* The flags affect the server process spawned by the parent build
process, rather than affecting the build process itself.
* The --inspect-brk flag causes the server process to pause just after
server code has loaded but before it begins to execute. This timing is
more useful than the Node.js --inspect-brk behavior, which is to pause
on the first instruction executed by the process, since that is too
early to set any useful breakpoints.
Implements https://github.com/meteor/meteor-feature-requests/issues/194.
The `meteor debug` command behaves like Node's `--inspect-brk` flag, in
that it attempts to pause the server before executing any server code.
However, simply passing the `--inspect-brk` flag to Node causes execution
to pause on the very first line of code, which is not good for setting any
breakpoints, because no server code has actually loaded yet.
Instead, the `meteor debug` command uses Node's `--inspect` flag to enable
debugging without an initial pause, then manually pauses at an appropriate
moment during server startup. Ideally, the pause should last until an
inspector client has been attached to the process, at which point the
developer has a chance to set any desired breakpoints, then clicks the
continue button to proceed with server startup.
The most difficult part of this process is detecting when the inspector
client has attached. Previously, the parent process listened for the child
process to print a "Debugger attached" message to STDERR, which happens as
a result of this `fprintf` call in Native C++:
7cff6e80bf/src/inspector_io.cc (L396)
However, this message was not printed in some cases, especially on Windows
(#9165), and required inter-process communication even in the ideal case.
All of that logic is gone now, thanks to this commit.
This commit takes advantage of a difference in behavior of the `debugger`
keyword depending on whether or not an inspector client is attached. When
no client is attached, the `debugger` keyword is a no-op that takes no
time (or very little time) to execute. Once a client has attached, the
`debugger` keyword triggers a breakpoint that lasts until the developer
explicitly continues execution through the client UI. Needless to say,
this makes the `debugger` keyword take longer than a no-op.
Because the `debugger` keyword does nothing until a client connects, we
can safely poll a `pause` function containing a `debugger` keyword at a
frequent interval (say, every 500ms). Once a client connects, the
`debugger` keyword will become active, pausing the server at exactly the
point we hoped. The difference is easy to detect by timing the `pause()`
function call. Once the `debugger` keyword becomes active, we stop polling
and allow server startup to continue.
Elegant!
Fixes#9165.
By my calculations, the sum of the sizes of the individual isopackets was
152MB, and the size of the combined isopacket is now just 36MB. That
remarkable difference goes to show how much duplication of transitive
dependencies was happening before this change.
That's a savings of 116MB for the (uncompressed) size of the meteor-tool
package. In Meteor 1.5.x, the meteor-tool package is about 544MB, but in
Meteor 1.6 it's considerably smaller: 373MB. In other words, this change
should reduce those sizes to 428MB (-21%) and 257MB (-31%), respectively.
* When awaiting Mongo, wait until the heartbeat has pulsed after election.
> Prologue: A heartbeat is used amongst members of a MongoDB replica set
to poll the status of said members.
When we are initiating a new replicaset for the test Mongo server, the
replicaset is not fully prepared to accept writes until the voting
members have negotiated and propagated their decision about who is the
"primary" to all members involved. This seems to be delayed by almost
_exactly_ the default heartbeat interval, which is 2000ms.
The heartbeat interval is marked as an "internal only" property in Mongo
so I was hesitant to lower it. It's also a new property in Mongo 3.2
which might explain why this cropped up a while ago.
I believe this heartbeat delay is the only explanation for why the
`rs.status()` (i.e. `replSetGetStatus`) believes it is ready before the
`mongod` has actually printed "transition to primary complete" to the
log.
Fixesmeteor/meteor#9026.
* Replace addl. variable occurrence with the new `firstMemberState` var.
