Commit Graph

5027 Commits

Author SHA1 Message Date
Slava Kim
8a4b58d987 More on deep/shallow cloning 2013-12-02 19:17:36 -08:00
Slava Kim
034d2e628b Remove a couple of EJSON clones we don't need 2013-12-02 19:17:36 -08:00
Nick Martin
5f1b721823 Fix meteor run to look at the OPLOG_URL environment variable. 2013-12-02 19:17:36 -08:00
Slava Kim
8a00f1d713 Fix basic fields projection. 2013-12-02 19:17:36 -08:00
Slava Kim
2bbd69ad2c Tests to prove the basic fields implementation was wrong. 2013-12-02 19:17:35 -08:00
Slava Kim
e793c9d948 Remove unnecessary check.
This removed check will never be false as we pass the oplog selector that looks
for 'o.drop' and we assume that you can get only one command per oplog record.
2013-12-02 19:17:35 -08:00
Slava Kim
008c74df4b OPLOG_URL is accessed only by appConfig. 2013-12-02 19:17:35 -08:00
Slava Kim
111de8d803 Oplog observe handles 'drop collection' or db.c.drop() 2013-12-02 19:17:35 -08:00
Slava Kim
adb46face0 Basic handling for fields in oplog
Just run through projection before calling callbacks or keeping fields in published IdMap, or diffing.
2013-12-02 19:17:35 -08:00
David Glasser
74b4bd2bd6 Make DocFetcher more async.
This should use fewer fibers. nim, can you benchmark?
2013-12-02 19:17:35 -08:00
David Glasser
81c23f4baa minor refactor and add XXX suggestion 2013-12-02 19:17:35 -08:00
David Glasser
2fb5e66010 Some renames and minor fixes. 2013-12-02 19:17:34 -08:00
David Glasser
a5d805e633 delete dead code 2013-12-02 19:17:34 -08:00
David Glasser
70a28a6229 better way of not using more fibers 2013-12-02 19:17:34 -08:00
David Glasser
60f7aa75cd a few random notes 2013-12-02 19:17:34 -08:00
David Glasser
062b475134 don't die 2013-12-02 19:17:34 -08:00
David Glasser
2e5f96a867 clear lots of vars on oplog stop
also don't publish facts by default
2013-12-02 19:17:34 -08:00
David Glasser
5aecf9cbf4 don't use fibers in callWhenProcessedLatest 2013-12-02 19:17:34 -08:00
David Glasser
2d170a0971 Add some facts to oplog. 2013-12-02 19:17:34 -08:00
David Glasser
c7329ff617 Extra error checking around calling callbacks. 2013-12-02 19:17:33 -08:00
David Glasser
2f5ed9a3d3 Stop using weird "curiousity" nomenclature. 2013-12-02 19:17:33 -08:00
David Glasser
24913f862b When FETCHING, process add/delete/replace directly
Unless they are for one of the currently-fetching IDs.

Note that this can reorder observes! But so can poll-based observe. That
said, I think it can reorder them in a more noticable way.

Specifically, with poll-based observe, if you observe two operations A1
and A2 on a doc A, then you are guaranteed to have seen any operation on
any document that occured before A1. This is not the case after this
commit for oplog-based observe. (Not sure if it was before this
commit...)
2013-12-02 19:17:33 -08:00
David Glasser
2f1ebbbcc7 Restructure code to "allow" misordered sequencers.
Throw in an assertion that they aren't misordered anyway. This will be
removed before merge. But it should never fire for a singleton replset.
2013-12-02 19:17:33 -08:00
David Glasser
3960fcc963 More usable error messages for misordered oplog. 2013-12-02 19:17:33 -08:00
David Glasser
6ff2c1c9ce Set Mongo connection pool size better.
Fixes (for now) misordered oplog errors.
2013-12-02 19:17:33 -08:00
David Glasser
b4a598b3e5 Close connection to DB used to set up oplog. 2013-12-02 19:17:33 -08:00
David Glasser
9cc8ca6c2b Don't fire write fence until observes are steady. 2013-12-02 19:17:32 -08:00
David Glasser
f797a49beb Support oplog when MONGO_URL is a replset.
Node's URL parser doesn't support Mongo's "URLs" which contain commas in
the hostname part. Instead of trying to parse, just extract the database
name from ... the database!
2013-12-02 19:17:32 -08:00
David Glasser
bcbc73bc4a Add more info to the misordered oplog errors. 2013-12-02 19:17:32 -08:00
David Glasser
0d45c5f009 Limit poolsize for "last oplog entry" to 1.
This ensures that we allocate sequencers in increasing order.
2013-12-02 19:17:32 -08:00
David Glasser
19e41af82a Only use one connection for oplog tailing. 2013-12-02 19:17:32 -08:00
David Glasser
599a2f928e Separate oplog tail queries from other operations.
This prevents the "look up last oplog entry" queries from taking several
seconds 1/5 of the time when the query is on the same pooled connection
as the awaitdata tail query.
2013-12-02 19:17:32 -08:00
David Glasser
aae9fc23db replace sequencer writes with reads. 2013-12-02 19:17:32 -08:00
Slava Kim
f24be0684f Use MinimongoTest object to export private functions for unit tests 2013-12-02 19:17:31 -08:00
David Glasser
73b45f2fa2 Actually use Slava's function in oplog tailing.
Tests pass, but I have not confirmed that it actually has the desired
end-to-end effect.
2013-12-02 19:17:31 -08:00
David Glasser
81d77f916b Link to a PR and use latest version. 2013-12-02 19:17:31 -08:00
David Glasser
acb6decb6a Make run-tools-tests.sh pass. 2013-12-02 19:17:31 -08:00
David Glasser
0ddf232773 Restore accidentally-deleted mongo restart code.
Thanks, cli-test.sh!
2013-12-02 19:17:31 -08:00
Slava Kim
3dc3ad9997 Simplify getPaths - method for getting important key paths from a selector
object.
2013-12-02 19:17:31 -08:00
Slava Kim
12f5f4e87e Another test to ensure correct observation of object literals. 2013-12-02 19:17:31 -08:00
David Glasser
75f6037203 Set internal Mongo replay flag when tailing oplog. 2013-12-02 19:17:30 -08:00
David Glasser
3c39614e50 Fix typo in test. 2013-12-02 19:17:30 -08:00
Slava Kim
37120d186c One more test 2013-12-02 19:17:30 -08:00
Slava Kim
7fc451da3e Glasser's comments 2013-12-02 19:17:30 -08:00
Slava Kim
1c17d8af44 More work with numeric keys to be more precise answering if modifier affects selector. 2013-12-02 19:17:30 -08:00
Slava Kim
b5d9df66c1 Tests for "observe array element" case
ex.: collection.find({ 'foo.0.bar': 2 })
// => find docs with bar of first element of array foo being 2
2013-12-02 19:17:30 -08:00
Slava Kim
a75470bcf3 Fix the way we remove integer fields 2013-12-02 19:17:30 -08:00
Slava Kim
2dc25d77ae Fix prefix check. Remove all numeric keys from sel 2013-12-02 19:17:30 -08:00
David Glasser
df73dca222 Make a comment about something important into an XXX comment. 2013-12-02 19:17:29 -08:00
David Glasser
10f3e9c90f Add a comment about a non-redundant stopped check. 2013-12-02 19:17:29 -08:00