github/meteor
1
1
Fork 0
mirror of https://github.com/meteor/meteor.git synced 2026-05-02 03:01:46 -04:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'release-0.8.1'

Browse Source
This commit is contained in:
Emily Stark
2014-04-30 11:02:42 -07:00
parent f016894e7b 532e9f32a8
commit f4044c2fde
191 changed files with 5530 additions and 5915 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
.mailmap
View File

@@ -8,7 +8,10 @@
# For any emails that show up in the shortlog that aren't in one of
# these lists, figure out their GitHub username and add them.
GITHUB: aldeed <eric@dairystatedesigns.com>
GITHUB: AlexeyMK <alexey@alexeymk.com>
GITHUB: apendua <apendua@gmail.com>
GITHUB: arbesfeld <arbesfeld@gmail.com>
GITHUB: DenisGorbachev <Denis.Gorbachev@faster-than-wind.ru>
GITHUB: EOT <eot@gmx.at>
GITHUB: FooBarWidget <honglilai@gmail.com>
@@ -20,10 +23,12 @@ GITHUB: awwx <andrew.wilcox@gmail.com>
GITHUB: cmather <mather.chris@gmail.com>
GITHUB: codeinthehole <david.winterbottom@gmail.com>
GITHUB: dandv <ddascalescu+github@gmail.com>
GITHUB: davegonzalez <gonzalez.dalex@gmail.com>
GITHUB: emgee3 <hello@gravitronic.com>
GITHUB: icellan <icellan@icellan.com>
GITHUB: jacott <geoffjacobsen@gmail.com>
GITHUB: jfhamlin <jfhamlin@gmail.com>
GITHUB: justinsb <justin@fathomdb.com>
GITHUB: marcandre <github@marc-andre.ca>
GITHUB: mart-jansink <m.jansink@gmail.com>
GITHUB: meawoppl <meawoppl@gmail.com>
@@ -33,7 +38,9 @@ GITHUB: mitar <mitar.github@tnode.com>
GITHUB: mizzao <mizzao@gmail.com>
GITHUB: mquandalle <maxime.quandalle@gmail.com>
GITHUB: nathan-muir <ndmuir@gmail.com>
GITHUB: Neftedollar <oildollar@gmail.com>
GITHUB: paulswartz <paulswartz@gmail.com>
GITHUB: Pent <jon@empire5design.com>
GITHUB: queso <joshua.owens@gmail.com>
GITHUB: rdickert <robert.dickert@gmail.com>
GITHUB: rgould <rwgould@gmail.com>
@@ -50,6 +57,7 @@ METEOR: dgreensp <dgreenspan@alum.mit.edu>
METEOR: estark37 <emily@meteor.com>
METEOR: estark37 <estark37@gmail.com>
METEOR: glasser <glasser@meteor.com>
METEOR: glasser <glasser@davidglasser.net>
METEOR: gschmidt <geoff@geoffschmidt.com>
METEOR: karayu <lele.yu@gmail.com>
METEOR: n1mmy <nim@meteor.com>
@@ -57,3 +65,4 @@ METEOR: sixolet <naomi@meteor.com>
METEOR: Slava <slava@meteor.com>
METEOR: stubailo <sashko@mit.edu>
METEOR: ekatek <ekate@meteor.com>

125
History.md
View File

@@ -1,5 +1,130 @@
## v.NEXT
#### Meteor Accounts
* Fix a security flaw in OAuth1 and OAuth2 implementations. If you are
using any OAuth accounts packages (such as `accounts-google` or
`accounts-twitter`), we recommend that you update immediately and log
out your users' current sessions with the following MongoDB command:
$ db.users.update({}, { $set: { 'services.resume.loginTokens': [] } }, { multi: true });
* OAuth redirect URLs are now required to be on the same origin as your app.
* Log out a user's other sessions when they change their password.
* Store pending OAuth login results in the database instead of
in-memory, so that an OAuth flow succeeds even if different requests
go to different server processes.
* When validateLoginAttempt callbacks return false, don't override a more
specific error message.
* Add `Random.secret()` for generating security-critical secrets like
login tokens.
* `Meteor.logoutOtherClients` now calls the user callback when other
login tokens have actually been removed from the database, not when
they have been marked for eventual removal. #1915
* Rename `Oauth` to `OAuth`. `Oauth` is now an alias for backwards
compatibility.
* Add `oauth-encryption` package for encrypting sensitive account
credentials in the database.
* A validate login hook can now override the exception thrown from
`beginPasswordExchange` like it can for other login methods.
* Remove an expensive observe over all users in the `accounts-base`
package.
#### Blaze
* Disallow `javascript:` URLs in URL attribute values by default, to
help prevent cross-site scripting bugs. Call
`UI._allowJavascriptUrls()` to allow them.
* Fix `UI.toHTML` on templates containing `{{#with}}`.
* Fix `{{#with}}` over a data context that is mutated. #2046
* Clean up autoruns when calling `UI.toHTML`.
* Add support for `{{!-- block comments --}}` in Spacebars. Block comments may
contain `}}`, so they are more useful than `{{! normal comments}}` for
commenting out sections of Spacebars templates.
* Don't dynamically insert `<tbody>` tags in reactive tables
* When handling a custom jQuery event, additional arguments are
no longer lost -- they now come after the template instance
argument. #1988
#### DDP and MongoDB
* Extend latency compensation to support an arbitrary sequence of
inserts in methods. Previously, documents created inside a method
stub on the client would eventually be replaced by new documents
from the server, causing the screen to flicker. Calling `insert`
inside a method body now generates the same ID on the client (inside
the method stub) and on the server. A sequence of inserts also
generates the same sequence of IDs. Code that wants a random stream
that is consistent between method stub and real method execution can
get one with `DDP.randomStream`.
https://trello.com/c/moiiS2rP/57-pattern-for-creating-multiple-database-records-from-a-method
* DDP now has an implementation of bidirectional heartbeats which is consistent
across SockJS and websocket transports. This enables connection keepalive and
allows servers and clients to more consistently and efficiently detect
disconnection.
* The DDP protocol version number has been incremented to "pre2" (adding
randomSeed and heartbeats).
* The oplog observe driver handles errors communicating with MongoDB
better and knows to re-poll all queries after a MongoDB failover.
* Fix bugs involving mutating DDP method arguments.
#### meteor command-line tool
* Move boilerplate HTML from tools to webapp. Change internal
`Webapp.addHtmlAttributeHook` API.
* Add `meteor list-sites` command for listing the sites that you have
deployed to meteor.com with your Meteor developer account.
* Third-party template languages can request that their generated source loads
before other JavaScript files, just like *.html files, by passing the
isTemplate option to Plugin.registerSourceHandler.
* You can specify a particular interface for the dev mode runner to bind to with
`meteor -p host:port`.
* Don't include proprietary tar tags in bundle tarballs.
* Convert relative URLs to absolute URLs when merging CSS files.
#### Upgraded dependencies
* Node.js from 0.10.25 to 0.10.26.
* MongoDB driver from 1.3.19 to 1.4.1
* stylus: 0.42.3 (from 0.42.2)
* showdown: 0.3.1
* css-parse: an unreleased version (from 1.7.0)
* css-stringify: an unreleased version (from 1.4.1)
Patches contributed by GitHub users aldeed, apendua, arbesfeld, awwx, dandv,
davegonzalez, emgee3, justinsb, mquandalle, Neftedollar, Pent, sdarnell,
and timhaines.
## v0.8.0.1
* Fix security flaw in OAuth1 implementation. Clients can no longer

92
LICENSE.txt
View File

@@ -561,6 +561,65 @@ html5: https://github.com/aredridel/html5
Copyright (c) 2010 Aria Stewart <aredridel@nbtsc.org>
----------
node-aes-gcm: https://github.com/xorbit/node-aes-gcm
----------
Copyright (c) 2013 Patrick Van Oosterwijck
----------
nan: https://github.com/rvagg/nan
----------
Copyright 2013, NAN contributors:
- Rod Vagg <https://github.com/rvagg>
- Benjamin Byholm <https://github.com/kkoopa>
- Trevor Norris <https://github.com/trevnorris>
- Nathan Rajlich <https://github.com/TooTallNate>
- Brett Lawson <https://github.com/brett19>
- Ben Noordhuis <https://github.com/bnoordhuis>
(the "Original Author")
All rights reserved.
MIT +no-false-attribs License
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
Distributions of all or part of the Software intended to be used
by the recipients as they would use the unmodified Software,
containing modifications that substantially alter, remove, or
disable functionality of the Software, outside of the documented
configuration mechanisms provided by the Software, shall be
modified such that the Original Author's bug reporting email
addresses and urls are either replaced with the contact information
of the parties responsible for the changes, or removed entirely.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
Except where noted, this license applies to any and all software
programs and associated documentation files created by the
Original Author, when distributed with the Software.
==============
Apache License
@@ -1509,6 +1568,39 @@ OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to <http://unlicense.org/>
----------
npm-install-checks: https://github.com/npm/npm-install-checks
----------
Copyright (c) Robert Kowalski and Isaac Z. Schlueter ("Authors")
All rights reserved.
The BSD License
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS
BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
----------
tough-cookie: https://github.com/goinstant/tough-cookie
----------

2
docs/.meteor/release
View File

@@ -1 +1 @@
0.8.0.1
0.8.1

21
docs/client/api.html
View File

@@ -1710,6 +1710,11 @@ example, to support GitHub login, run `$ meteor add accounts-github` and use the
Session.set('errorMessage', err.reason || 'Unknown error');
});
Login service configuration is sent from the server to the client over DDP when
your app starts up; you may not call the login function until the configuration
is loaded. The function `Accounts.loginServicesConfigured()` is a reactive data
source that will return true once the login service is configured; you should
not make login buttons visible or active until it is true.
{{> api_box currentUser}}
@@ -1848,10 +1853,18 @@ proceed. If the callback returns a falsy value or throws an
exception, the login is aborted. Throwing a `Meteor.Error` will
report the error reason to the user.
All registered validate login callbacks are called, even if one of the
callbacks aborts the login. The later callbacks will see the
`allowed` field set to `false` since the login will now not be
successful.
All registered validate login callbacks are called, even if one of the callbacks
aborts the login. The later callbacks will see the `allowed` field set to
`false` since the login will now not be successful. This allows later callbacks
to override an error from a previous callback; for example, you could override
the "Incorrect password" error with a different message.
Validate login callbacks that aren't explicitly trying to override a previous
error generally have no need to run if the attempt has already been determined
to fail, and should start with
if (!attempt.allowed)
return false;
{{> api_box accounts_onLogin}}

27
docs/client/api.js
View File

@@ -58,12 +58,18 @@ Template.api.settings = {
id: "meteor_settings",
name: "Meteor.settings",
locus: "Anywhere",
descr: ["`Meteor.settings` contains deployment-specific configuration options. " +
"You can initialize settings by passing the `--settings` option (which takes a file containing JSON data) to " +
"`meteor run` or `meteor deploy`, " +
"or by setting your server process's `METEOR_SETTINGS` environment variable to a JSON string. " +
"If you don't provide any settings, `Meteor.settings` will be an empty object. If the settings object contains a key named `public`, then " +
"`Meteor.settings.public` will be available on the client as well as the server. All other properties of `Meteor.settings` are only defined on the server."]
descr: ["`Meteor.settings` contains deployment-specific configuration " +
"options. You can initialize settings by passing the `--settings` " +
"option (which takes the name of a file containing JSON data) to " +
"`meteor run` or `meteor deploy`. When running your server " +
"directly (e.g. from a bundle), you instead specify settings by " +
"putting the JSON directly into the `METEOR_SETTINGS` environment " +
"variable. " +
"If you don't provide any settings, `Meteor.settings` will be an " +
"empty object. If the settings object contains a key named " +
"`public`, then `Meteor.settings.public` will be available on the " +
"client as well as the server. All other properties of " +
"`Meteor.settings` are only defined on the server."]
};
Template.api.release = {
@@ -1158,6 +1164,11 @@ Template.api.accounts_config = {
name: "loginExpirationInDays",
type: "Number",
descr: "The number of days from when a user logs in until their token expires and they are logged out. Defaults to 90. Set to `null` to disable login expiration."
},
{
name: "oauthSecretKey",
type: "String",
descr: "When using the `oauth-encryption` package, the 16 byte key using to encrypt sensitive account credentials in the database, encoded in base64. This option may only be specifed on the server. See packages/oauth-encryption/README.md for details."
}
]
};
@@ -1678,7 +1689,7 @@ Template.api.httpcall = {
args: [
{name: "method",
type: "String",
descr: 'The HTTP method to use: "`GET`", "`POST`", "`PUT`", or "`DELETE`".'},
descr: 'The [HTTP method](http://en.wikipedia.org/wiki/HTTP_method) to use, such as "`GET`", "`POST`", or "`HEAD`".'},
{name: "url",
type: "String",
descr: 'The URL to retrieve.'},
@@ -1713,7 +1724,7 @@ Template.api.httpcall = {
descr: "Maximum time in milliseconds to wait for the request before failing. There is no timeout by default."},
{name: "followRedirects",
type: "Boolean",
descr: "If true, transparently follow HTTP redirects. Cannot be set to false on the client."}
descr: "If `true`, transparently follow HTTP redirects. Cannot be set to `false` on the client. Default `true`."}
]
};

2
docs/client/concepts.html
View File

@@ -741,7 +741,7 @@ To get started, run
This command will generate a fully-contained Node.js application in the form of
a tarball. To run this application, you need to provide Node.js 0.10 and a
MongoDB server. (The current release of Meteor has been tested with Node
0.10.25; older versions contain a serious bug that can cause production servers
0.10.26; older versions contain a serious bug that can cause production servers
to stall.) You can then run the application by invoking node, specifying the
HTTP port for the application to listen on, and the MongoDB endpoint. If
you don't already have a MongoDB server, we can recommend our friends at

97
docs/client/docs.css
View File

@@ -173,7 +173,7 @@ a:hover {
/** Main pane **/
#main {
margin: 10px;
margin: 10px 10px 10px 60px;
line-height: 1.3;
color: #333333;
}
@@ -407,10 +407,34 @@ dl.callbacks {
/** layout control **/
/* default to no sidebar */
#nav {
#menu-ico {
font-size: 30px;
float: right;
position: fixed;
top: 3px;
left: 6px;
}
#menu-ico.hidden {
display: none;
}
/* default to no sidebar */
#nav {
display: block;
background: #FFF;
position: fixed;
width: 260px;
height: 100%;
top: 0;
left: -220px;
}
#nav.show {
left: 0;
overflow: auto;
}
.github-ribbon {
display: none;
}
@@ -419,37 +443,44 @@ pre {
}
@media (min-width: 768px) {
/* ipad portrait or better */
#main {
width: 440px;
height: 100%;
margin-left: 260px; /* nav width + padding */
padding: 30px;
}
#nav {
display: block;
width: 200px;
position: fixed;
overflow: auto;
height: 100%;
top: 0;
left: 0;
}
.main-headline {
display: none;
}
/* ipad portrait or better */
#main {
width: 440px;
height: 100%;
margin-left: 260px; /* nav width + padding */
padding: 30px;
}
#nav {
display: block;
width: 200px;
position: fixed;
overflow: auto;
height: 100%;
top: 0;
left: 0;
}
.main-headline {
display: none;
}
#menu-ico {
display: none;
}
}
@media (min-width: 1024px) {
/* ipad landscape and desktop */
#main {
width: 610px;
margin-left: 330px; /* nav width + padding */
}
#nav {
width: 270px;
}
.github-ribbon {
display: block;
}
/* ipad landscape and desktop */
#main {
width: 610px;
margin-left: 330px; /* nav width + padding */
}
#nav {
width: 270px;
}
.github-ribbon {
display: block;
}
#menu-ico {
display: none;
}
}

3
docs/client/docs.html
View File

@@ -8,7 +8,7 @@
</head>
<body>
<div id="nav">
<div id="nav" class="hide">
{{> nav }}
</div>
<div id="main">
@@ -23,6 +23,7 @@
</body>
<template name="nav">
<div id="menu-ico"><a href="#">&#9776;</a></div>
<div id="nav-inner">
{{#each sections}}
{{#if type "spacer"}}

25
docs/client/docs.js
View File

@@ -88,8 +88,18 @@ Meteor.startup(function () {
// Make external links open in a new tab.
$('a:not([href^="#"])').attr('target', '_blank');
// Hide menu by tapping on background
$('#main').on('click', function () {
hideMenu();
});
});
var hideMenu = function () {
$('#nav').removeClass('show');
$('#menu-ico').removeClass('hidden');
};
var toc = [
{name: "Meteor " + Template.headline.release(), id: "top"}, [
"Quick start",
@@ -348,6 +358,7 @@ var toc = [
"force-ssl",
"jquery",
"less",
"oauth-encryption",
"random",
"spiderable",
"stylus",
@@ -405,7 +416,7 @@ Template.nav.sections = function () {
Template.nav.type = function (what) {
return this.type === what;
}
};
Template.nav.maybe_current = function () {
return Session.equals("section", this.id) ? "current" : "";
@@ -415,6 +426,18 @@ Template.nav_section.depthIs = function (n) {
return this.depth === n;
};
// Show hidden TOC when menu icon is tapped
Template.nav.events({
'click #menu-ico' : function () {
$('#nav').addClass('show');
$('#menu-ico').addClass('hidden');
},
// Hide TOC when selecting an item
'click a' : function () {
hideMenu();
}
});
UI.registerHelper('dstache', function() {
return '{{';
});

1
docs/client/packages.html
View File

@@ -28,6 +28,7 @@ and removed with:
{{> pkg_force_ssl}}
{{> pkg_jquery}}
{{> pkg_less}}
{{> pkg_oauth_encryption}}
{{> pkg_random}}
{{> pkg_spiderable}}
{{> pkg_stylus}}

9
docs/client/packages/oauth-encryption.html Normal file
View File

@@ -0,0 +1,9 @@
<template name="pkg_oauth_encryption">
{{#markdown}}
## `oauth-encryption`
Encrypts sensitive account credential information stored in the
database. See packages/oauth-encryption/README.md for details.
{{/markdown}}
</template>

15
docs/client/packages/random.html
View File

@@ -10,9 +10,18 @@ servers that don't have enough entropy to seed the cryptographically strong
generator).
<dl class="callbacks">
{{#dtdd "Random.id()"}}
Returns a unique identifier, such as `"Jjwjg6gouWLXhMGKW"`, that is likely to
be unique in the whole world.
{{#dtdd "Random.id([n])"}}
Returns a unique identifier, such as `"Jjwjg6gouWLXhMGKW"`, that is
likely to be unique in the whole world. The optional argument `n`
specifies the length of the identifier in characters and defaults to 17.
{{/dtdd}}
{{#dtdd "Random.secret([n])"}}
Returns a random string of printable characters with 6 bits of
entropy per character. The optional argument `n` specifies the length of
the secret string and defaults to 43 characters, or 256 bits of
entropy. Use `Random.secret` for security-critical secrets that are
intended for machine, rather than human, consumption.
{{/dtdd}}
{{#dtdd "Random.fraction()"}}

8
docs/client/packages/spiderable.html
View File

@@ -33,6 +33,14 @@ If you deploy your application with `meteor bundle`, you must install
`$PATH`. If you use `meteor deploy` this is already taken care of.
{{/warning}}
{{#warning}}
When running your page, `spiderable` will wait for all publications
to be ready. Make sure that all of your [`publish functions`](#meteor_publish)
either return a cursor (or an array of cursors), or eventually call
[`this.ready()`](#publish_ready). Otherwise, the `phantomjs` executions
will fail.
{{/warning}}
{{/markdown}}
</template>

2
docs/lib/release-override.js
View File

@@ -1,5 +1,5 @@
// While galaxy apps are on their own special meteor releases, override
// Meteor.release here.
if (Meteor.isClient) {
Meteor.release = Meteor.release ? "0.8.0.1" : undefined;
Meteor.release = Meteor.release ? "0.8.1" : undefined;
}

2
examples/clock/.meteor/release
View File

@@ -1 +1 @@
0.8.0.1
0.8.1

2
examples/leaderboard/.meteor/release
View File

@@ -1 +1 @@
0.8.0.1
0.8.1

2
examples/parties/.meteor/release
View File

@@ -1 +1 @@
0.8.0.1
0.8.1

2
examples/todos/.meteor/release
View File

@@ -1 +1 @@
0.8.0.1
0.8.1

2
examples/wordplay/.meteor/release
View File

@@ -1 +1 @@
0.8.0.1
0.8.1

2
meteor
View File

@@ -1,6 +1,6 @@
#!/bin/bash
BUNDLE_VERSION=0.3.33
BUNDLE_VERSION=0.3.34
# OS Check. Put here because here is where we download the precompiled
# bundles that are arch specific.

66
packages/accounts-base/accounts_client.js
View File

@@ -215,38 +215,40 @@ Meteor.logout = function (callback) {
};
Meteor.logoutOtherClients = function (callback) {
// Call the `logoutOtherClients` method. Store the login token that we get
// back and use it to log in again. The server is not supposed to close
// connections on the old token for 10 seconds, so we should have time to
// store our new token and log in with it before being disconnected. If we get
// disconnected, then we'll immediately reconnect with the new token. If for
// some reason we get disconnected before storing the new token, then the
// worst that will happen is that we'll have a flicker from trying to log in
// with the old token before storing and logging in with the new one.
Accounts.connection.apply('logoutOtherClients', [], { wait: true },
function (error, result) {
if (error) {
callback && callback(error);
} else {
var userId = Meteor.userId();
storeLoginToken(userId, result.token, result.tokenExpires);
// If the server hasn't disconnected us yet by deleting our
// old token, then logging in now with the new valid token
// will prevent us from getting disconnected. If the server
// has already disconnected us due to our old invalid token,
// then we would have already tried and failed to login with
// the old token on reconnect, and we have to make sure a
// login method gets sent here with the new token.
Meteor.loginWithToken(result.token, function (err) {
if (err &&
storedLoginToken() &&
storedLoginToken().token === result.token) {
makeClientLoggedOut();
}
callback && callback(err);
});
}
});
// We need to make two method calls: one to replace our current token,
// and another to remove all tokens except the current one. We want to
// call these two methods one after the other, without any other
// methods running between them. For example, we don't want `logout`
// to be called in between our two method calls (otherwise the second
// method call would return an error). Another example: we don't want
// logout to be called before the callback for `getNewToken`;
// otherwise we would momentarily log the user out and then write a
// new token to localStorage.
//
// To accomplish this, we make both calls as wait methods, and queue
// them one after the other, without spinning off the event loop in
// between. Even though we queue `removeOtherTokens` before
// `getNewToken`, we won't actually send the `removeOtherTokens` call
// until the `getNewToken` callback has finished running, because they
// are both wait methods.
Accounts.connection.apply(
'getNewToken',
[],
{ wait: true },
function (err, result) {
if (! err) {
storeLoginToken(Meteor.userId(), result.token, result.tokenExpires);
}
}
);
Accounts.connection.apply(
'removeOtherTokens',
[],
{ wait: true },
function (err) {
callback && callback(err);
}
);
};

12
packages/accounts-base/accounts_common.js
View File

@@ -53,6 +53,18 @@ Accounts.config = function(options) {
"server; some configuration options may not take effect.");
}
// We need to validate the oauthSecretKey option at the time
// Accounts.config is called. We also deliberately don't store the
// oauthSecretKey in Accounts._options.
if (_.has(options, "oauthSecretKey")) {
if (Meteor.isClient)
throw new Error("The oauthSecretKey option may only be specified on the server");
if (! Package["oauth-encryption"])
throw new Error("The oauth-encryption package must be loaded to set oauthSecretKey");
Package["oauth-encryption"].OAuthEncryption.loadKey(options.oauthSecretKey);
options = _.omit(options, "oauthSecretKey");
}
// validate option keys
var VALID_KEYS = ["sendVerificationEmail", "forbidClientAccountCreation",
"restrictCreationByEmailDomain", "loginExpirationInDays"];

342
packages/accounts-base/accounts_server.js
View File

@@ -73,12 +73,19 @@ var validateLogin = function (connection, attempt) {
}
catch (e) {
attempt.allowed = false;
// XXX this means the last thrown error overrides previous error
// messages. Maybe this is surprising to users and we should make
// overriding errors more explicit. (see
// https://github.com/meteor/meteor/issues/1960)
attempt.error = e;
return true;
}
if (! ret) {
attempt.allowed = false;
attempt.error = new Meteor.Error(403, "Login forbidden");
// don't override a specific error provided by a previous
// validator or the initial attempt (eg "incorrect password").
if (!attempt.error)
attempt.error = new Meteor.Error(403, "Login forbidden");
}
return true;
});
@@ -227,6 +234,9 @@ var attemptLogin = function (methodInvocation, methodName, methodArgs, result) {
if (!result)
throw new Error("result is required");
// XXX A programming error in a login handler can lead to this occuring, and
// then we don't call onLogin or onLoginFailure callbacks. Should
// tryLoginMethod catch this case and turn it into an error?
if (!result.userId && !result.error)
throw new Error("A login method must specify a userId or an error");
@@ -245,6 +255,9 @@ var attemptLogin = function (methodInvocation, methodName, methodArgs, result) {
if (user)
attempt.user = user;
// validateLogin may mutate `attempt` by adding an error and changing allowed
// to false, but that's the only change it can make (and the user's callbacks
// only get a clone of `attempt`).
validateLogin(methodInvocation.connection, attempt);
if (attempt.allowed) {
@@ -296,6 +309,9 @@ Accounts._reportLoginFailure = function (methodInvocation, methodName, methodArg
validateLogin(methodInvocation.connection, attempt);
failedLogin(methodInvocation.connection, attempt);
// validateLogin may mutate attempt to set a new error message. Return
// the modified version.
return attempt;
};
@@ -418,6 +434,17 @@ Meteor.methods({
// use. Tests set Accounts._noConnectionCloseDelayForTest to delete tokens
// immediately instead of using a delay.
//
// XXX COMPAT WITH 0.7.2
// This single `logoutOtherClients` method has been replaced with two
// methods, one that you call to get a new token, and another that you
// call to remove all tokens except your own. The new design allows
// clients to know when other clients have actually been logged
// out. (The `logoutOtherClients` method guarantees the caller that
// the other clients will be logged out at some point, but makes no
// guarantees about when.) This method is left in for backwards
// compatibility, especially since application code might be calling
// this method directly.
//
// @returns {Object} Object with token and tokenExpires keys.
logoutOtherClients: function () {
var self = this;
@@ -435,7 +462,7 @@ Meteor.methods({
var tokens = user.services.resume.loginTokens;
var newToken = Accounts._generateStampedLoginToken();
var userId = self.userId;
Meteor.users.update(self.userId, {
Meteor.users.update(userId, {
$set: {
"services.resume.loginTokensToDelete": tokens,
"services.resume.haveLoginTokensToDelete": true
@@ -456,8 +483,60 @@ Meteor.methods({
tokenExpires: Accounts._tokenExpiration(newToken.when)
};
} else {
throw new Error("You are not logged in.");
throw new Meteor.Error("You are not logged in.");
}
},
// Generates a new login token with the same expiration as the
// connection's current token and saves it to the database. Associates
// the connection with this new token and returns it. Throws an error
// if called on a connection that isn't logged in.
//
// @returns Object
// If successful, returns { token: <new token>, id: <user id>,
// tokenExpires: <expiration date> }.
getNewToken: function () {
var self = this;
var user = Meteor.users.findOne(self.userId, {
fields: { "services.resume.loginTokens": 1 }
});
if (! self.userId || ! user) {
throw new Meteor.Error("You are not logged in.");
}
// Be careful not to generate a new token that has a later
// expiration than the curren token. Otherwise, a bad guy with a
// stolen token could use this method to stop his stolen token from
// ever expiring.
var currentHashedToken = Accounts._getLoginToken(self.connection.id);
var currentStampedToken = _.find(
user.services.resume.loginTokens,
function (stampedToken) {
return stampedToken.hashedToken === currentHashedToken;
}
);
if (! currentStampedToken) { // safety belt: this should never happen
throw new Meteor.Error("Invalid login token");
}
var newStampedToken = Accounts._generateStampedLoginToken();
newStampedToken.when = currentStampedToken.when;
Accounts._insertLoginToken(self.userId, newStampedToken);
return loginUser(self, self.userId, newStampedToken);
},
// Removes all tokens except the token associated with the current
// connection. Throws an error if the connection is not logged
// in. Returns nothing on success.
removeOtherTokens: function () {
var self = this;
if (! self.userId) {
throw new Meteor.Error("You are not logged in.");
}
var currentToken = Accounts._getLoginToken(self.connection.id);
Meteor.users.update(self.userId, {
$pull: {
"services.resume.loginTokens": { hashedToken: { $ne: currentToken } }
}
});
}
});
@@ -492,7 +571,7 @@ Accounts._setAccountData = function (connectionId, field, value) {
Meteor.server.onConnection(function (connection) {
accountData[connection.id] = {connection: connection};
connection.onClose(function () {
removeConnectionFromToken(connection.id);
removeTokenFromConnection(connection.id);
delete accountData[connection.id];
});
});
@@ -551,26 +630,32 @@ Accounts._clearAllLoginTokens = function (userId) {
);
};
// hashed token -> list of connection ids
var connectionsByLoginToken = {};
// connection id -> observe handle for the login token that this
// connection is currently associated with, or null. Null indicates that
// we are in the process of setting up the observe.
var userObservesForConnections = {};
// test hook
Accounts._getTokenConnections = function (token) {
return connectionsByLoginToken[token];
Accounts._getUserObserve = function (connectionId) {
return userObservesForConnections[connectionId];
};
// Remove the connection from the list of open connections for the connection's
// token.
var removeConnectionFromToken = function (connectionId) {
var token = Accounts._getLoginToken(connectionId);
if (token) {
connectionsByLoginToken[token] = _.without(
connectionsByLoginToken[token],
connectionId
);
if (_.isEmpty(connectionsByLoginToken[token]))
delete connectionsByLoginToken[token];
// Clean up this connection's association with the token: that is, stop
// the observe that we started when we associated the connection with
// this token.
var removeTokenFromConnection = function (connectionId) {
if (_.has(userObservesForConnections, connectionId)) {
var observe = userObservesForConnections[connectionId];
if (observe === null) {
// We're in the process of setting up an observe for this
// connection. We can't clean up that observe yet, but if we
// delete the null placeholder for this connection, then the
// observe will get cleaned up as soon as it has been set up.
delete userObservesForConnections[connectionId];
} else {
delete userObservesForConnections[connectionId];
observe.stop();
}
}
};
@@ -580,62 +665,76 @@ Accounts._getLoginToken = function (connectionId) {
// newToken is a hashed token.
Accounts._setLoginToken = function (userId, connection, newToken) {
removeConnectionFromToken(connection.id);
removeTokenFromConnection(connection.id);
Accounts._setAccountData(connection.id, 'loginToken', newToken);
if (newToken) {
if (! _.has(connectionsByLoginToken, newToken))
connectionsByLoginToken[newToken] = [];
connectionsByLoginToken[newToken].push(connection.id);
// Now that we've added the connection to the
// connectionsByLoginToken map for the token, the connection will
// be closed if the token is removed from the database. However
// at this point the token might have already been deleted, which
// wouldn't have closed the connection because it wasn't in the
// map yet.
// Set up an observe for this token. If the token goes away, we need
// to close the connection. We defer the observe because there's
// no need for it to be on the critical path for login; we just need
// to ensure that the connection will get closed at some point if
// the token gets deleted.
//
// We also did need to first add the connection to the map above
// (and now remove it here if the token was deleted), because we
// could be getting a response from the database that the token
// still exists, but then it could be deleted in another fiber
// before our `findOne` call returns... and then that other fiber
// would need for the connection to be in the map for it to close
// the connection.
//
// We defer this check because there's no need for it to be on the critical
// path for login; we just need to ensure that the connection will get
// closed at some point if the token has been deleted.
// Initially, we set the observe for this connection to null; this
// signifies to other code (which might run while we yield) that we
// are in the process of setting up an observe for this
// connection. Once the observe is ready to go, we replace null with
// the real observe handle (unless the placeholder has been deleted,
// signifying that the connection was closed already -- in this case
// we just clean up the observe that we started).
userObservesForConnections[connection.id] = null;
Meteor.defer(function () {
if (! Meteor.users.findOne({
var foundMatchingUser;
// Because we upgrade unhashed login tokens to hashed tokens at
// login time, sessions will only be logged in with a hashed
// token. Thus we only need to observe hashed tokens here.
var observe = Meteor.users.find({
_id: userId,
"services.resume.loginTokens.hashedToken": newToken
})) {
removeConnectionFromToken(connection.id);
'services.resume.loginTokens.hashedToken': newToken
}, { fields: { _id: 1 } }).observeChanges({
added: function () {
foundMatchingUser = true;
},
removed: function () {
connection.close();
// The onClose callback for the connection takes care of
// cleaning up the observe handle and any other state we have
// lying around.
}
});
// If the user ran another login or logout command we were waiting for
// the defer or added to fire, then we let the later one win (start an
// observe, etc) and just stop our observe now.
//
// Similarly, if the connection was already closed, then the onClose
// callback would have called removeTokenFromConnection and there won't be
// an entry in userObservesForConnections. We can stop the observe.
if (Accounts._getAccountData(connection.id, 'loginToken') !== newToken ||
!_.has(userObservesForConnections, connection.id)) {
observe.stop();
return;
}
if (userObservesForConnections[connection.id] !== null) {
throw new Error("Non-null user observe for connection " +
connection.id + " while observe was being set up?");
}
userObservesForConnections[connection.id] = observe;
if (! foundMatchingUser) {
// We've set up an observe on the user associated with `newToken`,
// so if the new token is removed from the database, we'll close
// the connection. But the token might have already been deleted
// before we set up the observe, which wouldn't have closed the
// connection because the observe wasn't running yet.
connection.close();
}
});
}
};
// Close all open connections associated with any of the tokens in
// `tokens`.
var closeConnectionsForTokens = function (tokens) {
_.each(tokens, function (token) {
if (_.has(connectionsByLoginToken, token)) {
// safety belt. close should defer potentially yielding callbacks.
Meteor._noYieldsAllowed(function () {
_.each(connectionsByLoginToken[token], function (connectionId) {
var connection = Accounts._getAccountData(connectionId, 'connection');
if (connection)
connection.close();
});
});
}
});
};
// Login handler for resume tokens.
Accounts.registerLoginHandler("resume", function(options) {
if (!options.resume)
@@ -735,7 +834,7 @@ Accounts.registerLoginHandler("resume", function(options) {
// (Also used by Meteor Accounts server and tests).
//
Accounts._generateStampedLoginToken = function () {
return {token: Random.id(), when: (new Date)};
return {token: Random.secret(), when: (new Date)};
};
///
@@ -797,6 +896,67 @@ maybeStopExpireTokensInterval = function () {
expireTokenInterval = Meteor.setInterval(expireTokens,
EXPIRE_TOKENS_INTERVAL_MS);
///
/// OAuth Encryption Support
///
var OAuthEncryption = Package["oauth-encryption"] && Package["oauth-encryption"].OAuthEncryption;
var usingOAuthEncryption = function () {
return OAuthEncryption && OAuthEncryption.keyIsLoaded();
};
// OAuth service data is temporarily stored in the pending credentials
// collection during the oauth authentication process. Sensitive data
// such as access tokens are encrypted without the user id because
// we don't know the user id yet. We re-encrypt these fields with the
// user id included when storing the service data permanently in
// the users collection.
//
var pinEncryptedFieldsToUser = function (serviceData, userId) {
_.each(_.keys(serviceData), function (key) {
var value = serviceData[key];
if (OAuthEncryption && OAuthEncryption.isSealed(value))
value = OAuthEncryption.seal(OAuthEncryption.open(value), userId);
serviceData[key] = value;
});
};
// Encrypt unencrypted login service secrets when oauth-encryption is
// added.
//
// XXX For the oauthSecretKey to be available here at startup, the
// developer must call Accounts.config({oauthSecretKey: ...}) at load
// time, instead of in a Meteor.startup block, because the startup
// block in the app code will run after this accounts-base startup
// block. Perhaps we need a post-startup callback?
Meteor.startup(function () {
if (!usingOAuthEncryption())
return;
var ServiceConfiguration =
Package['service-configuration'].ServiceConfiguration;
ServiceConfiguration.configurations.find( {$and: [
{ secret: {$exists: true} },
{ "secret.algorithm": {$exists: false} }
] } ).
forEach(function (config) {
ServiceConfiguration.configurations.update(
config._id,
{ $set: {
secret: OAuthEncryption.seal(config.secret)
} }
);
});
});
///
/// CREATE USER HOOKS
///
@@ -833,6 +993,11 @@ Accounts.insertUserDoc = function (options, user) {
// collections)
user = _.extend({createdAt: new Date(), _id: Random.id()}, user);
if (user.services)
_.each(user.services, function (serviceData) {
pinEncryptedFieldsToUser(serviceData, user._id);
});
var fullUser;
if (onCreateUserHook) {
fullUser = onCreateUserHook(options, user);
@@ -968,6 +1133,8 @@ Accounts.updateOrCreateUserFromExternalService = function(
var user = Meteor.users.findOne(selector);
if (user) {
pinEncryptedFieldsToUser(serviceData, user._id);
// We *don't* process options (eg, profile) for update, but we do replace
// the serviceData (eg, so that we keep an unexpired access token and
// don't cache old email addresses in serviceData.email).
@@ -1104,6 +1271,10 @@ Meteor.methods({
Package['service-configuration'].ServiceConfiguration;
if (ServiceConfiguration.configurations.findOne({service: options.service}))
throw new Meteor.Error(403, "Service " + options.service + " already configured");
if (_.has(options, "secret") && usingOAuthEncryption())
options.secret = OAuthEncryption.seal(options.secret);
ServiceConfiguration.configurations.insert(options);
}
});
@@ -1180,44 +1351,3 @@ Meteor.startup(function () {
deleteSavedTokens(user._id, user.services.resume.loginTokensToDelete);
});
});
///
/// LOGGING OUT DELETED USERS
///
// When login tokens are removed from the database, close any sessions
// logged in with those tokens.
//
// Because we upgrade unhashed login tokens to hashed tokens at login
// time, sessions will only be logged in with a hashed token. Thus we
// only need to pull out hashed tokens here.
var closeTokensForUser = function (userTokens) {
closeConnectionsForTokens(_.compact(_.pluck(userTokens, "hashedToken")));
};
// Like _.difference, but uses EJSON.equals to compute which values to return.
var differenceObj = function (array1, array2) {
return _.filter(array1, function (array1Value) {
return ! _.some(array2, function (array2Value) {
return EJSON.equals(array1Value, array2Value);
});
});
};
Meteor.users.find({}, { fields: { "services.resume": 1 }}).observe({
changed: function (newUser, oldUser) {
var removedTokens = [];
if (newUser.services && newUser.services.resume &&
oldUser.services && oldUser.services.resume) {
removedTokens = differenceObj(oldUser.services.resume.loginTokens || [],
newUser.services.resume.loginTokens || []);
} else if (oldUser.services && oldUser.services.resume) {
removedTokens = oldUser.services.resume.loginTokens || [];
}
closeTokensForUser(removedTokens);
},
removed: function (oldUser) {
if (oldUser.services && oldUser.services.resume)
closeTokensForUser(oldUser.services.resume.loginTokens || []);
}
});

76
packages/accounts-base/accounts_tests.js
View File

@@ -1,3 +1,9 @@
Meteor.methods({
getCurrentLoginToken: function () {
return Accounts._getLoginToken(this.connection.id);
}
});
// XXX it'd be cool to also test that the right thing happens if options
// *are* validated, but Accounts._options is global state which makes this hard
// (impossible?)
@@ -297,3 +303,73 @@ Tinytest.addAsync(
);
}
);
Tinytest.add(
'accounts - get new token',
function (test) {
// Test that the `getNewToken` method returns us a valid token, with
// the same expiration as our original token.
var userId = Accounts.insertUserDoc({}, { username: Random.id() });
var stampedToken = Accounts._generateStampedLoginToken();
Accounts._insertLoginToken(userId, stampedToken);
var conn = DDP.connect(Meteor.absoluteUrl());
conn.call('login', { resume: stampedToken.token });
test.equal(conn.call('getCurrentLoginToken'),
Accounts._hashLoginToken(stampedToken.token));
var newTokenResult = conn.call('getNewToken');
test.equal(newTokenResult.tokenExpires,
Accounts._tokenExpiration(stampedToken.when));
test.equal(conn.call('getCurrentLoginToken'),
Accounts._hashLoginToken(newTokenResult.token));
conn.disconnect();
// A second connection should be able to log in with the new token
// we got.
var secondConn = DDP.connect(Meteor.absoluteUrl());
secondConn.call('login', { resume: newTokenResult.token });
secondConn.disconnect();
}
);
Tinytest.addAsync(
'accounts - remove other tokens',
function (test, onComplete) {
// Test that the `removeOtherTokens` method removes all tokens other
// than the caller's token, thereby logging out and closing other
// connections.
var userId = Accounts.insertUserDoc({}, { username: Random.id() });
var stampedTokens = [];
var conns = [];
_.times(2, function (i) {
stampedTokens.push(Accounts._generateStampedLoginToken());
Accounts._insertLoginToken(userId, stampedTokens[i]);
var conn = DDP.connect(Meteor.absoluteUrl());
conn.call('login', { resume: stampedTokens[i].token });
test.equal(conn.call('getCurrentLoginToken'),
Accounts._hashLoginToken(stampedTokens[i].token));
conns.push(conn);
});
conns[0].call('removeOtherTokens');
simplePoll(
function () {
var tokens = _.map(conns, function (conn) {
return conn.call('getCurrentLoginToken');
});
return ! tokens[1] &&
tokens[0] === Accounts._hashLoginToken(stampedTokens[0].token);
},
function () { // success
_.each(conns, function (conn) {
conn.disconnect();
});
onComplete();
},
function () { // timed out
throw new Error("accounts - remove other tokens timed out");
}
);
}
);

3
packages/accounts-base/package.js
View File

@@ -30,6 +30,8 @@ Package.on_use(function (api) {
// it's loaded.
api.use('autopublish', 'server', {weak: true});
api.use('oauth-encryption', 'server', {weak: true});
api.export('Accounts');
api.add_files('accounts_common.js', ['client', 'server']);
@@ -50,5 +52,6 @@ Package.on_test(function (api) {
api.use('tinytest');
api.use('random');
api.use('test-helpers');
api.use('oauth-encryption');
api.add_files('accounts_tests.js', 'server');
});

8
packages/accounts-oauth/oauth_client.js
View File

@@ -2,14 +2,18 @@
// access in the popup this should log the user in, otherwise
// nothing should happen.
Accounts.oauth.tryLoginAfterPopupClosed = function(credentialToken, callback) {
var credentialSecret = OAuth._retrieveCredentialSecret(credentialToken) || null;
Accounts.callLoginMethod({
methodArguments: [{oauth: {credentialToken: credentialToken}}],
methodArguments: [{oauth: {
credentialToken: credentialToken,
credentialSecret: credentialSecret
}}],
userCallback: callback && function (err) {
// Allow server to specify a specify subclass of errors. We should come
// up with a more generic way to do this!
if (err && err instanceof Meteor.Error &&
err.error === Accounts.LoginCancelledError.numericError) {
callback(new Accounts.LoginCancelledError(err.details));
callback(new Accounts.LoginCancelledError(err.reason));
} else {
callback(err);
}

19
packages/accounts-oauth/oauth_server.js
View File

@@ -4,9 +4,19 @@ Accounts.registerLoginHandler(function (options) {
if (!options.oauth)
return undefined; // don't handle
check(options.oauth, {credentialToken: String});
check(options.oauth, {
credentialToken: String,
// When an error occurs while retrieving the access token, we store
// the error in the pending credentials table, with a secret of
// null. The client can call the login method with a secret of null
// to retrieve the error.
credentialSecret: Match.OneOf(null, String)
});
if (!Oauth.hasCredential(options.oauth.credentialToken)) {
var result = OAuth.retrieveCredential(options.oauth.credentialToken,
options.oauth.credentialSecret);
if (!result) {
// OAuth credentialToken is not recognized, which could be either
// because the popup was closed by the user before completion, or
// some sort of error where the oauth provider didn't talk to our
@@ -23,9 +33,10 @@ Accounts.registerLoginHandler(function (options) {
// XXX we want `type` to be the service name such as "facebook"
return { type: "oauth",
error: new Meteor.Error(
Accounts.LoginCancelledError.numericError, "Login canceled") };
Accounts.LoginCancelledError.numericError,
"No matching login attempt found") };
}
var result = Oauth.retrieveCredential(options.oauth.credentialToken);
if (result instanceof Error)
// We tried to login, but there was a fatal error. Report it back
// to the user.

2
packages/accounts-oauth/package.js
View File

@@ -11,7 +11,7 @@ Package.on_use(function (api) {
api.use('accounts-base', ['client', 'server']);
// Export Accounts (etc) to packages using this one.
api.imply('accounts-base', ['client', 'server']);
api.use('oauth', 'server');
api.use('oauth');
api.add_files('oauth_common.js');
api.add_files('oauth_client.js', 'client');

37
packages/accounts-password/password_server.js
View File

@@ -73,12 +73,13 @@ Meteor.methods({beginPasswordExchange: function (request) {
// the second step method ('login') is called. If a user calls
// 'beginPasswordExchange' but then never calls the second step
// 'login' method, no login hook will fire.
Accounts._reportLoginFailure(self, 'beginPasswordExchange', arguments, {
// The validate login hooks can mutate the exception to be thrown.
var attempt = Accounts._reportLoginFailure(self, 'beginPasswordExchange', arguments, {
type: 'password',
error: err,
userId: user && user._id
});
throw err;
throw attempt.error;
}
// Save results so we can verify them later.
@@ -198,10 +199,20 @@ Meteor.methods({changePassword: function (options) {
if (!verifier)
throw new Meteor.Error(400, "Invalid verifier");
// XXX this should invalidate all login tokens other than the current one
// (or it should assign a new login token, replacing existing ones)
Meteor.users.update({_id: this.userId},
{$set: {'services.password.srp': verifier}});
// It would be better if this removed ALL existing tokens and replaced
// the token for the current connection with a new one, but that would
// be tricky, so we'll settle for just replacing all tokens other than
// the one for the current connection.
var currentToken = Accounts._getLoginToken(this.connection.id);
Meteor.users.update(
{ _id: this.userId },
{
$set: { 'services.password.srp': verifier },
$pull: {
'services.resume.loginTokens': { hashedToken: { $ne: currentToken } }
}
}
);
var ret = {passwordChanged: true};
if (serialized)
@@ -253,7 +264,7 @@ Accounts.sendResetPasswordEmail = function (userId, email) {
if (!email || !_.contains(_.pluck(user.emails || [], 'address'), email))
throw new Error("No such email for user.");
var token = Random.id();
var token = Random.secret();
var when = new Date();
Meteor.users.update(userId, {$set: {
"services.password.reset": {
@@ -302,7 +313,7 @@ Accounts.sendEnrollmentEmail = function (userId, email) {
throw new Error("No such email for user.");
var token = Random.id();
var token = Random.secret();
var when = new Date();
Meteor.users.update(userId, {$set: {
"services.password.reset": {
@@ -313,14 +324,14 @@ Accounts.sendEnrollmentEmail = function (userId, email) {
}});
var enrollAccountUrl = Accounts.urls.enrollAccount(token);
var options = {
to: email,
from: Accounts.emailTemplates.from,
subject: Accounts.emailTemplates.enrollAccount.subject(user),
text: Accounts.emailTemplates.enrollAccount.text(user, enrollAccountUrl)
};
if (typeof Accounts.emailTemplates.enrollAccount.html === 'function')
options.html =
Accounts.emailTemplates.enrollAccount.html(user, enrollAccountUrl);
@@ -425,7 +436,7 @@ Accounts.sendVerificationEmail = function (userId, address) {
var tokenRecord = {
token: Random.id(),
token: Random.secret(),
address: address,
when: new Date()};
Meteor.users.update(
@@ -433,14 +444,14 @@ Accounts.sendVerificationEmail = function (userId, address) {
{$push: {'services.email.verificationTokens': tokenRecord}});
var verifyEmailUrl = Accounts.urls.verifyEmail(tokenRecord.token);
var options = {
to: address,
from: Accounts.emailTemplates.from,
subject: Accounts.emailTemplates.verifyEmail.subject(user),
text: Accounts.emailTemplates.verifyEmail.text(user, verifyEmailUrl)
};
if (typeof Accounts.emailTemplates.verifyEmail.html === 'function')
options.html =
Accounts.emailTemplates.verifyEmail.html(user, verifyEmailUrl);

196
packages/accounts-password/password_tests.js
View File

@@ -33,7 +33,12 @@ if (Meteor.isClient) (function () {
}, 10 * 1000, 100);
};
var invalidateLoginsStep = function (test, expect) {
Meteor.call("testInvalidateLogins", true, expect(function (error) {
Meteor.call("testInvalidateLogins", 'fail', expect(function (error) {
test.isFalse(error);
}));
};
var hideActualLoginErrorStep = function (test, expect) {
Meteor.call("testInvalidateLogins", 'hide', expect(function (error) {
test.isFalse(error);
}));
};
@@ -191,6 +196,56 @@ if (Meteor.isClient) (function () {
logoutStep
]);
testAsyncMulti("passwords - changing password logs out other clients", [
function (test, expect) {
this.username = Random.id();
this.email = Random.id() + '-intercept@example.com';
this.password = 'password';
this.password2 = 'password2';
Accounts.createUser(
{ username: this.username, email: this.email, password: this.password },
loggedInAs(this.username, test, expect));
},
// Log in a second connection as this user.
function (test, expect) {
var self = this;
self.secondConn = DDP.connect(Meteor.absoluteUrl());
self.secondConn.call('login',
{ user: { username: self.username }, password: self.password },
expect(function (err, result) {
test.isFalse(err);
self.secondConn.setUserId(result.id);
test.isTrue(self.secondConn.userId());
self.secondConn.onReconnect = function () {
self.secondConn.apply(
'login',
[{ resume: result.token }],
{ wait: true },
function (err, result) {
self.secondConn.setUserId(result && result.id || null);
}
);
};
}));
},
function (test, expect) {
var self = this;
Accounts.changePassword(self.password, self.password2, expect(function (err) {
test.isFalse(err);
}));
},
// Now that we've changed the password, wait until the second
// connection gets logged out.
function (test, expect) {
var self = this;
pollUntil(expect, function () {
return self.secondConn.userId() === null;
}, 10 * 1000, 100);
}
]);
testAsyncMulti("passwords - new user hooks", [
function (test, expect) {
@@ -418,23 +473,60 @@ if (Meteor.isClient) (function () {
test.equal(Meteor.userId(), null);
}));
},
logoutStep,
function (test, expect) {
var self = this;
// Test that Meteor.logoutOtherClients logs out a second
// authentcated connection while leaving Accounts.connection
// logged in.
var secondConn = DDP.connect(Meteor.absoluteUrl());
var token;
// copied from livedata/client_convenience.js
self.ddpUrl = '/';
if (typeof __meteor_runtime_config__ !== "undefined") {
if (__meteor_runtime_config__.DDP_DEFAULT_CONNECTION_URL)
self.ddpUrl = __meteor_runtime_config__.DDP_DEFAULT_CONNECTION_URL;
}
// XXX can we get the url from the existing connection somehow
// instead?
var expectSecondConnLoggedOut = expect(function (err, result) {
test.isTrue(err);
});
var expectAccountsConnLoggedIn = expect(function (err, result) {
test.isFalse(err);
});
var expectSecondConnLoggedIn = expect(function (err, result) {
test.equal(result.token, token);
test.isFalse(err);
Meteor.logoutOtherClients(function (err) {
test.isFalse(err);
secondConn.call('login', { resume: token },
expectSecondConnLoggedOut);
Accounts.connection.call('login', {
resume: Accounts._storedLoginToken()
}, expectAccountsConnLoggedIn);
});
});
Meteor.loginWithPassword(
self.username,
self.password,
expect(function (err) {
test.isFalse(err);
token = Accounts._storedLoginToken();
test.isTrue(token);
secondConn.call('login', { resume: token },
expectSecondConnLoggedIn);
})
);
},
logoutStep,
// The tests below this point are for the deprecated
// `logoutOtherClients` method.
function (test, expect) {
var self = this;
// Test that Meteor.logoutOtherClients logs out a second authenticated
// connection while leaving Accounts.connection logged in.
var token;
var userId;
self.secondConn = DDP.connect(self.ddpUrl);
self.secondConn = DDP.connect(Meteor.absoluteUrl());
var expectLoginError = expect(function (err) {
test.isTrue(err);
@@ -461,7 +553,6 @@ if (Meteor.isClient) (function () {
token = Accounts._storedLoginToken();
self.beforeLogoutOthersToken = token;
test.isTrue(token);
userId = Meteor.userId();
self.secondConn.call("login", { resume: token },
expectSecondConnLoggedIn);
}));
@@ -495,41 +586,9 @@ if (Meteor.isClient) (function () {
);
},
logoutStep,
function (test, expect) {
var self = this;
// Test that, when we call logoutOtherClients, if the server disconnects
// us before the logoutOtherClients callback runs, then we still end up
// logged in.
var expectServerLoggedIn = expect(function (err, result) {
test.isFalse(err);
test.isTrue(Meteor.userId());
test.equal(result, Meteor.userId());
});
Meteor.loginWithPassword(
self.username,
self.password,
expect(function (err) {
test.isFalse(err);
test.isTrue(Meteor.userId());
// The test is only useful if things interleave in the following order:
// - logoutOtherClients runs on the server
// - onReconnect fires and sends a login method with the old token,
// which results in an error
// - logoutOtherClients callback runs and stores the new token and
// logs in with it
// In practice they seem to interleave this way, but I'm not sure how
// to make sure that they do.
Meteor.logoutOtherClients(function (err) {
test.isFalse(err);
Meteor.call("getUserId", expectServerLoggedIn);
});
})
);
},
logoutStep,
function (test, expect) {
var self = this;
// Test that deleting a user logs out that user's connections.
@@ -562,6 +621,28 @@ if (Meteor.isClient) (function () {
})
);
},
validateLoginsStep,
function (test, expect) {
Meteor.loginWithPassword(
"no such user",
"some password",
expect(function (error) {
test.isTrue(error);
test.equal(error.reason, 'User not found');
})
);
},
hideActualLoginErrorStep,
function (test, expect) {
Meteor.loginWithPassword(
"no such user",
"some password",
expect(function (error) {
test.isTrue(error);
test.equal(error.reason, 'hide actual error');
})
);
},
validateLoginsStep
]);
@@ -725,7 +806,7 @@ if (Meteor.isServer) (function () {
// XXX would be nice to test Accounts.config({forbidClientAccountCreation: true})
Tinytest.addAsync(
'passwords - login tokens cleaned up',
'passwords - login token observes get cleaned up',
function (test, onComplete) {
var username = Random.id();
Accounts.createUser({
@@ -737,8 +818,7 @@ if (Meteor.isServer) (function () {
test,
function (clientConn, serverConn) {
serverConn.onClose(function () {
test.isFalse(_.contains(
Accounts._getTokenConnections(token), serverConn.id));
test.isFalse(Accounts._getUserObserve(serverConn.id));
onComplete();
});
var result = clientConn.call('login', {
@@ -748,9 +828,25 @@ if (Meteor.isServer) (function () {
test.isTrue(result);
var token = Accounts._getAccountData(serverConn.id, 'loginToken');
test.isTrue(token);
test.isTrue(_.contains(
Accounts._getTokenConnections(token), serverConn.id));
clientConn.disconnect();
// We poll here, instead of just checking `_getUserObserve`
// once, because the login method defers the creation of the
// observe, and setting up the observe yields, so we could end
// up here before the observe has been set up.
simplePoll(
function () {
return !! Accounts._getUserObserve(serverConn.id);
},
function () {
test.isTrue(Accounts._getUserObserve(serverConn.id));
clientConn.disconnect();
},
function () {
test.fail("timed out waiting for user observe for connection " +
serverConn.id);
onComplete();
}
);
},
onComplete
);

24
packages/accounts-password/password_tests_setup.js
View File

@@ -15,14 +15,14 @@ Accounts.onCreateUser(function (options, user) {
});
// connection id -> true
// connection id -> action
var invalidateLogins = {};
Meteor.methods({
testInvalidateLogins: function (flag) {
if (flag)
invalidateLogins[this.connection.id] = true;
testInvalidateLogins: function (action) {
if (action)
invalidateLogins[this.connection.id] = action;
else
delete invalidateLogins[this.connection.id];
}
@@ -30,9 +30,19 @@ Meteor.methods({
Accounts.validateLoginAttempt(function (attempt) {
return ! (attempt &&
attempt.connection &&
invalidateLogins[attempt.connection.id]);
var action =
attempt &&
attempt.connection &&
invalidateLogins[attempt.connection.id];
if (! action)
return true;
else if (action === 'fail')
return false;
else if (action === 'hide')
throw new Meteor.Error(403, 'hide actual error');
else
throw new Error('unknown action: ' + action);
});

2
packages/appcache/appcache-server.js
View File

@@ -57,7 +57,7 @@ var browserEnabled = function(request) {
WebApp.addHtmlAttributeHook(function (request) {
if (browserEnabled(request))
return 'manifest="/app.manifest"';
return { manifest: "/app.manifest" };
else
return null;
});

36
packages/deps/deps.js
View File

@@ -15,10 +15,23 @@ var setCurrentComputation = function (c) {
Deps.active = !! c;
};
// _assign is like _.extend or the upcoming Object.assign.
// Copy src's own, enumerable properties onto tgt and return
// tgt.
var _hasOwnProperty = Object.prototype.hasOwnProperty;
var _assign = function (tgt, src) {
for (var k in src) {
if (_hasOwnProperty.call(src, k))
tgt[k] = src[k];
}
return tgt;
};
var _debugFunc = function () {
// lazy evaluation because `Meteor` does not exist right away
return (typeof Meteor !== "undefined" ? Meteor._debug :
((typeof console !== "undefined") && console.log ? console.log :
((typeof console !== "undefined") && console.log ?
function () { console.log.apply(console, arguments); } :
function () {}));
};
@@ -36,7 +49,7 @@ var _throwOrLog = function (from, e) {
// where `_noYieldsAllowed` is a no-op. `f` may be a computation
// function or an onInvalidate callback.
var callWithNoYieldsAllowed = function (f, comp) {
if (Meteor.isClient) {
if ((typeof Meteor === 'undefined') || Meteor.isClient) {
f(comp);
} else {
Meteor._noYieldsAllowed(function () {
@@ -60,7 +73,7 @@ var inCompute = false;
// `true` if the `_throwFirstError` option was passed in to the call
// to Deps.flush that we are in. When set, throw rather than log the
// first error encountered while flushing. Before throwing the error,
// finish flushing (from a catch block), logging any subsequent
// finish flushing (from a finally block), logging any subsequent
// errors.
var throwFirstError = false;
@@ -116,7 +129,7 @@ Deps.Computation = function (f, parent) {
}
};
_.extend(Deps.Computation.prototype, {
_assign(Deps.Computation.prototype, {
// http://docs.meteor.com/#computation_oninvalidate
onInvalidate: function (f) {
@@ -213,7 +226,7 @@ Deps.Dependency = function () {
this._dependentsById = {};
};
_.extend(Deps.Dependency.prototype, {
_assign(Deps.Dependency.prototype, {
// http://docs.meteor.com/#dependency_depend
//
// Adds `computation` to this set if it is not already
@@ -255,7 +268,7 @@ _.extend(Deps.Dependency.prototype, {
}
});
_.extend(Deps, {
_assign(Deps, {
// http://docs.meteor.com/#deps_flush
flush: function (_opts) {
// XXX What part of the comment below is still true? (We no longer
@@ -279,6 +292,7 @@ _.extend(Deps, {
willFlush = true;
throwFirstError = !! (_opts && _opts._throwFirstError);
var finishedTry = false;
try {
while (pendingComputations.length ||
afterFlushCallbacks.length) {
@@ -300,11 +314,13 @@ _.extend(Deps, {
}
}
}
} catch (e) {
inFlush = false; // needed before calling `Deps.flush()` again
Deps.flush({_throwFirstError: false}); // finish flushing
throw e;
finishedTry = true;
} finally {
if (! finishedTry) {
// we're erroring
inFlush = false; // needed before calling `Deps.flush()` again
Deps.flush({_throwFirstError: false}); // finish flushing
}
willFlush = false;
inFlush = false;
}

2
packages/ejson/ejson.js
View File

@@ -114,6 +114,8 @@ var builtinConverters = [
},
fromJSONValue: function (obj) {
var typeName = obj.$type;
if (!_.has(customTypes, typeName))
throw new Error("Custom EJSON type " + typeName + " is not defined");
var converter = customTypes[typeName];
return converter(obj.$value);
}

9
packages/facebook/facebook_client.js
View File

@@ -15,12 +15,13 @@ Facebook.requestCredential = function (options, credentialRequestCompleteCallbac
var config = ServiceConfiguration.configurations.findOne({service: 'facebook'});
if (!config) {
credentialRequestCompleteCallback && credentialRequestCompleteCallback(new ServiceConfiguration.ConfigError("Service not configured"));
credentialRequestCompleteCallback && credentialRequestCompleteCallback(
new ServiceConfiguration.ConfigError());
return;
}
var credentialToken = Random.id();
var mobile = /Android|webOS|iPhone|iPad|iPod|BlackBerry/i.test(navigator.userAgent);
var credentialToken = Random.secret();
var mobile = /Android|webOS|iPhone|iPad|iPod|BlackBerry|Windows Phone/i.test(navigator.userAgent);
var display = mobile ? 'touch' : 'popup';
var scope = "email";
@@ -32,7 +33,7 @@ Facebook.requestCredential = function (options, credentialRequestCompleteCallbac
'&redirect_uri=' + Meteor.absoluteUrl('_oauth/facebook?close') +
'&display=' + display + '&scope=' + scope + '&state=' + credentialToken;
Oauth.showPopup(
OAuth.showPopup(
loginUrl,
_.bind(credentialRequestCompleteCallback, null, credentialToken)
);

8
packages/facebook/facebook_server.js
View File

@@ -3,7 +3,7 @@ Facebook = {};
var querystring = Npm.require('querystring');
Oauth.registerService('facebook', 2, null, function(query) {
OAuth.registerService('facebook', 2, null, function(query) {
var response = getTokenResponse(query);
var accessToken = response.accessToken;
@@ -44,7 +44,7 @@ var isJSON = function (str) {
var getTokenResponse = function (query) {
var config = ServiceConfiguration.configurations.findOne({service: 'facebook'});
if (!config)
throw new ServiceConfiguration.ConfigError("Service not configured");
throw new ServiceConfiguration.ConfigError();
var responseContent;
try {
@@ -54,7 +54,7 @@ var getTokenResponse = function (query) {
params: {
client_id: config.appId,
redirect_uri: Meteor.absoluteUrl("_oauth/facebook?close"),
client_secret: config.secret,
client_secret: OAuth.openSecret(config.secret),
code: query.code
}
}).content;
@@ -96,5 +96,5 @@ var getIdentity = function (accessToken) {
};
Facebook.retrieveCredential = function(credentialToken) {
return Oauth.retrieveCredential(credentialToken);
return OAuth.retrieveCredential(credentialToken);
};

8
packages/github/github_client.js
View File

@@ -14,10 +14,11 @@ Github.requestCredential = function (options, credentialRequestCompleteCallback)
var config = ServiceConfiguration.configurations.findOne({service: 'github'});
if (!config) {
credentialRequestCompleteCallback && credentialRequestCompleteCallback(new ServiceConfiguration.ConfigError("Service not configured"));
credentialRequestCompleteCallback && credentialRequestCompleteCallback(
new ServiceConfiguration.ConfigError());
return;
}
var credentialToken = Random.id();
var credentialToken = Random.secret();
var scope = (options && options.requestPermissions) || [];
var flatScope = _.map(scope, encodeURIComponent).join('+');
@@ -29,8 +30,7 @@ Github.requestCredential = function (options, credentialRequestCompleteCallback)
'&redirect_uri=' + Meteor.absoluteUrl('_oauth/github?close') +
'&state=' + credentialToken;
Oauth.showPopup(
OAuth.showPopup(
loginUrl,
_.bind(credentialRequestCompleteCallback, null, credentialToken),
{width: 900, height: 450}

10
packages/github/github_server.js
View File

@@ -1,6 +1,6 @@
Github = {};
Oauth.registerService('github', 2, null, function(query) {
OAuth.registerService('github', 2, null, function(query) {
var accessToken = getAccessToken(query);
var identity = getIdentity(accessToken);
@@ -8,7 +8,7 @@ Oauth.registerService('github', 2, null, function(query) {
return {
serviceData: {
id: identity.id,
accessToken: accessToken,
accessToken: OAuth.sealSecret(accessToken),
email: identity.email,
username: identity.login
},
@@ -24,7 +24,7 @@ if (Meteor.release)
var getAccessToken = function (query) {
var config = ServiceConfiguration.configurations.findOne({service: 'github'});
if (!config)
throw new ServiceConfiguration.ConfigError("Service not configured");
throw new ServiceConfiguration.ConfigError();
var response;
try {
@@ -37,7 +37,7 @@ var getAccessToken = function (query) {
params: {
code: query.code,
client_id: config.clientId,
client_secret: config.secret,
client_secret: OAuth.openSecret(config.secret),
redirect_uri: Meteor.absoluteUrl("_oauth/github?close"),
state: query.state
}
@@ -68,5 +68,5 @@ var getIdentity = function (accessToken) {
Github.retrieveCredential = function(credentialToken) {
return Oauth.retrieveCredential(credentialToken);
return OAuth.retrieveCredential(credentialToken);
};

7
packages/google/google_client.js
View File

@@ -16,11 +16,12 @@ Google.requestCredential = function (options, credentialRequestCompleteCallback)
var config = ServiceConfiguration.configurations.findOne({service: 'google'});
if (!config) {
credentialRequestCompleteCallback && credentialRequestCompleteCallback(new ServiceConfiguration.ConfigError("Service not configured"));
credentialRequestCompleteCallback && credentialRequestCompleteCallback(
new ServiceConfiguration.ConfigError());
return;
}
var credentialToken = Random.id();
var credentialToken = Random.secret();
// always need this to get user id from google.
var requiredScope = ['profile'];
@@ -53,7 +54,7 @@ Google.requestCredential = function (options, credentialRequestCompleteCallback)
loginUrl += '&hd=' + encodeURIComponent(Accounts._options.restrictCreationByEmailDomain);
}
Oauth.showPopup(
OAuth.showPopup(
loginUrl,
_.bind(credentialRequestCompleteCallback, null, credentialToken),
{ height: 406 }

8
packages/google/google_server.js
View File

@@ -5,7 +5,7 @@ Google.whitelistedFields = ['id', 'email', 'verified_email', 'name', 'given_name
'family_name', 'picture', 'locale', 'timezone', 'gender'];
Oauth.registerService('google', 2, null, function(query) {
OAuth.registerService('google', 2, null, function(query) {
var response = getTokens(query);
var accessToken = response.accessToken;
@@ -38,7 +38,7 @@ Oauth.registerService('google', 2, null, function(query) {
var getTokens = function (query) {
var config = ServiceConfiguration.configurations.findOne({service: 'google'});
if (!config)
throw new ServiceConfiguration.ConfigError("Service not configured");
throw new ServiceConfiguration.ConfigError();
var response;
try {
@@ -46,7 +46,7 @@ var getTokens = function (query) {
"https://accounts.google.com/o/oauth2/token", {params: {
code: query.code,
client_id: config.clientId,
client_secret: config.secret,
client_secret: OAuth.openSecret(config.secret),
redirect_uri: Meteor.absoluteUrl("_oauth/google?close"),
grant_type: 'authorization_code'
}});
@@ -79,5 +79,5 @@ var getIdentity = function (accessToken) {
Google.retrieveCredential = function(credentialToken) {
return Oauth.retrieveCredential(credentialToken);
return OAuth.retrieveCredential(credentialToken);
};

3
packages/html-tools/package.js
View File

@@ -1,6 +1,7 @@
Package.describe({
summary: "Standards-compliant HTML tools"
summary: "Standards-compliant HTML tools",
internal: true
});
Package.on_use(function (api) {

3
packages/html5-tokenizer/README.md
View File

@@ -1,3 +0,0 @@
Code comes from https://github.com/aredridel/html5 a.k.a. `npm html5`, but
just the tokenizer, no parsing and no `jsdom` dependency.

90
packages/html5-tokenizer/buffer.js
View File

@@ -1,90 +0,0 @@
var Buffer = HTML5.Buffer = function Buffer() {
this.data = '';
this.start = 0;
this.committed = 0;
this.eof = false;
};
Buffer.prototype = {
slice: function() {
if(this.start >= this.data.length) {
if(!this.eof) throw HTML5.DRAIN;
return HTML5.EOF;
}
return this.data.slice(this.start, this.data.length);
},
char: function() {
if(!this.eof && this.start >= this.data.length - 1) throw HTML5.DRAIN;
if(this.start >= this.data.length) {
return HTML5.EOF;
}
return this.data[this.start++];
},
advance: function(amount) {
this.start += amount;
if(this.start >= this.data.length) {
if(!this.eof) throw HTML5.DRAIN;
return HTML5.EOF;
} else {
if(this.committed > this.data.length / 2) {
// Sliiiide
this.data = this.data.slice(this.committed);
this.start = this.start - this.committed;
this.committed = 0;
}
}
},
matchWhile: function(re) {
if(this.eof && this.start >= this.data.length ) return '';
var r = new RegExp("^"+re+"+");
var m = r.exec(this.slice());
if(m) {
if(!this.eof && m[0].length == this.data.length - this.start) throw HTML5.DRAIN;
this.advance(m[0].length);
return m[0];
} else {
return '';
}
},
matchUntil: function(re) {
var m, s;
s = this.slice();
if(s === HTML5.EOF) {
return '';
} else if(m = new RegExp(re + (this.eof ? "|\0|$" : "|\0")).exec(this.slice())) {
var t = this.data.slice(this.start, this.start + m.index);
this.advance(m.index);
return t.toString();
} else {
throw HTML5.DRAIN;
}
},
append: function(data) {
this.data += data;
},
shift: function(n) {
if(!this.eof && this.start + n >= this.data.length) throw HTML5.DRAIN;
if(this.eof && this.start >= this.data.length) return HTML5.EOF;
var d = this.data.slice(this.start, this.start + n).toString();
this.advance(Math.min(n, this.data.length - this.start));
return d;
},
peek: function(n) {
if(!this.eof && this.start + n >= this.data.length) throw HTML5.DRAIN;
if(this.eof && this.start >= this.data.length) return HTML5.EOF;
return this.data.slice(this.start, Math.min(this.start + n, this.data.length)).toString();
},
length: function() {
return this.data.length - this.start - 1;
},
unget: function(d) {
if(d === HTML5.EOF) return;
this.start -= (d.length);
},
undo: function() {
this.start = this.committed;
},
commit: function() {
this.committed = this.start;
}
};

759
packages/html5-tokenizer/constants.js
View File

@@ -1,759 +0,0 @@
HTML5 = (typeof HTML5 === 'undefined' ? {} : HTML5);
HTML5.CONTENT_MODEL_FLAGS = [
'PCDATA',
'RCDATA',
'CDATA',
'SCRIPT_CDATA',
'PLAINTEXT'
];
HTML5.Marker = {type: 'Marker', data: 'this is a marker token'};
(function() {
function EOF() {
}
EOF.prototype = {
toString: function() { throw new Error("EOF added as string"); }
};
HTML5.EOF = new EOF();
})();
HTML5.EOF_TOK = {type: 'EOF', data: 'End of File' };
HTML5.DRAIN = -2;
HTML5.SCOPING_ELEMENTS = [
'applet', 'caption', 'html', 'table', 'td', 'th',
'marquee', 'object', 'math:mi', 'math:mo', 'math:mn', 'math:ms', 'math:mtext',
'math:annotation-xml', 'svg:foreignObject', 'svg:desc', 'svg:title'
];
HTML5.LIST_SCOPING_ELEMENTS = [
'ol', 'ul',
'applet', 'caption', 'html', 'table', 'td', 'th',
'marquee', 'object', 'math:mi', 'math:mo', 'math:mn', 'math:ms', 'math:mtext',
'math:annotation-xml', 'svg:foreignObject', 'svg:desc', 'svg:title'
];
HTML5.BUTTON_SCOPING_ELEMENTS = [
'button',
'applet', 'caption', 'html', 'table', 'td', 'th',
'marquee', 'object', 'math:mi', 'math:mo', 'math:mn', 'math:ms', 'math:mtext',
'math:annotation-xml', 'svg:foreignObject', 'svg:desc', 'svg:title'
];
HTML5.TABLE_SCOPING_ELEMENTS = [
'table', 'html'
];
HTML5.SELECT_SCOPING_ELEMENTS = [
'option', 'optgroup'
];
HTML5.FORMATTING_ELEMENTS = [
'a',
'b',
'big',
'code',
'em',
'font',
'i',
'nobr',
's',
'small',
'strike',
'strong',
'tt',
'u'
];
HTML5.SPECIAL_ELEMENTS = [
'address',
'area',
'base',
'basefont',
'bgsound',
'blockquote',
'body',
'br',
'center',
'col',
'colgroup',
'dd',
'dir',
'div',
'dl',
'dt',
'embed',
'fieldset',
'form',
'frame',
'frameset',
'h1',
'h2',
'h3',
'h4',
'h5',
'h6',
'head',
'hr',
'iframe',
'image',
'img',
'input',
'isindex',
'li',
'link',
'listing',
'menu',
'meta',
'noembed',
'noframes',
'noscript',
'ol',
'optgroup',
'option',
'p',
'param',
'plaintext',
'pre',
'script',
'select',
'spacer',
'style',
'tbody',
'textarea',
'tfoot',
'thead',
'title',
'tr',
'ul',
'wbr'
];
HTML5.SPACE_CHARACTERS_IN = "\t\n\x0B\x0C\x20\u0012\r";
HTML5.SPACE_CHARACTERS = "[\t\n\x0B\x0C\x20\r]";
HTML5.SPACE_CHARACTERS_R = /^[\t\n\x0B\x0C \r]/;
HTML5.TABLE_INSERT_MODE_ELEMENTS = [
'table',
'tbody',
'tfoot',
'thead',
'tr'
];
HTML5.ASCII_LOWERCASE = 'abcdefghijklmnopqrstuvwxyz';
HTML5.ASCII_UPPERCASE = HTML5.ASCII_LOWERCASE.toUpperCase();
HTML5.ASCII_LETTERS = "[a-zA-Z]";
HTML5.ASCII_LETTERS_R = /^[a-zA-Z]/;
HTML5.DIGITS = '0123456789';
HTML5.DIGITS_R = new RegExp('^[0123456789]');
HTML5.HEX_DIGITS = HTML5.DIGITS + 'abcdefABCDEF';
HTML5.HEX_DIGITS_R = new RegExp('^[' + HTML5.DIGITS + 'abcdefABCDEF' +']' );
// Heading elements need to be ordered
HTML5.HEADING_ELEMENTS = [
'h1',
'h2',
'h3',
'h4',
'h5',
'h6'
];
HTML5.VOID_ELEMENTS = [
'base',
'link',
'meta',
'hr',
'br',
'img',
'embed',
'param',
'area',
'col',
'input'
];
HTML5.CDATA_ELEMENTS = [
'title',
'textarea'
];
HTML5.RCDATA_ELEMENTS = [
'style',
'script',
'xmp',
'iframe',
'noembed',
'noframes',
'noscript'
];
HTML5.BOOLEAN_ATTRIBUTES = {
'_global': ['irrelevant'],
// Fixme?
'style': ['scoped'],
'img': ['ismap'],
'audio': ['autoplay', 'controls'],
'video': ['autoplay', 'controls'],
'script': ['defer', 'async'],
'details': ['open'],
'datagrid': ['multiple', 'disabled'],
'command': ['hidden', 'disabled', 'checked', 'default'],
'menu': ['autosubmit'],
'fieldset': ['disabled', 'readonly'],
'option': ['disabled', 'readonly', 'selected'],
'optgroup': ['disabled', 'readonly'],
'button': ['disabled', 'autofocus'],
'input': ['disabled', 'readonly', 'required', 'autofocus', 'checked', 'ismap'],
'select': ['disabled', 'readonly', 'autofocus', 'multiple'],
'output': ['disabled', 'readonly']
};
HTML5.ENCODINGS = [
'ansi_x3.4-1968',
'iso-ir-6',
'ansi_x3.4-1986',
'iso_646.irv:1991',
'ascii',
'iso646-us',
'us-ascii',
'us',
'ibm367',
'cp367',
'csascii',
'ks_c_5601-1987',
'korean',
'iso-2022-kr',
'csiso2022kr',
'euc-kr',
'iso-2022-jp',
'csiso2022jp',
'iso-2022-jp-2',
'',
'iso-ir-58',
'chinese',
'csiso58gb231280',
'iso_8859-1:1987',
'iso-ir-100',
'iso_8859-1',
'iso-8859-1',
'latin1',
'l1',
'ibm819',
'cp819',
'csisolatin1',
'iso_8859-2:1987',
'iso-ir-101',
'iso_8859-2',
'iso-8859-2',
'latin2',
'l2',
'csisolatin2',
'iso_8859-3:1988',
'iso-ir-109',
'iso_8859-3',
'iso-8859-3',
'latin3',
'l3',
'csisolatin3',
'iso_8859-4:1988',
'iso-ir-110',
'iso_8859-4',
'iso-8859-4',
'latin4',
'l4',
'csisolatin4',
'iso_8859-6:1987',
'iso-ir-127',
'iso_8859-6',
'iso-8859-6',
'ecma-114',
'asmo-708',
'arabic',
'csisolatinarabic',
'iso_8859-7:1987',
'iso-ir-126',
'iso_8859-7',
'iso-8859-7',
'elot_928',
'ecma-118',
'greek',
'greek8',
'csisolatingreek',
'iso_8859-8:1988',
'iso-ir-138',
'iso_8859-8',
'iso-8859-8',
'hebrew',
'csisolatinhebrew',
'iso_8859-5:1988',
'iso-ir-144',
'iso_8859-5',
'iso-8859-5',
'cyrillic',
'csisolatincyrillic',
'iso_8859-9:1989',
'iso-ir-148',
'iso_8859-9',
'iso-8859-9',
'latin5',
'l5',
'csisolatin5',
'iso-8859-10',
'iso-ir-157',
'l6',
'iso_8859-10:1992',
'csisolatin6',
'latin6',
'hp-roman8',
'roman8',
'r8',
'ibm037',
'cp037',
'csibm037',
'ibm424',
'cp424',
'csibm424',
'ibm437',
'cp437',
'437',
'cspc8codepage437',
'ibm500',
'cp500',
'csibm500',
'ibm775',
'cp775',
'cspc775baltic',
'ibm850',
'cp850',
'850',
'cspc850multilingual',
'ibm852',
'cp852',
'852',
'cspcp852',
'ibm855',
'cp855',
'855',
'csibm855',
'ibm857',
'cp857',
'857',
'csibm857',
'ibm860',
'cp860',
'860',
'csibm860',
'ibm861',
'cp861',
'861',
'cp-is',
'csibm861',
'ibm862',
'cp862',
'862',
'cspc862latinhebrew',
'ibm863',
'cp863',
'863',
'csibm863',
'ibm864',
'cp864',
'csibm864',
'ibm865',
'cp865',
'865',
'csibm865',
'ibm866',
'cp866',
'866',
'csibm866',
'ibm869',
'cp869',
'869',
'cp-gr',
'csibm869',
'ibm1026',
'cp1026',
'csibm1026',
'koi8-r',
'cskoi8r',
'koi8-u',
'big5-hkscs',
'ptcp154',
'csptcp154',
'pt154',
'cp154',
'utf-7',
'utf-16be',
'utf-16le',
'utf-16',
'utf-8',
'iso-8859-13',
'iso-8859-14',
'iso-ir-199',
'iso_8859-14:1998',
'iso_8859-14',
'latin8',
'iso-celtic',
'l8',
'iso-8859-15',
'iso_8859-15',
'iso-8859-16',
'iso-ir-226',
'iso_8859-16:2001',
'iso_8859-16',
'latin10',
'l10',
'gbk',
'cp936',
'ms936',
'gb18030',
'shift_jis',
'ms_kanji',
'csshiftjis',
'euc-jp',
'gb2312',
'big5',
'csbig5',
'windows-1250',
'windows-1251',
'windows-1252',
'windows-1253',
'windows-1254',
'windows-1255',
'windows-1256',
'windows-1257',
'windows-1258',
'tis-620',
'hz-gb-2312'
];
HTML5.E = {
"null-character":
"Null character in input stream, replaced with U+FFFD.",
"incorrectly-placed-solidus":
"Solidus (/) incorrectly placed in tag.",
"incorrect-cr-newline-entity":
"Incorrect CR newline entity, replaced with LF.",
"illegal-windows-1252-entity":
"Entity used with illegal number (windows-1252 reference).",
"cant-convert-numeric-entity":
"Numeric entity couldn't be converted to character " +
"(codepoint U+%(charAsInt)08x).",
"illegal-codepoint-for-numeric-entity":
"Numeric entity represents an illegal codepoint=> " +
"U+%(charAsInt)08x.",
"numeric-entity-without-semicolon":
"Numeric entity didn't end with ';'.",
"expected-numeric-entity-but-got-eof":
"Numeric entity expected. Got end of file instead.",
"expected-numeric-entity":
"Numeric entity expected but none found.",
"named-entity-without-semicolon":
"Named entity didn't end with ';'.",
"expected-named-entity":
"Named entity expected. Got none.",
"attributes-in-end-tag":
"End tag contains unexpected attributes.",
"expected-tag-name-but-got-right-bracket":
"Expected tag name. Got '>' instead.",
"expected-tag-name-but-got-question-mark":
"Expected tag name. Got '?' instead. (HTML doesn't " +
"support processing instructions.)",
"expected-tag-name":
"Expected tag name. Got something else instead",
"expected-closing-tag-but-got-right-bracket":
"Expected closing tag. Got '>' instead. Ignoring '</>'.",
"expected-closing-tag-but-got-eof":
"Expected closing tag. Unexpected end of file.",
"expected-closing-tag-but-got-char":
"Expected closing tag. Unexpected character '%(data)' found.",
"eof-in-tag-name":
"Unexpected end of file in the tag name.",
"expected-attribute-name-but-got-eof":
"Unexpected end of file. Expected attribute name instead.",
"eof-in-attribute-name":
"Unexpected end of file in attribute name.",
"duplicate-attribute":
"Dropped duplicate attribute on tag.",
"expected-end-of-tag-name-but-got-eof":
"Unexpected end of file. Expected = or end of tag.",
"expected-attribute-value-but-got-eof":
"Unexpected end of file. Expected attribute value.",
"eof-in-attribute-value-double-quote":
"Unexpected end of file in attribute value (\").",
"eof-in-attribute-value-single-quote":
"Unexpected end of file in attribute value (').",
"eof-in-attribute-value-no-quotes":
"Unexpected end of file in attribute value.",
"expected-dashes-or-doctype":
"Expected '--' or 'DOCTYPE'. Not found.",
"incorrect-comment":
"Incorrect comment.",
"eof-in-comment":
"Unexpected end of file in comment.",
"eof-in-comment-end-dash":
"Unexpected end of file in comment (-)",
"unexpected-dash-after-double-dash-in-comment":
"Unexpected '-' after '--' found in comment.",
"eof-in-comment-double-dash":
"Unexpected end of file in comment (--).",
"unexpected-char-in-comment":
"Unexpected character in comment found.",
"need-space-after-doctype":
"No space after literal string 'DOCTYPE'.",
"expected-doctype-name-but-got-right-bracket":
"Unexpected > character. Expected DOCTYPE name.",
"expected-doctype-name-but-got-eof":
"Unexpected end of file. Expected DOCTYPE name.",
"eof-in-doctype-name":
"Unexpected end of file in DOCTYPE name.",
"eof-in-doctype":
"Unexpected end of file in DOCTYPE.",
"expected-space-or-right-bracket-in-doctype":
"Expected space or '>'. Got '%(data)'",
"unexpected-end-of-doctype":
"Unexpected end of DOCTYPE.",
"unexpected-char-in-doctype":
"Unexpected character in DOCTYPE.",
"eof-in-bogus-doctype":
"Unexpected end of file in bogus doctype.",
"eof-in-innerhtml":
"Unexpected EOF in inner html mode.",
"unexpected-doctype":
"Unexpected DOCTYPE. Ignored.",
"non-html-root":
"html needs to be the first start tag.",
"expected-doctype-but-got-eof":
"Unexpected End of file. Expected DOCTYPE.",
"unknown-doctype":
"Erroneous DOCTYPE.",
"expected-doctype-but-got-chars":
"Unexpected non-space characters. Expected DOCTYPE.",
"expected-doctype-but-got-start-tag":
"Unexpected start tag (%(name)). Expected DOCTYPE.",
"expected-doctype-but-got-end-tag":
"Unexpected end tag (%(name)). Expected DOCTYPE.",
"end-tag-after-implied-root":
"Unexpected end tag (%(name)) after the (implied) root element.",
"expected-named-closing-tag-but-got-eof":
"Unexpected end of file. Expected end tag (%(name)).",
"two-heads-are-not-better-than-one":
"Unexpected start tag head in existing head. Ignored.",
"unexpected-end-tag":
"Unexpected end tag (%(name)). Ignored.",
"unexpected-start-tag-out-of-my-head":
"Unexpected start tag (%(name)) that can be in head. Moved.",
"unexpected-start-tag":
"Unexpected start tag (%(name)).",
"missing-end-tag":
"Missing end tag (%(name)).",
"missing-end-tags":
"Missing end tags (%(name)).",
"unexpected-start-tag-implies-end-tag":
"Unexpected start tag (%(startName)) " +
"implies end tag (%(endName)).",
"unexpected-start-tag-treated-as":
"Unexpected start tag (%(originalName)). Treated as %(newName).",
"deprecated-tag":
"Unexpected start tag %(name). Don't use it!",
"unexpected-start-tag-ignored":
"Unexpected start tag %(name). Ignored.",
"expected-one-end-tag-but-got-another":
"Unexpected end tag (%(gotName). " +
"Missing end tag (%(expectedName)).",
"end-tag-too-early":
"End tag (%(name)) seen too early. Expected other end tag.",
"end-tag-too-early-named":
"Unexpected end tag (%(gotName)). Expected end tag (%(expectedName).",
"end-tag-too-early-ignored":
"End tag (%(name)) seen too early. Ignored.",
"adoption-agency-1.1":
"End tag (%(name) violates step 1, " +
"paragraph 1 of the adoption agency algorithm.",
"adoption-agency-1.2":
"End tag (%(name) violates step 1, " +
"paragraph 2 of the adoption agency algorithm.",
"adoption-agency-1.3":
"End tag (%(name) violates step 1, " +
"paragraph 3 of the adoption agency algorithm.",
"unexpected-end-tag-treated-as":
"Unexpected end tag (%(originalName)). Treated as %(newName).",
"no-end-tag":
"This element (%(name)) has no end tag.",
"unexpected-implied-end-tag-in-table":
"Unexpected implied end tag (%(name)) in the table phase.",
"unexpected-implied-end-tag-in-table-body":
"Unexpected implied end tag (%(name)) in the table body phase.",
"unexpected-char-implies-table-voodoo":
"Unexpected non-space characters in " +
"table context caused voodoo mode.",
"unpexted-hidden-input-in-table":
"Unexpected input with type hidden in table context.",
"unexpected-start-tag-implies-table-voodoo":
"Unexpected start tag (%(name)) in " +
"table context caused voodoo mode.",
"unexpected-end-tag-implies-table-voodoo":
"Unexpected end tag (%(name)) in " +
"table context caused voodoo mode.",
"unexpected-cell-in-table-body":
"Unexpected table cell start tag (%(name)) " +
"in the table body phase.",
"unexpected-cell-end-tag":
"Got table cell end tag (%(name)) " +
"while required end tags are missing.",
"unexpected-end-tag-in-table-body":
"Unexpected end tag (%(name)) in the table body phase. Ignored.",
"unexpected-implied-end-tag-in-table-row":
"Unexpected implied end tag (%(name)) in the table row phase.",
"unexpected-end-tag-in-table-row":
"Unexpected end tag (%(name)) in the table row phase. Ignored.",
"unexpected-select-in-select":
"Unexpected select start tag in the select phase " +
"treated as select end tag.",
"unexpected-input-in-select":
"Unexpected input start tag in the select phase.",
"unexpected-start-tag-in-select":
"Unexpected start tag token (%(name)) in the select phase. " +
"Ignored.",
"unexpected-end-tag-in-select":
"Unexpected end tag (%(name)) in the select phase. Ignored.",
"unexpected-table-element-start-tag-in-select-in-table":
"Unexpected table element start tag (%(name))s in the select in table phase.",
"unexpected-table-element-end-tag-in-select-in-table":
"Unexpected table element end tag (%(name))s in the select in table phase.",
"unexpected-char-after-body":
"Unexpected non-space characters in the after body phase.",
"unexpected-start-tag-after-body":
"Unexpected start tag token (%(name))" +
"in the after body phase.",
"unexpected-end-tag-after-body":
"Unexpected end tag token (%(name))" +
" in the after body phase.",
"unexpected-char-in-frameset":
"Unepxected characters in the frameset phase. Characters ignored.",
"unexpected-start-tag-in-frameset":
"Unexpected start tag token (%(name))" +
" in the frameset phase. Ignored.",
"unexpected-frameset-in-frameset-innerhtml":
"Unexpected end tag token (frameset " +
"in the frameset phase (innerHTML).",
"unexpected-end-tag-in-frameset":
"Unexpected end tag token (%(name))" +
" in the frameset phase. Ignored.",
"unexpected-char-after-frameset":
"Unexpected non-space characters in the " +
"after frameset phase. Ignored.",
"unexpected-start-tag-after-frameset":
"Unexpected start tag (%(name))" +
" in the after frameset phase. Ignored.",
"unexpected-end-tag-after-frameset":
"Unexpected end tag (%(name))" +
" in the after frameset phase. Ignored.",
"expected-eof-but-got-char":
"Unexpected non-space characters. Expected end of file.",
"expected-eof-but-got-start-tag":
"Unexpected start tag (%(name))" +
". Expected end of file.",
"expected-eof-but-got-end-tag":
"Unexpected end tag (%(name))" +
". Expected end of file.",
"unexpected-end-table-in-caption":
"Unexpected end table tag in caption. Generates implied end caption.",
"end-html-in-innerhtml":
"Unexpected html end tag in inner html mode.",
"expected-self-closing-tag":
"Expected a > after the /.",
"self-closing-end-tag":
"Self closing end tag.",
"eof-in-table":
"Unexpected end of file. Expected table content.",
"html-in-foreign-content":
"HTML start tag \"%(name)\" in a foreign namespace context.",
"unexpected-start-tag-in-table":
"Unexpected %(name). Expected table content."
};
HTML5.Models = {PCDATA: 'PCDATA', RCDATA: 'RCDATA', CDATA: 'CDATA', SCRIPT_CDATA: 'SCRIPT_CDATA'};
HTML5.TAGMODES = {
select: 'inSelect',
td: 'inCell',
th: 'inCell',
tr: 'inRow',
tbody: 'inTableBody',
thead: 'inTableBody',
tfoot: 'inTableBody',
caption: 'inCaption',
colgroup: 'inColumnGroup',
table: 'inTable',
head: 'inBody',
body: 'inBody',
frameset: 'inFrameset'
};
HTML5.SVGAttributeMap = {
attributename: 'attributeName',
attributetype: 'attributeType',
basefrequency: 'baseFrequency',
baseprofile: 'baseProfile',
calcmode: 'calcMode',
clippathunits: 'clipPathUnits',
contentscripttype: 'contentScriptType',
contentstyletype: 'contentStyleType',
diffuseconstant: 'diffuseConstant',
edgemode: 'edgeMode',
externalresourcesrequired: 'externalResourcesRequired',
filterres: 'filterRes',
filterunits: 'filterUnits',
glyphref: 'glyphRef',
gradienttransform: 'gradientTransform',
gradientunits: 'gradientUnits',
kernelmatrix: 'kernelMatrix',
kernelunitlength: 'kernelUnitLength',
keypoints: 'keyPoints',
keysplines: 'keySplines',
keytimes: 'keyTimes',
lengthadjust: 'lengthAdjust',
limitingconeangle: 'limitingConeAngle',
markerheight: 'markerHeight',
markerunits: 'markerUnits',
markerwidth: 'markerWidth',
maskcontentunits: 'maskContentUnits',
maskunits: 'maskUnits',
numoctaves: 'numOctaves',
pathlength: 'pathLength',
patterncontentunits: 'patternContentUnits',
patterntransform: 'patternTransform',
patternunits: 'patternUnits',
pointsatx: 'pointsAtX',
pointsaty: 'pointsAtY',
pointsatz: 'pointsAtZ',
preservealpha: 'preserveAlpha',
preserveaspectratio: 'preserveAspectRatio',
primitiveunits: 'primitiveUnits',
refx: 'refX',
refy: 'refY',
repeatcount: 'repeatCount',
repeatdur: 'repeatDur',
requiredextensions: 'requiredExtensions',
requiredfeatures: 'requiredFeatures',
specularconstant: 'specularConstant',
specularexponent: 'specularExponent',
spreadmethod: 'spreadMethod',
startoffset: 'startOffset',
stddeviation: 'stdDeviation',
stitchtiles: 'stitchTiles',
surfacescale: 'surfaceScale',
systemlanguage: 'systemLanguage',
tablevalues: 'tableValues',
targetx: 'targetX',
targety: 'targetY',
textlength: 'textLength',
viewbox: 'viewBox',
viewtarget: 'viewTarget',
xchannelselector: 'xChannelSelector',
ychannelselector: 'yChannelSelector',
zoomandpan: 'zoomAndPan'
};

2235
packages/html5-tokenizer/entities.js
View File

File diff suppressed because it is too large Load Diff

23
packages/html5-tokenizer/events.js
View File

@@ -1,23 +0,0 @@
// dgreenspan's minimal implementation of events.EventEmitter
toyevents = {
EventEmitter: function EventEmitter() {
this._listeners = {};
}
};
toyevents.EventEmitter.prototype.addListener = function (type, f) {
if (! f)
return;
this._listeners[type] = this._listeners[type] || [];
this._listeners[type].push(f);
};
toyevents.EventEmitter.prototype.emit = function (type, data) {
var funcs = this._listeners[type];
if (! funcs)
return;
for (var i = 0, f; f = funcs[i]; i++)
f(data);
};

28
packages/html5-tokenizer/html5_tokenizer.js
View File

@@ -1,28 +0,0 @@
HTML5Tokenizer = {
tokenize: function (inputString) {
var tokens = [];
var tokenizer = new HTML5.Tokenizer(inputString);
tokenizer.addListener('token', function (tok) {
tokens.push(tok);
});
tokenizer.tokenize();
return tokens;
}
// Incremental tokenization turns out not to be useful
// for inspecting intermediate tokenizer state, just
// for async streaming.
//
// tokenizeIncremental: function (tokenFunc) {
// var emitter = new toyevents.EventEmitter();
// var tokenizer = new HTML5.Tokenizer(emitter);
// tokenizer.addListener('token', tokenFunc);
// return {
// add: function (str) {
// emitter.emit('data', str);
// },
// finish: function () {
// emitter.emit('end');
// }
// };
// }
};

16
packages/html5-tokenizer/package.js
View File

@@ -1,16 +0,0 @@
Package.describe({
summary: "HTML5 tokenizer"
});
Package.on_use(function (api) {
api.export('HTML5Tokenizer');
api.add_files(['entities.js', 'constants.js', 'buffer.js',
'events.js', 'tokenizer.js',
'html5_tokenizer.js']);
});
Package.on_test(function (api) {
api.use('html5-tokenizer');
api.use('tinytest');
api.add_files('tokenizer_tests.js', ['server']);
});

965
packages/html5-tokenizer/tokenizer.js
View File

@@ -1,965 +0,0 @@
var Models = HTML5.Models;
var events = toyevents;
function keys(h) {
var r = [];
for(var k in h) {
r.push(k);
}
return r;
}
var last = function (array) {
return array[array.length - 1];
};
HTML5.debug = function () {};
var ENTITY_KEYS = keys(HTML5.ENTITIES);
var t = HTML5.Tokenizer = function HTML5Tokenizer(input, document, tree) {
var state;
var buffer = new HTML5.Buffer();
var escapeFlag = false;
var lastFourChars = '';
var current_token = null;
var script_buffer = null;
var content_model = Models.PCDATA;
var source;
tree = tree || {open_elements: []};
function data_state(buffer) {
var c = buffer.char();
if (c !== HTML5.EOF && (content_model == Models.CDATA || content_model == Models.RCDATA || content_model == Models.SCRIPT_CDATA)) {
lastFourChars += c;
if (lastFourChars.length >= 4) {
lastFourChars = lastFourChars.substr(-4);
}
}
if (content_model == Models.SCRIPT_CDATA) {
if (script_buffer === null) {
script_buffer = '';
}
}
if (c === HTML5.EOF) {
emitToken(HTML5.EOF_TOK);
buffer.commit();
return false;
} else if (c === '\0' && (content_model == Models.SCRIPT_CDATA || content_model == Models.PLAINTEXT || content_model == Models.RAWTEXT || content_model == Models.RCDATA)) {
emitToken({type: 'Characters', data: "\ufffd"});
buffer.commit();
} else if (c == '&' && (content_model == Models.PCDATA || content_model == Models.RCDATA) && !escapeFlag) {
newState(entity_data_state);
} else if (c == '-' && (content_model == Models.CDATA || content_model == Models.RCDATA || content_model == Models.SCRIPT_CDATA) && !escapeFlag && lastFourChars == '<!--') {
escapeFlag = true;
emitToken({type: 'Characters', data: c});
buffer.commit();
} else if (c == '<' && !escapeFlag && (content_model == Models.PCDATA || content_model == Models.RCDATA || content_model == Models.CDATA || content_model == Models.SCRIPT_CDATA)) {
newState(tag_open_state);
} else if (c == '>' && escapeFlag && (content_model == Models.CDATA || content_model == Models.RCDATA || content_model == Models.SCRIPT_CDATA) && lastFourChars.match(/-->$/)) {
escapeFlag = false;
emitToken({type: 'Characters', data: c});
buffer.commit();
} else if (HTML5.SPACE_CHARACTERS_R.test(c)) {
emitToken({type: 'SpaceCharacters', data: c + buffer.matchWhile(HTML5.SPACE_CHARACTERS)});
buffer.commit();
} else {
var o = buffer.matchUntil("[&<>-]");
if (o !== HTML5.EOF) {
c = c + o;
}
emitToken({type: 'Characters', data: c});
lastFourChars += c;
lastFourChars = lastFourChars.slice(-4);
buffer.commit();
}
return true;
}
var entity_data_state = function entity_data_state(buffer) {
var entity = consume_entity(buffer);
if (entity) {
emitToken({type: 'Characters', data: entity});
} else {
emitToken({type: 'Characters', data: '&'});
}
newState(data_state);
return true;
};
this.tokenize = function() {
if (this.pump) this.pump();
};
var emitToken = function emitToken(tok) {
tok = normalize_token(tok);
if (content_model == Models.SCRIPT_CDATA && (tok.type == 'Characters' || tok.type == 'SpaceCharacters') && !buffer.eof) {
HTML5.debug('tokenizer.addScriptData', tok);
script_buffer += tok.data;
} else {
HTML5.debug('tokenizer.token', tok);
this.emit('token', tok);
}
}.bind(this);
function consume_entity(buffer, from_attr) {
var char = null;
var chars = buffer.char();
var c;
if (chars === HTML5.EOF) return false;
if (chars.match(HTML5.SPACE_CHARACTERS) || chars == '<' || chars == '&') {
buffer.unget(chars);
} else if (chars[0] == '#') { // Maybe a numeric entity
c = buffer.shift(2);
if (c === HTML5.EOF) {
buffer.unget(chars);
return false;
}
chars += c;
if (chars[1] && chars[1].toLowerCase() == 'x' && HTML5.HEX_DIGITS_R.test(chars[2])) {
// Hex entity
buffer.unget(chars[2]);
char = consume_numeric_entity(buffer, true);
} else if (chars[1] && HTML5.DIGITS_R.test(chars[1])) {
// Decimal entity
buffer.unget(chars.slice(1));
char = consume_numeric_entity(buffer, false);
} else {
// Not numeric
buffer.unget(chars);
parse_error("expected-numeric-entity");
}
} else {
var filteredEntityList = ENTITY_KEYS.filter(function(e) {
return e[0] == chars[0];
});
var entityName = null;
var matches = function(e) {
return e.indexOf(chars) === 0;
};
while(true) {
if (filteredEntityList.some(matches)) {
filteredEntityList = filteredEntityList.filter(matches);
c = buffer.char();
if (c !== HTML5.EOF) {
chars += c;
} else {
break;
}
} else {
break;
}
if (HTML5.ENTITIES[chars]) {
entityName = chars;
if (entityName[entityName.length - 1] == ';') break;
}
}
if (entityName) {
char = HTML5.ENTITIES[entityName];
if (entityName[entityName.length - 1] != ';' && this.from_attribute && (HTML5.ASCII_LETTERS_R.test(chars.substr(entityName.length, 1) || HTML5.DIGITS.test(chars.substr(entityName.length, 1))))) {
buffer.unget(chars);
char = '&';
} else {
buffer.unget(chars.slice(entityName.length));
}
} else {
parse_error("expected-named-entity");
buffer.unget(chars);
}
}
return char;
}
function replaceEntityNumbers(c) {
switch(c) {
case 0x00: return 0xFFFD; // REPLACEMENT CHARACTER
case 0x13: return 0x0010; // Carriage return
case 0x80: return 0x20AC; // EURO SIGN
case 0x81: return 0x0081; // <control>
case 0x82: return 0x201A; // SINGLE LOW-9 QUOTATION MARK
case 0x83: return 0x0192; // LATIN SMALL LETTER F WITH HOOK
case 0x84: return 0x201E; // DOUBLE LOW-9 QUOTATION MARK
case 0x85: return 0x2026; // HORIZONTAL ELLIPSIS
case 0x86: return 0x2020; // DAGGER
case 0x87: return 0x2021; // DOUBLE DAGGER
case 0x88: return 0x02C6; // MODIFIER LETTER CIRCUMFLEX ACCENT
case 0x89: return 0x2030; // PER MILLE SIGN
case 0x8A: return 0x0160; // LATIN CAPITAL LETTER S WITH CARON
case 0x8B: return 0x2039; // SINGLE LEFT-POINTING ANGLE QUOTATION MARK
case 0x8C: return 0x0152; // LATIN CAPITAL LIGATURE OE
case 0x8D: return 0x008D; // <control>
case 0x8E: return 0x017D; // LATIN CAPITAL LETTER Z WITH CARON
case 0x8F: return 0x008F; // <control>
case 0x90: return 0x0090; // <control>
case 0x91: return 0x2018; // LEFT SINGLE QUOTATION MARK
case 0x92: return 0x2019; // RIGHT SINGLE QUOTATION MARK
case 0x93: return 0x201C; // LEFT DOUBLE QUOTATION MARK
case 0x94: return 0x201D; // RIGHT DOUBLE QUOTATION MARK
case 0x95: return 0x2022; // BULLET
case 0x96: return 0x2013; // EN DASH
case 0x97: return 0x2014; // EM DASH
case 0x98: return 0x02DC; // SMALL TILDE
case 0x99: return 0x2122; // TRADE MARK SIGN
case 0x9A: return 0x0161; // LATIN SMALL LETTER S WITH CARON
case 0x9B: return 0x203A; // SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
case 0x9C: return 0x0153; // LATIN SMALL LIGATURE OE
case 0x9D: return 0x009D; // <control>
case 0x9E: return 0x017E; // LATIN SMALL LETTER Z WITH CARON
case 0x9F: return 0x0178; // LATIN CAPITAL LETTER Y WITH DIAERESIS
default:
if ((c >= 0xD800 && c <= 0xDFFF) || c >= 0x10FFFF) { /// @todo. The spec says > 0x10FFFF, not >=. Section 8.2.4.69.
return 0xFFFD;
} else if ((c >= 0x0001 && c <= 0x0008) || (c >= 0x000E && c <= 0x001F) ||
(c >= 0x007F && c <= 0x009F) || (c >= 0xFDD0 && c <= 0xFDEF) ||
c == 0x000B || c == 0xFFFE || c == 0x1FFFE || c == 0x2FFFFE ||
c == 0x2FFFF || c == 0x3FFFE || c == 0x3FFFF || c == 0x4FFFE ||
c == 0x4FFFF || c == 0x5FFFE || c == 0x5FFFF || c == 0x6FFFE ||
c == 0x6FFFF || c == 0x7FFFE || c == 0x7FFFF || c == 0x8FFFE ||
c == 0x8FFFF || c == 0x9FFFE || c == 0x9FFFF || c == 0xAFFFE ||
c == 0xAFFFF || c == 0xBFFFE || c == 0xBFFFF || c == 0xCFFFE ||
c == 0xCFFFF || c == 0xDFFFE || c == 0xDFFFF || c == 0xEFFFE ||
c == 0xEFFFF || c == 0xFFFFE || c == 0xFFFFF || c == 0x10FFFE ||
c == 0x10FFFF) {
return c;
}
}
}
function consume_numeric_entity(buffer, hex) {
var allowed, radix;
if (hex) {
allowed = HTML5.HEX_DIGITS_R;
radix = 16;
} else {
allowed = HTML5.DIGITS_R;
radix = 10;
}
var chars = '';
var c = buffer.char();
while(c !== HTML5.EOF && allowed.test(c)) {
chars = chars + c;
c = buffer.char();
}
var charAsInt = parseInt(chars, radix);
var replacement = replaceEntityNumbers(charAsInt);
if (replacement) {
parse_error("invalid-numeric-entity-replaced", {old: charAsInt, 'new': replacement});
charAsInt = replacement;
}
var char = String.fromCharCode(charAsInt);
/*if (charAsInt <= 0x10FFFF && !(charAsInt >= 0xD800 && charAsInt <= 0xDFFF)) {
} else {
char = String.fromCharCode(0xFFFD);
parse_error("cant-convert-numeric-entity");
} */
if (c !== ';') {
parse_error("numeric-entity-without-semicolon");
buffer.unget(c);
}
return char;
}
function process_entity_in_attribute(buffer) {
var entity = consume_entity(buffer);
if (entity) {
last(current_token.data).nodeValue += entity;
} else {
last(current_token.data).nodeValue += '&';
}
}
function process_solidus_in_tag(buffer) {
var data = buffer.peek(1);
if (current_token.type == 'StartTag' && data == '>') {
current_token.type = 'EmptyTag';
return true;
} else {
parse_error("incorrectly-placed-solidus");
return false;
}
}
function tag_open_state(buffer) {
var data = buffer.char();
if (content_model == Models.PCDATA) {
if (data === HTML5.EOF) {
parse_error("bare-less-than-sign-at-eof");
emitToken({type: 'Characters', data: '<'});
newState(data_state);
} else if (data !== HTML5.EOF && HTML5.ASCII_LETTERS_R.test(data)) {
current_token = {type: 'StartTag', name: data, data: []};
newState(tag_name_state);
} else if (data == '!') {
newState(markup_declaration_open_state);
} else if (data == '/') {
newState(close_tag_open_state);
} else if (data == '>') {
// XXX In theory it could be something besides a tag name. But
// do we really care?
parse_error("expected-tag-name-but-got-right-bracket");
emitToken({type: 'Characters', data: "<>"});
newState(data_state);
} else if (data == '?') {
// XXX In theory it could be something besides a tag name. But
// do we really care?
parse_error("expected-tag-name-but-got-question-mark");
buffer.unget(data);
newState(bogus_comment_state);
} else {
// XXX
parse_error("expected-tag-name");
emitToken({type: 'Characters', data: "<"});
buffer.unget(data);
newState(data_state);
}
} else {
// We know the content model flag is set to either RCDATA or CDATA or SCRIPT_CDATA
// now because this state can never be entered with the PLAINTEXT
// flag.
if (data === '/') {
newState(close_tag_open_state);
} else {
emitToken({type: 'Characters', data: "<"});
buffer.unget(data);
newState(data_state);
}
}
return true;
}
function close_tag_open_state(buffer) {
if (content_model == Models.RCDATA || content_model == Models.CDATA || content_model == Models.SCRIPT_CDATA) {
var chars = '';
if (current_token) {
for(var i = 0; i <= current_token.name.length; i++) {
var c = buffer.char();
if (c === HTML5.EOF) break;
chars += c;
}
buffer.unget(chars);
}
if (current_token &&
current_token.name.toLowerCase() == chars.slice(0, current_token.name.length).toLowerCase() &&
(chars.length > current_token.name.length ? new RegExp('[' + HTML5.SPACE_CHARACTERS_IN + '></\0]').test(chars.substr(-1)) : true)
) {
content_model = Models.PCDATA;
} else {
emitToken({type: 'Characters', data: '</'});
newState(data_state);
return true;
}
}
var data = buffer.char();
if (data === HTML5.EOF) {
parse_error("expected-closing-tag-but-got-eof");
emitToken({type: 'Characters', data: '</'});
buffer.unget(data);
newState(data_state);
} else if (HTML5.ASCII_LETTERS_R.test(data)) {
current_token = {type: 'EndTag', name: data, data: []};
newState(tag_name_state);
} else if (data == '>') {
parse_error("expected-closing-tag-but-got-right-bracket");
newState(data_state);
} else {
parse_error("expected-closing-tag-but-got-char", {data: data}); // param 1 is datavars:
buffer.unget(data);
newState(bogus_comment_state);
}
return true;
}
function tag_name_state(buffer) {
var data = buffer.char();
if (data === HTML5.EOF) {
parse_error('eof-in-tag-name');
emit_current_token();
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
newState(before_attribute_name_state);
} else if (HTML5.ASCII_LETTERS_R.test(data)) {
var c = buffer.matchWhile(HTML5.ASCII_LETTERS);
if (c !== HTML5.EOF) {
current_token.name += data + c;
} else {
current_token.name += data;
buffer.unget(c);
newState(data_state);
}
} else if (data == '>') {
emit_current_token();
} else if (data == '/') {
process_solidus_in_tag(buffer);
newState(self_closing_tag_state);
} else {
current_token.name += data;
}
buffer.commit();
return true;
}
function before_attribute_name_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("expected-attribute-name-but-got-eof");
emit_current_token();
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
buffer.matchWhile(HTML5.SPACE_CHARACTERS);
} else if (HTML5.ASCII_LETTERS_R.test(data)) {
current_token.data.push({nodeName: data, nodeValue: ""});
newState(attribute_name_state);
} else if (data == '>') {
emit_current_token();
} else if (data == '/') {
newState(self_closing_tag_state);
} else if (data == "'" || data == '"' || data == '=') {
parse_error("invalid-character-in-attribute-name");
current_token.data.push({nodeName: data, nodeValue: ""});
newState(attribute_name_state);
} else {
current_token.data.push({nodeName: data, nodeValue: ""});
newState(attribute_name_state);
}
return true;
}
function attribute_name_state(buffer) {
var data = buffer.shift(1);
var leavingThisState = true;
var emitToken = false;
if (data === HTML5.EOF) {
parse_error("eof-in-attribute-name");
newState(data_state);
emitToken = true;
} else if (data == '=') {
newState(before_attribute_value_state);
} else if (HTML5.ASCII_LETTERS_R.test(data)) {
last(current_token.data).nodeName += data + buffer.matchWhile(HTML5.ASCII_LETTERS);
leavingThisState = false;
} else if (data == '>') {
// XXX If we emit here the attributes are converted to a dict
// without being checked and when the code below runs we error
// because data is a dict not a list
emitToken = true;
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
newState(after_attribute_name_state);
} else if (data == '/') {
if (!process_solidus_in_tag(buffer)) {
newState(before_attribute_name_state);
}
} else if (data == "'" || data == '"') {
parse_error("invalid-character-in-attribute-name");
last(current_token.data).nodeName += data;
leavingThisState = false;
} else {
last(current_token.data).nodeName += data;
leavingThisState = false;
}
if (leavingThisState) {
// Attributes are not dropped at this stage. That happens when the
// start tag token is emitted so values can still be safely appended
// to attributes, but we do want to report the parse error in time.
if (this.lowercase_attr_name) {
last(current_token.data).nodeName = last(current_token.data).nodeName.toLowerCase();
}
for (var k in current_token.data.slice(0, -1)) {
// FIXME this is a fucking mess.
if (current_token.data.slice(-1)[0] == current_token.data.slice(0, -1)[k].name) {
parse_error("duplicate-attribute");
break; // Don't emit more than one of these errors
}
}
if (emitToken) emit_current_token();
} else {
buffer.commit();
}
return true;
}
function after_attribute_name_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("expected-end-of-tag-but-got-eof");
emit_current_token();
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
buffer.matchWhile(HTML5.SPACE_CHARACTERS);
} else if (data == '=') {
newState(before_attribute_value_state);
} else if (data == '>') {
emit_current_token();
} else if (HTML5.ASCII_LETTERS_R.test(data)) {
current_token.data.push({nodeName: data, nodeValue: ""});
newState(attribute_name_state);
} else if (data == '/') {
newState(self_closing_tag_state);
} else {
current_token.data.push({nodeName: data, nodeValue: ""});
newState(attribute_name_state);
}
return true;
}
function before_attribute_value_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("expected-attribute-value-but-got-eof");
emit_current_token();
newState(attribute_value_unquoted_state);
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
buffer.matchWhile(HTML5.SPACE_CHARACTERS);
} else if (data == '"') {
newState(attribute_value_double_quoted_state);
} else if (data == '&') {
newState(attribute_value_unquoted_state);
buffer.unget(data);
} else if (data == "'") {
newState(attribute_value_single_quoted_state);
} else if (data == '>') {
emit_current_token();
} else if (data == '=') {
parse_error("equals-in-unquoted-attribute-value");
last(current_token.data).nodeValue += data;
newState(attribute_value_unquoted_state);
} else {
last(current_token.data).nodeValue += data;
newState(attribute_value_unquoted_state);
}
return true;
}
function attribute_value_double_quoted_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-attribute-value-double-quote");
newState(data_state);
} else if (data == '"') {
newState(after_attribute_value_state);
} else if (data == '&') {
process_entity_in_attribute(buffer);
} else {
var s = buffer.matchUntil('["&]');
if (s !== HTML5.EOF) data = data + s;
last(current_token.data).nodeValue += data;
}
return true;
}
function attribute_value_single_quoted_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-attribute-value-single-quote");
emit_current_token();
} else if (data == "'") {
newState(after_attribute_value_state);
} else if (data == '&') {
process_entity_in_attribute(buffer);
} else {
last(current_token.data).nodeValue += data + buffer.matchUntil("['&]");
}
return true;
}
function attribute_value_unquoted_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-attribute-value-no-quotes");
buffer.commit();
emit_current_token();
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
newState(before_attribute_name_state);
} else if (data == '&') {
process_entity_in_attribute(buffer);
} else if (data == '>') {
emit_current_token();
} else if (data == '"' || data == "'" || data == '=') {
parse_error("unexpected-character-in-unquoted-attribute-value");
last(current_token.data).nodeValue += data;
} else {
var o = buffer.matchUntil("["+ HTML5.SPACE_CHARACTERS_IN + '&<>' +"]");
if (o === HTML5.EOF) {
parse_error("eof-in-attribute-value-no-quotes");
emit_current_token();
}
// Commit here since this state is re-enterable and its outcome won't change with more data.
buffer.commit();
last(current_token.data).nodeValue += data + o;
}
return true;
}
function after_attribute_value_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error( "unexpected-EOF-after-attribute-value");
emit_current_token();
buffer.unget(data);
newState(data_state);
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
newState(before_attribute_name_state);
} else if (data == '>') {
emit_current_token();
newState(data_state);
} else if (data == '/') {
newState(self_closing_tag_state);
} else {
emitToken({type: 'ParseError', data: "unexpected-character-after-attribute-value"});
buffer.unget(data);
newState(before_attribute_name_state);
}
return true;
}
function self_closing_tag_state(buffer) {
var c = buffer.shift(1);
if (c === HTML5.EOF) {
parse_error("eof-in-tag-name");
buffer.unget(c);
newState(data_state);
} else if (c == '>') {
current_token.self_closing = true;
emit_current_token();
newState(data_state);
} else {
parse_error("expected-self-closing-tag");
buffer.unget(c);
newState(before_attribute_name_state);
}
return true;
}
function bogus_comment_state(buffer) {
var s = buffer.matchUntil('>');
if (s === HTML5.EOF) {
s = '';
}
var tok = {type: 'Comment', data: s};
buffer.char();
emitToken(tok);
newState(data_state);
return true;
}
function markup_declaration_open_state(buffer) {
var chars = buffer.shift(2);
if (chars === '--') {
current_token = {type: 'Comment', data: ''};
newState(comment_start_state);
} else {
var newchars = buffer.shift(5);
if (newchars === HTML5.EOF || chars === HTML5.EOF) {
parse_error("expected-dashes-or-doctype");
newState(bogus_comment_state);
buffer.unget(chars);
return true;
}
chars += newchars;
if (chars.toUpperCase() == 'DOCTYPE') {
current_token = {type: 'Doctype', name: '', publicId: null, systemId: null, correct: true};
newState(doctype_state);
} else if (last(tree.open_elements) && last(tree.open_elements).namespace && chars == '[CDATA[') {
newState(cdata_section_state);
} else {
parse_error("expected-dashes-or-doctype");
buffer.unget(chars);
newState(bogus_comment_state);
}
}
return true;
}
function cdata_section_state(buffer) {
var data = buffer.matchUntil(/\]\]>/);
var slice;
if (/\]\]>$/.match(data)) {
slice = 4;
} else {
slice = 0;
}
emitToken({type: 'Characters', data: data.slice(0, data.length - slice)});
newState(data_state);
}
function comment_start_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-comment");
emitToken(current_token);
newState(data_state);
} else if (data == '-') {
newState(comment_start_dash_state);
} else if (data == '>') {
parse_error("incorrect comment");
emitToken(current_token);
newState(data_state);
} else {
current_token.data += data + buffer.matchUntil('-');
newState(comment_state);
}
return true;
}
function comment_start_dash_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-comment");
emitToken(current_token);
newState(data_state);
} else if (data == '-') {
newState(comment_end_state);
} else if (data == '>') {
parse_error("incorrect-comment");
emitToken(current_token);
newState(data_state);
} else {
var s = buffer.matchUntil('-');
if (s !== HTML5.EOF) data = data + s;
current_token.data += '-' + data;
newState(comment_state);
}
return true;
}
function comment_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-comment");
emitToken(current_token);
newState(data_state);
} else if (data == '-') {
newState(comment_end_dash_state);
} else {
current_token.data += data + buffer.matchUntil('-');
}
return true;
}
function comment_end_dash_state(buffer) {
var data = buffer.char();
if (data === HTML5.EOF) {
parse_error("eof-in-comment-end-dash");
emitToken(current_token);
newState(data_state);
} else if (data == '-') {
newState(comment_end_state);
} else {
current_token.data += '-' + data + buffer.matchUntil('-');
// Consume the next character which is either a "-" or an :EOF as
// well so if there's a "-" directly after the "-" we go nicely to
// the "comment end state" without emitting a ParseError there.
buffer.char();
}
return true;
}
function comment_end_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("eof-in-comment-double-dash");
emitToken(current_token);
newState(data_state);
} else if (data == '>') {
emitToken(current_token);
newState(data_state);
} else if (data == '-') {
parse_error("unexpected-dash-after-double-dash-in-comment");
current_token.data += data;
} else {
// XXX
parse_error("unexpected-char-in-comment");
current_token.data += '--' + data;
newState(comment_state);
}
return true;
}
function doctype_state(buffer) {
var data = buffer.shift(1);
if (HTML5.SPACE_CHARACTERS_R.test(data)) {
newState(before_doctype_name_state);
} else {
parse_error("need-space-after-doctype");
buffer.unget(data);
newState(before_doctype_name_state);
}
return true;
}
function before_doctype_name_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
parse_error("expected-doctype-name-but-got-eof");
current_token.correct = false;
emit_current_token();
newState(data_state);
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
} else if (data == '>') {
parse_error("expected-doctype-name-but-got-right-bracket");
current_token.correct = false;
emit_current_token();
newState(data_state);
} else {
current_token.name = data.toLowerCase();
newState(doctype_name_state);
}
return true;
}
function doctype_name_state(buffer) {
var data = buffer.shift(1);
if (data === HTML5.EOF) {
current_token.correct = false;
buffer.unget(data);
parse_error("eof-in-doctype");
emit_current_token();
newState(data_state);
} else if (HTML5.SPACE_CHARACTERS_R.test(data)) {
newState(bogus_doctype_state);
} else if (data == '>') {
emit_current_token();
newState(data_state);
} else {
current_token.name += data.toLowerCase();
}
buffer.commit();
return true;
}
function bogus_doctype_state(buffer) {
var data = buffer.shift(1);
current_token.correct = false;
if (data === HTML5.EOF) {
throw(new Error("Unimplemented!"));
} else if (data == '>') {
emit_current_token();
newState(data_state);
}
return true;
}
function parse_error(message, context) {
emitToken({type: 'ParseError', data: message});
HTML5.debug('tokenizer.parseError', message, context);
}
function emit_current_token() {
var tok = current_token;
switch(tok.type) {
case 'StartTag':
case 'EndTag':
case 'EmptyTag':
if (tok.type == 'EndTag' && tok.self_closing) {
parse_error('self-closing-end-tag');
}
break;
}
if (current_token.name.toLowerCase() == "script" && tok.type == 'EndTag' && script_buffer) {
emitToken({ type: 'Characters', data: script_buffer });
script_buffer = null;
}
emitToken(tok);
newState(data_state);
}
function normalize_token(token) {
if (token.type == 'EmptyTag') {
if (HTML5.VOID_ELEMENTS.indexOf(token.name) == -1) {
parse_error('incorrectly-placed-solidus');
}
token.type = 'StartTag';
}
if (token.type == 'StartTag') {
token.name = token.name.toLowerCase();
if (token.data.length !== 0) {
var data = {};
// the first value for each key wins
token.data.reverse();
token.data.forEach(function(e) {
data[e.nodeName.toLowerCase()] = e.nodeValue;
});
token.data = [];
for(var k in data) {
token.data.push({nodeName: k, nodeValue: data[k]});
}
// restore original attribute order
token.data.reverse();
}
} else if (token.type == 'EndTag') {
if (token.data.length !== 0) parse_error('attributes-in-end-tag');
token.name = token.name.toLowerCase();
}
return token;
}
if (typeof input === 'undefined') throw(new Error("No input given"));
this.document = document;
this.__defineSetter__('content_model', function(model) {
HTML5.debug('tokenizer.content_model=', model);
content_model = model;
});
this.__defineGetter__('content_model', function() {
return content_model;
});
function newState(newstate) {
HTML5.debug('tokenizer.state=', newstate.name);
state = newstate;
buffer.commit();
}
newState(data_state);
if (input instanceof events.EventEmitter) {
source = input;
this.pump = null;
} else {
source = new events.EventEmitter();
this.pump = function() {
source.emit('data', input);
source.emit('end');
};
}
source.addListener('data', function(data) {
if (typeof data !== 'string') data = data.toString();
buffer.append(data);
try {
while(state(buffer));
} catch(e) {
if (e != HTML5.DRAIN) {
throw(e);
} else {
HTML5.debug('tokenizer.drain', 'Drain');
buffer.undo();
}
}
});
source.addListener('end', function() {
buffer.eof = true;
while(state(buffer));
this.emit('end');
}.bind(this));
};
t.prototype = new events.EventEmitter();

36
packages/html5-tokenizer/tokenizer_tests.js
View File

@@ -1,36 +0,0 @@
Tinytest.add("html5-tokenizer - basic", function (test) {
var run = function (input, expectedTokens) {
test.equal(HTML5Tokenizer.tokenize(input),
expectedTokens);
};
run('<p>foo',
[ { type: 'StartTag', name: 'p', data: [] },
{ type: 'Characters', data: 'foo' },
{ type: 'EOF', data: 'End of File' } ]);
run('<!DOCTYPE html>',
[ { type: 'Doctype', name: 'html', correct: true,
publicId: null, systemId: null },
{ type: 'EOF', data: 'End of File' } ]);
run('<a b c=d> </a>',
[ { type: 'StartTag', name: 'a',
data: [{nodeName: 'b', nodeValue: ''},
{nodeName: 'c', nodeValue: 'd'}] },
{ type: 'SpaceCharacters', data: ' ' },
{ type: 'EndTag', name: 'a', data: [] },
{ type: 'EOF', data: 'End of File' } ]);
run('<3',
[{ type: 'ParseError', data: 'expected-tag-name' },
{ type: 'Characters', data: '<' },
{ type: 'Characters', data: '3' },
{ type: 'EOF', data: 'End of File' } ]);
run('<!--foo-->',
[{ type: 'Comment', data: 'foo' },
{ type: 'EOF', data: 'End of File' } ]);
});

10
packages/htmljs/html.js
View File

@@ -178,10 +178,14 @@ callReactiveFunction = function (func) {
stopWithLater = function (instance) {
if (instance.materialized && instance.materialized.isWith) {
if (Deps.active)
if (Deps.active) {
instance.materialized();
else
instance.data.stop();
} else {
if (instance.data) // `UI.With`
instance.data.stop();
else if (instance.v) // `Spacebars.With`
instance.v.stop();
}
}
};

3
packages/htmljs/package.js
View File

@@ -1,5 +1,6 @@
Package.describe({
summary: "Small library for expressing HTML trees"
summary: "Small library for expressing HTML trees",
internal: true
});
Package.on_use(function (api) {

2
packages/http/httpcall_common.js
View File

@@ -1,5 +1,5 @@
makeErrorByStatus = function(statusCode, content) {
var MAX_LENGTH = 160; // if you change this, also change the appropriate test
var MAX_LENGTH = 500; // if you change this, also change the appropriate test
var truncate = function(str, length) {
return str.length > length ? str.slice(0, length) + '...' : str;

4
packages/http/httpcall_tests.js
View File

@@ -116,8 +116,8 @@ testAsyncMulti("httpcall - errors", [
// in test_responder.js we make a very long response body, to make sure
// that we truncate messages. first of all, make sure we didn't make that
// message too short, so that we can be sure we're verifying that we truncate.
test.isTrue(error.response.content.length > 180);
test.isTrue(error.message.length < 180); // make sure we truncate.
test.isTrue(error.response.content.length > 520);
test.isTrue(error.message.length < 520); // make sure we truncate.
};
HTTP.call("GET", url_prefix()+"/fail", expect(error500Callback));

6
packages/http/test_responder.js
View File

@@ -10,8 +10,10 @@ var respond = function(req, res) {
return;
} else if (req.url === "/fail") {
res.statusCode = 500;
res.end("SOME SORT OF SERVER ERROR. " +
"MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. MAKE THIS VERY LONG TO MAKE SURE WE TRUNCATE. ");
res.end("SOME SORT OF SERVER ERROR. foo" +
_.times(100, function () {
return "MAKE THIS LONG TO TEST THAT WE TRUNCATE";
}).join(' '));
return;
} else if (req.url === "/redirect") {
res.statusCode = 301;

6
packages/js-analyze/.npm/package/npm-shrinkwrap.json generated
View File

@@ -1,8 +1,5 @@
{
"dependencies": {
"esprima": {
"version": "https://github.com/ariya/esprima/tarball/2a41dbf0ddadade0b09a9a7cc9a0c8df9c434018"
},
"escope": {
"version": "1.0.0",
"dependencies": {
@@ -10,6 +7,9 @@
"version": "1.3.1"
}
}
},
"esprima": {
"version": "https://github.com/ariya/esprima/tarball/2a41dbf0ddadade0b09a9a7cc9a0c8df9c434018"
}
}
}

1
packages/less/plugin/compile-less.js
View File

@@ -15,6 +15,7 @@ Plugin.registerSourceHandler("less", function (compileStep) {
var source = compileStep.read().toString('utf8');
var options = {
filename: compileStep.inputPath,
// Use fs.readFileSync to process @imports. This is the bundler, so
// that's not going to cause concurrency issues, and it means that (a)
// we don't have to use Futures and (b) errors thrown by bugs in less

16
packages/livedata/.npm/package/npm-shrinkwrap.json generated
View File

@@ -1,5 +1,13 @@
{
"dependencies": {
"faye-websocket": {
"version": "0.7.2",
"dependencies": {
"websocket-driver": {
"version": "0.3.2"
}
}
},
"sockjs": {
"version": "0.3.8",
"dependencies": {
@@ -15,14 +23,6 @@
}
}
}
},
"faye-websocket": {
"version": "0.7.2",
"dependencies": {
"websocket-driver": {
"version": "0.3.2"
}
}
}
}
}

66
packages/livedata/DDP.md
View File

@@ -7,7 +7,7 @@ DDP is a protocol between a client and a server that supports two operations:
client informed about the contents of those documents as they change over
time.
This document specifies the version "pre1" of DDP. It's a rough description of
This document specifies the version "pre2" of DDP. It's a rough description of
the protocol and not intended to be entirely definitive.
## General Message Structure:
@@ -71,6 +71,23 @@ connections. The client can rely on the server sending a `failed`
message if a better version is possible as a result of the client or
the server having been upgraded.
## Heartbeats
### Messages:
* `ping`
- `id`: optional string (identifier used to correlate with response)
* `pong`
- `id`: optional string (same as received in the `ping` message)
### Procedure:
At any time after the connection is established either side may send a `ping`
message. The sender may chose to include an `id` field in the `ping`
message. When the other side receives a `ping` it must immediately respond with
a `pong` message. If the received `ping` message includes an `id` field, the
`pong` message must include the same `id` field.
## Managing Data:
### Messages:
@@ -170,6 +187,8 @@ the server having been upgraded.
- `method`: string (method name)
- `params`: optional array of EJSON items (parameters to the method)
- `id`: string (an arbitrary client-determined identifier for this method call)
- `randomSeed`: optional JSON value (an arbitrary client-determined seed
for pseudo-random generators)
* `result` (server -> client):
- `id`: string (the id passed to 'method')
- `error`: optional Error (an error thrown by the method (or method-not-found)
@@ -193,6 +212,19 @@ the server having been upgraded.
* There is no particular required ordering between `result` and `updated`
messages for a method call.
* The client may provide a randomSeed JSON value. If provided, this value is
used to seed pseudo-random number generation. By using the same seed with
the same algorithm, the same pseudo-random values can be generated on the
client and the server. In particular, this is used for generating ids for
newly created documents. If randomSeed is not provided, then values
generated on the server and the client will not be identical.
* Currently randomSeed is expected to be a string, and the algorithm by which
values are produced from this is not yet documented. It will likely be
formally specified in future when we are confident that the complete
requirements are known, or when a compatible implementation requires this
to be specified.
## Errors:
Errors appear in `result` and `nosub` messages in an optional error field. An
@@ -260,3 +292,35 @@ of EJSON should not rely on key order, if possible.
> MongoDB relies on key order. When using EJSON with MongoDB, the
> implementation of EJSON must preserve key order.
## Appendix 2: randomSeed backward/forward compatibility
randomSeed was added into DDP pre2, but it does not break backward or forward
compatibility.
If the client stub and the server produce documents that are different in any
way, Meteor will reconcile this difference. This may cause 'flicker' in the UI
as the values change on the client to reflect what happened on the server, but
the final result will be correct: the server and the client will agree.
Consistent id generation / randomSeed does not alter the syncing process, and
thus will (at worst) be the same:
* If neither the server nor the client support randomSeed, we will get the
classical/flicker behavior.
* If the client supports randomSeed, but the server does not, the server will
ignore randomSeed, as it ignores any unknown properties in a DDP method call.
Different ids will be generated, but this will be fixed by syncing.
* If the server supports randomSeed, but the client does not, the server will
generate unseeded random values (providing a randomSeed is optional); different
ids will be generated; and again this will be fixed by syncing.
* If both client and server support randomSeed, but different ids are
generated, either because the generation procedure is buggy, or the stub
behaves differently to the server, then syncing will fix this.
* If both client and server support randomSeed, in the normal case the ids
generated will be the same, and syncing will be a no-op.

102
packages/livedata/heartbeat.js Normal file
View File

@@ -0,0 +1,102 @@
// Heartbeat options:
// heartbeatInterval: interval to send pings, in milliseconds.
// heartbeatTimeout: timeout to close the connection if a reply isn't
// received, in milliseconds.
// sendPing: function to call to send a ping on the connection.
// onTimeout: function to call to close the connection.
Heartbeat = function (options) {
var self = this;
self.heartbeatInterval = options.heartbeatInterval;
self.heartbeatTimeout = options.heartbeatTimeout;
self._sendPing = options.sendPing;
self._onTimeout = options.onTimeout;
self._heartbeatIntervalHandle = null;
self._heartbeatTimeoutHandle = null;
};
_.extend(Heartbeat.prototype, {
stop: function () {
var self = this;
self._clearHeartbeatIntervalTimer();
self._clearHeartbeatTimeoutTimer();
},
start: function () {
var self = this;
self.stop();
self._startHeartbeatIntervalTimer();
},
_startHeartbeatIntervalTimer: function () {
var self = this;
self._heartbeatIntervalHandle = Meteor.setTimeout(
_.bind(self._heartbeatIntervalFired, self),
self.heartbeatInterval
);
},
_startHeartbeatTimeoutTimer: function () {
var self = this;
self._heartbeatTimeoutHandle = Meteor.setTimeout(
_.bind(self._heartbeatTimeoutFired, self),
self.heartbeatTimeout
);
},
_clearHeartbeatIntervalTimer: function () {
var self = this;
if (self._heartbeatIntervalHandle) {
Meteor.clearTimeout(self._heartbeatIntervalHandle);
self._heartbeatIntervalHandle = null;
}
},
_clearHeartbeatTimeoutTimer: function () {
var self = this;
if (self._heartbeatTimeoutHandle) {
Meteor.clearTimeout(self._heartbeatTimeoutHandle);
self._heartbeatTimeoutHandle = null;
}
},
// The heartbeat interval timer is fired when we should send a ping.
_heartbeatIntervalFired: function () {
var self = this;
self._heartbeatIntervalHandle = null;
self._sendPing();
// Wait for a pong.
self._startHeartbeatTimeoutTimer();
},
// The heartbeat timeout timer is fired when we sent a ping, but we
// timed out waiting for the pong.
_heartbeatTimeoutFired: function () {
var self = this;
self._heartbeatTimeoutHandle = null;
self._onTimeout();
},
pingReceived: function () {
var self = this;
// We know the connection is alive if we receive a ping, so we
// don't need to send a ping ourselves. Reset the interval timer.
if (self._heartbeatIntervalHandle) {
self._clearHeartbeatIntervalTimer();
self._startHeartbeatIntervalTimer();
}
},
pongReceived: function () {
var self = this;
// Receiving a pong means we won't timeout, so clear the timeout
// timer and start the interval again.
if (self._heartbeatTimeoutHandle) {
self._clearHeartbeatTimeoutTimer();
self._startHeartbeatIntervalTimer();
}
}
});

8
packages/livedata/livedata_common.js
View File

@@ -1,6 +1,6 @@
DDP = {};
SUPPORTED_DDP_VERSIONS = [ 'pre1' ];
SUPPORTED_DDP_VERSIONS = [ 'pre2', 'pre1' ];
LivedataTest.SUPPORTED_DDP_VERSIONS = SUPPORTED_DDP_VERSIONS;
@@ -31,6 +31,12 @@ MethodInvocation = function (options) {
// On the server, the connection this method call came in on.
this.connection = options.connection;
// The seed for randomStream value generation
this.randomSeed = options.randomSeed;
// This is set by RandomStream.get; and holds the random stream state
this.randomStream = null;
};
_.extend(MethodInvocation.prototype, {

131
packages/livedata/livedata_connection.js
View File

@@ -31,10 +31,13 @@ var Connection = function (url, options) {
onDDPVersionNegotiationFailure: function (description) {
Meteor._debug(description);
},
heartbeatInterval: 35000,
heartbeatTimeout: 15000,
// These options are only for testing.
reloadWithOutstanding: false,
supportedDDPVersions: SUPPORTED_DDP_VERSIONS,
retry: true
retry: true,
respondToPings: true
}, options);
// If set, called when we reconnect, queuing method calls _before_ the
@@ -64,6 +67,9 @@ var Connection = function (url, options) {
self._nextMethodId = 1;
self._supportedDDPVersions = options.supportedDDPVersions;
self._heartbeatInterval = options.heartbeatInterval;
self._heartbeatTimeout = options.heartbeatTimeout;
// Tracks methods which the user has tried to call but which have not yet
// called their user callback (ie, they are waiting on their result or for all
// of their writes to be written to the local cache). Map from method ID to
@@ -224,6 +230,17 @@ var Connection = function (url, options) {
options.onDDPVersionNegotiationFailure(description);
}
}
else if (msg.msg === 'ping') {
if (options.respondToPings)
self._send({msg: "pong", id: msg.id});
if (self._heartbeat)
self._heartbeat.pingReceived();
}
else if (msg.msg === 'pong') {
if (self._heartbeat) {
self._heartbeat.pongReceived();
}
}
else if (_.include(['added', 'changed', 'removed', 'ready', 'updated'], msg.msg))
self._livedata_data(msg);
else if (msg.msg === 'nosub')
@@ -291,12 +308,21 @@ var Connection = function (url, options) {
});
};
var onDisconnect = function () {
if (self._heartbeat) {
self._heartbeat.stop()
self._heartbeat = null;
}
};
if (Meteor.isServer) {
self._stream.on('message', Meteor.bindEnvironment(onMessage, Meteor._debug));
self._stream.on('reset', Meteor.bindEnvironment(onReset, Meteor._debug));
self._stream.on('disconnect', Meteor.bindEnvironment(onDisconnect, Meteor._debug));
} else {
self._stream.on('message', onMessage);
self._stream.on('reset', onReset);
self._stream.on('disconnect', onDisconnect);
}
};
@@ -493,7 +519,7 @@ _.extend(Connection.prototype, {
self._subscriptions[id] = {
id: id,
name: name,
params: params,
params: EJSON.clone(params),
inactive: false,
ready: false,
readyDeps: (typeof Deps !== "undefined") && new Deps.Dependency,
@@ -596,6 +622,17 @@ _.extend(Connection.prototype, {
// onResultReceived: Function - a callback to call as soon as the method
// result is received. the data written by
// the method may not yet be in the cache!
// returnStubValue: Boolean - If true then in cases where we would have
// otherwise discarded the stub's return value
// and returned undefined, instead we go ahead
// and return it. Specifically, this is any
// time other than when (a) we are already
// inside a stub or (b) we are in Node and no
// callback was provided. Currently we require
// this flag to be explicitly passed to reduce
// the likelihood that stub return values will
// be confused with server return values; we
// may improve this in future.
// @param callback {Optional Function}
apply: function (name, args, options, callback) {
var self = this;
@@ -618,6 +655,10 @@ _.extend(Connection.prototype, {
);
}
// Keep our args safe from mutation (eg if we don't send the message for a
// while because of a wait method).
args = EJSON.clone(args);
// Lazily allocate method ID once we know that it'll be needed.
var methodId = (function () {
var id;
@@ -628,6 +669,27 @@ _.extend(Connection.prototype, {
};
})();
var enclosing = DDP._CurrentInvocation.get();
var alreadyInSimulation = enclosing && enclosing.isSimulation;
// Lazily generate a randomSeed, only if it is requested by the stub.
// The random streams only have utility if they're used on both the client
// and the server; if the client doesn't generate any 'random' values
// then we don't expect the server to generate any either.
// Less commonly, the server may perform different actions from the client,
// and may in fact generate values where the client did not, but we don't
// have any client-side values to match, so even here we may as well just
// use a random seed on the server. In that case, we don't pass the
// randomSeed to save bandwidth, and we don't even generate it to save a
// bit of CPU and to avoid consuming entropy.
var randomSeed = null;
var randomSeedGenerator = function () {
if (randomSeed === null) {
randomSeed = makeRpcSeed(enclosing, name);
}
return randomSeed;
};
// Run the stub, if we have one. The stub is supposed to make some
// temporary writes to the database to give the user a smooth experience
// until the actual result of executing the method comes back from the
@@ -640,18 +702,17 @@ _.extend(Connection.prototype, {
// to do a RPC, so we use the return value of the stub as our return
// value.
var enclosing = DDP._CurrentInvocation.get();
var alreadyInSimulation = enclosing && enclosing.isSimulation;
var stub = self._methodHandlers[name];
if (stub) {
var setUserId = function(userId) {
self.setUserId(userId);
};
var invocation = new MethodInvocation({
isSimulation: true,
userId: self.userId(),
setUserId: setUserId
setUserId: setUserId,
randomSeed: function () { return randomSeedGenerator(); }
});
if (!alreadyInSimulation)
@@ -660,11 +721,12 @@ _.extend(Connection.prototype, {
try {
// Note that unlike in the corresponding server code, we never audit
// that stubs check() their arguments.
var ret = DDP._CurrentInvocation.withValue(invocation, function () {
var stubReturnValue = DDP._CurrentInvocation.withValue(invocation, function () {
if (Meteor.isServer) {
// Because saveOriginals and retrieveOriginals aren't reentrant,
// don't allow stubs to yield.
return Meteor._noYieldsAllowed(function () {
// re-clone, so that the stub can't affect our caller's values
return stub.apply(invocation, EJSON.clone(args));
});
} else {
@@ -685,12 +747,12 @@ _.extend(Connection.prototype, {
// we'll end up returning undefined.
if (alreadyInSimulation) {
if (callback) {
callback(exception, ret);
callback(exception, stubReturnValue);
return undefined;
}
if (exception)
throw exception;
return ret;
return stubReturnValue;
}
// If an exception occurred in a stub, and we're ignoring it
@@ -725,18 +787,25 @@ _.extend(Connection.prototype, {
// Send the RPC. Note that on the client, it is important that the
// stub have finished before we send the RPC, so that we know we have
// a complete list of which local documents the stub wrote.
var message = {
msg: 'method',
method: name,
params: args,
id: methodId()
};
// Send the randomSeed only if we used it
if (randomSeed !== null) {
message.randomSeed = randomSeed;
}
var methodInvoker = new MethodInvoker({
methodId: methodId(),
callback: callback,
connection: self,
onResultReceived: options.onResultReceived,
wait: !!options.wait,
message: {
msg: 'method',
method: name,
params: args,
id: methodId()
}
message: message
});
if (options.wait) {
@@ -761,7 +830,7 @@ _.extend(Connection.prototype, {
if (future) {
return future.wait();
}
return undefined;
return options.returnStubValue ? stubReturnValue : undefined;
},
// Before calling a method stub, prepare all stores to track changes and allow
@@ -834,6 +903,14 @@ _.extend(Connection.prototype, {
self._stream.send(stringifyDDP(obj));
},
// We detected via DDP-level heartbeats that we've lost the
// connection. Unlike `disconnect` or `close`, a lost connection
// will be automatically retried.
_lostConnection: function () {
var self = this;
self._stream._lostConnection();
},
status: function (/*passthrough args*/) {
var self = this;
return self._stream.status.apply(self._stream, arguments);
@@ -893,6 +970,28 @@ _.extend(Connection.prototype, {
_livedata_connected: function (msg) {
var self = this;
if (self._version !== 'pre1' && self._heartbeatInterval !== 0) {
self._heartbeat = new Heartbeat({
heartbeatInterval: self._heartbeatInterval,
heartbeatTimeout: self._heartbeatTimeout,
onTimeout: function () {
if (Meteor.isClient) {
// only print on the client. this message is useful on the
// browser console to see that we've lost connection. on the
// server (eg when doing server-to-server DDP), it gets
// kinda annoying. also this matches the behavior with
// sockjs timeouts.
Meteor._debug("Connection timeout. No DDP heartbeat received.");
}
self._lostConnection();
},
sendPing: function () {
self._send({msg: 'ping'});
}
});
self._heartbeat.start();
}
// If this is a reconnect, we'll have to reset all stores.
if (self._lastSessionId)
self._resetStores = true;

87
packages/livedata/livedata_connection_tests.js
View File

@@ -17,12 +17,16 @@ var makeConnectMessage = function (session) {
return msg;
}
// Tests that stream got a message that matches expected.
// Expected is normally an object, and allows a wildcard value of '*',
// which will then match any value.
// Returns the message (parsed as a JSON object if expected is an object);
// which is particularly handy if you want to extract a value that was
// matched as a wildcard.
var testGotMessage = function (test, stream, expected) {
var retVal = undefined;
if (stream.sent.length === 0) {
test.fail({error: 'no message received', expected: expected});
return retVal;
return undefined;
}
var got = stream.sent.shift();
@@ -42,17 +46,10 @@ var testGotMessage = function (test, stream, expected) {
_.each(keysWithStarValues, function (k) {
expected[k] = got[k];
});
if (keysWithStarValues.length === 1) {
retVal = got[keysWithStarValues[0]];
} else {
retVal = _.map(keysWithStarValues, function (k) {
return got[k];
});
}
}
test.equal(got, expected);
return retVal;
return got;
};
var startAndConnect = function(test, stream) {
@@ -275,6 +272,7 @@ Tinytest.add("livedata stub - this", function (test) {
conn.call('test_this', _.identity);
// satisfy method, quiesce connection
var message = JSON.parse(stream.sent.shift());
test.isUndefined(message.randomSeed);
test.equal(message, {msg: 'method', method: 'test_this',
params: [], id:message.id});
test.length(stream.sent, 0);
@@ -322,9 +320,11 @@ if (Meteor.isClient) {
test.equal(counts, {added: 1, removed: 0, changed: 0, moved: 0});
// get response from server
var message = JSON.parse(stream.sent.shift());
test.equal(message, {msg: 'method', method: 'do_something',
params: ['friday!'], id:message.id});
var message = testGotMessage(test, stream, {msg: 'method',
method: 'do_something',
params: ['friday!'],
id: '*',
randomSeed: '*'});
test.equal(coll.find({}).count(), 1);
test.equal(coll.find({value: 'friday!'}).count(), 1);
@@ -357,6 +357,7 @@ if (Meteor.isClient) {
// test we still send a method request to server
var message2 = JSON.parse(stream.sent.shift());
test.isUndefined(message2.randomSeed);
test.equal(message2, {msg: 'method', method: 'do_something_else',
params: ['monday'], id: message2.id});
@@ -401,6 +402,7 @@ Tinytest.add("livedata stub - mutating method args", function (test) {
// Method should be called with original arg, not mutated arg.
var message = JSON.parse(stream.sent.shift());
test.isUndefined(message.randomSeed);
test.equal(message, {msg: 'method', method: 'mutateArgs',
params: [{foo: 50}], id: message.id});
test.length(stream.sent, 0);
@@ -453,9 +455,11 @@ if (Meteor.isClient) {
conn.call('do_something', _.identity);
// see we only send message for outer methods
var message = JSON.parse(stream.sent.shift());
test.equal(message, {msg: 'method', method: 'do_something',
params: [], id:message.id});
var message = testGotMessage(test, stream, {msg: 'method',
method: 'do_something',
params: [],
id: '*',
randomSeed: '*'});
test.length(stream.sent, 0);
// but inner method runs locally.
@@ -566,6 +570,7 @@ Tinytest.add("livedata stub - reconnect", function (test) {
// The non-wait method should send, but not the wait method.
var methodMessage = JSON.parse(stream.sent.shift());
test.isUndefined(methodMessage.randomSeed);
test.equal(methodMessage, {msg: 'method', method: 'do_something',
params: [], id:methodMessage.id});
test.equal(stream.sent.length, 0);
@@ -623,6 +628,7 @@ Tinytest.add("livedata stub - reconnect", function (test) {
o.expectCallbacks({added: 1, changed: 1});
var waitMethodMessage = JSON.parse(stream.sent.shift());
test.isUndefined(waitMethodMessage.randomSeed);
test.equal(waitMethodMessage, {msg: 'method', method: 'do_something_else',
params: [], id: waitMethodMessage.id});
test.equal(stream.sent.length, 0);
@@ -633,6 +639,7 @@ Tinytest.add("livedata stub - reconnect", function (test) {
// wait method done means we can send the third method
test.equal(stream.sent.length, 1);
var laterMethodMessage = JSON.parse(stream.sent.shift());
test.isUndefined(laterMethodMessage.randomSeed);
test.equal(laterMethodMessage, {msg: 'method', method: 'do_something_later',
params: [], id: laterMethodMessage.id});
@@ -677,7 +684,7 @@ if (Meteor.isClient) {
// Method sent.
var methodId = testGotMessage(
test, stream, {msg: 'method', method: 'writeSomething',
params: [], id: '*'});
params: [], id: '*', randomSeed: '*'}).id;
test.equal(stream.sent.length, 0);
// Get some data.
@@ -744,7 +751,7 @@ if (Meteor.isClient) {
// Method sent.
var methodId2 = testGotMessage(
test, stream, {msg: 'method', method: 'writeSomething',
params: [], id: '*'});
params: [], id: '*', randomSeed: '*'}).id;
test.equal(stream.sent.length, 0);
// Get some data.
@@ -777,7 +784,7 @@ if (Meteor.isClient) {
testGotMessage(test, stream, makeConnectMessage(SESSION_ID + 1));
var slowMethodId = testGotMessage(
test, stream,
{msg: 'method', method: 'slowMethod', params: [], id: '*'});
{msg: 'method', method: 'slowMethod', params: [], id: '*'}).id;
// Still holding out hope for session resumption, so nothing updated yet.
test.equal(coll.find().count(), 2);
test.equal(coll.findOne(stubWrittenId2), {_id: stubWrittenId2, foo: 'bar'});
@@ -840,7 +847,7 @@ Tinytest.add("livedata stub - reconnect method which only got data", function (t
// Method sent.
var methodId = testGotMessage(
test, stream, {msg: 'method', method: 'doLittle',
params: [], id: '*'});
params: [], id: '*'}).id;
test.equal(stream.sent.length, 0);
// Get some data.
@@ -930,7 +937,7 @@ if (Meteor.isClient) {
// Method sent.
var insertMethodId = testGotMessage(
test, stream, {msg: 'method', method: 'insertSomething',
params: [], id: '*'});
params: [], id: '*', randomSeed: '*'}).id;
test.equal(stream.sent.length, 0);
// Call update method.
@@ -943,7 +950,7 @@ if (Meteor.isClient) {
// Method sent.
var updateMethodId = testGotMessage(
test, stream, {msg: 'method', method: 'updateIt',
params: [stubWrittenId], id: '*'});
params: [stubWrittenId], id: '*'}).id;
test.equal(stream.sent.length, 0);
// Get some data... slightly different than what we wrote.
@@ -1019,7 +1026,7 @@ if (Meteor.isClient) {
// first method sent
var firstMethodId = testGotMessage(
test, stream, {msg: 'method', method: 'no-op',
params: [], id: '*'});
params: [], id: '*'}).id;
test.equal(stream.sent.length, 0);
// ack the first method
@@ -1029,7 +1036,7 @@ if (Meteor.isClient) {
// Wait method sent.
var waitMethodId = testGotMessage(
test, stream, {msg: 'method', method: 'no-op',
params: [], id: '*'});
params: [], id: '*'}).id;
test.equal(stream.sent.length, 0);
// ack the wait method
@@ -1039,7 +1046,7 @@ if (Meteor.isClient) {
// insert method sent.
var insertMethodId = testGotMessage(
test, stream, {msg: 'method', method: 'insertSomething',
params: [], id: '*'});
params: [], id: '*', randomSeed: '*'}).id;
test.equal(stream.sent.length, 0);
// ack the insert method
@@ -1285,6 +1292,25 @@ Tinytest.add("livedata connection - onReconnect prepends messages correctly with
]);
});
Tinytest.add("livedata connection - ping without id", function (test) {
var stream = new StubStream();
var conn = newConnection(stream);
startAndConnect(test, stream);
stream.receive({msg: 'ping'});
testGotMessage(test, stream, {msg: 'pong'});
});
Tinytest.add("livedata connection - ping with id", function (test) {
var stream = new StubStream();
var conn = newConnection(stream);
startAndConnect(test, stream);
var id = Random.id();
stream.receive({msg: 'ping', id: id});
testGotMessage(test, stream, {msg: 'pong', id: id});
});
var getSelfConnectionUrl = function () {
if (Meteor.isClient) {
return Meteor._relativeToSiteRootUrl("/");
@@ -1429,7 +1455,7 @@ Tinytest.add("livedata connection - onReconnect with sent messages", function(te
// Test that we sent just the login message.
var loginId = testGotMessage(
test, stream, {msg: 'method', method: 'do_something',
params: ['login'], id: '*'});
params: ['login'], id: '*'}).id;
// we connect.
stream.receive({msg: 'connected', session: Random.id()});
@@ -1444,7 +1470,7 @@ Tinytest.add("livedata connection - onReconnect with sent messages", function(te
testGotMessage(
test, stream, {msg: 'method', method: 'do_something',
params: ['one'], id: '*'});
params: ['one'], id: '*'}).id;
});
@@ -1469,7 +1495,7 @@ Tinytest.add("livedata stub - reconnect double wait method", function (test) {
// Method sent.
var halfwayId = testGotMessage(
test, stream, {msg: 'method', method: 'halfwayMethod',
params: [], id: '*'});
params: [], id: '*'}).id;
test.equal(stream.sent.length, 0);
// Get the result. This means it will not be resent.
@@ -1483,7 +1509,7 @@ Tinytest.add("livedata stub - reconnect double wait method", function (test) {
testGotMessage(test, stream, makeConnectMessage(SESSION_ID));
var reconnectId = testGotMessage(
test, stream, {msg: 'method', method: 'reconnectMethod',
params: [], id: '*'});
params: [], id: '*'}).id;
test.length(stream.sent, 0);
// Still holding out hope for session resumption, so no callbacks yet.
test.equal(output, []);
@@ -1574,6 +1600,7 @@ if (Meteor.isClient) {
test.equal(coll.findOne(), {_id: "foo", bar: 42});
// It also sends the method message.
var methodMessage = JSON.parse(stream.sent.shift());
test.isUndefined(methodMessage.randomSeed);
test.equal(methodMessage, {msg: 'method', method: '/' + collName + '/insert',
params: [{_id: "foo", bar: 42}],
id: methodMessage.id});

103
packages/livedata/livedata_server.js
View File

@@ -216,7 +216,7 @@ _.extend(SessionCollectionView.prototype, {
/* Session */
/******************************************************************************/
var Session = function (server, version, socket) {
var Session = function (server, version, socket, options) {
var self = this;
self.id = Random.id();
@@ -262,13 +262,16 @@ var Session = function (server, version, socket) {
// temporary and will go away in the near future.
self._socketUrl = socket.url;
// Allow tests to disable responding to pings.
self._respondToPings = options.respondToPings;
// This object is the public interface to the session. In the public
// API, it is called the `connection` object. Internally we call it
// a `connectionHandle` to avoid ambiguity.
self.connectionHandle = {
id: self.id,
close: function () {
self.server._closeSession(self);
self.close();
},
onClose: function (fn) {
var cb = Meteor.bindEnvironment(fn, "connection onClose callback");
@@ -290,6 +293,20 @@ var Session = function (server, version, socket) {
self.startUniversalSubs();
}).run();
if (version !== 'pre1' && options.heartbeatInterval !== 0) {
self.heartbeat = new Heartbeat({
heartbeatInterval: options.heartbeatInterval,
heartbeatTimeout: options.heartbeatTimeout,
onTimeout: function () {
self.destroy();
},
sendPing: function () {
self.send({msg: 'ping'});
}
});
self.heartbeat.start();
}
Package.facts && Package.facts.Facts.incrementServerFact(
"livedata", "sessions", 1);
};
@@ -391,6 +408,15 @@ _.extend(Session.prototype, {
destroy: function () {
var self = this;
// Already destroyed.
if (!self.inQueue)
return;
if (self.heartbeat) {
self.heartbeat.stop();
self.heartbeat = null;
}
if (self.socket) {
self.socket.close();
self.socket._meteorSession = null;
@@ -417,6 +443,19 @@ _.extend(Session.prototype, {
});
},
// Destroy this session and unregister it at the server.
close: function () {
var self = this;
// Unconditionally destroy this session, even if it's not
// registered at the server.
self.destroy();
// Unregister the session. This will also call `destroy`, but
// that's OK because `destroy` is idempotent.
self.server._closeSession(self);
},
// Send a message (doing nothing if no socket is connected right now.)
// It should be a JSON object (it will be stringified.)
send: function (msg) {
@@ -457,6 +496,32 @@ _.extend(Session.prototype, {
if (!self.inQueue) // we have been destroyed.
return;
// Respond to ping and pong messages immediately without queuing.
// If the negotiated DDP version is "pre1" which didn't support
// pings, preserve the "pre1" behavior of responding with a "bad
// request" for the unknown messages.
//
// Fibers are needed because heartbeat uses Meteor.setTimeout, which
// needs a Fiber. We could actually use regular setTimeout and avoid
// these new fibers, but it is easier to just make everything use
// Meteor.setTimeout and not think too hard.
if (self.version !== 'pre1' && msg_in.msg === 'ping') {
if (self._respondToPings)
self.send({msg: "pong", id: msg_in.id});
if (self.heartbeat)
Fiber(function () {
self.heartbeat.pingReceived();
}).run();
return;
}
if (self.version !== 'pre1' && msg_in.msg === 'pong') {
if (self.heartbeat)
Fiber(function () {
self.heartbeat.pongReceived();
}).run();
return;
}
self.inQueue.push(msg_in);
if (self.workerRunning)
return;
@@ -530,14 +595,18 @@ _.extend(Session.prototype, {
var self = this;
// reject malformed messages
// XXX should also reject messages with unknown attributes?
// For now, we silently ignore unknown attributes,
// for forwards compatibility.
if (typeof (msg.id) !== "string" ||
typeof (msg.method) !== "string" ||
(('params' in msg) && !(msg.params instanceof Array))) {
(('params' in msg) && !(msg.params instanceof Array)) ||
(('randomSeed' in msg) && (typeof msg.randomSeed !== "string"))) {
self.sendError("Malformed method invocation", msg);
return;
}
var randomSeed = msg.randomSeed || null;
// set up to mark the method as satisfied once all observers
// (and subscriptions) have reacted to any writes that were
// done.
@@ -572,7 +641,8 @@ _.extend(Session.prototype, {
userId: self.userId,
setUserId: setUserId,
unblock: unblock,
connection: self.connectionHandle
connection: self.connectionHandle,
randomSeed: randomSeed
});
try {
var result = DDPServer._CurrentWriteFence.withValue(fence, function () {
@@ -1047,9 +1117,20 @@ _.extend(Subscription.prototype, {
/* Server */
/******************************************************************************/
Server = function () {
Server = function (options) {
var self = this;
// The default heartbeat interval is 30 seconds on the server and 35
// seconds on the client. Since the client doesn't need to send a
// ping as long as it is receiving pings, this means that pings
// normally go from the server to the client.
self.options = _.defaults(options || {}, {
heartbeatInterval: 30000,
heartbeatTimeout: 15000,
// For testing, allow responding to pings to be disabled.
respondToPings: true
});
// Map of callbacks to call when a new connection comes in to the
// server and completes DDP version negotiation. Use an object instead
// of an array so we can safely remove one from the list while
@@ -1120,7 +1201,7 @@ Server = function () {
socket.on('close', function () {
if (socket._meteorSession) {
Fiber(function () {
self._closeSession(socket._meteorSession);
socket._meteorSession.close();
}).run();
}
});
@@ -1142,7 +1223,7 @@ _.extend(Server.prototype, {
if (msg.version === version) {
// Creating a new session
socket._meteorSession = new Session(self, version, socket);
socket._meteorSession = new Session(self, version, socket, self.options);
self.sessions[socket._meteorSession.id] = socket._meteorSession;
self.onConnectionHook.each(function (callback) {
if (socket._meteorSession)
@@ -1318,12 +1399,14 @@ _.extend(Server.prototype, {
isSimulation: false,
userId: userId,
setUserId: setUserId,
connection: connection
connection: connection,
randomSeed: makeRpcSeed(currentInvocation, name)
});
try {
var result = DDP._CurrentInvocation.withValue(invocation, function () {
return maybeAuditArgumentChecks(
handler, invocation, args, "internal call to '" + name + "'");
handler, invocation, EJSON.clone(args), "internal call to '" +
name + "'");
});
} catch (e) {
exception = e;

3
packages/livedata/package.js
View File

@@ -49,6 +49,7 @@ Package.on_use(function (api) {
// _idParse, _idStringify.
api.use('minimongo', ['client', 'server']);
api.add_files('heartbeat.js', ['client', 'server']);
api.add_files('livedata_server.js', 'server');
@@ -56,6 +57,7 @@ Package.on_use(function (api) {
api.add_files('crossbar.js', 'server');
api.add_files('livedata_common.js', ['client', 'server']);
api.add_files('random_stream.js', ['client', 'server']);
api.add_files('livedata_connection.js', ['client', 'server']);
@@ -77,6 +79,7 @@ Package.on_test(function (api) {
api.add_files('livedata_test_service.js', ['client', 'server']);
api.add_files('session_view_tests.js', ['server']);
api.add_files('crossbar_tests.js', ['server']);
api.add_files('random_stream_tests.js', ['client', 'server']);
api.use('http', 'client');
api.add_files(['stream_tests.js'], 'client');

103
packages/livedata/random_stream.js Normal file
View File

@@ -0,0 +1,103 @@
// RandomStream allows for generation of pseudo-random values, from a seed.
//
// We use this for consistent 'random' numbers across the client and server.
// We want to generate probably-unique IDs on the client, and we ideally want
// the server to generate the same IDs when it executes the method.
//
// For generated values to be the same, we must seed ourselves the same way,
// and we must keep track of the current state of our pseudo-random generators.
// We call this state the scope. By default, we use the current DDP method
// invocation as our scope. DDP now allows the client to specify a randomSeed.
// If a randomSeed is provided it will be used to seed our random sequences.
// In this way, client and server method calls will generate the same values.
//
// We expose multiple named streams; each stream is independent
// and is seeded differently (but predictably from the name).
// By using multiple streams, we support reordering of requests,
// as long as they occur on different streams.
//
// @param options {Optional Object}
// seed: Array or value - Seed value(s) for the generator.
// If an array, will be used as-is
// If a value, will be converted to a single-value array
// If omitted, a random array will be used as the seed.
RandomStream = function (options) {
var self = this;
this.seed = [].concat(options.seed || randomToken());
this.sequences = {};
};
// Returns a random string of sufficient length for a random seed.
// This is a placeholder function; a similar function is planned
// for Random itself; when that is added we should remove this function,
// and call Random's randomToken instead.
function randomToken() {
return Random.hexString(20);
};
// Returns the random stream with the specified name, in the specified scope.
// If scope is null (or otherwise falsey) then we will use Random, which will
// give us as random numbers as possible, but won't produce the same
// values across client and server.
// However, scope will normally be the current DDP method invocation, so
// we'll use the stream with the specified name, and we should get consistent
// values on the client and server sides of a method call.
RandomStream.get = function (scope, name) {
if (!name) {
name = "default";
}
if (!scope) {
// There was no scope passed in;
// the sequence won't actually be reproducible.
return Random;
}
var randomStream = scope.randomStream;
if (!randomStream) {
scope.randomStream = randomStream = new RandomStream({
seed: scope.randomSeed
});
}
return randomStream._sequence(name);
};
// Returns the named sequence of pseudo-random values.
// The scope will be DDP._CurrentInvocation.get(), so the stream will produce
// consistent values for method calls on the client and server.
DDP.randomStream = function (name) {
var scope = DDP._CurrentInvocation.get();
return RandomStream.get(scope, name);
};
// Creates a randomSeed for passing to a method call.
// Note that we take enclosing as an argument,
// though we expect it to be DDP._CurrentInvocation.get()
// However, we often evaluate makeRpcSeed lazily, and thus the relevant
// invocation may not be the one currently in scope.
// If enclosing is null, we'll use Random and values won't be repeatable.
makeRpcSeed = function (enclosing, methodName) {
var stream = RandomStream.get(enclosing, '/rpc/' + methodName);
return stream.hexString(20);
};
_.extend(RandomStream.prototype, {
// Get a random sequence with the specified name, creating it if does not exist.
// New sequences are seeded with the seed concatenated with the name.
// By passing a seed into Random.create, we use the Alea generator.
_sequence: function (name) {
var self = this;
var sequence = self.sequences[name] || null;
if (sequence === null) {
var sequenceSeed = self.seed.concat(name);
for (var i = 0; i < sequenceSeed.length; i++) {
if (_.isFunction(sequenceSeed[i])) {
sequenceSeed[i] = sequenceSeed[i]();
}
}
self.sequences[name] = sequence = Random.createWithSeeds.apply(null, sequenceSeed);
}
return sequence;
}
});

44
packages/livedata/random_stream_tests.js Normal file
View File

@@ -0,0 +1,44 @@
Tinytest.add("livedata - DDP.randomStream", function (test) {
var randomSeed = Random.id();
var context = { randomSeed: randomSeed };
var sequence = DDP._CurrentInvocation.withValue(context, function () {
return DDP.randomStream('1');
});
var seeds = sequence.alea.args;
test.equal(seeds.length, 2);
test.equal(seeds[0], randomSeed);
test.equal(seeds[1], '1');
var id1 = sequence.id();
// Clone the sequence by building it the same way RandomStream.get does
var sequenceClone = Random.createWithSeeds.apply(null, seeds);
var id1Cloned = sequenceClone.id();
var id2Cloned = sequenceClone.id();
test.equal(id1, id1Cloned);
// We should get the same sequence when we use the same key
sequence = DDP._CurrentInvocation.withValue(context, function () {
return DDP.randomStream('1');
});
seeds = sequence.alea.args;
test.equal(seeds.length, 2);
test.equal(seeds[0], randomSeed);
test.equal(seeds[1], '1');
// But we should be at the 'next' position in the stream
var id2 = sequence.id();
// Technically these could be equal, but likely to be a bug if hit
// http://search.dilbert.com/comic/Random%20Number%20Generator
test.notEqual(id1, id2);
test.equal(id2, id2Cloned);
});
Tinytest.add("livedata - DDP.randomStream with no-args", function (test) {
DDP.randomStream().id();
});

2
packages/livedata/stream_client_common.js
View File

@@ -87,7 +87,7 @@ _.extend(LivedataTest.ClientStream.prototype, {
on: function (name, callback) {
var self = this;
if (name !== 'message' && name !== 'reset')
if (name !== 'message' && name !== 'reset' && name !== 'disconnect')
throw new Error("unknown event type: " + name);
if (!self.eventCallbacks[name])

2
packages/livedata/stream_client_nodejs.js
View File

@@ -96,6 +96,8 @@ _.extend(LivedataTest.ClientStream.prototype, {
self.client = null;
client.close();
}
_.each(self.eventCallbacks.disconnect, function (callback) { callback(); });
},
_clearConnectionTimer: function () {

18
packages/livedata/stream_client_sockjs.js
View File

@@ -12,13 +12,15 @@ LivedataTest.ClientStream = function (url, options) {
// how long between hearing heartbeat from the server until we declare
// the connection dead. heartbeats come every 25s (stream_server.js)
// the connection dead. heartbeats come every 45s (stream_server.js)
//
// NOTE: this is a workaround until sockjs detects heartbeats on the
// client automatically.
// https://github.com/sockjs/sockjs-client/issues/67
// https://github.com/sockjs/sockjs-node/issues/68
self.HEARTBEAT_TIMEOUT = 60000;
// NOTE: this is a older timeout mechanism. We now send heartbeats at
// the DDP level (https://github.com/meteor/meteor/pull/1865), and
// expect those timeouts to kill a non-responsive connection before
// this timeout fires. This is kept around for compatibility (when
// talking to a server that doesn't support DDP heartbeats) and can be
// removed later.
self.HEARTBEAT_TIMEOUT = 100*1000;
self.rawUrl = url;
self.socket = null;
@@ -88,6 +90,8 @@ _.extend(LivedataTest.ClientStream.prototype, {
self.socket.close();
self.socket = null;
}
_.each(self.eventCallbacks.disconnect, function (callback) { callback(); });
},
_clearConnectionAndHeartbeatTimers: function () {
@@ -104,7 +108,7 @@ _.extend(LivedataTest.ClientStream.prototype, {
_heartbeat_timeout: function () {
var self = this;
Meteor._debug("Connection timeout. No heartbeat received.");
Meteor._debug("Connection timeout. No sockjs heartbeat received.");
self._lostConnection();
},

2
packages/livedata/stream_server.js
View File

@@ -23,7 +23,7 @@ StreamServer = function () {
log: function() {},
// this is the default, but we code it explicitly because we depend
// on it in stream_client:HEARTBEAT_TIMEOUT
heartbeat_delay: 25000,
heartbeat_delay: 45000,
// The default disconnect_delay is 5 seconds, but if the server ends up CPU
// bound for that much time, SockJS might not notice that the user has
// reconnected because the timer (of disconnect_delay ms) can fire before

3
packages/livedata/stub_stream.js
View File

@@ -30,6 +30,9 @@ _.extend(StubStream.prototype, {
// no-op
},
_lostConnection: function () {
// no-op
},
// Methods for tests
receive: function (data) {

10
packages/meetup/meetup_client.js
View File

@@ -13,9 +13,15 @@ Meetup.requestCredential = function (options, credentialRequestCompleteCallback)
var config = ServiceConfiguration.configurations.findOne({service: 'meetup'});
if (!config) {
credentialRequestCompleteCallback && credentialRequestCompleteCallback(new ServiceConfiguration.ConfigError("Service not configured"));
credentialRequestCompleteCallback && credentialRequestCompleteCallback(
new ServiceConfiguration.ConfigError());
return;
}
// For some reason, meetup converts underscores to spaces in the state
// parameter when redirecting back to the client, so we use
// `Random.id()` here (alphanumerics) instead of `Random.secret()`
// (base 64 characters).
var credentialToken = Random.id();
var scope = (options && options.requestPermissions) || [];
@@ -34,7 +40,7 @@ Meetup.requestCredential = function (options, credentialRequestCompleteCallback)
if (_.without(scope, 'basic').length)
height += 130;
Oauth.showPopup(
OAuth.showPopup(
loginUrl,
_.bind(credentialRequestCompleteCallback, null, credentialToken),
{width: 900, height: height}

8
packages/meetup/meetup_server.js
View File

@@ -1,6 +1,6 @@
Meetup = {};
Oauth.registerService('meetup', 2, null, function(query) {
OAuth.registerService('meetup', 2, null, function(query) {
var accessToken = getAccessToken(query);
var identity = getIdentity(accessToken);
@@ -17,7 +17,7 @@ Oauth.registerService('meetup', 2, null, function(query) {
var getAccessToken = function (query) {
var config = ServiceConfiguration.configurations.findOne({service: 'meetup'});
if (!config)
throw new ServiceConfiguration.ConfigError("Service not configured");
throw new ServiceConfiguration.ConfigError();
var response;
try {
@@ -25,7 +25,7 @@ var getAccessToken = function (query) {
"https://secure.meetup.com/oauth2/access", {headers: {Accept: 'application/json'}, params: {
code: query.code,
client_id: config.clientId,
client_secret: config.secret,
client_secret: OAuth.openSecret(config.secret),
grant_type: 'authorization_code',
redirect_uri: Meteor.absoluteUrl("_oauth/meetup?close"),
state: query.state
@@ -56,5 +56,5 @@ var getIdentity = function (accessToken) {
Meetup.retrieveCredential = function(credentialToken) {
return Oauth.retrieveCredential(credentialToken);
return OAuth.retrieveCredential(credentialToken);
};

8
packages/meteor-developer/meteor_developer_client.js
View File

@@ -10,13 +10,11 @@ var requestCredential = function (credentialRequestCompleteCallback) {
});
if (!config) {
credentialRequestCompleteCallback &&
credentialRequestCompleteCallback(
new ServiceConfiguration.ConfigError("Service not configured")
);
credentialRequestCompleteCallback(new ServiceConfiguration.ConfigError());
return;
}
var credentialToken = Random.id();
var credentialToken = Random.secret();
var loginUrl =
METEOR_DEVELOPER_URL + "/oauth2/authorize?" +
@@ -25,7 +23,7 @@ var requestCredential = function (credentialRequestCompleteCallback) {
"client_id=" + config.clientId +
"&redirect_uri=" + Meteor.absoluteUrl("_oauth/meteor-developer?close");
Oauth.showPopup(
OAuth.showPopup(
loginUrl,
_.bind(credentialRequestCompleteCallback, null, credentialToken),
{

12
packages/meteor-developer/meteor_developer_server.js
View File

@@ -1,12 +1,12 @@
MeteorDeveloperAccounts = {};
Oauth.registerService("meteor-developer", 2, null, function (query) {
OAuth.registerService("meteor-developer", 2, null, function (query) {
var response = getTokens(query);
var accessToken = response.accessToken;
var identity = getIdentity(accessToken);
var serviceData = {
accessToken: accessToken,
accessToken: OAuth.sealSecret(accessToken),
expiresAt: (+new Date) + (1000 * response.expiresIn)
};
@@ -16,7 +16,7 @@ Oauth.registerService("meteor-developer", 2, null, function (query) {
// that we don't lose old ones (since we only get this on the first
// log in attempt)
if (response.refreshToken)
serviceData.refreshToken = response.refreshToken;
serviceData.refreshToken = OAuth.sealSecret(response.refreshToken);
return {
serviceData: serviceData,
@@ -35,7 +35,7 @@ var getTokens = function (query) {
service: 'meteor-developer'
});
if (!config)
throw new ServiceConfiguration.ConfigError("Service not configured");
throw new ServiceConfiguration.ConfigError();
var response;
try {
@@ -45,7 +45,7 @@ var getTokens = function (query) {
grant_type: "authorization_code",
code: query.code,
client_id: config.clientId,
client_secret: config.secret,
client_secret: OAuth.openSecret(config.secret),
redirect_uri: Meteor.absoluteUrl("_oauth/meteor-developer?close")
}
}
@@ -94,5 +94,5 @@ var getIdentity = function (accessToken) {
};
MeteorDeveloperAccounts.retrieveCredential = function (credentialToken) {
return Oauth.retrieveCredential(credentialToken);
return OAuth.retrieveCredential(credentialToken);
};

13
packages/meteor/url_server.js
View File

@@ -1,6 +1,15 @@
if (process.env.ROOT_URL &&
typeof __meteor_runtime_config__ === "object") {
__meteor_runtime_config__.ROOT_URL = process.env.ROOT_URL;
var pathPrefix = Npm.require('url').parse(__meteor_runtime_config__.ROOT_URL).pathname;
__meteor_runtime_config__.ROOT_URL_PATH_PREFIX = pathPrefix === "/" ? "" : pathPrefix;
if (__meteor_runtime_config__.ROOT_URL) {
var parsedUrl = Npm.require('url').parse(__meteor_runtime_config__.ROOT_URL);
// Sometimes users try to pass, eg, ROOT_URL=mydomain.com.
if (!parsedUrl.host) {
throw Error("$ROOT_URL, if specified, must be an URL");
}
var pathPrefix = parsedUrl.pathname;
__meteor_runtime_config__.ROOT_URL_PATH_PREFIX = pathPrefix === "/" ? "" : pathPrefix;
} else {
__meteor_runtime_config__.ROOT_URL_PATH_PREFIX = "";
}
}

32
packages/minifiers/.npm/package/npm-shrinkwrap.json generated
View File

@@ -1,5 +1,21 @@
{
"dependencies": {
"css-parse": {
"version": "https://github.com/reworkcss/css-parse/tarball/aa7e23285375ca621dd20250bac0266c6d8683a5"
},
"css-stringify": {
"version": "https://github.com/reworkcss/css-stringify/tarball/a7fe6de82e055d41d1c5923ec2ccef06f2a45efa",
"dependencies": {
"source-map": {
"version": "0.1.33",
"dependencies": {
"amdefine": {
"version": "0.1.0"
}
}
}
}
},
"uglify-js": {
"version": "2.4.7",
"dependencies": {
@@ -26,22 +42,6 @@
"version": "1.0.1"
}
}
},
"css-parse": {
"version": "1.7.0"
},
"css-stringify": {
"version": "1.4.1",
"dependencies": {
"source-map": {
"version": "0.1.31",
"dependencies": {
"amdefine": {
"version": "0.1.0"
}
}
}
}
}
}
}

0
packages/minifiers/beautify_tests.js → packages/minifiers/beautify-tests.js
View File

7
packages/minifiers/minification.js
View File

@@ -95,6 +95,13 @@ traverse.page = function(node) {
+ emit('}');
};
traverse['font-face'] = function(node){
return emit('@font-face', node.position, true)
+ emit('{')
+ mapVisit(node.declarations)
+ emit('}');
};
traverse.rule = function(node) {
var decls = node.declarations;
if (!decls.length) return '';

2
packages/minifiers/minifiers-tests.js
View File

@@ -1,3 +1,4 @@
Tinytest.add("minifiers - simple css minification", function (test) {
var t = function (css, expected, desc) {
test.equal(CssTools.minifyCss(css), expected, desc);
@@ -18,4 +19,3 @@ Tinytest.add("minifiers - simple css minification", function (test) {
'a{font:12px Helvetica,Arial,Nautica;background:url("/some/nice/picture.png")}', 'removing quotes in font and url (if possible)');
t('/* no comments */ a { color: red; }', 'a{color:red}', 'remove comments');
});

48
packages/minifiers/minifiers.js
View File

@@ -2,6 +2,8 @@ UglifyJSMinify = Npm.require('uglify-js').minify;
var cssParse = Npm.require('css-parse');
var cssStringify = Npm.require('css-stringify');
var path = Npm.require('path');
var url = Npm.require('url');
CssTools = {
parseCss: cssParse,
@@ -52,6 +54,8 @@ CssTools = {
break;
}
CssTools.rewriteCssUrls(ast);
var imports = ast.stylesheet.rules.splice(0, importCount);
newAst.stylesheet.rules = newAst.stylesheet.rules.concat(imports);
@@ -71,6 +75,50 @@ CssTools = {
});
return newAst;
},
// We are looking for all relative urls defined with the `url()` functional
// notation and rewriting them to the equivalent absolute url using the
// `position.source` path provided by css-parse
// For performance reasons this function acts by side effect by modifying the
// given AST without doing a deep copy.
rewriteCssUrls: function (ast) {
var isRelative = function(path) {
return path.charAt(0) !== '/';
};
_.each(ast.stylesheet.rules, function(rule, ruleIndex) {
var basePath = path.dirname(rule.position.source);
_.each(rule.declarations, function(declaration, declarationIndex) {
var parts, resource, absolutePath, quotes, oldCssUrl, newCssUrl;
var value = declaration.value;
// Match css values containing some functional calls to `url(URI)` where
// URI is optionally quoted.
// Note that a css value can contains other elements, for instance:
// background: top center url("background.png") black;
// or even multiple url(), for instance for multiple backgrounds.
var cssUrlRegex = /url\s*\(\s*(['"]?)(.+?)\1\s*\)/gi;
while (parts = cssUrlRegex.exec(value)) {
oldCssUrl = parts[0];
quotes = parts[1];
resource = url.parse(parts[2]);
// Rewrite relative paths to absolute paths.
// We don't rewrite urls starting with a protocol definition such as
// http, https, or data.
if (isRelative(resource.path) && resource.protocol === null) {
absolutePath = path.join(basePath, resource.path);
newCssUrl = "url(" + quotes + absolutePath + quotes + ")";
value = value.replace(oldCssUrl, newCssUrl);
}
}
declaration.value = value;
});
});
}
};

10
packages/minifiers/package.js
View File

@@ -5,8 +5,8 @@ Package.describe({
Npm.depends({
"uglify-js": "2.4.7",
"css-parse": "1.7.0",
"css-stringify": "1.4.1"
"css-parse": "https://github.com/reworkcss/css-parse/tarball/aa7e23285375ca621dd20250bac0266c6d8683a5",
"css-stringify": "https://github.com/reworkcss/css-stringify/tarball/a7fe6de82e055d41d1c5923ec2ccef06f2a45efa"
});
Package.on_use(function (api) {
@@ -19,5 +19,9 @@ Package.on_test(function (api) {
api.use('minifiers', 'server');
api.use('tinytest');
api.add_files('beautify_tests.js', 'server');
api.add_files([
'beautify-tests.js',
'minifiers-tests.js',
'urlrewriting-tests.js'
], 'server');
});

32
packages/minifiers/urlrewriting-tests.js Normal file
View File

@@ -0,0 +1,32 @@
Tinytest.add("minifiers - url rewriting when merging", function (test) {
var stylesheet = function(backgroundPath) {
return "body { color: green; background: top center url(" + backgroundPath + ") black, bottom center url(" + backgroundPath + "); }"
};
var filename = 'dir/subdir/style.css';
var parseOptions = { source: filename, position: true };
var t = function(relativeUrl, absoluteUrl, desc) {
var ast1 = CssTools.parseCss(stylesheet(relativeUrl), parseOptions);
var ast2 = CssTools.parseCss(stylesheet(absoluteUrl), parseOptions);
CssTools.rewriteCssUrls(ast1);
test.equal(CssTools.stringifyCss(ast1), CssTools.stringifyCss(ast2), desc);
};
t('../image.png', 'dir/image.png', 'parent directory');
t('./../image.png', 'dir/image.png', 'parent directory');
t('../subdir2/image.png', 'dir/subdir2/image.png', 'cousin directory');
t('../../image.png', 'image.png', 'grand parent directory');
t('./image.png', 'dir/subdir/image.png', 'current directory');
t('./child/image.png', 'dir/subdir/child/image.png', 'child directory');
t('child/image.png', 'dir/subdir/child/image.png', 'child directory');
t('/image.png', '/image.png', 'absolute url');
t('"/image.png"', '"/image.png"', 'double quoted url');
t("'/image.png'", "'/image.png'", 'single quoted url');
t('"./../image.png"', '"dir/image.png"', 'quoted parent directory');
t('http://i.imgur.com/fBcdJIh.gif', 'http://i.imgur.com/fBcdJIh.gif', 'complete URL');
t('"http://i.imgur.com/fBcdJIh.gif"', '"http://i.imgur.com/fBcdJIh.gif"', 'complete quoted URL');
t('data:image/png;base64,iVBORw0K=', 'data:image/png;base64,iVBORw0K=', 'data URI');
});

9
packages/mongo-livedata/.npm/package/npm-shrinkwrap.json generated
View File

@@ -1,10 +1,15 @@
{
"dependencies": {
"mongodb": {
"version": "https://github.com/meteor/node-mongodb-native/tarball/779bbac916a751f305d84c727a6cc7dfddab7924",
"version": "1.4.1",
"dependencies": {
"bson": {
"version": "0.2.2"
"version": "0.2.7",
"dependencies": {
"nan": {
"version": "0.8.0"
}
}
},
"kerberos": {
"version": "0.0.3"

5
packages/mongo-livedata/allow_tests.js
View File

@@ -132,7 +132,8 @@ if (Meteor.isServer) {
}
}, {
insert: function(userId, doc) {
return doc.cantInsert2;
// Don't allow explicit ID to be set by the client.
return _.has(doc, '_id');
},
update: function(userId, doc, fields, modifier) {
return -1 !== _.indexOf(fields, 'verySecret');
@@ -560,7 +561,7 @@ if (Meteor.isClient) {
// insert with one allow and other deny. denied.
function (test, expect) {
collection.insert(
{canInsert: true, cantInsert2: true},
{canInsert: true, _id: Random.id()},
expect(function (err, res) {
test.equal(err.error, 403);
test.equal(collection.find().count(), 0);

99
packages/mongo-livedata/collection.js
View File

@@ -40,13 +40,15 @@ Meteor.Collection = function (name, options) {
switch (options.idGeneration) {
case 'MONGO':
self._makeNewID = function () {
return new Meteor.Collection.ObjectID();
var src = name ? DDP.randomStream('/collection/' + name) : Random;
return new Meteor.Collection.ObjectID(src.hexString(24));
};
break;
case 'STRING':
default:
self._makeNewID = function () {
return Random.id();
var src = name ? DDP.randomStream('/collection/' + name) : Random;
return src.id();
};
break;
}
@@ -207,6 +209,13 @@ _.extend(Meteor.Collection.prototype, {
if (args.length < 2) {
return { transform: self._transform };
} else {
check(args[1], Match.Optional(Match.ObjectIncluding({
fields: Match.Optional(Object),
sort: Match.Optional(Match.OneOf(Object, Array)),
limit: Match.Optional(Number),
skip: Match.Optional(Number)
})));
return _.extend({
transform: self._transform
}, args[1]);
@@ -367,7 +376,19 @@ _.each(["insert", "update", "remove"], function (name) {
|| insertId instanceof Meteor.Collection.ObjectID))
throw new Error("Meteor requires document _id fields to be non-empty strings or ObjectIDs");
} else {
insertId = args[0]._id = self._makeNewID();
var generateId = true;
// Don't generate the id if we're the client and the 'outermost' call
// This optimization saves us passing both the randomSeed and the id
// Passing both is redundant.
if (self._connection && self._connection !== Meteor.server) {
var enclosing = DDP._CurrentInvocation.get();
if (!enclosing) {
generateId = false;
}
}
if (generateId) {
insertId = args[0]._id = self._makeNewID();
}
}
} else {
args[0] = Meteor.Collection._rewriteSelector(args[0]);
@@ -394,10 +415,14 @@ _.each(["insert", "update", "remove"], function (name) {
// On inserts, always return the id that we generated; on all other
// operations, just return the result from the collection.
var chooseReturnValueFromCollectionResult = function (result) {
if (name === "insert")
if (name === "insert") {
if (!insertId && result) {
insertId = result;
}
return insertId;
else
} else {
return result;
}
};
var wrappedCallback;
@@ -437,7 +462,7 @@ _.each(["insert", "update", "remove"], function (name) {
}
ret = chooseReturnValueFromCollectionResult(
self._connection.apply(self._prefix + name, args, wrappedCallback)
self._connection.apply(self._prefix + name, args, {returnStubValue: true}, wrappedCallback)
);
} else {
@@ -630,13 +655,31 @@ Meteor.Collection.prototype._defineMutationMethods = function() {
m[self._prefix + method] = function (/* ... */) {
// All the methods do their own validation, instead of using check().
check(arguments, [Match.Any]);
var args = _.toArray(arguments);
try {
if (this.isSimulation) {
// For an insert, if the client didn't specify an _id, generate one
// now; because this uses DDP.randomStream, it will be consistent with
// what the client generated. We generate it now rather than later so
// that if (eg) an allow/deny rule does an insert to the same
// collection (not that it really should), the generated _id will
// still be the first use of the stream and will be consistent.
//
// However, we don't actually stick the _id onto the document yet,
// because we want allow/deny rules to be able to differentiate
// between arbitrary client-specified _id fields and merely
// client-controlled-via-randomSeed fields.
var generatedId = null;
if (method === "insert" && !_.has(args[0], '_id')) {
generatedId = self._makeNewID();
}
if (this.isSimulation) {
// In a client simulation, you can do any mutation (even with a
// complex selector).
if (generatedId !== null)
args[0]._id = generatedId;
return self._collection[method].apply(
self._collection, _.toArray(arguments));
self._collection, args);
}
// This is the server receiving a method call from the client.
@@ -644,7 +687,7 @@ Meteor.Collection.prototype._defineMutationMethods = function() {
// We don't allow arbitrary selectors in mutations from the client: only
// single-ID selectors.
if (method !== 'insert')
throwIfSelectorIsNotId(arguments[0], method);
throwIfSelectorIsNotId(args[0], method);
if (self._restricted) {
// short circuit if there is no way it will pass.
@@ -656,12 +699,14 @@ Meteor.Collection.prototype._defineMutationMethods = function() {
var validatedMethodName =
'_validated' + method.charAt(0).toUpperCase() + method.slice(1);
var argsWithUserId = [this.userId].concat(_.toArray(arguments));
return self[validatedMethodName].apply(self, argsWithUserId);
args.unshift(this.userId);
generatedId !== null && args.push(generatedId);
return self[validatedMethodName].apply(self, args);
} else if (self._isInsecure()) {
if (generatedId !== null)
args[0]._id = generatedId;
// In insecure mode, allow any mutation (with a simple selector).
return self._collection[method].apply(self._collection,
_.toArray(arguments));
return self._collection[method].apply(self._collection, args);
} else {
// In secure mode, if we haven't called allow or deny, then nothing
// is permitted.
@@ -706,30 +751,46 @@ Meteor.Collection.prototype._isInsecure = function () {
return self._insecure;
};
var docToValidate = function (validator, doc) {
var docToValidate = function (validator, doc, generatedId) {
var ret = doc;
if (validator.transform)
ret = validator.transform(EJSON.clone(doc));
if (validator.transform) {
ret = EJSON.clone(doc);
// If you set a server-side transform on your collection, then you don't get
// to tell the difference between "client specified the ID" and "server
// generated the ID", because transforms expect to get _id. If you want to
// do that check, you can do it with a specific
// `C.allow({insert: f, transform: null})` validator.
if (generatedId !== null) {
ret._id = generatedId;
}
ret = validator.transform(ret);
}
return ret;
};
Meteor.Collection.prototype._validatedInsert = function(userId, doc) {
Meteor.Collection.prototype._validatedInsert = function (userId, doc,
generatedId) {
var self = this;
// call user validators.
// Any deny returns true means denied.
if (_.any(self._validators.insert.deny, function(validator) {
return validator(userId, docToValidate(validator, doc));
return validator(userId, docToValidate(validator, doc, generatedId));
})) {
throw new Meteor.Error(403, "Access denied");
}
// Any allow returns true means proceed. Throw error if they all fail.
if (_.all(self._validators.insert.allow, function(validator) {
return !validator(userId, docToValidate(validator, doc));
return !validator(userId, docToValidate(validator, doc, generatedId));
})) {
throw new Meteor.Error(403, "Access denied");
}
// If we generated an ID above, insert it now: after the validation, but
// before actually inserting.
if (generatedId !== null)
doc._id = generatedId;
self._collection.insert.call(self._collection, doc);
};

47
packages/mongo-livedata/mongo_driver.js
View File

@@ -113,6 +113,7 @@ MongoConnection = function (url, options) {
options = options || {};
self._connectCallbacks = [];
self._observeMultiplexers = {};
self._onFailoverHook = new Hook;
var mongoOptions = {db: {safe: true}, server: {}, replSet: {}};
@@ -144,18 +145,42 @@ MongoConnection = function (url, options) {
mongoOptions.replSet.poolSize = options.poolSize;
}
MongoDB.connect(url, mongoOptions, function(err, db) {
MongoDB.connect(url, mongoOptions, Meteor.bindEnvironment(function(err, db) {
if (err)
throw err;
self.db = db;
// We keep track of the ReplSet's primary, so that we can trigger hooks when
// it changes. The Node driver's joined callback seems to fire way too
// often, which is why we need to track it ourselves.
self._primary = null;
// First, figure out what the current primary is, if any.
if (self.db.serverConfig._state.master)
self._primary = self.db.serverConfig._state.master.name;
self.db.serverConfig.on(
'joined', Meteor.bindEnvironment(function (kind, doc) {
if (kind === 'primary') {
if (doc.primary !== self._primary) {
self._primary = doc.primary;
self._onFailoverHook.each(function (callback) {
callback();
return true;
});
}
} else if (doc.me === self._primary) {
// The thing we thought was primary is now something other than
// primary. Forget that we thought it was primary. (This means that
// if a server stops being primary and then starts being primary again
// without another server becoming primary in the middle, we'll
// correctly count it as a failover.)
self._primary = null;
}
}));
Fiber(function () {
// drain queue of pending callbacks
_.each(self._connectCallbacks, function (c) {
c(db);
});
}).run();
});
// drain queue of pending callbacks
_.each(self._connectCallbacks, function (c) {
c(db);
});
}));
self._docFetcher = new DocFetcher(self);
self._oplogHandle = null;
@@ -229,6 +254,12 @@ MongoConnection.prototype._maybeBeginWrite = function () {
return {committed: function () {}};
};
// Internal interface: adds a callback which is called when the Mongo primary
// changes. Returns a stop handle.
MongoConnection.prototype._onFailover = function (callback) {
return this._onFailoverHook.register(callback);
};
//////////// Public API //////////

395
packages/mongo-livedata/mongo_livedata_tests.js
View File

@@ -2,6 +2,10 @@
// the selector (or inserted document) contains fail: true.
var TRANSFORMS = {};
// We keep track of the collections, so we can refer to them by name
var COLLECTIONS = {};
if (Meteor.isServer) {
Meteor.methods({
createInsecureCollection: function (name, options) {
@@ -15,6 +19,7 @@ if (Meteor.isServer) {
options.transform = TRANSFORMS[options.transformName];
}
var c = new Meteor.Collection(name, options);
COLLECTIONS[name] = c;
c._insecure = true;
Meteor.publish('c-' + name, function () {
return c.find();
@@ -23,6 +28,32 @@ if (Meteor.isServer) {
});
}
// We store the generated id, keyed by collection, for each insert
// This is so we can test the stub and the server generate the same id
var INSERTED_IDS = {};
Meteor.methods({
insertObjects: function (collectionName, doc, count) {
var c = COLLECTIONS[collectionName];
var ids = [];
for (var i = 0; i < count; i++) {
var id = c.insert(doc);
INSERTED_IDS[collectionName] = (INSERTED_IDS[collectionName] || []).concat([id]);
ids.push(id);
}
return ids;
},
upsertObject: function (collectionName, selector, modifier) {
var c = COLLECTIONS[collectionName];
return c.upsert(selector, modifier);
},
doMeteorCall: function (name /*, arguments */) {
var args = Array.prototype.slice.call(arguments);
return Meteor.call.apply(null, args);
}
});
var runInFence = function (f) {
if (Meteor.isClient) {
f();
@@ -1216,13 +1247,13 @@ if (Meteor.isServer) {
testAsyncMulti('mongo-livedata - empty documents, ' + idGeneration, [
function (test, expect) {
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName);
Meteor.subscribe('c-' + this.collectionName, expect());
}
var coll = new Meteor.Collection(collectionName, collectionOptions);
}, function (test, expect) {
var coll = new Meteor.Collection(this.collectionName, collectionOptions);
coll.insert({}, expect(function (err, id) {
test.isFalse(err);
@@ -1236,28 +1267,29 @@ testAsyncMulti('mongo-livedata - empty documents, ' + idGeneration, [
// See https://github.com/meteor/meteor/issues/594.
testAsyncMulti('mongo-livedata - document with length, ' + idGeneration, [
function (test, expect) {
var self = this;
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName, collectionOptions);
Meteor.subscribe('c-' + this.collectionName, expect());
}
}, function (test, expect) {
var self = this;
var coll = self.coll = new Meteor.Collection(self.collectionName, collectionOptions);
self.coll = new Meteor.Collection(collectionName, collectionOptions);
self.coll.insert({foo: 'x', length: 0}, expect(function (err, id) {
coll.insert({foo: 'x', length: 0}, expect(function (err, id) {
test.isFalse(err);
test.isTrue(id);
self.docId = id;
test.equal(self.coll.findOne(self.docId),
test.equal(coll.findOne(self.docId),
{_id: self.docId, foo: 'x', length: 0});
}));
},
function (test, expect) {
var self = this;
self.coll.update(self.docId, {$set: {length: 5}}, expect(function (err) {
var coll = self.coll;
coll.update(self.docId, {$set: {length: 5}}, expect(function (err) {
test.isFalse(err);
test.equal(self.coll.findOne(self.docId),
test.equal(coll.findOne(self.docId),
{_id: self.docId, foo: 'x', length: 5});
}));
}
@@ -1265,13 +1297,14 @@ testAsyncMulti('mongo-livedata - document with length, ' + idGeneration, [
testAsyncMulti('mongo-livedata - document with a date, ' + idGeneration, [
function (test, expect) {
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName, collectionOptions);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName, collectionOptions);
Meteor.subscribe('c-' + this.collectionName, expect());
}
}, function (test, expect) {
var coll = new Meteor.Collection(collectionName, collectionOptions);
var coll = new Meteor.Collection(this.collectionName, collectionOptions);
var docId;
coll.insert({d: new Date(1356152390004)}, expect(function (err, id) {
test.isFalse(err);
@@ -1292,23 +1325,24 @@ testAsyncMulti('mongo-livedata - document goes through a transform, ' + idGenera
return doc;
};
TRANSFORMS["seconds"] = seconds;
var collectionOptions = {
self.collectionOptions = {
idGeneration: idGeneration,
transform: seconds,
transformName: "seconds"
};
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName, collectionOptions);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName, collectionOptions);
Meteor.subscribe('c-' + this.collectionName, expect());
}
self.coll = new Meteor.Collection(collectionName, collectionOptions);
}, function (test, expect) {
var self = this;
self.coll = new Meteor.Collection(self.collectionName, self.collectionOptions);
var obs;
var expectAdd = expect(function (doc) {
test.equal(doc.seconds(), 50);
});
var expectRemove = expect (function (doc) {
var expectRemove = expect(function (doc) {
test.equal(doc.seconds(), 50);
obs.stop();
});
@@ -1357,12 +1391,14 @@ testAsyncMulti('mongo-livedata - transform sets _id if not present, ' + idGenera
transform: justId,
transformName: "justId"
};
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName, collectionOptions);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName, collectionOptions);
Meteor.subscribe('c-' + this.collectionName, expect());
}
self.coll = new Meteor.Collection(collectionName, collectionOptions);
}, function (test, expect) {
var self = this;
self.coll = new Meteor.Collection(this.collectionName, collectionOptions);
self.coll.insert({}, expect(function (err, id) {
test.isFalse(err);
test.isTrue(id);
@@ -1371,24 +1407,25 @@ testAsyncMulti('mongo-livedata - transform sets _id if not present, ' + idGenera
}
]);
var bin = EJSONTest.base64Decode(
"TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyBy" +
"ZWFzb24sIGJ1dCBieSB0aGlzIHNpbmd1bGFyIHBhc3Npb24gZnJv" +
"bSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgdGhl" +
"IG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaWdo" +
"dCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdl" +
"bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9y" +
"dCB2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4=");
testAsyncMulti('mongo-livedata - document with binary data, ' + idGeneration, [
function (test, expect) {
// XXX probably shouldn't use EJSON's private test symbols
var bin = EJSONTest.base64Decode(
"TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyBy" +
"ZWFzb24sIGJ1dCBieSB0aGlzIHNpbmd1bGFyIHBhc3Npb24gZnJv" +
"bSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgdGhl" +
"IG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaWdo" +
"dCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdl" +
"bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9y" +
"dCB2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4=");
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName, collectionOptions);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName, collectionOptions);
Meteor.subscribe('c-' + this.collectionName, expect());
}
var coll = new Meteor.Collection(collectionName, collectionOptions);
}, function (test, expect) {
var coll = new Meteor.Collection(this.collectionName, collectionOptions);
var docId;
coll.insert({b: bin}, expect(function (err, id) {
test.isFalse(err);
@@ -1405,13 +1442,13 @@ testAsyncMulti('mongo-livedata - document with binary data, ' + idGeneration, [
testAsyncMulti('mongo-livedata - document with a custom type, ' + idGeneration, [
function (test, expect) {
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName, collectionOptions);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName, collectionOptions);
Meteor.subscribe('c-' + this.collectionName, expect());
}
var coll = new Meteor.Collection(collectionName, collectionOptions);
}, function (test, expect) {
var coll = new Meteor.Collection(this.collectionName, collectionOptions);
var docId;
// Dog is implemented at the top of the file, outside of the idGeneration
// loop (so that we only call EJSON.addType once).
@@ -1727,6 +1764,8 @@ var asyncUpsertTestName = function (useNetwork, useDirectCollection,
// collections, and run the tests for both the Meteor.Collection and the
// LocalCollection. On the server, we test mongo-backed collections, for both
// the Meteor.Collection and the MongoConnection.
//
// XXX Rewrite with testAsyncMulti, that would simplify things a lot!
_.each(Meteor.isServer ? [false] : [true, false], function (useNetwork) {
_.each(useNetwork ? [false] : [true, false], function (useDirectCollection) {
_.each([true, false], function (useUpdate) {
@@ -1737,10 +1776,16 @@ _.each(Meteor.isServer ? [false] : [true, false], function (useNetwork) {
(useUpdate ? "_update_" : "") +
(useNetwork ? "_network_" : "") +
(useDirectCollection ? "_direct_" : "");
var next0 = function () {
// Test starts here.
upsert(coll, useUpdate, {_id: 'foo'}, {_id: 'foo', foo: 'bar'}, next1);
};
if (useNetwork) {
Meteor.call("createInsecureCollection", collName, collectionOptions);
coll = new Meteor.Collection(collName, collectionOptions);
Meteor.subscribe("c-" + collName);
Meteor.subscribe("c-" + collName, next0);
} else {
var opts = _.clone(collectionOptions);
if (Meteor.isClient)
@@ -1762,8 +1807,9 @@ _.each(Meteor.isServer ? [false] : [true, false], function (useNetwork) {
upsert(coll, useUpdate, {_id: 'foo'}, {foo: 'baz'}, next2);
};
// Test starts here.
upsert(coll, useUpdate, {_id: 'foo'}, {_id: 'foo', foo: 'bar'}, next1);
if (! useNetwork) {
next0();
}
var t1, t2, result2;
var next2 = function (err, result) {
@@ -1904,24 +1950,24 @@ if (Meteor.isClient) {
var collName = "livedata_upsert_collection_"+run;
Meteor.call("createInsecureCollection", collName, collectionOptions);
coll = new Meteor.Collection(collName, collectionOptions);
Meteor.subscribe("c-" + collName);
coll.insert({ _id: "foo" });
coll.insert({ _id: "bar" });
coll.update({ _id: "foo" }, { $set: { foo: 1 } }, { multi: true }, function (err, result) {
test.isFalse(err);
test.equal(result, 1);
coll.update({ _id: "foo" }, { _id: "foo", foo: 2 }, function (err, result) {
Meteor.subscribe("c-" + collName, function () {
coll.insert({ _id: "foo" });
coll.insert({ _id: "bar" });
coll.update({ _id: "foo" }, { $set: { foo: 1 } }, { multi: true }, function (err, result) {
test.isFalse(err);
test.equal(result, 1);
coll.update({ _id: "baz" }, { $set: { foo: 1 } }, function (err, result) {
coll.update({ _id: "foo" }, { _id: "foo", foo: 2 }, function (err, result) {
test.isFalse(err);
test.equal(result, 0);
coll.remove({ _id: "foo" }, function (err, result) {
test.equal(result, 1);
coll.remove({ _id: "baz" }, function (err, result) {
test.equal(result, 0);
onComplete();
test.equal(result, 1);
coll.update({ _id: "baz" }, { $set: { foo: 1 } }, function (err, result) {
test.isFalse(err);
test.equal(result, 0);
coll.remove({ _id: "foo" }, function (err, result) {
test.equal(result, 1);
coll.remove({ _id: "baz" }, function (err, result) {
test.equal(result, 0);
onComplete();
});
});
});
});
@@ -2104,17 +2150,18 @@ Tinytest.add('mongo-livedata - rewrite selector', function (test) {
testAsyncMulti('mongo-livedata - specified _id', [
function (test, expect) {
var collectionName = Random.id();
this.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName);
Meteor.subscribe('c-' + collectionName);
Meteor.call('createInsecureCollection', this.collectionName);
Meteor.subscribe('c-' + this.collectionName, expect());
}
}, function (test, expect) {
var expectError = expect(function (err, result) {
test.isTrue(err);
var doc = coll.findOne();
test.equal(doc.name, "foo");
});
var coll = new Meteor.Collection(collectionName);
var coll = new Meteor.Collection(this.collectionName);
coll.insert({_id: "foo", name: "foo"}, expect(function (err1, id) {
test.equal(id, "foo");
var doc = coll.findOne();
@@ -2125,13 +2172,197 @@ testAsyncMulti('mongo-livedata - specified _id', [
}
]);
// Consistent id generation tests
function collectionInsert (test, expect, coll, index) {
var clientSideId = coll.insert({name: "foo"}, expect(function (err1, id) {
test.equal(id, clientSideId);
var o = coll.findOne(id);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
function collectionUpsert (test, expect, coll, index) {
var upsertId = '123456' + index;
coll.upsert(upsertId, {$set: {name: "foo"}}, expect(function (err1, result) {
test.equal(result.insertedId, upsertId);
test.equal(result.numberAffected, 1);
var o = coll.findOne(upsertId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
function collectionUpsertExisting (test, expect, coll, index) {
var clientSideId = coll.insert({name: "foo"}, expect(function (err1, id) {
test.equal(id, clientSideId);
var o = coll.findOne(id);
test.isTrue(_.isObject(o));
// We're not testing sequencing/visibility rules here, so skip this check
// test.equal(o.name, 'foo');
}));
coll.upsert(clientSideId, {$set: {name: "bar"}}, expect(function (err1, result) {
test.equal(result.insertedId, clientSideId);
test.equal(result.numberAffected, 1);
var o = coll.findOne(clientSideId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'bar');
}));
};
function functionCallsInsert (test, expect, coll, index) {
Meteor.call("insertObjects", coll._name, {name: "foo"}, 1, expect(function (err1, ids) {
test.notEqual((INSERTED_IDS[coll._name] || []).length, 0);
var stubId = INSERTED_IDS[coll._name][index];
test.equal(ids.length, 1);
test.equal(ids[0], stubId);
var o = coll.findOne(stubId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
function functionCallsUpsert (test, expect, coll, index) {
var upsertId = '123456' + index;
Meteor.call("upsertObject", coll._name, upsertId, {$set:{name: "foo"}}, expect(function (err1, result) {
test.equal(result.insertedId, upsertId);
test.equal(result.numberAffected, 1);
var o = coll.findOne(upsertId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
function functionCallsUpsertExisting (test, expect, coll, index) {
var id = coll.insert({name: "foo"});
var o = coll.findOne(id);
test.notEqual(null, o);
test.equal(o.name, 'foo');
Meteor.call("upsertObject", coll._name, id, {$set:{name: "bar"}}, expect(function (err1, result) {
test.equal(result.numberAffected, 1);
test.equal(result.insertedId, undefined);
var o = coll.findOne(id);
test.isTrue(_.isObject(o));
test.equal(o.name, 'bar');
}));
};
function functionCalls3Inserts (test, expect, coll, index) {
Meteor.call("insertObjects", coll._name, {name: "foo"}, 3, expect(function (err1, ids) {
test.notEqual((INSERTED_IDS[coll._name] || []).length, 0);
test.equal(ids.length, 3);
for (var i = 0; i < 3; i++) {
var stubId = INSERTED_IDS[coll._name][(3 * index) + i];
test.equal(ids[i], stubId);
var o = coll.findOne(stubId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}
}));
};
function functionChainInsert (test, expect, coll, index) {
Meteor.call("doMeteorCall", "insertObjects", coll._name, {name: "foo"}, 1, expect(function (err1, ids) {
test.notEqual((INSERTED_IDS[coll._name] || []).length, 0);
var stubId = INSERTED_IDS[coll._name][index];
test.equal(ids.length, 1);
test.equal(ids[0], stubId);
var o = coll.findOne(stubId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
function functionChain2Insert (test, expect, coll, index) {
Meteor.call("doMeteorCall", "doMeteorCall", "insertObjects", coll._name, {name: "foo"}, 1, expect(function (err1, ids) {
test.notEqual((INSERTED_IDS[coll._name] || []).length, 0);
var stubId = INSERTED_IDS[coll._name][index];
test.equal(ids.length, 1);
test.equal(ids[0], stubId);
var o = coll.findOne(stubId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
function functionChain2Upsert (test, expect, coll, index) {
var upsertId = '123456' + index;
Meteor.call("doMeteorCall", "doMeteorCall", "upsertObject", coll._name, upsertId, {$set:{name: "foo"}}, expect(function (err1, result) {
test.equal(result.insertedId, upsertId);
test.equal(result.numberAffected, 1);
var o = coll.findOne(upsertId);
test.isTrue(_.isObject(o));
test.equal(o.name, 'foo');
}));
};
_.each( {collectionInsert: collectionInsert,
collectionUpsert: collectionUpsert,
functionCallsInsert: functionCallsInsert,
functionCallsUpsert: functionCallsUpsert,
functionCallsUpsertExisting: functionCallsUpsertExisting,
functionCalls3Insert: functionCalls3Inserts,
functionChainInsert: functionChainInsert,
functionChain2Insert: functionChain2Insert,
functionChain2Upsert: functionChain2Upsert}, function (fn, name) {
_.each( [1, 3], function (repetitions) {
_.each( [1, 3], function (collectionCount) {
_.each( ['STRING', 'MONGO'], function (idGeneration) {
testAsyncMulti('mongo-livedata - consistent _id generation ' + name + ', ' + repetitions + ' repetitions on ' + collectionCount + ' collections, idGeneration=' + idGeneration, [ function (test, expect) {
var collectionOptions = { idGeneration: idGeneration };
this.collections = _.times(collectionCount, function () {
var collectionName = "consistentid_" + Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName, collectionOptions);
Meteor.subscribe('c-' + collectionName, expect());
}
return (COLLECTIONS[collectionName] = new Meteor.Collection(collectionName, collectionOptions));
});
}, function (test, expect) {
// now run the actual test
for (var i = 0; i < repetitions; i++) {
for (var j = 0; j < collectionCount; j++) {
fn(test, expect, this.collections[j], i);
}
}
}]);
});
});
});
});
testAsyncMulti('mongo-livedata - empty string _id', [
function (test, expect) {
var self = this;
self.collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', self.collectionName);
Meteor.subscribe('c-' + self.collectionName);
Meteor.subscribe('c-' + self.collectionName, expect());
}
self.coll = new Meteor.Collection(self.collectionName);
try {
@@ -2326,7 +2557,7 @@ testAsyncMulti("mongo-livedata - update handles $push with $each correctly", [
var collectionName = Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName);
Meteor.subscribe('c-' + collectionName);
Meteor.subscribe('c-' + collectionName, expect());
}
self.collection = new Meteor.Collection(collectionName);
@@ -2579,7 +2810,7 @@ testAsyncMulti("mongo-livedata - oplog - update EJSON", [
var collectionName = "ejson" + Random.id();
if (Meteor.isClient) {
Meteor.call('createInsecureCollection', collectionName);
Meteor.subscribe('c-' + collectionName);
Meteor.subscribe('c-' + collectionName, expect());
}
self.collection = new Meteor.Collection(collectionName);
@@ -2676,3 +2907,21 @@ var waitUntilOplogCaughtUp = function () {
oplogHandle.waitUntilCaughtUp();
};
Meteor.isServer && Tinytest.add("mongo-livedata - cursor dedup stop", function (test) {
var coll = new Meteor.Collection(Random.id());
_.times(100, function () {
coll.insert({foo: 'baz'});
});
var handler = coll.find({}).observeChanges({
added: function (id) {
coll.update(id, {$set: {foo: 'bar'}});
}
});
handler.stop();
// Previously, this would print
// Exception in queued task: TypeError: Object.keys called on non-object
// Unfortunately, this test didn't fail before the bugfix, but it at least
// would print the error and no longer does.
// See https://github.com/meteor/meteor/issues/2070
});

6
packages/mongo-livedata/observe_multiplex.js
View File

@@ -131,6 +131,10 @@ _.extend(ObserveMultiplexer.prototype, {
_applyCallback: function (callbackName, args) {
var self = this;
self._queue.queueTask(function () {
// If we stopped in the meantime, do nothing.
if (!self._handles)
return;
// First, apply the change to the cache.
// XXX We could make applyChange callbacks promise not to hang on to any
// state from their arguments (assuming that their supplied callbacks
@@ -151,7 +155,7 @@ _.extend(ObserveMultiplexer.prototype, {
// use a handle that got removed, because removeHandle does not use the
// queue; thus, we iterate over an array of keys that we control.)
_.each(_.keys(self._handles), function (handleId) {
var handle = self._handles[handleId];
var handle = self._handles && self._handles[handleId];
if (!handle)
return;
var callback = handle['_' + callbackName];

77
packages/mongo-livedata/oplog_observe_driver.js
View File

@@ -151,6 +151,13 @@ OplogObserveDriver = function (options) {
}
));
// When Mongo fails over, we need to repoll the query, in case we processed an
// oplog entry that got rolled back.
self._stopHandles.push(self._mongoHandle._onFailover(finishIfNeedToPollQuery(
function () {
self._needToPollQuery();
})));
// Give _observeChanges a chance to add the new ObserveHandle to our
// multiplexer, so that the added calls get streamed.
Meteor.defer(finishIfNeedToPollQuery(function () {
@@ -424,7 +431,6 @@ _.extend(OplogObserveDriver.prototype, {
var thisGeneration = ++self._fetchGeneration;
self._needToFetch = new LocalCollection._IdMap;
var waiting = 0;
var anyError = null;
var fut = new Future;
// This loop is safe, because _currentlyFetching will not be updated
// during this loop (in fact, it is never mutated).
@@ -435,8 +441,15 @@ _.extend(OplogObserveDriver.prototype, {
finishIfNeedToPollQuery(function (err, doc) {
try {
if (err) {
if (!anyError)
anyError = err;
Meteor._debug("Got exception while fetching documents: " +
err);
// If we get an error from the fetcher (eg, trouble connecting
// to Mongo), let's just abandon the fetch phase altogether
// and fall back to polling. It's not like we're getting live
// updates anyway.
if (self._phase !== PHASE.QUERYING) {
self._needToPollQuery();
}
} else if (!self._stopped && self._phase === PHASE.FETCHING
&& self._fetchGeneration === thisGeneration) {
// We re-check the generation in case we've had an explicit
@@ -456,15 +469,15 @@ _.extend(OplogObserveDriver.prototype, {
}));
});
fut.wait();
// XXX do this even if we've switched to PHASE.QUERYING?
if (anyError)
throw anyError;
// Exit now if we've had a _pollQuery call (here or in another fiber).
if (self._phase === PHASE.QUERYING)
return;
self._currentlyFetching = null;
}
self._beSteady();
// We're done fetching, so we can be steady, unless we've had a _pollQuery
// call (here or in another fiber).
if (self._phase !== PHASE.QUERYING)
self._beSteady();
}));
},
_beSteady: function () {
@@ -599,22 +612,40 @@ _.extend(OplogObserveDriver.prototype, {
_runQuery: function () {
var self = this;
var newResults = new LocalCollection._IdMap;
var newBuffer = new LocalCollection._IdMap;
var newResults, newBuffer;
// Query 2x documents as the half excluded from the original query will go
// into unpublished buffer to reduce additional Mongo lookups in cases when
// documents are removed from the published set and need a replacement.
// XXX needs more thought on non-zero skip
// XXX 2 is a "magic number" meaning there is an extra chunk of docs for
// buffer if such is needed.
var cursor = self._cursorForQuery({ limit: self._limit * 2 });
cursor.forEach(function (doc, i) {
if (!self._limit || i < self._limit)
newResults.set(doc._id, doc);
else
newBuffer.set(doc._id, doc);
});
// This while loop is just to retry failures.
while (true) {
// If we've been stopped, we don't have to run anything any more.
if (self._stopped)
return;
newResults = new LocalCollection._IdMap;
newBuffer = new LocalCollection._IdMap;
// Query 2x documents as the half excluded from the original query will go
// into unpublished buffer to reduce additional Mongo lookups in cases
// when documents are removed from the published set and need a
// replacement.
// XXX needs more thought on non-zero skip
// XXX 2 is a "magic number" meaning there is an extra chunk of docs for
// buffer if such is needed.
var cursor = self._cursorForQuery({ limit: self._limit * 2 });
try {
cursor.forEach(function (doc, i) {
if (!self._limit || i < self._limit)
newResults.set(doc._id, doc);
else
newBuffer.set(doc._id, doc);
});
break;
} catch (e) {
// During failover (eg) if we get an exception we should log and retry
// instead of crashing.
Meteor._debug("Got exception while polling query: " + e);
Meteor._sleepForMs(100);
}
}
self._publishNewResults(newResults, newBuffer);
},
@@ -846,4 +877,4 @@ var modifierCanBeDirectlyApplied = function (modifier) {
});
};
MongoTest.OplogObserveDriver = OplogObserveDriver;
MongoInternals.OplogObserveDriver = OplogObserveDriver;

27
packages/mongo-livedata/oplog_tailing.js
View File

@@ -101,12 +101,25 @@ _.extend(OplogHandle.prototype, {
// be ready.
self._readyFuture.wait();
// We need to make the selector at least as restrictive as the actual
// tailing selector (ie, we need to specify the DB name) or else we might
// find a TS that won't show up in the actual tail stream.
var lastEntry = self._oplogLastEntryConnection.findOne(
OPLOG_COLLECTION, self._baseOplogSelector,
{fields: {ts: 1}, sort: {$natural: -1}});
while (!self._stopped) {
// We need to make the selector at least as restrictive as the actual
// tailing selector (ie, we need to specify the DB name) or else we might
// find a TS that won't show up in the actual tail stream.
try {
var lastEntry = self._oplogLastEntryConnection.findOne(
OPLOG_COLLECTION, self._baseOplogSelector,
{fields: {ts: 1}, sort: {$natural: -1}});
break;
} catch (e) {
// During failover (eg) if we get an exception we should log and retry
// instead of crashing.
Meteor._debug("Got exception while reading last entry: " + e);
Meteor._sleepForMs(100);
}
}
if (self._stopped)
return;
if (!lastEntry) {
// Really, nothing in the oplog? Well, we've processed everything.
@@ -166,7 +179,7 @@ _.extend(OplogHandle.prototype, {
// Find the last oplog entry.
var lastOplogEntry = self._oplogLastEntryConnection.findOne(
OPLOG_COLLECTION, {}, {sort: {$natural: -1}});
OPLOG_COLLECTION, {}, {sort: {$natural: -1}, fields: {ts: 1}});
var oplogSelector = _.clone(self._baseOplogSelector);
if (lastOplogEntry) {

6
packages/mongo-livedata/package.js
View File

@@ -13,9 +13,7 @@ Package.describe({
});
Npm.depends({
// 1.3.19, plus a patch to add oplogReplay flag:
// https://github.com/mongodb/node-mongodb-native/pull/1108
mongodb: "https://github.com/meteor/node-mongodb-native/tarball/779bbac916a751f305d84c727a6cc7dfddab7924"
mongodb: "1.4.1"
});
Package.on_use(function (api) {
@@ -46,6 +44,8 @@ Package.on_use(function (api) {
// If the facts package is loaded, publish some statistics.
api.use('facts', 'server', {weak: true});
api.use('callback-hook', 'server');
// Stuff that should be exposed via a real API, but we haven't yet.
api.export('MongoInternals', 'server');
// For tests only.

0
packages/html5-tokenizer/.gitignore → packages/oauth-encryption/.gitignore vendored
View File

74
packages/oauth-encryption/README.md Normal file
View File

@@ -0,0 +1,74 @@
# oauth-encryption
Encrypts sensitive login secrets stored in the database such as a
login service's application secret key and users' access tokens.
## Generating a Key
The encryption key is 16 bytes, encoded in base64.
To generate a key:
$ ~/.meteor/tools/latest/bin/node -e 'console.log(require("crypto").randomBytes(16).toString("base64"))'
## Using oauth-encryption with accounts
On the server only, use the `oauthSecretKey` option to `Accounts.config`:
Accounts.config({oauthSecretKey: "onsqJ+1e4iGFlV0nhZYobg=="});
This call to `Accounts.config` should be made at load time (place at
the top level of your source file), not called from inside of a
`Meteor.startup` block.
To avoid storing the secret key in your application's source code, you
can use [`Meteor.settings`](http://docs.meteor.com/#meteor_settings):
Accounts.config({oauthSecretKey: Meteor.settings.oauthSecretKey});
## Migrating unencrypted user tokens
This example for Twitter shows how existing unencrypted user tokens
can be encrypted. The query finds user documents which have a Twitter
access token but not the `algorithm` field which is created when the
token is encrypted. The relevant fields in the service data are then
encrypted.
Meteor.users.find({ $and: [
{ 'services.twitter.accessToken': {$exists: true} },
{ 'services.twitter.accessToken.algorithm': {$exists: false} }
] }).
forEach(function (userDoc) {
var set = {};
_.each(['accessToken', 'accessTokenSecret', 'refreshToken'], function (field) {
var plaintext = userDoc.services.twitter[field];
if (!_.isString(plaintext))
return;
set['services.twitter.' + field] = OAuthEncryption.seal(
userDoc.services.twitter[field],
userDoc._id
);
});
Meteor.users.update(userDoc._id, {$set: set});
});
## Using oauth-encryption without accounts
If you're using the oauth packages directly instead of through the
Meteor accounts packages, you can load the OAuth encryption key
directly using `OAuthEncryption.loadKey`:
OAuthEncryption.loadKey("onsqJ+1e4iGFlV0nhZYobg==");
If you call `retrieveCredential` (such as
`Twitter.retrieveCredential`) as part of your process, you'll find
when using oauth-encryption that the sensitive service data fields
will be encrypted.
You can decrypt them using `OAuth.openSecrets`:
var credentials = Twitter.retrieveCredential(token);
var serviceData = OAuth.openSecrets(credentials.serviceData);

146
packages/oauth-encryption/encrypt.js Normal file
View File

@@ -0,0 +1,146 @@
var crypto = Npm.require("crypto");
// XXX We hope to be able to use the `crypto` module exclusively when
// Node supports GCM in version 0.11.
var gcm = Npm.require("node-aes-gcm");
OAuthEncryption = {};
var gcmKey = null;
// Node leniently ignores non-base64 characters when parsing a base64
// string, but we want to provide a more informative error message if
// the developer doesn't use base64 encoding.
//
// Note that an empty string is valid base64 (denoting 0 bytes).
//
// Exported for the convenience of tests.
//
OAuthEncryption._isBase64 = function (str) {
return _.isString(str) && /^[A-Za-z0-9\+\/]*\={0,2}$/.test(str);
};
// Loads the OAuth secret key, which must be 16 bytes in length
// encoded in base64.
//
// The key may be `null` which reverts to having no key (mainly used
// by tests).
//
OAuthEncryption.loadKey = function (key) {
if (key === null) {
gcmKey = null;
return;
}
if (! OAuthEncryption._isBase64(key))
throw new Error("The OAuth encryption key must be encoded in base64");
var buf = new Buffer(key, "base64");
if (buf.length !== 16)
throw new Error("The OAuth encryption AES-128-GCM key must be 16 bytes in length");
gcmKey = buf;
};
// Encrypt `data`, which may be any EJSON-compatible object, using the
// previously loaded OAuth secret key.
//
// The `userId` argument is optional. The data is encrypted as { data:
// *, userId: * }. When the result of `seal` is passed to `open`, the
// same user id must be supplied, which prevents user specific
// credentials such as access tokens from being used by a different
// user.
//
// We would actually like the user id to be AAD (additional
// authenticated data), but the node crypto API does not currently have
// support for specifying AAD.
//
OAuthEncryption.seal = function (data, userId) {
if (! gcmKey) {
throw new Error("No OAuth encryption key loaded");
}
var plaintext = new Buffer(EJSON.stringify({
data: data,
userId: userId
}));
var iv = crypto.randomBytes(12);
var result = gcm.encrypt(gcmKey, iv, plaintext, new Buffer([]) /* aad */);
return {
iv: iv.toString("base64"),
ciphertext: result.ciphertext.toString("base64"),
algorithm: "aes-128-gcm",
authTag: result.auth_tag.toString("base64")
};
};
// Decrypt the passed ciphertext (as returned from `seal`) using the
// previously loaded OAuth secret key.
//
// `userId` must match the user id passed to `seal`: if the user id
// wasn't specified, it must not be specified here, if it was
// specified, it must be the same user id.
//
// To prevent an attacker from breaking the encryption key by
// observing the result of sending manipulated ciphertexts, `open`
// throws "decryption unsuccessful" on any error.
OAuthEncryption.open = function (ciphertext, userId) {
if (! gcmKey)
throw new Error("No OAuth encryption key loaded");
try {
if (ciphertext.algorithm !== "aes-128-gcm") {
throw new Error();
}
var result = gcm.decrypt(
gcmKey,
new Buffer(ciphertext.iv, "base64"),
new Buffer(ciphertext.ciphertext, "base64"),
new Buffer([]), /* aad */
new Buffer(ciphertext.authTag, "base64")
);
if (! result.auth_ok) {
throw new Error();
}
var err;
var data;
try {
data = EJSON.parse(result.plaintext.toString());
} catch (e) {
err = new Error();
}
if (data.userId !== userId) {
err = new Error();
}
if (err) {
throw err;
} else {
return data.data;
}
} catch (e) {
throw new Error("decryption failed");
}
};
OAuthEncryption.isSealed = function (maybeCipherText) {
return maybeCipherText &&
OAuthEncryption._isBase64(maybeCipherText.iv) &&
OAuthEncryption._isBase64(maybeCipherText.ciphertext) &&
OAuthEncryption._isBase64(maybeCipherText.authTag) &&
_.isString(maybeCipherText.algorithm);
};
OAuthEncryption.keyIsLoaded = function () {
return !! gcmKey;
};

150
packages/oauth-encryption/encrypt_tests.js Normal file
View File

@@ -0,0 +1,150 @@
Tinytest.add("oauth-encryption - loadKey", function (test) {
test.throws(
function () {
OAuthEncryption.loadKey("my encryption key");
},
"The OAuth encryption key must be encoded in base64"
);
test.throws(
function () {
OAuthEncryption.loadKey(new Buffer([1, 2, 3, 4, 5]).toString("base64"));
},
"The OAuth encryption AES-128-GCM key must be 16 bytes in length"
);
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
OAuthEncryption.loadKey(null);
});
Tinytest.add("oauth-encryption - seal", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var ciphertext = OAuthEncryption.seal({a: 1, b: 2});
test.isTrue(new Buffer(ciphertext.iv, "base64").length === 12);
test.isTrue(OAuthEncryption._isBase64(ciphertext.ciphertext));
test.isTrue(ciphertext.algorithm === "aes-128-gcm");
test.isTrue(OAuthEncryption._isBase64(ciphertext.authTag));
OAuthEncryption.loadKey(null);
});
Tinytest.add("oauth-encryption - open successful", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var userId = "rH6rNSWd2hBTfkwcc";
var ciphertext = OAuthEncryption.seal({a: 1, b: 2}, userId);
var decrypted = OAuthEncryption.open(ciphertext, userId);
test.equal(decrypted, {a: 1, b: 2});
OAuthEncryption.loadKey(null);
});
Tinytest.add("oauth-encryption - open with wrong key", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var userId = "rH6rNSWd2hBTfkwcc";
var ciphertext = OAuthEncryption.seal({a: 1, b: 2}, userId);
OAuthEncryption.loadKey(
new Buffer([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9]).
toString("base64")
);
test.throws(
function () {
OAuthEncryption.open(ciphertext, userId);
},
"decryption failed"
);
OAuthEncryption.loadKey(null);
});
Tinytest.add("oauth-encryption - open with wrong userId", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var userId = "rH6rNSWd2hBTfkwcc";
var ciphertext = OAuthEncryption.seal({a: 1, b: 2}, userId);
var differentUser = "3FPxY2mBNeBpigm86";
test.throws(
function () {
OAuthEncryption.open(ciphertext, differentUser);
},
"decryption failed"
);
OAuthEncryption.loadKey(null);
});
Tinytest.add("oauth-encryption - seal and open with no userId", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var ciphertext = OAuthEncryption.seal({a: 1, b: 2});
var decrypted = OAuthEncryption.open(ciphertext);
test.equal(decrypted, {a: 1, b: 2});
});
Tinytest.add("oauth-encryption - open modified ciphertext", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var ciphertext = OAuthEncryption.seal({a: 1, b: 2});
var b = new Buffer(ciphertext.ciphertext, "base64");
b[0] = b[0] ^ 1;
ciphertext.ciphertext = b.toString("base64");
test.throws(
function () {
OAuthEncryption.open(ciphertext);
},
"decryption failed"
);
});
Tinytest.add("oauth-encryption - isSealed", function (test) {
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
var userId = "rH6rNSWd2hBTfkwcc";
var ciphertext = OAuthEncryption.seal({a: 1, b: 2}, userId);
test.isTrue(OAuthEncryption.isSealed(ciphertext));
test.isFalse(OAuthEncryption.isSealed("abcdef"));
test.isFalse(OAuthEncryption.isSealed({a: 1, b: 2}));
OAuthEncryption.loadKey(null);
});
Tinytest.add("oauth-encryption - keyIsLoaded", function (test) {
OAuthEncryption.loadKey(null);
test.isFalse(OAuthEncryption.keyIsLoaded());
OAuthEncryption.loadKey(
new Buffer([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]).
toString("base64")
);
test.isTrue(OAuthEncryption.keyIsLoaded());
OAuthEncryption.loadKey(null);
});

17
packages/oauth-encryption/package.js Normal file
View File

@@ -0,0 +1,17 @@
// Uses the node-aes-gcm NPM module from the dev bundle (because
// binary modules aren't working yet).
Package.describe({
summary: "Encrypt account secrets stored in the database"
});
Package.on_use(function (api) {
api.export("OAuthEncryption", ["server"]);
api.add_files("encrypt.js", ["server"]);
});
Package.on_test(function (api) {
api.use("tinytest");
api.use("oauth-encryption");
api.add_files("encrypt_tests.js", ["server"]);
});

3
packages/oauth/deprecated.js Normal file
View File

@@ -0,0 +1,3 @@
// XXX COMPAT WITH 0.8.0
Oauth = OAuth;

Some files were not shown because too many files have changed in this diff Show More