20 Commits

Author SHA1 Message Date
David Glasser
5a5204e3a4 Remove closures around package files. (hint: git blame -w)
The bundler now adds closures around client files and the server adds closures
around server files.
2013-03-19 15:13:53 -07:00
Avital Oliver
78ca755d3d Npm.require unified requireNpm and __meteor_bootstrap__.require 2013-03-19 15:11:04 -07:00
David Glasser
70f1412c0a Avoid crash on spiderable errors. Fixes #832. 2013-03-18 10:22:37 -07:00
David Greenspan
975454d4e4 Merge branch 'devel' into deps-radical 2013-03-04 13:46:50 -08:00
Morten Henriksen
a60f4eeddd Fixed #721
Testing for bots should be case insensitive - facebook bots are not all
lowercase - they adapted CamelCase on some servers. I've added the
linkedin bot just for the sake of it.
2013-02-28 14:37:23 -08:00
David Greenspan
f6aabffab3 port all packages to new Deps (tests pass) 2013-02-25 17:52:51 -08:00
David Glasser
aec2e586d8 spiderable: Correctly kill phantomjs after timeout. Fixes #571.
The previous version only killed the wrapper bash script, not the child. We now
work around the broken (socketpair) stdin presented to Node's child processes by
using a bash heredoc rather than "cat |"; this means we can use exec, which only
gives us one process to clean up.

While we're at it, simplify code by switching from child_process.spawn to
child_process.execFile, which does the work of capturing stdout and setting a
timeout for us automatically.
2013-01-08 22:51:40 -08:00
Nick Martin
cb0d7238a1 Convert to use regexps for user agents, so we can match multiple Facebook agents. 2012-10-29 19:14:22 -07:00
ayalgelles
2b72391ccf using a list of agents to serve 2012-10-29 19:14:22 -07:00
ayalgelles
40010607af serve facebookexternalhit user agent with spiderable 2012-10-29 19:14:22 -07:00
David Glasser
6c6fa5a6b1 Fix race condition in spiderable.
If the "check to see if page is ready" interval ran when "Meteor" is defined but
"Meteor.status" is not yet defined, an error message about trying to call an
undefined function would be dumped into the output source before the page
itself.

This can be easily reproduced by changing the timer interval from 100 to 1.
2012-09-24 10:41:14 -07:00
David Glasser
e27304c999 Work around Node 0.8 brokenness with using /dev/stdin in subprocesses. 2012-09-19 13:50:06 -07:00
Nick Martin
0397945fd8 Add hack in livedata_connection to support knowing when all data is received on all connections. 2012-08-03 22:00:25 -07:00
Nick Martin
07c703270c Use subscription readiness instead of timeout.
This only looks at default_connection, though, which isn't good. May need to patch livedata.
2012-08-03 20:59:38 -07:00
Nick Martin
24c2fe9d7f Add timeout so requests can't take forever. 2012-08-03 19:06:14 -07:00
Nick Martin
ee7fc09c16 Handle ? in query string better. 2012-08-03 18:12:59 -07:00
Nick Martin
81cb0239f0 Check for errors from phantomjs. 2012-08-03 18:11:25 -07:00
Nick Martin
7632dc0386 Avoid writing a tempfile by using '/dev/stdin'. Hacky, but functional. 2012-08-02 23:06:41 -07:00
Nick Martin
0d27fa40f2 Fix encoding. 2012-08-02 22:39:28 -07:00
Nick Martin
dc34cff990 add spiderable package 2012-08-02 22:21:13 -07:00