Geoff Schmidt
5661a590f2
Move spiderable docs from docs.meteor.com to readme
2014-10-13 18:37:23 -07:00
Sashko Stubailo
36da5aecb4
Make all version numbers not have rc
2014-10-13 14:09:37 -07:00
Sashko Stubailo
698fbedb10
Bump all of the version numbers to rc.0
2014-10-07 17:10:48 -07:00
Sashko Stubailo
1e53f6b598
Bump all version numbers again after cherry-picking unipackage fix
2014-10-01 17:06:57 -07:00
Sashko Stubailo
0129c3f5ac
Bump all of the version numbers again, to republish with the unipackage.json fix
2014-10-01 15:43:05 -07:00
Sashko Stubailo
eecc272e4a
Bump every single package version number due to a change in compilation
2014-09-29 23:44:50 -07:00
Avital Oliver
c23408bdd7
versions for 0.9.1
2014-09-04 15:41:39 -07:00
Avital Oliver
864fffc9e9
Bump package versions in preparation for 0.9.1-rc0
2014-08-28 11:00:35 -07:00
Sashko Stubailo
41910019e1
Renamed Deps to Tracker in all packages
2014-08-27 20:05:22 -07:00
Emily Stark
b0e0ccb5a4
Bump package versions for 0.9.0.1
2014-08-27 15:16:49 -07:00
Emily Stark
1b639e5853
Bump package versions for 0.9.0.1-rc1
2014-08-27 14:28:03 -07:00
Justin SB
f1a7455d4c
Spiderable now uses a flag to determine when the initial JS has loaded
...
Other approaches seemed to be heuristics that broke down in edge cases:
a timeout wasn't guaranteed if your JS was slow; page.load in PhantomJS
wasn't firing reliably because of long polling.
2014-08-27 12:25:45 -07:00
David Glasser
563860c769
Version number bumps, this time for real
2014-08-05 19:03:16 -07:00
David Glasser
845286b925
Merge branch 'devel' into packaging
2014-08-05 13:13:27 -07:00
MaximDubrovin
6345d92f07
move phantom_script.js setInterval into page.open callback
...
Until I made this I always got only ```<head/>``` with it's content, ```<body>``` was empty.
It seems setInterval script was finished earlier then url content was loaded to the page. Maybe because I have subscriptions with response time lower then 100ms so they were ready very quickly — database server in the same data center.
http://phantomjs.org/api/webpage/method/open.html
2014-08-01 16:44:04 -07:00
David Glasser
996d347139
Merge branch 'devel' into packaging
...
Conflicts:
docs/client/api.html
docs/client/concepts.html
docs/client/introduction.html
meteor
packages/html-tools/package.js
packages/spacebars-common/package.js
packages/webapp/webapp_server.js
tools/bundler.js
2014-07-21 16:18:25 -07:00
David Glasser
39fc2ac12c
Move spiderable phantom script to an asset
2014-07-18 16:45:16 -07:00
ekatek
fb709c9563
remove versions files from core packages
2014-06-16 22:11:31 -07:00
David Glasser
c5e5305a3b
Merge branch 'devel' into packaging
...
Conflicts:
tools/bundler.js
tools/packages.js
2014-06-11 12:02:57 -07:00
Emily Stark
560ba659e1
Use ROOT_URL instead of Host header in spiderable
2014-05-21 20:40:40 -07:00
ekatek
beb0e59b9d
new version files from teh new solver? Also, package.js correct with tests
2014-05-09 17:36:25 -07:00
ekatek
031890d2c5
don't store buildtime deps in the catalog
2014-05-04 16:56:58 -07:00
ekatek
23ad149a78
better versions files
2014-04-25 15:06:45 -07:00
ekatek
10d2047d12
clean up
2014-04-25 15:01:20 -07:00
ekatek
c84c8d6097
alphabetize package version lock files
2014-04-24 18:18:21 -07:00
Geoff Schmidt
66b7253695
Take all packages to 1.0.0 (that aren't already there)
2014-03-07 12:00:20 -08:00
David Glasser
06267daa93
Comment: we only partially do AJAX crawling spec
...
Fixes #487
2014-03-03 20:55:37 -08:00
Nick Martin
fbfcb4859b
Increase buffer size for phantomjs in spiderable. Our docs were too big.
2013-10-18 13:28:25 -07:00
Nick Martin
57c87a403d
OCD
2013-10-08 20:03:02 -07:00
Nick Martin
30b08fc012
Export Spiderable symbol, so people can override user agents. Also add twitterbot to agents. Fixes #1300 .
2013-08-19 19:51:01 -07:00
Geoff Schmidt
3d1c09794f
Comprehensive namespace cleanup.
2013-07-25 18:54:40 -07:00
David Glasser
053db8e9e9
Begin to refactor webapp package.
...
Move stuff from __meteor_bootstrap__ to WebApp exported symbol.
Note that the WebApp API continues to be subject to change.
2013-07-02 16:15:12 -07:00
David Glasser
2d827e74fc
Factor webapp-specific stuff (and keepalive) out of boot.js into webapp package.
...
Replace type: "bare" with type: "server" in attributes.json. This now just
means "only make one server process" rather than controlling the boot script.
Programs that are type: "traditional" that want to be a webapp now need to
explicitly depend on the webapp package.
2013-05-13 11:58:58 -07:00
Naomi Seyfer
3aa28e54ab
some changes necessary to make various packages work together
2013-05-13 11:58:57 -07:00
David Glasser
8dbee56e77
Add .gitignore to package directories to match linker branch.
...
This way switching between linker and devel doesn't leave you with lots of junk
in git status.
2013-05-13 10:57:37 -07:00
David Glasser
308395da46
Spiderable.userAgentRegExps can now be modified to change what user agents are
...
treated as spiders by the spiderable package.
2013-04-09 16:03:57 -07:00
David Glasser
5a5204e3a4
Remove closures around package files. (hint: git blame -w)
...
The bundler now adds closures around client files and the server adds closures
around server files.
2013-03-19 15:13:53 -07:00
Avital Oliver
78ca755d3d
Npm.require unified requireNpm and __meteor_bootstrap__.require
2013-03-19 15:11:04 -07:00
David Glasser
70f1412c0a
Avoid crash on spiderable errors. Fixes #832 .
2013-03-18 10:22:37 -07:00
David Greenspan
975454d4e4
Merge branch 'devel' into deps-radical
2013-03-04 13:46:50 -08:00
Morten Henriksen
a60f4eeddd
Fixed #721
...
Testing for bots should be case insensitive - facebook bots are not all
lowercase - they adapted CamelCase on some servers. I've added the
linkedin bot just for the sake of it.
2013-02-28 14:37:23 -08:00
David Greenspan
f6aabffab3
port all packages to new Deps (tests pass)
2013-02-25 17:52:51 -08:00
David Glasser
aec2e586d8
spiderable: Correctly kill phantomjs after timeout. Fixes #571 .
...
The previous version only killed the wrapper bash script, not the child. We now
work around the broken (socketpair) stdin presented to Node's child processes by
using a bash heredoc rather than "cat |"; this means we can use exec, which only
gives us one process to clean up.
While we're at it, simplify code by switching from child_process.spawn to
child_process.execFile, which does the work of capturing stdout and setting a
timeout for us automatically.
2013-01-08 22:51:40 -08:00
Nick Martin
cb0d7238a1
Convert to use regexps for user agents, so we can match multiple Facebook agents.
2012-10-29 19:14:22 -07:00
ayalgelles
2b72391ccf
using a list of agents to serve
2012-10-29 19:14:22 -07:00
ayalgelles
40010607af
serve facebookexternalhit user agent with spiderable
2012-10-29 19:14:22 -07:00
David Glasser
6c6fa5a6b1
Fix race condition in spiderable.
...
If the "check to see if page is ready" interval ran when "Meteor" is defined but
"Meteor.status" is not yet defined, an error message about trying to call an
undefined function would be dumped into the output source before the page
itself.
This can be easily reproduced by changing the timer interval from 100 to 1.
2012-09-24 10:41:14 -07:00
David Glasser
e27304c999
Work around Node 0.8 brokenness with using /dev/stdin in subprocesses.
2012-09-19 13:50:06 -07:00
Nick Martin
0397945fd8
Add hack in livedata_connection to support knowing when all data is received on all connections.
2012-08-03 22:00:25 -07:00
Nick Martin
07c703270c
Use subscription readiness instead of timeout.
...
This only looks at default_connection, though, which isn't good. May need to patch livedata.
2012-08-03 20:59:38 -07:00