Adding new Facebook user-agent to spiderable

According to Facebook
(https://developers.facebook.com/docs/sharing/webmasters/crawler), they
have a new (May 2014) crawler named Facebot. Spiderable doesn't
currently work for Facebot. It should.

PS. I shed a tear along with @stuabilo that this is even a thing.
This commit is contained in:
Alexey Komissarouk
2015-10-23 06:25:16 +09:00
parent dd2f166fe9
commit 9b437b1646

View File

@@ -14,6 +14,7 @@ var urlParser = Npm.require('url');
// here. I shed a silent tear.
Spiderable.userAgentRegExps = [
/^facebookexternalhit/i,
/^Facebot/,
/^linkedinbot/i,
/^twitterbot/i,
/^slackbot-linkexpanding/i