github/reddit
1
1
Fork 0
mirror of https://github.com/reddit-archive/reddit.git synced 2026-04-05 03:00:15 -04:00
Code Issues Packages Projects Releases Wiki Activity

Also scrapes https pages.

Browse Source
This commit is contained in:
Edho Arief
2012-06-13 07:02:52 -07:00
committed by Neil Williams
parent 64d068529e
commit 13feb89274
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
r2/r2/lib/scraper.py
View File

@@ -96,7 +96,7 @@ def fetch_url(url, referer = None, retries = 1, dimension = False):
nothing = None if dimension else (None, None)
url = clean_url(url)
#just basic urls
if not url.startswith('http://'):
if not (url.startswith('http://') or url.startswith('https://')):
return nothing
while True:
try: