Commit Graph

7374 Commits

Author SHA1 Message Date
Matt Lee
ee89b8ea4d Scraper: Convert imgur gif urls to gifv for scraping 2016-03-28 17:50:57 -04:00
Matt Lee
2dac6a26f1 Scraper: Add YouTube scraper 2016-03-28 17:50:57 -04:00
Jack Ward
239649429b Allow resize property in CSS filter
A CSS property supported by most browsers.
See https://drafts.csswg.org/css-ui-3/#resize
2016-03-28 12:02:53 -07:00
Lev Neiman
ae687a6d9f Vagrant: Update Ubuntu Image 2016-03-24 15:11:46 -07:00
Chris Slowe
512ec9dd6f Decode loid cookies on screenview events
Also stop extra-quoting them on the backend. Apparently, pylons already quotes cookies on the way out, leading to some cooke being doubly escaped. Correspondingly, aggressively decode loidcreated in JS to counteract any badly-formed cookies.
2016-03-20 21:51:15 -07:00
Chad Birch
b0c8f424d9 Scraper og:image check: handle no content attr
The thumbnail scraper is crashing on unusual og:image tags, when they
don't have a content= attribute. Example from the wild:

<meta property="og:image">

This just gets rid of the assumption that the attr will be there, so
that it can carry on finding a thumbnail as normal if an unusual tag is
encountered.
2016-03-17 16:36:50 -06:00
powerlanguage
c0ae5a25d8 Remove references to gold partners. 2016-03-17 13:12:47 -06:00
Chad Birch
c39ba57796 Lumen API: don't send AUTHENTICATION_TOKEN header
Lumen is making some changes to the way they check for authentication
tokens in API requests. Instead of updating this header, we're just
going to remove it since it was already unnecessary to be sending (since
our auth token is being sent in the JSON data).
2016-03-17 12:50:58 -06:00
Chad Birch
754b4bff99 Thumbnail scraper: handle EXIF Orientation tags
This adds support for EXIF Orientation tags to the thumbnail scraper.
Information about the tag and meaning of its values from
http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html
2016-03-17 12:50:54 -06:00
Brian Simpson
97fca24689 builder: Add all sorts to SORT_OPERATOR_BY_NAME 2016-03-16 17:04:50 -07:00
Brian Simpson
16060950ca CommentOrderer: Add event to count sort usage 2016-03-16 17:04:43 -07:00
Brian Simpson
a2001e4a9e get_comment_order: Add counters and timers by number of comments
This will let us see how often different number of comments are viewed
and how much time it takes to generate the order. This will help us decide
how to set precomputed_comment_sort_min_comments.
2016-03-16 17:04:35 -07:00
Daniel Ellis
862a7e0fc5 wiki: Fix broken diffs on long line-lengths. 2016-03-15 15:59:00 -07:00
Daniel Ellis
08fa64ed25 oauth2: Add stats. 2016-03-15 15:32:35 -07:00
Brian Simpson
f60f362fde CommentOrderer: add the option to precompute the suggested sort 2016-03-15 15:26:13 -07:00
scarow
a0eaf1b317 Utm tags are removed from url 2016-03-15 15:11:43 -07:00
scarow
c393ffc488 Add trademark text when creating subreddit
Add trademark text and new description on create subreddit form, as per mockup
2016-03-15 15:11:43 -07:00
Brian Simpson
30be5f5dd0 get_comment_order: Add timers around reading cache vs full loading 2016-03-15 13:41:07 -07:00
Brian Simpson
d3e643a574 CommentOrderer.read_cache: don't try to filter MissingChildrenTuple
First pop it off the ordered list if it exists, then filter out
too deep and too many comments.
2016-03-15 13:41:01 -07:00
Daniel Ellis
64dbfd30d0 comments: Check comment age against subreddit archive age. 2016-03-15 11:08:50 -07:00
Brian Simpson
b1e616b0fd commentstree_q: Process up to 200 items at a time
When precomputing comment sorts the queues can back up. Hopefully this will
let them process faster.
2016-03-14 15:52:02 -07:00
Brian Simpson
d2db50e07f GET_morechildren: Make sure that link is a Link object 2016-03-14 11:31:07 -07:00
Neil Williams
79123b696c Revert "Fix URL generation in dev-mode debugger on HTTPS"
This reverts commit 1d99057d0bd6037d66ac2218fd80a9e58138ecd8.

There have been complaints of redirect loops etc. I'm going to debug
this further before recommitting.
2016-03-14 10:11:51 -07:00
Brian Simpson
4f229785ec write_comment_orders: Don't record step by step timings 2016-03-14 10:01:44 -07:00
scarow
0d993529ad Newsletter only shows on Listings pages
Issue: newsletterbar was showing up on unintended pages (namely, our 404 error page, and Contact Us page). The newsletter prompt should only show up on Listings pages.

Fix: show_newsletterbar now defaults to False, and build_listing in the listingcontroller explicitly sets show_newsletterbar=True on render_cls instantiation. Because the new default for show_newsletterbar is False, I was able to remove the places where that bool was explicitly set (all in pages.py)

fixup: use self.show_newsletterbar over getattr
2016-03-14 09:53:30 -07:00
Chris Slowe
4e045fd831 Experiments: add a decorator for whitelisting loid-based experiments
This will allow us to use loid-based experiments along side the pagecache,
and intelligently vary the cache accordingly.
2016-03-11 10:57:11 -08:00
Chris Slowe
178d7a2ae0 LoId: add loid_created to bucketing_event and ensure proper unquoting everywhere 2016-03-11 10:48:08 -08:00
Chris Slowe
2276f0988e Experiments: add A/B tests for logged out users with a loid.
* add global `g.enable_loggedout_experiments`
 * The new syntax available is `{"experiment": { "loggedout": true, ... }}`
 * Unify the code path to get whether the experiment is enabled with the one to get the experiment variant.
 * add additional test cases to `experiment_test.py` and organize `loid` mocking there.
 * Fixup mocking in `feature_test.py`

*NOTE*: this is a first pass, and still won't work:
 1. on cached pages if we turn the CDN back on
 2. on the *first* page impression by a user before they get a loid (though this case is handled).
2016-03-11 10:47:48 -08:00
Chris Slowe
53e522c51a Experiments: Users excluded from experiments should have feature.is_enabled(user) == False
This patch also breaks off experiment_test.py from feature_test.py for improved readability.
2016-03-11 10:34:26 -08:00
xiongchiamiov
fcbff995b1 Feature flags: add support for experiments
We've long wanted the ability to run A/B tests; they make it much easier to
test out potential behavior, and reduce the self-selection bias you get through
opt-in betas.  This commit adds A/B-type experiment support to the feature
flagging system.

Currently, only server-side checks and tests on logged-out users are directly
supported.  This is purely to scope down the feature enough to make it easier
to get a v1 out.
2016-03-11 10:31:41 -08:00
Chad Birch
5b4ccc9ed6 AutoMod: ensure comment on a post sends orangered
Normally, if a moderator leaves a top-level comment on a post and
distinguishes it, we send an orangered to the author of the post even if
they have "send replies to my inbox" disabled for that post. This is
done since a distinguished top-level comment is often used to give important
information to the author (such as why the post was removed).

However, since AutoMod distinguishes its comments "directly" by just
setting the attribute, this logic wasn't being checked for its comments,
and it hasn't been sending orangereds in cases where it should be. This
commit fixes that, and ensures that it'll always send one whenever it's
commenting on a post.
2016-03-10 17:11:56 -07:00
Chad Birch
86114aa0a6 Distinguish notification: reduce query updates
If a comment gets distinguished and hasn't already been sent to the
parent's author's inbox (due to them having replies disabled), we send a
notification. This is so that users see if a moderator leaves a comment
in reply to one of their posts (often as a reason for removing it).

However, when doing this, we were previously calling queries.new_comment
to send the notification. There are multiple other unnecessary updates
done by this, which can result in strange errors like the comment
showing up twice in the subreddit's comments listing.

This commit changes to just call queries.update_comment_notifications
directly, since that's the only step in new_comment that we actually
needed to be doing.
2016-03-10 17:11:54 -07:00
Chad Birch
b9aa26249f new_comment: rearrange mutator creation
This moves the creation of a CachedQueryMutator inside
update_comment_notifications, to simplify the process of calling it
directly.

Related to that, the CachedQueryMutator inside new_comment is
now only created when it's needed in the (rare) spam-related cases,
since it no longer needs to create one to be able to call
update_comment_notifications.
2016-03-10 17:11:51 -07:00
Chad Birch
3881b3b961 Image preview JSON: account for max ratio
If an image is more than 2x taller than wide, we crop it down to a 2x
ratio when generating the image previews. However, the calculation of
the different preview resolutions wasn't accounting for this maximum,
and was still using the original ratio. Because of this, for tall
images, the reported preview heights were incorrect.
2016-03-10 16:40:48 -07:00
Brian Simpson
b53d138b6f Make it possible to precompute comment order
The comment order will be computed and stored in permacache when
a comment is added or a comment is voted on. Then when the link's
comment listing is viewed it can be read from permacache.
2016-03-10 15:10:14 -08:00
Brian Simpson
21f3785a67 add_comments: Write scores before writing CommentTree
get_comment_scores expects scores to exist for all comments present
in the CommentTree. To ensure this is true, write the scores before
writing the CommentTree.
2016-03-10 15:09:12 -08:00
Brian Simpson
476ac39e45 add_comments: Handle rebuild and combine locks 2016-03-10 15:09:07 -08:00
Brian Simpson
af81459727 Update comment scores in comment_tree.add_comments
This combines score updating with comment tree updating
2016-03-10 15:08:20 -08:00
Kevin O'Connor
29cc2149f4 Only redirect OAuth authorize requests as a result of user action.
Redirecting to the client's `redirect_uri` for any error was an
open redirect. Only redirecting as a result of user action ensures
we're checking errors and only sending a resource owner to a new
page if they've seen the authorize page and have taken some action.

Thanks to /u/avlidienbrunn for reporting!
2016-03-10 13:40:29 -08:00
Kevin O'Connor
a3929831c7 Add additional OAuth error messages. 2016-03-10 13:38:20 -08:00
Chris Slowe
113fb36369 Testing: fix promote_test.py 2016-03-10 10:41:47 -08:00
Matt Lee
955623c6dd Fix broken media test 2016-03-10 10:39:50 -08:00
Chris Slowe
b128f3814c Testing: fix oauth2 new-attr tracking 2016-03-10 10:38:21 -08:00
Chris Slowe
d6f810a3a4 Events: start storing oauth client name and app_type. 2016-03-10 10:29:14 -08:00
Daniel Ellis
3760877b26 mobile: Change gild message on first mobile login. 2016-03-10 10:10:02 -08:00
Neil Williams
158dd2820d Fix URL generation in dev-mode debugger on HTTPS
The interactive debugger was previously generating HTTP URLs even when
on HTTPS. This is because it uses wsgi.url_scheme to determine which
scheme to use when generating URLs and because we're using the paste
server in development, this environment variable was not being set
appropriately. PasteDeploy's PrefixMiddleware will set the variable
correctly based on X-Forwarded-Scheme / X-Forwarded-Proto which we
were already sending from haproxy.
2016-03-09 16:06:33 -08:00
Daniel Ellis
e86333e986 mobile: Gild the first login on an official mobile app. 2016-03-09 15:50:46 -08:00
Chris Slowe
36675c811e Installer: install-reddit.sh needs to pull down install.cfg. 2016-03-08 17:48:05 -08:00
Brian Simpson
cf819333a7 Fix oembed ForbiddenError 2016-03-08 16:00:57 -08:00
Jack Lawson
922c9d7486 Load related_subreddits from thing.site in sub/about/edit json 2016-03-08 14:19:22 -08:00