7956 Commits

Author SHA1 Message Date
Brian Simpson
6675a93ca9 memoize: Stop writing to old key/old pool
NOTE: we were using the old key to get a lock on updating the
memoized result, but will now use the new key to get the lock. This
means that during the deploy we can have multiple processes updating
the memoized result. This is inefficient but should be fine.
2016-08-16 13:46:49 -07:00
Brian Simpson
a983dcc600 memoize: Read from new key/new pool 2016-08-16 13:46:40 -07:00
Brian Simpson
7b6c8d55b6 promote_test: Patch memoizecache 2016-08-16 13:46:29 -07:00
Brian Simpson
c7b0f9b14b memoize: Don't set module level memoizecache, just use the g attr 2016-08-16 13:45:41 -07:00
Brian Simpson
679a8c01d7 tests: Patch memoizecache 2016-08-16 13:45:23 -07:00
Brian Simpson
eeabd7eb0d memoize: Dual write to new mcrouter pool
Reads are served from the old key and pool. Writes go to both the
old key/old pool and the new key/new pool.
2016-08-16 13:44:56 -07:00
Brian Simpson
21fced6bfb popular_searches: Use stalecache for memoized results 2016-08-16 13:43:48 -07:00
Brian Simpson
6552628af4 RegistrationInfo: Use stalecache for memoized html chunk
The note on the registration page is stored in the wiki and is fronted
by memoizecache because it is loaded very frequently (it's on the login
page). Stale the cache lookup to reduce the number of requests that
fall through to memoizecache. It's currently requested ~1600/s.
2016-08-16 13:43:43 -07:00
Brian Simpson
9bb3167fd5 CommentTree.add_comments: Add orphaned comments to the tree
Adding them to the tree doesn't hurt anything--they still won't be
displayed until their parent comment is in the tree, but it does allow
for easier cleanup in case comments get processed out of order. Once their
parent is added to the tree everything will be consistent and fine.
2016-08-11 13:45:47 -07:00
Brian Simpson
4bcc145941 Refactor CommentTree to isolate locking and simplify interface 2016-08-11 13:45:42 -07:00
Brian Simpson
8a83763b02 add_comments: Don't raise InconsistentCommentTreeError
We don't handle the exception and fix the inconsistency, so it's better
to just log the error and continue.
2016-08-11 13:45:29 -07:00
Brian Simpson
a343cfe47b add_comments: Don't use mutation lock around write_comment_orders
The CommentTree is updated first, then it is read within write_comment_orders.
There's no issue of eventual consistency so the read will always get the
most up to date version of the CommentTree.
2016-08-11 13:45:23 -07:00
Brian Simpson
533eb60c92 Remove support for different comment tree versions 2016-08-11 13:44:02 -07:00
Brian Simpson
e844fbda73 Relation: Reduce _cache_ttl to 1 hour 2016-08-09 11:44:27 -07:00
Brian Simpson
6c4033b933 Relation: Reduce _rel_cache_ttl to 1 hr
_rel_cache is used to store (t1, t2, name) --> rel_id to make
Relation._fast_query work without falling through to a postgres
query. Reducing the TTL will let us recover from inconsistencies
more quickly, but we'll have to watch to make sure the more frequent
expirations don't put too much pressure on postgres.
2016-08-09 11:44:26 -07:00
Brian Simpson
ea226bbc10 Subreddit._by_name: Set 1 hour cache TTL
In the case of cache outages the values in cache can become out of
sync with reality. If a subreddit is created during an outage it will
be impossible to access it with Subreddit._by_name until the cache is
corrected. Setting a TTL will ensure this happens eventually.
2016-08-09 11:44:23 -07:00
Zean Tsoi
05b24d44da Specify plugins to be installed by default on Vagrant builds
Plugins must meet the following requirements in order to be installed
on a Vagrant build:

1. Be listed in `plugins` in the Vagrantfile
2. Be cloned to the designated install directory
3. Adhere to plugin naming conventions

This applies to all open-source plugins.
2016-08-04 13:13:14 -07:00
Kevin O'Connor
5a27265b29 Allow employees to distinguish messages. 2016-08-01 16:45:53 -07:00
Chris Slowe
f6ec25f330 Add request.parsed_agent, defined in r2.lib.utils.Agent 2016-07-28 14:47:06 -07:00
Brian Simpson
ab3d8e4304 Subreddit._by_name: Handle UnicodeEncodeError 2016-07-28 13:33:14 -07:00
Ricky Ramirez
21cddb73a4 Revert "commentstree_q: Add a min size for fastlane."
This reverts commit 32dece6abec82be66d034905720203da6ee12bf7.
I had an incorrect assumption that the number of updates to permacache
significantly contributes to the instability we were having with cassandra.
Other changes to comment tree processing have made cassandra more stable so
all this does is slow down updates unnecessarily.
2016-07-28 13:10:10 -07:00
Ricky Ramirez
9ca6616694 tdb_sql: Remove redundant indexes.
These indexes duplicate the primary key and just take up extra space and IOPS.
2016-07-28 13:10:08 -07:00
Chris Slowe
2a1d0f0ad2 fix evencollector testing after c1b1350 2016-07-28 12:59:25 -07:00
Katie Atkinson
0efc7b12f5 Extend httpagentparser to detect reddit clients (#5434) 2016-07-28 12:14:53 -07:00
Chris Slowe
6bf197a7df Refactor loid.py to more closely resemble the planned v1 format. 2016-07-28 12:10:21 -07:00
Neil Williams
17ff73f70e Install ZK on travis and wait for it to start
This should fix Travis and occasional failures in Drone where ZK isn't
present / wasn't started yet.
2016-07-28 12:07:21 -07:00
Chris Slowe
c6b766bf71 Add forward compatibility to v1 LoId spec (at least for the id) 2016-07-28 12:06:36 -07:00
ethaksus
b66145a980 Add CSS mask and mask-border properties
Allow use of the mask property to allow raster-based masking of elements and borders.
2016-07-28 11:48:35 -07:00
Chris Slowe
c1b1350aff Allow for event tests which test more than one event. 2016-07-28 11:31:10 -07:00
Chris Slowe
7672399c37 Add new epoch-millisecond functions to utils 2016-07-28 11:22:01 -07:00
Chad Birch
e8d8be8a8a Link.affects_karma_type: remove is_promoted_post
This check is related to the code for the upcoming "User Promoted Posts"
(see https://redd.it/4upf11), which hasn't been open-sourced yet. So
this attr doesn't exist yet, and isn't working. This should be removed
for now but can be added back in when the rest of the code is
open-sourced.
2016-07-27 11:22:43 -06:00
MelissaCole
8cc3f46ad9 Expose show_media_preview subreddit pref via API 2016-07-26 10:40:30 -07:00
MelissaCole
42215e12dd Add show_media to SubredditJsonTemplate 2016-07-26 10:40:21 -07:00
13steinj
12bde5e416 Flairlist.html: fix improper usage of utils.html:error_field
GET_flairlisting sets `USER_DOESNT_EXIST` error if the chosen user is nonexistent, however it's not rendering due to FlairPane checking for a `('USER_DOESNT_EXIST', 'name')` error by mistake. This commit resolves the issue.
2016-07-26 10:39:51 -07:00
Brian Simpson
ba3700e15e add_comments: Don't exit immediately on InconsistentCommentTreeError
add_comments is the consumer for commentstree_q and processed batches of
comments from multiple links. Previously on encountering an
InconsistentCommentTreeError we would return immediately and not process
the rest of the comments in the batch. This created additional inconsistent
CommentTrees.
2016-07-22 12:20:53 -07:00
Chad Birch
5d1c7252c5 Account: Add ability to disable karma (manually)
This will allow us to disable tracking karma for certain accounts
(specifically AutoModerator) to prevent the size of their Account
object from ballooning due to a huge number of karma attributes.
2016-07-19 10:38:41 -07:00
Chad Birch
521d40e328 Relabel "link karma" as "post karma"
This changes "link karma" to "post karma" in user-visible places:

* The sidebar on user profile pages
* The tooltip for hovering over the number in the header
* The check name in AutoModerator (but adds an alias for "link_karma" so
  old rules will continue to work)
2016-07-19 10:37:57 -07:00
Chad Birch
f55bdbdf9f Self posts: start giving karma
This commit starts giving and tracking karma for self-posts as well as
link posts. The karma is stored separately internally, but displayed and
otherwise treated as part of link karma. This will allow us to easily
revert this change and go back to the original link-only karma values if
desired.
2016-07-19 10:37:41 -07:00
Brian Simpson
0905c1a0ce Relation._fast_query: Ignore cache errors on sgm set
These sets are just used to populate the cache after a miss so they can
be ignored.
2016-07-19 09:17:55 -07:00
Brian Simpson
36bf9968f2 Relation._rel_cache: Use a 12 hour TTL 2016-07-18 13:54:05 -07:00
Brian Simpson
f1a60035ac example.ini: Remove unused caches 2016-07-18 13:54:05 -07:00
Brian Simpson
9c371f08e2 Use mcrouter for relcache and srmembercache 2016-07-18 13:54:05 -07:00
Brian Simpson
eaafb102c6 SRMember: set _cache_prefix and _rel_cache_prefix 2016-07-18 13:54:05 -07:00
Brian Simpson
5c237abef4 Relation: rename _fast_cache to _rel_cache 2016-07-18 13:54:05 -07:00
Brian Simpson
74fd3e280a Relation: Refactor _fast_cache_key to use a prefix 2016-07-18 13:54:05 -07:00
Brian Simpson
d85d88453c Trophy: Disable _fast_query
Trophy._fast_query is never used, and Trophy overrides the "name"
property so _fast_query wouldn't work if it were used. Also the "name"
is set to the string representation of the creation time so it contains
whitespace characters which are not allowed in cache keys.
2016-07-18 12:23:21 -07:00
Brian Simpson
34fd68c507 Relation: Allow optional disabling of _fast_query
Some relations are never queried this way so it's a waste to set the
cache values for them.
2016-07-18 12:23:16 -07:00
Brian Simpson
6c76b67b2c BaseController: better separate and document __before__ and __call__ 2016-07-18 12:23:10 -07:00
Nelson Pecora
cdfe26096f add omitscript arg to oembed endpoint 2016-07-18 09:44:06 -07:00
Brian Simpson
af0525815a Subreddit._by_name: Use set rather than add when populating cache
We cache a sentinal value for when the subreddit doesn't exist. Doing
an add won't overwrite that existing value when the subreddit is created.
2016-07-15 08:39:59 -07:00