The base LiveList set up is similar to LiveConfig; it sets a watch and
will get notified by ZooKeeper when the data changes. Sometimes we don't
need data very frequently, if at all, and would rather eschew the watch
for a pull based model (asking ZK for the data only when needed.) This
is particularly helpful if we're doing write-only changes to a
datastructure read only in one place (such as a queue processor).
This buys us two things: one fewer memcache roundtrip per request to
determine if the IP is throttled or not, and the ability to use CIDR for
specifying blocks.
Google's ipaddress-py library as currently being modified (6f231f50aace)
for inclusion in the Python stdlib is included here to help us
manipulate CIDR ranges etc.
The reddit PPA version of Cassandra no longer auto-starts on install
because that was really obnoxious in production. We'll have to make
sure to start the service after installing it.
Previously, it would treat anything that wasn't (case insensitively)
equal to "true" as False. The new configparser will ensure that the
value is either "true" or "false" and not "slkdjfljksdf", reducing the
risk of accidental False by typo.
The dynamic configuration system has two components: the app, which
reads, and the writer script. The latter is meant for use by humans, and
converts a [live_config] section of the INI file into JSON for storage
in ZooKeeper. The app will read this data on startup and place a watch
on the node to be notified, by ZooKeeper, of changes. This means that
running the writer script with new data will automatically propagate the
changes to every app very quickly, without restart.
The writer script relies on a human-entered password to authenticate
with ZooKeeper. The reddit app uses a different set of credentials
(specified in the INI file) to obtain read-only access to the
configuration data.
Also adds a new "live_config" spec to reddit and plugins. This spec is
parsed at write-time only and the parsed values are stored as JSON in
ZooKeeper.
The first phase loads the entry points and instantiates plugin objects.
This phase is useful for inspecting plugin metadata without having a
full environment configured (routing etc.). This phase is moved earlier
in the init process, to the first moment after the configuration has
been parsed.
The second phase does the heavy work of making individual plugins inject
themselves into the otherwise ready reddit environment. This phase stays
where plugin loading took place before (after global setup).
Uses both transaction id and campaign id to uniquely identify a bid. This
fixes a bug where querying for one bid in some cases caused a multiple rows
returned exception - a side effect of the campaign migration, which left both
old and new bids in the db.
This change also includes the campaign id when attempting to recreate a lost
bid record because... bid records are supposed to have campaign ids. I'm not
sure what was happening with the bid records that didn't.
This check was raising a value error if run on the 31st against an expiration
date in a month that only had 30 days. Now we pick the 1st day of the month,
which is guaranteed to exist.
Also uses the correct timezone.
There have been some None object has no date() attribute errors in the logs.
Not sure how one of start/end are being set to None and not the other, but
went ahead and moved the call to date() inside the conditional that checks
whether both exist to make sure the correct error message gets set.
Makes get_schedule return a list of errored campaigns instead of just logging
them so they can be handled by the calling function.
Updates make_daily_promotions to first launch as many campaigns as possible,
then raise an exception with a list of errors at the end. The exception is
useful for triggering alerts.
See https://github.com/reddit/reddit-public/issues/570
When getting a list of user subreddits, we'll check if they're
subscribed to any automatic subreddits and remove them from the mix
before doing the random selection. Then they'll be added back in. This
is important to ensure that maintenance announcements aren't missed by
users because the announcement wasn't in their subset of subscriptions.
On reddit.com the automatic subreddits are /r/blog and /r/announcements.
Some evil people are using the toolbar URLs to spam their links and then
framebusting to ditch the toolbar. This patch makes it so that if the
framed URL is on a shame-banned domain, the framing will abort and
return a 404.
This will allow us to collect information about how users found
subreddits that they subscribe to. We can use this information to test
the effectiveness of new ways of discovering and subscribing to
subreddits.
Specifically, when the subscribe button is clicked, for the current page
and previous page: the URL, referrer URL, and the type of UI element
clicked are sent. We'll use this to answer questions like:
* "did clicking on gizmo A lead to users subscribing to subreddit B?"
* "why did we see a spike in subscriptions to subreddit X today?"
