Memoize creates very large objects that take up lots of space
and has weird behaviour when it gets evicted. We get some isolation
and monitoring by moving it to its own pool. Since it is sensitive
to split-brain stuff, we start out here by dual-writing to the new
pool.
If the relevant configuration options are left blank in the ini file, we
will pass None to boto for credentials allowing it to look in other
places, including the AWS instance metadata used by Roles for EC2.
The paste-supplied gzip middleware is primarily for testing and is
insufficient in that it:
1) is unable to require a minimum size for gzipping
2) does not correctly add the "Vary: Accept-Encoding" header
it is also somewhat difficult to add those features given its structure.
Much new code expects the query cache to always be present, especially
for stuff that is 100% Cassandra based and doesn't even have a Postgres
query to back it. These options are just cluttering up our code since
they're useless now.
This means that stylesheets can have all the advantages of other static
files, such as not having session cookies in the request. In addition,
it also means that subreddit objects are drastically smaller in memcache
which saves internal bandwidth and increases cache capacity.
If a specific link has ridiculous traffic, we can inform the
apps (via ZK live config) that votes on it should be sent to a
separate, dedicated, queue to avoid overwhelming the rest of the
system.
Note that pylons.wsgiapp uses pylons.i18n's get_lang()
and friends, instead of our modified versions in
r2.lib.translation. This means that setting 'lang' in the
conf causes it to try and 'help' us by looking up PO files
in r2/i18n. Changing the ini files to use 'site_lang' as the
key gets around that.
The dynamic configuration system has two components: the app, which
reads, and the writer script. The latter is meant for use by humans, and
converts a [live_config] section of the INI file into JSON for storage
in ZooKeeper. The app will read this data on startup and place a watch
on the node to be notified, by ZooKeeper, of changes. This means that
running the writer script with new data will automatically propagate the
changes to every app very quickly, without restart.
The writer script relies on a human-entered password to authenticate
with ZooKeeper. The reddit app uses a different set of credentials
(specified in the INI file) to obtain read-only access to the
configuration data.
Also adds a new "live_config" spec to reddit and plugins. This spec is
parsed at write-time only and the parsed values are stored as JSON in
ZooKeeper.
