====
* Uninstall the python Cassandra package that we previously depended on (it has a namespace conflict with the new package we depend on). To find it:
$ python -c "import cassandra; print cassandra.__file__"
and rm -r the .egg directory after "site-packages/"
* This version relies on cython, so if "make" fails, you may have to install cython via your distro's package manager.
$ cd reddit/r2
$ python setup.py develop # possibly with "sudo" depending on your install
$ make
* Cassandra is now required for the caching layer. An example storage-conf.xml can be found in reddit/srv/cassandra. Make sure that the additional <Keyspace> items are included in your conf file.
* remove the query_queue_reader services if they are running. add new gen_time_listings.sh instead. Suggested cron:
0 */2 * * * $SCRIPTS/gen_time_listings.sh year '("month","year")'
*/5 * * * * $SCRIPTS/gen_time_listings.sh week '("day","week")'
* * * * * $SCRIPTS/gen_time_listings.sh hour '("hour",)'
where $SCRIPTS is the location of this script directory
Features and Bugfixes
====
* Mobile reddit:
* templates are in r2/templates of the form *.compact
* css is r2/public/static/css/compact.css
* beginning of a sass-based (http://sass-lang.com/) compact.scss
* reachable via .compact extension or from the "i" subdomain.
* Cassandra is now *required*, and votes are currently written out to both cassandra and postgres (part of an eventual migration).
* attempt to make the db connection code a little smarter.
* A dropped DB connection will mark the connection dead and randomly attempt to reconnect.
* A dropped db connection on start will permanently mark the connection as dead.
* Calculate the time-filtered top/controversy listings using mapreduce instead of prec_links (new cron job in reddit/scripts)
* allow default user/pass for database to be specified with '*' to fallback on db_user and db_pass in the INI file
* Search feedback buttons
* make deleted comments not show up in your inbox.
* move last_visited into cassandra
* Swallow rare, race-conditiony POST_save/hide/subscribe problems
* Apparently we haven't been breaking properly for the past few weeks.
* The error pages have been broken for the past week.
* Fix /r/all+all
* Provides about.json to the api for subreddits (it already works for users)
* Make sure Accounts have data before checking their name and msgtime
* new bigger arrows for mobile site and a the alien is fingerless now
* Try to update the _fast_query_timestamps less often to alleviate some of the load it causes on the Accounts DB
* Swallow rare, race-conditiony POST_save/hide/subscribe problems
* bugfixes in unicode handling
* Nag nonparticpating jurors 20% of the time
* fix thumbnails for sponsored links
* non admins can't change the thumb of a live link
* the uploader updates the status properly to 'saved'
* Two new 404 aliens
* enable a comment page cache if the user isn't author of any of the comments and the page is a non-permalink. votes and friends are updated by javascript
Additions
* descriptive example.ini
* create "trusted_sponsor" who get auto-accepted
* split off url cache from the permacache, with major clean-up to app_globals
* Move is_banned_domain() call into POST_submit() so it can take the sanitized url
* Redirect /r/asdfasdfasdf to the search page, not the create page.
* updates to draw_load.py
* orangered envelope goes to unread, not all.
* Remove "enable whitelisting" checkbox, and switch to new failiens
* Put verification link on email prefs page
Bugfixes:
* split off a minimal api controller for onload calls (which don't care about c.site or c.user, making them cheaper)
* Assume admins want to keep their cookies forever just like everyone else
* /randomrising bugfix
* JSON error page was rendering in html. it now returns "{ error: ${code} }"
* Make the new targeted self-serve promotion the default
* add TheOatmeal's comic to our source. (http://www.theoatmeal.com)
* add targetting
* add rerunning or suplementing existing campaigns
* move all sponsored link listings to be precomputed
* deputy moderation
* /about/trials and /about/modqueue
* Have trials train the spam filter
* Allow admins to see all trials, site-wide, at /r/all/about/trials
* Allow trials to be ended while votes are still coming in when the jury opinion is near-unanimous
* offload scraping to embed.ly
* Use boto for s3cp instead of forking curl
Additions:
* migrated all graphs to use Flot
* Implement per-user suspiciousness quota and print over-quota trace
* Keep track of how many questionable links a user has outstanding at any given time
* add links to the FAQ and walkthru
* add a differnt thumbnail for self posts (thanks to licensplate)
* Add some visual tracking for comments trees
* Relabel: ban / unban => remove / approve
* Use big remove / approve buttons when the listing relates to moderation,
or when the item has reports or is on trial
* Merge unban, ignore, and a new approve this non-banned link into one
* Be a little smarter about the way we allow caching of subreddit stylesheets
* make cassandra's cache chain cache negative results (and yes that commit message is bigger than the patch)
* add All, friends, random, and mod to topbar
* add a local render precache (optional)
* better noimage.png from paradox460
* Add a 'legacy' mode to CMemcache that will allow us to use it as a drop-in replacement for testing purposes. Modifies python-memcached to use the same 'crc' hashing algorithm that pylibmc does, and removes a feature in python-memcached that we aren't using. Contributed by Joe Terranova <joeterranova@gmail.com>
* Everyone sees cake
* Added g.system_user
Bugfixes:
* improve validation of media embed, and remove a todo from link.py by putting all of the child div stuff in pages
* Feed/JSON updates
* Add selftext to RSS feeds
* Add subreddit names to Comments in JSON
* add is_self param to indicate self posts
* Don't shade the background of child comments of spam
* shorted the cache lifetime for AllSR to 1 min
* Tell everyone that we actually own the trademarks we claim to own (footer update)
* You no longer get an orange alien when one of your reddits sends an automated message to a user
* Cassandra
* Add new cassandra libraries that we'll need to setup.py
* Select Cassandra seeds at random rather than in order
* Bugfix in CassandraCache.delete and a faster permacache migration function
* Like other caches, CassandraCaches need to be able to take (and ignore) a 'time' parameter
* add Cassandra to the permacache chain
* beginning of jury duty (later called deputy moderation)
Additions:
* Make /r/friends much cheaper at the expense of sorting
* Add Jury.delete_old(), which removes Account-Trial relations > 3 days old
* Make the pretty_button() template function's callback optional, so that
actionless pretty-buttons can be used on the admin details page
* make .embed listings work for permalink pages (think of this as a first pass to getting blog comments working). Adds 'limit' and 'depth' parameter to permalink pages
* Added final redditheader.html pretty-button class
* new iframe ads; also make button.js static
* Usage sampling
Bugfixes:
* Stop adding batched time query recalculations to the queue at all except through the catch_up_batch_queries function
* Superflous comma might be causing IE7 to barf
* Change the byurl keys again, to fit in memcaches 251 character limit
* Indentation error causing non-sponsors to be able to get to the advert listing
* Move to a custom build of pylibmc that doesn't hold the GIL during blocking operations
* Convert some cache.gets to cache.get_multis, and implement our own thread-safety around pylibmc's client
* Make search caching a little smarter for time searches
* Make the ads not be cached for 30 seconds each, ie. more random
* fix deleted things on profile pages
* Discount 1.6.1
* Lines beginning with spaces are considered code. I don't know why markdown.py didn't trigger this.
* tables in mark down: why not?
* validation of resulting HTML vial libxml to prevent hax.
* private RSS and JSON feeds
* optional whitelists for subreddits
* Moderator messaging
Additions:
* destination sanitization to cut down on XSRF
* cosmetic fix to spam and reported listing
* make the rss feeds on messages useful
* /admin/errors
* Of the types of listings hitting the precomputers (top/controversy by hour/day/week/month/year), the ones over long periods of time don't change often. So we can try to run them at most once per day, and then merge in the day's listings.
* google analytics
* logging queue
* Created empty commentspanel.xml, errorpage.xml, login.xml
* add subreddit rules/info box to submit page
* add 'via' link on messages in moderator inbox
* add a show=all get parameter to link listings to optionally ignore hiding preferences.
* Raise edited timelimit to three mins
* Remove UI that makes it look like you can edit deleted selftexts
* Make it clearer to admins when a link is deleted
* Fix [S] leak on deleted comments
* Fix /user/[deleted] misrendering
* New house ads system
* updated so that minimalcontrollers actually can use the page cache.
* Added /admin/usage
Bugfixes:
* Reduce the number of results that we request from Solr and simplify that caching a bit
* Require a secret key to shut down app-servers
* Make get_title a little more resilient to malformed documents and slow remote servers
* Cause the SearchBuilder to only byID the results that it's going to render instead of all 1000
* Remove ability for an author to XSS himself
* fix spam listings and an xsrf
* More verbose VDestination
* Fixing the famous ?limit=0.1 error, and one last password-validation one
* distinguish deleted comments' and deleted links' error messages
* Don't allow ridiculously long log lines to widen the page
* Bug with HardCache.add() when existing key is expired
* Add adminbox next to domain
* Activate negative-result caching for HardCache chain
* begin migration to pylibmc:
* Add pylibmc to the list of required packages in preparation for the replacement of the memcached library
* Start using pylibmc for the rendercaches
* Tweak the computation of the normalized hot page to be a bit faster when the precomputer is available, by relying on the precomputer's internal permacached structure.
* Default to a SelfEmptyingCache for scripts run from `paster run'. Note that processes that run forever are still responsible for resetting their local-caches, but this can now be done with g.reset_caches()
* threaded messaging patch part 1: backend changes only. This will allow migrate.py to be run to move new onto inbox and will start tracking message trees for users.
* Specify some queries to run at most once per day
* Refactored safemarkdown() and added soup testing
Additions:
* Added _byID_rel()
* Made error messages more verbose for:
1. byID lookups of too-big thing_ids
2. memcache failures
3. Solr Nones
* Award._all_awards() now sorts by date
* Trophy.by_{account,award}() now cache properly
* new feedback page with helpful links
* Try to reduce the length of the query-queue by not adding known-long queries at all, rather than adding them and skipping them
* whitespace clean up
* simplify the 'why did my CC get denied' email checking.
* added missing translation strings and users now get PMs when they are added as translators
Bugfixes:
* Fix a bug in unsaving
* BeautifulSoup stopped hosting 3.0.7a, but 3.0.8 still uses the good parser
* Better search error handling
* Properly reset the cache-chains (incl. the hardcache; d'oh!) per-request
* Fix an attribute error on listings where some items have author_ids and some don't
* Bug when forcing recalculation of memoized functions
* the subreddit creation and edit form aren't dealing with errors properly
* buttons fix
* self service sponsored links (initial version with no targeting)
* credit card processing with authorize.net (and corresponding interaction code)
* hardcache -- for persistent cache keys with expiration
* Awards code
* messaging overhaul round 1. Added mark as unread as well as message sorting
Additions:
* add TedScraper and improve youtube scraper (ala tritelife); move traffic link on sponsored links and make them still visible when the promotion is over
* compressify all of our pngs, and set it up so thumbs, sr images, and the sprite get compressed every time they are generated
* rate limit logins (naive way)
* add a limit to the length of the moderator sidebox.
* add over18 thumbnail handling and NSFW label on nsfw content
* make NSFW label optional, and make the operation of the pref checkboxes sensible
* add the option (for moderators) to remove a subreddit from the default set
* Combine the queue handling code to be easily plumbed, and use it to precompute /comments
Bugfixes:
* trap duplicate vote error
* Fixed new mail mouseover on toolbar
* spammers can't send empty bodied emails any more (they will get rejected as they should).
===
- add confidence sorting to comments
* common values are precomputed for speedier response
* best is made the default sort on comment pages
- messages will now be delivered once one is moderator/contributor/banned
- UI updates to messaging page, including added show parent functionality to messages
- Remove the rate-limit on comments on your own self-posts
- Give users some leeway in editing their comments: don't show an edit star if the edit is within the first few minutes of a comment's lifetime
- Office Assistant will help users when they write to admins
Backend
===
- Replace the postgres-based query_queue with an AMQP based one
* Set up amqp queues for async tasks such as search updates and the scrapers
* service monitor updates, adding queue-tracking support
- Allow find_recent_broken_things to specify both from_time and to_time
- add a ini file parameter to disallow db writes (to create read-only reddit instances for crawlers)
New features
===
- self-serve advertisement:
* complete overhaul of sponsored link code
* functions for talking with authorize.net
* added pay domain and https support
* added ability to share traffic from sponsored links
* auto-reject promotions that are too old and unpaid for
- awards
- allow widget to have its links to have a target (in case it is iframed)
- automatic_reddits:
* Don't show automatic_reddits in the horizontal topbar
- Listing numbers are always in order with no gaps
- add support for sprites for common (r2.lib.contrib.nymph)
Admin
===
- added a takedown page for dealing with DMCA requests properly
* status code 404 on takedown pages
* JSON returns same string as in the explanation text
* nofollow on markdown in explanation
* title and image optional
- Added /c/(comment_id) for admins
- updates to JS to rate-limit voting, commenting, and anything else that could be just as easily done by a script-kiddie to cheat.
- make ad frame dynamic and add tracking pixel
- add the ability to add a sponsored banner to the rightbox of a reddit
- add the ability to show custom css on cnamed and/or non-cnamed versions of a reddit
- allow us to ignore reports from report-spammers.
Bugfixes
===
- Fix sorting of duplicate links (patch by Chromakode)
- fix traffic bug on main traffic page when it is the first of the month.
- toolbar redirects to comments page on self posts rather than generating the frame
- half-assed unicode handling in menus giving us bugs again. Switched to the whole-ass approach
- added Thing._byID36
- Support /help/foo/bar
* start showing embedded videos inside of an <iframe>. This allows us to
embed videos from external sites without having to trust their DOM security
Note that the media_object property on new links is now a dictionary,
but old ones will still be strings
potentially expensive queries
* don't look up default_subreddits in languages that we never store
* start respecting Subreddit.allow_top again
* fix assert syntax in cache.py
