Commit Graph

52 Commits

Author SHA1 Message Date
Neil Williams
bc289092d9 Remove unused app restart code obviated by uWSGI. 2011-09-21 11:49:45 -07:00
Neil Williams
5721e4683f Add validator that checks a secret key or admin cookie. 2011-09-15 15:29:29 -07:00
Neil Williams
7f7dc9476a Remove the RequestLogMiddleware.
It doesn't play nicely with the new uWSGI setup and we don't use it.
2011-08-24 18:02:08 -07:00
Andre D
a86261f352 Add a contextual description metatag for scrapers. 2011-08-08 13:13:30 -07:00
Sean Creeley
4b9fee7926 Use an API key for the embed.ly API.
Open sorcerers: you will need an API key for media scraping to work
now. Please visit http://embed.ly to sign up. It's free for <10K
URLs a month.
2011-08-02 13:43:42 -07:00
Logan Hanks
0dcddae22f Add user "flair" for subreddits.
Flair is a new relation between subreddits and accounts (stored in a manner
similar to but distinct from subreddit membership). This relation can have
a text field and a CSS class name associated with it (this data is actually
stored under the account). The flair data is then incorporated into any
mention of the account within the context of the subreddit (namely, on links
and comments submitted by the user).
2011-07-28 11:21:47 -07:00
Neil Williams
4758ac8d2e Make read-only mode slightly less frustrating.
Also includes some work on heavy-load mode which
isn't being used right now.
2011-07-22 22:04:55 -07:00
Brian Simpson
51bcbfbc97 Method _key_from_url() now returns case sensitive results for special domains (e.g. imgur.com, youtube.com). They are defined as case_sensitive_domains in the .ini file 2011-06-20 16:07:28 -07:00
Max Goodman
5e2f134b95 Make template_debug skip stripping whitespace. 2011-05-17 15:17:44 -07:00
gabe
92fdd422fb Added option to disable captcha verification 2011-05-16 14:53:20 -07:00
westmaas
e6178bedb1 Make default header image URL configurable. 2011-05-16 14:52:15 -07:00
gabe
63d518f4c9 Make min age for subreddit creation configurable. 2011-05-16 14:52:09 -07:00
westmaas
d411cd55a6 Added support for disabling adds via config file 2011-05-16 14:52:03 -07:00
Neil Williams
68a06c568b April 2011 Merge
* Show "this post is archived" message for things we won't allow votes on.
* Don't mark messages read when pulled via RSS
* Make compact (smartphone) interface respect user's toolbar
* preferences.
* Make mobile interface more friendly for kindles.
* Fix bug that caused comment tree corruption.
* Use cachebuster on traffic pixel for more accurate tracking.
* Make apps restart themselves after a configurable number of requests.
* Move to pycassa 1.0.8.
* Fix bug in calculations for "best" sort.
* Fixes for Firefox Mobile
* Add a global flag to disable editing of the wiki.
* Move the child-comment collapse button to the left.
* Updated list of disallows in robots.txt to save needless hits from
  crawlers.
* Fix vote_q by splitting it into vote_link_q and vote_comment_q.
* Fix bug where /reddits crashes due to 'promos' subreddit.
2011-04-18 16:30:48 -07:00
ketralnis
7fff900bbe February 2011 Merge
* Upgrade discount to 1.6.8
* Start reading Votes, Saves, and Hides from Cassandra (while still dual-writing all three)
* Some XSS fixes
* Significant Gold buying and gifting improvements
  - Move /api/ipn to /ipn
* Allow non-US countries to buy sponsored links
* Increase embed.ly scope
* redd.it support
* Allow postgres port number to be specified in ini file (this changes the format of the .ini file)
* Upgrade Cassandra to 0.7
  - Change g.urlcache to LinksByURL
  - Translate storage-conf.xml to cassandra.yaml
  - TTL support (and enable on Hides)
  - Move permacache keyspace to inside reddit keyspace
* The stalecache: a local memcached that contains slightly old information to speed up some lookups
* Switch to patched Paste that is hopefully download.gz-proof
* Don't store votes on things > 30 days old
* Many many bugfixes/small features
2011-02-23 12:00:44 -08:00
ketralnis
37e2ba9892 * Combine cassandra clusters into a single one
* Add a new Cassandra CF for comment-sort caching
* Add JSONP support
* Support non-auto-renewing PayPal IPNs
* Gold accounting
* Gold features: comments tracking, larger comment limit
* Autorenewing gold
* Google checkout support
* Profile-page sorting for all
* Title-text on the logo
* Hardcache sharding and profiling
* Self serve enhancements
* Add /r/foo/faq
* Make self-centred reddits allow for more verbose selftext
* Much better down-handling of databases
* Add the ability to take a thread-dump from a running process
* Remove the comscore tracker
* Add comments to modqueue (does not back-populate)
2010-10-18 11:52:44 -07:00
ketralnis
0ae8f2fb96 21 Jul 2010 merge
* Use Flaptor's Indextank product for search, including santip's patch
* for their new API
* Add Cassandra ConsistencyLevels to the ini file, and storage-conf.xml
* to the public repo
* Patch contributed by umbrae in ticket #929: Add jumpToContent support
* for Keyboard Accessibility
* reddit gold
    - paypal/postcard support
    - friends with benefits
    - profile-page sorting for gold members
    - move domain listings into the permacache
2010-07-21 17:48:12 -07:00
KeyserSosa
9a4271f641 Upgrade Instructions
====
   * Uninstall the python Cassandra package that we previously depended on (it has a namespace conflict with the new package we depend on).  To find it:

     $ python -c "import cassandra; print cassandra.__file__"

    and rm -r the .egg directory after "site-packages/"

   * This version relies on cython, so if "make" fails, you may have to install cython via your distro's package manager.

     $ cd reddit/r2
     $ python setup.py develop # possibly with "sudo" depending on your install
     $ make

   * Cassandra is now required for the caching layer.  An example storage-conf.xml can be found in reddit/srv/cassandra.  Make sure that the additional <Keyspace> items are included in your conf file.

   * remove the query_queue_reader services if they are running.  add new gen_time_listings.sh instead.  Suggested cron:

    0    */2 *   *   *   $SCRIPTS/gen_time_listings.sh year '("month","year")'
    */5  *   *   *   *   $SCRIPTS/gen_time_listings.sh week '("day","week")'
    *    *   *   *   *   $SCRIPTS/gen_time_listings.sh hour '("hour",)'

   where $SCRIPTS is the location of this script directory

Features and Bugfixes
====
 * Mobile reddit:
   * templates are in r2/templates of the form *.compact
   * css is r2/public/static/css/compact.css
     * beginning of a sass-based (http://sass-lang.com/) compact.scss
   * reachable via .compact extension or from the "i" subdomain.
 * Cassandra is now *required*, and votes are currently written out to both cassandra and postgres (part of an eventual migration).
 * attempt to make the db connection code a little smarter.
   * A dropped DB connection will mark the connection dead and randomly attempt to reconnect.
   * A dropped db connection on start will permanently mark the connection as dead.
 * Calculate the time-filtered top/controversy listings using mapreduce instead of prec_links (new cron job in reddit/scripts)
 * allow default user/pass for database to be specified with '*' to fallback on db_user and db_pass in the INI file
 * Search feedback buttons
 * make deleted comments not show up in your inbox.
 * move last_visited into cassandra
 * Swallow rare, race-conditiony POST_save/hide/subscribe problems
 * Apparently we haven't been breaking properly for the past few weeks.
2010-06-16 17:01:50 -07:00
KeyserSosa
08c431bd85 * Comply with the spec on 304 errors so Chrome won't barf download.gz all over the place
* The error pages have been broken for the past week.
 * Fix /r/all+all
 * Provides about.json to the api for subreddits (it already works for users)
 * Make sure Accounts have data before checking their name and msgtime
 * new bigger arrows for mobile site and a the alien is fingerless now
 * Try to update the _fast_query_timestamps less often to alleviate some of the load it causes on the Accounts DB
 * Swallow rare, race-conditiony POST_save/hide/subscribe problems
2010-05-28 14:55:13 -07:00
Jeremy Edberg
253c4730f9 * Add services and scripts to public repos for ease of reddit installation
* bring discount up to date
 * fix consistency errors in the self-serve traffic page
2010-05-18 16:43:06 -07:00
Jeremy Edberg
e6838895e0 New Features
* Two new 404 aliens
    * enable a comment page cache if the user isn't author of any of the comments and the page is a non-permalink.  votes and friends are updated by javascript

    Additions
    * descriptive example.ini
    * create "trusted_sponsor" who get auto-accepted
    * split off url cache from the permacache, with major clean-up to app_globals
    * Move is_banned_domain() call into POST_submit() so it can take the sanitized url
    * Redirect /r/asdfasdfasdf to the search page, not the create page.
    * updates to draw_load.py
    * orangered envelope goes to unread, not all.
    * Remove "enable whitelisting" checkbox, and switch to new failiens
    * Put verification link on email prefs page

    Bugfixes:
    * split off a minimal api controller for onload calls (which don't care about c.site or c.user, making them cheaper)
    * Assume admins want to keep their cookies forever just like everyone else
    * /randomrising bugfix
    * JSON error page was rendering in html.  it now returns "{ error: ${code} }"
2010-05-17 13:31:25 -07:00
KeyserSosa
e87f520d45 New Features:
* Make the new targeted self-serve promotion the default
      * add TheOatmeal's comic to our source.  (http://www.theoatmeal.com)
      * add targetting
      * add rerunning or suplementing existing campaigns
      * move all sponsored link listings to be precomputed
    * deputy moderation
      * /about/trials and /about/modqueue
      * Have trials train the spam filter
      * Allow admins to see all trials, site-wide, at /r/all/about/trials
      * Allow trials to be ended while votes are still coming in when the jury opinion is near-unanimous
    * offload scraping to embed.ly
    * Use boto for s3cp instead of forking curl

    Additions:
    * migrated all graphs to use Flot
    * Implement per-user suspiciousness quota and print over-quota trace
      * Keep track of how many questionable links a user has outstanding at any given time
    * add links to the FAQ and walkthru
    * add a differnt thumbnail for self posts (thanks to licensplate)
    * Add some visual tracking for comments trees
    * Relabel:  ban / unban => remove / approve
    * Use big remove / approve buttons when the listing relates to moderation,
      or when the item has reports or is on trial
    * Merge unban, ignore, and a new approve this non-banned link into one
    * Be a little smarter about the way we allow caching of subreddit stylesheets
    * make cassandra's cache chain cache negative results (and yes that commit message is bigger than the patch)
    * add All, friends, random, and mod to topbar
    * add a local render precache (optional)
    * better noimage.png from paradox460
    * Add a 'legacy' mode to CMemcache that will allow us to use it as a drop-in replacement for testing purposes. Modifies python-memcached to use the same 'crc' hashing algorithm that pylibmc does, and removes a feature in python-memcached that we aren't using. Contributed by Joe Terranova <joeterranova@gmail.com>
    * Everyone sees cake
    * Added g.system_user

    Bugfixes:
    * improve validation of media embed, and remove a todo from link.py by putting all of the child div stuff in pages
    * Feed/JSON updates
      * Add selftext to RSS feeds
      * Add subreddit names to Comments in JSON
      * add is_self param to indicate self posts
    * Don't shade the background of child comments of spam
    * shorted the cache lifetime for AllSR to 1 min
    * Tell everyone that we actually own the trademarks we claim to own (footer update)
    * You no longer get an orange alien when one of your reddits sends an automated message to a user
2010-05-17 13:28:21 -07:00
ketralnis
67814d543b Features:
* Cassandra
      * Add new cassandra libraries that we'll need to setup.py
      * Select Cassandra seeds at random rather than in order
      * Bugfix in CassandraCache.delete and a faster permacache migration function
      * Like other caches, CassandraCaches need to be able to take (and ignore) a 'time' parameter
      * add Cassandra to the permacache chain
    * beginning of jury duty (later called deputy moderation)

    Additions:
    * Make /r/friends much cheaper at the expense of sorting
    * Add Jury.delete_old(), which removes Account-Trial relations > 3 days old
    * Make the pretty_button() template function's callback optional, so that
      actionless pretty-buttons can be used on the admin details page
    * make .embed listings work for permalink pages (think of this as a first pass to getting blog comments working).  Adds 'limit' and 'depth' parameter to permalink pages
    * Added final redditheader.html pretty-button class
    * new iframe ads; also make button.js static
    * Usage sampling

    Bugfixes:
    * Stop adding batched time query recalculations to the queue at all except through the catch_up_batch_queries function
    * Superflous comma might be causing IE7 to barf
    * Change the byurl keys again, to fit in memcaches 251 character limit
    * Indentation error causing non-sponsors to be able to get to the advert listing
    * Move to a custom build of pylibmc that doesn't hold the GIL during blocking operations
    * Convert some cache.gets to cache.get_multis, and implement our own thread-safety around pylibmc's client
    * Make search caching a little smarter for time searches
    * Make the ads not be cached for 30 seconds each, ie. more random
    * fix deleted things on profile pages
2010-05-17 13:27:59 -07:00
Mike
a402d48de3 New features:
* Discount 1.6.1
      * Lines beginning with spaces are considered code. I don't know why markdown.py didn't trigger this.
      * tables in mark down: why not?
      * validation of resulting HTML vial libxml to prevent hax.
    * private RSS and JSON feeds
    * optional whitelists for subreddits
    * Moderator messaging

    Additions:
    * destination sanitization to cut down on XSRF
    * cosmetic fix to spam and reported listing
    * make the rss feeds on messages useful
    * /admin/errors
    * Of the types of listings hitting the precomputers (top/controversy by hour/day/week/month/year), the ones over long periods of time don't change often. So we can try to run them at most once per day, and then merge in the day's listings.
    * google analytics
    * logging queue
    * Created empty commentspanel.xml, errorpage.xml, login.xml
    * add subreddit rules/info box to submit page
    * add 'via' link on messages in moderator inbox
    * add a show=all get parameter to link listings to optionally ignore hiding preferences.
    * Raise edited timelimit to three mins
    * Remove UI that makes it look like you can edit deleted selftexts
    * Make it clearer to admins when a link is deleted
    * Fix [S] leak on deleted comments
    * Fix /user/[deleted] misrendering
    * New house ads system
    * updated so that minimalcontrollers actually can use the page cache.
    * Added /admin/usage

    Bugfixes:
    * Reduce the number of results that we request from Solr and simplify that caching a bit
    * Require a secret key to shut down app-servers
    * Make get_title a little more resilient to malformed documents and slow remote servers
    * Cause the SearchBuilder to only byID the results that it's going to render instead of all 1000
    * Remove ability for an author to XSS himself
    * fix spam listings and an xsrf
    * More verbose VDestination
    * Fixing the famous ?limit=0.1 error, and one last password-validation one
    * distinguish deleted comments' and deleted links' error messages
    * Don't allow ridiculously long log lines to widen the page
    * Bug with HardCache.add() when existing key is expired
    * Add adminbox next to domain
2010-05-17 13:27:40 -07:00
ketralnis
2869eaf8b9 New features:
* Activate negative-result caching for HardCache chain
    * begin migration to pylibmc:
       * Add pylibmc to the list of required packages in preparation for the replacement of the memcached library
       * Start using pylibmc for the rendercaches
    * Tweak the computation of the normalized hot page to be a bit faster when the precomputer is available, by relying on the precomputer's internal permacached structure.
    * Default to a SelfEmptyingCache for scripts run from `paster run'.  Note that processes that run forever are still responsible for resetting their local-caches, but this can now be done with g.reset_caches()
    * threaded messaging patch part 1: backend changes only.  This will allow migrate.py to be run to move new onto inbox and will start tracking message trees for users.
    * Specify some queries to run at most once per day
    * Refactored safemarkdown() and added soup testing

    Additions:
    * Added _byID_rel()
    * Made error messages more verbose for:
       1. byID lookups of too-big thing_ids
       2. memcache failures
       3. Solr Nones
    * Award._all_awards() now sorts by date
    * Trophy.by_{account,award}() now cache properly
    * new feedback page with helpful links
    * Try to reduce the length of the query-queue by not adding known-long queries at all, rather than adding them and skipping them
    * whitespace clean up
    * simplify the 'why did my CC get denied' email checking.
    * added missing translation strings and users now get PMs when they are added as translators

    Bugfixes:
    * Fix a bug in unsaving
    * BeautifulSoup stopped hosting 3.0.7a, but 3.0.8 still uses the good parser
    * Better search error handling
    * Properly reset the cache-chains (incl. the hardcache; d'oh!) per-request
    * Fix an attribute error on listings where some items have author_ids and some don't
    * Bug when forcing recalculation of memoized functions
    * the subreddit creation and edit form aren't dealing with errors properly
    * buttons fix
2010-05-17 13:27:17 -07:00
KeyserSosa
5ef76b96c7 New features:
* self service sponsored links (initial version with no targeting)
       * credit card processing with authorize.net (and corresponding interaction code)
    * hardcache -- for persistent cache keys with expiration
    * Awards code
    * messaging overhaul round 1.  Added mark as unread as well as message sorting

  Additions:
    * add TedScraper and improve youtube scraper (ala tritelife); move traffic link on sponsored links and make them still visible when the promotion is over
    * compressify all of our pngs, and set it up so thumbs, sr images, and the sprite get compressed every time they are generated
    * rate limit logins (naive way)
    * add a limit to the length of the moderator sidebox.
    * add over18 thumbnail handling and NSFW label on nsfw content
    * make NSFW label optional, and make the operation of the pref checkboxes sensible
    * add the option (for moderators) to remove a subreddit from the default set
    * Combine the queue handling code to be easily plumbed, and use it to precompute /comments

  Bugfixes:
    * trap duplicate vote error
    * Fixed new mail mouseover on toolbar
    * spammers can't send empty bodied emails any more (they will get rejected as they should).
2010-05-17 13:26:22 -07:00
KeyserSosa
bf9f43ccac Messaging/commenting
===
 - add confidence sorting to comments
   * common values are precomputed for speedier response
   * best is made the default sort on comment pages
 - messages will now be delivered once one is moderator/contributor/banned
 - UI updates to messaging page, including added show parent functionality to messages
 - Remove the rate-limit on comments on your own self-posts
 - Give users some leeway in editing their comments: don't show an edit star if the edit is within the first few minutes of a comment's lifetime
 - Office Assistant will help users when they write to admins

Backend
===
 - Replace the postgres-based query_queue with an AMQP based one
   * Set up amqp queues for async tasks such as search updates and the scrapers
   * service monitor updates, adding queue-tracking support
 - Allow find_recent_broken_things to specify both from_time and to_time
 - add a ini file parameter to disallow db writes (to create read-only reddit instances for crawlers)

New features
===
 - self-serve advertisement:
   * complete overhaul of sponsored link code
   * functions for talking with authorize.net
   * added pay domain and https support
   * added ability to share traffic from sponsored links
   * auto-reject promotions that are too old and unpaid for
 - awards
 - allow widget to have its links to have a target (in case it is iframed)
 - automatic_reddits:
   * Don't show automatic_reddits in the horizontal topbar
 - Listing numbers are always in order with no gaps
 - add support for sprites for common (r2.lib.contrib.nymph)

Admin
===
 - added a takedown page for dealing with DMCA requests properly
   * status code 404 on takedown pages
   * JSON returns same string as in the explanation text
   * nofollow on markdown in explanation
   * title and image optional
 - Added /c/(comment_id) for admins
 - updates to JS to rate-limit voting, commenting, and anything else that could be just as easily done by a script-kiddie to cheat.
 - make ad frame dynamic and add tracking pixel
 - add the ability to add a sponsored banner to the rightbox of a reddit
 - add the ability to show custom css on cnamed and/or non-cnamed versions of a reddit
 - allow us to ignore reports from report-spammers.

Bugfixes
===
 - Fix sorting of duplicate links (patch by Chromakode)
 - fix traffic bug on main traffic page when it is the first of the month.
 - toolbar redirects to comments page on self posts rather than generating the frame
 - half-assed unicode handling in menus giving us bugs again.  Switched to the whole-ass approach
 - added Thing._byID36
 - Support /help/foo/bar
2009-12-01 13:42:06 -08:00
ketralnis
2cf3490329 * add some new scrapers
* start showing embedded videos inside of an <iframe>. This allows us to
  embed videos from external sites without having to trust their DOM security

Note that the media_object property on new links is now a dictionary,
but old ones will still be strings
2009-07-21 14:31:11 -07:00
Mike
852b989f9e srbar updates 2009-07-16 12:14:14 -07:00
KeyserSosa
9c4ebbbdfd Allow submissions with short enough titled to be made into T-shirts thru spreadshirt. 2009-07-15 11:03:15 -07:00
Mike
7ee5b70c7f toolbar 2.1 2009-06-03 11:32:21 -07:00
ketralnis
36ed3fe248 Don't show very young links on the new page, so that the thumbnail fetcher et al have had time to act on them 2009-05-13 14:38:46 -07:00
KeyserSosa
0745f5bf18 make 1/2 of the default reddits English reddits if the user hasn't set their language preferences and is coming from a non-en-default browser. Also clean up and remvove old subredditbox code. 2009-05-13 11:13:00 -07:00
Mike
2de837f52a New help system and footer 2009-05-06 15:39:16 -07:00
Jeremy Edberg
9813d94741 Make reddit compatible with Postgres 8.3 and Python 2.6 2009-04-29 16:38:28 -07:00
KeyserSosa
47370d50b4 make locale INI file dependent 2009-03-16 15:21:28 -07:00
KeyserSosa
780d9bb987 example.ini bugfixes and first trials of site traffic integration 2009-03-16 14:48:11 -07:00
ketralnis
f6fbb05a79 Cache solr query results for a short time. Speeds up domain listings and the 'next' button on the search page 2009-02-27 11:59:57 -08:00
spez
1855d7034e fix example.ini 2009-01-28 16:33:37 -05:00
spez
2a5f891abe Moved database configuration from databases.py to the ini.
Added support for load-balancing database reads.
2009-01-28 16:29:12 -05:00
spez
ae6ba7a78e added a separate set of memcaches for the render cache 2008-12-16 15:51:43 -08:00
ketralnis
3c19fd2feb * decorate promoted links
* interface for creating promoted links
* store one organic list per unique list of subreddits rather than per user
2008-11-06 10:50:50 -08:00
KeyserSosa
e9589594d2 missing except block in share, cache cleanup, and example.ini updates 2008-09-26 11:26:34 -07:00
ketralnis
b9396594df Allow a link to be forced into the organic link box for site-wide announcements 2008-09-18 12:20:16 -07:00
KeyserSosa
7a30629bf7 added the ability for subreddit moderators to upload custom images for use in their custom stylesheets 2008-09-16 15:11:21 -07:00
KeyserSosa
21b7f4b7a7 fix to cname handling and graceful bookmarklet rejection. 2008-09-09 12:03:03 -07:00
ketralnis
8aac82ec88 Added forgotten example.ini parameter 2008-09-03 12:03:17 -07:00
ketralnis
6bcef0037b 1. Allow a reddit to have a cname, like www.proggit.com, that renders
the listing for that reddit

2. Allow a reddit to have a custom CSS stylesheet that appears to
   visitors

3. Allow a reddit to upload a custom reddit alien logo
2008-08-26 07:15:02 -07:00
Steve Huffman
1861a04e45 fixed sorting issue where the global sorts were being overwritten. added the query_queue parameters to example.ini. 2008-08-22 14:03:58 -07:00
KeyserSosa
7ce107f2ac sharing 2008-07-17 16:32:11 -07:00