Commit Graph

93 Commits

Author SHA1 Message Date
Brian Simpson
5863fb6b8f pixel: Use user id36 rather than user name. 2015-01-06 04:01:12 -05:00
Brian Simpson
00e03edbfd Traffic processing: Validate clicks by checking response code.
Double checking in the click app and in the processing scripts was
difficult. Just trust the click app and assume any request that got
a 302 response is valid.
2014-12-12 17:00:29 -05:00
Brian Simpson
af04006baf click: unquote destination before unmangling query string. 2014-12-10 13:09:19 -08:00
Brian Simpson
966bb14675 Click redirect: fix encoding of destination url.
Some advertisers set their ad's url to an intermediate tracker so
they can independently track clicks. This results in a series of
redirects like this:

reddit tracker > intermediate tracker > final destination

The ad's url is communicated to the reddit tracker through a query
parameter which is urlencoded on reddit.com and then unquoted when
being handled by the reddit tracker. This unquoting causes problems
if there is an intermediate tracker with its own query string
that needs to be urlencoded. This commit adds handling for those query
strings.
2014-12-10 13:09:18 -08:00
Brian Simpson
a2e41ed4a6 click: Don't unquote destination url.
The url is already unquoted correctly and double unquoting can cause
problems with unicode characters.
2014-11-04 09:21:45 -05:00
Brian Simpson
78631fa746 Properly encode arguments for click tracker. 2014-10-09 05:55:50 -04:00
Brian Simpson
4bd2eb6ab8 Remove support for old click and impression hashes. 2014-10-09 05:55:50 -04:00
Brian Simpson
0a67287684 tracker: Delete unused adtracker_url. 2014-10-07 16:21:59 -04:00
Brian Simpson
4a9d7457bd tracker: Use constant_time_compare for hash check. 2014-10-07 16:21:54 -04:00
Brian Simpson
5a012fb789 Support new click and impression hashes that don't include IP.
Support new and old style hashes in verify.c and the click redirect app,
but only generate old style hashes.
2014-10-07 16:21:26 -04:00
umbrae
ee5ea8ca3c Inbox_counts: corrections on deletes, spams, edits
Conflicts:

	r2/r2/controllers/listingcontroller.py
2014-10-03 11:46:48 -07:00
umbrae
aad94d3f80 Inbox counts: fix typo in backfill script 2014-10-03 11:46:45 -07:00
umbrae
09c98d6dde Inbox counts: Add dark unread counts badge, start writing to inbox_count 2014-10-03 11:46:34 -07:00
Brian Simpson
eb9f0ae0e3 PromotionWeights: speed up queries by using distinct.
The queries are used to find the ids of PromoCampaign or Link objects
and we don't need the many (one per campaign per subreddit target per day)
PromotionWeights objects.
2014-10-01 02:44:16 -04:00
umbrae
9361596d68 Register hooks on app load rather than inline 2014-08-23 00:09:05 -07:00
umbrae
c0bff7498b Support 'all' in compute_time_listings 2014-07-17 13:03:13 -07:00
Brian Simpson
c18dcac467 Add subreddit gildings backfill script. 2014-06-11 14:39:06 -04:00
Roger Ostrander
dc68b16776 Trylater: Enable temporary subreddit bans 2014-06-05 14:45:28 -07:00
Brian Simpson
20f57a17eb Add GeoIP service. 2014-05-28 12:57:10 -07:00
Neil Williams
90cfcaaecc Update license headers to 2014.
Ok, now I'm getting some angst in my commit messages like my
predecessors had.  I understand now.  It's a terrible burden.  Why must
the calendar progress?  Why must numbers increment?  The world is
forever turning.

The future is here.

It is 2014.
2014-05-02 16:26:31 -04:00
Brian Simpson
17fcb723fc fetch_trackers: allow up to 100 ids. 2014-04-01 21:41:39 -04:00
Neil Williams
d2ccc40733 Automatically delete password hashes of deleted accounts.
The password hash is no longer necessary once an account is deleted (and
after a period of time for safety in case it needs to be restored)
2014-02-26 12:45:55 -08:00
Roger Ostrander
89762c93f0 Add TryLater: a system for scheduling events. 2014-02-26 12:45:55 -08:00
Chad Birch
7b24dacd77 compute_time_listings MINID query: order by date 2014-02-26 11:44:08 -08:00
Neil Williams
3511b08110 Combine and generalize the time listing precomputer scripts.
Previously, the subreddit/domain and account precomputers were separate.
This merges the two and improves their portability in the process.
Because of the increased portability, the precomputer can now be added
to the install script by default.
2014-02-13 13:50:52 -08:00
Neil Williams
5e249f4773 Make all moderators have a modmsgtime attribute.
This attribute can serve as a handy indicator that a user is a moderator
somewhere and can therefore replace the more costly modship lookup in
reddit_base.
2014-01-12 10:08:07 -05:00
bsimpson63
2a05f17161 No intermediate storage step in mr_process_hour.pig.
Made possible by upgrading pig to 0.10.
2013-12-05 04:04:35 -05:00
Neil Williams
c3c2f200fc read_secrets: Don't transform INI options on output. 2013-11-19 12:43:28 -08:00
Neil Williams
3366083663 Create a vault for secret tokens and move some into it.
This is intended to reduce the number of critical secrets stored in the
INI file.  An initial subset of secrets is moved into the vault to test
things out.
2013-11-19 12:16:58 -08:00
Neil Williams
342ad24409 Factor out INI file parsing into util. 2013-11-19 12:16:58 -08:00
Chad Birch
05c06df947 Userpage gildings tab: start writing
Includes backfill script to be run before the other commit that actually
adds the tab
2013-10-22 12:43:19 -06:00
Max Goodman
aaf1e51abb write-live-config: Update to reflect PluginLoader API change. 2013-10-21 11:37:17 -07:00
Jason Harvey
96fdb4dcb5 Cleanup temp files if dying due to replication lag. 2013-10-07 16:57:18 -07:00
Chad Birch
7722a73d15 Userpage gilded tab: start writing
Includes backfill script to be run before second commit to actually add
tab is deployed.
2013-10-07 15:45:06 -06:00
Neil Williams
f2cf180597 Remove remnants of legacy stylesheet systems. 2013-10-04 11:53:47 -07:00
Neil Williams
d81b953053 Revert "upload_static_files_to_s3: Don't list the whole bucket to find changes."
This reverts commit c67d688c9da437a87e8cf2fa1f94aa32be4de29f.
2013-10-04 11:53:47 -07:00
Andre D
544e4994a5 ModAction: Add a migrate script. 2013-09-18 08:56:55 -04:00
Neil Williams
4303841341 Add ImagesByWikiPage model and start dual-writing.
This model will initially be used to transfer subreddit images (used in
the stylesheet) off to a new system, but is intended to be used for
per-wikipage images in the future as well.
2013-08-19 16:11:50 -07:00
Neil Williams
bcea327b51 manage-consumers: Allow for subtyped consumers.
This is immediately useful for commentstree_q but may also help if we
move vote_q procs onto a single box.
2013-06-19 16:12:24 -07:00
Neil Williams
2265908f20 upload_static_files_to_s3: Don't list the whole bucket to find changes.
The static files S3 bucket has been getting a lot larger recently,
between subreddit stylesheets being in there and the static file cleaner
being disabled due to a bug. This is causing the deploy process to take
upwards of 3 minutes just to determine that no files need to be uploaded
to the bucket.

As a short-term workaround, this changes the uploader to check each key
individually with an S3 HEAD request rather than listing the whole
bucket. This is slower than best case of listing the bucket, but is
significantly faster than the current condition (~25 second runtime
now).
2013-06-16 22:35:40 -07:00
Neil Williams
8af415476b Update and fix license headers for 2013.
Angst!
2013-03-19 20:03:40 -07:00
bsimpson63
c886c59cc8 Add promoted_link tools. 2013-02-27 17:16:11 -05:00
Ricky Ramirez
4aec9618e6 write_live_config: Fail correctly if there is an error. 2013-02-19 14:49:56 -05:00
Ricky Ramirez
a9b5d2a7b0 write_live_config: Exit with non-zero code on user abort. 2013-02-19 14:49:50 -05:00
Neil Williams
77efa9ab43 wrap-job: Don't cd into r2/r2/ when we just want to be in r2/.
This was causing Pygments to fail to import correctly in jobs since
r2/r2/ has a commands.py which was getting relatively imported by
accident. Yuck.
2013-02-14 14:18:47 -08:00
Neil Williams
cd88393fa6 Fix manage-consumers not correctly iterating over files. 2013-01-28 13:39:01 -08:00
Neil Williams
73f518adea fetch_trackers: Use X-Forwarded-For when behind a trusted reverse proxy. 2013-01-28 12:08:22 -08:00
Neil Williams
417e8e6bb5 fetch_trackers: Make trackers.py self-packaging for Elastic Beanstalk.
This should have no effect on actually running the script, but is useful
if you do want to use Eb or likely some other service like GAE.
2013-01-28 12:08:22 -08:00
Roger Ostrander
2ab402c41d Domainban: Use Zookeeper as a backend 2013-01-25 12:35:08 -08:00
Neil Williams
afeabf204f Move consumer count info to individual files.
This is more easily composable among multiple puppet recipes etc.
2013-01-22 14:52:31 -08:00