This is to resolve the problem that we can generate markdown with named entities
like ` `. These are valid in XHTML's DTD but because entities aren't
namespaced they aren't valid in Atom documents, even within the XHTML namespace.
The fix is to switch to the RSS style double-escaped HTML type for `<content>`
blocks
Another approach would be to used numeric entities instead, but since we allow
users to type HTML entities we'd have to replace them too. That's a pretty large
change for such a small problem.
As a side effect of this approach, this means we also have to get rid of the
"[23 comments]" label on Link's feed items because the double-escaping breaks
the StringTemplate replacement
We have our Mako filters set to escape HTML by default. Unfortunately RSS
requires double escaping in some places and not in others, so there isn't a
reasonable default. Here I have done a pass through the `.xml` templates to find
user data that's ending up single-escaped and added double escaping to them.
This requirement is because RSS grew HTML support organically in a way that
clients can't tell if a field actually contains HTML or not. Sometimes it's
double escaped, sometimes it's not. Clients have to take a guess by sniffing for
`>` characters and hoping they get it right. This also means that it's
impossible for servers to reliably tell the clients which data this field
contains.
This is a bit of a ticking time bomb. Users may find ways to sneak in HTML in
the date field, or we may add new templates that forget to do double escaping on
little-used fields. I recommend that we switch these to use Atom which always
indicates whether the fields contain HTML or not. Work has started on this
conversion in another branch.
* ref https://www.reddit.com/r/AskNetsec/comments/41larg/titleheadbody_idmsgfeedsummarybodyimg/
* ref https://reddit.atlassian.net/browse/INFRA-721
* ref https://bugzilla.mozilla.org/show_bug.cgi?id=1240603
The cause of the bug is that if we fail to start because someone else has already started, we still delete their files.
From job-02 right now:
write_permacache() # comment ("day", "week+
write_permacache() # link ("day","week")
write_permacache() # link ("day","week")
write_permacache() # comment ("day", "week+
write_permacache() # link ("month","year")
write_permacache() # comment ("month", "ye+
* Make them lockless, because mr_top is the only one that ever writes to it.
This avoids a lot of memcached round trips
* Don't set them to permacache_memcaches. Delete from it instead.
This keeps us from blowing out that whole cache with listings that will never
be read out of every time mr_top runs.
* Fix a performance bug in _mr_tools.mr_reduce_max_per_key
timeouts.js and locked.js have pretty much the same code for managing
their popups. Pull this out into a file to reduce duplicate code, and
also make it easier to manage more of the same type of popup
If the cached query doesn't exist yet, "data" will be an empty list.
If we try to insert >= 1000 items, the chunk of code under the
"would_truncate" condition will run and raise "IndexError: list index
out of range"
Some account information for users in permanent suspension, such as karma, are
still accessible through the api. This info is hidden on the site, so it should
be treated similarly in the API response.
This needs to be called on posts being removed by the "all" filter
setting to update some info on them and put them into the correct cached
queries so that they show up in pages where the mods expect them.
You can run `vagrant up` from any directory within a project--vagrant
will look for a Vagrantfile in the current directory and if it doesn't
find one walk up one directory, and repeat until it finds a Vagrantfile.
Previously the `code_share_host_path` was set assuming that `vagrant up`
would always be run from the root reddit directory. Now it is set from the
fixed known location of the Vagrantfile.
This was previously being done in the spam-filter itself, so if there
are site issues causing the filter to fall behind, subreddits that use
the "all" setting to initially remove all posts would have it stop
working until the filter was able to catch up.
This moves the check into the actual posting process so that it isn't
dependent on the filter at all (and will also make this setting function
on open-source installs where the spam-filter code isn't available).
For events with a notion of a `target`, this stores:
* `target._age` as `target_age_seconds` in seconds. This is derivable from the event timestamp and the `target_created_ts` but simpler to query if it's denormalized
* For `Link`s, store the `target.title` as `target_title`. The primary goal here is to add more context around reports in Interana.
Also add a test for the report event and fixup the existing tests.
Don't allow sharing, saving, or hiding a deleted post. A user can unsave and
unhide a deleted post so that it's not stuck in the user's saved/hidden lists.
The share feature is currently broken on promoted links that have comments
disabled. Since the share feature relies on links having a comments thread,
hide it for any link with comments disabled.
Neither of these are used at all, so this code can be simplified a bit
by dropping them. get_cols was being specifically set to False in the
code so everything related to it being True was completely dead code.
