* Test props in order of least likely to use an API call
* Enables simplifying shadowbanned test and allows testing for more properties on shadowbanned user
* Fix existing bot removal
* Return response to client after testing client rather than after managers build to avoid long response time if bot has many subreddits
* Normalize (depopulate from snoowrap) mod note raw data so it can be constructed agnostic of source (cache or api)
* Implement cache GET for modnotes with default TTL of 60 seconds
* Refactor mod note action and implement cache PUT when new notes are added
I was under the impression primary keys were always indexed but that is not the case, at least for postgres. This migration explicitly creates uniques indexes for all tables that use random ids and adds other indexes to filter/premise/result tables on other FK strings. Improves event retrieval timing dramatically.
* Implement separate language detection functionality
* Clearer/simpler sentiment processing
* Add languageHints to help coerce low confidence language detection
* Add test cases for lang detection, sentiment detection, and sentiment tests
* Fix neutral range -- was not using normalized score range
* Build NLP container ad-hoc so only supported languages are included from npm
* Use vader/wink as heuristics for detecting language when content is very short
* Add languageHint option for sentiment config to make coercing a confident sentiment easier
* Refactor lang processing to fail the sentiment test rather than throwing an error when language is not support/not confident -- provides more insight into outcome
* Refactor to use interval in browser to call to api proxy endpoint and get live stats directly instead of using websockets. Generally simplifies things.
* Remove empty/superfluous data from cache stats returned for live data
Prevents CM from iterating through n+1 pages of polling sources (mostly unmoderated) due to a source-of-truth change
See comments for scenario this helps avoid
* Get streamed logs directly from api (through proxy) in browser using streaming apis instead of through client websockets
* Use observer visibility to determine which logs to stream
* Timeout and abort any streaming logs if tab hasn't been visible for more than 15 seconds
TODO streaming system logs
* Use browserify to include logform functions and triple-beam symbols in client js
* Implement default log transform function in client js
* Remove formatted message and transport data from non-streaming log data sent to client
* Fix missing assignment for filtered activities after removing activity
* Remove 'processing' state from delayed activity lifecycle
* Allows delayed queue to immediately remove activity after pushing to firehose -- simplifies lifecycle since another function (queue) doesn't have to handle this
* Removing delayed activity function call from queue logic reduces calls to database
* Likelihood an activity is cancelled while also processing is small (i hope...)
* Re-order item refresh logic in activity handling so delayed items are fetched before any proxy properties are accessed
* Catch errors on adding delayed activities from DB and just log -- not essential function
Use 'name' since this should *always* be present on a submission/comment -- when snoowrap creates an empty proxy it only includes 'name'.
* Potentially fixes facet of #64
* Fixes shouldRefresh undefined when activity is from delayed activities in database (empty proxy)
* Refactor from async while depending on queue state into interval that always runs and just checks paused status of queue -- eliminates need to restart while loop if queue state is not running
* Add canary debug output for delayed activities to be able to know if it is actually running
* Use duration field as SECONDS and remove additional time-based field (not necessary)
* Refactor dayjs usage in UI to parse action delay duration and correctly display "time until dispatch" to show if duration is negative
* Implement english-only scoring with wink https://github.com/winkjs/wink-sentiment (AFINN, emojis)
* Implement english-only scoring with NLP.js https://github.com/axa-group/nlp.js/blob/master/docs/v3/sentiment-analysis.md (AFINN, Senticon, Pattern)
* Refactor language processing into standalone functions for future use
* Add limited multi-langauge support
* Can run sentiment with NLP.js on english, german, spanish, and french
* Normalize all scores to range between -1 and +1
* Improve score accuracy by averaging all scores
* Add deprecation warnings to rules when building if properties should be migrated to window
* Add `debug` option to window to increase verbosity of filter logging. Default to false.
* Fix object assigment when building hash key for window filters
* Further cleanup for circular dependencies by moving some filter and logging functions into respective files
* Fix list function passed from author activities convenience method
* Move author history caching into main activity fetching function
* Do a better job at rehydrating snoowrap objects from cache data -- set as fetched, substitute relationships for non-fetching objects, and remove listing related objects
* Cache key for results based on window and pre-filter only -- post filter can be done after fetching cached results (Should save api calls!)
* Move some interfaces and types into own files to breakup huge interfaces file
* Refactor window shape for config and "full" usage in app to support subreddit/item filtering
* Refactor author activities into resources class so we can take advantage of caching on subreddit/item filter results
