* Improve action criteria to FullCriteria function
* Don't include undefined properties
* Iterate entries with switch to simplify property transformations
* Fix mod action test switch case matching to be case-insensitive (same as key)
* Fix missing/bad assignments for filtering mod actions
* Fix typo usage of foundNoteResult in modActions case block
* Throw error if mod action criteria isn't recognized as note/log instead of silently falling back to log
* Test props in order of least likely to use an API call
* Enables simplifying shadowbanned test and allows testing for more properties on shadowbanned user
* Fix existing bot removal
* Return response to client after testing client rather than after managers build to avoid long response time if bot has many subreddits
* Normalize (depopulate from snoowrap) mod note raw data so it can be constructed agnostic of source (cache or api)
* Implement cache GET for modnotes with default TTL of 60 seconds
* Refactor mod note action and implement cache PUT when new notes are added
I was under the impression primary keys were always indexed but that is not the case, at least for postgres. This migration explicitly creates uniques indexes for all tables that use random ids and adds other indexes to filter/premise/result tables on other FK strings. Improves event retrieval timing dramatically.
* Implement separate language detection functionality
* Clearer/simpler sentiment processing
* Add languageHints to help coerce low confidence language detection
* Add test cases for lang detection, sentiment detection, and sentiment tests
* Fix neutral range -- was not using normalized score range
* Build NLP container ad-hoc so only supported languages are included from npm
* Use vader/wink as heuristics for detecting language when content is very short
* Add languageHint option for sentiment config to make coercing a confident sentiment easier
* Refactor lang processing to fail the sentiment test rather than throwing an error when language is not support/not confident -- provides more insight into outcome
typeorm depends on ts-node as an *optional peer* dependency -- ts-node can be used in the typeorm cli to parse entities and run migrations.
However CM doesn't use typeorm CLI for running production so its not needed. And ts-node depends on typescript so npm install --production always installs both at about ~30MB.
I couldn't find a good way to remove peer deps ONLY for typeorm so instead just manually remove the folders from the prod install in the final layer of the docker.
* Refactor to use interval in browser to call to api proxy endpoint and get live stats directly instead of using websockets. Generally simplifies things.
* Remove empty/superfluous data from cache stats returned for live data
Prevents CM from iterating through n+1 pages of polling sources (mostly unmoderated) due to a source-of-truth change
See comments for scenario this helps avoid
* Get streamed logs directly from api (through proxy) in browser using streaming apis instead of through client websockets
* Use observer visibility to determine which logs to stream
* Timeout and abort any streaming logs if tab hasn't been visible for more than 15 seconds
TODO streaming system logs
* Use browserify to include logform functions and triple-beam symbols in client js
* Implement default log transform function in client js
* Remove formatted message and transport data from non-streaming log data sent to client
* Fix missing assignment for filtered activities after removing activity
* Remove 'processing' state from delayed activity lifecycle
* Allows delayed queue to immediately remove activity after pushing to firehose -- simplifies lifecycle since another function (queue) doesn't have to handle this
* Removing delayed activity function call from queue logic reduces calls to database
* Likelihood an activity is cancelled while also processing is small (i hope...)
* Re-order item refresh logic in activity handling so delayed items are fetched before any proxy properties are accessed
* Catch errors on adding delayed activities from DB and just log -- not essential function
Use 'name' since this should *always* be present on a submission/comment -- when snoowrap creates an empty proxy it only includes 'name'.
* Potentially fixes facet of #64
* Fixes shouldRefresh undefined when activity is from delayed activities in database (empty proxy)
* Refactor from async while depending on queue state into interval that always runs and just checks paused status of queue -- eliminates need to restart while loop if queue state is not running
* Add canary debug output for delayed activities to be able to know if it is actually running
* Use duration field as SECONDS and remove additional time-based field (not necessary)
* Refactor dayjs usage in UI to parse action delay duration and correctly display "time until dispatch" to show if duration is negative
