466 Commits

Author SHA1 Message Date
FoxxMD
d2d945db2c Merge branch 'edge' 0.9.1 2021-09-21 15:08:28 -04:00
FoxxMD
c5018183e0 fix(attribution): Improve parsing of domain type to fix bug with galleries
* Add `redditMedia` as distinct domain type from `self` for more granular aggregation
* Use `redditMedia` to fix bug where video and galleries were being counted as `media`
2021-09-20 16:34:29 -04:00
FoxxMD
c5358f196d feat(author): Handle shadowbanned users
* Allow checking if user is shadowbanned via authorIs (AuthorCriteria)
* try-catch on history get or author criteria to try to detect shadowbanned user for a more descriptive error
2021-09-20 13:49:35 -04:00
FoxxMD
1d9f8245f9 feat(tooling): scope-based sorting with BC note for git cliff generation 2021-09-20 11:51:25 -04:00
FoxxMD
20b37f3a40 Initial git cliff config 2021-09-20 11:03:37 -04:00
FoxxMD
910f7f79ef Merge branch 'edge' 0.9.0 2021-09-20 10:54:32 -04:00
FoxxMD
641892cd3e fix: Fix activity push to manager
Should only be using firehose
2021-09-20 09:37:32 -04:00
FoxxMD
1dfb9779e7 feat(attribution): Allow specifying aggregateOn filter when using domain blacklist
May not make sense all the time but a properly configured config could take advantage of this
2021-09-17 15:14:36 -04:00
FoxxMD
40111c54a2 feat(message): Add a markdown formatted 'ruleSummary' property to content template data 2021-09-17 14:38:39 -04:00
FoxxMD
b4745e3b45 feat(message): Implement arbitrary message recipient to enable modmail
* Can send message to any entity (user/subreddit) using 'to' property, or leave unspecified to send to author of activity
* Parse entity type (user or subreddit) from to value and ensure its in a valid format we can understand with regex
2021-09-17 13:36:28 -04:00
FoxxMD
838da497ce feat: Add more detail to actioned events and logging for action results 2021-09-17 12:46:00 -04:00
FoxxMD
01755eada5 feat: De-dup activities from different polling sources
Previously CM would process the same activity multiple times if it was ingested from two different polling sources (modqueue and unmoderated/newSub). Introduce queue control flow to ensure activity is de-duped or refreshed before processing if this scenario occurs.

* Use a queue (firehose) to bottleneck all activities from different sources before pushing to worker queues
* Keep track of items currently ingested but not completely processed and use firehose to de-dupe queued items (flag to refresh) or re-queue if currently processing (and flag to refresh)
2021-09-17 11:50:49 -04:00
FoxxMD
1ff59ad6e8 feat: Add report count comparison to comment/submission state 2021-09-17 10:21:46 -04:00
FoxxMD
d8fd8e6140 feat: Add score (karma) comparison to comment/submission state 2021-09-17 10:13:21 -04:00
FoxxMD
255ffdb417 fix(recent): Deduplicate present subreddits 2021-09-16 16:48:00 -04:00
FoxxMD
f0199366a0 feat(history)!: Implement subreddit state and subreddit name parsing
* Implement total threshold to compare filtered activities against window activities

BREAKING CHANGE: include/exclude now filters POST activity window and all comparisons are done on those filtered activities against window activities
2021-09-16 15:36:06 -04:00
FoxxMD
20c724cab5 fix: Fix bug where non-media domains were not counted for attribution rule 2021-09-16 15:33:59 -04:00
FoxxMD
a670975f14 feat(repeat activity): Implement subreddit state and regex parsing 2021-09-16 14:12:16 -04:00
FoxxMD
ee13feaf57 feat(recent activity): Implement subreddit state and regex parsing for recent activity
* SubredditState can be used to check some subreddit attributes alongside, or in place of, a subreddit name
* Regex parsing for subreddit name string in recent activity
2021-09-16 13:34:19 -04:00
FoxxMD
23a24b4448 feat(regex)!: Simplify regex parsing from config
Reduce regex complexity in config by parsing a normal regex straight from config string value (including flags)

BREAKING CHANGE: regex must now be enclosed in forward slashes, flags must be on regex value, and regexFlags property has been removed
2021-09-16 10:53:33 -04:00
FoxxMD
a11b667d5e Merge branch 'edge' 0.8.1 2021-09-13 16:16:55 -04:00
FoxxMD
269b1620b9 fix(regex): fix when to add match sample to summary
Only include summary if there were non-zero matches
2021-09-07 15:04:30 -04:00
FoxxMD
6dee734440 fix(recent activity): fix subreddit summary in result
* Only include if any subreddits with activity found
* Use correct subreddit array
2021-09-07 15:03:56 -04:00
FoxxMD
3aea422eff fix(cache): check manager has cache (valid config) before trying to get actioned events 2021-09-07 14:56:03 -04:00
FoxxMD
e707e5a9a8 fix(manager): revert commented notification line from debugging 2021-09-07 14:07:00 -04:00
FoxxMD
2a24eea3a5 fix(rule): fix regex rule matching default behavior and improve log message
* Default to global flag if none specified so that all matches per activity are found
* Improve result message section ordering and display a sample of up to 4 found matches
2021-09-07 14:06:30 -04:00
FoxxMD
8ad8297c0e feat(rule): improve recent activity result (log message) by listing on subreddits with found activity 2021-09-07 14:04:26 -04:00
FoxxMD
0b94a14ac1 feat(ui): improve actioned events interactions
* Refactor api to get all accessible events, sorted by time, when subreddit is not specified
* Add subreddit name to actioned event data to differentiate between events
* Show actioned events link in "All" subreddit view
* Remove user-select css style (left over from config template)
* Format timestamp to be more human friendly
* Remove success/triggered text and just use checkmarks (same as log)
2021-09-07 13:33:35 -04:00
FoxxMD
a04e0d2a9b fix(cache): Set actioned events not to expire in cache 2021-09-07 13:26:30 -04:00
FoxxMD
3a1348c370 feat(ui): move actioned events link to "Actions Run" statistic
More intuitive location
2021-09-07 12:59:51 -04:00
FoxxMD
507818037f feat(cache): refactor actioned events into cache for persistence and make number stored configurable
* refactor actioned events into bot-configured cache so they can be persisted between restarts
* add config params for actionedEventsMax and actionedEventsDefault to allow defining defaults at operator/bot/subreddit level
2021-09-07 12:55:19 -04:00
FoxxMD
2c1f6daf4f Implement load config from URL for editor 2021-09-01 10:15:46 -04:00
FoxxMD
fef79472fe re-add missing heartbeat and improve bot exception handling
* Missed heartbeat during client-server refactor somehow...oops. Re-add heartbeat behavior
* Refactor nanny functionality to use date check rather than loop -- behaves same as heartbeat now
* use http retry handling in nanny to handle reddit outages
* try-catch on nanny and heartbeat for better exception handling at bot-level
* await health loop so we can catch bot-level exceptions in app to prevent entire app from crashing
2021-08-31 11:02:03 -04:00
FoxxMD
885e3fa765 Merge branch 'edge' 0.8.0 2021-08-26 16:04:01 -04:00
FoxxMD
0b2c0e6451 Add karma threshold for recent activity rule 2021-08-26 12:04:17 -04:00
FoxxMD
15806b5f1f Add regex documentation 2021-08-26 11:26:59 -04:00
FoxxMD
bf42cdf356 Fix author criteria hash 2021-08-25 15:26:41 -04:00
FoxxMD
e21acd86db Fix maxAge optional 2021-08-25 12:57:02 -04:00
FoxxMD
5dca1c9602 Refactor caching ttl data type to be more intuitive and flexible
* (BC) instead of 0 being 'disabled' must now be 'false'
* (BC) 0 now means cache indefinitely
* 'true' is an alias for '0'
2021-08-25 12:12:41 -04:00
FoxxMD
5274584d92 Improve readability/functionality for caching functions
* Make keys for readable by using plaintext for unique values and only hashing objects
* Improve author criteria caching by excluding item identifier from hash since result should be same at subreddit-level
2021-08-25 11:49:28 -04:00
FoxxMD
1d386c53a5 Make usernote author usage blind to author type
If it has to handle cached activities this will help
2021-08-25 11:30:19 -04:00
FoxxMD
d6e351b195 Fix missing TTL settings 2021-08-25 11:21:55 -04:00
FoxxMD
ea32dc0b62 Fix shared cache namespacing and key count
* Add special SHARED namespace for subreddits using default cache -- remove ns when cache is dedicated
* Check for redis cache type and include prefix pattern when getting key count
2021-08-24 15:59:15 -04:00
FoxxMD
dca57bb19e Fix key count display
Correctly count shared and non-shared
2021-08-24 15:58:04 -04:00
FoxxMD
43919f7f9c Fix connected users since cache can now be persistent 2021-08-24 15:57:48 -04:00
FoxxMD
a176b51148 Fix storing cache user result 2021-08-24 15:08:21 -04:00
FoxxMD
75ac5297df Refactor caching prefix building to provide unique defaults 2021-08-24 14:36:13 -04:00
FoxxMD
0ef2b99bd6 Refactor web to use more caching
* Implement invite code/data to use caching
* refactor operator config so session and invite use same cache instance
2021-08-24 14:10:12 -04:00
FoxxMD
9596a476b5 Improve caching for redis
* Re-add operator-level caching config so a global default cache config can be defined
* Expand provider options with index property so additional, redis specific, params can be provided
* namespace (prefix) bot and subreddit-level (When not shared) redis connections
* refactor subreddit and author name usage since it differs when objects are deserialized
* as type guard for submission based on instance type OR object shape hint since deserialized activities are plain objects
2021-08-24 13:10:13 -04:00
FoxxMD
92f52cada5 Better implementation of user cache result
* When check is triggered also store rule results in cache -- makes actioned events more complete when actions run from cached result
* Add config option to toggle run actions on/off from cached check (Defaults to on)
2021-08-24 10:30:37 -04:00