95 Commits

Author SHA1 Message Date
FoxxMD
c5018183e0 fix(attribution): Improve parsing of domain type to fix bug with galleries
* Add `redditMedia` as distinct domain type from `self` for more granular aggregation
* Use `redditMedia` to fix bug where video and galleries were being counted as `media`
2021-09-20 16:34:29 -04:00
FoxxMD
c5358f196d feat(author): Handle shadowbanned users
* Allow checking if user is shadowbanned via authorIs (AuthorCriteria)
* try-catch on history get or author criteria to try to detect shadowbanned user for a more descriptive error
2021-09-20 13:49:35 -04:00
FoxxMD
1dfb9779e7 feat(attribution): Allow specifying aggregateOn filter when using domain blacklist
May not make sense all the time but a properly configured config could take advantage of this
2021-09-17 15:14:36 -04:00
FoxxMD
b4745e3b45 feat(message): Implement arbitrary message recipient to enable modmail
* Can send message to any entity (user/subreddit) using 'to' property, or leave unspecified to send to author of activity
* Parse entity type (user or subreddit) from to value and ensure its in a valid format we can understand with regex
2021-09-17 13:36:28 -04:00
FoxxMD
1ff59ad6e8 feat: Add report count comparison to comment/submission state 2021-09-17 10:21:46 -04:00
FoxxMD
d8fd8e6140 feat: Add score (karma) comparison to comment/submission state 2021-09-17 10:13:21 -04:00
FoxxMD
f0199366a0 feat(history)!: Implement subreddit state and subreddit name parsing
* Implement total threshold to compare filtered activities against window activities

BREAKING CHANGE: include/exclude now filters POST activity window and all comparisons are done on those filtered activities against window activities
2021-09-16 15:36:06 -04:00
FoxxMD
a670975f14 feat(repeat activity): Implement subreddit state and regex parsing 2021-09-16 14:12:16 -04:00
FoxxMD
ee13feaf57 feat(recent activity): Implement subreddit state and regex parsing for recent activity
* SubredditState can be used to check some subreddit attributes alongside, or in place of, a subreddit name
* Regex parsing for subreddit name string in recent activity
2021-09-16 13:34:19 -04:00
FoxxMD
23a24b4448 feat(regex)!: Simplify regex parsing from config
Reduce regex complexity in config by parsing a normal regex straight from config string value (including flags)

BREAKING CHANGE: regex must now be enclosed in forward slashes, flags must be on regex value, and regexFlags property has been removed
2021-09-16 10:53:33 -04:00
FoxxMD
507818037f feat(cache): refactor actioned events into cache for persistence and make number stored configurable
* refactor actioned events into bot-configured cache so they can be persisted between restarts
* add config params for actionedEventsMax and actionedEventsDefault to allow defining defaults at operator/bot/subreddit level
2021-09-07 12:55:19 -04:00
FoxxMD
0b2c0e6451 Add karma threshold for recent activity rule 2021-08-26 12:04:17 -04:00
FoxxMD
e21acd86db Fix maxAge optional 2021-08-25 12:57:02 -04:00
FoxxMD
5dca1c9602 Refactor caching ttl data type to be more intuitive and flexible
* (BC) instead of 0 being 'disabled' must now be 'false'
* (BC) 0 now means cache indefinitely
* 'true' is an alias for '0'
2021-08-25 12:12:41 -04:00
FoxxMD
9596a476b5 Improve caching for redis
* Re-add operator-level caching config so a global default cache config can be defined
* Expand provider options with index property so additional, redis specific, params can be provided
* namespace (prefix) bot and subreddit-level (When not shared) redis connections
* refactor subreddit and author name usage since it differs when objects are deserialized
* as type guard for submission based on instance type OR object shape hint since deserialized activities are plain objects
2021-08-24 13:10:13 -04:00
FoxxMD
92f52cada5 Better implementation of user cache result
* When check is triggered also store rule results in cache -- makes actioned events more complete when actions run from cached result
* Add config option to toggle run actions on/off from cached check (Defaults to on)
2021-08-24 10:30:37 -04:00
FoxxMD
cfb228de73 Update schema documentation 2021-08-20 14:52:51 -04:00
FoxxMD
2fd1ffed19 Rewrite docs to explain server-client architecture 2021-08-20 13:47:55 -04:00
FoxxMD
bb2c5f076c Implement subreddit exclusion option for bot config 2021-08-20 11:08:09 -04:00
FoxxMD
2655ae6041 Update schema 2021-08-18 17:52:51 -04:00
FoxxMD
3b905e6961 General improvements for ui and error handling
* Move Bot webhook listeners for crash/shutdown into Bot code itself
* Simplify running modes
* Parse web operator into user data
* Add basic status tooltip for bots in ui
2021-08-10 16:39:39 -04:00
FoxxMD
707547effc Implement separate credentials for web 2021-08-10 12:02:03 -04:00
FoxxMD
3f1d1bc6d0 Rewrite age regex for schema to not use named capture groups (just regular capture groups) 2021-08-04 09:55:32 -04:00
FoxxMD
ad45f75267 More caching improvements
* Refactor cache-manager creation to use built in "none" store to simplify cache usage
* Implement comment check cache result flow
2021-08-03 13:06:34 -04:00
FoxxMD
643790d3bd Add 'enable' parameter for actions config 2021-08-03 10:54:50 -04:00
FoxxMD
a531d7e4e0 Add submission state for link flair text/css 2021-08-03 10:01:05 -04:00
FoxxMD
8d5d44bf0d Update schema 2021-08-02 16:46:49 -04:00
FoxxMD
af1ea5543e Implement caching for more components
* Implement caching for specific activities
* Implement/refactor item is criteria to cache activities and results
2021-08-02 15:10:49 -04:00
FoxxMD
0a3409cfef Add title matching on SubmissionState filter 2021-08-01 22:07:19 -04:00
FoxxMD
8b1d3cb170 Implement MessageAction
* enable bot to send private messages as self or modmail
* add necessary permissions to oauth helper to make this possible
* update schema with message action structure
2021-07-31 15:05:49 -04:00
FoxxMD
90df5f45a8 Add parent submission state testing when checking a comment activity 2021-07-31 14:22:17 -04:00
FoxxMD
2a5f812dba Implement defining multiple operators 2021-07-30 14:19:49 -04:00
FoxxMD
5f30dd8ce9 Refactor caching interface to simplify
No need to have it be optional string provider and it unnecessarily complicates json schema/configuration
2021-07-30 12:37:07 -04:00
FoxxMD
bf336ca55a Move the schema/settings for operator into json schema (viewer)
easier to read, has full examples, and validation
2021-07-30 12:00:42 -04:00
FoxxMD
fd4c2a38e7 Implement max queue workers as configurable
Globally or per-subreddit (max global)
2021-07-29 13:15:32 -04:00
FoxxMD
7175965e3d Update missed naming changes 2021-07-29 11:26:26 -04:00
FoxxMD
6b6124d76e Implement event hooks for notifications
* Implemented a generic notification manager with several event types
* Implemented discord notification provider
2021-07-27 12:24:36 -04:00
FoxxMD
7cda47183f Simplify cli options and improve web server checks
* Check for in-use web port before starting so we can display a useful error message
* Remove 'web' command and instead make it an optional argument for `run` (default to web)
* Update dockerfile to use run command
* Remove remaining commander option defaults since they are defined in config builder now
2021-07-26 12:18:51 -04:00
FoxxMD
b2b4988246 Consolidate all caching
* Use cache manager instance in UserNotes and callback to report stats
* Simplify sub resource config interface and setter
* Use cache manager for express session
* Add session cache provider to operator config
2021-07-23 14:48:33 -04:00
FoxxMD
73c3052c69 Refactor how configuration is parsed
* Implement a weak/strong interface for operator (app) config
* Consolidate config parsing code into ConfigBuilder
* Implement parsing configuration from a json file
* Refactor configuration parsing so there is a clear hierarchy to where and how config is overridden
2021-07-22 15:17:41 -04:00
FoxxMD
4fbb3edf8b Lots of regex fixes 2021-07-21 16:50:49 -04:00
FoxxMD
c69d66c001 Initial implementation of Regex Rule 2021-07-21 15:47:26 -04:00
FoxxMD
c9bdae66dd Implement configurable shared modqueue for operator convenience
If operator is running subreddits for lop-sided traffic burdens then /r/mod may be over the 100 activity limit, all from subreddit X while sub Y and Z have few activities. In this scenario Y and Z would never run since X would take up all results. Therefore default to individual modqueues and make shared modqueue opt-in by operator
2021-07-19 23:04:47 -04:00
FoxxMD
5df4fd5ccc Separate event polling and activity processing systems to improve robustness
* Use async/queue for handling activity processing on each manager
* Refactor polling to push activities into activity queue
* Refactor manager state to control activity processing queue and event polling independently
* Pause activity processing and wait until no work is being done to do config update
* Add way more logging for new systems
* Add basic ui controls/view for new systems
2021-07-19 16:32:41 -04:00
FoxxMD
b25001b7af Reduce false positives for repeat self/comment activities
* For self submissions use "title + body" to identify repeats instead of just body -- body could potentially be empty
* Add "minWordCount" with default of 1 so that blank comments are ignored
2021-07-18 21:30:44 -04:00
FoxxMD
be9dcdee1c Refactor polling streams to reduce usage and improve error catching
* Re-implement all polling streams with extended poll class for catching errors
* Increase default polling interval to 30 seconds and limit to 50
* When using default interval/limit for mod polling use a shared stream between all managers and poll to /r/mod -- reduces api calls for polling mod streams to 1 regardless of how many subreddits are managed
2021-07-12 13:44:07 -04:00
FoxxMD
15824e5d0f Implement delay before processing
Using criteria for Activity being N seconds old allows for delay with possibility of immediate processing to avoid api call

Closes #23
2021-07-07 15:23:59 -04:00
FoxxMD
f591c3a05a Implement more powerful content parsing options
* Can get wiki pages from other subreddits
* Can fetch from an external url
2021-07-06 10:27:30 -04:00
FoxxMD
529b8fc03e Further improvements for subreddit name parsing
* Allow whitespace on either side of regex value to parse since its automatically trimmed by getting capture group
* Implement sub name parsing everywhere subreddits can be specified and update documentation to remove prefix restrictions
2021-07-05 16:06:15 -04:00
FoxxMD
54eef5620d Update interfaces and documentation for new filters and item states 2021-07-05 15:39:08 -04:00