Compare commits

..

621 Commits

Author SHA1 Message Date
FoxxMD
fc51928054 Merge branch 'edge' 2022-02-02 16:59:56 -05:00
FoxxMD
c07276a3be fix(logging): Fix typo in error transform 2022-02-01 13:13:27 -05:00
FoxxMD
4a2297f5cd docs: Add github sponsor link 2022-02-01 12:01:34 -05:00
FoxxMD
f8967d55c4 feat(repeat): Use newer text comparison technique to improve repeat detection
* Use same technique as repost rule which has high accuracy and let false-positives
* Implement ability to see similarity score, case sensitivity, and text transformations
2022-01-31 14:08:21 -05:00
FoxxMD
e2590e50f8 Merge branch 'edge' 2022-01-28 17:27:51 -05:00
FoxxMD
7e8745d226 fix(polling): Fix shared polling behavior for nanny mode changes
* On hard limit stop shared streams
* On nanny mode turned off restart any stopped shared streams
2022-01-27 16:49:03 -05:00
FoxxMD
e2efc85833 fix(polling): Fix running state not changed on error
* Set running to false when error is caught. Was not caught on last stream refactor which changed polling behavior to end if any error is caught rather than waiting for external source to clear interval
* Add debugging/error messages on polling start/stop
2022-01-27 16:47:43 -05:00
FoxxMD
41038b9bcd feat(logging): Implement richer errors everywhere
* Use ErrorWithCause so we can get and print a chain of error causes
* Make reddit error response in stack trace more readable by replacing them with a "translated" parent response and add them as the cause
* Properly handle error formatting for winston by looking at shape of log object for error rather than testing instanceof (see comments in errorAwareFormat)
* Fix formatting in web interface for log lines with white-space pre css and properly splitting timestamp from rest of the message
2022-01-27 16:27:03 -05:00
FoxxMD
9fe8c9568c refactor: Move SimpleError into main Errors module 2022-01-27 11:48:23 -05:00
FoxxMD
9614f7a209 refactor(logging): Implement snoowrap errors "the right way" and implement consolidated logging function
* Implement declaration file for snoowrap errors so they can be imported directly
* Implement logging function to handle boilerplate for known error responses (reddit HTTP response, rate limit, etc.)
2022-01-27 11:43:39 -05:00
FoxxMD
8dbaaf6798 fix(logging): Defaults for log file dir 2022-01-26 12:28:56 -05:00
FoxxMD
c14ad6cb76 feat(logging): Implement separate logging options for each transport type
* Add properties for file, console, and stream in logging object of operator config
* Each property inherits a (useful) subset of winston transport options
2022-01-26 12:09:03 -05:00
FoxxMD
adda280dd3 fix(logging): Fix parsing log dir
* Correct else condition to use log dir when value is not true
* Set level to 'debug' on init logger if no value is provided to help with debugging
2022-01-26 10:27:01 -05:00
FoxxMD
15fd47bdb4 fix(polling): Correct typings for stream getter and check isFinished for Listing 2022-01-26 10:11:06 -05:00
FoxxMD
78b6d8b7b6 feat(polling): Add debug messages when streams are stopped 2022-01-26 10:00:09 -05:00
FoxxMD
61bc63ccc5 fix(polling): Emit config change event to bot only after manager has rebuilt polling 2022-01-26 09:50:30 -05:00
FoxxMD
05df8b7fe2 fix(polling): Use manager eventState to control shared stream callback rather than removing callback when events are stopped
Should prevent edge cases where shared streams are re-parsed while managers are stopped (hard limit) and then removed due to there being no callbacks
2022-01-25 18:07:15 -05:00
FoxxMD
3cb7dffb90 fix(polling): Prevent endless loop when trying to enforce continuity on a stream with no items returned 2022-01-25 09:25:59 -05:00
FoxxMD
d0aafc34b9 feat(remove): Add option to mark activity as spam 2022-01-21 13:03:05 -05:00
FoxxMD
d2e1b5019f chore: Update packages 2022-01-21 13:02:31 -05:00
FoxxMD
aaed0d3419 Merge branch 'edge' 2022-01-21 10:46:11 -05:00
FoxxMD
2a77c71645 fix(usernotes): Fix wiki entity handling to avoid unhandled rejection
Since snoowrap's WikiPage isn't a "real" object setting it as a property on the class means if it rejects the whole application crashes. Fix this by building wiki proxy every time we need it before awaiting promise for edit/retrieval so that promise scope is bound to the function we are in (that has try-catch)
2022-01-20 14:10:39 -05:00
FoxxMD
780e5c185e refactor(author filter): Strongly structure comparison/matching data for more consistent manipulation and output
* Use interface for comparison results at both criteria property level and criteria level
* Implement summary functions to build string results of comparisons
* Output all comparisons to debug and provide summaries to verbose (when applicable)
2022-01-20 14:08:54 -05:00
FoxxMD
38e2a4e69a fix(filter): Missing return on flair failure comparison 2022-01-19 15:49:44 -05:00
FoxxMD
7e0c34b6a3 fix(userflair): Fix wrong assignment for css 2022-01-19 13:10:11 -05:00
FoxxMD
e3ceb90d6f fix(filter): Fix default excludeCondition type
* Expected (prior) behavior is that all exclude criteria must pass, not just one
* Fix missing AND condition logic when all conditions pass
2022-01-19 13:09:45 -05:00
FoxxMD
6977e3bcdf feat(author): Add flair template criteria for author/submission
* Add filtering by flairTemplate id for author/submission
* Refactor flair properties for author/submission to accept string or array of strings
2022-01-19 12:48:58 -05:00
FoxxMD
f382cddc2a fix(filter): Change array merging behavior for authorIs defaults to be more sane
* Don't just overwrite (duh)
* Drop any default filters that include object keys that are also present in user-defined filters -- this way user-defined always takes precedence on merge
2022-01-19 11:52:18 -05:00
FoxxMD
99a5642bdf fix(ui): Change time formatting from 12 to 24 hour 2022-01-18 16:49:07 -05:00
FoxxMD
174d832ab0 docs: Pretty up readme header 2022-01-18 16:08:40 -05:00
FoxxMD
3ee7586fe2 fix(approve): Fix touched entity 2022-01-18 13:37:56 -05:00
FoxxMD
e2c724b4ae feat(approve): Implement approving parent submission of comment 2022-01-18 13:37:22 -05:00
FoxxMD
d581f19a36 feat(logs): Use log objects in api to improve parsing client-side
* Add options for /logs endpoint to stream objects instead of strings
* Always return log objects from /status endpoint -- fixes bug where all bots/subreddits got lines from logs that had newlines
* Return context-aware, formatted log lines to client to reduce line length IE if returning to botA -> subA then do not need to include labels for botA,subA #40
* Shorten timestamp to just time and wrap full timestamp in tooltip #40
* Emit log objects to client to reduce parsing complexity (don't have to regex for bot/subreddit name)
2022-01-18 12:59:59 -05:00
FoxxMD
48dea24bea feat: Improve first-run display in ui and add system view
* Fix bugs in UI when bot does not have a name (configured incorrectly)
* Implement instance system log view for operators
2022-01-18 10:38:39 -05:00
FoxxMD
5fc2a693a0 fix(config): Fix empty yaml config document initialization 2022-01-18 00:06:52 -05:00
FoxxMD
7be0722140 fix(bot): Fix limit rate expiration getter when there is no client initialized 2022-01-18 00:06:24 -05:00
FoxxMD
6ab9fe4bf4 feat(config): Implement persisting bots from invite process to application and config
* write to config when bot is added
* replace/add based on existing bot
* implement specify instance from instances user is operator of
* implement specify subreddits to run on using comma-separated list
* rewrite invite flow ending to be more clear on results and next steps
2022-01-17 17:47:27 -05:00
FoxxMD
5811af0342 feat(config): Refactor config parsing to preserve comments and enable writing
* use node-comment and yaml@next to keep comment information intact
* store ast/source version of parsed config for operator
* implement generic yaml/json operator config classes to keep everything organized and simplify marshalling source to js/string
* refactor file parsing and json/yaml parsing to have better single responsibility
2022-01-17 15:51:43 -05:00
FoxxMD
ed2924264a feat(util): Better check for file/dir permissions 2022-01-17 11:18:23 -05:00
FoxxMD
e9394ccf2e refactor(tooling): Ignore sqlite files 2022-01-17 09:52:18 -05:00
FoxxMD
dec72f95c6 docs: Add discord invite link 2022-01-14 16:42:01 -05:00
FoxxMD
bc7eff8928 Merge branch 'edge' 2022-01-14 15:27:09 -05:00
FoxxMD
80c11b2c7f refactor(filter): Consolidate authorIs logic and add additional control to exclude logic
* Add excludeCondition to control how exclude sets are tested (and/or)
* Refactor authorIs logic from check/rule/action into standalone function (DRY)
* Simplify filter defaults -- don't need to specify automoderator since it is always a mod
2022-01-14 10:51:29 -05:00
FoxxMD
e6a2a86828 feat(config): Implement default filter criteria behavior
* Add default behavior config to operator and manager config
* Implement configurable behavior when filter is present on check
* Add defaults to exclude mods and automoderator from checks
2022-01-13 16:46:32 -05:00
FoxxMD
96749be571 refactor(polling): Simplify and cleanup all polling logic
* Remove unused clearProcessing code
* Use same data structures (Map) for storing polling objects in both Manager and Bot to reduce cognitive load and re-use some logic
* Rename "mod" streams to "shared" streams
* Implement detection and updating of polling when manager config changes
* Implement detection and updating of shared streams on manager config update
* Use shared retry handler for manager polling to better handle general reddit api issues (all polling stops faster)
* Move initial polling buffer into polling object (instead of in manager) for better logic encapsulation and add debug logging for it
* Add more debug logging for manager/bot poll building
2022-01-13 11:39:16 -05:00
FoxxMD
6b7e8e7749 feat(polling): Implement shared streams for all polling sources
* Refactor polling config to use new 'shared' string list of polling sources and deprecate 'sharedMod' property
* Refactor how shared sources are built to look for shared intention in manager polling options before creating
* Implement continuity check for comment/submission polling to ensure no activities are missed
* Add debug logging to polling
2022-01-12 15:47:43 -05:00
FoxxMD
43b29432a2 refactor(auth): Refactor auth data structures to consolidate logic
* Add abstract user class with auth methods with implementations for client/server
* Refactor client/server logic to use class methods instead of inline auth checks

Closes #71
2022-01-12 09:57:38 -05:00
FoxxMD
ff84946068 feat(regex): Experimental support for parsing regex expressions from fetched URL
* Support fetching from reddit wiki
* Support fetching from raw URL
* Support parsing and fetching from gist, github blob, and regexr (very experimental)
2022-01-11 14:05:57 -05:00
FoxxMD
7cdde99864 fix(recent): Potential fix for reddit ACID issues on history retrieval 2022-01-11 13:00:51 -05:00
FoxxMD
8eee1fe2e1 fix(recent): Remove code that should have been deleted during refactor
Refactored recent to use batch subreddit testing but forgot to remove old, individual subreddit testing, code so activities were being counted twice
2022-01-11 10:15:16 -05:00
FoxxMD
6fc09864f6 fix: Don't delete property from object
Object passed by ref, duh
2022-01-11 10:13:48 -05:00
FoxxMD
1510980ce3 fix(util): Ensure provided state description is reattached to strong sub state 2022-01-11 10:13:14 -05:00
FoxxMD
56005f0f28 fix(bot): Fix own profile detection when building managers 2022-01-11 09:52:44 -05:00
FoxxMD
03b655515c fix(server): Fix logs not persisting for managers
* Change manager acquisition so all managers belong to a bot before they start logging so all logs are captured correctly
* Fix log capture logic that prevented all subreddits from being populated
2022-01-11 09:45:25 -05:00
FoxxMD
edd874f356 fix(server): Correctly filter bots and managers on auth on server 2022-01-11 09:15:52 -05:00
FoxxMD
7f13debe3b fix(client): Make sure all moderated subreddits are fetched 2022-01-10 16:17:24 -05:00
Matt Foxx
1565bdbf1a Merge pull request #67 from rysie/feature/dry-run-buttons
Run/Dry run buttons
2022-01-10 14:54:42 -05:00
FoxxMD
ec4cee8c77 refactor(ui): Fix and simplify button logic
* Fix url query selector to constrain to sub
* Use shared class between run buttons to simplify class modification and click event
2022-01-10 14:54:17 -05:00
FoxxMD
d6954533a0 Merge branch 'edge' 2022-01-10 12:32:14 -05:00
Matt Foxx
04b8762926 Merge pull request #68 from rysie/feature/flair-docs
feat(docs): User flair and submission flair docs
2022-01-10 12:31:56 -05:00
FoxxMD
dcc5f87c30 refactor(docs): Clean up flair docs
* Fix regex escaped characters
* Use authorIs
* make flair action type usage more clear
2022-01-10 12:22:16 -05:00
FoxxMD
66d9c0b2a7 fix(auth): Fix bug allowing any valid moderator to see all instances 2022-01-10 10:27:13 -05:00
FoxxMD
00e7cad423 fix(auth): Logout bot after auth flow is complete 2022-01-10 10:26:45 -05:00
Marcin Macinski
bc541d00d4 feat(docs): User flair and submission flair docs 2022-01-08 00:02:37 +01:00
Marcin Macinski
c5b27628b0 feat(ui): Run/Dry run buttons 2022-01-07 23:32:12 +01:00
FoxxMD
ba53233640 Merge branch 'edge' 2022-01-07 09:31:14 -05:00
Matt Foxx
ede86d285b Merge pull request #62 from rysie/user-flair-action
UserFlairAction added
2022-01-06 14:55:27 -05:00
FoxxMD
52f6aabb69 feat: Prevent bot from running on reports/comments it just created
Cache reported items or new comments made by bot for a short time (default to twice polling interval, 1 minute) to prevent bot from running on things it did itself
2022-01-06 14:54:17 -05:00
FoxxMD
18175f3662 feat(item filter): Support checking for different report types: total, user, mod 2022-01-06 13:13:10 -05:00
FoxxMD
68a272d305 fix(ui): Fix subreddit intersection check for bot related routes
Remove any prefixed r/ from a bot's subreddits when checking intersection with user subreddits
2022-01-06 12:29:19 -05:00
FoxxMD
3dac91fafc fix(recent): Fix default behavior for submissionReference based on activity type
Eliminates noisy logging when it's not specified but activity is comment
2022-01-06 12:09:04 -05:00
FoxxMD
e5bb8c2a38 fix(bot): Reduce retries for more aggressive fallback on reddit api issues
* Reduce retry for snoowrap to 2 since we do our own error handling in-app and 2 is enough for the occasional, non-systemic blip
* Reduce manager retries
2022-01-05 20:46:54 -05:00
FoxxMD
61e0baf3fd feat(recent): Add combined karma to template variables 2022-01-05 17:08:15 -05:00
FoxxMD
37e9d1fcc2 fix(polling): Fix set timeout args 2022-01-05 14:28:19 -05:00
FoxxMD
5e70ca1cb6 fix: Fix and improve code related to stopping bots when reddit api is not OK
* Fix polling timeout to actually stop on error by simplifying timeout and waiting until response is OK to recreate next timeout call
* Use "unexpected exception" retry count for all non well-known "reddit blip" responses in retry handler rather than failing immediately AND log this distinction
* Fix managers not emitting errors from checks
* Fix bot not awaiting retry handler on manager error emit
* Increase nanny loop delay on error to reduce api pressure when there are many bots running
* (unrelated) set bot as running before starting managers so UI is available earlier
2022-01-05 12:58:17 -05:00
FoxxMD
7f7ed18927 refactor(server): return app earlier so UI is available earlier
Bot init can finish asynchronously without any negative affect to server/client. Returning earlier means we can access server info earlier in startup
2022-01-05 12:50:55 -05:00
FoxxMD
efed3381fd feat(config): Allow top-level operator snoowrap config 2022-01-05 10:39:43 -05:00
FoxxMD
5ac5d65a28 refactor(userflair): Fix dryrun usage and add unflair functionality
* Can flair user on comment/submission
* fix dryrun if-else block (maybe a debugging artifact?)
* allow all properties to be undefined/null/empty and use as intention to unflair user
2022-01-03 21:02:21 -05:00
FoxxMD
1ac7ad4724 Merge branch 'edge' 2022-01-03 16:35:01 -05:00
FoxxMD
0ae74fdce1 docs: Add missing config box screenshot 2022-01-03 13:42:35 -05:00
FoxxMD
845173822c docs(onboarding): Update moderator getting started to reflect new editor capabilities 2022-01-03 13:40:17 -05:00
FoxxMD
edb3036957 feat(editor): Resolve raw url from github blob url 2022-01-03 13:31:54 -05:00
FoxxMD
3790f0e061 feat(editor): Implement wiki page creation
If page is not created yet detect it and require more perms for re-authorization
2022-01-03 13:23:31 -05:00
FoxxMD
e3e4e4abff feat(editor): Resolve raw gist url from gist id 2022-01-03 11:50:23 -05:00
FoxxMD
fd9b83437b feat(editor): Use manager format to infer format for editor
* Provide wiki format in manager data to status api
* Remove additional View link and instead infer format for querystring from manager data
2022-01-03 11:17:04 -05:00
FoxxMD
05694f115c fix(editor): Fix how initial editor value is set
For some reason providing the data directly to a new model doesn't trigger validation and also had some other weird effects. Instead, using an empty string as initial value and then set model value to data afterwards -- which fixes everything b/c idk why.
2022-01-03 11:15:29 -05:00
FoxxMD
70ee157198 feat(config): Make manager aware of config format
* Default format to yaml
* Add detected config format as property to manager
* When neither format is valid use starting character to (naively) detect json or not
* Reduce config error noise by only show one format error based on likely type and print other to debug
2022-01-03 10:58:02 -05:00
FoxxMD
bbb4ec3c2d fix: Fix missing hyphen character on regex for parsing reddit entity name
Reddit users can have hyphens in their names. Slight tradeoff for allowing hyphens for subreddit names (they are non existent) to allow all valid reddit user names is worth it.

Found and corrected by @prometheus-22
2021-12-31 00:04:25 -05:00
FoxxMD
acb72551ec fix(bot): Add missing invite check in healthloop 2021-12-30 23:09:09 -05:00
FoxxMD
bf6affe592 feat(bot): Add basic programmatic moderator invitation acceptance
* Store subreddits to try to accept invites from in bot's default cache
* Handle invitation scenarios (none, modself missing, accepted) and starting manager after invitation
* Implement basic invitation acceptance list control in UI for bot operators
2021-12-30 22:52:37 -05:00
FoxxMD
8c2cb02a46 feat(bot): Add modself oauth permission
Used for accepting moderator invitations
2021-12-30 20:25:51 -05:00
FoxxMD
73e2af2100 feat(manager): Improve wiki page creation
* Only try page creation if response error is a 404
* Improve permission error descriptions
* Only create if it can also set page permissions to improve security
2021-12-30 19:08:47 -05:00
FoxxMD
ba4c4af5a7 Update invite view 2021-12-30 18:00:11 -05:00
FoxxMD
9ad21ee2dd feat(bot): Add ability to create non-existing wiki page and change visibility 2021-12-25 21:13:55 -05:00
FoxxMD
b32c4f213c docs(yaml): Finish adding yaml examples
#61
2021-12-25 16:42:44 -05:00
FoxxMD
7e01c8d1f8 docs(yaml): add yaml examples fo activities window
#61
2021-12-25 16:28:58 -05:00
FoxxMD
aee158ecc9 docs(yaml): add attribution and advanced concepts yaml examples
#61
2021-12-25 16:18:13 -05:00
FoxxMD
8cd2243c2d docs(yaml): add history and author yaml examples
#61
2021-12-25 16:07:46 -05:00
FoxxMD
4969789532 docs(yaml): add recentActivity yaml examples
#61
2021-12-25 15:53:54 -05:00
FoxxMD
1dcfdc14d1 docs(yaml): add regex yaml examples
#61
2021-12-25 15:23:34 -05:00
FoxxMD
f1c9b64f64 docs(yaml): Add more examples
#61
2021-12-23 21:35:18 -05:00
FoxxMD
2e5a61566b docs(yaml): create yaml versions of subreddit ready examples
#61
2021-12-23 12:34:50 -05:00
Matt Foxx
85761fa662 Merge pull request #65 from rysie/edge-submission-flair-fix
Flair action fix: import + assigning flair by flair_template_id
2021-12-23 09:33:34 -05:00
Marcin Macinski
0b1a6bd77b Flair action fix: import + assigning flair by flair_template_id 2021-12-23 12:34:08 +01:00
Marcin Macinski
51e299ca99 Merge branch 'edge' into user-flair-action 2021-12-22 01:13:33 +01:00
Marcin Macinski
7696f3c2ff UserFlairAction added 2021-12-22 00:45:59 +01:00
FoxxMD
1c9ed41e70 feat(ui): Implement basic wiki editing capabilities for editor
* Use stored scope to determine if user can save
* Only show save action if loaded from a subreddit
* Implement re-authorization flow through popup window and sockets to update status in editor
* Implement wiki location endpoint for server and wiki save endpoint for client
2021-12-21 16:30:57 -05:00
FoxxMD
2d67f9f57d refactor(ui): Migrate all editor usage to monaco-yaml base
* monaco-yaml can also do json validation since its just normal monaco
* simplifies config.ejs greatly not having to maintain two different monaco implementation, at the expense of a larger project
2021-12-21 14:22:26 -05:00
FoxxMD
975bcb6ad7 feat(ui): Enable additional scopes usage
* Store scopes in user auth object/sessions
* Implement requesting additional scopes through login route
2021-12-21 12:30:33 -05:00
FoxxMD
2a282a0d6f Merge branch 'edge' 2021-12-21 09:35:21 -05:00
FoxxMD
0d087521a7 feat(docs): Specify yaml can be validated in editor 2021-12-21 09:35:06 -05:00
FoxxMD
fb5fc961cc feat(docs): Add youtube credentials requirements for repost rule 2021-12-21 09:33:24 -05:00
FoxxMD
c04b305881 feat(docs): Add link-spamming regex config 2021-12-21 09:22:35 -05:00
FoxxMD
5c5e9a26aa refactor(ui): Unify editor look-and-feel
* Use named entry files for webpack so we can access them from config.ejs
* Move some of the validation logic to config.ejs so it can be used by json editor as well
* Use same controls for loading files/schemas/urls for both editors
* Add validation errors box for both editors
2021-12-20 23:30:41 -05:00
Matt Foxx
477d1a10ae Merge pull request #60 from rysie/edge
Support setting user flair by flair_template_id
2021-12-20 18:37:36 -05:00
Marcin Macinski
bbee92699c Support setting user flair by flair_template_id 2021-12-21 00:10:40 +01:00
FoxxMD
7f09043cdf fix(ui): Add missing yaml asset files
Need to unignore them...
2021-12-20 15:56:36 -05:00
FoxxMD
768a199c40 feat(ui): Add quick-n-dirty YAML editor
Using one-off project cm-yaml http://github.com/foxxmd/cm-yaml to run a static version of monaco-yaml inside CM
2021-12-20 15:42:37 -05:00
FoxxMD
6e4b0c7719 feat(regex): Add match sample to rule results so it can be used in content template 2021-12-20 12:33:40 -05:00
FoxxMD
89b21e6073 fix(repost): Use average of all algos to get high score
Ensures no anomalous scores dominate
2021-12-18 18:44:52 -05:00
FoxxMD
da611c5894 fix(repost): Fix max weighted score and formatting 2021-12-18 18:22:38 -05:00
FoxxMD
2c90a260c0 fix(stream): Remove processing clear behavior for polling to try to fix weird polling behavior 2021-12-18 18:08:00 -05:00
FoxxMD
f081598da6 feat(repost): Improve text matching and default values
* Use 3 different matching algorithms using the highest score out of the three
* Weight score based on length of the sentence
* Increase minimum number of words to 3
* Enforce min word count on external (youtube) comments
2021-12-18 17:42:43 -05:00
FoxxMD
55f45163a4 feat(repost): Fix some bugs and improve result data
* Wrong hash check
* Add closest match summary and percent for use in content
2021-12-18 14:54:01 -05:00
FoxxMD
e4dfa9dde3 refactor(docker): reduce image size with best practices
* use --no-cache so repository index isn't included in image
* use FROM instructions to build project with dev dependencies, then copy built project to new stage and install production-only libraries
* properly ignore docs and node_modules with dockerignore
2021-12-17 15:10:22 -05:00
FoxxMD
0e395792db refactor: Remove unusused ts-auto-guard package
Has @ts-morph as a dependency which is a huge package
2021-12-17 14:36:03 -05:00
FoxxMD
dcbeb784e8 refactor: Remove set-random-interval
* depended on and always downloaded an entire, older typescript version (even with production install) -- was not necessary for one function
* refactor project to use newer TS version (specify any type for catch blocks to fix compiler errors)
2021-12-17 14:25:48 -05:00
FoxxMD
aeaeb6ce27 refactor(docker): Remove monaco editor local dependency to reduce image size
Wasn't necessary to have it installed locally (can use CDN) and is not a part of the core experience
2021-12-17 14:00:54 -05:00
FoxxMD
d6a29c5914 fix(bot): Actually get all moderated subreddits
Previously was only getting first page of moderated subreddits (listing). Make sure to get all pages.
2021-12-17 13:24:09 -05:00
FoxxMD
c1224121d4 feat(subreddit): Implement user profile filtering for SubredditState
* Add convenience property for filtering subreddits by user profile prefix (u_)
* Do some smart property comparison to check if SubredditState name is regex or not -- use additional user prof check if it is so we don't clobber existing name regex

Closes #56
2021-12-17 13:01:47 -05:00
FoxxMD
9790e681ea docs(regex): Add subreddit-ready example for removing discord link spam 2021-12-16 16:27:06 -05:00
FoxxMD
a48a850c98 Update winston repository package format/information 2021-12-16 14:58:43 -05:00
FoxxMD
b8369a9e9f Package bump for security
https://github.com/advisories/GHSA-93q8-gq69-wqmw
2021-12-14 12:06:48 -05:00
FoxxMD
0c31bdf25e refactor(repost): Improve repost criteria configuration and add documentation
* Simplify usage b/w comment-submission by removing "criteria" in main criteria property -- comment checks can just use additional properties
* Consolidated occurrence count/time into one property to allow and/or operands on both conditions (more powerful!)
* Added documentation describing repost configuration
* Added repost configuration examples
2021-12-07 16:30:05 -05:00
FoxxMD
4b14e581dd fix: Add more staggering on heartbeat check
Always stagger subreddit checks
2021-12-06 17:31:00 -05:00
FoxxMD
b2846efd2b fix: Improve falloff behavior when reddit api error are encountered
* Turn off snoowrap request queuing
* Aggregate errors from all managers at bot-level and force all to stop (and clear queue/polling) if a small threshold is met
* Add activity refresh on check into try-catch so delayed activities in queue don't cause a loop if they fail due to api issues
2021-12-06 17:16:04 -05:00
FoxxMD
a787e4515b feat(repost): Implement occurrence count and time comparisons
* User-defined set of comparisons for testing how many reposts were found
* User-defined set of comparisons for testing when a slice/all of reposts were created
2021-12-06 13:28:02 -05:00
FoxxMD
f63e2a0ec4 Merge branch 'topComments' into edge 2021-12-01 17:34:51 -05:00
FoxxMD
9d0e098db1 feat(rule): Implement repost rule
* checks for both submission/comment
* search by submission title, url, dups/crossposts, and external (just youtube for now)
* user-defined string sameness for all search facets
* user-defined case-sensitivity and regex-based transformations for activity/repost item values
* cache comment checks
* implemented youtube client for retrieving video comments
2021-12-01 17:34:25 -05:00
FoxxMD
181390f0eb fix: Correct error description for reading a config file
Was using copy-pasted statement from wiki error which was confusing
2021-12-01 09:33:03 -05:00
FoxxMD
a8c7b1dac9 Interim implementation of repost rule 2021-11-30 20:42:40 -05:00
FoxxMD
fd5a92758d Merge branch 'edge' 2021-11-28 19:43:20 -05:00
FoxxMD
027199d788 docs(heroku): Add information on dyno usage 2021-11-15 17:27:42 -05:00
FoxxMD
2a9f01b928 fix(heroku): Add OPERATOR to heroku deploy template 2021-11-15 17:26:34 -05:00
FoxxMD
cf54502f0d fix(heroku): Add redirect uri env to heroku deploy template 2021-11-15 17:26:34 -05:00
FoxxMD
2a3663ccc9 refactor(heroku): heroku-specific docker change
* Add image for web and worker
* Refactor port usage to conform to heroku's usage
2021-11-15 17:26:34 -05:00
FoxxMD
dc2eeffcb5 fix(bot): Remove own profile "sub" from default moderated subreddits to run on
The user profile for the bot account is returning as a moderated "subreddit" EX (/u/ContextMotBot) but its not useable by CM because it does not have a wiki and users don't use this as a sub. Filter it out of moderated subreddits when using the default behavior.
2021-11-15 13:19:21 -05:00
FoxxMD
39daa11f2d Merge branch 'edge' 2021-11-15 12:53:28 -05:00
FoxxMD
93de38a845 fix(filter): Missing 'op' property test on itemIs for comment 2021-11-15 12:47:34 -05:00
FoxxMD
43caaca1f2 fix(config): Fix wrong provider data used when bot has no caching config
Use previously built defaultProvider, instead of hardcoded memory config, so any cache config from operator waterfalls to bot/subreddit

Closes #46
2021-11-02 15:55:03 -04:00
FoxxMD
7bcc0195fe fix(author): change include/exclude on RULE to be optional
One of these properties must be present for the rule to be valid. This is enforced at runtime. Change the schema so that both are optional from the config-side though.
2021-11-02 09:41:55 -04:00
FoxxMD
dac6541e28 Merge branch 'edge' 2021-11-01 16:12:43 -04:00
FoxxMD
2504a34a34 refactor(docker): Remove old steps from node-canvas dep
Not using node-canvas anymore so dockerfile can be simplified #48
2021-11-01 15:55:12 -04:00
FoxxMD
e19639ad0d fix(ui): Forgot to add tailwin css to git
Oops!
2021-11-01 15:06:50 -04:00
FoxxMD
b8084e02b5 Merge branch 'colorImprovements' into edge 2021-11-01 14:58:01 -04:00
FoxxMD
97906281e6 Merge branch 'edge' 2021-11-01 14:55:10 -04:00
FoxxMD
2cea119657 feat(ui): Add tooltip descriptions for depleted/limit reset stats
Closes #36
2021-11-01 14:52:35 -04:00
FoxxMD
6f16d289dd feat(ui): Add lang attribute for html
Closes #35
2021-11-01 14:22:20 -04:00
FoxxMD
a96575c6b3 refactor(ui): Use local tailwind css asset
Closes #34
2021-11-01 14:19:47 -04:00
FoxxMD
0a82e83352 refactor(ui)!: Implement colorbind color changes for rest of UI and remove dark/light mode
* Apply changes made in ac409dce to the rest of the application
* Remove dark/light mode -- now always dark mode (easier to maintain this way)
* Remove dependency on tailwind dark css
2021-11-01 14:14:49 -04:00
FoxxMD
d5e1cdec61 fix(criteria): Improve criteria filtering for removed/deleted activities and logging
* Use different logging messages when criteria is not available due to mod permissions (property not available to non-mods)
* Change logging level for missing/unavailable criteria to reduce logging noise. On unavailable use debug, on missing use warn
* Improve activity removed/deleted detection based on whether activity is moddable by current user
2021-11-01 13:25:54 -04:00
FoxxMD
ef40c25b09 feat(attribution): Add additional subreddit and activity filtering functionality to criteria
* Refactor subreddit filtering with include/exclude to use subreddit state
* Add submissionState and commentState filters
2021-11-01 11:26:55 -04:00
FoxxMD
6370a2976a Merge branch 'edge' into colorImprovements 2021-11-01 10:23:40 -04:00
FoxxMD
d8180299ea fix(author): Fix missing true return statement for author flair check 2021-11-01 10:23:22 -04:00
FoxxMD
ac409dce3d POC color improvements for color blindness
Needs another pass and consolidate/clean up for pages other than status
2021-10-20 21:12:31 -04:00
FoxxMD
56c007c20d feat(author): Implement author profile description regex/string testing
May test "description" in authorIs as a regular express of string literal -- or as an array of the aforementioned values
2021-10-20 19:52:46 -04:00
FoxxMD
487f13f704 Merge branch 'edge' 2021-10-12 11:56:51 -04:00
FoxxMD
00b9d87cdc Remove unused types package 2021-10-12 11:56:29 -04:00
FoxxMD
2c797e0b9b docs(image): Add documentation for image comparison
#26
2021-10-12 11:10:14 -04:00
FoxxMD
4a2b27bfbf fix(recent): Revert debug change for concurrency on activity comparisons 2021-10-12 11:08:53 -04:00
FoxxMD
463a4dc0eb feat(image): Implement perceptual hashing image comparison
* Generate perceptual hashes using blockhash-js of images that can be cache/stored
* Take advantage of reddit thumbnail code ImageData to hash lower-res to begin with (but represent full url)
* Refactor imageDetection config so hash and pixel approaches have different configs
* Cache phash results to reduce reddit traffic and speed up performance

Addresses cpu/memory issues with pixel comparison. Allow pixel for finer comparisons if needed using tiered thresholds. Closes #26
2021-10-11 15:28:48 -04:00
FoxxMD
4b3bea661d Merge branch 'imageComparisonOptimization' into edge
# Conflicts:
#	src/Rule/RecentActivityRule.ts
2021-10-08 13:50:49 -04:00
FoxxMD
976f310f51 refactor(image): Remove resemblejs dependency
* Refactor image comparison to use pixelmatch only so resemblejs can be removed (too much memory usage)
* Heavier usage of sharp to get images into same dimensions prior to pixelmatch
* Refactor image conversion into ImageData to clean up utils/recent activity rule
2021-10-08 13:49:46 -04:00
FoxxMD
4d8d3dc266 fix(docker): Add libvips dependency 2021-10-08 13:47:35 -04:00
FoxxMD
ce9e678c4c fix(image): clone sharp instance so converting between outputs doesn't cause issues 2021-10-07 18:06:05 -04:00
FoxxMD
8cf30b6b7d refactor: Introduce staggered startup for bots/polling to decrease load on host/reddit and improve image comparison performance
* Implement staggered startup for bots (reddit accounts, top-level)
* Implement staggered startup for managers (subreddits) and subreddit polling
* Introduce random -1/+1 second to polling interval for every stream to ensure none are synced so there is no instantaneous spike in cpu/traffic/memory on host/reddit
* Add user-configurable stagger interval for shared mod polling
* Implement second image comparison approach with pixelmatch for reduced memory usage when image dimensions are exactly the same
* Use sharp to resize images to 400 width max when using resemblejs to reduce memory usage
2021-10-07 17:13:27 -04:00
FoxxMD
2b6d08f8a5 Dummy commit
To get github actions to run
2021-10-06 18:03:26 -04:00
FoxxMD
f8fc63991f feat(image): Leverage reddit image previews for download/comparison
* Refactor image acquisition/parsing to use on-demand fetching and track different resolutions
* Try to use smaller previews (under 1000px), when possible, for comparing images and downloading
* Do image comparisons in parallel
2021-10-06 17:33:56 -04:00
FoxxMD
d96a1f677c refactor(image): Profile image comparison analysis time to debug statement 2021-10-06 14:14:36 -04:00
FoxxMD
b14689791c refactor(image): Document resemble compare options for performance
* Disable color comparison is slower than nothing
* Restrict comparison to a max of 800x800 pixels is slower than nothing
2021-10-06 14:13:50 -04:00
FoxxMD
b70c877e44 refactor(polling): Improve processed list configuration/usage
* Change 'after' type to string duration for friendlier configuration
* Decrease list size trigger === limit instead of 2x (not necessary to have a list that big for polling new)
* Increase initial shared mod polling to max limit (100)
2021-10-06 12:54:23 -04:00
FoxxMD
041655376a feat(filter): Implement age test for comment/submission
Closes #27
2021-10-06 12:54:23 -04:00
FoxxMD
e1eab7696b feat(polling): Implement memory bloat mitigation for long-running SnooStorm polling
The list of processed activities SnooStorm uses to ensure only new activities are emitted when polling is never cleared. MayorMonty/Snoostorm#35

To mitigate the memory bloat this creates when RCB runs for a long time on high-volume subreddits implement user-configurable (with defaults) behavior for clearing the processed activity list. Default values ensure clearing the list does not interfere with checking for new activities.
2021-10-05 14:45:10 -04:00
FoxxMD
65d1d36d53 feat(cache): Implement batching for subreddit info
When using full criteria for subreddit state we can save a ton of api calls by get info for all uncached subreddits at the same time rather than individually
2021-09-29 16:20:02 -04:00
FoxxMD
120d776fc2 feat(ui): Display subreddit and subreddit crit statistics in cache call breakdown popup 2021-09-29 10:47:33 -04:00
FoxxMD
425e16295b refactor(cache): Better cache usage for some known state conditions
* Don't store subreddit state cache results for now since nothing computationally expensive or requires api requests
* Return early on item state check if there is nothing to check so we don't store an empty result in cache
2021-09-29 10:47:05 -04:00
FoxxMD
dd7e9d72cc fix(subreddit): Fix subreddit state testing edge case and undefined criteria
* allow both over_18 and over18 criteria in case user accidentally used name from sub/comm state
* correctly determine if subreddit property exists when testing
* fix cache hit subreddit name logging
2021-09-29 10:21:16 -04:00
FoxxMD
55535ddd62 fix(regex): Fix regex generation for simple subreddit strings
* Trim value before parsing
* If not a valid regex string then when generating regex from simple string add qualifiers for beginning/end of string so any matches must be exact
2021-09-29 09:22:40 -04:00
FoxxMD
631e21452c Merge branch 'edge' 2021-09-28 16:36:13 -04:00
FoxxMD
be6fa4dd50 fix(cache): Fix accidental re-use of maps 2021-09-24 16:13:58 -04:00
FoxxMD
0d7a82836f refactor(cache): Move bot usage stats into cache
* Moving into cache means stats will persist after restart (yay!)
* Refactored stats structure to be simpler
2021-09-24 15:24:19 -04:00
FoxxMD
d9a59b6824 feat(recent): Print log statement when image processing is causing rule to take a long time 2021-09-23 13:23:58 -04:00
FoxxMD
ddbf8c3189 fix(recent): Actually use filtered activities when using submission as reference 2021-09-23 12:58:43 -04:00
FoxxMD
8393c471b2 fix(image): Dynamically import resemblejs for better compatibility on systems not supporting node-canvas
* By dynamically importing the module any user not using image comparison will not be affected by a lack of node-canvas dependency
* try-catch on import and provide a helpful error message about node-canvas dep
2021-09-23 10:38:34 -04:00
FoxxMD
fe66a2e8f7 fix(docker): Update build to build node-canvas from source 2021-09-23 10:08:26 -04:00
FoxxMD
4b0284102d fix: Improve image comparison threshold and results for typescript 2021-09-22 22:15:00 -04:00
FoxxMD
95529f14a8 feat(recent): Implement pixel-level image comparison when using a reference (image) submission 2021-09-22 16:52:56 -04:00
FoxxMD
26af2c4e4d fix(recent): don't include submission being checked when filtering by reference 2021-09-22 10:29:06 -04:00
FoxxMD
044c293f34 fix(attribution): Update aggregateOn defaults to align with expected behavior
Majority of mods that have used this rule assume it does not aggregate on reddit domains by default (only external links), which is reasonable.
So update the default to follow this assumption.
2021-09-22 10:11:25 -04:00
FoxxMD
a082c9e593 doc(attribution): Remove unused useSubmissionAsReference property 2021-09-22 09:36:44 -04:00
FoxxMD
4f3685a1f5 Merge branch 'edge' 2021-09-21 15:18:38 -04:00
FoxxMD
e242c36c09 fix(tooling): Fix tag pattern for git cliff 2021-09-21 15:18:26 -04:00
FoxxMD
d2d945db2c Merge branch 'edge' 2021-09-21 15:08:28 -04:00
FoxxMD
c5018183e0 fix(attribution): Improve parsing of domain type to fix bug with galleries
* Add `redditMedia` as distinct domain type from `self` for more granular aggregation
* Use `redditMedia` to fix bug where video and galleries were being counted as `media`
2021-09-20 16:34:29 -04:00
FoxxMD
c5358f196d feat(author): Handle shadowbanned users
* Allow checking if user is shadowbanned via authorIs (AuthorCriteria)
* try-catch on history get or author criteria to try to detect shadowbanned user for a more descriptive error
2021-09-20 13:49:35 -04:00
FoxxMD
1d9f8245f9 feat(tooling): scope-based sorting with BC note for git cliff generation 2021-09-20 11:51:25 -04:00
FoxxMD
20b37f3a40 Initial git cliff config 2021-09-20 11:03:37 -04:00
FoxxMD
910f7f79ef Merge branch 'edge' 2021-09-20 10:54:32 -04:00
FoxxMD
641892cd3e fix: Fix activity push to manager
Should only be using firehose
2021-09-20 09:37:32 -04:00
FoxxMD
1dfb9779e7 feat(attribution): Allow specifying aggregateOn filter when using domain blacklist
May not make sense all the time but a properly configured config could take advantage of this
2021-09-17 15:14:36 -04:00
FoxxMD
40111c54a2 feat(message): Add a markdown formatted 'ruleSummary' property to content template data 2021-09-17 14:38:39 -04:00
FoxxMD
b4745e3b45 feat(message): Implement arbitrary message recipient to enable modmail
* Can send message to any entity (user/subreddit) using 'to' property, or leave unspecified to send to author of activity
* Parse entity type (user or subreddit) from to value and ensure its in a valid format we can understand with regex
2021-09-17 13:36:28 -04:00
FoxxMD
838da497ce feat: Add more detail to actioned events and logging for action results 2021-09-17 12:46:00 -04:00
FoxxMD
01755eada5 feat: De-dup activities from different polling sources
Previously CM would process the same activity multiple times if it was ingested from two different polling sources (modqueue and unmoderated/newSub). Introduce queue control flow to ensure activity is de-duped or refreshed before processing if this scenario occurs.

* Use a queue (firehose) to bottleneck all activities from different sources before pushing to worker queues
* Keep track of items currently ingested but not completely processed and use firehose to de-dupe queued items (flag to refresh) or re-queue if currently processing (and flag to refresh)
2021-09-17 11:50:49 -04:00
FoxxMD
1ff59ad6e8 feat: Add report count comparison to comment/submission state 2021-09-17 10:21:46 -04:00
FoxxMD
d8fd8e6140 feat: Add score (karma) comparison to comment/submission state 2021-09-17 10:13:21 -04:00
FoxxMD
255ffdb417 fix(recent): Deduplicate present subreddits 2021-09-16 16:48:00 -04:00
FoxxMD
f0199366a0 feat(history)!: Implement subreddit state and subreddit name parsing
* Implement total threshold to compare filtered activities against window activities

BREAKING CHANGE: include/exclude now filters POST activity window and all comparisons are done on those filtered activities against window activities
2021-09-16 15:36:06 -04:00
FoxxMD
20c724cab5 fix: Fix bug where non-media domains were not counted for attribution rule 2021-09-16 15:33:59 -04:00
FoxxMD
a670975f14 feat(repeat activity): Implement subreddit state and regex parsing 2021-09-16 14:12:16 -04:00
FoxxMD
ee13feaf57 feat(recent activity): Implement subreddit state and regex parsing for recent activity
* SubredditState can be used to check some subreddit attributes alongside, or in place of, a subreddit name
* Regex parsing for subreddit name string in recent activity
2021-09-16 13:34:19 -04:00
FoxxMD
23a24b4448 feat(regex)!: Simplify regex parsing from config
Reduce regex complexity in config by parsing a normal regex straight from config string value (including flags)

BREAKING CHANGE: regex must now be enclosed in forward slashes, flags must be on regex value, and regexFlags property has been removed
2021-09-16 10:53:33 -04:00
FoxxMD
a11b667d5e Merge branch 'edge' 2021-09-13 16:16:55 -04:00
FoxxMD
269b1620b9 fix(regex): fix when to add match sample to summary
Only include summary if there were non-zero matches
2021-09-07 15:04:30 -04:00
FoxxMD
6dee734440 fix(recent activity): fix subreddit summary in result
* Only include if any subreddits with activity found
* Use correct subreddit array
2021-09-07 15:03:56 -04:00
FoxxMD
3aea422eff fix(cache): check manager has cache (valid config) before trying to get actioned events 2021-09-07 14:56:03 -04:00
FoxxMD
e707e5a9a8 fix(manager): revert commented notification line from debugging 2021-09-07 14:07:00 -04:00
FoxxMD
2a24eea3a5 fix(rule): fix regex rule matching default behavior and improve log message
* Default to global flag if none specified so that all matches per activity are found
* Improve result message section ordering and display a sample of up to 4 found matches
2021-09-07 14:06:30 -04:00
FoxxMD
8ad8297c0e feat(rule): improve recent activity result (log message) by listing on subreddits with found activity 2021-09-07 14:04:26 -04:00
FoxxMD
0b94a14ac1 feat(ui): improve actioned events interactions
* Refactor api to get all accessible events, sorted by time, when subreddit is not specified
* Add subreddit name to actioned event data to differentiate between events
* Show actioned events link in "All" subreddit view
* Remove user-select css style (left over from config template)
* Format timestamp to be more human friendly
* Remove success/triggered text and just use checkmarks (same as log)
2021-09-07 13:33:35 -04:00
FoxxMD
a04e0d2a9b fix(cache): Set actioned events not to expire in cache 2021-09-07 13:26:30 -04:00
FoxxMD
3a1348c370 feat(ui): move actioned events link to "Actions Run" statistic
More intuitive location
2021-09-07 12:59:51 -04:00
FoxxMD
507818037f feat(cache): refactor actioned events into cache for persistence and make number stored configurable
* refactor actioned events into bot-configured cache so they can be persisted between restarts
* add config params for actionedEventsMax and actionedEventsDefault to allow defining defaults at operator/bot/subreddit level
2021-09-07 12:55:19 -04:00
FoxxMD
2c1f6daf4f Implement load config from URL for editor 2021-09-01 10:15:46 -04:00
FoxxMD
fef79472fe re-add missing heartbeat and improve bot exception handling
* Missed heartbeat during client-server refactor somehow...oops. Re-add heartbeat behavior
* Refactor nanny functionality to use date check rather than loop -- behaves same as heartbeat now
* use http retry handling in nanny to handle reddit outages
* try-catch on nanny and heartbeat for better exception handling at bot-level
* await health loop so we can catch bot-level exceptions in app to prevent entire app from crashing
2021-08-31 11:02:03 -04:00
FoxxMD
885e3fa765 Merge branch 'edge' 2021-08-26 16:04:01 -04:00
FoxxMD
0b2c0e6451 Add karma threshold for recent activity rule 2021-08-26 12:04:17 -04:00
FoxxMD
15806b5f1f Add regex documentation 2021-08-26 11:26:59 -04:00
FoxxMD
bf42cdf356 Fix author criteria hash 2021-08-25 15:26:41 -04:00
FoxxMD
e21acd86db Fix maxAge optional 2021-08-25 12:57:02 -04:00
FoxxMD
5dca1c9602 Refactor caching ttl data type to be more intuitive and flexible
* (BC) instead of 0 being 'disabled' must now be 'false'
* (BC) 0 now means cache indefinitely
* 'true' is an alias for '0'
2021-08-25 12:12:41 -04:00
FoxxMD
5274584d92 Improve readability/functionality for caching functions
* Make keys for readable by using plaintext for unique values and only hashing objects
* Improve author criteria caching by excluding item identifier from hash since result should be same at subreddit-level
2021-08-25 11:49:28 -04:00
FoxxMD
1d386c53a5 Make usernote author usage blind to author type
If it has to handle cached activities this will help
2021-08-25 11:30:19 -04:00
FoxxMD
d6e351b195 Fix missing TTL settings 2021-08-25 11:21:55 -04:00
FoxxMD
ea32dc0b62 Fix shared cache namespacing and key count
* Add special SHARED namespace for subreddits using default cache -- remove ns when cache is dedicated
* Check for redis cache type and include prefix pattern when getting key count
2021-08-24 15:59:15 -04:00
FoxxMD
dca57bb19e Fix key count display
Correctly count shared and non-shared
2021-08-24 15:58:04 -04:00
FoxxMD
43919f7f9c Fix connected users since cache can now be persistent 2021-08-24 15:57:48 -04:00
FoxxMD
a176b51148 Fix storing cache user result 2021-08-24 15:08:21 -04:00
FoxxMD
75ac5297df Refactor caching prefix building to provide unique defaults 2021-08-24 14:36:13 -04:00
FoxxMD
0ef2b99bd6 Refactor web to use more caching
* Implement invite code/data to use caching
* refactor operator config so session and invite use same cache instance
2021-08-24 14:10:12 -04:00
FoxxMD
9596a476b5 Improve caching for redis
* Re-add operator-level caching config so a global default cache config can be defined
* Expand provider options with index property so additional, redis specific, params can be provided
* namespace (prefix) bot and subreddit-level (When not shared) redis connections
* refactor subreddit and author name usage since it differs when objects are deserialized
* as type guard for submission based on instance type OR object shape hint since deserialized activities are plain objects
2021-08-24 13:10:13 -04:00
FoxxMD
92f52cada5 Better implementation of user cache result
* When check is triggered also store rule results in cache -- makes actioned events more complete when actions run from cached result
* Add config option to toggle run actions on/off from cached check (Defaults to on)
2021-08-24 10:30:37 -04:00
FoxxMD
a482e852c5 Add empty state to actioned events view 2021-08-24 10:24:38 -04:00
FoxxMD
e9055e5205 Better display for actioned events
* Use checks triggered display instead of own events actions stats (since it already exists and is the same)
* More visual separation between action events using shadowed boxes
* Move timestamp and title into a header (more visual distinction) and use short hyperlink
* Move rule summary up into check field
2021-08-23 16:58:06 -04:00
FoxxMD
df2c40d9c1 Use local time format for actioned events timestamp 2021-08-23 16:56:25 -04:00
FoxxMD
fc4eeb47fa Replace newlines with spaces to make comment peeks more compact 2021-08-23 16:49:40 -04:00
FoxxMD
9fb3eaa611 Implement actioned event history
* Refactor action return value to return results/success/error
* Store action event data after triggered from manager
* Display last 25 actioned events in ui
2021-08-23 14:33:40 -04:00
FoxxMD
23394ab5c2 Use clearer wording for scope error 2021-08-23 12:16:32 -04:00
FoxxMD
5417b26417 Read operator config as json5/yaml 2021-08-23 12:10:39 -04:00
FoxxMD
b6d638d6c5 Implement easier way to determine if reddit api error is due to insufficient oauth scope 2021-08-23 12:00:49 -04:00
FoxxMD
af1dd09e2d Remove batch usernote actions for now 2021-08-23 11:51:01 -04:00
FoxxMD
c42e56c68f Fix window duration when no activities are returned from history rule subreddit filtering 2021-08-20 16:36:52 -04:00
FoxxMD
561a007850 Fix instance qs name on check url 2021-08-20 16:36:07 -04:00
FoxxMD
465c3c9acf Merge branch 'edge' 2021-08-20 15:02:24 -04:00
FoxxMD
6cee8691f5 Remove debugger statement 2021-08-20 15:02:12 -04:00
FoxxMD
cfb228de73 Update schema documentation 2021-08-20 14:52:51 -04:00
FoxxMD
82a1a393de Rewrite docs for mod/operator to include bot authentication approaches 2021-08-20 14:27:32 -04:00
FoxxMD
2fd1ffed19 Rewrite docs to explain server-client architecture 2021-08-20 13:47:55 -04:00
FoxxMD
7b00e1c54b Add more screenshots and information to web ui section 2021-08-20 12:11:36 -04:00
FoxxMD
bb2c5f076c Implement subreddit exclusion option for bot config 2021-08-20 11:08:09 -04:00
FoxxMD
8a9212def2 Use winston-duplex transport package
Dogfooding
2021-08-19 14:06:16 -04:00
FoxxMD
a9a5bd0066 Security hardening
* Update "no access" render to reflect current architecture
* Get rid of defaultBot middleware and replace with client-side parsing
* Do not reveal instance information when user does not have access to a route
2021-08-19 11:59:00 -04:00
FoxxMD
f27b4a03e9 Fix access bug and allow specifying invite code
* Fix any authenticated user being able to access dashboard (condition flipped)
* User can specify invite code so urls are friendly and can be recreated on instance restart
2021-08-19 11:12:41 -04:00
FoxxMD
ce87285283 Add schema toggle link on config editor 2021-08-19 10:55:47 -04:00
FoxxMD
220c6cdd8b Disable minimap on config editor 2021-08-19 09:30:19 -04:00
FoxxMD
17440025b9 Config interaction greatly improved
* Replace json view with monaco-editor with schema loaded based on url param
* Add route for unauthenticated config editing
* Auto-load subreddit config when "view" is clicked from subreddit view
2021-08-18 21:34:46 -04:00
FoxxMD
2655ae6041 Update schema 2021-08-18 17:52:51 -04:00
FoxxMD
a5d7b473a0 Remove undefined keys on default bot parsing 2021-08-18 16:52:41 -04:00
FoxxMD
67a04c6cc6 Use winston stream for log streaming 2021-08-18 16:42:19 -04:00
FoxxMD
c687ddbe57 Return 500 if client tries to access instance too early in init 2021-08-18 16:41:59 -04:00
FoxxMD
980ff7da02 Fix duplex transport duplicated data listeners 2021-08-18 16:41:10 -04:00
FoxxMD
0f84a7cf6b Lots of log (streaming) improvements
* Use autolinker to deal with url parsing in logs and sanitize html
* Fix missing mergeArr arguments on child loggers
* Implement DuplexTransport to make streaming from winston less verbose and allow access to log object
* Refactor log parsing on server/client side to deal with log objects or strings
2021-08-18 15:47:31 -04:00
FoxxMD
51a93439bb Implement automatic bot adding on auth callback 2021-08-17 16:35:49 -04:00
FoxxMD
18f115987b Refactor bot auth flow to use invites and require operator 2021-08-17 16:19:31 -04:00
FoxxMD
34faf56d5d Refactor how logs are handled for parsing from server based on bot/sub 2021-08-17 13:23:16 -04:00
FoxxMD
d09a2df1e0 Send web logs on status response error 2021-08-17 12:30:26 -04:00
FoxxMD
5349171913 Fix bot/manager determination on user authentication 2021-08-17 12:23:11 -04:00
FoxxMD
e283d81fdf Implement an offline view for instances 2021-08-17 11:26:48 -04:00
FoxxMD
a606d6558c More front-end updates 2021-08-17 09:29:07 -04:00
FoxxMD
cc058388d0 Log streaming refactoring (front-end) 2021-08-16 17:42:25 -04:00
FoxxMD
4bbd170c1d Second pass at refactoring to use new instance/bot/subreddit architecture (front-end) 2021-08-16 16:53:57 -04:00
FoxxMD
c817716aa1 First pass at refactoring to use new instance/bot/subreddit architecture 2021-08-16 14:14:42 -04:00
FoxxMD
33f9b4a091 Fudge the client for now... 2021-08-13 16:26:07 -04:00
FoxxMD
8d8e4405e0 Implement graceful bot teardown and api endpoint to rebuild bot 2021-08-13 15:52:18 -04:00
FoxxMD
ee302ee430 Reorganize routing for server
* Much more readable and maintainable with separate files for the big routes
* Properly extend express.Request interface
2021-08-13 13:53:28 -04:00
FoxxMD
acbac54903 More cases to cover when bot is not running or in first-time setup 2021-08-12 15:05:34 -04:00
FoxxMD
3858070cee Restrict url access per bot based on web operator or bot operator status 2021-08-12 14:02:59 -04:00
FoxxMD
ac5ace1f61 Use forwarded IP header if available when logging user access 2021-08-12 14:02:35 -04:00
FoxxMD
3d79a9217a Refactor the oauth helper flow
* Integrate into the main client
* Do more checks for correct credentials before auth code flow init
* Implement special bypass token to help with authorization flow with account other than operator/logged in
* Add more credential options to helper page (can provide separate id/secret) and permission options
2021-08-12 11:57:54 -04:00
FoxxMD
4b6261517c Implement log streaming retries and fix log settings
* Max 5 retries using got built-in behavior, with logged warnings/errors
* Reload client window after log setting update as its easier to do setup logs from a fresh load then in-situ
2021-08-11 15:59:57 -04:00
FoxxMD
d1960c68bb Add logging for when user connects to logs 2021-08-11 14:54:20 -04:00
FoxxMD
a8cc40e95d Cleanup streams and add default status data when no connection 2021-08-11 14:54:05 -04:00
FoxxMD
5c76f9ab1c Continue to simplify stream handling
* Cleanup stream usage for winston and simplify adding to logger
* Use pipeline with delimiter stream for parsing logs from client
* End log streaming with promise abort (cleaner)
2021-08-10 23:10:06 -04:00
FoxxMD
a5d3c809aa Better handling for clients with bad connections 2021-08-10 17:14:07 -04:00
FoxxMD
3b905e6961 General improvements for ui and error handling
* Move Bot webhook listeners for crash/shutdown into Bot code itself
* Simplify running modes
* Parse web operator into user data
* Add basic status tooltip for bots in ui
2021-08-10 16:39:39 -04:00
FoxxMD
707547effc Implement separate credentials for web 2021-08-10 12:02:03 -04:00
FoxxMD
6b02350d96 Routing fixes and reduce complexity for determining accessible managers 2021-08-10 10:27:41 -04:00
FoxxMD
7ff8094156 More ui improvements for new architecture 2021-08-09 22:56:29 -04:00
FoxxMD
82c673c8a6 More cleanup of ui 2021-08-09 22:23:40 -04:00
FoxxMD
7f742d3a30 Feature parity for main page 2021-08-09 21:32:20 -04:00
FoxxMD
2442fc2483 More refactoring for client/server decoupling
Most things working now...
2021-08-09 17:44:55 -04:00
FoxxMD
e762cc29ef Big ol mess for now 2021-08-06 22:04:29 -04:00
FoxxMD
88db6767eb Remove extra tags for major/minor 2021-08-05 14:55:54 -04:00
FoxxMD
161251a943 Merge branch 'edge' 2021-08-05 14:40:06 -04:00
FoxxMD
6e4b1b68e3 Update tag trigger pattern 2021-08-05 13:53:30 -04:00
FoxxMD
a6212897b3 Increase cache hit sample size 2021-08-04 14:44:07 -04:00
FoxxMD
7b8a89e918 Add cache stat averages for key hits 2021-08-04 14:40:47 -04:00
FoxxMD
efd31c5f21 Improve log streaming 2021-08-04 12:45:22 -04:00
FoxxMD
868bac9f1a Fix dry run parsing for arguments 2021-08-04 12:44:51 -04:00
FoxxMD
adf18cc7ee Better check for default cache key count
Look for the first manager using the default rather than assuming the first one configured will have it
2021-08-04 09:56:09 -04:00
FoxxMD
3f1d1bc6d0 Rewrite age regex for schema to not use named capture groups (just regular capture groups) 2021-08-04 09:55:32 -04:00
FoxxMD
ce4cb96d9a Merge branch 'edge' 2021-08-03 23:39:14 -04:00
FoxxMD
4457e3957d Implement json pretty print to html on config view 2021-08-03 23:38:48 -04:00
FoxxMD
c317f95953 Merge branch 'edge' 2021-08-03 22:43:02 -04:00
FoxxMD
2eda6c5fe1 Typo fix 2021-08-03 22:42:42 -04:00
FoxxMD
1108216a50 Improve formatting for cache section
* compact the calls/miss stats into one line
* combine cache type stats and description
* improve cache tips documentation
2021-08-03 21:22:03 -04:00
FoxxMD
b9215e944a Fix batch usernote save logic
* timeout returns no argument but we have bound the function so just reassign this to self
* fix errenous save left over from previous changes
* keep track of number of usernotes to save and log to debug
* log to debug when executing save immediately due to cache miss
2021-08-03 15:08:02 -04:00
FoxxMD
a976171e3a Merge branch 'develop' into batchUserNotes
# Conflicts:
#	src/Subreddit/UserNotes.ts
2021-08-03 14:36:20 -04:00
FoxxMD
b773afbe38 Add comment check stats to cache breakdown 2021-08-03 14:29:12 -04:00
FoxxMD
045e2c1d33 Fix check cache result behavior
* Don't set result in cache if it already exists
* Fix resource manager init issue with cache manager?
* Make check cache logging statements clearer
2021-08-03 14:25:11 -04:00
FoxxMD
ad45f75267 More caching improvements
* Refactor cache-manager creation to use built in "none" store to simplify cache usage
* Implement comment check cache result flow
2021-08-03 13:06:34 -04:00
FoxxMD
643790d3bd Add 'enable' parameter for actions config 2021-08-03 10:54:50 -04:00
FoxxMD
a531d7e4e0 Add submission state for link flair text/css 2021-08-03 10:01:05 -04:00
FoxxMD
be065f919c Implement POC batch usernotes save flow 2021-08-02 21:23:04 -04:00
FoxxMD
8d5d44bf0d Update schema 2021-08-02 16:46:49 -04:00
FoxxMD
bbd8a6633e Implement check enabled state
To make it easier to turn on/off a check without having to comment out the entire thing in config
2021-08-02 16:46:07 -04:00
FoxxMD
038e5d086b Fix missing check on optimization 2021-08-02 16:05:40 -04:00
FoxxMD
5422b181c0 Optimize isItem test when only testing comment's submission state 2021-08-02 16:01:30 -04:00
FoxxMD
d0e0515990 Merge branch 'edge' 2021-08-02 15:44:57 -04:00
FoxxMD
931dfa67fd Display more cache info in web interface
* Show total miss and percent
* Move breakdown into tooltip
* Show item crit, submission, and comment cache stats in breakdown
2021-08-02 15:44:25 -04:00
FoxxMD
af1ea5543e Implement caching for more components
* Implement caching for specific activities
* Implement/refactor item is criteria to cache activities and results
2021-08-02 15:10:49 -04:00
FoxxMD
fd7a6edeb6 Fix activity window criteria checks for 'any' condition
Should check time before count
2021-08-02 09:19:02 -04:00
FoxxMD
0a3409cfef Add title matching on SubmissionState filter 2021-08-01 22:07:19 -04:00
FoxxMD
89b2932495 Missed some slashes 2021-07-31 23:27:47 -04:00
FoxxMD
3a05e43ce9 Clearer error message on wiki content failed response 2021-07-31 23:08:32 -04:00
FoxxMD
8b1d3cb170 Implement MessageAction
* enable bot to send private messages as self or modmail
* add necessary permissions to oauth helper to make this possible
* update schema with message action structure
2021-07-31 15:05:49 -04:00
FoxxMD
90df5f45a8 Add parent submission state testing when checking a comment activity 2021-07-31 14:22:17 -04:00
FoxxMD
ba4b4a69a7 Better formatting for downloaded wiki content 2021-07-31 12:35:08 -04:00
FoxxMD
e3d4ffa36d Disable cloning on cache
* Lodash has issues iterating over properties because the items can be a `proxy`
* We're not modifying the items at any point anyway so cloning to preserve state isn't necessary
* We want to get proxy objects back (potentially) since we'll still be able to use them for retrieving more data later
2021-07-31 12:29:06 -04:00
FoxxMD
cdddd8de48 Merge branch 'edge' 2021-07-30 18:17:38 -04:00
FoxxMD
7f1429395c Fix links in getting started 2021-07-30 18:17:13 -04:00
FoxxMD
f598215d88 Merge branch 'edge' 2021-07-30 14:46:51 -04:00
FoxxMD
c92e6775cb Add config download option 2021-07-30 14:38:12 -04:00
FoxxMD
2a5f812dba Implement defining multiple operators 2021-07-30 14:19:49 -04:00
FoxxMD
54905da782 Implement extended memory store to allow pruning 2021-07-30 13:48:58 -04:00
FoxxMD
5f30dd8ce9 Refactor caching interface to simplify
No need to have it be optional string provider and it unnecessarily complicates json schema/configuration
2021-07-30 12:37:07 -04:00
FoxxMD
547f57b99f Some configuration override clarification 2021-07-30 12:05:07 -04:00
FoxxMD
bf336ca55a Move the schema/settings for operator into json schema (viewer)
easier to read, has full examples, and validation
2021-07-30 12:00:42 -04:00
FoxxMD
4716ac8c0a Add docs for thresholds and expand caching docs 2021-07-30 10:43:05 -04:00
FoxxMD
79a518edbc Move the getting started section up so its more visible 2021-07-30 10:06:09 -04:00
FoxxMD
b72a3fea7f Fill out some TODOs for docs (checks, filters) 2021-07-30 10:04:51 -04:00
FoxxMD
58603f17f4 Mention the schema editor in more places 2021-07-30 10:04:49 -04:00
FoxxMD
99b5a01835 Clean up subreddit-ready examples 2021-07-30 10:04:40 -04:00
FoxxMD
fd41c23128 Add subreddit-ready examples #14 2021-07-29 17:00:00 -04:00
FoxxMD
3230c4b30b Update repeat activity to be activity agnostic 2021-07-29 16:46:41 -04:00
FoxxMD
38507c8990 Fix anchor 2021-07-29 15:21:13 -04:00
FoxxMD
136098354b Improve getting started documentation #14 2021-07-29 15:19:56 -04:00
FoxxMD
29fc9a3a2d Add maxWorker to operator config docs 2021-07-29 13:39:58 -04:00
FoxxMD
0c7218571c Merge branch 'edge' 2021-07-29 13:25:16 -04:00
FoxxMD
fd4c2a38e7 Implement max queue workers as configurable
Globally or per-subreddit (max global)
2021-07-29 13:15:32 -04:00
FoxxMD
f89dca5d77 Use botname in more places
* Add operator option for naming the bot, otherwise default to authenticated account name
* Use botname as discord webhook name
2021-07-29 12:02:36 -04:00
FoxxMD
acc7c49e0e Merge branch 'edge' 2021-07-29 11:27:42 -04:00
FoxxMD
7175965e3d Update missed naming changes 2021-07-29 11:26:26 -04:00
FoxxMD
3ec7d3530d Update actions to not trigger on documentation updates 2021-07-29 11:23:14 -04:00
FoxxMD
01839512d5 Merge branch 'edge' 2021-07-29 11:14:33 -04:00
FoxxMD
d37958e5c8 Update links and branding in web server pages 2021-07-29 11:14:09 -04:00
FoxxMD
bfbbb3466a Update footer link 2021-07-29 11:01:11 -04:00
FoxxMD
775613374b Update dockerhub image tagging
* Update repository to new naming
* Use 'edge' instead of 'develop'
2021-07-29 10:51:43 -04:00
FoxxMD
44c8bd9a6a Rebrand to conform to reddit 3rd party app naming guidelines
They don't like having reddit as part of the name unless its "for reddit" -- so just remove reddit from name altogether and also drop "bot" as its superfluous. Shorter name is better anyway
2021-07-29 10:50:58 -04:00
FoxxMD
45e61b8bc7 Update tooltips 2021-07-29 10:29:20 -04:00
FoxxMD
4680640b0c Merge branch 'develop' 2021-07-28 16:58:36 -04:00
FoxxMD
897802b234 Update npm start command 2021-07-28 16:04:49 -04:00
FoxxMD
82b353c6d9 Update package-lock 2021-07-28 15:37:59 -04:00
FoxxMD
254d2ca896 Add local install requirements 2021-07-28 15:37:44 -04:00
FoxxMD
5a531f0122 Remove remaining js/map files from src 2021-07-28 13:57:54 -04:00
FoxxMD
0afd87ab1b Doc improvements
* Reorganize some main readme contents into separate docs
* Add full bot authentication guide
* Add screenshots and web interface section

Some link fixes and clarifications

Fix another link

Fix another link

More doc cleanup

More doc cleanup

More doc cleanup

Add link to docs in main readme summary
2021-07-28 13:47:24 -04:00
FoxxMD
c1ab3b11f4 Ignore github folder for docker build 2021-07-28 11:31:44 -04:00
Matt Foxx
222fe0aeac Create dockerhub.yml
(cherry picked from commit b813ebdd96)
2021-07-28 11:31:26 -04:00
FoxxMD
ceb98d04bb Update application version 2021-07-28 11:30:04 -04:00
Matt Foxx
b813ebdd96 Create dockerhub.yml 2021-07-28 11:27:04 -04:00
FoxxMD
4865259ae8 Improve app exit notifications and fix wiki location
* Refactor server so app is passed back to main index.js so we can handle SIGTERM in a central location and determine if exit was based on uncaught error or not
* Fix missing assignment of default wikiLocation to manager
* await discord notifier so on app exit the notification is actually sent before exit
2021-07-28 10:45:12 -04:00
FoxxMD
2616439f5f Better formatting for check triggered notification 2021-07-28 09:35:17 -04:00
FoxxMD
0eddac35fa Fix per subreddit cache request rate display 2021-07-28 09:33:36 -04:00
FoxxMD
fb3047ca82 Remove error wording (may be normal shutdown) 2021-07-27 13:08:56 -04:00
FoxxMD
193ecfba2f Fix truthy condition for notifyOnTrigger 2021-07-27 12:56:53 -04:00
FoxxMD
ef3475e519 Add a blank notification manager for initial manager creation 2021-07-27 12:42:31 -04:00
FoxxMD
a003e18360 Add log statement when sending notification 2021-07-27 12:33:27 -04:00
FoxxMD
6b6124d76e Implement event hooks for notifications
* Implemented a generic notification manager with several event types
* Implemented discord notification provider
2021-07-27 12:24:36 -04:00
FoxxMD
e4f18e8f06 Fix cache call request rate when over 10 minutes 2021-07-26 18:45:53 -04:00
FoxxMD
24963ec333 More op doc fixes 2021-07-26 17:10:12 -04:00
FoxxMD
2ab6ee3715 Typo fix 2021-07-26 17:07:07 -04:00
FoxxMD
802884f686 Add operator configuration documentation 2021-07-26 17:06:02 -04:00
FoxxMD
67ed8ab4ee Add more caching info and tooltips for major components 2021-07-26 14:47:58 -04:00
FoxxMD
1e6d61ac31 Add 401 http status to retry handler
Seen this occasionally on polling and is actually a reddit issue, not client, so should be able to retry on it
2021-07-26 12:20:10 -04:00
FoxxMD
7cda47183f Simplify cli options and improve web server checks
* Check for in-use web port before starting so we can display a useful error message
* Remove 'web' command and instead make it an optional argument for `run` (default to web)
* Update dockerfile to use run command
* Remove remaining commander option defaults since they are defined in config builder now
2021-07-26 12:18:51 -04:00
FoxxMD
a9edd4b998 Fix miss percent formatting 2021-07-26 09:55:03 -04:00
FoxxMD
9e1d5f1dd4 Ui polish
* Add more caching information
* Add more visual distinction for selected subreddit
* Add querystring based on shown subreddit so view persists between reload
2021-07-23 17:03:30 -04:00
FoxxMD
4617e06459 Fix missing await on usernote cache call 2021-07-23 16:44:21 -04:00
FoxxMD
b2b4988246 Consolidate all caching
* Use cache manager instance in UserNotes and callback to report stats
* Simplify sub resource config interface and setter
* Use cache manager for express session
* Add session cache provider to operator config
2021-07-23 14:48:33 -04:00
FoxxMD
e1c24133dd Use name instead of user id for banning 2021-07-23 13:26:53 -04:00
FoxxMD
12a4e0436e Simplify resource stats generation 2021-07-23 11:46:37 -04:00
FoxxMD
484931d8b5 Fix bool middleware running on wrong route 2021-07-23 11:10:53 -04:00
FoxxMD
abf2674f80 Detect noisy html errors from reddit 2021-07-22 21:37:43 -04:00
FoxxMD
1f3cfbeef9 Fix cache miss value comparison 2021-07-22 21:37:08 -04:00
FoxxMD
2b21885a55 Fix default cache stuff when subreddit manager isn't built 2021-07-22 17:59:45 -04:00
FoxxMD
232925e691 Maybe fix error on ban 2021-07-22 17:53:47 -04:00
FoxxMD
a91b9ab146 Use more robust caching implementation
* use node-cache-manager so operator has a choice of memory or redis
* BC update TTL values to be in seconds instead of milliseconds
* Count requests and misses for cache
* display cache stats in ui
2021-07-22 17:47:19 -04:00
FoxxMD
73c3052c69 Refactor how configuration is parsed
* Implement a weak/strong interface for operator (app) config
* Consolidate config parsing code into ConfigBuilder
* Implement parsing configuration from a json file
* Refactor configuration parsing so there is a clear hierarchy to where and how config is overridden
2021-07-22 15:17:41 -04:00
FoxxMD
4fbb3edf8b Lots of regex fixes 2021-07-21 16:50:49 -04:00
FoxxMD
c69d66c001 Initial implementation of Regex Rule 2021-07-21 15:47:26 -04:00
FoxxMD
9b53974152 Fix formatting for rolling average 2021-07-21 12:14:28 -04:00
FoxxMD
13d3ed2314 Improve repeat pattern detection for repeat activity rule
* Fuzzy match on all identifiers to enable detecting small variations in title/body
* For reddit media submissions (image/video) check title instead of url
* For submissions with external url also check title
2021-07-21 12:09:51 -04:00
FoxxMD
9d7505fa38 More UI improvements
* Move theme toggle to title to make room in the subreddit tabs when there are many, many subreddits
* Implement check permalink with subreddit context -- can run any permalink on a subreddit's config
2021-07-21 12:07:37 -04:00
FoxxMD
1b94316987 Fix log level for api nanny 2021-07-20 20:30:06 -04:00
FoxxMD
9316019b01 Implement api nanny mode to help with heavy api usage 2021-07-20 20:15:15 -04:00
FoxxMD
4642f67104 Implement loading operator arguments (main program) from file 2021-07-20 16:38:35 -04:00
FoxxMD
a78692d7a0 Even more polish 2021-07-20 13:11:37 -04:00
FoxxMD
3627a5f60a More UI polish 2021-07-20 12:48:32 -04:00
FoxxMD
6b04ea0a9d Compact stats/info 2021-07-20 00:57:47 -04:00
FoxxMD
f6217547ae Some ui cleanup 2021-07-20 00:23:48 -04:00
FoxxMD
f1b24eb4a2 Add unmoderated/modqueue check actions to ui 2021-07-19 23:16:27 -04:00
FoxxMD
c9bdae66dd Implement configurable shared modqueue for operator convenience
If operator is running subreddits for lop-sided traffic burdens then /r/mod may be over the 100 activity limit, all from subreddit X while sub Y and Z have few activities. In this scenario Y and Z would never run since X would take up all results. Therefore default to individual modqueues and make shared modqueue opt-in by operator
2021-07-19 23:04:47 -04:00
FoxxMD
6ab162888b try-catch all the things 2021-07-19 22:37:50 -04:00
FoxxMD
dd202ac790 Add console to error/rejection transports 2021-07-19 22:37:39 -04:00
FoxxMD
eead88c9a7 Remove secondary client usage which was causing bug
For some reason using client *again* here was causing the primary client usage to lose response data. I think perhaps it needs to be cloned or need to create a new instance all-together rather than trying to re-use.
2021-07-19 22:37:22 -04:00
FoxxMD
f548be8060 Fix "All" actions type 2021-07-19 16:44:03 -04:00
FoxxMD
5df4fd5ccc Separate event polling and activity processing systems to improve robustness
* Use async/queue for handling activity processing on each manager
* Refactor polling to push activities into activity queue
* Refactor manager state to control activity processing queue and event polling independently
* Pause activity processing and wait until no work is being done to do config update
* Add way more logging for new systems
* Add basic ui controls/view for new systems
2021-07-19 16:32:41 -04:00
FoxxMD
b25001b7af Reduce false positives for repeat self/comment activities
* For self submissions use "title + body" to identify repeats instead of just body -- body could potentially be empty
* Add "minWordCount" with default of 1 so that blank comments are ignored
2021-07-18 21:30:44 -04:00
FoxxMD
8733717cda Remove submission title on comment item peek
Erroneously returning proxy object instead of actual submission. Dont' really need the title since its in the permalink so remove it entirely to avoid another api call
2021-07-18 21:05:56 -04:00
FoxxMD
6167d45e49 Refactor configuration loading to be independent of manager initialization
* Makes it easier to deal with never-successful subs (from startup) and invalid configs during run
* Paves the way for adding managers in-situ
* Add validConfig property to manager to track this and in UI
* Track if user manually stopped manager so we don't try to restart on heartbeat
2021-07-16 16:57:25 -04:00
FoxxMD
694842226b Cover case where user declines oauth 2021-07-16 15:35:25 -04:00
FoxxMD
02ad661226 Cover case where outside user logs in 2021-07-16 15:18:38 -04:00
FoxxMD
3be62f5560 Use partials for rendering head/title 2021-07-16 15:18:01 -04:00
FoxxMD
eb84df5133 Set dark theme as default 2021-07-16 12:12:17 -04:00
FoxxMD
4b11e348ba Update some logging to be more subreddit specific 2021-07-16 12:08:19 -04:00
FoxxMD
9edacf29fa Refactor log tracking to handle limit's per suberddit
Makes the 'limit' option in UI more true -- keep track of $max number of logs per subreddit and then combine at render time
2021-07-16 11:50:34 -04:00
FoxxMD
56c13474d9 Cover edge case where logs don't exist for a subreddit 2021-07-16 10:15:47 -04:00
FoxxMD
66a4144b7b retry on all stream types 2021-07-16 10:11:17 -04:00
FoxxMD
16880775fb Increase max polling retries
If reddit has a general outage this is where we want to wait the longest before giving up entirely since polling is infrequent (relative to running checks)
2021-07-16 10:11:03 -04:00
FoxxMD
d69d0e972c End stream when error occurs
Stream should also stop polling. It's up to the retry handler and individual usage of stream in app/manager to restart it
2021-07-16 10:10:18 -04:00
FoxxMD
a9350c2828 Fix retry handler and broaden request conditions
* fix other retry compare statement
* check request for status code missing (timeout, socket timeout, address unavailable, etc.) or valid status code
* clarify wording in retry handler logging
2021-07-16 10:09:38 -04:00
FoxxMD
2fe06f21d9 Fix stats margin 2021-07-15 23:04:09 -04:00
FoxxMD
42d71a918f Implement URL checker 2021-07-15 22:58:20 -04:00
FoxxMD
0aa2b24f39 Implement actions 2021-07-15 20:22:27 -04:00
FoxxMD
4771efa32a Add status indicator to subreddit tab name 2021-07-15 20:01:07 -04:00
FoxxMD
1d9f4f32b8 Implement logout and reduce footprint of light dark toggle 2021-07-15 19:54:12 -04:00
FoxxMD
d84e6f1905 Implement oauth helper ui 2021-07-15 19:34:01 -04:00
FoxxMD
ae19d1c9a1 Fix log check 2021-07-15 17:10:31 -04:00
FoxxMD
f9c7cf433e Switch to web as default run command and provide fallback
* Use 'web' as run command for dockerfile
* If on 'web' there is no redirect uri fallback to cli (doesn't break compatibility this way)
2021-07-15 15:44:47 -04:00
FoxxMD
2917233728 More startup logging and better logging on invalid/missing credentials 2021-07-15 15:31:56 -04:00
FoxxMD
6dfb5823ba Add web ui port as configurable 2021-07-15 15:02:41 -04:00
FoxxMD
14e7275f64 Implement operator-specific UI components
* Add api limit, reset, and heartbeat interval to All overview when operator is viewing
* Stream op stats on log emit
* Add env/arg for setting "Operated by"
2021-07-15 14:57:05 -04:00
FoxxMD
1fbe6b708d More layout improvements 2021-07-15 13:37:05 -04:00
FoxxMD
495213bba9 Improvements to web ui
* Implement stats tracking per manager
* Render stats per sub and overall in ui
* Clean up ui layout
2021-07-15 12:54:32 -04:00
FoxxMD
15e031d448 Implement web ui
* Refactor program options to allow running as web
* Implement authentication using reddit oauth
* Use session in memory to store identification and accessible subreddits
* Implement socket.io with shared session to enable streaming logs
* Implement log streaming with per-subreddit views
2021-07-15 01:18:21 -04:00
FoxxMD
6994bbe770 Remove errenous delayUntil default 2021-07-13 14:32:46 -04:00
FoxxMD
a3c923bda6 Fix passing dryrun arg 2021-07-13 10:31:29 -04:00
FoxxMD
a40c4c5e58 Fix erroneous debug change for heartbeat interval 2021-07-12 13:45:19 -04:00
FoxxMD
be9dcdee1c Refactor polling streams to reduce usage and improve error catching
* Re-implement all polling streams with extended poll class for catching errors
* Increase default polling interval to 30 seconds and limit to 50
* When using default interval/limit for mod polling use a shared stream between all managers and poll to /r/mod -- reduces api calls for polling mod streams to 1 regardless of how many subreddits are managed
2021-07-12 13:44:07 -04:00
FoxxMD
07b34caffb Add proxy options and handle polling errors
* Implement proxied snoowrap requests
* Extend snoostorm class so timeout/errors during polling can be caught
2021-07-12 10:01:53 -04:00
FoxxMD
c5a3404242 Remove erroneous debugging addition 2021-07-08 14:13:02 -04:00
FoxxMD
1e03b38f0a Add polling info on options build 2021-07-07 17:19:56 -04:00
FoxxMD
f64be77e70 Change log level of processing delay statement 2021-07-07 17:02:00 -04:00
FoxxMD
a3da77874b Ignore doc updates for docker 2021-07-07 16:42:06 -04:00
FoxxMD
a9f740c9fa Fix example links 2021-07-07 16:40:45 -04:00
FoxxMD
00e6346cdb Fix wording 2021-07-07 16:38:55 -04:00
FoxxMD
951359ac39 Add tuning documentation for repeat activity 2021-07-07 16:10:24 -04:00
FoxxMD
15824e5d0f Implement delay before processing
Using criteria for Activity being N seconds old allows for delay with possibility of immediate processing to avoid api call

Closes #23
2021-07-07 15:23:59 -04:00
FoxxMD
e7c794ec85 Fix activities example 2021-07-07 13:16:04 -04:00
FoxxMD
70e426de7e Fix dajys link 2021-07-07 13:14:27 -04:00
FoxxMD
cc2518d086 Refactor documentation and ENHANCE 2021-07-07 13:06:51 -04:00
FoxxMD
5517c75d4c Update examples 2021-07-07 09:44:12 -04:00
FoxxMD
8e2fee6d50 Fix heartbeat interval 2021-07-06 16:52:45 -04:00
FoxxMD
ed8be6dda2 Refactor app/manager building for in-situ updates
* Separate manager instantiation from configuration flow so config can be reloaded
* Move wiki page parsing into manager for better encapsulation
* Check for wiki revision date on heartbeat and on checks if older than one minute
* Catch config parsing issues and retry on next heartbeat
2021-07-06 16:28:18 -04:00
FoxxMD
00e38b5560 Use correct media property for anchor parsing 2021-07-06 12:59:46 -04:00
FoxxMD
9cac11f436 Implement author parsing for audio/podcast related media
Parse spotify and anchor.fm media sources
2021-07-06 11:33:00 -04:00
FoxxMD
f591c3a05a Implement more powerful content parsing options
* Can get wiki pages from other subreddits
* Can fetch from an external url
2021-07-06 10:27:30 -04:00
FoxxMD
39fad91c7f Fix missing author criteria 2021-07-05 17:20:35 -04:00
FoxxMD
529b8fc03e Further improvements for subreddit name parsing
* Allow whitespace on either side of regex value to parse since its automatically trimmed by getting capture group
* Implement sub name parsing everywhere subreddits can be specified and update documentation to remove prefix restrictions
2021-07-05 16:06:15 -04:00
FoxxMD
54eef5620d Update interfaces and documentation for new filters and item states 2021-07-05 15:39:08 -04:00
FoxxMD
99537fbebb Fix missing filtering behavior on repeat and add remove check
* Add missing include/exclude behavior for counting repeat submissions
* Add parameter to enable user to specify if removed activities should be included
2021-07-05 15:38:49 -04:00
FoxxMD
4c3f9ee082 Fix remove check on remove action 2021-07-05 15:37:37 -04:00
FoxxMD
5b028b6a45 Fix some item state checks and implement subreddit filtering on window
* Fix how removed check is performed since there are different behaviors for submission/comment
* Add filtered and deleted states for item check
* Add subreddit filters (include/exclude) on window criteria
2021-07-05 15:37:19 -04:00
FoxxMD
859bcf9213 Implement subreddit name parser to allow more lax input
Use regex to extract subreddit name regardless of prefix
2021-07-05 15:34:59 -04:00
FoxxMD
e790f7c260 Fix issue when activities retrieved for attribution rule 2021-06-25 15:20:31 -04:00
FoxxMD
20358294ce Fix domain ident aliases when not aggregating parent domain 2021-06-25 10:29:19 -04:00
FoxxMD
e0f18dc0a2 Add typescript dep 2021-06-25 10:28:39 -04:00
FoxxMD
9a788a8323 Add file logging for uncaught rejection/exceptions 2021-06-23 16:05:58 -04:00
FoxxMD
bed9a9682a Add item is key in log on found 2021-06-23 15:17:45 -04:00
FoxxMD
d39ce13209 Big improvements for Attribution Rule
* Move most qualifier properties into criteria
* Allow filtering on specific domains and consolidate "use sub as reference" into a special string to use here
* Above also enables using rule on Comment
* Enable more submission type filtering options and allow multiple options (link, media, self -- or a combination of any)
* Provide count and percent ranges in results
* Provide domains and friendly tiles in results
2021-06-23 15:09:56 -04:00
FoxxMD
4bd25e53b0 Better formatting for logging errors
Don't erase the original message if there is one
2021-06-23 14:56:20 -04:00
FoxxMD
ac87d5acfa Check if Activity is deleted before running checks 2021-06-23 14:55:45 -04:00
FoxxMD
0f541f1961 Try to fix attribution undefined issue on criteria results array having undefined index 2021-06-22 16:26:30 -04:00
FoxxMD
db2be949b4 Refactor polling to enable polling from multiple sources
* New polling sources: unmoderated and modqueue
* Can now poll from many sources
2021-06-22 14:47:20 -04:00
FoxxMD
8c6b18cf4d Add optional subreddit nickname to help make logs more readable 2021-06-22 13:00:53 -04:00
FoxxMD
add4204304 Make check labels more concise
* Shorten prefix to 'CHK'
* Truncate check names to 25 characters
2021-06-22 13:00:35 -04:00
FoxxMD
927d4ef07e Refactor recent activity to provide useful information on fail 2021-06-22 12:42:43 -04:00
FoxxMD
b8c12009ee Refactor history rule to provide useful logging information on fail 2021-06-22 11:59:43 -04:00
FoxxMD
7f9b4ce6a0 Refactor repeat rule to provide useful logging information on fail 2021-06-22 11:29:07 -04:00
FoxxMD
ad8a668a08 Make pass/fail symbols constants 2021-06-22 11:28:37 -04:00
FoxxMD
84c5e97c92 Refactor Attribution so fail condition also logs useful information 2021-06-22 10:52:29 -04:00
FoxxMD
03b2cb36ab Change pass/fail symbols so they aren't emojis
Makes logging more uniform
2021-06-22 10:52:01 -04:00
FoxxMD
93bdb89115 Shorten action logging label 2021-06-22 10:51:29 -04:00
FoxxMD
702e2ccccf Shorten Rule labels 2021-06-22 10:47:51 -04:00
FoxxMD
631d67928d Shorten kind for some rules to make logging more readable 2021-06-22 10:47:13 -04:00
FoxxMD
eea04344c0 Check for error timeout code 2021-06-21 22:53:50 -04:00
FoxxMD
7f29ade87b Refactor results to make check complexity easier to visualize in log
* Differentiate rule results from rule set results
* Implement function to parse all results for check and print and/or PLUS triggered state for all rule/sets in check
2021-06-21 17:10:24 -04:00
FoxxMD
cced86381b Fix minimum items for subthreshold 2021-06-21 13:57:31 -04:00
FoxxMD
01c575f2b2 Rename history criteria operator name to be consistent 2021-06-21 13:34:27 -04:00
FoxxMD
f1d04d4718 Fix regex named group 2021-06-21 13:33:49 -04:00
FoxxMD
6ca65079b3 Add author and item checks to Actions
So that Actions can be skipped based on item/author criteria
2021-06-21 13:00:45 -04:00
FoxxMD
73236e44ad Add karma comparisons and verified check for author criteria 2021-06-21 12:02:33 -04:00
FoxxMD
4bef85e1e4 Refactor most thresholds to use comparison operators (like automod) 2021-06-21 11:40:04 -04:00
FoxxMD
532f6aa3d8 Fix recent activity default thresholds 2021-06-20 14:20:39 -04:00
FoxxMD
e1e5b26264 One more duration fix 2021-06-20 14:15:06 -04:00
FoxxMD
46a583e20a Fix regex constant usage for duration 2021-06-20 14:01:00 -04:00
FoxxMD
24064dfe03 Refactor how activity window is parsed to use dayjs shorthand string
* Parse dayjs shorthand IE "90 days" alongside the rest of the activity types
* Update schema with clearer information on how activity window can be formed
2021-06-18 17:36:43 -04:00
FoxxMD
ad91901cc2 Refactor history criteria to use string regex parsing (like automod) 2021-06-18 16:33:13 -04:00
FoxxMD
58c51e56b1 Test Author account age using string comparison (like automod) 2021-06-18 13:52:52 -04:00
FoxxMD
9850ccb8f3 Fix dryrun usage for comment action 2021-06-17 15:04:11 -04:00
FoxxMD
79b82dab0f Refactor footer to be configurable at subreddit and Action level 2021-06-17 14:43:22 -04:00
FoxxMD
9c059beb85 Add usernotes to content template render data and contextualize bot footer 2021-06-17 13:13:46 -04:00
FoxxMD
88be7d8836 Missed ban case in action factory 2021-06-16 22:42:46 -04:00
FoxxMD
20acc12460 Add footer to comment/ban content
Closes #5
2021-06-16 22:29:28 -04:00
FoxxMD
60c0569e21 Add missing actions
Closes #19
2021-06-16 22:04:44 -04:00
FoxxMD
879807390d Add YAML as configuration language to readme 2021-06-16 21:30:43 -04:00
FoxxMD
08413dbe16 Implement YAML parsing 2021-06-16 21:27:48 -04:00
FoxxMD
75cbde8b8b Improve logging levels and add end-run stats
* Increase some noisy log statements to verbose
* Display action summary on info level
* verbose -- Display run stats (checks, rules, actions) at end of event
* verbose -- Display reddit api stats (initial, current, used) at end of event
2021-06-16 13:31:37 -04:00
FoxxMD
3acf268313 Check notes length before trying to get current note 2021-06-16 12:30:28 -04:00
FoxxMD
97b9391f3b Remove debug statement derp 2021-06-16 12:21:19 -04:00
FoxxMD
f8ec0d7ee0 Fix and/or condition logic for checks and rulesets 2021-06-16 12:20:38 -04:00
FoxxMD
0002c1bc11 Allow rules to be optional, increase startup logging, and change default log level
* Allow rules to be optional on json -- if no rules actions are run immediately after check passes author/item tests
* When verbose logging show much more detail about check stats, rules, and actions on startup
* Set verbose as default log level. New users should have more information of the box so they can understand how things work.
2021-06-16 11:38:45 -04:00
FoxxMD
a09f3fe4f1 Finally got action example working correctly 2021-06-16 10:39:27 -04:00
FoxxMD
daf66083d0 Schema documentation improvements 2021-06-16 10:33:36 -04:00
FoxxMD
7acd62d787 Refactor window criteria to actually work as described 2021-06-16 00:27:04 -04:00
FoxxMD
75889cc927 Add more error handling for reddit timeout issues 2021-06-15 17:05:14 -04:00
FoxxMD
db0440356c Refactor author usage to be more universal and change name to match item behavior
* (BC) rename authors to authorIs to match itemIs -- since they have the same behavior
* Add author filter to Check so it matches usage of itemIs
2021-06-15 16:12:06 -04:00
FoxxMD
016952128c Update documentation for Toolbox User Notes 2021-06-15 15:31:59 -04:00
FoxxMD
884966b8d3 Remove more debugging statements
ugh
2021-06-15 14:33:47 -04:00
FoxxMD
0ad7c66e9d Fix error display on config error 2021-06-15 14:26:42 -04:00
FoxxMD
c075e5fb24 Remove debug statements 2021-06-15 14:11:07 -04:00
FoxxMD
a3de885620 Add some action checks 2021-06-15 14:04:06 -04:00
FoxxMD
e29d19ada8 Implement itemIs test for Checks and Rules 2021-06-15 13:54:54 -04:00
FoxxMD
c52e1d5e1d Implement toolbox usernote action 2021-06-15 12:09:26 -04:00
FoxxMD
257563a3b8 Implement toolbox usernote author filter and rule criteria 2021-06-15 10:39:16 -04:00
FoxxMD
7761372091 Implement toolbox usernote read/write 2021-06-14 22:45:48 -04:00
FoxxMD
eb62e39975 Add unmoderated run command 2021-06-14 10:26:05 -04:00
FoxxMD
bdd72dc28e Add more schema examples 2021-06-12 00:54:23 -04:00
FoxxMD
e7b5a9bb60 Change repeat activity behavior when useSubmissionAsReference=true but not a link
Return false result instead of throwing an error since this is probably the expected behavior
2021-06-12 00:02:32 -04:00
FoxxMD
699f2577e5 Fix return value of author filter 2021-06-11 23:59:57 -04:00
FoxxMD
a22096a667 Fix snoowrap logger code
Accidentally not instantiating if debug not true
2021-06-11 23:43:52 -04:00
FoxxMD
a6e72dc79d Add rule ordering and api caching to advanced concepts 2021-06-11 19:35:25 -04:00
FoxxMD
962e44bf57 Fix some more links wtf 2021-06-11 19:20:16 -04:00
FoxxMD
2189d92725 Update recent activity examples descriptions 2021-06-11 19:17:09 -04:00
FoxxMD
14711efeb3 Use better link for example 2021-06-11 19:13:48 -04:00
FoxxMD
774b41c2a3 Add readmes for all sections 2021-06-11 19:12:11 -04:00
FoxxMD
4928b8b57a Add examples 2021-06-11 18:30:30 -04:00
FoxxMD
4b39794e2f Normalize named rule value 2021-06-11 18:29:19 -04:00
FoxxMD
c0ede0561c Add general percentages to history result data 2021-06-11 16:40:51 -04:00
FoxxMD
d7cea1f705 Add totalCount threshold property for recent activity rule 2021-06-11 16:00:17 -04:00
FoxxMD
3e29d7eb9f Remove noisy poll logging
Duh
2021-06-11 14:17:10 -04:00
FoxxMD
48ea60f886 Move heartbeat to app-level config 2021-06-11 14:15:54 -04:00
FoxxMD
1897d96a8f Add dryrun setting to help with testing
Can be configured at action, check, subreddit, or app level
2021-06-11 12:38:01 -04:00
FoxxMD
1279975a8a Add verbose log statements to log rule results and comment/report action contents 2021-06-11 11:56:55 -04:00
FoxxMD
7d0f7e8714 Decode content in render util instead of in each action 2021-06-11 11:44:36 -04:00
FoxxMD
f6b3f02e05 unescape content before report/comment 2021-06-11 10:36:37 -04:00
FoxxMD
1f439dc290 Re-add report length enforcement
Reddit returns 200 for longer content but then never actually makes the report
2021-06-11 10:28:51 -04:00
FoxxMD
0a299308fb Make history summary more succinct 2021-06-11 10:28:29 -04:00
FoxxMD
a84b39cc5a Wrap snoowrap logger function so arguments can be passed correctly to winston 2021-06-10 15:57:04 -04:00
FoxxMD
f64c6e0df5 Fix default for boolean option 2021-06-10 15:56:41 -04:00
FoxxMD
250313b6a8 Add separator between check name and description 2021-06-10 14:27:27 -04:00
FoxxMD
e4be9ed4e6 Display api limit on startup 2021-06-10 14:27:02 -04:00
FoxxMD
788af401b3 Support JSON5 for parsing configuration 2021-06-10 13:20:11 -04:00
FoxxMD
6bc74b383f Implement resource caching
User-configurable global/subreddit-level caching for author activities, wiki pages, and author checks
2021-06-10 13:13:57 -04:00
FoxxMD
51825a594d Handle errors from check iteration 2021-06-10 10:18:26 -04:00
FoxxMD
186d9ac4b7 Implement History rule 2021-06-09 17:09:33 -04:00
FoxxMD
ac02fdabfd Log link to invalid wiki page on error 2021-06-09 14:56:16 -04:00
FoxxMD
0eeb204371 Update ajv and fix schema generation issues
* Update ajv to fix #ref warning
* Consolidate ajv instantiation so config is always the same
* Remove propOrder flag for schema generation since its out of spec
2021-06-09 14:39:23 -04:00
FoxxMD
64a97ee048 Logging improvements
* Insert activity identifier into logging labels after subreddit using dynamic labels
* Simplify logger creation (don't need shuffle using improvements from above)
* Add logging to Actions
* Make check logging clearer and more succinct
* Log more information on startup
2021-06-09 13:36:32 -04:00
FoxxMD
318a1d3326 Set heroku url to default branch 2021-06-08 16:16:58 -04:00
FoxxMD
08db50426b Show check details and summary by default (info level) 2021-06-08 16:15:36 -04:00
FoxxMD
77f7a0167c Wiki value typo 2021-06-08 16:10:44 -04:00
FoxxMD
23a9f9d652 Remove potentially problematic heroku env 2021-06-08 16:09:00 -04:00
FoxxMD
72ed72ce4a Add heroku quick deploy button 2021-06-08 16:07:53 -04:00
FoxxMD
8cea19c7f2 Remove default env vars 2021-06-08 16:01:18 -04:00
FoxxMD
8eeaac2d53 Update heroku app file 2021-06-08 15:41:57 -04:00
FoxxMD
3cf838ba9f Create heroku app file 2021-06-08 15:39:14 -04:00
FoxxMD
16f3c2268b Create heroku file 2021-06-08 15:31:34 -04:00
FoxxMD
3be20b910d Fix missing return on activity filter 2021-06-08 14:03:49 -04:00
FoxxMD
78aed4321a Add support for reddit permalink when running check command 2021-06-08 13:55:03 -04:00
FoxxMD
0fe2fa8934 Add submission from comments convenience method 2021-06-08 13:46:45 -04:00
FoxxMD
37ba1dc1bf Fix default value when reference submission has no repeats 2021-06-08 13:46:08 -04:00
FoxxMD
5905c910b0 Implement name references for actions and rules
Action/Rule objects they can now be referenced by name from anywhere in the configuration
2021-06-08 12:40:00 -04:00
FoxxMD
d239d3c6cc Update dockerfile to use default run command 2021-06-08 00:52:23 -04:00
FoxxMD
16d0eebac6 Some small fixes for attribution 2021-06-08 00:35:34 -04:00
FoxxMD
1a393944c0 Refactor AttributionRule to be more robust and handle multiple window/thresholds
It's more useful to be abel to check thresholds for multiple windows to get a more holistic idea of attribution percents
2021-06-08 00:32:25 -04:00
FoxxMD
9f270010b7 Add trace to winston log levels so it can be used with snoowrap 2021-06-08 00:31:31 -04:00
FoxxMD
2548cff367 Friendly print schema validation errors 2021-06-08 00:30:57 -04:00
FoxxMD
c7acda46a0 Implement AttributionRule 2021-06-07 17:45:08 -04:00
FoxxMD
530675179b Refactor activity window
* Truncate items to window length when too many retrieved
* Correctly compare dates
2021-06-07 17:44:20 -04:00
FoxxMD
7960423678 Fix missing bold format character 2021-06-07 14:09:11 -04:00
FoxxMD
4ddb0f0963 Update readme with new cli syntax 2021-06-07 13:58:16 -04:00
FoxxMD
8a54ce15cd Refactor RepeatSubmission rule into RepeatActivity to allow more flexibility in use
* Refactor item repeat logic completely to simplify allow scenarios
* Can check for repeat comments now
* Add more context data and markdown content to display all repeats
2021-06-07 13:39:50 -04:00
FoxxMD
01161c3493 Implement specific activity checking through cli
* Refactor application input to use commando for extensibility
* Add all args/env as options for easy readout on command line
* Add 'check' command to allow running checks against a specific activity
* BC: must specify 'run' to run regular manager/unattended operation
2021-06-07 13:38:37 -04:00
FoxxMD
9970156a3d Fix leaf detection when undefined 2021-06-07 13:36:11 -04:00
FoxxMD
b437156d99 Clean up logging 2021-06-04 16:20:25 -04:00
FoxxMD
de3a279dc3 More verbose debug logging of checks added to subreddit manager on load 2021-06-04 15:07:30 -04:00
FoxxMD
86a6a75119 Add missing domain prefix to permalink 2021-06-04 15:03:32 -04:00
FoxxMD
9634b59b3a Add some temporal convenience logging
* heartbeat logging with configurable interval
* configurable api limit warning
2021-06-04 14:58:13 -04:00
FoxxMD
37f7c99155 Add snoodebug arg/env to control snoowrap debug output independently 2021-06-04 14:53:59 -04:00
FoxxMD
a99ab9a64a Add permalink to peek content to make logs more convenient 2021-06-04 14:53:29 -04:00
314 changed files with 43118 additions and 3298 deletions

View File

@@ -1,6 +1,8 @@
node_modules
Dockerfile
.dockerignore
.gitignore
.git
src/logs
.github
/docs/
/node_modules/

2
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,2 @@
github: [FoxxMD]
custom: ["bitcoincash:qqmpsh365r8n9jhp4p8ks7f7qdr7203cws4kmkmr8q"]

49
.github/workflows/dockerhub.yml vendored Normal file
View File

@@ -0,0 +1,49 @@
name: Publish Docker image to Dockerhub
on:
push:
branches:
- 'master'
- 'edge'
tags:
- '*.*.*'
# don't trigger if just updating docs
paths-ignore:
- '**.md'
jobs:
push_to_registry:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v2
- name: Log in to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v3
with:
images: foxxmd/context-mod
# generate Docker tags based on the following events/attributes
tags: |
type=raw,value=latest,enable=${{ endsWith(github.ref, 'master') }}
type=ref,event=branch,enable=${{ !endsWith(github.ref, 'master') }}
type=semver,pattern={{version}}
flavor: |
latest=false
- name: Build and push Docker image
uses: docker/build-push-action@v2
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

10
.gitignore vendored
View File

@@ -381,4 +381,14 @@ dist
.pnp.*
**/src/**/*.js
!src/Web/assets/public/yaml/*
**/src/**/*.map
/**/*.sqlite
/**/*.bak
*.yaml
*.json5
!src/Schema/*.json
!docs/**/*.json5
!docs/**/*.yaml
!docs/**/*.json

View File

@@ -5,7 +5,9 @@
<excludeFolder url="file://$MODULE_DIR$/temp" />
<excludeFolder url="file://$MODULE_DIR$/.tmp" />
<excludeFolder url="file://$MODULE_DIR$/tmp" />
<excludeFolder url="file://$MODULE_DIR$/src/logs" />
</content>
<content url="file://$MODULE_DIR$/node_modules" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>

View File

@@ -1,13 +1,17 @@
FROM node:16-alpine3.12
FROM node:16-alpine3.14 as base
ENV TZ=Etc/GMT
RUN apk update
# vips required to run sharp library for image comparison
RUN echo "http://dl-4.alpinelinux.org/alpine/v3.14/community" >> /etc/apk/repositories \
&& apk --no-cache add vips
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
WORKDIR /usr/app
FROM base as build
COPY package*.json ./
COPY tsconfig.json .
@@ -15,7 +19,13 @@ RUN npm install
ADD . /usr/app
RUN npm run build
RUN npm run build && rm -rf node_modules
FROM base as app
COPY --from=build /usr/app /usr/app
RUN npm install --production
ENV NPM_CONFIG_LOGLEVEL debug
@@ -24,4 +34,8 @@ RUN mkdir -p $log_dir
VOLUME $log_dir
ENV LOG_DIR=$log_dir
CMD [ "node", "src/index.js" ]
ARG webPort=8085
ENV PORT=$webPort
EXPOSE $PORT
CMD [ "node", "src/index.js", "run" ]

273
README.md
View File

@@ -1,10 +1,9 @@
# reddit-context-bot
# ContextMod [![Latest Release](https://img.shields.io/github/v/release/foxxmd/context-mod)](https://github.com/FoxxMD/context-mod/releases) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Docker Pulls](https://img.shields.io/docker/pulls/foxxmd/context-mod)](https://hub.docker.com/r/foxxmd/context-mod)
[![Latest Release](https://img.shields.io/github/v/release/foxxmd/reddit-context-bot)](https://github.com/FoxxMD/reddit-context-bot/releases)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker Pulls](https://img.shields.io/docker/pulls/foxxmd/reddit-context-bot)](https://hub.docker.com/r/foxxmd/reddit-context-bot)
<img src="/docs/logo.png" align="right"
alt="ContextMod logo" width="180" height="176">
**Context Bot** is an event-based, [reddit](https://reddit.com) moderation bot built on top of [snoowrap](https://github.com/not-an-aardvark/snoowrap) and written in [typescript](https://www.typescriptlang.org/).
**Context Mod** (CM) is an event-based, [reddit](https://reddit.com) moderation bot built on top of [snoowrap](https://github.com/not-an-aardvark/snoowrap) and written in [typescript](https://www.typescriptlang.org/).
It is designed to help fill in the gaps for [automoderator](https://www.reddit.com/wiki/automoderator/full-documentation) in regard to more complex behavior with a focus on **user-history based moderation.**
@@ -17,24 +16,37 @@ An example of the above that Context Bot can do now:
Some feature highlights:
* Simple rule-action behavior can be combined to create any level of complexity in behavior
* One instance can handle managing many subreddits (as many as it has moderator permissions in!)
* Per-subreddit configuration is handled by JSON stored in the subreddit wiki
* Any text-based actions (comment, submission, message, etc...) can be configured via a wiki page or raw text in JSON
* All text-based actions support [mustache](https://mustache.github.io) templating
* Server/client architecture
* Default/no configuration runs "All In One" behavior
* Additional configuration allows web interface to connect to multiple servers
* Each server instance can run multiple reddit accounts as bots
* **Per-subreddit configuration** is handled by YAML (**like automoderator!**) or JSON stored in the subreddit wiki
* Any text-based actions (comment, submission, message, usernotes, ban, etc...) can be configured via a wiki page or raw text and supports [mustache](https://mustache.github.io) [templating](/docs/actionTemplating.md)
* History-based rules support multiple "valid window" types -- [ISO 8601 Durations](https://en.wikipedia.org/wiki/ISO_8601#Durations), [Day.js Durations](https://day.js.org/docs/en/durations/creating), and submission/comment count limits.
* All rules support skipping behavior based on author criteria -- name, css flair/text, and moderator status
* Docker container support *(coming soon...)*
* Support Activity skipping based on:
* Author criteria (name, css flair/text, age, karma, moderator status, and [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes))
* Activity state (removed, locked, distinguished, etc.)
* Rules and Actions support named references (write once, reference anywhere)
* [**Image Comparisons**](/docs/imageComparison.md) via fingerprinting and/or pixel differences
* [**Repost detection**](/docs/examples/repost) with support for external services (youtube, etc...)
* Global/subreddit-level **API caching**
* Support for [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes) as criteria or Actions (writing notes)
* Docker container support
* Event notification via Discord
* **Web interface** for monitoring, administration, and oauth bot authentication
# Table of Contents
* [How It Works](#how-it-works)
* [Installation](#installation)
* [Configuration](#configuration)
* [Usage](#usage)
* [Getting Started](#getting-started)
* [Configuration And Documentation](#configuration-and-documentation)
* [Web UI and Screenshots](#web-ui-and-screenshots)
### How It Works
Context Bot's configuration is made up of an array of **Checks**. Each **Check** consists of :
Each subreddit using the RCB bot configures its behavior via their own wiki page.
When a monitored **Event** (new comment/submission, new modqueue item, etc.) is detected the bot runs through a list of **Checks** to determine what to do with the **Activity** from that Event. Each **Check** consists of :
#### Kind
@@ -42,228 +54,87 @@ Is this check for a submission or comment?
#### Rules
A list of **Rule** objects to run against the activity. If **any** Rule object is triggered by the activity then the Check runs its **Actions**
A list of **Rule** objects to run against the **Activity**. Triggered Rules can cause the whole Check to trigger and run its **Actions**
#### Actions
A list of **Action** objects that describe what the bot should do with the activity or author of the activity. The bot will run **all** Actions in this list.
A list of **Action** objects that describe what the bot should do with the **Activity** or **Author** of the activity (comment, remove, approve, etc.). The bot will run **all** Actions in this list.
___
The **Checks** for a subreddit are split up into **Submission Checks** and **Comment Checks** based on their **kind**. Each list of checks is run independently based on when events happen (submission or comment).
When an event occurs all Checks of that type are run in the order they were listed in the configuration. When one check is triggered (an action is performed) the remaining checks will not be run.
When an Event occurs all Checks of that type are run in the order they were listed in the configuration. When one check is triggered (an Action is performed) the remaining checks will not be run.
## Installation
___
[Learn more about the RCB lifecycle and core concepts in the docs.](/docs#how-it-works)
### Locally
## Getting Started
Clone this repository somewhere and then install from the working directory
#### Operators
```bash
git clone https://github.com/FoxxMD/reddit-context-bot.git .
cd reddit-context-bot
npm install
```
This guide is for users who want to **run their own bot on a ContextMod instance.**
### [Docker](https://hub.docker.com/r/foxxmd/reddit-context-bot)
See the [Operator's Getting Started Guide](/docs/gettingStartedOperator.md)
```
foxxmd/reddit-context-bot:latest
```
#### Moderators
Adding [**environmental variables**](#usage) to your `docker run` command will pass them through to the app EX:
```
docker run -e "CLIENT_ID=myId" ... foxxmd/reddit-context-bot
```
This guide is for **reddit moderators** who want to configure an existing CM bot to run on their subreddit.
## Configuration
See the [Moderator's Getting Started Guide](/docs/gettingStartedMod.md)
Context Bot's [configuration schema](/src/Schema/App.json) conforms to [JSON Schema](https://json-schema.org/) Draft 7.
## Configuration and Documentation
I suggest using [Atlassian JSON Schema Viewer](https://json-schema.app/start) ([direct link](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)) so you can view all documentation while also interactively writing and validating your config! From there you can drill down into any object, see its requirements, view an example JSON document, and live-edit your configuration on the right-hand side.
Context Bot's configuration can be written in YAML (like automoderator) or [JSON5](https://json5.org/). Its schema conforms to [JSON Schema Draft 7](https://json-schema.org/). Additionally, many **operator** settings can be passed via command line or environmental variables.
### Action Templating
* For **operators** (running the bot instance) see the [Operator Configuration](/docs/operatorConfiguration.md) guide
* For **moderators** consult the [app schema and examples folder](/docs/#configuration-and-usage)
Actions that can submit text (Report, Comment) will have their `content` values run through a [Mustache Template](https://mustache.github.io/). This means you can insert data generated by Rules into your text before the Action is performed.
[**Check the full docs for in-depth explanations of all concepts and examples**](/docs)
See here for a [cheatsheet](https://gist.github.com/FoxxMD/d365707cf99fdb526a504b8b833a5b78) and [here](https://www.tsmean.com/articles/mustache/the-ultimate-mustache-tutorial/) for a more thorough tutorial.
## Web UI and Screenshots
All Actions with `content` have access to this data:
### Dashboard
```json5
{
item: {
kind: 'string', // the type of item (comment/submission)
author: 'string', // name of the item author (reddit user)
permalink: 'string', // a url to the item
url: 'string', // if the item is a Submission then its URL (external for link type submission, reddit link for self-posts)
title: 'string', // if the item is a Submission, then the title of the Submission
},
rules: {
// contains all rules that were run and are accessible using the name, lowercased, with all spaces/dashes/underscores removed
}
}
CM comes equipped with a dashboard designed for use by both moderators and bot operators.
```
* Authentication via Reddit OAuth -- only accessible if you are the bot operator or a moderator of a subreddit the bot moderates
* Connect to multiple ContextMod instances (specified in configuration)
* Monitor API usage/rates
* Monitoring and administration **per subreddit:**
* Start/stop/pause various bot components
* View statistics on bot usage (# of events, checks run, actions performed) and cache usage
* View various parts of your subreddit's configuration and manually update configuration
* View **real-time logs** of what the bot is doing on your subreddit
* **Run bot on any permalink**
The properties of `rules` are accessible using the name, lower-cased, with all spaces/dashes/underscores. If no name is given `kind` is used as `name` Example:
![Subreddit View](docs/screenshots/subredditStatus.jpg)
```
"rules": [
{
"name": "My Custom-Recent Activity Rule", // mycustomrecentactivityrule
"kind": "recentActivity"
},
{
// name = repeatsubmission
"kind": "repeatSubmission",
}
]
```
### Bot Setup/Authentication
**To see what data is available for individual Rules [consult the schema](#configuration) for each Rule.**
A bot oauth helper allows operators to define oauth credentials/permissions and then generate unique, one-time invite links that allow moderators to authenticate their own bots without operator assistance. [Learn more about using the oauth helper.](docs/botAuthentication.md#cm-oauth-helper-recommended)
#### Quick Templating Tutorial
Operator view/invite link generation:
<details>
![Oauth View](docs/screenshots/oauth.jpg)
As a quick example for how you will most likely be using templating -- wrapping a variable in curly brackets, `{{variable}}`, will cause the variable value to be rendered instead of the brackets:
```
myVariable = 50;
myOtherVariable = "a text fragment"
template = "This is my template, the variable is {{myVariable}}, my other variable is {{myOtherVariable}}, and that's it!";
Moderator view/invite and authorization:
console.log(Mustache.render(template, {myVariable});
// will render...
"This is my template, the variable is 50, my other variable is a text fragment, and that's it!";
```
![Invite View](docs/screenshots/oauth-invite.jpg)
**Note: When accessing an object or its properties you must use dot notation**
```
const item = {
aProperty: 'something',
anotherObject: {
bProperty: 'something else'
}
}
const content = "My content will render the property {{item.aProperty}} like this, and another nested property {{item.anotherObject.bProperty}} like this."
```
</details>
### Configuration Editor
### Example Config
A built-in editor using [monaco-editor](https://microsoft.github.io/monaco-editor/) makes editing configurations easy:
Below is a configuration fulfilling the example given at the start of this readme:
* Automatic JSON or YAML syntax validation and formatting
* Automatic Schema (subreddit or operator) validation
* All properties are annotated via hover popups
* Unauthenticated view via `yourdomain.com/config`
* Authenticated view loads subreddit configurations by simple link found on the subreddit dashboard
* Switch schemas to edit either subreddit or operator configurations
<details>
<summary>Click to expand configuration</summary>
```json
{
"checks": [
{
"name": "repeatSpam",
"kind": "submission",
"rules": [
{
"kind": "repeatSubmission",
"gapAllowance": 2,
"threshold": 10
}
],
"actions": [
{
"kind": "remove"
},
{
"kind": "comment",
"content": "Thank you for your submission but we do not allow mass crossposting. Your submission has been removed",
"distingish": true
}
]
},
{
"name": "selfPromoActivity",
"kind": "submission",
"rules": [
{
"kind": "recentActivity",
"thresholds": [
{
"subreddits": [
"YouTubeSubscribeBoost",
"AdvertiseYourVideos"
]
}
]
}
],
"actions": [
{
"kind": "report",
"content": "User posted link {{rules.recentactivity.totalCount}} times in {{rules.recentactivity.subCount}} SP subs: {{rules.recentactivity.summary}}"
}
]
}
]
}
```
</details>
## Usage
`npm run start [list,of,subreddits] [...--options]`
CLI options take precedence over environmental variables
| CLI | Environmental Variable | Required | Description |
|------------------|------------------------|----------|----------------------------------------------------------------------------------------------------------------------------------|
| [First Argument] | | No | Comma-deliminated list of subreddits to run on if you don't want to run all the account has access to. |
| --clientId | CLIENT_ID | **Yes** | Your reddit application client id |
| --clientSecret | CLIENT_SECRET | **Yes** | Your reddit application client secret |
| --accessToken | ACCESS_TOKEN | **Yes** | A valid access token retrieved from completing the oauth flow for a user with your application. |
| --refreshToken | REFRESH_TOKEN | **Yes** | A valid refresh token retrieved from completing the oauth flow for a user with your application. |
| --logDir | LOG_DIR | No | The absolute path to where logs should be stored. use `false` to turn off log files. Defaults to `CWD/logs` |
| --logLevel | LOG_LEVEL | No | The minimum level to log at. Uses [Winston Log Levels](https://github.com/winstonjs/winston#logging-levels). Defaults to `info` |
| --wikiConfig | WIKI_CONFIG | No | The location of the bot configuration in the subreddit wiki. Defaults to `botconfig/contextbox` |
### Reddit App??
To use this bot you must do two things:
* Create a reddit application
* Authenticate that application to act as a user (login to the application with an account)
#### Create Application
Visit [your reddit preferences](https://www.reddit.com/prefs/apps) and at the bottom of the page go through the **create an(other) app** process.
* Choose **script**
* For redirect uri use https://not-an-aardvark.github.io/reddit-oauth-helper/
* Write down your **Client ID** and **Client Secret** somewhere
#### Authenticate an Account
Visit https://not-an-aardvark.github.io/reddit-oauth-helper/
* Input your **Client ID** and **Client Secret** in the text boxes with those names.
* Choose scopes. **It is very important you check everything on this list or Context Bot will not work correctly**
* edit
* flair
* history
* identity
* modcontributors
* modflair
* modposts
* modself
* mysubreddits
* read
* report
* submit
* wikiread
* Click **Generate tokens*, you will get a popup asking you to approve access (or login) -- **the account you approve access with is the account that Bot will control.**
* After approving an **Access Token** and **Refresh Token** will be shown at the bottom of the page. Write these down.
You should now have all the information you need to start the bot.
![Configuration View](docs/screenshots/editor.jpg)
## License

43
app.json Normal file
View File

@@ -0,0 +1,43 @@
{
"name": "Reddit Context Bot",
"description": "An event-based, reddit moderation bot built on top of snoowrap and written in typescript",
"repository": "https://github.com/FoxxMD/context-mod",
"stack": "container",
"env": {
"CLIENT_ID": {
"description": "Client ID for your Reddit application",
"value": "",
"required": true
},
"CLIENT_SECRET": {
"description": "Client Secret for your Reddit application",
"value": "",
"required": true
},
"REFRESH_TOKEN": {
"description": "Refresh token retrieved from authenticating an account with your Reddit Application",
"value": "",
"required": false
},
"ACCESS_TOKEN": {
"description": "Access token retrieved from authenticating an account with your Reddit Application",
"value": "",
"required": false
},
"REDIRECT_URI": {
"description": "Redirect URI you specified when creating your Reddit Application. Required if you want to use the web interface. In the provided example replace 'your-heroku-app-name' with the name of your HEROKU app.",
"value": "https://your-heroku-6app-name.herokuapp.com/callback",
"required": false
},
"OPERATOR": {
"description": "Your reddit username WITHOUT any prefixes EXAMPLE /u/FoxxMD => FoxxMD. Specified user will be recognized as an admin.",
"value": "",
"required": false
},
"WIKI_CONFIG": {
"description": "Relative url to contextbot wiki page EX https://reddit.com/r/subreddit/wiki/<path>",
"value": "botconfig/contextbot",
"required": false
}
}
}

67
cliff.toml Normal file
View File

@@ -0,0 +1,67 @@
# configuration file for git-cliff (0.1.0)
[changelog]
# changelog header
header = """
# Changelog
All notable changes to this project will be documented in this file.\n
"""
# template for the changelog body
# https://tera.netlify.app/docs/#introduction
body = """
{% if version %}\
## [{{ version | replace(from="v", to="") }}] - {{ timestamp | date(format="%Y-%m-%d") }}
{% else %}\
## [unreleased]
{% endif %}\
{% for group, commits in commits | group_by(attribute="group") %}
### {{ group | upper_first }}
{% for commit in commits
| filter(attribute="scope")
| sort(attribute="scope") %}
- *({{commit.scope}})* {{ commit.message | upper_first }}
{%- if commit.breaking %}
{% raw %} {% endraw %}- **BREAKING**: {{commit.breaking_description}}
{%- endif -%}
{%- endfor -%}
{%- for commit in commits %}
{%- if commit.scope -%}
{% else -%}
- *(No Category)* {{ commit.message | upper_first }}
{% if commit.breaking -%}
{% raw %} {% endraw %}- **BREAKING**: {{commit.breaking_description}}
{% endif -%}
{% endif -%}
{% endfor -%}
{% endfor %}
"""
# remove the leading and trailing whitespaces from the template
trim = true
# changelog footer
footer = """
<!-- generated by git-cliff -->
"""
[git]
# allow only conventional commits
# https://www.conventionalcommits.org
conventional_commits = true
# regex for parsing and grouping commits
commit_parsers = [
{ message = "^feat", group = "Features"},
{ message = "^fix", group = "Bug Fixes"},
{ message = "^doc", group = "Documentation"},
{ message = "^perf", group = "Performance"},
{ message = "^refactor", group = "Refactor"},
{ message = "^style", group = "Styling"},
{ message = "^test", group = "Testing"},
{ message = "^chore\\(release\\): prepare for", skip = true},
{ message = "^chore", group = "Miscellaneous Tasks"},
{ body = ".*security", group = "Security"},
]
# filter out the commits that are not matched by commit parsers
filter_commits = false
# glob pattern for matching git tags
tag_pattern = "[0-9]*"
# regex for skipping tags
skip_tags = "v0.1.0-beta.1"

354
docs/README.md Normal file
View File

@@ -0,0 +1,354 @@
# Documentation
# Table of Contents
* [Getting Started](#getting-started)
* [How It Works](#how-it-works)
* [Concepts](#concepts)
* [Check](#checks)
* [Rule](#rule)
* [Examples](#available-rules)
* [Rule Set](#rule-set)
* [Examples](#rule-set-examples)
* [Action](#action)
* [Examples](#available-actions)
* [Filters](#filters)
* [Configuration and Usage](#configuration-and-usage)
* [Common Resources](#common-resources)
* [Activities `window`](#activities-window)
* [Comparisons](#thresholds-and-comparisons)
* [Activity Templating](/docs/actionTemplating.md)
* [Image Comparisons](#image-comparisons)
* [Best Practices](#best-practices)
* [Named Rules](#named-rules)
* [Rule Order](#rule-order)
* [Caching](#caching)
* FAQ
## Getting Started
Review **at least** the **How It Works** and **Concepts** below, then:
* For **Operators** (running a bot instance) refer to [**Operator Getting Started**](/docs/gettingStartedOperator.md) guide
* For **Moderators** (configuring an existing bot for your subreddit) refer to the [**Moderator Getting Started**](/docs/gettingStartedMod.md) guide
## How It Works
Where possible Context Mod (CM) uses the same terminology as, and emulates the behavior, of **automoderator** so if you are familiar with that much of this may seem familiar to you.
CM's lifecycle looks like this:
#### 1) A new event in your subreddit is received by CM
The events CM watches for are configured by you. These can be new modqueue/unmoderated items, submissions, or comments.
#### 2) CM sequentially processes each Check in your configuration
A **Check** is a set of:
* One or more **Rules** that define what conditions should **trigger** this Check
* One or more **Actions** that define what the bot should do once the Check is **triggered**
#### 3) Each Check is processed, *in order*, until a Check is triggered
Once a Check is **triggered** no more Checks will be processed. This means all subsequent Checks in your configuration (in the order you listed them) are basically skipped.
#### 4) All Actions from that Check are executed
After all Actions are executed CM returns to waiting for the next Event.
## Concepts
Core, high-level concepts regarding how CM works.
### Checks
A **Check** is the main logical unit of behavior for the bot. It is equivalent to "if X then Y". A Check is comprised of:
* One or more **Rules** that are tested against an **Activity**
* One of more **Actions** that are performed when the **Rules** are satisfied
The bot's configuration can be made up of one or more **Checks** that are processed **in the order they are listed in the configuration.**
Once a Check is **triggered** (its Rules are satisfied and Actions performed) all subsequent Checks are skipped.
Some other important concepts regarding Checks:
* All Checks have a **kind** (defined in the configuration) that determine if they should run on **Submissions** or **Comments**
* Checks have a **condition** property that determines when they are considered **triggered**
* If the **condition** is `AND` then ALL of their **Rules** must be **triggered** for the Check to be **triggered**
* If the **condition** is `OR` then if ANY **Rules** is triggered **triggered** then the Check is **triggered**
Examples of different types of Checks can be found in the [subreddit-ready examples.](/docs/examples/subredditReady)
### Rule
A **Rule** is some set of **criteria** (conditions) that are tested against an Activity (comment/submission), a User, or a User's history. A Rule is considered **triggered** when the **criteria** for that rule are found to be **true** for whatever is being tested against.
There are generally three main properties for a Rule:
* **Critiera** -- The conditions/values you want to test for.
* **Activities Window** -- If applicable, the range of activities that the **criteria** will be tested against.
* **Rule-specific options** -- Any number of options that modify how the **criteria** are tested.
CM has different **Rules** that can test against different types of behavior and aspects of a User, their history, and the Activity (submission/common) being checked.
#### Available Rules
Find detailed descriptions of all the Rules, with examples, below:
* [Attribution](/docs/examples/attribution)
* [Recent Activity](/docs/examples/recentActivity)
* [Repeat Activity](/docs/examples/repeatActivity)
* [History](/docs/examples/history)
* [Author](/docs/examples/author)
* [Regex](/docs/examples/regex)
* [Repost](/docs/examples/repost)
### Rule Set
A **Rule Set** is a "grouped" set of `Rules` with a **trigger condition** specified.
Rule Sets can be used interchangeably with other **Rules** and **Rule Sets** in the `rules` list of a **Check**.
They allow you to create more complex trigger behavior by combining multiple rules into one "parent rule".
It consists of:
* **condition** -- Under what condition should the Rule Set be considered triggered?
* `AND` -- ALL Rules in the Rule Set must **trigger** in order for the Rule Set to **trigger.**
* `OR` -- ANY Rule in the Rule Set that is **triggered** will trigger the whole Rule Set.
* **rules** -- The **Rules** for the Rule Set.
Example
YAML
```yaml
condition: AND
# rules are an array
rules:
- aRule
```
JSON
```json5
{
"condition": "AND",
"rules": [
// all the rules go here
]
}
```
#### Rule Set Examples
* [**Detailed Example**](/docs/examples/advancedConcepts/ruleSets.json5)
### Action
An **Action** is some action the bot can take against the checked Activity (comment/submission) or Author of the Activity. CM has Actions for most things a normal reddit user or moderator can do.
#### Available Actions
* Remove (Comment/Submission)
* Flair (Submission)
* Ban (User)
* Approve (Comment/Submission)
* Comment (Reply to Comment/Submission)
* Lock (Comment/Submission)
* Report (Comment/Submission)
* [UserNote](/docs/examples/userNotes) (User, when /r/Toolbox is used)
For detailed explanation and options of what individual Actions can do [see the links in the `actions` property in the schema.](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json)
### Filters
**Checks, Rules, and Actions** all have two additional (optional) criteria "tests". These tests behave differently than rule/check triggers in that:
* When they **pass** the thing being tested continues to process as usual
* When they **fail** the thing being tested **is skipped, not failed.**
For **Checks** and **Actions** skipping means that the thing is not processed. The Action is not run, the Check is not triggered.
In the context of **Rules** (in a Check) skipping means the Rule does not get run BUT it does not fail. The Check will continue processing as if the Rule did not exist. However, if ALL Rules in a Check are skipped then the Check does "fail" (is not triggered).
#### Available Filters
##### Item Filter (`itemIs`)
This filter will test against the **state of the Activity currently being run.** Some criteria available to test against IE "Is the activity...":
* removed
* nsfw
* locked
* stickied
* deleted
* etc...
The `itemIs` filter is made up of an array (list) of `State` criteria objects. **All** criteria in the array must pass for this filter to pass.
There are two different State criteria depending on what type of Activity is being tested:
* Submission -- [SubmissionState](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FSubmissionState?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json)
* Comment -- [CommentState](https://json-schema.app/view/%23/%23%2Fdefinitions%2FCommentCheckJson/%23%2Fdefinitions%2FCommentState?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json)
##### Author Filter (`authorIs`)
This filter will test against the **Author of the Activity currently being run.** Some criteria available to test against:
* account age
* comment, link, and total karma
* subreddit flair text/css
* name
* User Notes
* verified
* etc...
The `authorIs` filter is made up two (optional) lists of [`AuthorCriteria`](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FAuthorOptions/%23%2Fdefinitions%2FAuthorCriteria?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) criteria objects that define how the test behaves:
* `include` list -- If **any** `AuthorCriteria` from this list passes then the `authorIs` test passes
* `exclude` list -- If **any** `AuthorCriteria` from this list **does not pass** then the `authorIs` test passes. **Note:** This property is ignored if `include` is also present IE you cannot use both properties at the same time.
Refer to the [app schema for `AuthorCriteria`](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FAuthorOptions/%23%2Fdefinitions%2FAuthorCriteria?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for all available properties to test against.
Some examples of using `authorIs` can be found in the [Author examples.](/docs/examples/author)
## Configuration And Usage
* For **Operator/Bot maintainers** see **[Operation Configuration](/docs/operatorConfiguration.md)**
* [CLI Usage](docs/operatorConfiguration.md#cli-usage)
* For **Moderators**
* Refer to the [examples folder](/docs/examples) or the [subreddit-ready examples](/docs/examples/subredditReady)
* as well as the [schema editor](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) which has
* fully annotated configuration data/structure
* generated examples in json/yaml
* built-in editor that automatically validates your config
## Common Resources
Technical information on recurring, common data/patterns used in CM.
### Activities `window`
Most **Rules** must define the **range of Activities (submissions and/or comments)** that will be used to check the criteria of the Rule. This range is defined wherever you see a `window` property in configuration.
Refer to the [Activities Window](/docs/activitiesWindow.md) documentation for a technical explanation with examples.
### Thresholds and Comparisons
Most rules/filters have criteria that require you to define a specific condition to test against. This can be anything from repeats of activities to account age.
In all of these scenarios the condition is defined using a subset of [comparison operators](https://www.codecademy.com/articles/fwd-js-comparison-logical) (very similar to how automoderator does things).
Available operators:
* `<` -- **less than** => `5 < 6` => 5 is less than 6
* `>` -- **greater than** => `6 > 5` => 6 is greater than 5
* `<=` -- **less than or equal to** => `5 <= 5` => 5 is less than **or equal to** 5
* `>=` -- **greater than or equal to** => `5 >= 5` => 5 is greater than **or equal to** 5
In the context of a rule/filter comparison you provide the comparison **omitting** the value that is being tested. An example...
The RepeatActivity rule has a `threshold` comparison to test against the number of repeat activities it finds
* You want the rule to trigger if it finds **4 or more repeat activities**
* The rule would be configured like this `"threshold": ">= 4"`
Essentially what this is telling the rule is `threshold: "x >= 4"` where `x` is the largest repeat of activities it finds.
#### Other Comparison Types
Other than comparison numeric values there are two other values that can be compared (depending on the criteria)
##### Percentages
Some criteria accept an optional **percentage** to compare against:
```
"threshold": "> 20%"
```
Refer to the individual rule/criteria schema to see what this percentage is comparing against.
##### Durations
Some criteria accept an optional **duration** to compare against:
```
"threshold": "< 1 month"
```
The duration value compares a time range from **now** to `duration value` time in the past.
Refer to [duration values in activity window documentation](/docs/activitiesWindow.md#duration-values) as well as the individual rule/criteria schema to see what this duration is comparing against.
### Image Comparisons
ContextMod implements two methods for comparing **image content**, perceptual hashing and pixel-to-pixel comparisons. Comparisons can be used to filter activities in some activities.
See [image comparison documentation](/docs/imageComparison.md) for a full reference.
## Best Practices
### Named Rules
All **Rules** in a subreddit's configuration can be assigned a **name** that can then be referenced from any other Check.
Create general-use rules so they can be reused and de-clutter your configuration. Additionally, CM will automatically cache the result of a rule so there is a performance and api usage benefit to re-using Rules.
See [ruleNameReuse.json5](/docs/examples/advancedConcepts/ruleNameReuse.json5) for a detailed configuration with annotations.
### Check Order
Checks are run in the order they appear in your configuration, therefore you should place your highest requirement/severe action checks at the top and lowest requirement/moderate actions at the bottom.
This is so that if an Activity warrants a more serious reaction that Check is triggered first rather than having a lower requirement check with less severe actions triggered and causing all subsequent Checks to be skipped.
* Attribution >50% AND Repeat Activity 8x AND Recent Activity in 2 subs => remove submission + ban
* Attribution >20% AND Repeat Activity 4x AND Recent Activity in 5 subs => remove submission + flair user restricted
* Attribution >20% AND Repeat Activity 2x => remove submission
* Attribution >20% AND History comments <30% => remove submission
* Attribution >15% => report
* Repeat Activity 2x => report
* Recent Activity in 3 subs => report
* Author not vetted => flair new user submission
### Rule Order
The ordering of your Rules within a Check/RuleSet can have an impact on Check performance (speed) as well as API usage.
Consider these three rules:
* Rule A -- Recent Activity => 3 subreddits => last 15 submissions
* Rule B -- Repeat Activity => last 3 days
* Rule C -- Attribution => >10% => last 90 days or 300 submissions
The first two rules are lightweight in their requirements -- Rule A can be completed in 1 API call, Rule B potentially completed in 1 Api call.
However, depending on how active the Author is, Rule C will take *at least* 3 API calls just to get all activities (Reddit limit 100 items per call).
If the Check is using `AND` condition for its rules (default) then if either Rule A or Rule B fail then Rule C will never run. This means 3 API calls never made plus the time waiting for each to return.
**It is therefore advantageous to list your lightweight Rules first in each Check.**
### Caching
ContextMod implements caching functionality for:
* author history (`window` criteria in rules)
* `authorIs` results
* `content` that uses wiki pages (on Comment/Report/Ban Actions)
* and User Notes
All of these use api requests so caching them reduces api usage.
Cached results can be re-used if the criteria in configuration is identical to a previously cached result. So...
* author history cache results are re-used if **`window` criteria on a Rule is identical to the `window` on another Rule** IE always use **7 Days** or always use **50 Items** for absolute counts.
* `authorIs` criteria is identical to another `authorIs` elsewhere in configuration..
* etc...
Re-use will result in less API calls and faster Check times.
PROTIP: You can monitor the re-use of cache in the `Cache` section of your subreddit on the web interface. See the tooltips in that section for a better breakdown of cache statistics.
## FAQ
TODO

72
docs/actionTemplating.md Normal file
View File

@@ -0,0 +1,72 @@
Actions that can submit text (Report, Comment) will have their `content` values run through a [Mustache Template](https://mustache.github.io/). This means you can insert data generated by Rules into your text before the Action is performed.
See here for a [cheatsheet](https://gist.github.com/FoxxMD/d365707cf99fdb526a504b8b833a5b78) and [here](https://www.tsmean.com/articles/mustache/the-ultimate-mustache-tutorial/) for a more thorough tutorial.
All Actions with `content` have access to this data:
```json5
{
item: {
kind: 'string', // the type of item (comment/submission)
author: 'string', // name of the item author (reddit user)
permalink: 'string', // a url to the item
url: 'string', // if the item is a Submission then its URL (external for link type submission, reddit link for self-posts)
title: 'string', // if the item is a Submission, then the title of the Submission,
botLink: 'string' // a link to the bot's FAQ
},
rules: {
// contains all rules that were run and are accessible using the name, lowercased, with all spaces/dashes/underscores removed
}
}
```
The properties of `rules` are accessible using the name, lower-cased, with all spaces/dashes/underscores. If no name is given `kind` is used as `name` Example:
```
"rules": [
{
"name": "My Custom-Recent Activity Rule", // mycustomrecentactivityrule
"kind": "recentActivity"
},
{
// name = repeatsubmission
"kind": "repeatActivity",
}
]
```
**To see what data is available for individual Rules [consult the schema](#configuration) for each Rule.**
#### Quick Templating Tutorial
As a quick example for how you will most likely be using templating -- wrapping a variable in curly brackets, `{{variable}}`, will cause the variable value to be rendered instead of the brackets:
```
myVariable = 50;
myOtherVariable = "a text fragment"
template = "This is my template, the variable is {{myVariable}}, my other variable is {{myOtherVariable}}, and that's it!";
console.log(Mustache.render(template, {myVariable});
// will render...
"This is my template, the variable is 50, my other variable is a text fragment, and that's it!";
```
**Note: When accessing an object or its properties you must use dot notation**
```
const item = {
aProperty: 'something',
anotherObject: {
bProperty: 'something else'
}
}
const content = "My content will render the property {{item.aProperty}} like this, and another nested property {{item.anotherObject.bProperty}} like this."
```

304
docs/activitiesWindow.md Normal file
View File

@@ -0,0 +1,304 @@
# Activity Window
Most **Rules** have a `window` property somewhere within their configuration. This property defines the range of **Activities** (submission and/or comments) that should be retrieved for checking the criteria of the Rule.
As an example if you want to run an **Recent Activity Rule** to check if a user has had activity in /r/mealtimevideos you also need to define what range of activities you want to look at from that user's history.
## `window` property overview (tldr)
The value of `window` can be any of these types:
* `number` count of activities
* `string` [duration](#duration-string-recommended) or [iso 8601](#an-iso-8601-duration-string)
* [duration `object`](#duration-object)
* [ActivityWindowCriteria `object`](#activitywindowcriteria)
Examples of all of the above
<details>
```yaml
# count, last 100 activities
window: 100
# duration, last 10 days
window: 10 days
# duration object, last 2 months and 5 days
window:
months: 2
days: 5
# iso 8601 string, last 15 minutes
window: PT15M
# ActivityWindowCriteria, last 100 activities or 6 weeks of activities (whichever is found first)
window:
count: 100
duration: 6 weeks
```
```json5
// count, last 100 activities
{
"window": 100
}
// duration string, last 10 days
{
"window": "10 days"
}
// duration object, last 2 months and 5 days
{
"window": {
"months": 2,
"days": 5,
}
}
// iso 8601 string, last 15 minutes
{
"window": "PT15M"
}
// ActivityWindowCriteria, last 100 activities or 6 weeks of activities (whichever is found first)
{
"window": {
"count": 100,
"duration": "6 weeks"
}
}
```
</details>
## Types of Ranges
There are two types of values that can be used when defining a range:
### Count
This is the **number** of activities you want to retrieve. It's straightforward -- if you want to look at the last 100 activities for a user you can use `100` as the value.
### Duration
A **duration of time** between which all activities will be retrieved. This is a **relative value** that calculates the actual range based on **the duration of time subtracted from when the rule is run.**
For example:
* Today is **July 15th**
* You define a duration of **10 days**
Then the range of activities to be retrieved will be between **July 5th and July 15th** (10 days).
#### Duration Values
The value used to define the duration can be **any of these three types**:
##### Duration String (recommended)
A string consisting of
* A [Dayjs unit of time](https://day.js.org/docs/en/durations/creating#list-of-all-available-units)
* The value of that unit of time
Examples:
* `9 days`
* `14 hours`
* `80 seconds`
You can ensure your string is valid by testing it [here.](https://regexr.com/61em3)
##### Duration Object
If you need to specify multiple units of time for your duration you can instead provide a [Dayjs duration **object**](https://day.js.org/docs/en/durations/creating#list-of-all-available-units) consisting of Dayjs unit-values.
Example
JSON
```json
{
"days": 4,
"hours": 6,
"minutes": 20
}
```
YAML
```yaml
window:
days: 4
hours: 6
minutes: 20
```
##### An ISO 8601 duration string
If you're a real nerd you can also use a [standard duration](https://en.wikipedia.org/wiki/ISO_8601#Durations)) string.
Examples
* `PT15M` (15 minutes)
Ensure your string is valid by testing it [here.](https://regexr.com/61em9)
## ActivityWindowCriteria
This is an object that lets you specify more granular conditions for your range.
The full object looks like this:
JSON
```json
{
"count": 100,
"duration": "10 days",
"satisfyOn": "any",
"subreddits": {
"include": ["mealtimevideos","pooptimevideos"],
"exclude": ["videos"]
}
}
```
YAML
```yaml
window:
count: 100
duration: 10 days
satisfyOn: any
subreddits:
include:
- mealtimevideos
- pooptimevideos
exclude:
- videos
```
### Specifying Range
You may use **one or both range properties.**
If both range properties are specified then the value `satisfyOn` determines how the final range is determined
#### Using `"satisfyOn": "any"` (default)
If **any** then Activities will be retrieved until one of the range properties is met, **whichever occurs first.**
Example
JSON
```json
{
"count": 80,
"duration": "90 days",
"satisfyOn": "any"
}
```
YAML
```yaml
window:
count: 80
duration: 90 days
satisfyOn: any
```
Activities are retrieved in chunks of 100 (or `count`, whichever is smaller)
* If 90 days of activities returns only 40 activities => returns 40 activities
* If 80 activities is only 20 days of range => 80 activities
#### Using `"satisfyOn": "all"`
If **all** then both ranges must be satisfied. Effectively, whichever range produces the most Activities will be the one that is used.
Example
JSON
```json
{
"count": 100,
"duration": "90 days",
"satisfyOn": "all"
}
```
YAML
```yaml
window:
count: 100
duration: 90 days
satisfyOn: all
```
Activities are retrieved in chunks of 100 (or `count`, whichever is smaller)
* If at 90 days of activities => 40 activities retrieved
* continue retrieving results until 100 activities
* so range is >90 days of activities
* If at 100 activities => 20 days of activities retrieved
* continue retrieving results until 90 days of range
* so results in >100 activities
### Filtering Activities
You may filter retrieved Activities using an array of subreddits.
**Note:** Activities are filtered **before** range check is made so you will always end up with specified range (but may require more api calls if many activities are filtered out)
#### Include
Use **include** to specify which subreddits should be included from results
Example where only activities from /r/mealtimevideos and /r/modsupport will be returned
JSON
```json
{
"count": 100,
"duration": "90 days",
"satisfyOn": "any",
"subreddits": {
"include": ["mealtimevideos","modsupport"]
}
}
```
YAML
```yaml
window:
count: 100
duruation: 90 days
satisfyOn: any
subreddits:
include:
- mealtimevideos
- modsupport
```
#### Exclude
Use **exclude** to specify which subreddits should NOT be in the results
Example where activities from /r/mealtimevideos and /r/modsupport will not be returned in results
JSON
```json
{
"count": 100,
"duration": "90 days",
"satisfyOn": "any",
"subreddits": {
"exclude": ["mealtimevideos","modsupport"]
}
}
```
YAML
```yaml
window:
count: 100
duruation: 90 days
satisfyOn: any
subreddits:
exclude:
- mealtimevideos
- modsupport
```
**Note:** `exclude` will be ignored if `include` is also present.

109
docs/botAuthentication.md Normal file
View File

@@ -0,0 +1,109 @@
**Note:** This is for **bot operators.** If you are a subreddit moderator check out the **[Getting Started Guide](/docs/gettingStartedMod.md)**
Before you can start using your bot on reddit there are a few steps you must take:
* Create your bot account IE the reddit account that will be the "bot"
* Create a Reddit application
* Authenticate your bot account with the application
At the end of this process you will have this info:
* clientId
* clientSecret
* refreshToken
* accessToken
* redirectUri
**Note:** If you already have this information you can skip this guide **but make sure your redirect uri is correct if you plan on using the web interface.**
# Table Of Contents
* [Creating an Application](#create-application)
* [Authenticate Your Bot](#authenticate-your-bot-account)
* [Using CM OAuth Helper](#cm-oauth-helper-recommended)
* [Using Aardvark OAuth Helper](#aardvark-oauth-helper)
* [Provide Credentials to CM](#provide-credentials-to-cm)
# Create Application
Visit [your reddit preferences](https://www.reddit.com/prefs/apps) and at the bottom of the page go through the **create an(other) app** process.
* Give it a **name**
* Choose **web app**
* If you know what you will use for **redirect uri** go ahead and use it, otherwise use **http://localhost:8085/callback**
Click **create app**.
Then write down your **Client ID, Client Secret, and Redirect Uri** somewhere (or keep this webpage open)
# Authenticate Your Bot Account
There are **two ways** you can authenticate your bot account. It is recommended to use the CM oauth helper.
## CM OAuth Helper (Recommended)
This method will use CM's built in oauth flow. It is recommended because it will ensure your bot is authenticated with the correct oauth permissions.
### Start CM with Client ID/Secret and Operator
Start the application and provide these to your configuration:
* **Client ID**
* **Client Secret**
* **Redirect URI**
* **Operator**
It is important you define **Operator** because the auth route is **protected.** You must login to the application in order to access the route.
Refer to the [operator config guide](/docs/operatorConfiguration.md) if you need help with this.
Examples:
* CLI - `node src/index.js --clientId=myId --clientSecret=mySecret --redirectUri="http://localhost:8085/callback" --operator=FoxxMD`
* Docker - `docker run -e "CLIENT_ID=myId" -e "CLIENT_SECRET=mySecret" -e "OPERATOR=FoxxMD" -e "REDIRECT_URI=http://localhost:8085/callback" foxxmd/context-mod`
### Create An Auth Invite
Then open the CM web interface (default is [http://localhost:8085](http://localhost:8085)) and login.
After logging in you should be automatically redirected the auth page. If you are not then visit [http://localhost:8085/auth/helper](http://localhost:8085/auth/helper))
Follow the directions in the helper to create an **auth invite link.** Open this link and then follow the directions to authenticate your bot. At the end of the process you will receive an **Access Token** and **Refresh Token**
## Aardvark OAuth Helper
This method should only be used if you cannot use the [CM OAuth Helper method](#cm-oauth-helper-recommended) because you cannot access the CM web interface.
* Visit [https://not-an-aardvark.github.io/reddit-oauth-helper/](https://not-an-aardvark.github.io/reddit-oauth-helper/) and follow the instructions given.
* **Note:** You will need to update your **redirect uri.**
* Input your **Client ID** and **Client Secret** in the text boxes with those names.
* Choose scopes. **It is very important you check everything on this list or CM may not work correctly**
* edit
* flair
* history
* identity
* modcontributors
* modflair
* modposts
* modself
* mysubreddits
* read
* report
* submit
* wikiread
* wikiedit (if you are using Toolbox User Notes)
* Click **Generate tokens**, you will get a popup asking you to approve access (or login) -- **the account you approve access with is the account that Bot will control.**
* After approving an **Access Token** and **Refresh Token** will be shown at the bottom of the page. Save these to use with CM.
# Provide Credentials to CM
At the end of the last step you chose you should now have this information saved somewhere:
* clientId
* clientSecret
* refreshToken
* accessToken
* redirectUri
This is all the information you need to run your bot with CM.
Using these credentials follow the [operator config guide](/docs/operatorConfiguration.md) to finish setting up your CM instance.

29
docs/examples/README.md Normal file
View File

@@ -0,0 +1,29 @@
# Examples
This directory contains example of valid, ready-to-go configurations for Context Mod for the purpose of:
* showcasing what the bot can do
* providing best practices for writing your configuration
* providing generally useful configurations **that can be used immediately** or as a jumping-off point for your configuration
### Examples Overview
* Rules
* [Attribution](/docs/examples/attribution)
* [Recent Activity](/docs/examples/recentActivity)
* [Repeat Activity](/docs/examples/repeatActivity)
* [History](/docs/examples/history)
* [Author](/docs/examples/author)
* [Regex](/docs/examples/regex)
* [Repost](/docs/examples/repost)
* [Author and post flairs](/docs/examples/onlyfansFlair)
* [Toolbox User Notes](/docs/examples/userNotes)
* [Advanced Concepts](/docs/examples/advancedConcepts)
* [Rule Sets](/docs/examples/advancedConcepts/ruleSets.json5)
* [Name Rules](/docs/examples/advancedConcepts/ruleNameReuse.json5)
* [Check Ordering](/docs/examples/advancedConcepts)
* [Subreddit-ready examples](/docs/examples/subredditReady)
PROTIP: You can edit/build on examples by using the [schema editor.](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json)

View File

@@ -0,0 +1,56 @@
### Named Rules
See **Rule Name Reuse Examples [YAML](/docs/examples/advancedConcepts/ruleNameReuse.yaml) | [JSON](/docs/examples/advancedConcepts/ruleNameReuse.json5)**
### Check Order
Checks are run in the order they appear in your configuration, therefore you should place your highest requirement/severe action checks at the top and lowest requirement/moderate actions at the bottom.
This is so that if an Activity warrants a more serious reaction that Check is triggered first rather than having a lower requirement check with less severe actions triggered and causing all subsequent Checks to be skipped.
* Attribution >50% AND Repeat Activity 8x AND Recent Activity in 2 subs => remove submission + ban
* Attribution >20% AND Repeat Activity 4x AND Recent Activity in 5 subs => remove submission + flair user restricted
* Attribution >20% AND Repeat Activity 2x => remove submission
* Attribution >20% AND History comments <30% => remove submission
* Attribution >15% => report
* Repeat Activity 2x => report
* Recent Activity in 3 subs => report
* Author not vetted => flair new user submission
### Rule Sets
The `rules` array on a `Checks` can contain both `Rule` objects and `RuleSet` objects.
A **Rule Set** is a "nested" set of `Rule` objects with a passing condition specified. These allow you to create more complex trigger behavior by combining multiple rules.
See **ruleSets [YAML](/docs/examples/advancedConcepts/ruleSets.yaml) | [JSON](/docs/examples/advancedConcepts/ruleSets.json5)** for a complete example as well as consulting the [schema](https://json-schema.app/view/%23%2Fdefinitions%2FRuleSetJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json).
### Rule Order
The ordering of your Rules within a Check/RuleSet can have an impact on Check performance (speed) as well as API usage.
Consider these three rules:
* Rule A -- Recent Activity => 3 subreddits => last 15 submissions
* Rule B -- Repeat Activity => last 3 days
* Rule C -- Attribution => >10% => last 90 days or 300 submissions
The first two rules are lightweight in their requirements -- Rule A can be completed in 1 API call, Rule B potentially completed in 1 Api call.
However, depending on how active the Author is, Rule C will take *at least* 3 API calls just to get all activities (Reddit limit 100 items per call).
If the Check is using `AND` condition for its rules (default) then if either Rule A or Rule B fail then Rule C will never run. This means 3 API calls never made plus the time waiting for each to return.
**It is therefore advantageous to list your lightweight Rules first in each Check.**
### API Caching
Context Mod implements some basic caching functionality for **Author Activities** and wiki pages (on Comment/Report Actions).
**Author Activities** are cached for a subreddit-configurable amount of time (10 seconds by default). A cached activities set can be re-used if the **window on a Rule is identical to the window on another Rule**.
This means that when possible you should re-use window values.
IE If you want to check an Author's Activities for a time range try to always use **7 Days** or always use **50 Items** for absolute counts.
Re-use will result in less API calls and faster Check times.

View File

@@ -0,0 +1,75 @@
{
"checks": [
{
"name": "Auto Remove SP Karma",
"description": "Remove submission because author has self-promo >10% and posted in karma subs recently",
"kind": "submission",
"rules": [
// named rules can be referenced at any point in the configuration (where they occur does not matter)
// and can be used in any Check
// Note: rules do not transfer between subreddit configurations
"freekarmasub",
{
"name": "attr10all",
"kind": "attribution",
"criteria": [
{
"threshold": "> 10%",
"window": "90 days"
},
{
"threshold": "> 10%",
"window": 100
}
],
}
],
"actions": [
{
"kind": "remove"
},
{
"kind": "comment",
"content": "Your submission was removed because you are over reddit's threshold for self-promotion and recently posted this content in a karma sub"
}
]
},
{
"name": "Free Karma On Submission Alert",
"description": "Check if author has posted this submission in 'freekarma' subreddits",
"kind": "submission",
"rules": [
{
// rules can be re-used throughout a configuration by referencing them by name
//
// The rule name itself can only contain spaces, hyphens and underscores
// The value used to reference it will have all of these removed, and lower-cased
//
// so to reference this rule use the value 'freekarmasub'
"name": "Free_Karma-SUB",
"kind": "recentActivity",
"lookAt": "submissions",
"useSubmissionAsReference":true,
"thresholds": [
{
"threshold": ">= 1",
"subreddits": [
"DeFreeKarma",
"FreeKarma4U",
"FreeKarma4You",
"upvote"
]
}
],
"window": "7 days"
}
],
"actions": [
{
"kind": "report",
"content": "Submission posted {{rules.freekarmasub.totalCount}} times in karma {{rules.freekarmasub.subCount}} subs over {{rules.freekarmasub.window}}: {{rules.freekarmasub.subSummary}}"
}
]
},
]
}

View File

@@ -0,0 +1,52 @@
checks:
- name: Auto Remove SP Karma
description: >-
Remove submission because author has self-promo >10% and posted in karma
subs recently
kind: submission
rules:
# named rules can be referenced at any point in the configuration (where they occur does not matter)
# and can be used in any Check
# Note: rules do not transfer between subreddit configurations
- freekarmasub
- name: attr10all
kind: attribution
criteria:
- threshold: '> 10%'
window: 90 days
- threshold: '> 10%'
window: 100
actions:
- kind: remove
- kind: comment
content: >-
Your submission was removed because you are over reddit's threshold
for self-promotion and recently posted this content in a karma sub
- name: Free Karma On Submission Alert
description: Check if author has posted this submission in 'freekarma' subreddits
kind: submission
rules:
# rules can be re-used throughout a configuration by referencing them by name
#
# The rule name itself can only contain spaces, hyphens and underscores
# The value used to reference it will have all of these removed, and lower-cased
#
# so to reference this rule use the value 'freekarmasub'
- name: Free_Karma-SUB
kind: recentActivity
lookAt: submissions
useSubmissionAsReference: true
thresholds:
- threshold: '>= 1'
subreddits:
- DeFreeKarma
- FreeKarma4U
- FreeKarma4You
- upvote
window: 7 days
actions:
- kind: report
content: >-
Submission posted {{rules.freekarmasub.totalCount}} times in karma
{{rules.freekarmasub.subCount}} subs over
{{rules.freekarmasub.window}}: {{rules.freekarmasub.subSummary}}

View File

@@ -0,0 +1,84 @@
{
"checks": [
{
"name": "Self Promo All or low comment",
"description": "SP >10% of all activities or >10% of submissions with low comment engagement",
"kind": "submission",
"rules": [
{
// this attribution rule is looking at all activities
//
// we want want this one rule to trigger the check because >10% of all activity (submission AND comments) is a good requirement
"name": "attr10all",
"kind": "attribution",
"criteria": [
{
"threshold": "> 10%",
"window": "90 days"
},
{
"threshold": "> 10%",
"window": 100
}
],
},
{
// this is a **Rule Set**
//
// it is made up of "nested" rules with a pass condition (AND/OR)
// if the nested rules pass the condition then the Rule Set triggers the Check
//
// AND = all nested rules must be triggered to make the Rule Set trigger
// AND = any of the nested Rules will be the Rule Set trigger
"condition": "AND",
// in this check we use an Attribution >10% on ONLY submissions, which is a lower requirement then the above attribution rule
// and combine it with a History rule looking for low comment engagement
// to make a "higher" requirement Rule Set our of two low requirement Rules
"rules": [
{
"name": "attr20sub",
"kind": "attribution",
"criteria": [
{
"threshold": "> 10%",
"thresholdOn": "submissions",
"window": "90 days"
},
{
"threshold": "> 10%",
"thresholdOn": "submissions",
"window": 100
}
],
"lookAt": "media"
},
{
"name": "lowOrOpComm",
"kind": "history",
"criteriaJoin": "OR",
"criteria": [
{
"window": "90 days",
"comment": "< 50%"
},
{
"window": "90 days",
"comment": "> 40% OP"
}
]
}
]
}
],
"actions": [
{
"kind": "remove"
},
{
"kind": "comment",
"content": "Your submission was removed because you are over reddit's threshold for self-promotion or exhibit low comment engagement"
}
]
},
],
}

View File

@@ -0,0 +1,53 @@
checks:
- name: Self Promo All or low comment
description: >-
SP >10% of all activities or >10% of submissions with low comment
engagement
kind: submission
rules:
# this attribution rule is looking at all activities
#
# we want want this one rule to trigger the check because >10% of all activity (submission AND comments) is a good requirement
- name: attr10all
kind: attribution
criteria:
- threshold: '> 10%'
window: 90 days
- threshold: '> 10%'
window: 100
# this is a RULE SET
#
# it is made up of "nested" rules with a pass condition (AND/OR)
# if the nested rules pass the condition then the Rule Set triggers the Check
#
# AND = all nested rules must be triggered to make the Rule Set trigger
# AND = any of the nested Rules will be the Rule Set trigger
- condition: AND
# in this check we use an Attribution >10% on ONLY submissions, which is a lower requirement then the above attribution rule
# and combine it with a History rule looking for low comment engagement
# to make a "higher" requirement Rule Set our of two low requirement Rules
rules:
- name: attr20sub
kind: attribution
criteria:
- threshold: '> 10%'
thresholdOn: submissions
window: 90 days
- threshold: '> 10%'
thresholdOn: submissions
window: 100
lookAt: media
- name: lowOrOpComm
kind: history
criteriaJoin: OR
criteria:
- window: 90 days
comment: < 50%
- window: 90 days
comment: '> 40% OP'
actions:
- kind: remove
- kind: comment
content: >-
Your submission was removed because you are over reddit's threshold
for self-promotion or exhibit low comment engagement

View File

@@ -0,0 +1,14 @@
# Attribution
The **Attribution** rule will aggregate an Author's content Attribution (youtube channels, twitter, website domains, etc.) and can check on their totals or percentages of all Activities over a time period:
* Total # of attributions
* As percentage of all Activity or only Submissions
* Look at all domains or only media (youtube, vimeo, etc.)
* Include self posts (by reddit domain) or not
Consult the [schema](https://json-schema.app/view/%23/%23%2Fdefinitions%2FCheckJson/%23%2Fdefinitions%2FAttributionJSONConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the rule's properties.
### Examples
* Self Promotion as percentage of all Activities [YAML](/docs/examples/attribution/redditSelfPromoAll.yaml) | [JSON](/docs/examples/attribution/redditSelfPromoAll.json5) - Check if Author is submitting much more than they comment.
* Self Promotion as percentage of Submissions [YAML](/docs/examples/attribution/redditSelfPromoSubmissionsOnly.yaml) | [JSON](/docs/examplesm/attribution/redditSelfPromoSubmissionsOnly.json5) - Check if any of Author's aggregated submission origins are >10% of their submissions

View File

@@ -0,0 +1,39 @@
{
"checks": [
{
"name": "Self Promo Activities",
"description": "Check if any of Author's aggregated submission origins are >10% of entire history",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "attr10all",
"kind": "attribution",
// criteria defaults to OR -- so either of these criteria will trigger the rule
"criteria": [
{
// threshold can be a percent or an absolute number
"threshold": "> 10%",
// The default is "all" -- calculate percentage of entire history (submissions & comments)
// "thresholdOn": "all",
// look at last 90 days of Author's activities (comments and submissions)
"window": "90 days"
},
{
"threshold": "> 10%",
// look at Author's last 100 activities (comments and submissions)
"window": 100
}
],
}
],
"actions": [
{
"kind": "report",
"content": "{{rules.attr10all.largestPercent}}% of {{rules.attr10all.activityTotal}} items over {{rules.attr10all.window}}"
}
]
}
]
}

View File

@@ -0,0 +1,27 @@
checks:
- name: Self Promo Activities
description: >-
Check if any of Author's aggregated submission origins are >10% of entire
history
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: attr10all
kind: attribution
# criteria defaults to OR -- so either of these criteria will trigger the rule
criteria:
- threshold: '> 10%' # threshold can be a percent or an absolute number
# The default is "all" -- calculate percentage of entire history (submissions & comments)
#thresholdOn: all
#
# look at last 90 days of Author's activities (comments and submissions)
window: 90 days
- threshold: '> 10%'
# look at Author's last 100 activities (comments and submissions)
window: 100
actions:
- kind: report
content: >-
{{rules.attr10all.largestPercent}}% of
{{rules.attr10all.activityTotal}} items over
{{rules.attr10all.window}}

View File

@@ -0,0 +1,24 @@
checks:
- name: Self Promo Submissions
description: >-
Check if any of Author's aggregated submission origins are >10% of their
submissions
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: attr10sub
kind: attribution
# criteria defaults to OR -- so either of these criteria will trigger the rule
criteria:
- threshold: '> 10%' # threshold can be a percent or an absolute number
thresholdOn: submissions # calculate percentage of submissions, rather than entire history (submissions & comments)
window: 90 days # look at last 90 days of Author's activities (comments and submissions)
- threshold: '> 10%'
thresholdOn: submissions
window: 100 # look at Author's last 100 activities (comments and submissions)
actions:
- kind: report
content: >-
{{rules.attr10sub.largestPercent}}% of
{{rules.attr10sub.activityTotal}} items over
{{rules.attr10sub.window}}

View File

@@ -0,0 +1,40 @@
{
"checks": [
{
"name": "Self Promo Submissions",
"description": "Check if any of Author's aggregated submission origins are >10% of their submissions",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "attr10sub",
"kind": "attribution",
// criteria defaults to OR -- so either of these criteria will trigger the rule
"criteria": [
{
// threshold can be a percent or an absolute number
"threshold": "> 10%",
// calculate percentage of submissions, rather than entire history (submissions & comments)
"thresholdOn": "submissions",
// look at last 90 days of Author's activities (comments and submissions)
"window": "90 days"
},
{
"threshold": "> 10%",
"thresholdOn": "submissions",
// look at Author's last 100 activities (comments and submissions)
"window": 100
}
],
}
],
"actions": [
{
"kind": "report",
"content": "{{rules.attr10sub.largestPercent}}% of {{rules.attr10sub.activityTotal}} items over {{rules.attr10sub.window}}"
}
]
}
]
}

View File

@@ -0,0 +1,38 @@
# Author
## Rule
The **Author** rule triggers if any [AuthorCriteria](https://json-schema.app/view/%23%2Fdefinitions%2FAuthorCriteria?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) from a list are either **included** or **excluded**, depending on which property you put them in.
**AuthorCriteria** that can be checked:
* name (u/userName)
* author's subreddit flair text
* author's subreddit flair css
* author's subreddit mod status
* [Toolbox User Notes](/docs/examples/userNotes)
The Author **Rule** is best used in conjunction with other Rules to short-circuit a Check based on who the Author is. It is easier to use a Rule to do this then to write **author filters** for every Rule (and makes Rules more re-useable).
Consult the [schema](https://json-schema.app/view/%23%2Fdefinitions%2FAuthorRuleJSONConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the rule's properties.
### Examples
* Basic examples
* Flair new user Submission [YAML](/docs/examples/author/flairNewUserSubmission.yaml) | [JSON](/docs/examples/author/flairNewUserSubmission.json5) - If the Author does not have the `vet` flair then flair the Submission with `New User`
* Flair vetted user Submission [YAML](/docs/examples/author/flairNewUserSubmission.yaml) | [JSON](/docs/examples/author/flairNewUserSubmission.json5) - If the Author does have the `vet` flair then flair the Submission with `Vetted`
* Used with other Rules
* Ignore vetted user [YAML](/docs/examples/author/flairNewUserSubmission.yaml) | [JSON](/docs/examples/author/flairNewUserSubmission.json5) - Short-circuit the Check if the Author has the `vet` flair
## Filter
All **Rules** and **Checks** have an optional `authorIs` property that takes an [AuthorOptions](https://json-schema.app/view/%23%2Fdefinitions%2FAuthorOptions?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) object.
**This property works the same as the Author Rule except that:**
* On **Rules** if all criteria fail the Rule is **skipped.**
* If a Rule is skipped **it does not fail or pass** and so does not affect the outcome of the Check.
* However, if all Rules on a Check are skipped the Check will fail.
* On **Checks** if all criteria fail the Check **fails**.
### Examples
* Skip recent activity check based on author [YAML](/docs/examples/author/authorFilter.yaml) | [JSON](/docs/examples/author/authorFilter.json5) - Skip a Recent Activity check for a set of subreddits if the Author of the Submission has any set of flairs.

View File

@@ -0,0 +1,69 @@
{
"checks": [
{
"name": "Karma/Meme Sub Activity",
"description": "Report on karma sub activity or meme sub activity if user isn't a memelord",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "freekarma",
"kind": "recentActivity",
"lookAt": "submissions",
"thresholds": [
{
"threshold": ">= 1",
"subreddits": [
"DeFreeKarma",
"FreeKarma4U",
]
}
],
"window": "7 days"
},
{
"name": "noobmemer",
"kind": "recentActivity",
// authors filter will be checked before a rule is run. If anything passes then the Rule is skipped -- it is not failed or triggered.
// if *all* Rules for a Check are skipped due to authors filter then the Check will fail
"authorIs": {
// each property (include/exclude) can contain multiple AuthorCriteria
// if any AuthorCriteria passes its test the Rule is skipped
//
// for an AuthorCriteria to pass all properties present on it must pass
//
// if "include" is present it will always run and exclude will be skipped
// "include:" []
"exclude": [
// for this to pass the Author of the Submission must not have the flair "Supreme Memer" and have the name "user1" or "user2"
{
"flairText": ["Supreme Memer"],
"names": ["user1","user2"]
},
{
// for this to pass the Author of the Submission must not have the flair "Decent Memer"
"flairText": ["Decent Memer"]
}
]
},
"lookAt": "submissions",
"thresholds": [
{
"threshold": ">= 1",
"subreddits": [
"dankmemes",
]
}
],
"window": "7 days"
}
],
"actions": [
{
"kind": "report",
"content": "Author has posted in free karma sub, or in /r/dankmemes and does not have meme flair in this subreddit"
}
]
}
]
}

View File

@@ -0,0 +1,48 @@
checks:
- name: Karma/Meme Sub Activity
description: Report on karma sub activity or meme sub activity if user isn't a memelord
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: freekarma
kind: recentActivity
lookAt: submissions
thresholds:
- threshold: '>= 1'
subreddits:
- DeFreeKarma
- FreeKarma4U
window: 7 days
- name: noobmemer
kind: recentActivity
# authors filter will be checked before a rule is run. If anything passes then the Rule is skipped -- it is not failed or triggered.
# if *all* Rules for a Check are skipped due to authors filter then the Check will fail
authorIs:
# each property (include/exclude) can contain multiple AuthorCriteria
# if any AuthorCriteria passes its test the Rule is skipped
#
# for an AuthorCriteria to pass all properties present on it must pass
#
# if include is present it will always run and exclude will be skipped
#-include:
exclude:
# for this to pass the Author of the Submission must not have the flair "Supreme Memer" and have the name "user1" or "user2"
- flairText:
- Supreme Memer
names:
- user1
- user2
# for this to pass the Author of the Submission must not have the flair "Decent Memer"
- flairText:
- Decent Memer
lookAt: submissions
thresholds:
- threshold: '>= 1'
subreddits:
- dankmemes
window: 7 days
actions:
- kind: report
content: >-
Author has posted in free karma sub, or in /r/dankmemes and does not
have meme flair in this subreddit

View File

@@ -0,0 +1,29 @@
{
"checks": [
{
"name": "Flair New User Sub",
"description": "Flair submission as sketchy if user does not have vet flair",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "newflair",
"kind": "author",
// rule will trigger if Author does not have "vet" flair text
"exclude": [
{
"flairText": ["vet"]
}
]
}
],
"actions": [
{
"kind": "flair",
"text": "New User",
"css": "orange"
}
]
}
]
}

View File

@@ -0,0 +1,16 @@
checks:
- name: Flair New User Sub
description: Flair submission as sketchy if user does not have vet flair
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: newflair
kind: author
# rule will trigger if Author does not have "vet" flair text
exclude:
- flairText:
- vet
actions:
- kind: flair
text: New User
css: orange

View File

@@ -0,0 +1,29 @@
{
"checks": [
{
"name": "Flair Vetted User Submission",
"description": "Flair submission as Approved if user has vet flair",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "newflair",
"kind": "author",
// rule will trigger if Author has "vet" flair text
"include": [
{
"flairText": ["vet"]
}
]
}
],
"actions": [
{
"kind": "flair",
"text": "Vetted",
"css": "green"
}
]
}
]
}

View File

@@ -0,0 +1,16 @@
checks:
- name: Flair Vetted User Submission
description: Flair submission as Approved if user has vet flair
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: newflair
kind: author
# rule will trigger if Author has "vet" flair text
include:
- flairText:
- vet
actions:
- kind: flair
text: Vetted
css: green

View File

@@ -0,0 +1,75 @@
{
"checks": [
{
"name": "non-vetted karma/meme activity",
"description": "Report if Author has SP and has recent karma/meme sub activity and isn't vetted",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
// The Author Rule is best used in conjunction with other Rules --
// instead of having to write an AuthorFilter for every Rule where you want to skip it based on Author criteria
// you can write one Author Rule and make it fail on the required criteria
// so that the check fails and Actions don't run
"name": "nonvet",
"kind": "author",
"exclude": [
{
"flairText": ["vet"]
}
]
},
{
"name": "attr10",
"kind": "attribution",
"criteria": [
{
"threshold": "> 10%",
"window": "90 days"
},
{
"threshold": "> 10%",
"window": 100
}
],
},
{
"name": "freekarma",
"kind": "recentActivity",
"lookAt": "submissions",
"thresholds": [
{
"threshold": ">= 1",
"subreddits": [
"DeFreeKarma",
"FreeKarma4U",
]
}
],
"window": "7 days"
},
{
"name": "memes",
"kind": "recentActivity",
"lookAt": "submissions",
"thresholds": [
{
"threshold": ">= 3",
"subreddits": [
"dankmemes",
]
}
],
"window": "7 days"
}
],
// will NOT run if the Author for this Submission has the flair "vet"
"actions": [
{
"kind": "report",
"content": "Author has posted in free karma or meme subs recently"
}
]
}
]
}

View File

@@ -0,0 +1,45 @@
checks:
- name: non-vetted karma/meme activity
description: >-
Report if Author has SP and has recent karma/meme sub activity and isn't
vetted
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
# The Author Rule is best used in conjunction with other Rules --
# instead of having to write an AuthorFilter for every Rule where you want to skip it based on Author criteria
# you can write one Author Rule and make it fail on the required criteria
# so that the check fails and Actions don't run
- name: nonvet
kind: author
exclude:
- flairText:
- vet
- name: attr10
kind: attribution
criteria:
- threshold: '> 10%'
window: 90 days
- threshold: '> 10%'
window: 100
- name: freekarma
kind: recentActivity
lookAt: submissions
thresholds:
- threshold: '>= 1'
subreddits:
- DeFreeKarma
- FreeKarma4U
window: 7 days
- name: memes
kind: recentActivity
lookAt: submissions
thresholds:
- threshold: '>= 3'
subreddits:
- dankmemes
window: 7 days
# will NOT run if the Author for this Submission has the flair "vet"
actions:
- kind: report
content: Author has posted in free karma or meme subs recently

View File

@@ -0,0 +1,13 @@
# History
The **History** rule can check an Author's submission/comment statistics over a time period:
* Submission total or percentage of All Activity
* Comment total or percentage of all Activity
* Comments made as OP (commented in their own Submission) total or percentage of all Comments
Consult the [schema](https://json-schema.app/view/%23%2Fdefinitions%2FHistoryJSONConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the rule's properties.
### Examples
* Low Comment Engagement [YAML](/docs/examples/history/lowEngagement.yaml) | [JSON](/docs/examples/history/lowEngagement.json5) - Check if Author is submitting much more than they comment.
* OP Comment Engagement [YAML](/docs/examples/history/opOnlyEngagement.yaml) | [JSON](/docs/examples/history/opOnlyEngagement.json5) - Check if Author is mostly engaging only in their own content

View File

@@ -0,0 +1,30 @@
{
"checks": [
{
"name": "Low Comment Engagement",
"description": "Check if Author is submitting much more than they comment",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "lowComm",
"kind": "history",
"criteria": [
{
// look at last 90 days of Author's activities
"window": "90 days",
// trigger if less than 30% of their activities in this time period are comments
"comment": "< 30%"
},
]
}
],
"actions": [
{
"kind": "report",
"content": "Low engagement: comments were {{rules.lowcomm.commentPercent}} of {{rules.lowcomm.activityTotal}} over {{rules.lowcomm.window}}"
}
]
}
]
}

View File

@@ -0,0 +1,21 @@
checks:
- name: Low Comment Engagement
description: Check if Author is submitting much more than they comment
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: lowComm
kind: history
criteria:
- comment: '< 30%'
window:
# get author's last 90 days of activities or 100 activities, whichever is less
duration: 90 days
count: 100
# trigger if less than 30% of their activities in this time period are comments
actions:
- kind: report
content: >-
Low engagement: comments were {{rules.lowcomm.commentPercent}} of
{{rules.lowcomm.activityTotal}} over {{rules.lowcomm.window}}

View File

@@ -0,0 +1,30 @@
{
"checks": [
{
"name": "Engaging Own Content Only",
"description": "Check if Author is mostly engaging in their own content only",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "opOnly",
"kind": "history",
"criteria": [
{
// look at last 90 days of Author's activities
"window": "90 days",
// trigger if more than 60% of their activities in this time period are comments as OP
"comment": "> 60% OP"
},
]
}
],
"actions": [
{
"kind": "report",
"content": "Selfish OP: {{rules.oponly.opPercent}} of {{rules.oponly.commentTotal}} comments over {{rules.oponly.window}} are as OP"
}
]
}
]
}

View File

@@ -0,0 +1,22 @@
checks:
- name: Engaging Own Content Only
description: Check if Author is mostly engaging in their own content only
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: opOnly
kind: history
criteria:
# trigger if more than 60% of their activities in this time period are comments as OP
- comment: '> 60% OP'
window:
# get author's last 90 days of activities or 100 activities, whichever is less
duration: 90 days
count: 100
actions:
- kind: report
content: >-
Selfish OP: {{rules.oponly.opPercent}} of
{{rules.oponly.commentTotal}} comments over {{rules.oponly.window}}
are as OP

View File

@@ -0,0 +1,9 @@
# Flair users and submissions
Flair users and submissions based on certain keywords from submitter's profile.
Consult [User Flair schema](https://json-schema.app/view/%23%2Fdefinitions%2FUserFlairActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) and [Submission Flair schema](https://json-schema.app/view/%23%2Fdefinitions%2FFlairActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the rule's properties.
### Examples
* OnlyFans submissions [YAML](/docs/examples/onlyFansFlair/onlyFansFlair.yaml) | [JSON](/docs/examples/onlyfansFlair/onlyfansFlair.json5) - Check whether submitter has typical OF keywords in their profile and flair both author + submission accordingly.

View File

@@ -0,0 +1,68 @@
{
"checks": [
{
"name": "Flair OF submitters",
"description": "Flair submission as OF if user does not have Verified flair and has certain keywords in their profile",
"kind": "submission",
"authorIs": {
"exclude": [
{
"flairCssClass": ["verified"]
}
]
},
"rules": [
{
"name": "OnlyFans strings in description",
"kind": "author",
"include": [
{
"description": [
"/(cashapp|allmylinks|linktr|onlyfans\\.com)/i",
"/(see|check|my|view) (out|of|onlyfans|kik|skype|insta|ig|profile|links)/i",
"my links",
"$"
]
}
]
}
],
"actions": [
{
"name": "Set OnlyFans user flair",
"kind": "userflair",
"flair_template_id": "put-your-onlyfans-user-flair-id-here"
},
{
"name":"Set OF Creator SUBMISSION flair",
"kind": "flair",
"flair_template_id": "put-your-onlyfans-post-flair-id-here"
}
]
},
{
"name": "Flair posts of OF submitters",
"description": "Flair submission as OnlyFans if submitter has OnlyFans userflair (override post flair set by submitter)",
"kind": "submission",
"rules": [
{
"name": "Include OF submitters",
"kind": "author",
"include": [
{
"flairCssClass": ["onlyfans"]
}
]
}
],
"actions": [
{
"name":"Set OF Creator SUBMISSION flair",
"kind": "flair",
"flair_template_id": "put-your-onlyfans-post-flair-id-here"
}
]
}
]
}

View File

@@ -0,0 +1,38 @@
checks:
- name: Flair OF submitters
description: Flair submission as OF if user does not have Verified flair and has
certain keywords in their profile
kind: submission
authorIs:
exclude:
- flairCssClass:
- verified
rules:
- name: OnlyFans strings in description
kind: author
include:
- description:
- '/(cashapp|allmylinks|linktr|onlyfans\.com)/i'
- '/(see|check|my|view) (out|of|onlyfans|kik|skype|insta|ig|profile|links)/i'
- my links
- "$"
actions:
- name: Set OnlyFans user flair
kind: userflair
flair_template_id: put-your-onlyfans-user-flair-id-here
- name: Set OF Creator SUBMISSION flair
kind: flair
flair_template_id: put-your-onlyfans-post-flair-id-here
- name: Flair posts of OF submitters
description: Flair submission as OnlyFans if submitter has OnlyFans userflair (override post flair set by submitter)
kind: submission
rules:
- name: Include OF submitters
kind: author
include:
- flairCssClass:
- onlyfans
actions:
- name: Set OF Creator SUBMISSION flair
kind: flair
flair_template_id: put-your-onlyfans-post-flair-id-here

View File

@@ -0,0 +1,10 @@
# Recent Activity
The **Recent Activity** rule can check if an Author has made any Submissions/Comments in a list of defined Subreddits.
Consult the [schema](https://json-schema.app/view/%23%2Fdefinitions%2FRecentActivityRuleJSONConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the rule's properties.
### Examples
* Free Karma Subreddits [YAML](/docs/examples/recentActivity/freeKarma.yaml) | [JSON](/docs/examples/recentActivity/freeKarma.json5) - Check if the Author has recently posted in any "free karma" subreddits
* Submission in Free Karma Subreddits [YAML](/docs/examples/recentActivity/freeKarmaOnSubmission.yaml) | [JSON](/docs/examples/recentActivity/freeKarmaOnSubmission.json5) - Check if the Author has posted the Submission this check is running on in any "free karma" subreddits recently

View File

@@ -0,0 +1,40 @@
{
"checks": [
{
"name": "Free Karma Alert",
"description": "Check if author has posted in 'freekarma' subreddits",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "freekarma",
"kind": "recentActivity",
"useSubmissionAsReference": false,
// when `lookAt` is not present this rule will look for submissions and comments
// lookAt: "submissions"
// lookAt: "comments"
"thresholds": [
{
// for all subreddits, if the number of activities (sub/comment) is equal to or greater than 1 then the rule is triggered
"threshold": ">= 1",
"subreddits": [
"DeFreeKarma",
"FreeKarma4U",
"FreeKarma4You",
"upvote"
]
}
],
// will look at all of the Author's activities in the last 7 days
"window": "7 days"
}
],
"actions": [
{
"kind": "report",
"content": "{{rules.freekarma.totalCount}} activities in karma {{rules.freekarma.subCount}} subs over {{rules.freekarma.window}}: {{rules.freekarma.subSummary}}"
}
]
}
]
}

View File

@@ -0,0 +1,27 @@
checks:
- name: Free Karma Alert
description: Check if author has posted in 'freekarma' subreddits
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: freekarma
kind: recentActivity
# // when lookAt is not present this rule will look for submissions and comments
#lookAt: comments
useSubmissionAsReference: false
thresholds:
# if the number of activities (sub/comment) found CUMULATIVELY in the subreddits listed is
# equal to or greater than 1 then the rule is triggered
- threshold: '>= 1'
subreddits:
- DeFreeKarma
- FreeKarma4U
- FreeKarma4You
- upvote
window: 7 days
actions:
- kind: report
content: >-
{{rules.freekarma.totalCount}} activities in karma
{{rules.freekarma.subCount}} subs over {{rules.freekarma.window}}:
{{rules.freekarma.subSummary}}

View File

@@ -0,0 +1,41 @@
{
"checks": [
{
"name": "Free Karma On Submission Alert",
"description": "Check if author has posted this submission in 'freekarma' subreddits",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "freekarmasub",
"kind": "recentActivity",
// rule will only look at Author's submissions in these subreddits
"lookAt": "submissions",
// rule will only look at Author's submissions in these subreddits that have the same content (link) as the submission this event was made on
// In simpler terms -- rule will only check to see if the same link the author just posted is also posted in these subreddits
"useSubmissionAsReference":true,
"thresholds": [
{
// for all subreddits, if the number of activities (sub/comment) is equal to or greater than 1 then the rule is triggered
"threshold": ">= 1",
"subreddits": [
"DeFreeKarma",
"FreeKarma4U",
"FreeKarma4You",
"upvote"
]
}
],
// look at all of the Author's submissions in the last 7 days
"window": "7 days"
}
],
"actions": [
{
"kind": "report",
"content": "Submission posted {{rules.freekarmasub.totalCount}} times in karma {{rules.freekarmasub.subCount}} subs over {{rules.freekarmasub.window}}: {{rules.freekarmasub.subSummary}}"
}
]
}
]
}

View File

@@ -0,0 +1,26 @@
checks:
- name: Free Karma On Submission Alert
description: Check if author has posted this submission in 'freekarma' subreddits
kind: submission
rules:
- name: freekarmasub
kind: recentActivity
# rule will only look at Author's submissions in these subreddits
lookAt: submissions
# rule will only look at Author's submissions in these subreddits that have the same content (link) as the submission this event was made on
# In simpler terms -- rule will only check to see if the same link the author just posted is also posted in these subreddits
useSubmissionAsReference: true
thresholds:
- threshold: '>= 1'
subreddits:
- DeFreeKarma
- FreeKarma4U
- FreeKarma4You
- upvote
window: 7 days
actions:
- kind: report
content: >-
Submission posted {{rules.freekarmasub.totalCount}} times in karma
{{rules.freekarmasub.subCount}} subs over
{{rules.freekarmasub.window}}: {{rules.freekarmasub.subSummary}}

View File

@@ -0,0 +1,22 @@
The **Regex** rule matches on text content from a comment or submission in the same way automod uses regex. The rule, however, provides additional functionality automod does not:
* Can set the **number** of matches that trigger the rule (`matchThreshold`)
Which can then be used in conjunction with a [`window`](https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md) to match against activities from the history of the Author of the Activity being checked (including the Activity being checked):
* Can set the **number of Activities** that meet the `matchThreshold` to trigger the rule (`activityMatchThreshold`)
* Can set the **number of total matches** across all Activities to trigger the rule (`totalMatchThreshold`)
* Can set the **type of Activities** to check (`lookAt`)
* When an Activity is a Submission can **specify which parts of the Submission to match against** IE title, body, and/or url (`testOn`)
### Examples
* Trigger if regex matches against the current activity - [YAML](/docs/examples/regex/matchAnyCurrentActivity.yaml) | [JSON](/docs/examples/regex/matchAnyCurrentActivity.json5)
* Trigger if regex matches 5 times against the current activity - [YAML](/docs/examples/regex/matchThresholdCurrentActivity.yaml) | [JSON](/docs/examples/regex/matchThresholdCurrentActivity.json5)
* Trigger if regex matches against any part of a Submission - [YAML](/docs/examples/regex/matchSubmissionParts.yaml) | [JSON](/docs/examples/regex/matchSubmissionParts.json5)
* Trigger if regex matches any of Author's last 10 activities - [YAML](/docs/examples/regex/matchHistoryActivity.yaml) | [JSON](/docs/examples/regex/matchHistoryActivity.json5)
* Trigger if regex matches at least 3 of Author's last 10 activities - [YAML](/docs/examples/regex/matchActivityThresholdHistory.json5) | [JSON](/docs/examples/regex/matchActivityThresholdHistory.json5)
* Trigger if there are 5 regex matches in the Author's last 10 activities - [YAML](/docs/examples/regex/matchTotalHistoryActivity.yaml) | [JSON](/docs/examples/regex/matchTotalHistoryActivity.json5)
* Trigger if there are 5 regex matches in the Author's last 10 comments - [YAML](/docs/examples/regex/matchSubsetHistoryActivity.yaml) | [JSON](/docs/examples/regex/matchSubsetHistoryActivity.json5)
* Remove comments that are spamming discord links - [YAML](/docs/examples/regex/removeDiscordSpam.yaml) | [JSON](/docs/examples/regex/removeDiscordSpam.json5)
* Differs from just using automod because this config can allow one-off/organic links from users who DO NOT spam discord links but will still remove the comment if the user is spamming them

View File

@@ -0,0 +1,20 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
// triggers if more than 3 activities in the last 10 match the regex
{
"regex": "/fuck|shit|damn/",
// this differs from "totalMatchThreshold"
//
// activityMatchThreshold => # of activities from window must match regex
// totalMatchThreshold => # of matches across all activities from window must match regex
"activityMatchThreshold": "> 3",
// if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
// learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
"window": 10,
},
]
}

View File

@@ -0,0 +1,13 @@
name: swear
kind: regex
criteria:
# triggers if more than 3 activities in the last 10 match the regex
- regex: '/fuck|shit|damn/'
# this differs from "totalMatchThreshold"
#
# activityMatchThreshold => # of activities from window must match regex
# totalMatchThreshold => # of matches across all activities from window must match regex
activityMatchThreshold: '> 3'
# if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
# learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
window: 10

View File

@@ -0,0 +1,14 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
// triggers if current activity has more than 0 matches
{
"regex": "/fuck|shit|damn/",
// if "matchThreshold" is not specified it defaults to this -- default behavior is to trigger if there are any matches
// "matchThreshold": "> 0"
},
]
}

View File

@@ -0,0 +1,6 @@
name: swear
kind: regex
criteria:
- regex: '/fuck|shit|damn/'
# if "matchThreshold" is not specified it defaults to this -- default behavior is to trigger if there are any matches
#matchThreshold: "> 0"

View File

@@ -0,0 +1,15 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
// triggers if any activity in the last 10 (including current activity) match the regex
{
"regex": "/fuck|shit|damn/",
// if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
// learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
"window": 10,
},
]
}

View File

@@ -0,0 +1,8 @@
name: swear
kind: regex
criteria:
# triggers if any activity in the last 10 (including current activity) match the regex
- regex: '/fuck|shit|damn/'
# if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
# learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
window: 10

View File

@@ -0,0 +1,19 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
{
// triggers if the current activity has more than 0 matches
// if the activity is a submission then matches against title, body, and url
// if "testOn" is not provided then `title, body` are the defaults
"regex": "/fuck|shit|damn/",
"testOn": [
"title",
"body",
"url"
]
},
]
}

View File

@@ -0,0 +1,11 @@
name: swear
kind: regex
criteria:
- regex: '/fuck|shit|damn/'
# triggers if the current activity has more than 0 matches
# if the activity is a submission then matches against title, body, and url
# if "testOn" is not provided then `title, body` are the defaults
testOn:
- title
- body
- url

View File

@@ -0,0 +1,23 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
// triggers if there are more than 5 regex matches in the last 10 activities (comments only)
{
"regex": "/fuck|shit|damn/",
// this differs from "activityMatchThreshold"
//
// activityMatchThreshold => # of activities from window must match regex
// totalMatchThreshold => # of matches across all activities from window must match regex
"totalMatchThreshold": "> 5",
// if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
// learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
"window": 10,
// determines which activities from window to consider
//defaults to "all" (submissions and comments)
"lookAt": "comments",
},
]
}

View File

@@ -0,0 +1,16 @@
name: swear
kind: regex
criteria:
# triggers if there are more than 5 regex matches in the last 10 activities (comments only)
- regex: '/fuck|shit|damn/'
# this differs from "activityMatchThreshold"
#
# activityMatchThreshold => # of activities from window must match regex
# totalMatchThreshold => # of matches across all activities from window must match regex
totalMatchThreshold: '> 5'
# if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
# learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
window: 10
# determines which activities from window to consider
# defaults to "all" (submissions and comments)
lookAt: comments

View File

@@ -0,0 +1,13 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
{
"regex": "/fuck|shit|damn/",
// triggers if current activity has greater than 5 matches
"matchThreshold": "> 5"
},
]
}

View File

@@ -0,0 +1,6 @@
name: swear
kind: regex
criteria:
- regex: '/fuck|shit|damn/'
# triggers if current activity has greater than 5 matches
matchThreshold: '> 5'

View File

@@ -0,0 +1,21 @@
// goes inside
// "rules": []
{
"name": "swear",
"kind": "regex",
"criteria": [
// triggers if there are more than 5 regex matches in the last 10 activities (comments or submission)
{
// triggers if there are more than 5 *total matches* across the last 10 activities
"regex": "/fuck|shit|damn/",
// this differs from "activityMatchThreshold"
//
// activityMatchThreshold => # of activities from window must match regex
// totalMatchThreshold => # of matches across all activities from window must match regex
"totalMatchThreshold": "> 5",
// if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
// learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
"window": 10,
},
]
}

View File

@@ -0,0 +1,13 @@
name: swear
kind: regex
criteria:
# triggers if there are more than 5 regex matches in the last 10 activities (comments or submission)
- regex: '/fuck|shit|damn/'
# this differs from "activityMatchThreshold"
#
# activityMatchThreshold => # of activities from window must match regex
# totalMatchThreshold => # of matches across all activities from window must match regex
totalMatchThreshold: '> 5'
# if `window` is specified it tells the rule to check the current activity as well as the activities returned from `window`
# learn more about `window` here https://github.com/FoxxMD/context-mod/blob/master/docs/activitiesWindow.md
window: 10

View File

@@ -0,0 +1,73 @@
{
"checks": [
{
"name": "remove discord spam",
"notifyOnTrigger": true,
"description": "remove comments from users who are spamming discord links",
"kind": "comment",
"authorIs": {
"exclude": [
{
"isMod": true
}
]
},
"itemIs": [
{
"removed": false,
"approved": false,
}
],
"condition": "OR",
"rules": [
{
// set to false if you want to allow comments with a discord link ONLY IF
// the author doesn't have a history of spamming discord links
// -- basically allows one-off/organic discord links
"enable": true,
"name": "linkOnlySpam",
"kind": "regex",
"criteria": [
{
"name": "only link",
"regex": "/^.*(discord\\.gg\\/[\\w\\d]+)$/i",
}
]
},
{
"condition": "AND",
"rules": [
{
"name": "linkAnywhereSpam",
"kind": "regex",
"criteria": [
{
"name": "contains link anywhere",
"regex": "/^.*(discord\\.gg\\/[\\w\\d]+).*$/i",
}
]
},
{
"name": "linkAnywhereHistoricalSpam",
"kind": "regex",
"criteria": [
{
"name": "contains links anywhere historically",
"regex": "/^.*(discord\\.gg\\/[\\w\\d]+).*$/i",
"totalMatchThreshold": ">= 3",
"lookAt": "comments",
"window": 10
}
]
}
]
}
],
"actions": [
{
"kind": "remove"
}
]
}
]
}

View File

@@ -0,0 +1,36 @@
checks:
- name: remove discord spam
notifyOnTrigger: true
description: remove comments from users who are spamming discord links
kind: comment
authorIs:
exclude:
- isMod: true
itemIs:
- removed: false
approved: false
condition: OR
rules:
- enable: true
name: linkOnlySpam
kind: regex
criteria:
- name: only link
regex: '/^.*(discord\.gg\/[\w\d]+)$/i'
- condition: AND
rules:
- name: linkAnywhereSpam
kind: regex
criteria:
- name: contains link anywhere
regex: '/^.*(discord\.gg\/[\w\d]+).*$/i'
- name: linkAnywhereHistoricalSpam
kind: regex
criteria:
- name: contains links anywhere historically
regex: '/^.*(discord\.gg\/[\w\d]+).*$/i'
totalMatchThreshold: '>= 3'
lookAt: comments
window: 10
actions:
- kind: remove

View File

@@ -0,0 +1,49 @@
# Repeat Activity
The **Repeat Activity** rule will check for patterns of repetition in an Author's Submission/Comment history. Consult the [schema](https://json-schema.app/view/%23%2Fdefinitions%2FRepeatActivityJSONConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the rule's properties.
## Tuning
The most critical properties for this Rule are **gapAllowance** and **lookAt**.
### `lookAt`
Determines which Activities from a User's history are checked when looking for repeats.
Can be either:
* `all` -- All of a user's submissions and comments are considered
* `submissions` -- Only a user's submissions are considered
Defaults to `all`
### `gapAllowance`
`gapAllowance` determines how many **non-repeat Activities** are "allowed" between "in a row" submissions. `N` number of non-repeat activities will be thrown away during the count which allows checking for patterns with a bit of "fuzziness".
By default `gapAllowance: 0` so all repeats must be truly consecutive.
___
Consider the following example in a user's history:
* crossposts 2 times
* 1 comment
* crossposts 2 times
* 2 comments
* crossposts 4 times
Your goal is to remove a submission if it has been crossposted **5 times.**
With defaults for lookAt and gapAllowance this rule **would not be triggered** because no set of consecutive submissions was repeated 5 times.
With only `lookAt: "submissions"` this rule **would trigger** because all the comments would be ignored resulting in 8 repeats.
With only `gapAllowance: 1` this rule **would not trigger** because the 2 comment non-repeat would break the "in a row" count.
With only `gapAllowance: 2` this rule **would trigger** because the the 1 and 2 comment non-repeats would be thrown out resulting in 8 repeats.
**Note:** `lookAt: "submissions"` should be used with caution because all comments are thrown away. This isn't indicative of real repeat behavior if the user is a heavy commenter. For this reason the default is `all`.
## Examples
* Crosspost Spamming [JSON](/docs/examples/repeatActivity/crosspostSpamming.json5) | [YAML](/docs/examples/repeatActivity/crosspostSpamming.yaml) - Check if an Author is spamming their Submissions across multiple subreddits
* Burst-posting [JSON](/docs/examples/repeatActivity/burstPosting.json5) | [YAML](/docs/examples/repeatActivity/burstPosting.yaml) - Check if Author is crossposting their Submissions in short bursts

View File

@@ -0,0 +1,30 @@
{
"checks": [
{
"name": "Burstpost Spam",
"description": "Check if Author is crossposting in short bursts",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "burstpost",
"kind": "repeatActivity",
// will only look at Submissions in Author's history that contain the same content (link) as the Submission this check was initiated by
"useSubmissionAsReference": true,
// the number of non-repeat activities (submissions or comments) to ignore between repeat submissions
"gapAllowance": 3,
// if the Author has posted this Submission 6 times, ignoring 3 non-repeat activities between each repeat, then this rule will trigger
"threshold": ">= 6",
// look at all of the Author's submissions in the last 7 days
"window": "7 days"
}
],
"actions": [
{
"kind": "report",
"content": "Author has burst-posted this link {{rules.burstpost.largestRepeat}} times over {{rules.burstpost.window}}"
}
]
}
]
}

View File

@@ -0,0 +1,23 @@
checks:
- name: Burstpost Spam
description: Check if Author is crossposting in short bursts
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: burstpost
kind: repeatActivity
# will only look at Submissions in Author's history that contain the same content (link) as the Submission this check was initiated by
useSubmissionAsReference: true
# the number of non-repeat activities (submissions or comments) to ignore between repeat submissions
gapAllowance: 3
# if the Author has posted this Submission 6 times, ignoring 3 non-repeat activities between each repeat, then this rule will trigger
threshold: '>= 6'
# look at all of the Author's submissions in the last 7 days or 100 submissions
window:
duration: 7 days
count: 100
actions:
- kind: report
content: >-
Author has burst-posted this link {{rules.burstpost.largestRepeat}}
times over {{rules.burstpost.window}}

View File

@@ -0,0 +1,28 @@
{
"checks": [
{
"name": "Crosspost Spam",
"description": "Check if Author is spamming Submissions across subreddits",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "xpostspam",
"kind": "repeatActivity",
// will only look at Submissions in Author's history that contain the same content (link) as the Submission this check was initiated by
"useSubmissionAsReference": true,
// if the Author has posted this Submission 5 times consecutively then this rule will trigger
"threshold": ">= 5",
// look at all of the Author's submissions in the last 7 days
"window": "7 days"
}
],
"actions": [
{
"kind": "report",
"content": "Author has posted this link {{rules.xpostspam.largestRepeat}} times over {{rules.xpostspam.window}}"
}
]
}
]
}

View File

@@ -0,0 +1,19 @@
checks:
- name: Crosspost Spam
description: Check if Author is spamming Submissions across subreddits
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: xpostspam
kind: repeatActivity
# will only look at Submissions in Author's history that contain the same content (link) as the Submission this check was initiated by
useSubmissionAsReference: true
# if the Author has posted this Submission 5 times consecutively then this rule will trigger
threshold: '>= 5'
# look at all of the Author's submissions in the last 7 days
window: 7 days
actions:
- kind: report
content: >-
Author has posted this link {{rules.xpostspam.largestRepeat}} times
over {{rules.xpostspam.window}}

View File

@@ -0,0 +1,927 @@
The **Repost** rule is used to find reposts for both **Submissions** and **Comments**, depending on what type of **Check** it is used on.
Note: This rule is for searching **all of Reddit** for reposts, as opposed to just the Author of the Activity being checked. If you only want to check for reposts by the Author of the Activity being checked you should use the [Repeat Activity](/docs/examples/repeatActivity) rule.
# TLDR
Out of the box CM generates a repost rule with sensible default behavior without any configuration. You do not need to configure any of below options (facets, modifiers, criteria) yourself in order to have a working repost rule. Default behavior is as follows...
* When looking for Submission reposts CM will find any Submissions with
* a very similar title
* or independent of title...
* any crossposts/duplicates
* any submissions with the exact URL
* When looking for Comment reposts CM will do the above AND THEN
* compare the top 50 most-upvoted comments from the top 10 most-upvoted Submissions against the comment being checked
* compare any items found from external source (Youtube comments, etc...) against the comment being checked
# Configuration
## Search Facets
ContextMod has several ways to search for reposts -- all of which look at different elements of a Submission in order to find repost candidates. You can define any/all of these **Search Facets** you want to use to search Reddit inside the configuration for the Repost Rule in the `searchOn` property.
### Usage
Facets are specified in the `searchOn` array property within the rule's configuration.
**String**
Specify one or more types of facets as a string to use their default configurations
<details>
YAML
```yaml
kind: repost
criteria:
- searchOn:
- title
- url
- crossposts
```
JSON
```json5
{
"kind": "repost",
"criteria": [
{
// ...
"searchOn": ["title", "url", "crossposts"],
// ....
}
]
}
```
</details>
**Object**
**string** and object configurations can be mixed
<details>
```yaml
kind: repost
criteria:
- searchOn:
- title
- kind: url
matchScore: 90
- external
```
```json5
{
"kind": "repost",
"criteria": [
{
// ...
"searchOn": [
"title",
{
"kind": "url",
// could also specify multiple types to use the same config for all
//"kind": ["url", "duplicates"]
"matchScore": 90,
//...
},
"external"
],
// ....
}
]
}
```
</details>
### Facet Types
* **title** -- search reddit for Submissions with a similar title
* **url** -- search reddit for Submissions with the same URL
* **duplicates** -- get all Submissions **reddit has identified** as duplicates that are **NOT** crossposts
* these are found under *View discussions in other communities* (new reddit) or *other discussions* (old reddit) on the Submission
* **crossposts** -- get all Submissions where the current Submission is the source of an **official** crosspost
* this differs from duplicates in that crossposts use reddit's built-in crosspost functionality, respect subreddit crosspost rules, and link back to the original Submission
* **external** -- get items from the Submission's link source that may be reposted (currently implemented for **Comment Checks** only)
* When the Submission link is for...
* **Youtube** -- get top comments on video by replies/like count
* **NOTE:** An **API Key** for the [Youtube Data API](https://developers.google.com/youtube/v3) must be provided for this facet to work. This can be provided by the operator alongside [bot credentials](/docs/operatorConfiguration.md) or in the top-level `credentials` property for a [subreddit configuration.](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)
### Facet Modifiers
For all **Facets**, except for **external**, there are options that be configured to determine if the found Submissions is a "valid" repost IE filtering. These options can be configured **per facet**.
* **matchScore** -- The percentage, as a whole number, of a repost title that must match the title being checked in order to consider both a match
* **minWordCount** -- The minimum number of words a title must have
* **caseSensitive** -- If the match comparison should be case-sensitive (defaults to `false`)
Additionally, the current Activity's title and/or each repost's title can be transformed before matching:
* **transformations** -- An array of SearchAndReplace objects used to transform the repost's title
* **transformationsActivity** -- An array of SearchAndReplace objects used to transform the current Activity's title
#### Modifier Defaults
To make facets easier to use without configuration sensible defaults are applied to each when no other configuration is defined...
* **title**
* `matchScore: 85` -- The candidate repost's title must be at least 85% similar to the current Activity's title
* `minWordCount: 2` -- The candidate repost's title must have at least 2 words
For `url`,`duplicates`, and `crossposts` the only default is `matchScore: 0` because the assumption is you want to treat any actual dups/x-posts or exact URLs as reposts, regardless of their title.
## Additional Criteria Properties
A **criteria** object may also specify some additional tests to run against the reposts found from searching.
### For Submissions and Comments
#### Occurrences
Define a set of criteria to test against the **number of reposts**, **time reposts were created**, or both.
##### Count
<details>
```yaml
kind: repost
criteria:
- searchOn:
- title
- url
- crossposts
occurrences:
criteria:
- count:
condition: AND
test:
- '> 3'
- <= 5
```
```json5
{
"kind": "repost",
"criteria": [
{
// ...
"searchOn": ["title", "url", "crossposts"],
"occurrences": {
"criteria": [
{
// passes if BOTH tests are true
"count": {
"condition": "AND", // default is AND
"test": [
"> 3", // TRUE if there are GREATER THAN 3 reposts found
"<= 5" // TRUE if there are LESS THAN OR EQUAL TO 5 reposts found
]
}
}
],
}
}
]
}
```
</details>
##### Time
Define a test or array of tests to run against **when reposts were created**
<details>
```yaml
kind: repost
criteria:
- searchOn:
- title
- url
- crossposts
occurrences:
criteria:
- time:
condition: AND
test:
- testOn: all
condition: '> 3 months'
```
```json5
{
"kind": "repost",
"criteria": [
{
// ...
"searchOn": [
"title",
"url",
"crossposts"
],
"occurrences": {
"criteria": [
{
time: {
// how to test array of comparisons. AND => all must pass, OR => any must pass
"condition": "AND",
"test": [
{
// which of the found reposts to test the time comparison on
//
// "all" => ALL reposts must pass time comparison
// "any" => ANY repost must pass time comparison
// "newest" => The newest (closest in time to now) repost must pass time comparison
// "oldest" => The oldest (furthest in time from now) repost must pass time comparison
//
"testOn": "all",
// Tested items must be OLDER THAN 3 months
"condition": "> 3 months"
}
]
}
}
]
},
}
]
}
```
</details>
### For Comments
When the rule is run in a **Comment Check** you may specify text comparisons (like those found in Search Facets) to run on the contents of the repost comments *against* the contents of the comment being checked.
* **matchScore** -- The percentage, as a whole number, of a repost comment that must match the comment being checked in order to consider both a match (defaults to 85% IE `85`)
* **minWordCount** -- The minimum number of words a comment must have
* **caseSensitive** -- If the match comparison should be case-sensitive (defaults to `false`)
# Examples
Examples of a *full* CM configuration, including the Repost Rule, in various scenarios. In each scenario the parts of the configuration that affect the rule are indicated.
## Submissions
When the Repost Rule is run on a **Submission Check** IE the activity being checked is a Submission.
### Default Behavior (No configuration)
This is the same behavior described in the [TLDR](#TLDR) section above -- find any submissions with:
* a very similar title (85% or more the same)
* or ignoring title...
* any crossposts/duplicates
* any submissions with the exact URL
<details>
```yaml
polling:
- unmoderated
checks:
- name: subRepost
description: Check if submission has been reposted
kind: submission
condition: AND
rules:
- kind: repost
actions:
- kind: report
content: This submission was reposted
```
```json5
{
"polling": [
"unmoderated"
],
"checks": [
{
"name": "subRepost",
"description": "Check if submission has been reposted",
// kind specifies this check is for SUBMISSIONS
"kind": "submission",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost"
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This submission was reposted"
}
]
}
]
}
```
</details>
### Search by Title Only
Find any submissions with:
* a very similar title (85% or more the same)
<details>
```yaml
polling:
- unmoderated
checks:
- name: subRepost
description: Check if submission has been reposted
kind: submission
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- title
actions:
- kind: report
content: This submission was reposted
```
```json5
{
"polling": [
"unmoderated"
],
"checks": [
{
"name": "subRepost",
"description": "Check if submission has been reposted",
// kind specifies this check is for SUBMISSIONS
"kind": "submission",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
// specify only title to search on
"searchOn": [
"title" // uses default configuration since only string is specified
]
}
]
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This submission was reposted"
}
]
}
]
}
```
</details>
### Search by Title only and specify similarity percentage
* a very similar title (95% or more the same)
<details>
```yaml
polling:
- unmoderated
checks:
- name: subRepost
description: Check if submission has been reposted
kind: submission
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- kind: title
matchScore: '95'
actions:
- kind: report
content: This submission was reposted
```
```json5
{
"polling": [
"unmoderated"
],
"checks": [
{
"name": "subRepost",
"description": "Check if submission has been reposted",
// kind specifies this check is for SUBMISSIONS
"kind": "submission",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
// specify only title to search on
"searchOn": [
{
"kind": "title",
// titles must be 95% or more similar
"matchScore": "95"
}
]
}
]
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This submission was reposted"
}
]
}
]
}
```
</details>
### Search by Title, specify similarity percentage, AND any duplicates
<details>
```yaml
polling:
- unmoderated
checks:
- name: subRepost
description: Check if submission has been reposted
kind: submission
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- duplicates
- kind: title
matchScore: '95'
actions:
- kind: report
content: This submission was reposted
```
```json5
{
"polling": [
"unmoderated"
],
"checks": [
{
"name": "subRepost",
"description": "Check if submission has been reposted",
// kind specifies this check is for SUBMISSIONS
"kind": "submission",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
"searchOn": [
// look for duplicates (NON crossposts) using default configuration
"duplicates",
// search by title
{
"kind": "title",
// titles must be 95% or more similar
"matchScore": "95"
}
]
}
]
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This submission was reposted"
}
]
}
]
}
```
</details>
### Approve Submission if not reposted in the last month, by title
<details>
```yaml
polling:
- unmoderated
checks:
- name: subRepost
description: Check there are no reposts with same title in the last month
kind: submission
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- title
occurrences:
condition: OR
criteria:
- count:
test:
- < 1
- time:
test:
- testOn: newest
condition: '> 1 month'
actions:
- kind: approve
```
```json5
{
"polling": [
"unmoderated"
],
"checks": [
{
"name": "subRepost",
"description": "Check there are no reposts with same title in the last month",
// kind specifies this check is for SUBMISSIONS
"kind": "submission",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
"searchOn": [
"title"
],
"occurrences": {
// if EITHER criteria is TRUE then it "passes"
"condition": "OR",
"criteria": [
// first criteria:
// TRUE if there are LESS THAN 1 reposts (no reposts found)
{
"count": {
"test": ["< 1"]
}
},
// second criteria:
// TRUE if the newest repost is older than one month
{
"time": {
"test": [
{
"testOn": "newest",
"condition": "> 1 month"
}
]
}
}
]
},
}
]
},
//
// repost rule configuration is above
],
"actions": [
{
// approve this post since we know it is not a repost of anything within the last month
"kind": "approve",
}
]
}
]
}
```
</details>
## Comments
### Default Behavior (No configuration)
This is the same behavior described in the [TLDR](#TLDR) section above -- find any submissions with:
* a very similar title (85% or more the same)
* or ignoring title...
* any crossposts/duplicates
* any submissions with the exact URL
* If comment being checked is on a Submission for Youtube then get top 50 comments on youtube video as well...
AND THEN
* sort submissions by votes
* take top 20 (upvoted) comments from top 10 (upvoted) submissions
* sort comments by votes, take top 50 + top 50 external items
FINALLY
* filter all gathered comments by default `matchScore: 85` to find very similar matches
* rules is triggered if any are found
<details>
```yaml
polling:
- newComm
checks:
- name: commRepost
description: Check if comment has been reposted
kind: common
condition: AND
rules:
- kind: repost
actions:
- kind: report
content: This comment was reposted
```
```json5
{
"polling": [
"newComm"
],
"checks": [
{
"name": "commRepost",
"description": "Check if comment has been reposted",
// kind specifies this check is for COMMENTS
"kind": "common",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost"
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This comment was reposted"
}
]
}
]
}
```
</details>
### Search by external (youtube) comments only
<details>
```yaml
polling:
- newComm
checks:
- name: commRepost
description: Check if comment has been reposted from youtube
kind: comment
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- external
actions:
- kind: report
content: This comment was reposted from youtube
```
```json5
{
"polling": [
"newComm"
],
"checks": [
{
"name": "commRepost",
"description": "Check if comment has been reposted from youtube",
// kind specifies this check is for SUBMISSIONS
"kind": "comment",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
// specify only external (youtube) to search on
"searchOn": [
"external"
]
}
]
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This comment was reposted from youtube"
}
]
}
]
}
```
</details>
### Search by external (youtube) comments only, with higher comment match percentage
<details>
```yaml
polling:
- newComm
checks:
- name: commRepost
description: Check if comment has been reposted from youtube
kind: comment
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- external
matchScore: 95
actions:
- kind: report
content: This comment was reposted from youtube
```
```json5
{
"polling": [
"newComm"
],
"checks": [
{
"name": "commRepost",
"description": "Check if comment has been reposted from youtube",
// kind specifies this check is for SUBMISSIONS
"kind": "comment",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
// specify only external (youtube) to search on
"searchOn": [
"external"
],
"matchScore": 95 // matchScore for comments is on criteria instead of searchOn config...
},
]
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This comment was reposted from youtube"
}
]
}
]
}
```
</details>
### Search by external (youtube) comments and submission URL, with higher comment match percentage
<details>
```yaml
polling:
- newComm
checks:
- name: commRepost
description: Check if comment has been reposted
kind: comment
condition: AND
rules:
- kind: repost
criteria:
- searchOn:
- external
- url
matchScore: 95
actions:
- kind: report
content: >-
This comment was reposted from youtube or from submission with the
same URL
```
```json5
{
"polling": [
"newComm"
],
"checks": [
{
"name": "commRepost",
"description": "Check if comment has been reposted",
// kind specifies this check is for SUBMISSIONS
"kind": "comment",
"condition": "AND",
"rules": [
// repost rule configuration is below
//
{
"kind": "repost",
"criteria": [
{
// specify only external (youtube) to search on
"searchOn": [
"external",
// can specify any/all submission search facets to acquire comments from
"url"
],
"matchScore": 95 // matchScore for comments is on criteria instead of searchOn config...
},
]
},
//
// repost rule configuration is above
],
"actions": [
{
"kind": "report",
"content": "This comment was reposted from youtube or from submission with the same URL"
}
]
}
]
}
```
</details>

View File

@@ -0,0 +1,78 @@
Provided here are **complete, ready-to-go configuration** that can copy-pasted straight into your configuration wiki page to get going with ContextMod immediately.
These configurations attempt to provide sensible, non-destructive, default behavior for some common scenarios and subreddit types.
In most cases these will perform decently out-of-the-box but they are not perfect. You should still monitor bot behavior to see how it performs and will most likely still need to tweak these configurations to get your desired behavior.
All actions for these configurations are non-destructive in that:
* All instances where an activity would be modified (remove/ban/approve) will have `dryRun: true` set to prevent the action from actually being performed
* These instances will also have a `report` action detailing the action would have been performed
**You will have to remove the `report` action and `dryRun` settings yourself.** This is to ensure that you understand the behavior the bot will be performing. If you are unsure of this you should leave them in place until you are certain the behavior the bot is performing is acceptable.
**YAML** is the same format as **automoderator**
## Submission-based Behavior
### Remove submissions from users who have used 'freekarma' subs to bypass karma checks
[YAML](/docs/examples/subredditReady/freekarma.yaml) | [JSON](/docs/examples/subredditReady/freekarma.json5)
If the user has any activity (comment/submission) in known freekarma subreddits in the past (50 activities or 6 months) then remove the submission.
### Remove submissions from users who have crossposted the same submission 4 or more times
[YAML](/docs/examples/subredditReady/crosspostSpam.yaml) | [JSON](/docs/examples/subredditReady/crosspostSpam.yaml)
If the user has crossposted the same submission in the past (50 activities or 6 months) 4 or more times in a row then remove the submission.
### Remove submissions from users who have crossposted or used 'freekarma' subs
[YAML](/docs/examples/subredditReady/freeKarmaOrCrosspostSpam.yaml) | [JSON](/docs/examples/subredditReady/freeKarmaOrCrosspostSpam.json5)
Will remove submission if either of the above two behaviors is detected
### Remove link submissions where the user's history is comprised of 10% or more of the same link
[YAML](/docs/examples/subredditReady/selfPromo.yaml) | [JSON](/docs/examples/subredditReady/selfPromo.json5)
If the link origin (youtube author, twitter author, etc. or regular domain for non-media links)
* comprises 10% or more of the users **entire** history in the past (100 activities or 6 months)
* or comprises 10% or more of the users **submission** history in the past (100 activities or 6 months) and the user has low engagement (<50% of history is comments or 40%> of comment are as OP)
then remove the submission
## Comment-based behavior
### Remove comment if the user has posted the same comment 4 or more times in a row
[YAML](/docs/examples/subredditReady/commentSpam.yaml) | [JSON](/docs/examples/subredditReady/commentSpam.json5)
If the user made the same comment (with some fuzzy matching) 4 or more times in a row in the past (50 activities or 6 months) then remove the comment.
### Remove comment if it is discord invite link spam
[YAML](/docs/examples/subredditReady/discordSpam.yaml) | [JSON](/docs/examples/subredditReady/discordSpam.json5)
This rule goes a step further than automod can by being more discretionary about how it handles this type of spam.
* Remove the comment and **ban a user** if:
* Comment being checked contains **only** a discord link (no other text) AND
* Discord links appear **anywhere** in three or more of the last 10 comments the Author has made
otherwise...
* Remove the comment if:
* Comment being checked contains **only** a discord link (no other text) OR
* Comment contains a discord link **anywhere** AND
* Discord links appear **anywhere** in three or more of the last 10 comments the Author has made
Using these checks ContextMod can more easily distinguish between these use cases for a user commenting with a discord link:
* actual spammers who only spam a discord link
* users who may comment with a link but have context for it either in the current comment or in their history
* users who many comment with a link but it's a one-off event (no other links historically)
Additionally, you could modify both/either of these checks to not remove one-off discord link comments but still remove if the user has a historical trend for spamming links

View File

@@ -0,0 +1,42 @@
{
"polling": ["newComm"],
"checks": [
{
//
// Stop users who spam the same comment many times
//
// Remove a COMMENT if the user has crossposted it at least 4 times in recent history
//
"name": "low xp comment spam",
"description": "X-posted comment >=4x",
"kind": "comment",
"condition": "AND",
"rules": [
{
"name": "xPostLow",
"kind": "repeatActivity",
"gapAllowance": 2,
"threshold": ">= 4",
"window": {
"count": 50,
"duration": "6 months"
}
},
],
"actions": [
// remove this after confirming behavior is acceptable
{
"kind": "report",
"content": "Remove=> Posted same comment {{rules.xpostlow.largestRepeat}}x times"
},
//
//
{
"kind": "remove",
// remove the line below after confirming behavior is acceptable
"dryRun": true
}
]
}
]
}

View File

@@ -0,0 +1,25 @@
polling:
- newComm
checks:
# Stop users who spam the same comment many times
- name: low xp comment spam
description: X-posted comment >=4x
kind: comment
condition: AND
rules:
- name: xPostLow
kind: repeatActivity
# number of "non-repeat" comments allowed between "repeat comments"
gapAllowance: 2
# greater or more than 4 repeat comments triggers this rule
threshold: '>= 4'
# retrieve either last 50 comments or 6 months' of history, whichever is less
window:
count: 50
duration: 6 months
actions:
- kind: report
enable: true
content: 'Remove => Posted same comment {{rules.xpostlow.largestRepeat}}x times'
- kind: remove
enable: true

View File

@@ -0,0 +1,77 @@
{
"polling": ["unmoderated"],
"checks": [
{
//
// Stop users who post low-effort, crossposted spam
//
// Remove a SUBMISSION if the user has crossposted it at least 4 times in recent history AND
// less than 50% of their activity is comments OR more than 40% of those comments are as OP (in the own submissions)
//
"name": "low xp spam and engagement",
"description": "X-posted 4x and low comment engagement",
"kind": "submission",
"itemIs": [
{
"removed": false
}
],
"condition": "AND",
"rules": [
{
"name": "xPostLow",
"kind": "repeatActivity",
"gapAllowance": 2,
"threshold": ">= 4",
"window": {
"count": 50,
"duration": "6 months"
}
},
{
"name": "lowOrOpComm",
"kind": "history",
"criteriaJoin": "OR",
"criteria": [
{
"window": {
"count": 100,
"duration": "6 months"
},
"comment": "< 50%"
},
{
"window": {
"count": 100,
"duration": "6 months"
},
"comment": "> 40% OP"
}
]
}
],
"actions": [
// remove this after confirming behavior is acceptable
{
"kind": "report",
"content": "Remove=>{{rules.xpostlow.largestRepeat}} X-P => {{rules.loworopcomm.thresholdSummary}}"
},
//
//
{
"kind": "remove",
// remove the line below after confirming behavior is acceptable
"dryRun": true
},
// optionally remove "dryRun" from below if you want to leave a comment on removal
// PROTIP: the comment is bland, you should make it better
{
"kind": "comment",
"content": "Your submission has been removed because you cross-posted it {{rules.xpostlow.largestRepeat}} times and you have very low engagement outside of making submissions",
"distinguish": true,
"dryRun": true
}
]
}
]
}

View File

@@ -0,0 +1,48 @@
polling:
- unmoderated
checks:
# stop users who post low-effort, crossposted spam submissions
#
# Remove a SUBMISSION if the user has crossposted it at least 4 times in recent history AND
# less than 50% of their activity is comments OR more than 40% of those comments are as OP (in the own submissions)
- name: low xp spam and engagement
description: X-posted 4x and low comment engagement
kind: submission
itemIs:
- removed: false
condition: AND
rules:
- name: xPostLow
kind: repeatActivity
gapAllowance: 2
threshold: '>= 4'
window:
count: 50
duration: 6 months
- name: lowOrOpComm
kind: history
criteriaJoin: OR
criteria:
- window:
count: 100
duration: 6 months
comment: < 50%
- window:
count: 100
duration: 6 months
comment: '> 40% OP'
actions:
- kind: report
enable: true
content: >-
Remove=>{{rules.xpostlow.largestRepeat}} X-P =>
{{rules.loworopcomm.thresholdSummary}}
- kind: remove
enable: true
- kind: comment
enable: true
content: >-
Your submission has been removed because you cross-posted it
{{rules.xpostlow.largestRepeat}} times and you have very low
engagement outside of making submissions
distinguish: true

View File

@@ -0,0 +1,75 @@
{
"polling": ["newComm"],
"checks": [
{
"name": "ban discord only spammer",
"description": "ban a user who spams only a discord link many times historically",
"kind": "comment",
"condition": "AND",
"rules": [
"linkOnlySpam",
"linkAnywhereHistoricalSpam",
],
"actions": [
{
"kind": "remove"
},
{
"kind": "ban",
"content": "spamming discord links"
}
]
},
{
"name": "remove discord spam",
"description": "remove comments from users who only link to discord or mention discord link many times historically",
"kind": "comment",
"condition": "OR",
"rules": [
{
"name": "linkOnlySpam",
"kind": "regex",
"criteria": [
{
"name": "only link",
"regex": "/^.*(discord\\.gg\\/[\\w\\d]+)$/i",
}
]
},
{
"condition": "AND",
"rules": [
{
"name": "linkAnywhereSpam",
"kind": "regex",
"criteria": [
{
"name": "contains link anywhere",
"regex": "/^.*(discord\\.gg\\/[\\w\\d]+).*$/i",
}
]
},
{
"name": "linkAnywhereHistoricalSpam",
"kind": "regex",
"criteria": [
{
"name": "contains links anywhere historically",
"regex": "/^.*(discord\\.gg\\/[\\w\\d]+).*$/i",
"totalMatchThreshold": ">= 3",
"lookAt": "comments",
"window": 10
}
]
}
]
}
],
"actions": [
{
"kind": "remove"
}
]
}
]
}

View File

@@ -0,0 +1,46 @@
polling:
- newComm
checks:
- name: ban discord only spammer
description: ban a user who spams only a discord link many times historically
kind: comment
condition: AND
rules:
- linkOnlySpam
- linkAnywhereHistoricalSpam
actions:
- kind: remove
- kind: ban
content: spamming discord links
- name: remove discord spam
description: >-
remove comments from users who only link to discord or mention discord
link many times historically
kind: comment
condition: OR
rules:
- name: linkOnlySpam
kind: regex
criteria:
- name: only link
# single quotes are required to escape special characters
regex: '/^.*(discord\.gg\/[\w\d]+)$/i'
- condition: AND
rules:
- name: linkAnywhereSpam
kind: regex
criteria:
- name: contains link anywhere
# single quotes are required to escape special characters
regex: '/^.*(discord\.gg\/[\w\d]+).*$/i'
- name: linkAnywhereHistoricalSpam
kind: regex
criteria:
- name: contains links anywhere historically
# single quotes are required to escape special characters
regex: '/^.*(discord\.gg\/[\w\d]+).*$/i'
totalMatchThreshold: '>= 3'
lookAt: comments
window: 10
actions:
- kind: remove

View File

@@ -0,0 +1,138 @@
{
"polling": [
"unmoderated"
],
"checks": [
{
//
// Stop users who post low-effort, crossposted spam
//
// Remove a SUBMISSION if the user has crossposted it at least 4 times in recent history AND
// less than 50% of their activity is comments OR more than 40% of those comments are as OP (in the own submissions)
//
"name": "remove on low xp spam and engagement",
"description": "X-posted 4x and low comment engagement",
"kind": "submission",
"itemIs": [
{
"removed": false
}
],
"condition": "AND",
"rules": [
{
"name": "xPostLow",
"kind": "repeatActivity",
"gapAllowance": 2,
"threshold": ">= 4",
"window": {
"count": 50,
"duration": "6 months"
}
},
{
"name": "lowOrOpComm",
"kind": "history",
"criteriaJoin": "OR",
"criteria": [
{
"window": {
"count": 100,
"duration": "6 months"
},
"comment": "< 50%"
},
{
"window": {
"count": 100,
"duration": "6 months"
},
"comment": "> 40% OP"
}
]
}
],
"actions": [
// remove this after confirming behavior is acceptable
{
"kind": "report",
"content": "Remove=>{{rules.xpostlow.largestRepeat}} X-P => {{rules.loworopcomm.thresholdSummary}}"
},
//
//
{
"kind": "remove",
// remove the line below after confirming behavior is acceptable
"dryRun": true
},
// optionally remove "dryRun" from below if you want to leave a comment on removal
// PROTIP: the comment is bland, you should make it better
{
"kind": "comment",
"content": "Your submission has been removed because you cross-posted it {{rules.xpostlow.largestRepeat}} times and you have very low engagement outside of making submissions",
"distinguish": true,
"dryRun": true
}
]
},
{
//
// Remove submissions from users who have recent activity in freekarma subs within the last 50 activities or 6 months (whichever is less)
//
"name": "freekarma removal",
"description": "Remove submission if user has used freekarma sub recently",
"kind": "submission",
"itemIs": [
{
"removed": false
}
],
"condition": "AND",
"rules": [
{
"name": "freekarma",
"kind": "recentActivity",
"window": {
"count": 50,
"duration": "6 months"
},
"useSubmissionAsReference": false,
"thresholds": [
{
"subreddits": [
"FreeKarma4U",
"FreeKarma4You",
"KarmaStore",
"promote",
"shamelessplug",
"upvote"
]
}
]
}
],
"actions": [
// remove this after confirming behavior is acceptable
{
"kind": "report",
"content": "Remove=> {{rules.newtube.totalCount}} activities in freekarma subs"
},
//
//
{
"kind": "remove",
// remove the line below after confirming behavior is acceptable
"dryRun": true
},
// optionally remove "dryRun" from below if you want to leave a comment on removal
// PROTIP: the comment is bland, you should make it better
{
"kind": "comment",
"content": "Your submission has been removed because you have recent activity in 'freekarma' subs",
"distinguish": true,
"dryRun": true
}
]
}
]
}

View File

@@ -0,0 +1,84 @@
polling:
- unmoderated
checks:
# stop users who post low-effort, crossposted spam submissions
#
# Remove a SUBMISSION if the user has crossposted it at least 4 times in recent history AND
# less than 50% of their activity is comments OR more than 40% of those comments are as OP (in the own submissions)
- name: remove on low xp spam and engagement
description: X-posted 4x and low comment engagement
kind: submission
itemIs:
- removed: false
condition: AND
rules:
- name: xPostLow
kind: repeatActivity
gapAllowance: 2
threshold: '>= 4'
window:
count: 50
duration: 6 months
- name: lowOrOpComm
kind: history
criteriaJoin: OR
criteria:
- window:
count: 100
duration: 6 months
comment: < 50%
- window:
count: 100
duration: 6 months
comment: '> 40% OP'
actions:
- kind: report
enable: true
content: >-
Remove=>{{rules.xpostlow.largestRepeat}} X-P =>
{{rules.loworopcomm.thresholdSummary}}
- kind: remove
enable: false
- kind: comment
enable: true
content: >-
Your submission has been removed because you cross-posted it
{{rules.xpostlow.largestRepeat}} times and you have very low
engagement outside of making submissions
distinguish: true
dryRun: true
# Remove submissions from users who have recent activity in freekarma subs within the last 50 activities or 6 months (whichever is less)
- name: freekarma removal
description: Remove submission if user has used freekarma sub recently
kind: submission
itemIs:
- removed: false
condition: AND
rules:
- name: freekarma
kind: recentActivity
window:
count: 50
duration: 6 months
useSubmissionAsReference: false
thresholds:
- subreddits:
- FreeKarma4U
- FreeKarma4You
- KarmaStore
- promote
- shamelessplug
- upvote
actions:
- kind: report
enable: true
content: 'Remove=> {{rules.newtube.totalCount}} activities in freekarma subs'
- kind: remove
enable: false
- kind: comment
enable: true
content: >-
Your submission has been removed because you have recent activity in
'freekarma' subs
distinguish: true
dryRun: true

View File

@@ -0,0 +1,64 @@
{
"polling": [
"unmoderated"
],
"checks": [
{
//
// Remove submissions from users who have recent activity in freekarma subs within the last 50 activities or 6 months (whichever is less)
//
"name": "freekarma removal",
"description": "Remove submission if user has used freekarma sub recently",
"kind": "submission",
"itemIs": [
{
"removed": false
}
],
"condition": "AND",
"rules": [
{
"name": "freekarma",
"kind": "recentActivity",
"window": {
"count": 50,
"duration": "6 months"
},
"useSubmissionAsReference": false,
"thresholds": [
{
"subreddits": [
"FreeKarma4U",
"FreeKarma4You",
"KarmaStore",
"upvote"
]
}
]
}
],
"actions": [
// remove this after confirming behavior is acceptable
{
"kind": "report",
"content": "Remove=> {{rules.newtube.totalCount}} activities in freekarma subs"
},
//
//
{
"kind": "remove",
// remove the line below after confirming behavior is acceptable
"dryRun": true,
},
// optionally remove "dryRun" from below if you want to leave a comment on removal
// PROTIP: the comment is bland, you should make it better
{
"kind": "comment",
"content": "Your submission has been removed because you have recent activity in 'freekarma' subs",
"distinguish": true,
"dryRun": true,
}
]
}
]
}

View File

@@ -0,0 +1,35 @@
polling:
- unmoderated
checks:
# Remove submissions from users who have recent activity in freekarma subs within the last 50 activities or 6 months (whichever is less)
- name: freekarma removal
description: Remove submission if user has used freekarma sub recently
kind: submission
itemIs:
- removed: false
condition: AND
rules:
- name: freekarma
kind: recentActivity
window:
count: 50
duration: 6 months
useSubmissionAsReference: false
thresholds:
- subreddits:
- FreeKarma4U
- FreeKarma4You
- KarmaStore
- upvote
actions:
- kind: report
enable: true
content: 'Remove=> {{rules.newtube.totalCount}} activities in freekarma subs'
- kind: remove
enable: true
- kind: comment
enable: false
content: >-
Your submission has been removed because you have recent activity in
'freekarma' subs
distinguish: true

View File

@@ -0,0 +1,104 @@
{
"polling": [
"unmoderated"
],
"checks": [
{
//
// Stop users who make link submissions with a self-promotional agenda (with reddit's suggested 10% rule)
// https://www.reddit.com/wiki/selfpromotion#wiki_guidelines_for_self-promotion_on_reddit
//
// Remove a SUBMISSION if the link comprises more than or equal to 10% of users history (100 activities or 6 months) OR
//
// if link comprises 10% of submission history (100 activities or 6 months)
// AND less than 50% of their activity is comments OR more than 40% of those comments are as OP (in the own submissions)
//
"name": "Self-promo all AND low engagement",
"description": "Self-promo is >10% for all or just sub and low comment engagement",
"kind": "submission",
"condition": "OR",
"rules": [
{
"name": "attr",
"kind": "attribution",
"criteria": [
{
"threshold": ">= 10%",
"window": {
"count": 100,
"duration": "6 months"
},
"domains": [
"AGG:SELF"
]
}
],
},
{
"condition": "AND",
"rules": [
{
"name": "attrsub",
"kind": "attribution",
"criteria": [
{
"threshold": ">= 10%",
"thresholdOn": "submissions",
"window": {
"count": 100,
"duration": "6 months"
},
"domains": [
"AGG:SELF"
]
}
]
},
{
"name": "lowOrOpComm",
"kind": "history",
"criteriaJoin": "OR",
"criteria": [
{
"window": {
"count": 100,
"duration": "6 months"
},
"comment": "< 50%"
},
{
"window": {
"count": 100,
"duration": "6 months"
},
"comment": "> 40% OP"
}
]
}
]
}
],
"actions": [
{
"kind": "report",
"content": "{{rules.attr.largestPercent}}{{rules.attrsub.largestPercent}} of {{rules.attr.activityTotal}}{{rules.attrsub.activityTotal}} items ({{rules.attr.window}}{{rules.attrsub.window}}){{#rules.loworopcomm.thresholdSummary}} => {{rules.loworopcomm.thresholdSummary}}{{/rules.loworopcomm.thresholdSummary}}"
},
//
//
{
"kind": "remove",
// remove the line below after confirming behavior is acceptable
"dryRun": true
},
// optionally remove "dryRun" from below if you want to leave a comment on removal
// PROTIP: the comment is bland, you should make it better
{
"kind": "comment",
"content": "Your submission has been removed it comprises 10% or more of your recent history ({{rules.attr.largestPercent}}{{rules.attrsub.largestPercent}}). This is against [reddit's self promotional guidelines.](https://www.reddit.com/wiki/selfpromotion#wiki_guidelines_for_self-promotion_on_reddit)",
"distinguish": true,
"dryRun": true
}
]
}
]
}

View File

@@ -0,0 +1,71 @@
polling:
- unmoderated
checks:
#
# Stop users who make link submissions with a self-promotional agenda (with reddit's suggested 10% rule)
# https://www.reddit.com/wiki/selfpromotion#wiki_guidelines_for_self-promotion_on_reddit
#
# Remove a SUBMISSION if the link comprises more than or equal to 10% of users history (100 activities or 6 months) OR
#
# if link comprises 10% of submission history (100 activities or 6 months)
# AND less than 50% of their activity is comments OR more than 40% of those comments are as OP (in the own submissions)
#
- name: Self-promo all AND low engagement
description: Self-promo is >10% for all or just sub and low comment engagement
kind: submission
condition: OR
rules:
- name: attr
kind: attribution
criteria:
- threshold: '>= 10%'
window:
count: 100
duration: 6 months
domains:
- 'AGG:SELF'
- condition: AND
rules:
- name: attrsub
kind: attribution
criteria:
- threshold: '>= 10%'
thresholdOn: submissions
window:
count: 100
duration: 6 months
domains:
- 'AGG:SELF'
- name: lowOrOpComm
kind: history
criteriaJoin: OR
criteria:
- window:
count: 100
duration: 6 months
comment: < 50%
- window:
count: 100
duration: 6 months
comment: '> 40% OP'
actions:
- kind: report
enable: true
content: >-
{{rules.attr.largestPercent}}{{rules.attrsub.largestPercent}} of
{{rules.attr.activityTotal}}{{rules.attrsub.activityTotal}} items
({{rules.attr.window}}{{rules.attrsub.window}}){{#rules.loworopcomm.thresholdSummary}}
=>
{{rules.loworopcomm.thresholdSummary}}{{/rules.loworopcomm.thresholdSummary}}
- kind: remove
enable: false
- kind: comment
enable: true
content: >-
Your submission has been removed it comprises 10% or more of your
recent history
({{rules.attr.largestPercent}}{{rules.attrsub.largestPercent}}). This
is against [reddit's self promotional
guidelines.](https://www.reddit.com/wiki/selfpromotion#wiki_guidelines_for_self-promotion_on_reddit)
distinguish: true
dryRun: true

View File

@@ -0,0 +1,26 @@
# [Toolbox](https://www.reddit.com/r/toolbox/wiki/docs) [User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes)
Context Mod supports reading and writing [User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes) for the [Toolbox](https://www.reddit.com/r/toolbox/wiki/docs) extension.
**You must have Toolbox setup for your subreddit and at least one User Note created before you can use User Notes related features on Context Bot.**
[Click here for the Toolbox Quickstart Guide](https://www.reddit.com/r/toolbox/wiki/docs/quick_start)
## Filter
User Notes are an additional criteria on [AuthorCriteria](https://json-schema.app/view/%23%2Fdefinitions%2FAuthorCriteria?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) that can be used alongside other Author properties for both [filtering rules and in the AuthorRule.](/docs/examples/author/)
Consult the [schema](https://json-schema.app/view/%23%2Fdefinitions%2FUserNoteCriteria?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) for a complete reference of the **UserNoteCriteria** object that can be used in AuthorCriteria.
### Examples
* Do not tag user with Good User note [JSON](/docs/examples/userNotes/usernoteFilter.json5) | [YAML](/docs/examples/userNotes/usernoteFilter.yaml)
## Action
A User Note can also be added to the Author of a Submission or Comment with the [UserNoteAction.](https://json-schema.app/view/%23%2Fdefinitions%2FUserNoteActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json)
### Examples
* Add note on user doing self promotion [JSON](/docs/examples/userNotes/usernoteSP.json5) | [YAML](/docs/examples/userNotes/usernoteSP.yaml)

View File

@@ -0,0 +1,45 @@
{
"checks": [
{
"name": "Self Promo Activities",
"description": "Tag SP only if user does not have good contributor user note",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "attr10all",
"kind": "attribution",
"author": {
"exclude": [
{
// the key of the usernote type to look for https://github.com/toolbox-team/reddit-moderator-toolbox/wiki/Subreddit-Wikis%3A-usernotes#working-with-note-types
// rule will not run if current usernote on Author is of type 'gooduser'
"type": "gooduser"
}
]
},
"criteria": [
{
"threshold": "> 10%",
"window": "90 days"
},
{
"threshold": "> 10%",
"window": 100
}
],
}
],
"actions": [
{
"kind": "usernote",
// the key of usernote type
// https://github.com/toolbox-team/reddit-moderator-toolbox/wiki/Subreddit-Wikis%3A-usernotes#working-with-note-types
"type": "spamwarn",
// content is mustache templated as usual
"content": "Self Promotion: {{rules.attr10all.titlesDelim}} {{rules.attr10sub.largestPercent}}%"
}
]
}
]
}

View File

@@ -0,0 +1,27 @@
checks:
- name: Self Promo Activities
description: Tag SP only if user does not have good contributor user note
# check will run on a new submission in your subreddit and look at the Author of that submission
kind: submission
rules:
- name: attr10all
kind: attribution
author:
exclude:
# the key of the usernote type to look for https://github.com/toolbox-team/reddit-moderator-toolbox/wiki/Subreddit-Wikis%3A-usernotes#working-with-note-types
# rule will not run if current usernote on Author is of type 'gooduser'
- type: gooduser
criteria:
- threshold: '> 10%'
window: 90 days
- threshold: '> 10%'
window: 100
actions:
- kind: usernote
# the key of usernote type
# https://github.com/toolbox-team/reddit-moderator-toolbox/wiki/Subreddit-Wikis%3A-usernotes#working-with-note-types
type: spamwarn
# content is mustache templated
content: >-
Self Promotion: {{rules.attr10all.titlesDelim}}
{{rules.attr10sub.largestPercent}}%

View File

@@ -0,0 +1,36 @@
{
"checks": [
{
"name": "Self Promo Activities",
"description": "Check if any of Author's aggregated submission origins are >10% of entire history",
// check will run on a new submission in your subreddit and look at the Author of that submission
"kind": "submission",
"rules": [
{
"name": "attr10all",
"kind": "attribution",
"criteria": [
{
"threshold": "> 10%",
"window": "90 days"
},
{
"threshold": "> 10%",
"window": 100
}
],
}
],
"actions": [
{
"kind": "usernote",
// the key of usernote type
// https://github.com/toolbox-team/reddit-moderator-toolbox/wiki/Subreddit-Wikis%3A-usernotes#working-with-note-types
"type": "spamwarn",
// content is mustache templated as usual
"content": "Self Promotion: {{rules.attr10all.titlesDelim}} {{rules.attr10sub.largestPercent}}%"
}
]
}
]
}

View File

@@ -0,0 +1,23 @@
checks:
- name: Self Promo Activities
# check will run on a new submission in your subreddit and look at the Author of that submission
description: >-
Check if any of Author's aggregated submission origins are >10% of entire
history
kind: submission
rules:
- name: attr10all
kind: attribution
criteria:
- threshold: '> 10%'
window: 90 days
- threshold: '> 10%'
window: 100
actions:
- kind: usernote
# the key of usernote type
# https://github.com/toolbox-team/reddit-moderator-toolbox/wiki/Subreddit-Wikis%3A-usernotes#working-with-note-types
type: spamwarn
content: >-
Self Promotion: {{rules.attr10all.titlesDelim}}
{{rules.attr10sub.largestPercent}}%

175
docs/gettingStartedMod.md Normal file
View File

@@ -0,0 +1,175 @@
This getting started guide is for **reddit moderators** -- that is, someone who wants **an existing ContextMod bot to run on their subreddit.** If you are trying to run a ContextMod
instance (software) please refer to the [operator getting started](/docs/gettingStartedOperator.md) guide.
# Table of Contents
* [Prior Knowledge](#prior-knowledge)
* [Choose A Bot](#choose-a-bot)
* [Use The Operator's Bot](#use-the-operators-bot)
* [Bring Your Own Bot (BYOB)](#bring-your-own-bot-byob)
* [Creating Configuration](#configuring-the-bot)
* [Monitor the Bot](#monitor-the-bot)
# Prior Knowledge
Before continuing with this guide you should first make sure you understand how a ContextMod works. Please review this documentation:
* [How It Works](/docs/README.md#how-it-works)
* [Core Concepts](/docs/README.md#concepts)
# Choose A Bot
First determine what bot (reddit account) you want to run ContextMod with. (You may have already discussed this with your operator)
## Use the Operator's Bot
If the Operator has communicated that **you should add a bot they control as a moderator** to your subreddit this is the option you will use.
**Pros:**
* Do not have to create and keep track of another reddit account
* Easiest option in terms of setup for both moderators and operator
**Cons:**
* Shared api quota among other moderated subreddits (not great for high-volume subreddits)
___
Ensure that you are in communication with the **operator** of this bot. The bot **will only accept a moderator invitation if your subreddit has been whitelisted by the operator.** This is an intentional barrier to ensure moderators and the operator are familiar with their respective needs and have some form of trust.
Now invite the bot to moderate your subreddit. The bot should have at least these permissions:
* Manage Users
* Manage Posts and Comments
* Manage Flair
* Manage Wiki Pages
* Required to read the moderator-only visible wiki page used to configure the bot
* Required to read/write to [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes)
## Bring Your Own Bot (BYOB)
If the operator has communicated that **they want to use a bot you control** this is the option you will use.
**Pros:**
* **Dedicated API quota**
* This is basically a requirement if your subreddit has high-volume activity and you plan on running checks on comments
* More security guarantees since you control the account
* **Note:** authenticating an account does NOT give the operator access to view or change the email/password for the account
* Established history in your subreddit
**Cons:**
* You must have access to the credentials for the reddit account (bot)
___
The **operator** will send you an **invite link** that you will use to authenticate your bot with the operator's application. Example link: `https://operatorsUrl.com/auth/invite?invite=4kf9n3o03ncd4nd`
Review the information shown on the invite link webpage and then follow the directions shown to authorize your bot for the operator.
**Note:** There is information display **after** authentication that you will need to communicate to your operator -- **Refresh** and **Access** token values. Make sure you save these somewhere as the invite link is **one-use only.**
# Configuring the Bot
The bot's behavior is defined using a configuration, like automoderator, that is stored in the **wiki** of each subreddit it moderates.
The default location for this page is at `https://old.reddit.com/r/YOURSUBERDDIT/wiki/botconfig/contextbot`
## Setup wiki page
The bot automatically tries to create its configuration wiki page. You can find the result of this in the log for your subreddit in the web interface.
If this fails for some reason you can create the wiki page through the web interface by navigating to your subreddit's tab, opening the [built-in editor (click **View**)](/docs/screenshots/configBox.png), and following the directions in **Create configuration for...** link found there.
If neither of the above approaches work, or you do not wish to use the web interface, expand the section below for directions on how to manually setup the wiki page:
<details>
* Visit the wiki page of the subreddit you want the bot to moderate
* The default location the bot checks for a configuration is at `https://old.reddit.com/r/YOURSUBERDDIT/wiki/botconfig/contextbot`
* If the page does not exist create it
* Ensure the wiki page visibility is restricted
* On the wiki page click **settings** (**Page settings** in new reddit)
* Check the box for **Only mods may edit and view** and then **save**
</details>
## Procure a configuration
Now you need to make the actual configuration that will be used to configure the bot's behavior on your subreddit. This may have already been done for you by your operator or you may be copying a fellow moderator's configuration.
If you already have a configuration you may skip the below step and go directly to [saving your configuration](#saving-your-configuration)
### Using an Example Config
Visit the [Examples](https://github.com/FoxxMD/context-mod/tree/master/docs/examples) folder to find various examples of individual rules or see the [subreddit-ready examples.](/docs/examples/subredditReady)
After you have found a configuration to use as a starting point:
* Copy the URL for the configuration file EX `https://github.com/FoxxMD/context-mod/blob/master/docs/examples/subredditReady/freekarma.json5` and either:
* (Easiest) **Load** it into your [subreddit's built-in editor](#using-the-built-in-editor) and **Save**
* or on the file's page, click the **Raw** button, select all and copy to your clipboard, and [manually save to your wiki page](#manually-saving)
### Build Your Own Config
CM comes equipped with a [configuration explorer](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) to help you see all available options, with descriptions and examples, that can be used in your configuration.
To create or edit a configuration you should use **CM's buit-in editor** which features:
* syntax validation and formatting
* full configuration validation with error highlighting, hints, and fixes
* hover over properties to see documentation and examples
To use the editor either:
* [use your subreddit's built-in editor](#using-the-built-in-editor)
* or use the public editor at https://cm.foxxmd.dev/config
PROTIP: Find an [example config](#using-an-example-config) to use as a starting point and then build on it using the editor.
## Saving Your Configuration
### Using the built-in Editor
In the web interface each subreddit's tab has access to the built-in editor. Use this built-in editor to automatically create, load, or save the configuration for that subreddit's wiki.
* Visit the tab for the subreddit you want to edit the configuration of
* Open the [built-in editor by click **View**](/docs/screenshots/configBox.png)
* Edit your configuration
* Follow the directions on the **Save to r/..** link found at the top of the editor to automatically save your configuration
### Manually Saving
<details>
* Open the wiki page you created in the [wiki setup step](#setup-wiki-page) and click **edit**
* Copy-paste your configuration into the wiki text box
* Save the edited wiki page
</details>
___
The bot automatically checks for new configurations on your wiki page every 5 minutes. If your operator has the web interface accessible you may login there and force the config to update on your subreddit.
# Monitor the Bot
Monitoring the behavior of the bot is dependent on how your operator setup their instance. ContextMod comes with a built-in web interface that is secure and accessible only to moderates of subreddits it is running on. However there is some additional setup for the operator to perform in order to make this interface accessible publicly. If you do not have access to this interface please communicate with your operator.
After logging in to the interface you will find your subreddit in a tab at the top of the web page. Selecting your subreddit will give you access to:
* Current status of the bot
* Current status of your configuration
* Statistics pertaining to the number of checks/rules/actions run and cache usage
* **A real-time view for bot logs pertaining to your subreddit**
The logs are the meat and potatoes of the bot and the main source of feedback you have for fine-tuning the bot's behavior. The **verbose** log level will show you:
* The event being processed
* The individual results of triggered rules, per check
* The checks that were run and their rules
* The actions performed, with markdown content preview, of triggered checks
This information should enable you to tweak the criteria for your rules in order to get the required behavior from the bot.
Additionally, you can test your bot on any comment/submission by entering its permalink in the text bot at the top of the logs and selecting **Dry Run** -- this will run the bot on an Activity without actually performing any actions allowing you to preview the results of a run.

View File

@@ -0,0 +1,83 @@
This getting started guide is for **Operators** -- that is, someone who wants to run the actual software for a ContentMod bot. If you are a **Moderator** check out the [moderator getting started](/docs/gettingStartedMod.md) guide instead.
# Table of Contents
* [Installation](#installation)
* [Docker](#docker-recommended)
* [Locally](#locally)
* [Heroku](#heroku-quick-deployhttpsherokucomabout)
* [Bot Authentication](#bot-authentication)
* [Instance Configuration](#instance-configuration)
* [Run Your Bot and Start Moderating](#run-your-bot-and-start-moderating)
# Installation
In order to run a ContextMod instance you must first you must install it somewhere.
ContextMod can be run on almost any operating system but it is recommended to use Docker due to ease of deployment.
## Docker (Recommended)
PROTIP: Using a container management tool like [Portainer.io CE](https://www.portainer.io/products/community-edition) will help with setup/configuration tremendously.
### [Dockerhub](https://hub.docker.com/r/foxxmd/context-mod)
```
foxxmd/context-mod:latest
```
Adding **environmental variables** to your `docker run` command will pass them through to the app EX:
```
docker run -d -e "CLIENT_ID=myId" ... foxxmd/context-mod
```
### Locally
Requirements:
* Typescript >=4.3.5
* Node >=15
Clone this repository somewhere and then install from the working directory
```bash
git clone https://github.com/FoxxMD/context-mod.git .
cd context-mod
npm install
tsc -p .
```
### [Heroku Quick Deploy](https://heroku.com/about)
[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://dashboard.heroku.com/new?template=https://github.com/FoxxMD/context-mod)
This template provides a **web** and **worker** dyno for heroku.
* **Web** -- Will run the bot **and** the web interface for ContextMod.
* **Worker** -- Will run **just** the bot.
Be aware that Heroku's [free dyno plan](https://devcenter.heroku.com/articles/free-dyno-hours#dyno-sleeping) enacts some limits:
* A **Web** dyno will go to sleep (pause) after 30 minutes without web activity -- so your bot will ALSO go to sleep at this time
* The **Worker** dyno **will not** go to sleep but you will NOT be able to access the web interface. You can, however, still see how Cm is running by reading the logs for the dyno.
If you want to use a free dyno it is recommended you perform first-time setup (bot authentication and configuration, testing, etc...) with the **Web** dyno, then SWITCH to a **Worker** dyno so it can run 24/7.
# Bot Authentication
Next you need to create a bot and authenticate it with Reddit. Follow the [bot authentication guide](/docs/botAuthentication.md) to complete this step.
# Instance Configuration
Finally, you must provide the credentials you received from the **Bot Authentication** step to the ContextMod instance you installed earlier. Refer to the [Operator Configuration](/docs/operatorConfiguration.md) guide to learn how this can be done as there are multiple approaches depending on how you installed the software.
Additionally, at this step you can also tweak many more settings and behavior concerning how your CM bot will operate.
# Run Your Bot and Start Moderating
Congratulations! You should now have a fully authenticated bot running on ContextMod software.
In order for your Bot to operate on reddit though it **must be a moderator in the subreddit you want it to run in.** This may be your own subreddit or someone else's.
**Note: ContextMod does not currently handle moderation invites automatically** and may never have this functionality. Due to the fact that many of its behaviors are api-heavy and that subreddits can control their own configuration the api and resource (cpu/memory) usage of a ContextMod instance can be highly variable. It therefore does not make sense to allow any/all subreddits to automatically have access to an instance through automatically accepting moderator invites. So...if you are planning to run a ContextMod instance for subreddits other than those you moderate you should establish solid trust with moderators of that subreddit as well as a solid line of communication in order to ensure their configurations can be tailored to best fit their needs and your resources.
Once you have logged in as your bot and manually accepted the moderator invite you will need to restart your ContextMod instance in order for these changes to take effect.

237
docs/imageComparison.md Normal file
View File

@@ -0,0 +1,237 @@
# Overview
ContextMod supports comparing image content, for the purpose of detecting duplicates, with two different but complimentary systems. Image comparison behavior is available for the following rules:
* [Recent Activity](/docs/examples/recentActivity)
* Repeat Activity (In-progress)
To enable comparisons reference the example below (at the top-level of your rule) and configure as needed:
JSON
```json5
{
"name": "ruleWithImageDetection",
"kind": "recentActivity",
// Add block below...
//
"imageDetection": {
// enables image comparison
"enable": true,
// The difference, in percentage, between the reference submission and the submissions being checked
// must be less than this number to consider the images "the same"
"threshold": 5,
// optional
// set the behavior for determining if image comparison should occur on a URL:
//
// "extension" => try image detection if URL ends in a known image extension (jpeg, gif, png, bmp, etc.)
// "unknown" => try image detection if URL ends in known image extension OR there is no extension OR the extension is unknown (not video, html, doc, etc...)
// "all" => ALWAYS try image detection, regardless of URL extension
//
// if fetchBehavior is not defined then "extension" is the default
"fetchBehavior": "extension",
},
//
// And above ^^^
//...
}
```
YAML
```yaml
name: ruleWithImageDetection
kind: recentActivity
enable: true
threshold: 5
fetchBehavior: extension
```
**Perceptual Hashing** (`hash`) and **Pixel Comparisons** (`pixel`) may be used at the same time. Refer to the documentation below to see how they interact.
**Note:** Regardless of `fetchBehavior`, if the response from the URL does not indicate it is an image then image detection will not occur. IE Response `Content-Type` must contain `image`
## Prerequisites
Both image comparison systems require [Sharp](https://sharp.pixelplumbing.com/) as a dependency. Most modern operating systems running Node.js >= 12.13.0 do not require installing additional dependencies in order to use Sharp.
If you are using the docker image for ContextMod (`foxxmd/context-mod`) Sharp is built-in.
If you are installing ContextMod using npm then **Sharp should be installed automatically as an optional dependency.**
**If you do not want to install it automatically** install ContextMod with the following command:
```
npm install --no-optional
```
If you are using ContextMod as part of a larger project you may want to require Sharp in your own package:
```
npm install sharp@0.29.1 --save
```
# Comparison Systems
## Perceptual Hashing
[Perceptual Hashing](https://en.wikipedia.org/wiki/Perceptual_hashing) creates a text fingerprint of an image by:
* Dividing up the image into a grid
* Using an algorithm to derive a value from the pixels in each grid
* Adding up all the values to create a unique string (the "fingerprint")
An example of how a perceptual hash can work [can be found here.](https://www.hackerfactor.com/blog/?/archives/432-Looks-Like-It.html)
ContextMod uses [blockhash-js](https://github.com/commonsmachinery/blockhash-js) which is a javascript implementation of the algorithm described in the paper [Block Mean Value Based Image Perceptual Hashing by Bian Yang, Fan Gu and Xiamu Niu.](https://ieeexplore.ieee.org/document/4041692)
**Advantages**
* Low memory requirements and not CPU intensive
* Does not require any image transformations
* Hash results can be stored to make future comparisons even faster and skip downloading images (cached by url)
* Resolution-independent
**Disadvantages**
* Hash is weak when image differences are based only on color
* Hash is weak when image contains lots of text
* Higher accuracy requires larger calculation (more bits required)
**When should I use it?**
* General duplicate detection
* Comparing many images
* Comparing the same images often
### How To Use
If `imageDetection.enable` is `true` then hashing is enabled by default and no further configuration is required.
To further configure hashing refer to this code block:
```json5
{
"name": "ruleWithImageDetectionAndConfiguredHashing",
"kind": "recentActivity",
"imageDetection": {
"enable": true,
// Add block below...
//
"hash": {
// enable or disable hash comparisons (enabled by default)
"enable": true,
// determines accuracy of hash and granularity of hash comparison (comparison to other hashes)
// the higher the bits the more accurate the comparison
//
// NOTE: Hashes of different sizes (bits) cannot be compared. If you are caching hashes make sure all rules where results may be shared use the same bit count to ensure hashes can be compared. Otherwise hashes will be recomputed.
"bits": 32,
// default is 32 if not defined
//
// number of seconds to cache an image hash
"ttl": 60,
// default is 60 if not defined
//
// "High Confidence" Threshold
// If the difference in comparison is equal to or less than this number the images are considered the same and pixel comparison WILL NOT occur
//
// Defaults to the parent-level `threshold` value if not present
//
// Use null if you want pixel comparison to ALWAYS occur (softThreshold must be present)
"hardThreshold": 5,
//
// "Low Confidence" Threshold -- only used if `pixel` is enabled
// If the difference in comparison is:
//
// 1) equal to or less than this value and
// 2) the value is greater than `hardThreshold`
//
// the images will be compared using the `pixel` method
"softThreshold": 0,
},
//
// And above ^^^
//"pixel": {...}
}
//...
}
```
YAML
```yaml
name: ruleWithImageDetectionAndConfiguredHashing
kind: recentActivity
imageDetection:
enable: true
hash:
enable: true
bits: 32
ttl: 60
hardThreshold: 5
softThreshold: 0
```
## Pixel Comparison
This approach is as straight forward as it sounds. Both images are compared, pixel by pixel, to determine the difference between the two. ContextMod uses [pixelmatch](https://github.com/mapbox/pixelmatch) to do the comparison.
**Advantages**
* Extremely accurate, high-confidence on difference percentage
* Strong when comparing text-based images or color-only differences
**Disadvantages**
* High memory requirements (10-30MB per comparison) and CPU intensive
* Weak against similar images with different aspect ratios
* Requires image transformations (resize, crop) before comparison
* Can only store image-to-image results (no single image fingerprints)
**When should I use it?**
* Require very high accuracy in comparison results
* Comparing mostly text-based images or subtle color/detail differences
* As a secondary, high-confidence confirmation of comparison result after hashing
### How To Use
By default pixel comparisons **are not enabled.** They must be explicitly enabled in configuration.
Pixel comparisons will be performed in either of these scenarios:
* pixel is enabled, hashing is enabled and `hash.softThreshold` is defined
* When a comparison occurs that is less different than `softThreshold` but more different then `hardThreshold` (or `"hardThreshold": null`), then pixel comparison will occur as a high-confidence check
* Example
* hash comparison => 7% difference
* `"softThreshold": 10`
* `"hardThreshold": 4`
* `hash.enable` is `false` and `pixel.enable` is true
* hashing is skipped entirely and only pixel comparisons are performed
To configure pixel comparisons refer to this code block:
```json5
{
"name": "ruleWithImageDetectionAndPixelEnabled",
"kind": "recentActivity",
"imageDetection": {
//"hash": {...}
"pixel": {
// enable or disable pixel comparisons (disabled by default)
"enable": true,
// if the comparison difference percentage is equal to or less than this value the images are considered the same
//
// if not defined the value from imageDetection.threshold will be used
"threshold": 5
}
},
//...
}
```
YAML
```yaml
name: ruleWithImageDetectionAndPixelEnabled
kind: recentActivity
imageDetection:
pixel:
enable: true
threshold: 5
```

BIN
docs/logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

View File

@@ -0,0 +1,341 @@
The **Operator** configuration refers to configuration used configure to the actual application/bot. This is different
from the **Subreddit** configuration that is defined in each Subreddit's wiki and determines the rules/actions for
activities the Bot runs on.
# Table of Contents
* [Minimum Required Configuration](#minimum-required-configuration)
* [Defining Configuration](#defining-configuration)
* [CLI Usage](#cli-usage)
* [Examples](#example-configurations)
* [Minimum Config](#minimum-config)
* [Using Config Overrides](#using-config-overrides)
* [Cache Configuration](#cache-configuration)
# Minimum Required Configuration
| property | Server And Web | Server Only | Web/Bot-Auth Only |
|:--------------:|:------------------:|:------------------:|:------------------:|
| `clientId` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| `clientSecret` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| `redirectUri` | :heavy_check_mark: | :x: | :heavy_check_mark: |
| `refreshToken` | :heavy_check_mark: | :heavy_check_mark: | :x: |
| `accessToken` | :heavy_check_mark: | :heavy_check_mark: | :x: |
Refer to the **[Bot Authentication guide](/docs/botAuthentication.md)** to retrieve credentials.
# Defining Configuration
CM can be configured using **any or all** of the approaches below. Note that **at each level ALL configuration values are
optional** but the "required configuration" mentioned above must be available when all levels are combined.
Any values defined at a **lower-listed** level of configuration will override any values from a higher-listed
configuration.
* **ENV** -- Environment variables loaded from an [`.env`](https://github.com/toddbluhm/env-cmd) file (path may be
specified with `--file` cli argument)
* **ENV** -- Any already existing environment variables (exported on command line/terminal profile/etc.)
* **FILE** -- Values specified in a JSON configuration file using the structure [in the schema](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
* **ARG** -- Values specified as CLI arguments to the program (see [ClI Usage](#cli-usage) below)
**Note:** When reading the **schema** if the variable is available at a level of configuration other than **FILE** it will be
noted with the same symbol as above. The value shown is the default.
* To load a JSON configuration (for **FILE**) **from the command line** use the `-c` cli argument EX: `node src/index.js -c /path/to/JSON/config.json`
* To load a JSON configuration (for **FILE**) **using an environmental variable** use `OPERATOR_CONFIG` EX: `OPERATOR_CONFIG=/path/to/JSON/config.json`
[**See the Operator Config Schema here**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
## Defining Multiple Bots or CM Instances
One ContextMod instance can
* Run multiple bots (multiple reddit accounts -- each as a bot)
* Connect to many other, independent, ContextMod instances
However, the default configuration (using **ENV/ARG**) assumes your intention is to run one bot (one reddit account) on one CM instance without these additional features. This is to make this mode of operation easier for users with this intention.
To take advantage of this additional features you **must** use a **FILE** configuration. Learn about how this works and how to configure this scenario in the [Architecture Documentation.](/docs/serverClientArchitecture.md)
## CLI Usage
Running CM from the command line is accomplished with the following command:
```bash
node src/index.js run
```
Run `node src/index.js run help` to get a list of available command line options (denoted by **ARG** above):
<details>
```
Usage: index [options] [command]
Options:
-h, --help display help for command
Commands:
run [options] [interface] Monitor new activities from configured subreddits.
check [options] <activityIdentifier> [type] Run check(s) on a specific activity
unmoderated [options] <subreddits...> Run checks on all unmoderated activity in the modqueue
help [command] display help for command
Options:
-c, --operatorConfig <path> An absolute path to a JSON file to load all parameters from (default: process.env.OPERATOR_CONFIG)
-i, --clientId <id> Client ID for your Reddit application (default: process.env.CLIENT_ID)
-e, --clientSecret <secret> Client Secret for your Reddit application (default: process.env.CLIENT_SECRET)
-a, --accessToken <token> Access token retrieved from authenticating an account with your Reddit Application (default: process.env.ACCESS_TOKEN)
-r, --refreshToken <token> Refresh token retrieved from authenticating an account with your Reddit Application (default: process.env.REFRESH_TOKEN)
-u, --redirectUri <uri> Redirect URI for your Reddit application (default: process.env.REDIRECT_URI)
-t, --sessionSecret <secret> Secret use to encrypt session id/data (default: process.env.SESSION_SECRET || a random string)
-s, --subreddits <list...> List of subreddits to run on. Bot will run on all subs it has access to if not defined (default: process.env.SUBREDDITS)
-d, --logDir [dir] Absolute path to directory to store rotated logs in. Leaving undefined disables rotating logs (default: process.env.LOG_DIR)
-l, --logLevel <level> Minimum level to log at (default: process.env.LOG_LEVEL || verbose)
-w, --wikiConfig <path> Relative url to contextbot wiki page EX https://reddit.com/r/subreddit/wiki/<path> (default: process.env.WIKI_CONFIG || 'botconfig/contextbot')
--snooDebug Set Snoowrap to debug. If undefined will be on if logLevel='debug' (default: process.env.SNOO_DEBUG)
--authorTTL <s> Set the TTL (seconds) for the Author Activities shared cache (default: process.env.AUTHOR_TTL || 60)
--heartbeat <s> Interval, in seconds, between heartbeat checks. (default: process.env.HEARTBEAT || 300)
--softLimit <limit> When API limit remaining (600/10min) is lower than this subreddits will have SLOW MODE enabled (default: process.env.SOFT_LIMIT || 250)
--hardLimit <limit> When API limit remaining (600/10min) is lower than this all subreddit polling will be paused until api limit reset (default: process.env.SOFT_LIMIT || 250)
--dryRun Set all subreddits in dry run mode, overriding configurations (default: process.env.DRYRUN || false)
--proxy <proxyEndpoint> Proxy Snoowrap requests through this endpoint (default: process.env.PROXY)
--operator <name...> Username(s) of the reddit user(s) operating this application, used for displaying OP level info/actions in UI (default: process.env.OPERATOR)
--operatorDisplay <name> An optional name to display who is operating this application in the UI (default: process.env.OPERATOR_DISPLAY || Anonymous)
-p, --port <port> Port for web server to listen on (default: process.env.PORT || 8085)
-q, --shareMod If enabled then all subreddits using the default settings to poll "unmoderated" or "modqueue" will retrieve results from a shared request to /r/mod (default: process.env.SHARE_MOD || false)
-h, --help display help for command
```
</details>
# Example Configurations
## Minimum Config
Below are examples of the minimum required config to run the application using all three config approaches independently.
Using **FILE**
<details>
YAML
```yaml
bots:
- credentials:
clientId: f4b4df1c7b2
clientSecret: 34v5q1c56ub
refreshToken: 34_f1w1v4
accessToken: p75_1c467b2
```
JSON
```json5
{
"bots": [
{
"credentials": {
"clientId": "f4b4df1c7b2",
"clientSecret": "34v5q1c56ub",
"refreshToken": "34_f1w1v4",
"accessToken": "p75_1c467b2"
}
}
]
}
```
</details>
Using **ENV** (`.env`)
<details>
```
CLIENT_ID=f4b4df1c7b2
CLIENT_SECRET=34v5q1c56ub
REFRESH_TOKEN=34_f1w1v4
ACCESS_TOKEN=p75_1c467b2
```
</details>
Using **ARG**
<details>
```
node src/index.js run --clientId=f4b4df1c7b2 --clientSecret=34v5q1c56ub --refreshToken=34_f1w1v4 --accessToken=p75_1c467b2
```
</details>
## Using Config Overrides
An example of using multiple configuration levels together IE all are provided to the application:
**FILE**
<details>
```json
{
"logging": {
"level": "debug"
}
}
```
YAML
```yaml
logging:
level: debug
```
</details>
**ENV** (`.env`)
<details>
```
CLIENT_SECRET=34v5q1c56ub
REFRESH_TOKEN=34_f1w1v4
ACCESS_TOKEN=p75_1c467b2
SUBREDDITS=sub1,sub2,sub3
PORT=9008
```
</details>
**ARG**
<details>
```
node src/index.js run --subreddits=sub1 --clientId=34v5q1c56ub
```
</details>
When all three are used together they produce these variables at runtime for the application:
```
clientId: f4b4df1c7b2
clientSecret: 34v5q1c56ub
refreshToken: 34_f1w1v4
accessToken: accessToken
subreddits: sub1
port: 9008
log level: debug
```
## Configuring Client for Many Instances
See the [Architecture Docs](/docs/serverClientArchitecture.md) for more information.
<details>
YAML
```yaml
bots:
- credentials:
clientId: f4b4df1c7b2
clientSecret: 34v5q1c56ub
refreshToken: 34_f1w1v4
accessToken: p75_1c467b2
web:
credentials:
clientId: f4b4df1c7b2
clientSecret: 34v5q1c56ub
redirectUri: 'http://localhost:8085/callback'
clients:
# server application running on this same CM instance
- host: 'localhost:8095'
secret: localSecret
# a server application running somewhere else
- host: 'mySecondContextMod.com:8095'
secret: anotherSecret
api:
secret: localSecret
```
JSON
```json5
{
"bots": [
{
"credentials": {
"clientId": "f4b4df1c7b2",
"clientSecret": "34v5q1c56ub",
"refreshToken": "34_f1w1v4",
"accessToken": "p75_1c467b2"
}
}
],
"web": {
"credentials": {
"clientId": "f4b4df1c7b2",
"clientSecret": "34v5q1c56ub",
"redirectUri": "http://localhost:8085/callback"
},
"clients": [
// server application running on this same CM instance
{
"host": "localhost:8095",
"secret": "localSecret"
},
// a server application running somewhere else
{
// api endpoint and port
"host": "mySecondContextMod.com:8095",
"secret": "anotherSecret"
}
]
},
"api": {
"secret": "localSecret",
}
}
```
</details>
# Cache Configuration
CM implements two caching backend **providers**. By default all providers use `memory`:
* `memory` -- in-memory (non-persistent) backend
* `redis` -- [Redis](https://redis.io/) backend
Each `provider` object in configuration can be specified as:
* one of the above **strings** to use the **defaults settings** or
* an **object** with keys to override default settings
A caching object in the json configuration:
```json5
{
"provider": {
"store": "memory", // one of "memory" or "redis"
"ttl": 60, // the default max age of a key in seconds
"max": 500, // the maximum number of keys in the cache (for "memory" only)
// the below properties only apply to 'redis' provider
"host": 'localhost',
"port": 6379,
"auth_pass": null,
"db": 0,
}
}
```
YAML
```yaml
provider:
store: memory
ttl: 60
max: 500
host: localhost
port: 6379
auth_pass: null
db: 0
```

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

BIN
docs/screenshots/editor.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 125 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

BIN
docs/screenshots/oauth.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 226 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 479 KiB

View File

@@ -0,0 +1,71 @@
# Overview
ContextMod's high-level functionality is separated into two **independently run** applications.
Each application consists of an [Express](https://expressjs.com/) web server that executes the core logic for that application and communicates via HTTP API calls:
Applications:
* **Server** -- Responsible for **running the bots** and providing an API to retrieve information on and interact with them EX start/stop bot, reload config, retrieve operational status, etc.
* **Client** -- Responsible for serving the **web interface** and handling the bot oauth authentication flow between operators and moderators.
Both applications operate independently and can be run individually. The determination for which is run is made by environmental variables, operator config, or cli arguments.
# Authentication
Communication between the applications is secured using [Json Web Tokens](https://github.com/mikenicholson/passport-jwt) signed/encoded by a **shared secret** (HMAC algorithm). The secret is defined in the operator configuration.
# Configuration
## Default Mode
**ContextMod is designed to operate in a "monolith" mode by default.**
This is done by assuming that when configuration is provided by **environmental variables or CLI arguments** the user's intention is to run the client/server together with only one bot, as if ContextMod is a monolith application. When using these configuration types the same values are parsed to both the server/client to ensure interoperability/transparent usage for the operator. Some examples of this in the **operator configuration**:
* The **shared secret** for both client/secret cannot be defined using env/cli -- at runtime a random string is generated that is set for the value `secret` on both the `api` and `web` properties.
* The `bots` array cannot be defined using env/cli -- a single entry is generated by the configuration parser using the combined values provided from env/cli
* The `PORT` env/cli argument only applies to the `client` wev server to guarantee the default port for the `server` web server is used (so the `client` can connect to `server`)
**The end result of this default behavior is that an operator who does not care about running multiple CM instances does not need to know or understand anything about the client/server architecture.**
## Server
To run a ContextMod instance as **sever only (headless):**
* Config file -- define top-level `"mode":"server"`
* ENV -- `MODE=server`
* CLI - `node src/index.js run server`
The relevant sections of the **operator configuration** for the **Server** are:
* [`operator.name`](https://json-schema.app/view/%23/%23%2Fproperties%2Foperator?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json) -- Define the reddit users who will be able to have full access to this server regardless of moderator status
* `api`
### [`api`](https://json-schema.app/view/%23/%23%2Fproperties%2Fapi?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
* `port` - The port the Server will listen on for incoming api requests. Cannot be the same as the Client (when running on the same host)
* `secret` - The **shared secret** that will be used to verify incoming api requests coming from an authenticated Client.
* `friendly` - An optional string to identify this **Server** on the client. It is recommended to provide this otherwise it will default to `host:port`
## Client
To run a ContextMod instance as **client only:**
* Config file -- define top-level `"mode":"client"`
* ENV -- `MODE=client`
* CLI - `node src/index.js run client`
### [`web`](https://json-schema.app/view/%23/%23%2Fproperties%2Fweb?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
In the **operator configuration** the top-level `web` property defines the configuration for the **Client** application.
* `web.credentials` -- Defines the reddit oauth credentials used to authenticate users for the web interface
* Must contain a `redirectUri` property to work
* Credentials are parsed from ENV/CLI credentials when not specified (IE will be same as default bot)
* `web.operators` -- Parsed from `operator.name` if not specified IE will use same users as defined for the bot operators
* `port` -- the port the web interface will be served from, defaults to `8085`
* `clients` -- An array of `BotConnection` objects that specify what **Server** instances the web interface should connect to. Each object should have:
* `host` -- The URL specifying where the server api is listening ie `localhost:8085`
* `secret` -- The **shared secret** used to sign api calls. **This should be the same as `api.secret` on the server being connected to.**

29
heroku.Dockerfile Normal file
View File

@@ -0,0 +1,29 @@
FROM node:16-alpine3.14
ENV TZ=Etc/GMT
# vips required to run sharp library for image comparison
RUN echo "http://dl-4.alpinelinux.org/alpine/v3.14/community" >> /etc/apk/repositories \
&& apk --update add vips
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
WORKDIR /usr/app
COPY package*.json ./
COPY tsconfig.json .
RUN npm install
ADD . /usr/app
RUN npm run build
ENV NPM_CONFIG_LOGLEVEL debug
ARG log_dir=/home/node/logs
RUN mkdir -p $log_dir
VOLUME $log_dir
ENV LOG_DIR=$log_dir
CMD [ "node", "src/index.js", "run", "all", "--port $PORT"]

4
heroku.yml Normal file
View File

@@ -0,0 +1,4 @@
build:
docker:
web: heroku.Dockerfile
worker: heroku.Dockerfile

Some files were not shown because too many files have changed in this diff Show More