103 Commits

Author SHA1 Message Date
FoxxMD
7e2068d82a fix(author): Ensure automoderator is always detected as a moderator for author isMod test 2022-02-14 09:30:54 -05:00
FoxxMD
41038b9bcd feat(logging): Implement richer errors everywhere
* Use ErrorWithCause so we can get and print a chain of error causes
* Make reddit error response in stack trace more readable by replacing them with a "translated" parent response and add them as the cause
* Properly handle error formatting for winston by looking at shape of log object for error rather than testing instanceof (see comments in errorAwareFormat)
* Fix formatting in web interface for log lines with white-space pre css and properly splitting timestamp from rest of the message
2022-01-27 16:27:03 -05:00
FoxxMD
9fe8c9568c refactor: Move SimpleError into main Errors module 2022-01-27 11:48:23 -05:00
FoxxMD
9614f7a209 refactor(logging): Implement snoowrap errors "the right way" and implement consolidated logging function
* Implement declaration file for snoowrap errors so they can be imported directly
* Implement logging function to handle boilerplate for known error responses (reddit HTTP response, rate limit, etc.)
2022-01-27 11:43:39 -05:00
FoxxMD
8dbaaf6798 fix(logging): Defaults for log file dir 2022-01-26 12:28:56 -05:00
FoxxMD
c14ad6cb76 feat(logging): Implement separate logging options for each transport type
* Add properties for file, console, and stream in logging object of operator config
* Each property inherits a (useful) subset of winston transport options
2022-01-26 12:09:03 -05:00
FoxxMD
adda280dd3 fix(logging): Fix parsing log dir
* Correct else condition to use log dir when value is not true
* Set level to 'debug' on init logger if no value is provided to help with debugging
2022-01-26 10:27:01 -05:00
FoxxMD
780e5c185e refactor(author filter): Strongly structure comparison/matching data for more consistent manipulation and output
* Use interface for comparison results at both criteria property level and criteria level
* Implement summary functions to build string results of comparisons
* Output all comparisons to debug and provide summaries to verbose (when applicable)
2022-01-20 14:08:54 -05:00
FoxxMD
6977e3bcdf feat(author): Add flair template criteria for author/submission
* Add filtering by flairTemplate id for author/submission
* Refactor flair properties for author/submission to accept string or array of strings
2022-01-19 12:48:58 -05:00
FoxxMD
5811af0342 feat(config): Refactor config parsing to preserve comments and enable writing
* use node-comment and yaml@next to keep comment information intact
* store ast/source version of parsed config for operator
* implement generic yaml/json operator config classes to keep everything organized and simplify marshalling source to js/string
* refactor file parsing and json/yaml parsing to have better single responsibility
2022-01-17 15:51:43 -05:00
FoxxMD
5e70ca1cb6 fix: Fix and improve code related to stopping bots when reddit api is not OK
* Fix polling timeout to actually stop on error by simplifying timeout and waiting until response is OK to recreate next timeout call
* Use "unexpected exception" retry count for all non well-known "reddit blip" responses in retry handler rather than failing immediately AND log this distinction
* Fix managers not emitting errors from checks
* Fix bot not awaiting retry handler on manager error emit
* Increase nanny loop delay on error to reduce api pressure when there are many bots running
* (unrelated) set bot as running before starting managers so UI is available earlier
2022-01-05 12:58:17 -05:00
Marcin Macinski
bbee92699c Support setting user flair by flair_template_id 2021-12-21 00:10:40 +01:00
FoxxMD
f081598da6 feat(repost): Improve text matching and default values
* Use 3 different matching algorithms using the highest score out of the three
* Weight score based on length of the sentence
* Increase minimum number of words to 3
* Enforce min word count on external (youtube) comments
2021-12-18 17:42:43 -05:00
FoxxMD
dcbeb784e8 refactor: Remove set-random-interval
* depended on and always downloaded an entire, older typescript version (even with production install) -- was not necessary for one function
* refactor project to use newer TS version (specify any type for catch blocks to fix compiler errors)
2021-12-17 14:25:48 -05:00
FoxxMD
b2846efd2b fix: Improve falloff behavior when reddit api error are encountered
* Turn off snoowrap request queuing
* Aggregate errors from all managers at bot-level and force all to stop (and clear queue/polling) if a small threshold is met
* Add activity refresh on check into try-catch so delayed activities in queue don't cause a loop if they fail due to api issues
2021-12-06 17:16:04 -05:00
FoxxMD
a787e4515b feat(repost): Implement occurrence count and time comparisons
* User-defined set of comparisons for testing how many reposts were found
* User-defined set of comparisons for testing when a slice/all of reposts were created
2021-12-06 13:28:02 -05:00
FoxxMD
9d0e098db1 feat(rule): Implement repost rule
* checks for both submission/comment
* search by submission title, url, dups/crossposts, and external (just youtube for now)
* user-defined string sameness for all search facets
* user-defined case-sensitivity and regex-based transformations for activity/repost item values
* cache comment checks
* implemented youtube client for retrieving video comments
2021-12-01 17:34:25 -05:00
FoxxMD
a8c7b1dac9 Interim implementation of repost rule 2021-11-30 20:42:40 -05:00
FoxxMD
d5e1cdec61 fix(criteria): Improve criteria filtering for removed/deleted activities and logging
* Use different logging messages when criteria is not available due to mod permissions (property not available to non-mods)
* Change logging level for missing/unavailable criteria to reduce logging noise. On unavailable use debug, on missing use warn
* Improve activity removed/deleted detection based on whether activity is moddable by current user
2021-11-01 13:25:54 -04:00
FoxxMD
d8180299ea fix(author): Fix missing true return statement for author flair check 2021-11-01 10:23:22 -04:00
FoxxMD
56c007c20d feat(author): Implement author profile description regex/string testing
May test "description" in authorIs as a regular express of string literal -- or as an array of the aforementioned values
2021-10-20 19:52:46 -04:00
FoxxMD
65d1d36d53 feat(cache): Implement batching for subreddit info
When using full criteria for subreddit state we can save a ton of api calls by get info for all uncached subreddits at the same time rather than individually
2021-09-29 16:20:02 -04:00
FoxxMD
c5358f196d feat(author): Handle shadowbanned users
* Allow checking if user is shadowbanned via authorIs (AuthorCriteria)
* try-catch on history get or author criteria to try to detect shadowbanned user for a more descriptive error
2021-09-20 13:49:35 -04:00
FoxxMD
40111c54a2 feat(message): Add a markdown formatted 'ruleSummary' property to content template data 2021-09-17 14:38:39 -04:00
FoxxMD
20c724cab5 fix: Fix bug where non-media domains were not counted for attribution rule 2021-09-16 15:33:59 -04:00
FoxxMD
fc4eeb47fa Replace newlines with spaces to make comment peeks more compact 2021-08-23 16:49:40 -04:00
FoxxMD
8a9212def2 Use winston-duplex transport package
Dogfooding
2021-08-19 14:06:16 -04:00
FoxxMD
980ff7da02 Fix duplex transport duplicated data listeners 2021-08-18 16:41:10 -04:00
FoxxMD
0f84a7cf6b Lots of log (streaming) improvements
* Use autolinker to deal with url parsing in logs and sanitize html
* Fix missing mergeArr arguments on child loggers
* Implement DuplexTransport to make streaming from winston less verbose and allow access to log object
* Refactor log parsing on server/client side to deal with log objects or strings
2021-08-18 15:47:31 -04:00
FoxxMD
c817716aa1 First pass at refactoring to use new instance/bot/subreddit architecture 2021-08-16 14:14:42 -04:00
FoxxMD
8d8e4405e0 Implement graceful bot teardown and api endpoint to rebuild bot 2021-08-13 15:52:18 -04:00
FoxxMD
5c76f9ab1c Continue to simplify stream handling
* Cleanup stream usage for winston and simplify adding to logger
* Use pipeline with delimiter stream for parsing logs from client
* End log streaming with promise abort (cleaner)
2021-08-10 23:10:06 -04:00
FoxxMD
e762cc29ef Big ol mess for now 2021-08-06 22:04:29 -04:00
FoxxMD
868bac9f1a Fix dry run parsing for arguments 2021-08-04 12:44:51 -04:00
FoxxMD
af1ea5543e Implement caching for more components
* Implement caching for specific activities
* Implement/refactor item is criteria to cache activities and results
2021-08-02 15:10:49 -04:00
FoxxMD
fd7a6edeb6 Fix activity window criteria checks for 'any' condition
Should check time before count
2021-08-02 09:19:02 -04:00
FoxxMD
0a3409cfef Add title matching on SubmissionState filter 2021-08-01 22:07:19 -04:00
FoxxMD
8b1d3cb170 Implement MessageAction
* enable bot to send private messages as self or modmail
* add necessary permissions to oauth helper to make this possible
* update schema with message action structure
2021-07-31 15:05:49 -04:00
FoxxMD
90df5f45a8 Add parent submission state testing when checking a comment activity 2021-07-31 14:22:17 -04:00
FoxxMD
2a5f812dba Implement defining multiple operators 2021-07-30 14:19:49 -04:00
FoxxMD
54905da782 Implement extended memory store to allow pruning 2021-07-30 13:48:58 -04:00
FoxxMD
bfbbb3466a Update footer link 2021-07-29 11:01:11 -04:00
FoxxMD
802884f686 Add operator configuration documentation 2021-07-26 17:06:02 -04:00
FoxxMD
7cda47183f Simplify cli options and improve web server checks
* Check for in-use web port before starting so we can display a useful error message
* Remove 'web' command and instead make it an optional argument for `run` (default to web)
* Update dockerfile to use run command
* Remove remaining commander option defaults since they are defined in config builder now
2021-07-26 12:18:51 -04:00
FoxxMD
a91b9ab146 Use more robust caching implementation
* use node-cache-manager so operator has a choice of memory or redis
* BC update TTL values to be in seconds instead of milliseconds
* Count requests and misses for cache
* display cache stats in ui
2021-07-22 17:47:19 -04:00
FoxxMD
73c3052c69 Refactor how configuration is parsed
* Implement a weak/strong interface for operator (app) config
* Consolidate config parsing code into ConfigBuilder
* Implement parsing configuration from a json file
* Refactor configuration parsing so there is a clear hierarchy to where and how config is overridden
2021-07-22 15:17:41 -04:00
FoxxMD
9316019b01 Implement api nanny mode to help with heavy api usage 2021-07-20 20:15:15 -04:00
FoxxMD
4642f67104 Implement loading operator arguments (main program) from file 2021-07-20 16:38:35 -04:00
FoxxMD
c9bdae66dd Implement configurable shared modqueue for operator convenience
If operator is running subreddits for lop-sided traffic burdens then /r/mod may be over the 100 activity limit, all from subreddit X while sub Y and Z have few activities. In this scenario Y and Z would never run since X would take up all results. Therefore default to individual modqueues and make shared modqueue opt-in by operator
2021-07-19 23:04:47 -04:00
FoxxMD
dd202ac790 Add console to error/rejection transports 2021-07-19 22:37:39 -04:00