Commit Graph

873 Commits

Author SHA1 Message Date
FoxxMD
8085e8ced6 Merge branch 'edge' into databaseSupport 2022-04-19 09:30:29 -04:00
FoxxMD
749980abbe feat(repeat): Make identifying repeat sets from reference submission more fuzzy
When reference submission is a self post identifier may vary slightly since it is considering both title and body. Use string matching on identifies to find "close" matches for reference submission
2022-04-18 11:40:10 -04:00
FoxxMD
db83f1d0e7 Update schema after merge 2022-04-14 14:24:10 -04:00
FoxxMD
77960ba186 Merge branch 'edge' into databaseSupport
# Conflicts:
#	src/Schema/Action.json
#	src/Schema/App.json
#	src/Schema/OperatorConfig.json
#	src/Schema/Rule.json
#	src/Schema/RuleSet.json
#	src/Subreddit/SubredditResources.ts
2022-04-14 14:23:28 -04:00
FoxxMD
03fc562643 fix(filter): Remove/implement debug placeholder for bot self account substitution
oops
2022-04-14 13:44:07 -04:00
FoxxMD
9ef58dfd27 feat(filter): Implement testing for which moderator approved or removed an activity
Extends removed/approved criteria to accept a string, strings, or object with testing behavior and names
2022-04-14 13:31:56 -04:00
FoxxMD
1ce415bcd5 Merge branch 'edge' into databaseSupport
# Conflicts:
#	src/util.ts
2022-04-14 12:11:28 -04:00
FoxxMD
e4a9e47d6a fix(filter): Handle scenario where submissionState is incorrectly used on submission itemIs
* Check for found criteria results when building summary
* change log level from debug to warn when submissionState is detected on submission itemIs
2022-04-14 12:10:41 -04:00
FoxxMD
dc1a999376 Merge branch 'edge' into databaseSupport
# Conflicts:
#	src/Rule/RepeatActivityRule.ts
2022-04-14 10:07:35 -04:00
FoxxMD
61109fa4c0 feat(filter): Implement named filter reuse in configuration 2022-04-13 15:21:03 -04:00
FoxxMD
95eaa66080 feat(filter): Implement named filter criteria (author, item) infrastructure
Refactor internal code to handle anonymous or named criteria for filters
2022-04-13 11:28:32 -04:00
FoxxMD
bda4d82191 feat: Implement filtering subreddits by own profile
isOwnProfile in SubredditState allows checking if any Activity belongs to the user profile of the Author of the Activity being checked
2022-04-12 12:13:34 -04:00
FoxxMD
69cb823203 refactor(cache): Implement check caching correctly based on new check result data structure
* Use check result from DB if found
* Otherwise build simple rule results and partial check properties from cache data
2022-04-12 10:44:04 -04:00
FoxxMD
ec41c2f8ef feat: Show filter outcome as check summary when any failed 2022-04-12 10:43:02 -04:00
FoxxMD
964f9f9225 fix: show check name in events ui 2022-04-12 10:42:27 -04:00
FoxxMD
870b0068e9 refactor(ui): Refactor client and ui rendering to match data structures (rulesets) and render correctly 2022-04-12 09:44:10 -04:00
FoxxMD
26a563def3 fix: Correct display text for generic value comparison
Check if percent capture group is undefined OR empty
2022-04-12 08:53:55 -04:00
FoxxMD
a3685b8c81 refactor(database): Improve rule result associations
* Separate rule results from check result ownership since results can be re-used
* Implement ruleset data structure for better representation of results
* Add ruleset assoication to check result
* Partially refactor check caching (TODO)
2022-04-12 08:42:49 -04:00
FoxxMD
5710147291 fix(database): Use repository to save changes to manager state 2022-04-11 10:03:20 -04:00
FoxxMD
e7732c5a55 feat(filter): Implement exclude functionality for itemIs filter
* Same behavior/structure as authorIs
* Refactor interfaces to use a generic FilterOption type
* Allow simplified data for item/author filters -- when prop data is an array of criteria treat as "include OR"
2022-04-08 14:20:27 -04:00
FoxxMD
ccee0ef6c0 refactor: Move author interfaces and utility functions into common files 2022-04-08 10:42:16 -04:00
FoxxMD
2893f40ec5 fix(database): Check for undefined runs on CMEvent
Ensure that we only try to sort runs on an Event if they exist
2022-04-07 15:25:46 -04:00
FoxxMD
b670ad73c4 refactor(docker): Improve data directory flexibility and build order
* Create data directory if not /config
* Copy s6 files in base to make incremental builds faster
2022-04-07 15:23:14 -04:00
FoxxMD
6764995dfe refactor(docker)!: Switch to linuxserverio.io alpine as base and re-work init to make permissions less rigid
* Switching to lsio as base means inclusion of s6-overlay for improved init, handling of default permissions, and a maintained base for security/bug fixes
* Also means we can use PUID and PGID as variables instead of requiring --user in run command (more friendly to unraid, portainer, etc...)
* BREAKING: mount directory for configuration,etc. has been changed to /config by default to align with lsio
2022-04-07 14:58:10 -04:00
FoxxMD
11be4085be refactor(docker): Reclaim some space
* Clear npm cache after final production install to reclaim 100MB on image size
* More comprehensive dockerignore to prevent accidentally including development files
2022-04-07 13:10:33 -04:00
FoxxMD
31a3ae9a6b fix(database): Fix existing tables detection
* Use correct property from entity metadata which has all real db prefixes added (tablePath)
* Look for specified metadata table name instead of generic 'migrations'
* Simplify conditions (DRY)
2022-04-07 11:47:30 -04:00
FoxxMD
339b85e30a Merge branch 'edge' into databaseSupport
# Conflicts:
#	src/Action/UserNoteAction.ts
2022-04-07 10:54:30 -04:00
FoxxMD
f0eb1f040e feat(usernote): Make user note wiki edit reason actually useful
Include the note type, user affected, submission/comment id, and preview of text in edit reason
2022-04-07 09:53:33 -04:00
FoxxMD
5896623822 fix(ui): Improve action data parsing and fix undefined type 2022-04-07 09:26:49 -04:00
FoxxMD
3c6bd9f1e4 fix(ui): Fix passed value for instance id when no friendly name is set
Fixes #85
2022-04-07 09:17:11 -04:00
FoxxMD
af28ab5291 fix: Explicitly set sorting method for author activities
Don't rely on default sort chosen by reddit as comment/overview are sorted by new but submitted is sorted by hot (why??). Make sure requests always sort by new to prevent inconsistencies.
2022-04-06 19:03:45 -04:00
FoxxMD
27786d9e6f refactor(usernotes): Ensure notes subreddit is entirely independent of any parent object
To prevent any accidental referenced object property re-assignment only use subreddit name to construct subreddit for usernotes. Want to make sure notes don't get read/written to different subs by mistake...this shouldn't be happening but just to be double extra sure.
2022-04-06 13:26:40 -04:00
FoxxMD
dc25078b01 fix(cache): Fix missing re-hydration of cached subreddits 2022-04-06 13:03:16 -04:00
FoxxMD
565c0ea34f refactor(database): Move session secret logic into storage provider 2022-04-06 11:51:13 -04:00
FoxxMD
8a6e67aaae feat(database): Add more helpful logging for database init
* Normalize logging prefixes for database
* Make intent of sqljs location name change clearer
* Add warning for using sqljs in production
2022-04-06 11:26:45 -04:00
FoxxMD
5cbc669dfd refactor(database): Revert f4e5f1a2 and allow specifying cache/database for web independently
Since web DB needs to be built separately from server to allow migrations anyway might as well provide *all* the flexibility

* Reverts breaking change removing caching from web
* Add database config override in web config
2022-04-06 11:00:17 -04:00
FoxxMD
2b39644202 fix(database): Fix some lingering issue for web migrations
* Remove user name from log (there is no user)
* Fix endpoint verb for getting database logs
2022-04-06 09:39:56 -04:00
FoxxMD
3f1aeeb421 refactor(database): Separate web and server persistence
To make persistence align with client-server architecture

* Create separate datasource instances for client and server
* Refactor migration logic into service so code can be reused between client/server
* Implement endpoints for backup/migrate on client
* Move migrations into separate folders and move client sessions, invites, and web settings into own migration
* Defer session secret generation until client instantiation and save to database when possible
2022-04-05 16:32:52 -04:00
FoxxMD
f4e5f1a22f refactor(web)!: Migrate all web storage to use either database or session
* BREAKING: Replace web.caching with web.storage and restrict options to specifying either top-level database or cache
  * Simplifies init but provides options for different use cases
  * Default to database
* Implement generic storage provider for web persistence. Uses either cache or database
* Refactor session/invites to use storage provider
* Implement invite entity for database
* Allow specifying different storage provider for sessions
2022-04-05 13:18:18 -04:00
FoxxMD
c81703720b refactor(session): Migrate session storage into typeorm database 2022-04-05 10:56:47 -04:00
FoxxMD
ed06213c62 refactor(database): Simplify rule/action premise data structures
* Remove action/rule as standalone entities because they depend too much on premise data to make sense (not normalized). Instead, consolidate into premise entities.
* Breakout premise config into config/itemIs/authorIs data to make future querying easier
2022-04-04 13:37:51 -04:00
FoxxMD
402119d718 fix(app): Clearer startup logging 2022-03-31 13:13:07 -04:00
FoxxMD
2e8f90c1a8 feat(database): More database and config improvements for startup
* Improve errors when checking if file is readable or directory/file is writable
* Implement custom winston-typeorm logging mappings so migrations are output at INFO level
* Add typeorm logging options to database config and default to displaying error, warning, and migrations
* Ensure directory is writeable before using log rotation transport for winston
* Move init logger into index so its always available
* Simplify printing stack for SimpleError when it's a cause
* Better handling for access permissions when reading operator config file
* Add hint to errors relating to permissions access if app detects its running in docker
2022-03-31 13:05:07 -04:00
FoxxMD
408c495d63 refactor(dispatch): Move manager state peristence out of config-dependent resources for earlier access
So that syncing state doesn't require config to be valid
2022-03-31 09:51:50 -04:00
FoxxMD
f469dad106 refactor(dispatch): Add author name to database object to avoid have to fetch activities on startup
Storing author name means we can show delayed summary without having to fetch every delayed activity
2022-03-31 09:24:22 -04:00
FoxxMD
e9b76bc503 refactor(docker): Improve permissions and set default db driver
* Change project location to /home/node to avoid root permissions when creating WORKDIR
* Create default config directory with correct permissions in the event no volume is mounted
* Set default db driver to better-sqlite3 since we control environment -- it has better performance and we know we can use a pre-built binary b/c of base docker image
2022-03-30 16:20:07 -04:00
FoxxMD
db2c7d40da feat(config): Add DB_DRIVER env as convenience
* Makes it easier to switch between sqljs and sqlite
* Useful for docker
2022-03-30 16:17:35 -04:00
FoxxMD
cf909d49cc refactor(docker): Provide a default DATA_DIR to simplify mount binding 2022-03-30 12:56:41 -04:00
FoxxMD
4fdbb354ec feat(config): Improve/simplify configuration paths
* Add DATA_DIR env to allow specifying a "base directory" for all other config paths. Defaults to project root (same as existing behavior)
* OPERATOR_ENV OPERATOR_CONFIG LOG_DIR and database locations can now be specified as absolute OR relative (from DATA_DIR)
* Look for config.yaml and config.json if no OPERATOR_CONFIG is present
2022-03-30 10:55:55 -04:00
FoxxMD
dfbea31120 fix(manager): Fix missing dryrun value in runtime options when handling an activity 2022-03-29 17:02:26 -04:00