Merge branch 'edge'

since 'to' be now be templated a user can configure a message to send to the subreddit the action is being processed from using `to: 'r/{{item.subreddit}}'`

Dockerfile

@@ -120,6 +120,10 @@ ENV NPM_CONFIG_LOGLEVEL debug
 # can set database to use more performant better-sqlite3 since we control everything
 ENV DB_DRIVER=better-sqlite3
 
+# NODE_ARGS are expanded after `node` command in the entrypoint IE "node {NODE_ARGS} src/index.js run"
+# by default enforce better memory mangement by limiting max long-lived GC space to 512MB
+ENV NODE_ARGS="--max_old_space_size=512"
+
 ARG webPort=8085
 ENV PORT=$webPort
 EXPOSE $PORT

docker/root/etc/services.d/node/run

@@ -2,6 +2,8 @@
 
 # used https://github.com/linuxserver/docker-plex as a template
 
+# NODE_ARGS can be passed by ENV in docker command like "docker run foxxmd/context-mod -e NODE_ARGS=--optimize_for_size"
+
 exec \
 	s6-setuidgid abc \
-	/usr/local/bin/node /app/src/index.js run
+	/usr/local/bin/node $NODE_ARGS /app/src/index.js run

docs/operator/README.md

@@ -21,7 +21,7 @@ They are responsible for configuring the software at a high-level and managing a
 
 # Overview
 
-CM is composed of two applications that operate indepedently but are packaged together such that they act as one piece of software:
+CM is composed of two applications that operate independently but are packaged together such that they act as one piece of software:
 
 * **Server** -- Responsible for **running the bot(s)** and providing an API to retrieve information on and interact with them EX start/stop bot, reload config, retrieve operational status, etc.
 * **Client** -- Responsible for serving the **web interface** and handling the bot oauth authentication flow between operators and subreddits/bots.

docs/operator/configuration.md

@@ -136,6 +136,8 @@ You will need have this information available:
 
 See the [**example minimum configuration** below.](#minimum-config)
 
+This configuration can also be **generated** by CM if you start CM with **no configuration defined** and visit the web interface.
+
 # Bots
 
 Configured using the `bots` top-level property. Bot configuration can override and specify many more options than are available at the operator-level. Many of these can also set the defaults for each subreddit the bot runs:

docs/operator/gettingStarted.md

@@ -4,9 +4,7 @@ This getting started guide is for **Operators** -- that is, someone who wants to
 
 * [Installation](#installation)
 * [Create a Reddit Client](#create-a-reddit-client)
-* [Create a Minimum Configuration](#create-a-minimum-configuration)
-  * [Local Installation](#local-installation)
-  * [Docker Installation](#docker-installation)
+* [Start ContextMod](#start-contextmod)
 * [Add a Bot to CM](#add-a-bot-to-cm)
 * [Access The Dashboard](#access-the-dashboard)
 * [What's Next?](#whats-next)
@@ -19,29 +17,25 @@ Follow the [installation](/docs/operator/installation.md) documentation. It is r
 
 [Create a reddit client](/docs/operator/README.md#provisioning-a-reddit-client)
 
-# Create a Minimum Configuration
+# Start ContextMod 
 
-Using the information you received in the previous step [create a minimum file configuration](/docs/operator/configuration.md#minimum-configuration) save it as `config.yaml` somewhere.
+Start CM using the example command from your [installation](#installation) and visit http://localhost:8085
 
-# Start ContextMod With Configuration
+The First Time Setup page will ask you to input:
 
-## Local Installation
+* Client ID (from [Create a Reddit Client](#create-a-reddit-client))
+* Client Secret (from [Create a Reddit Client](#create-a-reddit-client))
+* Operator -- this is the username of your main Reddit account.
 
-If you [installed CM locally](/docs/installation.md#locally) move your configuration file `config.yaml` to the root of the project directory (where `package.json`) is located.
-
-From the root directory run this command to start CM
-
-```
-node src/index.js run
-```
-
-## Docker Installation
-
-If you [installed CM using Docker](/docs/installation.md#docker-recommended) make note of the directory you saved your minimum configuration to and substitute its full path for `host/path/folder` in the docker command show in the [docker install directions](/docs/operator/installation.md#docker-recommended)
+**Write Config** and then restart CM. You have now created the [minimum configuration](/docs/operator/configuration.md#minimum-configuration) required to run CM.
 
 # Add A Bot to CM
 
-Once CM is up and running use the [CM OAuth Helper](/docs/operator/addingBot.md#cm-oauth-helper-recommended) to add authorize and add a Bot to your CM instance.
+You should automatically be directed to the [Bot Invite Helper](/docs/operator/addingBot.md#cm-oauth-helper-recommended) used to authorize and add a Bot to your CM instance.
+
+Follow the directions here and **create an Authorization Invite** at the bottom of the page. 
+
+Next, login to Reddit with the account you will be using as the Bot and then visit the **Authorization Invite** link you created. Follow the steps there to finish adding the Bot to your CM instance.
 
 # Access The Dashboard
 
@@ -57,4 +51,4 @@ As an operator you should familiarize yourself with how the [operator configurat
 
 If you are also the moderator of the subreddit the bot will be running you should check out the [moderator getting started guide.](/docs/subreddit/gettingStarted.md#setup-wiki-page)
 
-You might also be interested in these [quick tips for using the web interface](/docs/webInterface.md)
+You might also be interested in these [quick tips for using the web interface](/docs/webInterface.md). Additionally, on the dashboard click the **Help** button at the top of the page to get a guided tour of the dashboard.

docs/operator/installation.md

@@ -10,9 +10,10 @@ PROTIP: Using a container management tool like [Portainer.io CE](https://www.por
 
 ### [Dockerhub](https://hub.docker.com/r/foxxmd/context-mod)
 
-An example of starting the container using the [minimum configuration](/docs/operator/operatorConfiguration.md#minimum-config) with a [configuration file](/docs/operator/operatorConfiguration.md#defining-configuration-via-file):
+An example of starting the container using the [minimum configuration](/docs/operator/configuration.md#minimum-config):
 
 * Bind the directory where your config file, logs, and database are located on your host machine into the container's default `DATA_DIR` by using `-v /host/path/folder:/config`
+  * Note: **You must do this** or else your configuration will be lost next time your container is updated.
 * Expose the web interface using the container port `8085`
 
 ```
@@ -76,3 +77,21 @@ Be aware that Heroku's [free dyno plan](https://devcenter.heroku.com/articles/fr
 * The **Worker** dyno **will not** go to sleep but you will NOT be able to access the web interface. You can, however, still see how Cm is running by reading the logs for the dyno.
 
 If you want to use a free dyno it is recommended you perform first-time setup (bot authentication and configuration, testing, etc...) with the **Web** dyno, then SWITCH to a **Worker** dyno so it can run 24/7.
+
+# Memory Management
+
+Node exhibits [lazy GC cleanup](https://github.com/FoxxMD/context-mod/issues/90#issuecomment-1190384006) which can result in memory usage for long-running CM instances increasing to unreasonable levels. This problem does not seem to be an issue with CM itself but with Node's GC approach. The increase does not affect CM's performance and, for systems with less memory, the Node *should* limit memory usage based on total available.
+
+In practice CM uses ~130MB for a single bot, single subreddit setup. Up to ~350MB for many (10+) bots or many (20+) subreddits.
+
+If you need to reign in CM's memory usage for some reason this can be addressed by setting an upper limit for memory usage with `node` args by using either:
+
+**--max_old_space_size=**
+
+Value is megabytes. This sets an explicit limit on GC memory usage.
+
+This is set by default in the [Docker](#docker-recommended) container using the env `NODE_ARGS` to `--max_old_space_size=512`. It can be disabled by overriding the ENV.
+
+**--optimize_for_size**
+
+Tells Node to optimize for (less) memory usage rather than some performance optimizations. This option is not memory size dependent. In practice performance does not seem to be affected and it reduces (but not entirely prevents) memory increases over long periods.

docs/subreddit/actionTemplating.md

@@ -1,12 +1,32 @@
-Actions that can submit text (Report, Comment, UserNote) will have their `content` values run through a [Mustache Template](https://mustache.github.io/). This means you can insert data generated by Rules into your text before the Action is performed.
+Actions that can submit text (Report, Comment, UserNote, Message, Ban, Submission) will have their `content` values run through a [Mustache Template](https://mustache.github.io/). This means you can insert data generated by Rules into your text before the Action is performed.
 
 See here for a [cheatsheet](https://gist.github.com/FoxxMD/d365707cf99fdb526a504b8b833a5b78) and [here](https://www.tsmean.com/articles/mustache/the-ultimate-mustache-tutorial/) for a more thorough tutorial.
 
 # Template Data
 
+Some data can always be accessed at the top-level. Example
+
+```
+This action was run from {{manager}} in Check {{check}}.
+
+The bot intro post is {{botLink}}
+
+Message the moderators of this subreddit using this [compose link]({{modmailLink}})
+```
+
+
+
+|     Name      |                                         Description                                         |                                                                         Example                                                                          |
+|---------------|---------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `manager`     | The name of the subreddit the bot is running in                                             | mealtimevideos                                                                                                                                           |
+| `check`       | The name of the Check that was triggered                                                    | myCheck                                                                                                                                                  |
+| `botLink`     | A link to the bot introduction                                                              | https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot                                                                     |
+| `modmailLink` | A link that opens reddit's DM compose with the subject line as the Activity being processed | https://www.reddit.com/message/compose?to=/r/mealtimevideos&message=https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot |
+
+
 ## Activity Data
 
-Activity data can be accessed using the `item` variable. Example
+**Activity data can be accessed using the `item` variable.** Example
 
 ```
 This activity is a {{item.kind}} with {{item.votes}} votes, created {{item.age}} ago.
@@ -19,14 +39,17 @@ Produces:
 
 All Actions with `content` have access to this data:
 
-| Name        | Description                                                                                         | Example                                                                              |
-|-------------|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
-| `kind`      | The Activity type (submission or comment)                                                           | submission                                                                           |
-| `author`    | Name of the Author of the Activity being processed                                                  | FoxxMD                                                                               |
-| `permalink` | URL to the Activity                                                                                 | https://reddit.com/r/mySuibreddit/comments/ab23f/my_post                             |
-| `votes`     | Number of upvotes                                                                                   | 69                                                                                   |
-| `age`       | The age of the Activity in a [human friendly format](https://day.js.org/docs/en/durations/humanize) | 5 minutes                                                                            |
-| `botLink`   | A URL to CM's introduction thread                                                                   | https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot |
+|     Name     |                                             Description                                             |                         Example                          |
+|--------------|-----------------------------------------------------------------------------------------------------|----------------------------------------------------------|
+| `kind`       | The Activity type (submission or comment)                                                           | submission                                               |
+| `author`     | Name of the Author of the Activity being processed                                                  | FoxxMD                                                   |
+| `permalink`  | URL to the Activity                                                                                 | https://reddit.com/r/mySuibreddit/comments/ab23f/my_post |
+| `votes`      | Number of upvotes                                                                                   | 69                                                       |
+| `age`        | The age of the Activity in a [human friendly format](https://day.js.org/docs/en/durations/humanize) | 5 minutes                                                |
+| `subreddit`  | The name of the subreddit the Activity is from                                                      | mealtimevideos                                           |
+| `id`         | The `Reddit Thing` ID for the Activity                                                              | t3_0tin1                                                 |
+| `title`      | As comments => the body of the comment. As Submission => title                                      | Test post please ignore                                  |
+| `shortTitle` | The same as `title` but truncated to 15 characters                                                  | test post pleas...                                       |
 
 ### Submissions
 
@@ -89,7 +112,39 @@ Produces
 
 > Submission was repeated 7 times
 
-#### Quick Templating Tutorial
+## Action Data
+
+### Summary
+
+A summary of what actions have already been run **when the template is rendered**  is available using the `actionSummary` variable. It is therefore important that the Action you want to produce the summary is run **after** any other Actions you want to get a summary for.
+
+Example:
+
+```
+A summary of actions processed for this activity, so far:
+
+{{actionSummary}}
+```
+
+Would produce:
+> A summary of actions processed for this activity, so far:
+>
+> * approve - ✘ - Item is already approved??
+> * lock - ✓
+> * modnote - ✓ - (SOLID_CONTRIBUTOR) User is good
+
+### Individual
+
+Individual **Actions** can be accessed using the name of the action, **lower-cased, with all spaces/dashes/underscores.** Example:
+
+```
+User was banned for {{actions.exampleban.duration}} for {{actions.exampleban.reason}}
+```
+Produces
+
+> User was banned for 4 days for toxic behavior
+
+# Quick Templating Tutorial
 
 As a quick example for how you will most likely be using templating -- wrapping a variable in curly brackets, `{{variable}}`, will cause the variable value to be rendered instead of the brackets:
 

docs/subreddit/components/README.md

@@ -29,6 +29,7 @@ This list is not exhaustive. [For complete documentation on a subreddit's config
   * [List of Actions](#list-of-actions)
     * [Approve](#approve)
     * [Ban](#ban)
+    * [Submission](#submission)
     * [Comment](#comment)
     * [Contributor (Add/Remove)](#contributor)
     * [Dispatch/Delay](#dispatch)
@@ -494,11 +495,30 @@ actions:
 
 ### Comment
 
-Reply to the Activity being processed with a comment. [Schema Documentation](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FCommentActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)
+Reply to an Activity with a comment. [Schema Documentation](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FCommentActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)
 
 * If the Activity is a Submission the comment is a top-level reply
 * If the Activity is a Comment the comment is a child reply
 
+#### Templating
+
+`content` can be [templated](#templating) and use [URL Tokens](#url-tokens)
+
+#### Targets
+
+Optionally, specify the Activity CM should reply to. **When not specified CM replies to the Activity being processed using `self`**
+
+Valid values: `self`, `parent`, or a Reddit permalink.
+
+`self` and `parent` are special targets that are relative to the Activity being processed:
+
+* When the Activity being processed is a **Submission** => `parent` logs a warning and does nothing
+* When the Activity being processed is a  **Comment**
+  * `self` => reply to Comment
+  * `parent` => make a top-level Comment in the **Submission** the Comment belong to
+
+If target is not self/parent then CM assumes the value is a **reddit permalink** and will attempt to make a Comment to that Activity
+
 ```yaml
 actions:
   - kind: comment
@@ -506,7 +526,71 @@ actions:
     distinguish: boolean # distinguish as a mod
     sticky: boolean # sticky comment
     lock: boolean # lock the comment after creation
+    targets: string # 'self' or 'parent' or 'https://reddit.com/r/someSubreddit/21nfdi....'
+```
 
+### Submission
+
+Create a Submission [Schema Documentation](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FSubmissionActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)
+
+The Submission type, Link or Self-Post, is determined based on the presence of `url` in the action's configuration.
+
+```yaml
+actions:
+  - kind: submission
+    title: string # required, the title of the submission. can be templated.
+    content: string # the body of the submission. can be templated
+    url: string # if specified the submission will be a Link Submission. can be templated
+    distinguish: boolean # distinguish as a mod
+    sticky: boolean # sticky comment
+    lock: boolean # lock the comment after creation
+    nsfw: boolean # mark submission as NSFW
+    spoiler: boolean # mark submission as a spoiler
+    flairId: string # flair template id for submission
+    flairText: string # flair text for submission
+    targets: string # 'self' or a subreddit name IE mealtimevideos
+```
+
+#### Templating
+
+`content`,`url`, and `title` can be [templated](#templating) and use [URL Tokens](#url-tokens)
+
+TIP: To create a Link Submission pointing to the Activity currently being processed use
+
+```yaml
+actions:
+  - kind: submission
+    url: {{item.permalink}}
+    # ...
+```
+
+#### Targets
+
+Optionally, specify the Subreddit the Submission should be made in. **When not specified CM uses `self`**
+
+Valid values: `self` or Subreddit Name
+
+* `self` => (**Default**) Create Submission in the same Subreddit of the Activity being processed
+* Subreddit Name => Create Submission in given subreddit IE `mealtimevideos`
+  * Your bot must be able to access and be able to post in the given subreddit
+
+Example: 
+
+```yaml
+actions:
+  - kind: comment
+    targets: mealtimevideos
+```
+
+To post to multiple subreddits use a list:
+
+```yaml
+actions:
+  - kind: comment
+    targets:
+      - self
+      - mealtimevideos
+      - anotherSubreddit 
 ```
 
 ### Contributor
@@ -608,31 +692,35 @@ Some other things to note:
 * If the `to` property is not specified then the message is sent to the Author of the Activity being processed
   * `to` may be a **User** (u/aUser) or a **Subreddit** (r/aSubreddit)
   * `to` **cannot** be a Subreddit when `asSubreddit: true` -- IE cannot send subreddit-to-subreddit messages
-* `content` can be [templated](#templating) and use [URL Tokens](#url-tokens)
+  * TIP: `to` can be templated -- to send a message to the subreddit the Activity being processed is in use `'r/{{item.subreddit}}'`
+* `content` and `title` can be [templated](#templating) and use [URL Tokens](#url-tokens)
 
 ```yaml
 actions:
   - kind: message
     asSubreddit: true
-    content: 'A message sent as the subreddit'
-    title: 'Title of the message'
-    to: 'u/aUser' # do not specify 'to' in order default to sending to Author of Activity being processed
+    content: 'A message sent as the subreddit' # can be templated
+    title: 'Title of the message' # can be templated
+    to: 'u/aUser' # do not specify 'to' in order default to sending to Author of Activity being processed. Can also be templated
 ```
 
 ### Remove
 
 Remove the Activity being processed. [Schema Documentation](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FRemoveActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fedge%2Fsrc%2FSchema%2FApp.json)
 
+* **note** can be [templated](#templating)
+* **reasonId** IDs can be found in the [editor](/docs/webInterface.md) using the **Removal Reasons** popup
+
+If neither note nor reasonId are included then no removal reason is added.
+
 ```yaml
 actions:
   - kind: remove
-    spam: boolean # optional, mark as spam on removal
+    spam: false # optional, mark as spam on removal
+    note: 'a moderator-readable note' # optional, a note only visible to moderators (new reddit only)
+    reasonId: '2n0f4674-365e-46d2-8fc7-a337d85d5340' # optional, the ID of a removal reason to add to the removal action (new reddit only)
 ```
 
-#### What About Removal Reason?
-
-Reddit does not support setting a removal reason through the API. Please complain in [r/modsupport](https://www.reddit.com/r/modsupport) or [r/redditdev](https://www.reddit.com/r/redditdev) to help get this added :)
-
 ### Report
 
 Report the Activity being processed. [Schema Documentation](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson/%23%2Fdefinitions%2FReportActionJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fedge%2Fsrc%2FSchema%2FApp.json)

docs/webInterface.md

@@ -20,6 +20,7 @@
 
 ## Web Dashboard Tips
 
+* Click the **Help** button at the top of the page to get a **guided tour of the dashboard**
 * Use the [**Overview** section](/docs/images/botOperations.png) to control the bot at a high-level
 * You can **manually run** the bot on any activity (comment/submission) by pasting its permalink into the [input field below the Overview section](/docs/images/runInput.png) and hitting one of the **run buttons**
     * **Dry run** will make the bot run on the activity but it will only **pretend** to run actions, if triggered. This is super useful for testing your config without consequences

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "redditcontextbot",
-  "version": "0.5.1",
+  "version": "0.11.4",
   "lockfileVersion": 2,
   "requires": true,
   "packages": {
     "": {
       "name": "redditcontextbot",
-      "version": "0.5.1",
+      "version": "0.11.4",
       "hasInstallScript": true,
       "license": "ISC",
       "dependencies": {
@@ -77,6 +77,7 @@
         "triple-beam": "^1.3.0",
         "typeorm": "^0.3.7",
         "typeorm-logger-adaptor": "^1.1.0",
+        "unique-names-generator": "^4.7.1",
         "vader-sentiment": "^1.1.3",
         "webhook-discord": "^3.7.7",
         "wink-sentiment": "^5.0.2",
@@ -9741,6 +9742,14 @@
         "node": ">= 0.8.x"
       }
     },
+    "node_modules/unique-names-generator": {
+      "version": "4.7.1",
+      "resolved": "https://registry.npmjs.org/unique-names-generator/-/unique-names-generator-4.7.1.tgz",
+      "integrity": "sha512-lMx9dX+KRmG8sq6gulYYpKWZc9RlGsgBR6aoO8Qsm3qvkSJ+3rAymr+TnV8EDMrIrwuFJ4kruzMWM/OpYzPoow==",
+      "engines": {
+        "node": ">=8"
+      }
+    },
     "node_modules/universalify": {
       "version": "0.1.2",
       "resolved": "https://registry.npmjs.org/universalify/-/universalify-0.1.2.tgz",
@@ -17837,6 +17846,11 @@
       "resolved": "https://registry.npmjs.org/unicode/-/unicode-14.0.0.tgz",
       "integrity": "sha512-BjinxTXkbm9Jomp/YBTMGusr4fxIG67fNGShHIRAL16Ur2GJTq2xvLi+sxuiJmInCmwqqev2BCFKyvbfp/yAkg=="
     },
+    "unique-names-generator": {
+      "version": "4.7.1",
+      "resolved": "https://registry.npmjs.org/unique-names-generator/-/unique-names-generator-4.7.1.tgz",
+      "integrity": "sha512-lMx9dX+KRmG8sq6gulYYpKWZc9RlGsgBR6aoO8Qsm3qvkSJ+3rAymr+TnV8EDMrIrwuFJ4kruzMWM/OpYzPoow=="
+    },
     "universalify": {
       "version": "0.1.2",
       "resolved": "https://registry.npmjs.org/universalify/-/universalify-0.1.2.tgz",

package.json

@@ -97,6 +97,7 @@
     "triple-beam": "^1.3.0",
     "typeorm": "^0.3.7",
     "typeorm-logger-adaptor": "^1.1.0",
+    "unique-names-generator": "^4.7.1",
     "vader-sentiment": "^1.1.3",
     "webhook-discord": "^3.7.7",
     "wink-sentiment": "^5.0.2",

src/Action/ActionFactory.ts

@@ -3,7 +3,7 @@ import LockAction, {LockActionJson} from "./LockAction";
 import {RemoveAction, RemoveActionJson} from "./RemoveAction";
 import {ReportAction, ReportActionJson} from "./ReportAction";
 import {FlairAction, FlairActionJson} from "./SubmissionAction/FlairAction";
-import Action, {ActionJson, StructuredActionJson} from "./index";
+import Action, {ActionJson, ActionRuntimeOptions, StructuredActionJson} from "./index";
 import {Logger} from "winston";
 import {UserNoteAction, UserNoteActionJson} from "./UserNoteAction";
 import ApproveAction, {ApproveActionConfig} from "./ApproveAction";
@@ -18,38 +18,41 @@ import {CancelDispatchAction, CancelDispatchActionJson} from "./CancelDispatchAc
 import ContributorAction, {ContributorActionJson} from "./ContributorAction";
 import {StructuredFilter} from "../Common/Infrastructure/Filters/FilterShapes";
 import {ModNoteAction, ModNoteActionJson} from "./ModNoteAction";
+import {SubmissionAction, SubmissionActionJson} from "./SubmissionAction";
 
 export function actionFactory
-(config: StructuredActionJson, logger: Logger, subredditName: string, resources: SubredditResources, client: ExtendedSnoowrap, emitter: EventEmitter): Action {
+(config: StructuredActionJson, runtimeOptions: ActionRuntimeOptions): Action {
     switch (config.kind) {
         case 'comment':
-            return new CommentAction({...config as StructuredFilter<CommentActionJson>, logger, subredditName, resources, client, emitter});
+            return new CommentAction({...config as StructuredFilter<CommentActionJson>, ...runtimeOptions});
+        case 'submission':
+            return new SubmissionAction({...config as StructuredFilter<SubmissionActionJson>, ...runtimeOptions});
         case 'lock':
-            return new LockAction({...config as StructuredFilter<LockActionJson>, logger, subredditName, resources, client, emitter});
+            return new LockAction({...config as StructuredFilter<LockActionJson>, ...runtimeOptions});
         case 'remove':
-            return new RemoveAction({...config as StructuredFilter<RemoveActionJson>, logger, subredditName, resources, client, emitter});
+            return new RemoveAction({...config as StructuredFilter<RemoveActionJson>, ...runtimeOptions});
         case 'report':
-            return new ReportAction({...config as StructuredFilter<ReportActionJson>, logger, subredditName, resources, client, emitter});
+            return new ReportAction({...config as StructuredFilter<ReportActionJson>, ...runtimeOptions});
         case 'flair':
-            return new FlairAction({...config as StructuredFilter<FlairActionJson>, logger, subredditName, resources, client, emitter});
+            return new FlairAction({...config as StructuredFilter<FlairActionJson>, ...runtimeOptions});
         case 'userflair':
-            return new UserFlairAction({...config as StructuredFilter<UserFlairActionJson>, logger, subredditName, resources, client, emitter});
+            return new UserFlairAction({...config as StructuredFilter<UserFlairActionJson>, ...runtimeOptions});
         case 'approve':
-            return new ApproveAction({...config as StructuredFilter<ApproveActionConfig>, logger, subredditName, resources, client, emitter});
+            return new ApproveAction({...config as StructuredFilter<ApproveActionConfig>, ...runtimeOptions});
         case 'usernote':
-            return new UserNoteAction({...config as StructuredFilter<UserNoteActionJson>, logger, subredditName, resources, client, emitter});
+            return new UserNoteAction({...config as StructuredFilter<UserNoteActionJson>, ...runtimeOptions});
         case 'ban':
-            return new BanAction({...config as StructuredFilter<BanActionJson>, logger, subredditName, resources, client, emitter});
+            return new BanAction({...config as StructuredFilter<BanActionJson>, ...runtimeOptions});
         case 'message':
-            return new MessageAction({...config as StructuredFilter<MessageActionJson>, logger, subredditName, resources, client, emitter});
+            return new MessageAction({...config as StructuredFilter<MessageActionJson>, ...runtimeOptions});
         case 'dispatch':
-            return new DispatchAction({...config as StructuredFilter<DispatchActionJson>, logger, subredditName, resources, client, emitter});
+            return new DispatchAction({...config as StructuredFilter<DispatchActionJson>, ...runtimeOptions});
         case 'cancelDispatch':
-            return new CancelDispatchAction({...config as StructuredFilter<CancelDispatchActionJson>, logger, subredditName, resources, client, emitter})
+            return new CancelDispatchAction({...config as StructuredFilter<CancelDispatchActionJson>, ...runtimeOptions})
         case 'contributor':
-            return new ContributorAction({...config as StructuredFilter<ContributorActionJson>, logger, subredditName, resources, client, emitter})
+            return new ContributorAction({...config as StructuredFilter<ContributorActionJson>, ...runtimeOptions})
         case 'modnote':
-            return new ModNoteAction({...config as StructuredFilter<ModNoteActionJson>, logger, subredditName, resources, client, emitter})
+            return new ModNoteAction({...config as StructuredFilter<ModNoteActionJson>, ...runtimeOptions})
         default:
             throw new Error('rule "kind" was not recognized.');
     }

src/Action/ApproveAction.ts

@@ -8,6 +8,7 @@ import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTarget, ActionTypes} from "../Common/Infrastructure/Atomic";
 import {asComment, asSubmission} from "../util";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class ApproveAction extends Action {
 
@@ -26,7 +27,7 @@ export class ApproveAction extends Action {
         this.targets = targets;
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
         const touchedEntities = [];
 

src/Action/BanAction.ts

@@ -7,10 +7,18 @@ import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
 import {truncateStringToLength} from "../util";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 const truncate = truncateStringToLength(100);
 const truncateLongMessage = truncateStringToLength(200);
 
+const truncateIfNotUndefined = (val: string | undefined) => {
+    if(val === undefined) {
+        return undefined;
+    }
+    return truncate(val);
+}
+
 export class BanAction extends Action {
 
     message?: string;
@@ -39,14 +47,13 @@ export class BanAction extends Action {
         return 'ban';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
-        const content = this.message === undefined ? undefined : await this.resources.getContent(this.message, item.subreddit);
-        const renderedBody = content === undefined ? undefined : await renderContent(content, item, ruleResults, this.resources.userNotes);
-        const renderedContent = renderedBody === undefined ? undefined : `${renderedBody}${await this.resources.generateFooter(item, this.footer)}`;
+        const renderedBody = await this.renderContent(this.message, item, ruleResults, actionResults);
+        const renderedContent = renderedBody === undefined ? undefined : `${renderedBody}${await this.resources.renderFooter(item, this.footer)}`;
 
-        const renderedReason = this.reason === undefined ? undefined : truncate(await renderContent(this.reason, item, ruleResults, this.resources.userNotes));
-        const renderedNote = this.note === undefined ? undefined : truncate(await renderContent(this.note, item, ruleResults, this.resources.userNotes));
+        const renderedReason = truncateIfNotUndefined(await this.renderContent(this.reason, item, ruleResults, actionResults) as string);
+        const renderedNote = truncateIfNotUndefined(await this.renderContent(this.note, item, ruleResults, actionResults) as string);
 
         const touchedEntities = [];
         let banPieces = [];
@@ -73,7 +80,13 @@ export class BanAction extends Action {
             dryRun,
             success: true,
             result: `Banned ${item.author.name} ${durText}${renderedReason !== undefined ? ` (${renderedReason})` : ''}`,
-            touchedEntities
+            touchedEntities,
+            data: {
+                message: renderedContent === undefined ? undefined : renderedContent,
+                reason: renderedReason,
+                note: renderedNote,
+                duration: durText
+            }
         };
     }
 
@@ -108,7 +121,6 @@ export interface BanActionConfig extends ActionConfig, Footer {
      *
      * If the length expands to more than 100 characters it will truncated with "..."
      *
-     * @maxLength 100
      * @examples ["repeat spam"]
      * */
     reason?: string
@@ -124,7 +136,6 @@ export interface BanActionConfig extends ActionConfig, Footer {
      *
      * If the length expands to more than 100 characters it will truncated with "..."
      *
-     * @maxLength 100
      * @examples ["Sock puppet for u/AnotherUser"]
      * */
     note?: string

src/Action/CancelDispatchAction.ts

@@ -12,6 +12,7 @@ import {isSubmission, parseDurationValToDuration} from "../util";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTarget, ActionTypes, InclusiveActionTarget} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class CancelDispatchAction extends Action {
     identifiers?: (string | null)[];
@@ -35,7 +36,7 @@ export class CancelDispatchAction extends Action {
         this.targets = !Array.isArray(target) ? [target] : target;
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         // see note in DispatchAction about missing runtimeDryrun
         const dryRun = this.dryRun;
 

src/Action/CommentAction.ts

@@ -1,12 +1,15 @@
 import Action, {ActionJson, ActionOptions} from "./index";
-import {Comment} from "snoowrap";
+import {Comment, VoteableContent} from "snoowrap";
 import Submission from "snoowrap/dist/objects/Submission";
 import {renderContent} from "../Utils/SnoowrapUtils";
 import {ActionProcessResult, Footer, RequiredRichContent, RichContent, RuleResult} from "../Common/interfaces";
-import {truncateStringToLength} from "../util";
+import {asComment, asSubmission, parseRedditThingsFromLink, truncateStringToLength} from "../util";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
-import {ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionTarget, ActionTypes, ArbitraryActionTarget} from "../Common/Infrastructure/Atomic";
+import {CMError} from "../Utils/Errors";
+import {SnoowrapActivity} from "../Common/Infrastructure/Reddit";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class CommentAction extends Action {
     content: string;
@@ -14,6 +17,7 @@ export class CommentAction extends Action {
     sticky: boolean = false;
     distinguish: boolean = false;
     footer?: false | string;
+    targets: ArbitraryActionTarget[]
 
     constructor(options: CommentActionOptions) {
         super(options);
@@ -23,71 +27,139 @@ export class CommentAction extends Action {
             sticky = false,
             distinguish = false,
             footer,
+            targets = ['self']
         } = options;
         this.footer = footer;
         this.content = content;
         this.lock = lock;
         this.sticky = sticky;
         this.distinguish = distinguish;
+        if (!Array.isArray(targets)) {
+            this.targets = [targets];
+        } else {
+            this.targets = targets;
+        }
     }
 
     getKind(): ActionTypes {
         return 'comment';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
-        const content = await this.resources.getContent(this.content, item.subreddit);
-        const body = await renderContent(content, item, ruleResults, this.resources.userNotes);
+        const body =  await this.renderContent(this.content, item, ruleResults, actionResults) as string;
 
-        const footer = await this.resources.generateFooter(item, this.footer);
+        const footer = await this.resources.renderFooter(item, this.footer);
 
         const renderedContent = `${body}${footer}`;
         this.logger.verbose(`Contents:\r\n${renderedContent.length > 100 ? `\r\n${renderedContent}` : renderedContent}`);
 
-        if(item.archived) {
-            this.logger.warn('Cannot comment because Item is archived');
-            return {
-                dryRun,
-                success: false,
-                result: 'Cannot comment because Item is archived'
-            };
-        }
+        let allErrors = true;
+        const targetResults: string[] = [];
         const touchedEntities = [];
-        let modifiers = [];
-        let reply: Comment;
-        if(!dryRun) {
-            // @ts-ignore
-           reply = await item.reply(renderedContent);
-           // add to recent so we ignore activity when/if it is discovered by polling
-           await this.resources.setRecentSelf(reply);
-           touchedEntities.push(reply);
-        }
-        if (this.lock) {
-            modifiers.push('Locked');
+
+        for (const target of this.targets) {
+
+            let targetItem = item;
+            let targetIdentifier = target;
+
+            if (target === 'parent') {
+                if (asSubmission(item)) {
+                    const noParent = `[Parent] Submission ${item.name} does not have a parent`;
+                    this.logger.warn(noParent);
+                    targetResults.push(noParent);
+                    continue;
+                }
+                targetItem = await this.resources.getActivity(this.client.getSubmission(item.link_id));
+            } else if (target !== 'self') {
+                const redditThings = parseRedditThingsFromLink(target);
+                let id = '';
+
+                try {
+                    if (redditThings.comment !== undefined) {
+                        id = redditThings.comment.id;
+                        targetIdentifier = `Permalink Comment ${id}`
+                        // @ts-ignore
+                        await this.resources.getActivity(this.client.getSubmission(redditThings.submission.id));
+                        targetItem = await this.resources.getActivity(this.client.getComment(redditThings.comment.id));
+                    } else if (redditThings.submission !== undefined) {
+                        id = redditThings.submission.id;
+                        targetIdentifier = `Permalink Submission ${id}`
+                        targetItem = await this.resources.getActivity(this.client.getSubmission(redditThings.submission.id));
+                    } else {
+                        targetResults.push(`[Permalink] Could not parse ${target} as a reddit permalink`);
+                        continue;
+                    }
+                } catch (err: any) {
+                    targetResults.push(`[${targetIdentifier}] error occurred while fetching activity: ${err.message}`);
+                    this.logger.warn(new CMError(`[${targetIdentifier}] error occurred while fetching activity`, {cause: err}));
+                    continue;
+                }
+            }
+
+            if (targetItem.archived) {
+                const archived = `[${targetIdentifier}] Cannot comment because Item is archived`;
+                this.logger.warn(archived);
+                targetResults.push(archived);
+                continue;
+            }
+
+            let modifiers = [];
+            let reply: Comment;
             if (!dryRun) {
-                // snoopwrap typing issue, thinks comments can't be locked
                 // @ts-ignore
-                await reply.lock();
+                reply = await targetItem.reply(renderedContent);
+                // add to recent so we ignore activity when/if it is discovered by polling
+                await this.resources.setRecentSelf(reply);
+                touchedEntities.push(reply);
             }
-        }
-        if (this.distinguish && !dryRun) {
-            modifiers.push('Distinguished');
-            if(this.sticky) {
-                modifiers.push('Stickied');
+
+            if (this.lock && targetItem.can_mod_post) {
+                if (!targetItem.can_mod_post) {
+                    this.logger.warn(`[${targetIdentifier}] Cannot lock because bot is not a moderator`);
+                } else {
+                    modifiers.push('Locked');
+                    if (!dryRun) {
+                        // snoopwrap typing issue, thinks comments can't be locked
+                        // @ts-ignore
+                        await reply.lock();
+                    }
+                }
             }
-            if(!dryRun) {
-                // @ts-ignore
-                await reply.distinguish({sticky: this.sticky});
+
+            if (this.distinguish) {
+                if (!targetItem.can_mod_post) {
+                    this.logger.warn(`[${targetIdentifier}] Cannot lock Distinguish/Sticky because bot is not a moderator`);
+                } else {
+                    modifiers.push('Distinguished');
+                    if (this.sticky) {
+                        modifiers.push('Stickied');
+                    }
+                    if (!dryRun) {
+                        // @ts-ignore
+                        await reply.distinguish({sticky: this.sticky});
+                    }
+                }
             }
+
+            const modifierStr = modifiers.length === 0 ? '' : ` == ${modifiers.join(' | ')} == =>`;
+            // @ts-ignore
+            targetResults.push(`${targetIdentifier}${modifierStr} created Comment ${dryRun ? 'DRYRUN' : (reply as SnoowrapActivity).name}`)
+            allErrors = false;
         }
 
-        const modifierStr = modifiers.length === 0 ? '' : `[${modifiers.join(' | ')}]`;
+
         return {
             dryRun,
-            success: true,
-            result: `${modifierStr}${truncateStringToLength(100)(body)}`,
+            success: !allErrors,
+            result: `${targetResults.join('\n')}${truncateStringToLength(100)(body)}`,
             touchedEntities,
+            data: {
+                body,
+                bodyShort: truncateStringToLength(100)(body),
+                comments: targetResults,
+                commentsFormatted: targetResults.map(x => `* ${x}`).join('\n')
+            }
         };
     }
 
@@ -97,7 +169,8 @@ export class CommentAction extends Action {
             lock: this.lock,
             sticky: this.sticky,
             distinguish: this.distinguish,
-            footer: this.footer
+            footer: this.footer,
+            targets: this.targets,
         }
     }
 }
@@ -115,6 +188,21 @@ export interface CommentActionConfig extends RequiredRichContent, Footer {
      * Distinguish the comment after creation?
      * */
     distinguish?: boolean,
+
+    /**
+     * Specify where this comment should be made
+     *
+     * Valid values: 'self' | 'parent' | [reddit permalink]
+     *
+     * 'self' and 'parent' are special targets that are relative to the Activity being processed:
+     * * When Activity is Submission => 'parent' does nothing
+     * * When Activity is Comment
+     *    * 'self' => reply to Activity
+     *    * 'parent' => make a top-level comment in the Submission the Comment is in
+     *
+     * If target is not self/parent then CM assumes the value is a reddit permalink and will attempt to make a comment to that Activity
+     * */
+    targets?: ArbitraryActionTarget | ArbitraryActionTarget[]
 }
 
 export interface CommentActionOptions extends CommentActionConfig, ActionOptions {
@@ -124,5 +212,5 @@ export interface CommentActionOptions extends CommentActionConfig, ActionOptions
  * Reply to the Activity. For a submission the reply will be a top-level comment.
  * */
 export interface CommentActionJson extends CommentActionConfig, ActionJson {
-kind: 'comment'
+    kind: 'comment'
 }

src/Action/ContributorAction.ts

@@ -7,6 +7,7 @@ import Comment from "snoowrap/dist/objects/Comment";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTarget, ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class ContributorAction extends Action {
 
@@ -25,7 +26,7 @@ export class ContributorAction extends Action {
         this.actionType = action;
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
 
         const contributors = await this.resources.getSubredditContributors();

src/Action/DispatchAction.ts

@@ -8,6 +8,7 @@ import {activityDispatchConfigToDispatch, isSubmission, parseDurationValToDurati
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTarget, ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class DispatchAction extends Action {
     dispatchData: ActivityDispatchConfig;
@@ -39,7 +40,7 @@ export class DispatchAction extends Action {
         this.targets = !Array.isArray(target) ? [target] : target;
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         // ignore runtimeDryrun here because "real run" isn't causing any reddit api calls to happen
         // -- basically if bot is in dryrun this should still run since we want the "full effect" of the bot
         // BUT if the action explicitly sets 'dryRun: true' then do not dispatch as they probably don't want to it actually going (intention?)

src/Action/LockAction.ts

@@ -5,13 +5,14 @@ import {ActionProcessResult, RuleResult} from "../Common/interfaces";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class LockAction extends Action {
     getKind(): ActionTypes {
         return 'lock';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
         const touchedEntities = [];
         //snoowrap typing issue, thinks comments can't be locked

src/Action/MessageAction.ts

@@ -16,6 +16,7 @@ import {ErrorWithCause} from "pony-cause";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class MessageAction extends Action {
     content: string;
@@ -48,27 +49,30 @@ export class MessageAction extends Action {
         return 'message';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
-        const content = await this.resources.getContent(this.content);
-        const body = await renderContent(content, item, ruleResults, this.resources.userNotes);
 
-        const footer = await this.resources.generateFooter(item, this.footer);
+        const body = await this.renderContent(this.content, item, ruleResults, actionResults);
+        const titleTemplate = this.title ?? `Concerning your ${isSubmission(item) ? 'Submission' : 'Comment'}`;
+        const subject = await this.renderContent(titleTemplate, item, ruleResults, actionResults) as string;
+
+        const footer = await this.resources.renderFooter(item, this.footer);
 
         const renderedContent = `${body}${footer}`;
 
         let recipient = item.author.name;
         if(this.to !== undefined) {
+            const renderedTo = await this.renderContent(this.to, item, ruleResults, actionResults) as string;
             // parse to value
             try {
-                const entityData = parseRedditEntity(this.to, 'user');
+                const entityData = parseRedditEntity(renderedTo, 'user');
                 if(entityData.type === 'user') {
                     recipient = entityData.name;
                 } else {
                     recipient = `/r/${entityData.name}`;
                 }
             } catch (err: any) {
-                throw new ErrorWithCause(`'to' field for message was not in a valid format. See ${REDDIT_ENTITY_REGEX_URL} for valid examples`, {cause: err});
+                throw new ErrorWithCause(`'to' field for message was not in a valid format, given value after templating: ${renderedTo} -- See ${REDDIT_ENTITY_REGEX_URL} for valid examples`, {cause: err});
             }
             if(recipient.includes('/r/') && this.asSubreddit) {
                 throw new SimpleError(`Cannot send a message as a subreddit to another subreddit. Requested recipient: ${recipient}`);
@@ -80,7 +84,7 @@ export class MessageAction extends Action {
             text: renderedContent,
             // @ts-ignore
             fromSubreddit: this.asSubreddit ? await item.subreddit.fetch() : undefined,
-            subject: this.title || `Concerning your ${isSubmission(item) ? 'Submission' : 'Comment'}`,
+            subject: subject,
         };
 
         const msgPreview = `\r\n
@@ -122,7 +126,7 @@ export interface MessageActionConfig extends RequiredRichContent, Footer {
     asSubreddit: boolean
 
     /**
-     * Entity to send message to.
+     * Entity to send message to. It can be templated.
      *
      * If not present Message be will sent to the Author of the Activity being checked.
      *
@@ -134,8 +138,9 @@ export interface MessageActionConfig extends RequiredRichContent, Footer {
      *
      * **Note:** Reddit does not support sending a message AS a subreddit TO another subreddit
      *
-     * @pattern ^\s*(\/[ru]\/|[ru]\/)*(\w+)*\s*$
-     * @examples ["aUserName","u/aUserName","r/aSubreddit"]
+     * **Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`
+     *
+     * @examples ["aUserName","u/aUserName","r/aSubreddit", "r/{{item.subreddit}}"]
      * */
     to?: string
 

src/Action/ModNoteAction.ts

@@ -9,6 +9,7 @@ import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes, ModUserNoteLabel} from "../Common/Infrastructure/Atomic";
 import {ModNote} from "../Subreddit/ModNotes/ModNote";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 
 export class ModNoteAction extends Action {
@@ -39,13 +40,12 @@ export class ModNoteAction extends Action {
         }
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
 
         const modLabel = this.type !== undefined ? toModNoteLabel(this.type) : undefined;
 
-        const content = await this.resources.getContent(this.content, item.subreddit);
-        const renderedContent = await renderContent(content, item, ruleResults, this.resources.userNotes);
+        const renderedContent = await this.renderContent(this.content, item, ruleResults, actionResults);
         this.logger.verbose(`Note:\r\n(${this.type}) ${renderedContent}`);
 
         // TODO see what changes are made for bulk fetch of notes before implementing this

src/Action/RemoveAction.ts

@@ -4,13 +4,17 @@ import Snoowrap, {Comment, Submission} from "snoowrap";
 import {activityIsRemoved} from "../Utils/SnoowrapUtils";
 import {ActionProcessResult, RuleResult} from "../Common/interfaces";
 import dayjs from "dayjs";
-import {isSubmission} from "../util";
+import {isSubmission, truncateStringToLength} from "../util";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
+const truncate = truncateStringToLength(100);
 export class RemoveAction extends Action {
     spam: boolean;
+    note?: string;
+    reasonId?: string;
 
     getKind(): ActionTypes {
         return 'remove';
@@ -20,21 +24,54 @@ export class RemoveAction extends Action {
         super(options);
         const {
             spam = false,
+            note,
+            reasonId,
         } = options;
         this.spam = spam;
+        this.note = note;
+        this.reasonId = reasonId;
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
         const touchedEntities = [];
+        let removeSummary = [];
         // issue with snoowrap typings, doesn't think prop exists on Submission
         // @ts-ignore
         if (activityIsRemoved(item)) {
             this.logger.warn('It looks like this Item is already removed!');
         }
         if (this.spam) {
+            removeSummary.push('Marked as SPAM');
             this.logger.verbose('Marking as spam on removal');
         }
+        const renderedNote = await this.renderContent(this.note, item, ruleResults, actionResults);
+        let foundReasonId: string | undefined;
+        let foundReason: string | undefined;
+
+        if(this.reasonId !== undefined) {
+            const reason = await this.resources.getSubredditRemovalReasonById(this.reasonId);
+            if(reason === undefined) {
+                const reasonWarn = [`Could not find any Removal Reason with the ID ${this.reasonId}!`];
+                if(renderedNote === undefined) {
+                    reasonWarn.push('Cannot add any Removal Reason because note is also empty!');
+                } else {
+                    reasonWarn.push('Will add Removal Reason but only with note.');
+                }
+                this.logger.warn(reasonWarn.join(''));
+            } else {
+                foundReason = truncate(reason.title);
+                foundReasonId = reason.id;
+                removeSummary.push(`Reason: ${truncate(foundReason)} (${foundReasonId})`);
+            }
+        }
+
+        if(renderedNote !== undefined) {
+            removeSummary.push(`Note: ${truncate(renderedNote)}`);
+        }
+
+        this.logger.verbose(removeSummary.join(' | '));
+
         if (!dryRun) {
             // @ts-ignore
             await item.remove({spam: this.spam});
@@ -44,6 +81,18 @@ export class RemoveAction extends Action {
                 // @ts-ignore
                 item.removed = true;
             }
+
+            if(foundReasonId !== undefined || renderedNote !== undefined) {
+                await this.client.addRemovalReason(item, renderedNote, foundReasonId);
+                item.mod_reason_by = this.resources.botAccount as string;
+                if(renderedNote !== undefined) {
+                    item.removal_reason = renderedNote;
+                }
+                if(foundReason !== undefined) {
+                    item.mod_reason_title = foundReason;
+                }
+            }
+
             await this.resources.resetCacheForItem(item);
             touchedEntities.push(item);
         }
@@ -51,7 +100,8 @@ export class RemoveAction extends Action {
         return {
             dryRun,
             success: true,
-            touchedEntities
+            touchedEntities,
+            result: removeSummary.join(' | ')
         }
     }
 
@@ -66,7 +116,22 @@ export interface RemoveOptions extends Omit<RemoveActionConfig, 'authorIs' | 'it
 }
 
 export interface RemoveActionConfig extends ActionConfig {
+    /** (Optional) Mark Activity as spam */
     spam?: boolean
+    /**  (Optional) A mod-readable note added to the removal reason for this Activity. Can use Templating.
+     *
+     * This note (and removal reasons) are only visible on New Reddit
+     * */
+    note?: string
+    /** (Optional) The ID of the Removal Reason to use
+     *
+     * Removal reasons are only visible on New Reddit
+     *
+     * To find IDs for removal reasons check the "Removal Reasons" popup located in the CM dashboard config editor for your subreddit
+     *
+     * More info on Removal Reasons: https://mods.reddithelp.com/hc/en-us/articles/360010094892-Removal-Reasons
+     * */
+    reasonId?: string
 }
 
 /**

src/Action/ReportAction.ts

@@ -7,6 +7,7 @@ import {ActionProcessResult, RichContent, RuleResult} from "../Common/interfaces
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 // https://www.reddit.com/dev/api/oauth#POST_api_report
 // denotes 100 characters maximum
@@ -25,10 +26,9 @@ export class ReportAction extends Action {
         return 'report';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
-        const content = await this.resources.getContent(this.content, item.subreddit);
-        const renderedContent = await renderContent(content, item, ruleResults, this.resources.userNotes);
+        const renderedContent = (await this.renderContent(this.content, item, ruleResults, actionResults) as string);
         this.logger.verbose(`Contents:\r\n${renderedContent}`);
         const truncatedContent = reportTrunc(renderedContent);
         const touchedEntities = [];

src/Action/SubmissionAction.ts

@@ -0,0 +1,329 @@
+import Action, {ActionJson, ActionOptions} from "./index";
+import {Comment, SubmitLinkOptions, SubmitSelfPostOptions, VoteableContent} from "snoowrap";
+import Submission from "snoowrap/dist/objects/Submission";
+import {renderContent} from "../Utils/SnoowrapUtils";
+import {ActionProcessResult, Footer, RequiredRichContent, RichContent, RuleResult} from "../Common/interfaces";
+import {asComment, asSubmission, parseRedditEntity, parseRedditThingsFromLink, sleep, truncateStringToLength} from "../util";
+import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
+import {runCheckOptions} from "../Subreddit/Manager";
+import {ActionTarget, ActionTypes, ArbitraryActionTarget} from "../Common/Infrastructure/Atomic";
+import {CMError} from "../Utils/Errors";
+import {SnoowrapActivity} from "../Common/Infrastructure/Reddit";
+import Subreddit from "snoowrap/dist/objects/Subreddit";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
+
+export class SubmissionAction extends Action {
+    content?: string;
+    lock: boolean = false;
+    sticky: boolean = false;
+    distinguish: boolean = false;
+    spoiler: boolean = false;
+    nsfw: boolean = false;
+    flairId?: string
+    flairText?: string
+    url?: string
+    title: string
+    footer?: false | string;
+    targets: ('self' | string)[]
+
+    constructor(options: SubmissionActionOptions) {
+        super(options);
+        const {
+            content,
+            lock = false,
+            sticky = false,
+            spoiler = false,
+            distinguish = false,
+            nsfw = false,
+            flairText,
+            flairId,
+            footer,
+            url,
+            title,
+            targets = ['self']
+        } = options;
+        this.footer = footer;
+        this.content = content;
+        this.lock = lock;
+        this.sticky = sticky;
+        if(this.sticky) {
+            this.distinguish = sticky;
+        } else {
+            this.distinguish = distinguish;
+        }
+        this.spoiler = spoiler;
+        this.nsfw = nsfw;
+        this.flairText = flairText;
+        this.flairId = flairId;
+        this.url = url;
+        this.title = title;
+        if (!Array.isArray(targets)) {
+            this.targets = [targets];
+        } else {
+            this.targets = targets;
+        }
+    }
+
+    getKind(): ActionTypes {
+        return 'submission';
+    }
+
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+        const dryRun = this.getRuntimeAwareDryrun(options);
+
+        const title = await this.renderContent(this.title, item, ruleResults, actionResults) as string;
+        this.logger.verbose(`Title: ${title}`);
+
+        const url = await this.renderContent(this.url, item, ruleResults, actionResults);
+
+        this.logger.verbose(`URL: ${url !== undefined ? url : '[No URL]'}`);
+
+        const body = await this.renderContent(this.content, item, ruleResults, actionResults);
+
+        let renderedContent: string | undefined = undefined;
+        if(body !== undefined) {
+            const footer = await this.resources.renderFooter(item, this.footer);
+            renderedContent = `${body}${footer}`;
+            this.logger.verbose(`Contents:\r\n${renderedContent.length > 100 ? `\r\n${renderedContent}` : renderedContent}`);
+        } else {
+           this.logger.verbose(`Contents: [No Body]`);
+        }
+
+
+        let allErrors = true;
+        const targetResults: string[] = [];
+        const touchedEntities = [];
+
+        let submittedOnce = false;
+
+        for (const targetVal of this.targets) {
+
+            //
+            if(submittedOnce) {
+                // delay submissions by 3 seconds (on previous successful call)
+                // to try to spread out load
+                await sleep(3000);
+            }
+
+            let target: Subreddit = item.subreddit;
+            let targetIdentifier = targetVal;
+
+            if (targetVal !== 'self') {
+                const subredditVal = parseRedditEntity(targetVal);
+
+                try {
+                    target = await this.resources.getSubreddit(subredditVal.name);
+                    targetIdentifier = `[Subreddit ${target.display_name}]`;
+                } catch (err: any) {
+                    targetResults.push(`[${targetIdentifier}] error occurred while fetching subreddit: ${err.message}`);
+                    if(!err.logged) {
+                        this.logger.warn(new CMError(`[${targetIdentifier}] error occurred while fetching subreddit`, {cause: err}));
+                    }
+                    continue;
+                }
+            }
+
+            // TODO check if we can post in subreddit
+
+            let modifiers = [];
+            let post: Submission | undefined;
+            if (!dryRun) {
+                let opts: SubmitLinkOptions | SubmitSelfPostOptions;
+                let type: 'self' | 'link';
+                const genericOpts = {
+                    title,
+                    subredditName: target.display_name,
+                    nsfw: this.nsfw,
+                    spoiler: this.spoiler,
+                    flairId: this.flairId,
+                    flairText: this.flairText,
+                };
+                if(url !== undefined) {
+                    type = 'link';
+                    opts = {
+                        ...genericOpts,
+                        url,
+                    };
+                    if(renderedContent !== undefined) {
+                        // @ts-ignore
+                        linkOpts.text = renderedContent;
+                    }
+                } else {
+                    type = 'self';
+                    opts = {
+                        ...genericOpts,
+                        text: renderedContent,
+                    }
+                }
+                // @ts-ignore
+                post = await this.tryPost(type, target, opts);
+                await this.resources.setRecentSelf(post as Submission);
+                if(post !== undefined)  {
+                    touchedEntities.push(post);
+                }
+            }
+
+            if (this.lock) {
+                if (post !== undefined && !post.can_mod_post) {
+                    this.logger.warn(`[${targetIdentifier}] Cannot lock because bot is not a moderator`);
+                } else {
+                    modifiers.push('Locked');
+                    if (!dryRun && post !== undefined) {
+                        // snoopwrap typing issue, thinks comments can't be locked
+                        // @ts-ignore
+                        await post.lock();
+                    }
+                }
+            }
+
+            if (this.distinguish) {
+                if (post !== undefined && !post.can_mod_post) {
+                    this.logger.warn(`[${targetIdentifier}] Cannot Distinguish/Sticky because bot is not a moderator`);
+                } else {
+                    modifiers.push('Distinguished');
+                    if (this.sticky) {
+                        modifiers.push('Stickied');
+                    }
+                    if (!dryRun && post !== undefined) {
+                        // @ts-ignore
+                        await post.distinguish({sticky: this.sticky});
+                    }
+                }
+            }
+
+            const modifierStr = modifiers.length === 0 ? '' : ` == ${modifiers.join(' | ')} == =>`;
+            const targetSummary = `${targetIdentifier} ${modifierStr} created Submission ${dryRun ? 'DRYRUN' : (post as SnoowrapActivity).name}`;
+            // @ts-ignore
+            targetResults.push(targetSummary)
+            this.logger.verbose(targetSummary);
+            allErrors = false;
+        }
+
+
+        return {
+            dryRun,
+            success: !allErrors,
+            result: `${targetResults.join('\n')}${this.url !== undefined ? `\nURL: ${this.url}` : ''}${body !== undefined ? truncateStringToLength(100)(body) : ''}`,
+            touchedEntities,
+            data: {
+                body,
+                bodyShort: body !== undefined ? truncateStringToLength(100)(body) : '',
+                submissions: targetResults.map(x => `* ${x}`).join('\n')
+            }
+        };
+    }
+
+    // @ts-ignore
+    protected async tryPost(type: 'self' | 'link', target: Subreddit, data: SubmitLinkOptions | SubmitSelfPostOptions, maxAttempts = 2): Promise<Submission> {
+        let post: Submission | undefined;
+        let error: any;
+        for (let i = 0; i <= maxAttempts; i++) {
+            try {
+                if (type === 'self') {
+                    // @ts-ignore
+                    post = await target.submitSelfpost(data as SubmitSelfPostOptions);
+                } else {
+                    // @ts-ignore
+                    post = await target.submitLink(data as SubmitLinkOptions);
+                }
+                break;
+            } catch (e: any) {
+                if (e.message.includes('RATELIMIT')) {
+                    // Looks like you've been doing that a lot. Take a break for 5 seconds before trying again
+                    await sleep(5000);
+                    error = e;
+                } else {
+                    throw e;
+                }
+            }
+        }
+        if (error !== undefined) {
+            throw error;
+        }
+        // @ts-ignore
+        return post;
+    }
+
+    protected getSpecificPremise(): object {
+        return {
+            content: this.content,
+            lock: this.lock,
+            sticky: this.sticky,
+            spoiler: this.spoiler,
+            distinguish: this.distinguish,
+            nsfw: this.nsfw,
+            flairId: this.flairId,
+            flairText: this.flairText,
+            url: this.url,
+            text: this.content !== undefined ? truncateStringToLength(50)(this.content) : undefined,
+            footer: this.footer,
+            targets: this.targets,
+        }
+    }
+}
+
+export interface SubmissionActionConfig extends RichContent, Footer {
+    /**
+     * Lock the Submission after creation?
+     * */
+    lock?: boolean,
+    /**
+     * Sticky the Submission after creation?
+     * */
+    sticky?: boolean,
+
+    nsfw?: boolean
+
+    spoiler?: boolean
+
+    /**
+     * The title of this Submission.
+     *
+     * Templated the same as **content**
+     * */
+    title: string
+
+    /**
+     * If Submission should be a Link, the URL to use
+     *
+     * Templated the same as **content**
+     *
+     * PROTIP: To make a Link Submission pointing to the Activity being processed use `{{item.permalink}}` as the URL value
+     * */
+    url?: string
+
+    /**
+     * Flair template to apply to this Submission
+     * */
+    flairId?: string
+
+    /**
+     * Flair text to apply to this Submission
+     * */
+    flairText?: string
+
+    /**
+     * Distinguish as Mod after creation?
+     * */
+    distinguish?: boolean
+
+    /**
+     * Specify where this Submission should be made
+     *
+     * Valid values: 'self' | [subreddit]
+     *
+     * * 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed
+     * * [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos
+     * */
+    targets?: ('self' | string) | ('self' | string)[]
+}
+
+export interface SubmissionActionOptions extends SubmissionActionConfig, ActionOptions {
+}
+
+/**
+ * Reply to the Activity. For a submission the reply will be a top-level comment.
+ * */
+export interface SubmissionActionJson extends SubmissionActionConfig, ActionJson {
+    kind: 'submission'
+}

src/Action/SubmissionAction/FlairAction.ts

@@ -6,6 +6,7 @@ import Comment from 'snoowrap/dist/objects/Comment';
 import {RuleResultEntity} from "../../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../../Subreddit/Manager";
 import {ActionTypes} from "../../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../../Common/Entities/ActionResultEntity";
 
 export class FlairAction extends Action {
     text: string;
@@ -26,7 +27,7 @@ export class FlairAction extends Action {
         return 'flair';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
         let flairParts = [];
         if(this.text !== '') {

src/Action/UserFlairAction.ts

@@ -4,6 +4,7 @@ import {ActionProcessResult, RuleResult} from '../Common/interfaces';
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export class UserFlairAction extends Action {
   text?: string;
@@ -22,7 +23,7 @@ export class UserFlairAction extends Action {
     return 'userflair';
   }
 
-  async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+  async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
     const dryRun = this.getRuntimeAwareDryrun(options);
     let flairParts = [];
 

src/Action/UserNoteAction.ts

@@ -8,6 +8,7 @@ import {ActionProcessResult, RuleResult} from "../Common/interfaces";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {runCheckOptions} from "../Subreddit/Manager";
 import {ActionTypes, UserNoteType} from "../Common/Infrastructure/Atomic";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 
 export class UserNoteAction extends Action {
@@ -27,10 +28,9 @@ export class UserNoteAction extends Action {
         return 'usernote';
     }
 
-    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
+    async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
         const dryRun = this.getRuntimeAwareDryrun(options);
-        const content = await this.resources.getContent(this.content, item.subreddit);
-        const renderedContent = await renderContent(content, item, ruleResults, this.resources.userNotes);
+        const renderedContent = (await this.renderContent(this.content, item, ruleResults, actionResults) as string);
         this.logger.verbose(`Note:\r\n(${this.type}) ${renderedContent}`);
 
         if (!this.allowDuplicate) {

src/Action/index.ts

@@ -19,6 +19,8 @@ import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 import {FindOptionsWhere} from "typeorm/find-options/FindOptionsWhere";
 import {ActionTypes} from "../Common/Infrastructure/Atomic";
 import {RunnableBaseJson, RunnableBaseOptions, StructuredRunnableBase} from "../Common/Infrastructure/Runnable";
+import { SubredditResources } from "../Subreddit/SubredditResources";
+import {SnoowrapActivity} from "../Common/Infrastructure/Reddit";
 
 export abstract class Action extends RunnableBase {
     name?: string;
@@ -29,6 +31,8 @@ export abstract class Action extends RunnableBase {
     managerEmitter: EventEmitter;
     // actionEntity: ActionEntity | null = null;
     actionPremiseEntity: ActionPremise | null = null;
+    checkName: string;
+    subredditName: string;
 
     constructor(options: ActionOptions) {
         super(options);
@@ -40,6 +44,7 @@ export abstract class Action extends RunnableBase {
             subredditName,
             dryRun = false,
             emitter,
+            checkName,
         } = options;
 
         this.name = name;
@@ -48,6 +53,8 @@ export abstract class Action extends RunnableBase {
         this.client = client;
         this.logger = logger.child({labels: [`Action ${this.getActionUniqueName()}`]}, mergeArr);
         this.managerEmitter = emitter;
+        this.checkName = checkName;
+        this.subredditName = subredditName;
     }
 
     abstract getKind(): ActionTypes;
@@ -112,7 +119,7 @@ export abstract class Action extends RunnableBase {
         }
     }
 
-    async handle(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionResultEntity> {
+    async handle(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionResultEntity> {
         const {dryRun: runtimeDryrun} = options;
         const dryRun = runtimeDryrun || this.dryRun;
 
@@ -148,10 +155,11 @@ export abstract class Action extends RunnableBase {
                 actRes.runReason = runReason;
                 return actRes;
             }
-            const results = await this.process(item, ruleResults, options);
+            const results = await this.process(item, ruleResults, actionResults, options);
             actRes.success = results.success;
             actRes.dryRun = results.dryRun;
             actRes.result = results.result;
+            actRes.data = results.data;
             actRes.touchedEntities = results.touchedEntities ?? [];
 
             return actRes;
@@ -166,20 +174,31 @@ export abstract class Action extends RunnableBase {
         }
     }
 
-    abstract process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult>;
+    abstract process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult>;
 
     getRuntimeAwareDryrun(options: runCheckOptions): boolean {
         const {dryRun: runtimeDryrun} = options;
         return runtimeDryrun || this.dryRun;
     }
+
+    async renderContent(template: string | undefined, item: SnoowrapActivity, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[]): Promise<string | undefined> {
+        if(template === undefined) {
+            return undefined;
+        }
+        return await this.resources.renderContent(template, item, ruleResults, actionResults, {manager: this.subredditName, check: this.checkName});
+    }
 }
 
-export interface ActionOptions extends Omit<ActionConfig, 'authorIs' | 'itemIs'>, RunnableBaseOptions {
-    //logger: Logger;
-    subredditName: string;
-    //resources: SubredditResources;
+export interface ActionRuntimeOptions {
+    checkName: string
+    subredditName: string
     client: ExtendedSnoowrap;
-    emitter: EventEmitter
+    emitter: EventEmitter;
+    resources: SubredditResources;
+    logger: Logger;
+}
+
+export interface ActionOptions extends Omit<ActionConfig, 'authorIs' | 'itemIs'>, RunnableBaseOptions, ActionRuntimeOptions {
 }
 
 export interface ActionConfig extends RunnableBaseJson {

src/App.ts

@@ -4,24 +4,27 @@ import {getLogger} from "./Utils/loggerFactory";
 import {DatabaseMigrationOptions, OperatorConfig, OperatorConfigWithFileContext, OperatorFileConfig} from "./Common/interfaces";
 import Bot from "./Bot";
 import LoggedError from "./Utils/LoggedError";
-import {mergeArr, sleep} from "./util";
-import {copyFile} from "fs/promises";
+import {generateRandomName, mergeArr, sleep} from "./util";
+import {copyFile, open} from "fs/promises";
 import {constants} from "fs";
-import {Connection} from "typeorm";
+import {Connection, DataSource, Repository} from "typeorm";
 import {ErrorWithCause} from "pony-cause";
 import {MigrationService} from "./Common/MigrationService";
 import {Invokee} from "./Common/Infrastructure/Atomic";
 import {DatabaseConfig} from "./Common/Infrastructure/Database";
+import {InviteData} from "./Web/Common/interfaces";
+import {BotInvite} from "./Common/Entities/BotInvite";
 
 export class App {
 
     bots: Bot[] = [];
     logger: Logger;
     dbLogger: Logger;
-    database: Connection
+    database: DataSource
     startedAt: Dayjs = dayjs();
     ranMigrations: boolean = false;
     migrationBlocker?: string;
+    friendly?: string;
 
     config: OperatorConfig;
 
@@ -30,6 +33,7 @@ export class App {
     fileConfig: OperatorFileConfig;
 
     migrationService: MigrationService;
+    inviteRepo: Repository<BotInvite>;
 
     constructor(config: OperatorConfigWithFileContext) {
         const {
@@ -49,6 +53,8 @@ export class App {
         this.logger = getLogger(config.logging);
         this.dbLogger = this.logger.child({labels: ['Database']}, mergeArr);
         this.database = database;
+        this.inviteRepo = this.database.getRepository(BotInvite);
+        this.friendly = this.config.api.friendly;
 
         this.logger.info(`Operators: ${name.length === 0 ? 'None Specified' : name.join(', ')}`)
 
@@ -114,6 +120,8 @@ export class App {
             return;
         }
 
+        await this.checkFriendlyName();
+
         if(this.bots.length > 0) {
             this.logger.info('Bots already exist, will stop and destroy these before building new ones.');
             await this.destroy(causedBy);
@@ -161,4 +169,54 @@ export class App {
             await b.destroy(causedBy);
         }
     }
+
+    async checkFriendlyName() {
+        if(this.friendly === undefined) {
+            let randFriendly: string = generateRandomName();
+            this.logger.verbose(`No friendly name set for Server. Generated: ${randFriendly}`);
+
+            const exists = async (name: string) => {
+                const existing = await this.inviteRepo.findBy({instance: name});
+                return existing.length > 0;
+            }
+            while (await exists(randFriendly)) {
+                let oldFriendly = randFriendly;
+                randFriendly = generateRandomName();
+                this.logger.verbose(`${oldFriendly} already exists! Generated: ${randFriendly}`);
+            }
+
+            this.friendly = randFriendly;
+            this.fileConfig.document.setFriendlyName(this.friendly);
+
+            const handle = await open(this.fileConfig.document.location as string, 'w');
+            await handle.writeFile(this.fileConfig.document.toString());
+            await handle.close();
+            this.logger.verbose(`Wrote ${randFriendly} as friendly server name to config.`);
+        }
+    }
+
+    async getInviteById(id: string): Promise<BotInvite | undefined> {
+        const invite = await this.inviteRepo.findOne({where: {id, instance: this.friendly}});
+        if(invite === null) {
+            return undefined;
+        }
+        return invite;
+    }
+
+    async getInviteIds(): Promise<string[]> {
+        if(!this.ranMigrations) {
+            // not ready!
+            return [];
+        }
+        const invites = await this.inviteRepo.findBy({instance: this.friendly});
+        return invites.map(x => x.id);
+    }
+
+    async addInvite(data: InviteData): Promise<InviteData> {
+        return await this.inviteRepo.save(new BotInvite(data));
+    }
+
+    async deleteInvite(id: string): Promise<void> {
+        await this.inviteRepo.delete({ id });
+    }
 }

src/Bot/index.ts

@@ -13,13 +13,13 @@ import {
     USER
 } from "../Common/interfaces";
 import {
-    createRetryHandler, difference,
+    createRetryHandler, symmetricalDifference,
     formatNumber, getExceptionMessage, getUserAgent,
     mergeArr,
     parseBool,
     parseDuration, parseMatchMessage, parseRedditEntity,
     parseSubredditName, partition, RetryOptions,
-    sleep
+    sleep, intersect
 } from "../util";
 import {Manager} from "../Subreddit/Manager";
 import {ExtendedSnoowrap, ProxiedSnoowrap} from "../Utils/SnoowrapClients";
@@ -43,8 +43,12 @@ import {FilterCriteriaDefaults} from "../Common/Infrastructure/Filters/FilterSha
 import {snooLogWrapper} from "../Utils/loggerFactory";
 import {InfluxClient} from "../Common/Influx/InfluxClient";
 import {Point} from "@influxdata/influxdb-client";
+import {BotInstanceFunctions, NormalizedManagerResponse} from "../Web/Common/interfaces";
+import {AuthorEntity} from "../Common/Entities/AuthorEntity";
+import {Guest, GuestEntityData} from "../Common/Entities/Guest/GuestInterfaces";
+import {guestEntitiesToAll, guestEntityToApiGuest} from "../Common/Entities/Guest/GuestEntity";
 
-class Bot {
+class Bot implements BotInstanceFunctions {
 
     client!: ExtendedSnoowrap;
     logger!: Logger;
@@ -99,6 +103,8 @@ class Bot {
     database: DataSource
     invokeeRepo: Repository<InvokeeType>;
     runTypeRepo: Repository<RunStateType>;
+    managerRepo: Repository<ManagerEntity>;
+    authorRepo: Repository<AuthorEntity>;
     botEntity!: BotEntity
 
     getBotName = () => {
@@ -160,6 +166,8 @@ class Bot {
         this.database = database;
         this.invokeeRepo = this.database.getRepository(InvokeeType);
         this.runTypeRepo = this.database.getRepository(RunStateType);
+        this.managerRepo = this.database.getRepository(ManagerEntity);
+        this.authorRepo = this.database.getRepository(AuthorEntity);
         this.config = config;
         this.dryRun = parseBool(dryRun) === true ? true : undefined;
         this.softLimit = softLimit;
@@ -378,7 +386,7 @@ class Bot {
     async testClient(initial = true) {
         try {
             // @ts-ignore
-            const user = this.client.getMe().fetch();
+            const user = await this.client.getMe().fetch();
             this.logger.info('Test API call successful');
             return user;
         } catch (err: any) {
@@ -456,7 +464,7 @@ class Bot {
                     return acc;
                 }
             }, []);
-            const notMatched = difference(normalizedOverrideNames, subsToRunNames);
+            const notMatched = symmetricalDifference(normalizedOverrideNames, subsToRunNames);
             if(notMatched.length > 0) {
                 this.logger.warn(`There are overrides defined for subreddits the bot is not running. Check your spelling! Overrides not matched: ${notMatched.join(', ')}`);
             }
@@ -672,15 +680,15 @@ class Bot {
             databaseConfig: {
                 retention = undefined
             } = {},
+            wikiConfig = this.wikiLocation,
         } = override || {};
 
-        const managerRepo = this.database.getRepository(ManagerEntity);
         const subRepo = this.database.getRepository(SubredditEntity)
         let subreddit = await subRepo.findOne({where: {id: sub.name}});
         if(subreddit === null) {
             subreddit = await subRepo.save(new SubredditEntity({id: sub.name, name: sub.display_name}))
         }
-        let managerEntity = await managerRepo.findOne({
+        let managerEntity = await this.managerRepo.findOne({
             where: {
                 bot: {
                     id: this.botEntity.id
@@ -689,12 +697,15 @@ class Bot {
                     id: subreddit.id
                 }
             },
+            relations: {
+                guests: true
+            }
         });
         if(managerEntity === undefined || managerEntity === null) {
             const invokee = await this.invokeeRepo.findOneBy({name: SYSTEM}) as InvokeeType;
             const runType = await this.runTypeRepo.findOneBy({name: STOPPED}) as RunStateType;
 
-            managerEntity = await managerRepo.save(new ManagerEntity({
+            managerEntity = await this.managerRepo.save(new ManagerEntity({
                 name: sub.display_name,
                 bot: this.botEntity,
                 subreddit: subreddit as SubredditEntity,
@@ -707,7 +718,7 @@ class Bot {
         const manager = new Manager(sub, this.client, this.logger, this.cacheManager, {
             dryRun: this.dryRun,
             sharedStreams: this.sharedStreams,
-            wikiLocation: this.wikiLocation,
+            wikiLocation: wikiConfig,
             botName: this.botName as string,
             maxWorkers: this.maxWorkers,
             filterCriteriaDefaults: this.filterCriteriaDefaults,
@@ -814,10 +825,15 @@ class Bot {
     async healthLoop() {
         while (this.running) {
             await sleep(5000);
-            await this.apiHealthCheck();
+            const time = dayjs().valueOf()
+            await this.apiHealthCheck(time);
+            await this.guestModCleanup();
             if (!this.running) {
                 break;
             }
+            for(const m of this.subManagers) {
+                await m.writeHealthMetrics(time);
+            }
             const now = dayjs();
             if (now.isSameOrAfter(this.nextNannyCheck)) {
                 try {
@@ -857,7 +873,7 @@ class Bot {
         return`API Usage Rolling Avg: ${formatNumber(this.apiRollingAvg)}/s | Est Depletion: ${depletion} (${formatNumber(this.depletedInSecs, {toFixed: 0})} seconds)`;
     }
 
-    async apiHealthCheck() {
+    async apiHealthCheck(time?: number) {
 
         const rollingSample = this.apiSample.slice(0, 7)
         rollingSample.unshift(this.client.ratelimitRemaining);
@@ -887,6 +903,10 @@ class Bot {
                 .intField('remaining', this.client.ratelimitRemaining)
                 .stringField('nannyMod', this.nannyMode ?? 'none');
 
+            if(time !== undefined) {
+                apiMeasure.timestamp(time);
+            }
+
             if(this.apiSample.length > 1) {
                 const curr = this.apiSample[0];
                 const last = this.apiSample[1];
@@ -902,6 +922,19 @@ class Bot {
 
     }
 
+    async guestModCleanup() {
+        const now = dayjs();
+
+        for(const m of this.subManagers) {
+            const expiredGuests = m.managerEntity.getGuests().filter(x => x.expiresAt.isBefore(now));
+            if(expiredGuests.length > 0) {
+                m.managerEntity.removeGuestById(expiredGuests.map(x => x.id));
+                m.logger.info(`Removed expired Guest Mods: ${expiredGuests.map(x => x.author.name).join(', ')}`);
+                await this.managerRepo.save(m.managerEntity);
+            }
+        }
+    }
+
     async retentionCleanup() {
         const now = dayjs();
         if(now.isSameOrAfter(this.nextRetentionCheck)) {
@@ -1092,6 +1125,117 @@ class Bot {
             throw err;
         }
     }
+
+    getManagerNames(): string[] {
+        return this.subManagers.map(x => x.displayLabel);
+    }
+
+    getSubreddits(normalized = true): string[] {
+        return normalized ? this.subManagers.map(x => parseRedditEntity(x.subreddit.display_name).name) : this.subManagers.map(x => x.subreddit.display_name);
+    }
+
+    getGuestManagers(user: string): NormalizedManagerResponse[] {
+        return this.subManagers.filter(x => x.managerEntity.getGuests().map(y => y.author.name).includes(user)).map(x => x.toNormalizedManager());
+    }
+
+    getGuestSubreddits(user: string): string[] {
+        return this.getGuestManagers(user).map(x => x.subredditNormal);
+    }
+
+    getAccessibleSubreddits(user: string, subreddits: string[] = []): string[] {
+        const normalSubs = subreddits.map(x => parseRedditEntity(x).name);
+        const moderatedSubs = intersect(normalSubs, this.getSubreddits());
+        const guestSubs = this.getGuestSubreddits(user);
+        return Array.from(new Set([...guestSubs, ...moderatedSubs]));
+    }
+
+    canUserAccessBot(user: string, subreddits: string[] = []) {
+        return this.getAccessibleSubreddits(user, subreddits).length > 0;
+    }
+
+    canUserAccessSubreddit(subreddit: string, user: string, subreddits: string[] = []): boolean {
+        return this.getAccessibleSubreddits(user, subreddits).includes(parseRedditEntity(subreddit).name);
+    }
+
+    async addGuest(userVal: string | string[], expiresAt: Dayjs, managerVal?: string | string[]) {
+        let managerNames: string[];
+        if(typeof managerVal === 'string') {
+            managerNames = [managerVal];
+        } else if(Array.isArray(managerVal)) {
+            managerNames = managerVal;
+        } else {
+            managerNames = this.subManagers.map(x => x.subreddit.display_name);
+        }
+
+        const cleanSubredditNames = managerNames.map(x => parseRedditEntity(x).name);
+        const userNames = typeof userVal === 'string' ? [userVal] : userVal;
+        const cleanUsers = userNames.map(x => parseRedditEntity(x.trim(), 'user').name);
+
+        const users: AuthorEntity[] = [];
+
+        for(const uName of cleanUsers) {
+            let user = await this.authorRepo.findOne({
+                where: {
+                    name: uName,
+                }
+            });
+
+            if(user === null) {
+                users.push(await this.authorRepo.save(new AuthorEntity({name: uName})));
+            } else {
+                users.push(user);
+            }
+        }
+
+        const newGuestData = users.map(x => ({author: x, expiresAt})) as GuestEntityData[];
+
+        let newGuests = new Map<string, Guest[]>();
+        const updatedManagerEntities: ManagerEntity[] = [];
+        for(const m of this.subManagers) {
+            if(!cleanSubredditNames.includes(m.subreddit.display_name)) {
+                continue;
+            }
+            const filteredGuests = m.managerEntity.addGuest(newGuestData);
+            updatedManagerEntities.push(m.managerEntity);
+            newGuests.set(m.displayLabel, filteredGuests.map(x => guestEntityToApiGuest(x)));
+            m.logger.info(`Added ${cleanUsers.join(', ')} as Guest`);
+        }
+
+        await this.managerRepo.save(updatedManagerEntities);
+
+        return newGuests;
+    }
+
+    async removeGuest(userVal: string | string[], managerVal?: string | string[]) {
+        let managerNames: string[];
+        if(typeof managerVal === 'string') {
+            managerNames = [managerVal];
+        } else if(Array.isArray(managerVal)) {
+            managerNames = managerVal;
+        } else {
+            managerNames = this.subManagers.map(x => x.subreddit.display_name);
+        }
+
+        const cleanSubredditNames = managerNames.map(x => parseRedditEntity(x).name);
+        const userNames = typeof userVal === 'string' ? [userVal] : userVal;
+        const cleanUsers = userNames.map(x => parseRedditEntity(x.trim(), 'user').name);
+
+        let newGuests = new Map<string, Guest[]>();
+        const updatedManagerEntities: ManagerEntity[] = [];
+        for(const m of this.subManagers) {
+            if(!cleanSubredditNames.includes(m.subreddit.display_name)) {
+                continue;
+            }
+            const filteredGuests = m.managerEntity.removeGuestByUser(cleanUsers);
+            updatedManagerEntities.push(m.managerEntity);
+            newGuests.set(m.displayLabel, filteredGuests.map(x => guestEntityToApiGuest(x)));
+            m.logger.info(`Removed ${cleanUsers.join(', ')} from Guests`);
+        }
+
+        await this.managerRepo.save(updatedManagerEntities);
+
+        return newGuests;
+    }
 }
 
 export default Bot;

src/Check/index.ts

@@ -222,7 +222,14 @@ export abstract class Check extends RunnableBase implements Omit<ICheck, 'postTr
                     this.actions.push(actionFactory({
                         ...aj,
                         dryRun: this.dryRun || aj.dryRun
-                    }, this.logger, subredditName, this.resources, this.client, this.emitter));
+                    }, {
+                        logger: this.logger,
+                        subredditName,
+                        resources: this.resources,
+                        client: this.client,
+                        emitter: this.emitter,
+                        checkName: this.name
+                    }));
                     // @ts-ignore
                     a.logger = this.logger;
                 } else {
@@ -564,7 +571,7 @@ export abstract class Check extends RunnableBase implements Omit<ICheck, 'postTr
             const dr = dryRun || this.dryRun;
             this.logger.debug(`${dr ? 'DRYRUN - ' : ''}Running Actions`);
             for (const a of this.actions) {
-                const res = await a.handle(item, ruleResults, options);
+                const res = await a.handle(item, ruleResults, runActions, options);
                 runActions.push(res);
             }
             this.logger.info(`${dr ? 'DRYRUN - ' : ''}Ran Actions: ${runActions.map(x => x.premise.getFriendlyIdentifier()).join(' | ')}`);

src/Common/Config/Operator/index.ts

@@ -1,11 +1,14 @@
 import YamlConfigDocument from "../YamlConfigDocument";
 import JsonConfigDocument from "../JsonConfigDocument";
 import {YAMLMap, YAMLSeq, Pair, Scalar} from "yaml";
-import {BotInstanceJsonConfig, OperatorJsonConfig} from "../../interfaces";
+import {BotInstanceJsonConfig, OperatorJsonConfig, WebCredentials} from "../../interfaces";
 import {assign} from 'comment-json';
 
 export interface OperatorConfigDocumentInterface {
     addBot(botData: BotInstanceJsonConfig): void;
+    setFriendlyName(name: string): void;
+    setWebCredentials(data: Required<WebCredentials>): void;
+    setOperator(name: string): void;
     toJS(): OperatorJsonConfig;
 }
 
@@ -29,6 +32,18 @@ export class YamlOperatorConfigDocument extends YamlConfigDocument implements Op
         }
     }
 
+    setFriendlyName(name: string) {
+        this.parsed.addIn(['api', 'friendly'], name);
+    }
+
+    setWebCredentials(data: Required<WebCredentials>) {
+        this.parsed.addIn(['web', 'credentials'], data);
+    }
+
+    setOperator(name: string) {
+        this.parsed.addIn(['operator', 'name'], name);
+    }
+
     toJS(): OperatorJsonConfig  {
         return super.toJS();
     }
@@ -68,6 +83,23 @@ export class JsonOperatorConfigDocument extends JsonConfigDocument implements Op
         }
     }
 
+    setFriendlyName(name: string) {
+        const api = this.parsed.api || {};
+        this.parsed.api = {...api, friendly: name};
+    }
+
+    setWebCredentials(data: Required<WebCredentials>) {
+        const {
+            web = {},
+        } = this.parsed;
+
+        this.parsed.web = {...web, credentials: data};
+    }
+
+    setOperator(name: string) {
+        this.parsed.operator = { name };
+    }
+
     toJS(): OperatorJsonConfig {
         return super.toJS();
     }

src/Common/Entities/ActionResultEntity.ts

@@ -56,6 +56,11 @@ export class ActionResultEntity extends TimeAwareRandomBaseEntity {
     @JoinColumn({name: 'premiseId'})
     premise!: ActionPremise;
 
+    /**
+     * Ephemeral -- only added during actual run time and used for action templating. Is not available after loading from DB.
+     * */
+    data?: any;
+
     touchedEntities: (Submission | Comment | RedditUser | string)[] = []
 
     set itemIs(data: ActivityStateFilterResult | IFilterResult<TypedActivityState> | undefined) {

src/Common/Entities/AuthorEntity.ts

@@ -12,4 +12,10 @@ export class AuthorEntity {
 
     @OneToMany(type => Activity, act => act.author)
     activities!: Activity[]
+
+    constructor(data?: any) {
+        if(data !== undefined) {
+            this.name = data.name;
+        }
+    }
 }

src/Common/Entities/Bot.ts

@@ -1,13 +1,68 @@
 import {Entity, Column, PrimaryColumn, OneToMany, PrimaryGeneratedColumn} from "typeorm";
 import {ManagerEntity} from "./ManagerEntity";
 import {RandomIdBaseEntity} from "./Base/RandomIdBaseEntity";
+import {BotGuestEntity, ManagerGuestEntity} from "./Guest/GuestEntity";
+import {Guest, GuestEntityData, HasGuests} from "./Guest/GuestInterfaces";
+import {SubredditInvite} from "./SubredditInvite";
 
 @Entity()
-export class Bot extends RandomIdBaseEntity {
+export class Bot extends RandomIdBaseEntity implements HasGuests {
 
     @Column("varchar", {length: 200})
     name!: string;
 
     @OneToMany(type => ManagerEntity, obj => obj.bot)
     managers!: Promise<ManagerEntity[]>
+
+    @OneToMany(type => BotGuestEntity, obj => obj.guestOf, {eager: true, cascade: ['insert', 'remove', 'update']})
+    guests!: BotGuestEntity[]
+
+    @OneToMany(type => SubredditInvite, obj => obj.bot, {eager: true, cascade: ['insert', 'remove', 'update']})
+    subredditInvites!: SubredditInvite[]
+
+    getGuests() {
+        const g = this.guests;
+        if (g === undefined) {
+            return [];
+        }
+        //return g.map(x => ({id: x.id, name: x.author.name, expiresAt: x.expiresAt})) as Guest[];
+        return g;
+    }
+
+    addGuest(val: GuestEntityData | GuestEntityData[]) {
+        const reqGuests = Array.isArray(val) ? val : [val];
+        const guests = this.guests;
+        for (const g of reqGuests) {
+            const existing = guests.find(x => x.author.name.toLowerCase() === g.author.name.toLowerCase());
+            if (existing !== undefined) {
+                // update existing guest expiresAt
+                existing.expiresAt = g.expiresAt;
+            } else {
+                guests.push(new BotGuestEntity({...g, guestOf: this}));
+            }
+        }
+        this.guests = guests
+        return guests;
+    }
+
+    removeGuestById(val: string | string[]) {
+        const reqGuests = Array.isArray(val) ? val : [val];
+        const guests = this.guests;
+        const filteredGuests = guests.filter(x => reqGuests.includes(x.id));
+        this.guests = filteredGuests;
+        return filteredGuests;
+    }
+
+    removeGuestByUser(val: string | string[]) {
+        const reqGuests = (Array.isArray(val) ? val : [val]).map(x => x.trim().toLowerCase());
+        const guests = this.guests;
+        const filteredGuests = guests.filter(x => reqGuests.includes(x.author.name.toLowerCase()));
+        this.guests =filteredGuests;
+        return filteredGuests;
+    }
+
+    removeGuests() {
+        this.guests = []
+        return [];
+    }
 }

src/Common/Entities/BotInvite.ts

@@ -1,13 +1,12 @@
 import {Column, Entity, PrimaryColumn} from "typeorm";
-import {TimeAwareBaseEntity} from "../Entities/Base/TimeAwareBaseEntity";
+import {TimeAwareBaseEntity} from "./Base/TimeAwareBaseEntity";
 import {InviteData} from "../../Web/Common/interfaces";
 import dayjs, {Dayjs} from "dayjs";
+import {TimeAwareRandomBaseEntity} from "./Base/TimeAwareRandomBaseEntity";
+import {parseRedditEntity} from "../../util";
 
-@Entity()
-export class Invite extends TimeAwareBaseEntity implements InviteData {
-
-    @PrimaryColumn('varchar', {length: 255})
-    id!: string
+@Entity({name: 'BotInvite'})
+export class BotInvite extends TimeAwareRandomBaseEntity implements InviteData {
 
     @Column("varchar", {length: 50})
     clientId!: string;
@@ -30,6 +29,12 @@ export class Invite extends TimeAwareBaseEntity implements InviteData {
     @Column()
     overwrite?: boolean;
 
+    @Column("simple-json")
+    guests?: string[]
+
+    @Column("text")
+    initialConfig?: string
+
     @Column("simple-json", {nullable: true})
     subreddits?: string[];
 
@@ -51,10 +56,9 @@ export class Invite extends TimeAwareBaseEntity implements InviteData {
         }
     }
 
-    constructor(data?: InviteData & { id: string, expiresIn?: number }) {
+    constructor(data?: InviteData) {
         super();
         if (data !== undefined) {
-            this.id = data.id;
             this.permissions = data.permissions;
             this.subreddits = data.subreddits;
             this.instance = data.instance;
@@ -63,9 +67,16 @@ export class Invite extends TimeAwareBaseEntity implements InviteData {
             this.redirectUri = data.redirectUri;
             this.creator = data.creator;
             this.overwrite = data.overwrite;
+            this.initialConfig = data.initialConfig;
+            if(data.guests !== undefined && data.guests !== null && data.guests.length > 0) {
+                const cleanGuests = data.guests.filter(x => x !== '').map(x => parseRedditEntity(x, 'user').name);
+                if(cleanGuests.length > 0) {
+                    this.guests = cleanGuests;
+                }
+            }
 
-            if (data.expiresIn !== undefined && data.expiresIn !== 0) {
-                this.expiresAt = dayjs().add(data.expiresIn, 'seconds');
+            if (data.expiresAt !== undefined && data.expiresAt !== 0) {
+                this.expiresAt = dayjs(data.expiresAt);
             }
         }
     }

src/Common/Entities/Guest/GuestEntity.ts

@@ -0,0 +1,119 @@
+import {ChildEntity, Column, Entity, JoinColumn, ManyToOne, TableInheritance} from "typeorm";
+import {AuthorEntity} from "../AuthorEntity";
+import { ManagerEntity } from "../ManagerEntity";
+import { Bot } from "../Bot";
+import {TimeAwareRandomBaseEntity} from "../Base/TimeAwareRandomBaseEntity";
+import dayjs, {Dayjs} from "dayjs";
+import {Guest, GuestAll, GuestEntityData} from "./GuestInterfaces";
+
+export interface GuestOptions<T extends ManagerEntity | Bot> extends GuestEntityData {
+    guestOf: T
+}
+
+@Entity({name: 'Guests'})
+@TableInheritance({ column: { type: "varchar", name: "type" } })
+export abstract class GuestEntity<T extends ManagerEntity | Bot> extends TimeAwareRandomBaseEntity {
+
+    @ManyToOne(type => AuthorEntity, undefined, {cascade: ['insert'], eager: true})
+    @JoinColumn({name: 'authorName'})
+    author!: AuthorEntity;
+
+    @Column({ name: 'expiresAt', nullable: true })
+    _expiresAt?: Date = new Date();
+
+    public get expiresAt(): Dayjs {
+        return dayjs(this._expiresAt);
+    }
+
+    public set expiresAt(d: Dayjs | undefined) {
+        if(d === undefined) {
+            this._expiresAt = d;
+        } else {
+            this._expiresAt = d.utc().toDate();
+        }
+    }
+
+    expiresAtTimestamp(): number | undefined {
+        if(this._expiresAt !== undefined) {
+           return this.expiresAt.valueOf();
+        }
+        return undefined;
+    }
+
+    protected constructor(data?: GuestOptions<T>) {
+        super();
+        if(data !== undefined) {
+            this.author = data.author;
+            this.expiresAt = data.expiresAt;
+        }
+    }
+}
+
+@ChildEntity('manager')
+export class ManagerGuestEntity extends GuestEntity<ManagerEntity> {
+
+    type: string = 'manager';
+
+    @ManyToOne(type => ManagerEntity, act => act.guests, {nullable: false, orphanedRowAction: 'delete'})
+    @JoinColumn({name: 'guestOfId', referencedColumnName: 'id'})
+    guestOf!: ManagerEntity
+
+    constructor(data?: GuestOptions<ManagerEntity>) {
+        super(data);
+        if(data !== undefined) {
+            this.guestOf = data.guestOf;
+        }
+    }
+}
+
+@ChildEntity('bot')
+export class BotGuestEntity extends GuestEntity<Bot> {
+
+    type: string = 'bot';
+
+    @ManyToOne(type => Bot, act => act.guests, {nullable: false, orphanedRowAction: 'delete'})
+    @JoinColumn({name: 'guestOfId', referencedColumnName: 'id'})
+    guestOf!: Bot
+
+    constructor(data?: GuestOptions<Bot>) {
+        super(data);
+        if(data !== undefined) {
+            this.guestOf = data.guestOf;
+            this.author = data.author;
+        }
+    }
+}
+
+export const guestEntityToApiGuest = (val: GuestEntity<any>): Guest => {
+    return {
+        id: val.id,
+        name: val.author.name,
+        expiresAt: val.expiresAtTimestamp(),
+    }
+}
+
+interface ContextualGuest extends Guest {
+    subreddit: string
+}
+
+export const guestEntitiesToAll = (val: Map<string, Guest[]>): GuestAll[] => {
+    const contextualGuests: ContextualGuest[] = Array.from(val.entries()).map(([sub, guests]) => guests.map(y => ({...y, subreddit: sub} as ContextualGuest))).flat(3);
+
+    const userMap = contextualGuests.reduce((acc, curr) => {
+        let u: GuestAll | undefined = acc.get(curr.name);
+        if (u === undefined) {
+            u = {name: curr.name, expiresAt: curr.expiresAt, subreddits: [curr.subreddit]};
+        } else {
+            if (!u.subreddits.includes(curr.subreddit)) {
+                u.subreddits.push(curr.subreddit);
+            }
+            if ((u.expiresAt === undefined && curr.expiresAt !== undefined) || (u.expiresAt !== undefined && curr.expiresAt !== undefined && curr.expiresAt < u.expiresAt)) {
+                u.expiresAt = curr.expiresAt;
+            }
+        }
+        acc.set(curr.name, u);
+        return acc;
+    }, new Map<string, GuestAll>());
+
+    return Array.from(userMap.values());
+}

src/Common/Entities/Guest/GuestInterfaces.ts

@@ -0,0 +1,28 @@
+import { Dayjs } from "dayjs"
+import {AuthorEntity} from "../AuthorEntity";
+
+export interface Guest {
+    id: string
+    name: string
+    expiresAt?: number
+}
+
+export interface GuestAll {
+    name: string
+    expiresAt?: number
+    subreddits: string[]
+}
+
+
+export interface GuestEntityData {
+    expiresAt?: Dayjs
+    author: AuthorEntity
+}
+
+export interface HasGuests {
+    getGuests: () => GuestEntityData[]
+    addGuest: (val: GuestEntityData | GuestEntityData[]) => GuestEntityData[]
+    removeGuestById: (val: string | string[]) => GuestEntityData[]
+    removeGuestByUser: (val: string | string[]) => GuestEntityData[]
+    removeGuests: () => GuestEntityData[]
+}

src/Common/Entities/ManagerEntity.ts

@@ -15,12 +15,14 @@ import {RunEntity} from "./RunEntity";
 import {Bot} from "./Bot";
 import {RandomIdBaseEntity} from "./Base/RandomIdBaseEntity";
 import {ManagerRunState} from "./EntityRunState/ManagerRunState";
-import { QueueRunState } from "./EntityRunState/QueueRunState";
+import {QueueRunState} from "./EntityRunState/QueueRunState";
 import {EventsRunState} from "./EntityRunState/EventsRunState";
 import {RulePremise} from "./RulePremise";
 import {ActionPremise} from "./ActionPremise";
-import { RunningStateTypes } from "../../Subreddit/Manager";
+import {RunningStateTypes} from "../../Subreddit/Manager";
 import {EntityRunState} from "./EntityRunState/EntityRunState";
+import {GuestEntity, ManagerGuestEntity} from "./Guest/GuestEntity";
+import {Guest, GuestEntityData, HasGuests} from "./Guest/GuestInterfaces";
 
 export interface ManagerEntityOptions {
     name: string
@@ -36,7 +38,7 @@ export type RunningStateEntities = {
 };
 
 @Entity({name: 'Manager'})
-export class ManagerEntity extends RandomIdBaseEntity implements RunningStateEntities {
+export class ManagerEntity extends RandomIdBaseEntity implements RunningStateEntities, HasGuests {
 
     @Column("varchar", {length: 200})
     name!: string;
@@ -56,12 +58,15 @@ export class ManagerEntity extends RandomIdBaseEntity implements RunningStateEnt
     @OneToMany(type => ActionPremise, obj => obj.manager)
     actions!: Promise<ActionPremise[]>
 
-    @OneToMany(type => CheckEntity, obj => obj.manager) // note: we will create author property in the Photo class below
+    @OneToMany(type => CheckEntity, obj => obj.manager)
     checks!: Promise<CheckEntity[]>
 
-    @OneToMany(type => RunEntity, obj => obj.manager) // note: we will create author property in the Photo class below
+    @OneToMany(type => RunEntity, obj => obj.manager)
     runs!: Promise<RunEntity[]>
 
+    @OneToMany(type => ManagerGuestEntity, obj => obj.guestOf, {eager: true, cascade: ['insert', 'remove', 'update']})
+    guests!: ManagerGuestEntity[]
+
     @OneToOne(() => EventsRunState, {cascade: ['insert', 'update'], eager: true})
     @JoinColumn()
     eventsState!: EventsRunState
@@ -85,4 +90,50 @@ export class ManagerEntity extends RandomIdBaseEntity implements RunningStateEnt
             this.managerState = data.managerState;
         }
     }
+
+    getGuests(): ManagerGuestEntity[] {
+        const g = this.guests;
+        if (g === undefined) {
+            return [];
+        }
+        //return g.map(x => ({id: x.id, name: x.author.name, expiresAt: x.expiresAt})) as Guest[];
+        return g;
+    }
+
+   addGuest(val: GuestEntityData | GuestEntityData[]) {
+        const reqGuests = Array.isArray(val) ? val : [val];
+        const guests = this.getGuests();
+        for (const g of reqGuests) {
+            const existing = guests.find(x => x.author.name.toLowerCase() === g.author.name.toLowerCase());
+            if (existing !== undefined) {
+                // update existing guest expiresAt
+                existing.expiresAt = g.expiresAt;
+            } else {
+                guests.push(new ManagerGuestEntity({...g, guestOf: this}));
+            }
+        }
+        this.guests = guests;
+        return guests;
+    }
+
+    removeGuestById(val: string | string[]) {
+        const reqGuests = Array.isArray(val) ? val : [val];
+        const guests = this.getGuests();
+        const filteredGuests = guests.filter(x => !reqGuests.includes(x.id));
+        this.guests = filteredGuests
+        return filteredGuests;
+    }
+
+    removeGuestByUser(val: string | string[]) {
+        const reqGuests = (Array.isArray(val) ? val : [val]).map(x => x.trim().toLowerCase());
+        const guests = this.getGuests();
+        const filteredGuests = guests.filter(x => !reqGuests.includes(x.author.name.toLowerCase()));
+        this.guests = filteredGuests;
+        return filteredGuests;
+    }
+
+    removeGuests() {
+        this.guests = [];
+        return [];
+    }
 }

src/Common/Entities/SubredditInvite.ts

@@ -0,0 +1,54 @@
+import {Column, Entity, JoinColumn, ManyToOne, PrimaryColumn} from "typeorm";
+import {InviteData, SubredditInviteData} from "../../Web/Common/interfaces";
+import dayjs, {Dayjs} from "dayjs";
+import {TimeAwareRandomBaseEntity} from "./Base/TimeAwareRandomBaseEntity";
+import {AuthorEntity} from "./AuthorEntity";
+import {Bot} from "./Bot";
+
+@Entity()
+export class SubredditInvite extends TimeAwareRandomBaseEntity implements SubredditInviteData {
+
+    subreddit!: string;
+
+    @Column("simple-json", {nullable: true})
+    guests?: string[]
+
+    @Column("text")
+    initialConfig?: string
+
+    @ManyToOne(type => Bot, bot => bot.subredditInvites, {nullable: false, orphanedRowAction: 'delete'})
+    @JoinColumn({name: 'botId', referencedColumnName: 'id'})
+    bot!: Bot;
+
+    @Column({name: 'expiresAt', nullable: true})
+    _expiresAt?: Date;
+
+    public get expiresAt(): Dayjs | undefined {
+        if (this._expiresAt === undefined) {
+            return undefined;
+        }
+        return dayjs(this._expiresAt);
+    }
+
+    public set expiresAt(d: Dayjs | undefined) {
+        if (d === undefined) {
+            this._expiresAt = d;
+        } else {
+            this._expiresAt = d.utc().toDate();
+        }
+    }
+
+    constructor(data?: SubredditInviteData & { expiresIn?: number }) {
+        super();
+        if (data !== undefined) {
+            this.subreddit = data.subreddit;
+            this.initialConfig = data.initialConfig;
+            this.guests = data.guests;
+
+
+            if (data.expiresIn !== undefined && data.expiresIn !== 0) {
+                this.expiresAt = dayjs().add(data.expiresIn, 'seconds');
+            }
+        }
+    }
+}

src/Common/ImageData.ts

@@ -1,15 +1,17 @@
 import fetch from "node-fetch";
 import {Submission} from "snoowrap/dist/objects";
 import {URL} from "url";
-import {absPercentDifference, getSharpAsync, isValidImageURL} from "../util";
+import {absPercentDifference, getExtension, getSharpAsync, isValidImageURL} from "../util";
 import {Sharp} from "sharp";
-import {blockhash} from "./blockhash/blockhash";
-import {SimpleError} from "../Utils/Errors";
+import {blockhashAndFlipped} from "./blockhash/blockhash";
+import {CMError, SimpleError} from "../Utils/Errors";
+import {FileHandle, open} from "fs/promises";
+import {ImageHashCacheData} from "./Infrastructure/Atomic";
 
 export interface ImageDataOptions {
     width?: number,
     height?: number,
-    url: string,
+    path: URL,
     variants?: ImageData[]
 }
 
@@ -17,19 +19,20 @@ class ImageData {
 
     width?: number
     height?: number
-    url: URL
+    path: URL
     variants: ImageData[] = []
     preferredResolution?: [number, number]
     sharpImg!: Sharp
-    hashResult!: string
+    hashResult?: string
+    hashResultFlipped?: string
     actualResolution?: [number, number]
 
     constructor(data: ImageDataOptions, aggressive = false) {
         this.width = data.width;
         this.height = data.height;
-        this.url = new URL(data.url);
-        if (!aggressive && !isValidImageURL(`${this.url.origin}${this.url.pathname}`)) {
-            throw new Error('URL did not end with a valid image extension');
+        this.path = data.path;
+        if (!aggressive && !isValidImageURL(`${this.path.origin}${this.path.pathname}`)) {
+            throw new Error('Path did not end with a valid image extension');
         }
         this.variants = data.variants || [];
     }
@@ -39,55 +42,90 @@ class ImageData {
         return await (await this.sharp()).clone().toFormat(format).toBuffer();
     }
 
-    async hash(bits: number, useVariantIfPossible = true): Promise<string> {
-        if(this.hashResult === undefined) {
+    async hash(bits: number = 16, useVariantIfPossible = true): Promise<Required<ImageHashCacheData>> {
+        if (this.hashResult === undefined || this.hashResultFlipped === undefined) {
             let ref: ImageData | undefined;
-            if(useVariantIfPossible && this.preferredResolution !== undefined) {
+            if (useVariantIfPossible && this.preferredResolution !== undefined) {
                 ref = this.getSimilarResolutionVariant(this.preferredResolution[0], this.preferredResolution[1]);
             }
-            if(ref === undefined) {
+            if (ref === undefined) {
                 ref = this;
             }
-            this.hashResult = await blockhash((await ref.sharp()).clone(), bits);
+            const [hash, hashFlipped] = await blockhashAndFlipped((await ref.sharp()).clone(), bits);
+            this.hashResult = hash;
+            this.hashResultFlipped = hashFlipped;
         }
-        return this.hashResult;
+        return {original: this.hashResult, flipped: this.hashResultFlipped};
     }
 
     async sharp(): Promise<Sharp> {
         if (this.sharpImg === undefined) {
+            let animated = false;
+            let getBuffer: () => Promise<Buffer>;
+            let fileHandle: FileHandle | undefined;
             try {
-                const response = await fetch(this.url.toString())
-                if (response.ok) {
-                    const ct = response.headers.get('Content-Type');
-                    if (ct !== null && ct.includes('image')) {
-                        const sFunc = await getSharpAsync();
-                        // if image is animated then we want to extract the first frame and convert it to a regular image
-                        // so we can compare two static images later (also because sharp can't use resize() on animated images)
-                        if(['gif','webp'].some(x => ct.includes(x))) {
-                            this.sharpImg = await sFunc(await (await sFunc(await response.buffer(), {pages: 1, animated: false})).png().toBuffer());
-                        } else {
-                            this.sharpImg = await sFunc(await response.buffer());
-                        }
-                        const meta = await this.sharpImg.metadata();
-                        if (this.width === undefined || this.height === undefined) {
-                            this.width = meta.width;
-                            this.height = meta.height;
-                        }
-                        this.actualResolution = [meta.width as number, meta.height as number];
-                    } else {
-                        throw new SimpleError(`Content-Type for fetched URL ${this.url} did not contain "image"`);
+                if (this.path.protocol === 'file:') {
+                    try {
+                        animated = ['gif', 'webp'].includes(getExtension(this.path.pathname));
+                        fileHandle = await open(this.path, 'r');
+                        getBuffer = async () => await (fileHandle as FileHandle).readFile();
+                    } catch (err: any) {
+                        throw new CMError(`Unable to retrieve local file ${this.path.toString()}`, {cause: err});
                     }
                 } else {
-                    throw new SimpleError(`URL response was not OK: (${response.status})${response.statusText}`);
+                    try {
+                        const response = await fetch(this.path.toString())
+                        if (response.ok) {
+                            const ct = response.headers.get('Content-Type');
+                            if (ct !== null && ct.includes('image')) {
+                                animated = ['gif', 'webp'].some(x => ct.includes(x));
+                                getBuffer = async () => await response.buffer();
+                            } else {
+                                throw new SimpleError(`Content-Type for fetched URL ${this.path.toString()} did not contain "image"`);
+                            }
+                        } else {
+                            throw new SimpleError(`Fetching ${this.path.toString()} => URL response was not OK: (${response.status})${response.statusText}`);
+                        }
+
+                    } catch (err: any) {
+                        if (!(err instanceof SimpleError)) {
+                            throw new CMError(`Error occurred while fetching response from URL ${this.path.toString()}`, {cause: err});
+                        } else {
+                            throw err;
+                        }
+                    }
+                }
+            } catch (err: any) {
+                throw new CMError('Unable to fetch image resource', {cause: err, isSerious: false});
+            }
+
+            try {
+
+                const sFunc = await getSharpAsync();
+                // if image is animated then we want to extract the first frame and convert it to a regular image
+                // so we can compare two static images later (also because sharp can't use resize() on animated images)
+                if (animated) {
+                    this.sharpImg = await sFunc(await (await sFunc(await getBuffer(), {
+                        pages: 1,
+                        animated: false
+                    }).trim().greyscale()).png().withMetadata().toBuffer());
+                } else {
+                    this.sharpImg = await sFunc(await sFunc(await getBuffer()).trim().greyscale().withMetadata().toBuffer());
                 }
 
+                if(fileHandle !== undefined) {
+                    await fileHandle.close();
+                }
+
+                const meta = await this.sharpImg.metadata();
+                if (this.width === undefined || this.height === undefined) {
+                    this.width = meta.width;
+                    this.height = meta.height;
+                }
+                this.actualResolution = [meta.width as number, meta.height as number];
 
             } catch (err: any) {
-                if(!(err instanceof SimpleError)) {
-                    throw new Error(`Error occurred while fetching response from URL: ${err.message}`);
-                } else {
-                    throw err;
-                }
+                throw new CMError('Error occurred while converting image buffer to Sharp object', {cause: err});
             }
         }
         return this.sharpImg;
@@ -107,8 +145,8 @@ class ImageData {
         return this.width !== undefined && this.height !== undefined;
     }
 
-    get baseUrl() {
-        return `${this.url.origin}${this.url.pathname}`;
+    get basePath() {
+        return `${this.path.origin}${this.path.pathname}`;
     }
 
     setPreferredResolutionByWidth(prefWidth: number) {
@@ -225,10 +263,23 @@ class ImageData {
         return [refSharp, compareSharp, width, height];
     }
 
+    toHashCache(): ImageHashCacheData {
+        return {
+            original: this.hashResult,
+            flipped: this.hashResultFlipped
+        }
+    }
+
+    setFromHashCache(data: ImageHashCacheData) {
+        const {original, flipped} = data;
+        this.hashResult = original;
+        this.hashResultFlipped = flipped;
+    }
+
     static fromSubmission(sub: Submission, aggressive = false): ImageData {
         const url = new URL(sub.url);
         const data: any = {
-            url,
+            path: url,
         };
         let variants = [];
         if (sub.preview !== undefined && sub.preview.enabled && sub.preview.images.length > 0) {
@@ -237,7 +288,7 @@ class ImageData {
             data.width = ref.width;
             data.height = ref.height;
 
-            variants = firstImg.resolutions.map(x => new ImageData(x));
+            variants = firstImg.resolutions.map(x => new ImageData({...x, path: new URL(x.url)}));
             data.variants = variants;
         }
         return new ImageData(data, aggressive);

src/Common/Infrastructure/Atomic.ts

@@ -1,3 +1,5 @@
+import {ActivityType} from "./Reddit";
+
 /**
  * A duration and how to compare it against a value
  *
@@ -148,6 +150,7 @@ export type RecordOutputOption = boolean | RecordOutputType | RecordOutputType[]
 export type PostBehaviorType = 'next' | 'stop' | 'nextRun' | string;
 export type onExistingFoundBehavior = 'replace' | 'skip' | 'ignore';
 export type ActionTarget = 'self' | 'parent';
+export type ArbitraryActionTarget = ActionTarget | string;
 export type InclusiveActionTarget = ActionTarget | 'any';
 export type DispatchSource = 'dispatch' | `dispatch:${string}`;
 export type NonDispatchActivitySource = 'poll' | `poll:${PollOn}` | 'user' | `user:${string}`;
@@ -174,6 +177,7 @@ export type ActivitySource = NonDispatchActivitySource | DispatchSource;
 export type ConfigFormat = 'json' | 'yaml';
 export type ActionTypes =
     'comment'
+    | 'submission'
     | 'lock'
     | 'remove'
     | 'report'
@@ -277,3 +281,89 @@ export interface UrlContext {
     value: string
     context: WikiContext | ExternalUrlContext
 }
+
+export interface ImageHashCacheData {
+    original?: string
+    flipped?: string
+}
+
+// https://www.reddit.com/message/compose?to=/r/mealtimevideos&message=https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot
+
+export interface BaseTemplateData {
+    botLink: string
+    modmailLink?: string
+    manager?: string
+    check?: string
+    //[key: string]: any
+}
+
+export interface ActivityTemplateData {
+    kind: ActivityType
+    author: string
+    votes: number
+    age: string
+    permalink: string
+    id: string
+    subreddit: string
+    title: string
+    shortTitle: string
+}
+
+export interface ModdedActivityTemplateData {
+    reports: number
+    modReports: number
+    userReports: number
+}
+
+export interface SubmissionTemplateData extends ActivityTemplateData, Partial<ModdedActivityTemplateData> {
+    nsfw: boolean
+    spoiler: boolean
+    op: boolean
+    upvoteRatio: string
+    url: string
+}
+
+export interface CommentTemplateData extends ActivityTemplateData, Partial<ModdedActivityTemplateData> {
+    op: boolean
+}
+
+export interface SubredditTemplateData {
+    subredditBreakdownFormatted: string
+    subredditBreakdown?: {
+        totalFormatted: string
+        submissionFormatted: string
+        commentFormatted: string
+    }
+}
+
+export interface RuleResultTemplateData {
+    kind: string
+    triggered: boolean
+    result: string
+    [key: string]: any
+}
+
+export interface ActionResultTemplateData {
+    kind: string
+    success: boolean
+    result: string
+    [key: string]: any
+}
+
+export interface ActionResultsTemplateData {
+    actionSummary: string
+    actions: {
+        [key: string]: ActionResultTemplateData
+    }
+}
+
+export interface RuleResultsTemplateData {
+    ruleSummary: string
+    rules: {
+        [key: string]: RuleResultTemplateData
+    }
+}
+
+export interface GenericContentTemplateData extends BaseTemplateData, Partial<RuleResultsTemplateData>, Partial<ActionResultsTemplateData> {
+    item?: (SubmissionTemplateData | CommentTemplateData)
+}

src/Common/Infrastructure/Filters/FilterCriteria.ts

@@ -454,7 +454,7 @@ export interface ActivityState {
     dispatched?: boolean | string | string[]
 
 
-    // can use ActivitySource | ActivitySource[] here because of issues with generating json schema, see ActivitySource comments
+    // cant use ActivitySource | ActivitySource[] here because of issues with generating json schema, see ActivitySource comments
     /**
      * Test where the current activity was sourced from.
      *

src/Common/Infrastructure/Reddit.ts

@@ -75,3 +75,31 @@ export const activityReports = (activity: SnoowrapActivity): Report[] => {
     }
     return reports;
 }
+
+export interface RawSubredditRemovalReasonData {
+    data: {
+        [key: string]: SubredditRemovalReason
+    },
+    order: [string]
+}
+
+export interface SubredditRemovalReason {
+    message: string
+    id: string,
+    title: string
+}
+
+export interface SubredditActivityAbsoluteBreakdown {
+    count: number
+    name: string
+}
+
+export interface SubredditActivityBreakdown extends SubredditActivityAbsoluteBreakdown {
+    percent: number
+}
+
+export interface SubredditActivityBreakdownByType {
+    total: SubredditActivityBreakdown[]
+    submission: SubredditActivityBreakdown[]
+    comment: SubredditActivityBreakdown[]
+}

src/Common/MigrationService.ts

@@ -5,6 +5,7 @@ import {DatabaseMigrationOptions} from "./interfaces";
 import {copyFile} from "fs/promises";
 import {constants} from "fs";
 import {ErrorWithCause} from "pony-cause";
+import {CMError} from "../Utils/Errors";
 
 export interface ExistingTable {
     table: Table
@@ -118,9 +119,10 @@ export class MigrationService {
                 try {
                     await this.backupDatabase();
                     continueBCBackedup = true;
-                } catch (err) {
-                    // @ts-ignore
-                    this.dbLogger.error(err, {leaf: 'Backup'});
+                } catch (err: any) {
+                    if(!(err instanceof CMError) || !err.logged) {
+                        this.dbLogger.error(err, {leaf: 'Backup'});
+                    }
                 }
             } else {
                 this.dbLogger.info('Configuration DID NOT specify migrations may be executed if automated backup is successful. Will not try to create a backup.');
@@ -154,25 +156,34 @@ YOU SHOULD BACKUP YOUR EXISTING DATABASE BEFORE CONTINUING WITH MIGRATIONS.`);
 
     async backupDatabase() {
         try {
-            if (this.database.options.type === 'sqljs' && this.database.options.location !== undefined) {
+            let location: string | undefined;
+            const canBackup = ['sqljs','better-sqlite3'].includes(this.database.options.type);
+            if(canBackup) {
+                if(this.database.options.type === 'sqljs') {
+                    location = this.database.options.location === ':memory:' ? undefined : this.database.options.location;
+                } else {
+                    location = this.database.options.database === ':memory:' || (typeof this.database.options.database !== 'string') ? undefined : this.database.options.database;
+                }
+            }
+            if (canBackup && location !== undefined) {
                 try {
                     const ts = Date.now();
-                    const backupLocation = `${this.database.options.location}.${ts}.bak`
+                    const backupLocation = `${location}.${ts}.bak`
                     this.dbLogger.info(`Detected sqljs (sqlite) database. Will try to make a backup at ${backupLocation}`, {leaf: 'Backup'});
-                    await copyFile(this.database.options.location, backupLocation, constants.COPYFILE_EXCL);
+                    await copyFile(location, backupLocation, constants.COPYFILE_EXCL);
                     this.dbLogger.info('Successfully created backup!', {leaf: 'Backup'});
                 } catch (err: any) {
                     throw new ErrorWithCause('Cannot make an automated backup of your configured database.', {cause: err});
                 }
             } else {
                 let msg = 'Cannot make an automated backup of your configured database.';
-                if (this.database.options.type !== 'sqljs') {
-                    msg += ' Only SQlite (sqljs database type) is implemented for automated backups right now, sorry :( You will need to manually backup your database.';
+                if (!canBackup) {
+                    msg += ' Only SQlite (sqljs or better-sqlite3 database type) is implemented for automated backups right now, sorry :( You will need to manually backup your database.';
                 } else {
                     // TODO don't throw for this??
                     msg += ' Database location is not defined (probably in-memory).';
                 }
-                throw new Error(msg);
+                throw new CMError(msg, {logged: true});
             }
         } catch (e: any) {
             this.dbLogger.error(e, {leaf: 'Backup'});

src/Common/Migrations/Database/MigrationUtil.ts

@@ -1,4 +1,4 @@
-import {TableIndex} from "typeorm";
+import {QueryRunner, TableIndex} from "typeorm";
 
 /**
  * Boilerplate for creating generic index
@@ -78,3 +78,15 @@ export const filterIndices = (prefix: string) => {
         itemIsIndex(prefix)
     ]
 }
+
+export const tableHasData = async (runner: QueryRunner, name: string): Promise<boolean | null> => {
+    const countRes = await runner.query(`select count(*) from ${name}`);
+    let hasRows = null;
+    if (Array.isArray(countRes) && countRes[0] !== null) {
+        const {
+            'count(*)': count
+        } = countRes[0] || {};
+        hasRows = count !== 0;
+    }
+    return hasRows;
+}

src/Common/Migrations/Database/Server/1658930394548-Guests.ts

@@ -0,0 +1,50 @@
+import {MigrationInterface, QueryRunner, Table} from "typeorm"
+import {createdAtColumn, createdAtIndex, idIndex, index, randomIdColumn, timeAtColumn} from "../MigrationUtil";
+
+export class Guests1658930394548 implements MigrationInterface {
+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        const dbType = queryRunner.connection.driver.options.type;
+
+        await queryRunner.createTable(
+            new Table({
+                name: 'Guests',
+                columns: [
+                    randomIdColumn(),
+                    {
+                        name: 'authorName',
+                        type: 'varchar',
+                        length: '200',
+                        isNullable: false,
+                    },
+                    {
+                        name: 'type',
+                        type: 'varchar',
+                        isNullable: false,
+                        length: '50'
+                    },
+                    {
+                        name: 'guestOfId',
+                        type: 'varchar',
+                        length: '20',
+                        isNullable: true
+                    },
+                    timeAtColumn('expiresAt', dbType, true),
+                    createdAtColumn(dbType),
+                ],
+                indices: [
+                    idIndex('Guests', true),
+                    createdAtIndex('guests'),
+                    index('guest', ['expiresAt'], false)
+                ]
+            }),
+            true,
+            true,
+            true
+        );
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+    }
+
+}

src/Common/Migrations/Database/Server/1660228987769-invites.ts

@@ -0,0 +1,145 @@
+import {MigrationInterface, QueryRunner, Table, TableColumn} from "typeorm"
+import {createdAtColumn, createdAtIndex, idIndex, index, randomIdColumn, tableHasData, timeAtColumn} from "../MigrationUtil";
+
+export class invites1660228987769 implements MigrationInterface {
+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        const dbType = queryRunner.connection.driver.options.type;
+
+        await queryRunner.createTable(
+            new Table({
+                name: 'SubredditInvite',
+                columns: [
+                    {
+                        name: 'id',
+                        type: 'varchar',
+                        length: '255',
+                        isPrimary: true,
+                    },
+                    {
+                        name: 'botId',
+                        type: 'varchar',
+                        length: '20',
+                        isNullable: false
+                    },
+                    {
+                        name: 'subreddit',
+                        type: 'varchar',
+                        length: '255',
+                        isNullable: false
+                    },
+                    {
+                        name: 'guests',
+                        type: 'text',
+                        isNullable: true
+                    },
+                    {
+                        name: 'initialConfig',
+                        type: 'text',
+                        isNullable: true
+                    },
+                    createdAtColumn(dbType),
+                    timeAtColumn('expiresAt', dbType, true)
+                ],
+            }),
+            true,
+            true,
+            true
+        );
+
+        if (await queryRunner.hasTable('Invite')) {
+
+            await queryRunner.renameTable('Invite', 'BotInvite');
+            const table = await queryRunner.getTable('BotInvite') as Table;
+
+            await queryRunner.addColumns(table, [
+                new TableColumn({
+                    name: 'initialConfig',
+                    type: 'text',
+                    isNullable: true
+                }),
+                new TableColumn({
+                    name: 'guests',
+                    type: 'text',
+                    isNullable: true
+                })
+            ]);
+
+            queryRunner.connection.logger.logSchemaBuild(`Table 'Invite' has been renamed 'BotInvite'. If there are existing rows on this table they will need to be recreated.`);
+
+        } else {
+
+            await queryRunner.createTable(
+                new Table({
+                    name: 'BotInvite',
+                    columns: [
+                        {
+                            name: 'id',
+                            type: 'varchar',
+                            length: '255',
+                            isPrimary: true,
+                        },
+                        {
+                            name: 'clientId',
+                            type: 'varchar',
+                            length: '255',
+                        },
+                        {
+                            name: 'clientSecret',
+                            type: 'varchar',
+                            length: '255',
+                        },
+                        {
+                            name: 'redirectUri',
+                            type: 'text',
+                        },
+                        {
+                            name: 'creator',
+                            type: 'varchar',
+                            length: '255',
+                        },
+                        {
+                            name: 'permissions',
+                            type: 'text'
+                        },
+                        {
+                            name: 'instance',
+                            type: 'varchar',
+                            length: '255',
+                            isNullable: true
+                        },
+                        {
+                            name: 'overwrite',
+                            type: 'boolean',
+                            isNullable: true,
+                        },
+                        {
+                            name: 'subreddits',
+                            type: 'text',
+                            isNullable: true
+                        },
+                        {
+                            name: 'guests',
+                            type: 'text',
+                            isNullable: true
+                        },
+                        {
+                            name: 'initialConfig',
+                            type: 'text',
+                            isNullable: true
+                        },
+                        createdAtColumn(dbType),
+                        timeAtColumn('expiresAt', dbType, true)
+                    ],
+                }),
+                true,
+                true,
+                true
+            );
+        }
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+    }
+
+}

src/Common/Migrations/Database/Server/1661183583080-submission.ts

@@ -0,0 +1,15 @@
+import { MigrationInterface, QueryRunner } from "typeorm"
+import {ActionType} from "../../../Entities/ActionType";
+
+export class submission1661183583080 implements MigrationInterface {
+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.manager.getRepository(ActionType).save([
+            new ActionType('submission')
+        ]);
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+    }
+
+}

src/Common/Migrations/Database/Web/1660588028346-removeInvites.ts

@@ -0,0 +1,33 @@
+import {MigrationInterface, QueryRunner} from "typeorm"
+import {tableHasData} from "../MigrationUtil";
+
+export class removeInvites1660588028346 implements MigrationInterface {
+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        const dbType = queryRunner.connection.driver.options.type;
+
+        if (dbType === 'sqljs' && await queryRunner.hasTable('Invite')) {
+            // const countRes = await queryRunner.query('select count(*) from Invite');
+            // let hasNoRows = null;
+            // if (Array.isArray(countRes) && countRes[0] !== null) {
+            //     const {
+            //         'count(*)': count
+            //     } = countRes[0] || {};
+            //     hasNoRows = count === 0;
+            // }
+
+            const hasRows = await tableHasData(queryRunner, 'Invite');
+
+            if (hasRows === false) {
+                await queryRunner.dropTable('Invite');
+            } else {
+                let prefix = hasRows === null ? `Could not determine if SQL.js 'web' database had the table 'Invite' --` : `SQL.js 'web' database had the table 'Invite' and it is not empty --`
+                queryRunner.connection.logger.logSchemaBuild(`${prefix} This table is being replaced by 'BotInvite' table in 'app' database. If you have existing invites you will need to recreate them.`);
+            }
+        }
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+    }
+
+}

src/Common/WebEntities/ClientSession.ts

@@ -1,5 +1,5 @@
 import { ISession } from "connect-typeorm";
-import { Column, Entity, Index, PrimaryColumn } from "typeorm";
+import { Column, Entity, Index, PrimaryColumn, DeleteDateColumn } from "typeorm";
 @Entity()
 export class ClientSession implements ISession {
     @Index()
@@ -12,6 +12,6 @@ export class ClientSession implements ISession {
     @Column("text")
     public json = "";
 
-    @Column({ name: 'destroyedAt', nullable: true })
+    @DeleteDateColumn({ name: 'destroyedAt', nullable: true })
     destroyedAt?: Date;
 }

src/Common/blockhash/blockhash.ts

@@ -107,8 +107,7 @@ var bmvbhash_even = function(data: BlockImageData, bits: number) {
     return bits_to_hexhash(result);
 };
 
-var bmvbhash = function(data: BlockImageData, bits: number) {
-    var result = [];
+var bmvbhash = function(data: BlockImageData, bits: number, calculateFlipped: boolean = false): string | [string, string] {
 
     var i, j, x, y;
     var block_width, block_height;
@@ -198,30 +197,51 @@ var bmvbhash = function(data: BlockImageData, bits: number) {
         }
     }
 
-    for (i = 0; i < bits; i++) {
-        for (j = 0; j < bits; j++) {
-            result.push(blocks[i][j]);
+    const blocksFlipped: number[][] | undefined = calculateFlipped ? [] : undefined;
+    if(blocksFlipped !== undefined) {
+        for(const row of blocks) {
+            const flippedRow = [...row];
+            flippedRow.reverse();
+            blocksFlipped.push(flippedRow);
         }
     }
 
-    translate_blocks_to_bits(result, block_width * block_height);
-    return bits_to_hexhash(result);
+    if(blocksFlipped !== undefined) {
+        const result = [];
+        const resultFlip = [];
+        for (i = 0; i < bits; i++) {
+            for (j = 0; j < bits; j++) {
+                result.push(blocks[i][j]);
+                resultFlip.push(blocksFlipped[i][j])
+            }
+        }
+
+        translate_blocks_to_bits(result, block_width * block_height);
+        translate_blocks_to_bits(resultFlip, block_width * block_height);
+        return [bits_to_hexhash(result), bits_to_hexhash(resultFlip)];
+    } else {
+        const result = [];
+        for (i = 0; i < bits; i++) {
+            for (j = 0; j < bits; j++) {
+                result.push(blocks[i][j]);
+            }
+        }
+
+        translate_blocks_to_bits(result, block_width * block_height);
+        return bits_to_hexhash(result);
+    }
 };
 
-var blockhashData = function(imgData: BlockImageData, bits: number, method: number) {
-    var hash;
+var blockhashData = function(imgData: BlockImageData, bits: number, method: number, calculateFlipped: boolean): string | [string, string] {
 
     if (method === 1) {
-        hash = bmvbhash_even(imgData, bits);
+        return bmvbhash_even(imgData, bits);
     }
     else if (method === 2) {
-        hash = bmvbhash(imgData, bits);
-    }
-    else {
-        throw new Error("Bad hashing method");
+        return bmvbhash(imgData, bits, calculateFlipped);
     }
 
-    return hash;
+    throw new Error("Bad hashing method");
 };
 
 export const blockhash = async function(src: Sharp, bits: number, method: number = 2): Promise<string> {
@@ -230,5 +250,14 @@ export const blockhash = async function(src: Sharp, bits: number, method: number
         width: info.width,
         height: info.height,
         data: buff,
-    }, bits, method);
+    }, bits, method, false) as string;
+};
+
+export const blockhashAndFlipped = async function(src: Sharp, bits: number, method: number = 2): Promise<[string, string]> {
+    const {data: buff, info} = await src.ensureAlpha().raw().toBuffer({resolveWithObject: true});
+    return blockhashData({
+        width: info.width,
+        height: info.height,
+        data: buff,
+    }, bits, method, true) as [string, string];
 };

src/Common/defaults.ts

@@ -42,4 +42,4 @@ export const filterCriteriaDefault: FilterCriteriaDefaults = {
 export const defaultDataDir = path.resolve(__dirname, '../..');
 export const defaultConfigFilenames = ['config.json', 'config.yaml'];
 
-export const VERSION = '0.11.4';
+export const VERSION = '0.12.1';

src/Common/interfaces.ts

@@ -1060,6 +1060,16 @@ export interface SubredditOverrides {
          * */
         retention?: EventRetentionPolicyRange
     }
+
+    /**
+     * The relative URL to the ContextMod wiki page EX `https://reddit.com/r/subreddit/wiki/<path>`
+     *
+     * This will override the default relative URL as well as any URL set at the bot-level
+     *
+     * @default "botconfig/contextbot"
+     * @examples ["botconfig/contextbot"]
+     * */
+    wikiConfig?: string
 }
 
 /**
@@ -1479,20 +1489,6 @@ export interface OperatorJsonConfig {
             storage?: 'database' | 'cache'
         }
 
-        /**
-         * Settings related to oauth flow invites
-         * */
-        invites?: {
-            /**
-            * Number of seconds an invite should be valid for
-             *
-             * If `0` or not specified (default) invites do not expire
-            *
-            * @default 0
-            * @examples [0]
-            * */
-            maxAge?: number
-        }
         /**
          * The default log level to filter to in the web interface
          *
@@ -1548,11 +1544,30 @@ export interface OperatorJsonConfig {
         secret?: string,
         /**
          * A friendly name for this server. This will override `friendly` in `BotConnection` if specified.
+         *
+         * If none is set one is randomly generated.
          * */
         friendly?: string,
     }
 
     credentials?: ThirdPartyCredentialsJsonConfig
+
+    dev?: {
+        /**
+         * Invoke `process.memoryUsage()` on an interval and send metrics to Influx
+         *
+         * Only works if Influx config is provided
+         * */
+        monitorMemory?: boolean
+        /**
+        * Interval, in seconds, to invoke `process.memoryUsage()` at
+        *
+        * Defaults to 15 seconds
+        *
+        * @default 15
+        * */
+        monitorMemoryInterval?: number
+    };
 }
 
 export interface RequiredOperatorRedditCredentials extends RedditCredentials {
@@ -1642,9 +1657,6 @@ export interface OperatorConfig extends OperatorJsonConfig {
             secret?: string,
             storage?: 'database' | 'cache'
         },
-        invites: {
-          maxAge: number
-        },
         logLevel?: LogLevel,
         maxLogs: number,
         clients: BotConnection[]
@@ -1659,6 +1671,10 @@ export interface OperatorConfig extends OperatorJsonConfig {
     databaseStatisticsDefaults: DatabaseStatisticsOperatorConfig
     bots: BotInstanceConfig[]
     credentials: ThirdPartyCredentialsJsonConfig
+    dev: {
+        monitorMemory: boolean
+        monitorMemoryInterval: number
+    }
 }
 
 export interface OperatorFileConfig {
@@ -1714,6 +1730,7 @@ export interface ActionProcessResult {
     dryRun: boolean,
     result?: string
     touchedEntities?: (Submission | Comment | RedditUser | string)[]
+    data?: any
 }
 
 export interface EventActivity {

src/Common/types.ts

@@ -21,7 +21,8 @@ import {ContributorActionJson} from "../Action/ContributorAction";
 import {SentimentRuleJSONConfig} from "../Rule/SentimentRule";
 import {ModNoteActionJson} from "../Action/ModNoteAction";
 import {IncludesData} from "./Infrastructure/Includes";
+import { SubmissionActionJson } from "../Action/SubmissionAction";
 
 export type RuleObjectJsonTypes = RecentActivityRuleJSONConfig | RepeatActivityJSONConfig | AuthorRuleJSONConfig | AttributionJSONConfig | HistoryJSONConfig | RegexRuleJSONConfig | RepostRuleJSONConfig | SentimentRuleJSONConfig
 
-export type ActionJson = CommentActionJson | FlairActionJson | ReportActionJson | LockActionJson | RemoveActionJson | ApproveActionJson | BanActionJson | UserNoteActionJson | MessageActionJson | UserFlairActionJson | DispatchActionJson | CancelDispatchActionJson | ContributorActionJson | ModNoteActionJson | string | IncludesData;
+export type ActionJson = CommentActionJson | SubmissionActionJson | FlairActionJson | ReportActionJson | LockActionJson | RemoveActionJson | ApproveActionJson | BanActionJson | UserNoteActionJson | MessageActionJson | UserFlairActionJson | DispatchActionJson | CancelDispatchActionJson | ContributorActionJson | ModNoteActionJson | string | IncludesData;

src/ConfigBuilder.ts

@@ -2,7 +2,7 @@ import winston, {Logger} from "winston";
 import {
     asNamedCriteria, asWikiContext,
     buildCachePrefix, buildFilter, castToBool,
-    createAjvFactory, fileOrDirectoryIsWriteable,
+    createAjvFactory, fileOrDirectoryIsWriteable, generateRandomName,
     mergeArr, mergeFilters,
     normalizeName,
     overwriteMerge,
@@ -98,6 +98,7 @@ import {SubredditResources} from "./Subreddit/SubredditResources";
 import {asIncludesData, IncludesData, IncludesString} from "./Common/Infrastructure/Includes";
 import ConfigParseError from "./Utils/ConfigParseError";
 import {InfluxClient} from "./Common/Influx/InfluxClient";
+import {BotInvite} from "./Common/Entities/BotInvite";
 
 export interface ConfigBuilderOptions {
     logger: Logger,
@@ -1224,9 +1225,6 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
                 maxAge: sessionMaxAge = 86400,
                 storage: sessionStorage = undefined,
             } = {},
-            invites: {
-                maxAge: inviteMaxAge = 0,
-            } = {},
             clients,
             credentials: webCredentials,
             operators,
@@ -1239,6 +1237,10 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
         } = {},
         credentials = {},
         bots = [],
+        dev: {
+            monitorMemory = false,
+            monitorMemoryInterval = 15
+        } = {},
     } = data;
 
     let cache: StrongCache;
@@ -1327,6 +1329,8 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
     }
     const webDbConfig = createDatabaseConfig(realdbConnectionWeb);
 
+    const appDataSource = await createAppDatabaseConnection(dbConfig, appLogger);
+
     let influx: InfluxClient | undefined = undefined;
     if(influxConfig !== undefined) {
         const tags = friendly !== undefined ? {server: friendly} : undefined;
@@ -1334,6 +1338,28 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
         await influx.isReady();
     }
 
+/*    let friendlyId: string;
+    if (friendly === undefined) {
+        let randFriendly: string = generateRandomName();
+        // see if we can get invites to check for unique name
+        // if this is a new instance will not be able to get it but try anyway
+        try {
+            const inviteRepo = appDataSource.getRepository(BotInvite);
+            const exists = async (name: string) => {
+                const existing = await inviteRepo.findBy({instance: name});
+                return existing.length > 0;
+            }
+            while (await exists(randFriendly)) {
+                randFriendly = generateRandomName();
+            }
+        } catch (e: any) {
+            // something went wrong, just ignore this
+        }
+        friendlyId = randFriendly;
+    } else {
+        friendlyId = friendly;
+    }*/
+
     const config: OperatorConfig = {
         mode,
         operator: {
@@ -1347,7 +1373,7 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
             frequency,
             minFrequency
         },
-        database: await createAppDatabaseConnection(dbConfig, appLogger),
+        database: appDataSource,
         databaseConfig: {
             connection: dbConfig,
             migrations,
@@ -1367,9 +1393,6 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
             },
             port,
             storage: webStorage,
-            invites: {
-                maxAge: inviteMaxAge,
-            },
             session: {
                 secret: sessionSecretFromConfig,
                 maxAge: sessionMaxAge,
@@ -1383,10 +1406,14 @@ export const buildOperatorConfigWithDefaults = async (data: OperatorJsonConfig):
         api: {
             port: apiPort,
             secret: apiSecret,
-            friendly
+            friendly,
         },
         bots: [],
         credentials,
+        dev: {
+            monitorMemory,
+            monitorMemoryInterval
+        }
     };
 
     config.bots = bots.map(x => buildBotConfig(x, config));

src/Rule/HistoryRule.ts

@@ -7,9 +7,10 @@ import {Rule, RuleJSONConfig, RuleOptions} from "./index";
 import Submission from "snoowrap/dist/objects/Submission";
 import dayjs from "dayjs";
 import {
+    asComment,
     asSubmission,
     FAIL,
-    formatNumber, getActivitySubredditName, historyFilterConfigToOptions, isSubmission,
+    formatNumber, getActivitySubredditName, historyFilterConfigToOptions, isComment, isSubmission,
     parseSubredditName,
     PASS,
     percentFromString, removeUndefinedKeys, toStrongSubredditState, windowConfigToWindowCriteria
@@ -20,6 +21,7 @@ import {CompareValueOrPercent} from "../Common/Infrastructure/Atomic";
 import {ActivityWindowConfig, ActivityWindowCriteria} from "../Common/Infrastructure/ActivityWindow";
 import {ErrorWithCause} from "pony-cause";
 import {comparisonTextOp, parseGenericValueOrPercentComparison} from "../Common/Infrastructure/Comparisons";
+import {getSubredditBreakdownByActivityType} from "../Utils/SnoowrapUtils";
 
 export interface CommentThresholdCriteria extends ThresholdCriteria {
     /**
@@ -206,10 +208,11 @@ export class HistoryRule extends Rule {
                 fOpTotal = filteredCounts.opTotal;
             }
 
+            let asOp = false;
             let commentTrigger = undefined;
             if(comment !== undefined) {
                 const {operator, value, isPercent, extra = ''} = parseGenericValueOrPercentComparison(comment);
-                const asOp = extra.toLowerCase().includes('op');
+                 asOp = extra.toLowerCase().includes('op');
                 if(isPercent) {
                     const per = value / 100;
                     if(asOp) {
@@ -264,7 +267,8 @@ export class HistoryRule extends Rule {
                 submissionTrigger,
                 commentTrigger,
                 totalTrigger,
-                triggered: (submissionTrigger === undefined || submissionTrigger === true) && (commentTrigger === undefined || commentTrigger === true) && (totalTrigger === undefined || totalTrigger === true)
+                triggered: (submissionTrigger === undefined || submissionTrigger === true) && (commentTrigger === undefined || commentTrigger === true) && (totalTrigger === undefined || totalTrigger === true),
+                subredditBreakdown: getSubredditBreakdownByActivityType(!asOp ? filteredActivities :  filteredActivities.filter(x => asSubmission(x) || x.is_submitter))
             });
         }
 
@@ -320,6 +324,7 @@ export class HistoryRule extends Rule {
             submissionTrigger,
             commentTrigger,
             totalTrigger,
+            subredditBreakdown,
         } = results;
 
         const data: any = {
@@ -338,6 +343,7 @@ export class HistoryRule extends Rule {
             submissionTrigger,
             commentTrigger,
             totalTrigger,
+            subredditBreakdown
         };
 
         let thresholdSummary = [];

src/Rule/RecentActivityRule.ts

@@ -42,6 +42,8 @@ import {
 } from "../Common/Infrastructure/Filters/FilterCriteria";
 import {ActivityWindow, ActivityWindowConfig} from "../Common/Infrastructure/ActivityWindow";
 import {comparisonTextOp, parseGenericValueOrPercentComparison} from "../Common/Infrastructure/Comparisons";
+import {ImageHashCacheData} from "../Common/Infrastructure/Atomic";
+import {getSubredditBreakdownByActivityType} from "../Utils/SnoowrapUtils";
 
 const parseLink = parseUsableLinkIdentifier();
 
@@ -195,21 +197,21 @@ export class RecentActivityRule extends Rule {
                 let filteredActivity: (Submission|Comment)[] = [];
                 let analysisTimes: number[] = [];
                 let referenceImage: ImageData | undefined;
+                let refHash: Required<ImageHashCacheData> | undefined;
                 if (this.imageDetection.enable) {
                     try {
                         referenceImage = ImageData.fromSubmission(item);
                         referenceImage.setPreferredResolutionByWidth(800);
                         if(this.imageDetection.hash.enable) {
-                            let refHash: string | undefined;
                             if(this.imageDetection.hash.ttl !== undefined) {
                                 refHash = await this.resources.getImageHash(referenceImage);
                                 if(refHash === undefined) {
                                     refHash = await referenceImage.hash(this.imageDetection.hash.bits);
-                                    await this.resources.setImageHash(referenceImage, refHash, this.imageDetection.hash.ttl);
-                                } else if(refHash.length !== bitsToHexLength(this.imageDetection.hash.bits)) {
+                                    await this.resources.setImageHash(referenceImage, this.imageDetection.hash.ttl);
+                                } else if(refHash.original.length !== bitsToHexLength(this.imageDetection.hash.bits)) {
                                     this.logger.warn('Reference image hash length did not correspond to bits specified in config. Recomputing...');
-                                    refHash = await referenceImage.hash(this.imageDetection.hash.bits);
-                                    await this.resources.setImageHash(referenceImage, refHash, this.imageDetection.hash.ttl);
+                                    await referenceImage.hash(this.imageDetection.hash.bits);
+                                    await this.resources.setImageHash(referenceImage, this.imageDetection.hash.ttl);
                                 }
                             } else {
                                 refHash = await referenceImage.hash(this.imageDetection.hash.bits);
@@ -244,29 +246,38 @@ export class RecentActivityRule extends Rule {
                     }
                     // only do image detection if regular URL comparison and other conditions fail first
                     // to reduce CPU/bandwidth usage
-                    if (referenceImage !== undefined) {
+                    if (referenceImage !== undefined && refHash !== undefined) {
                         try {
                             let imgData =  ImageData.fromSubmission(x);
                             imgData.setPreferredResolutionByWidth(800);
                             if(this.imageDetection.hash.enable) {
-                                let compareHash: string | undefined;
+                                let compareHash: Required<ImageHashCacheData> | undefined;
                                 if(this.imageDetection.hash.ttl !== undefined) {
                                     compareHash = await this.resources.getImageHash(imgData);
                                 }
-                                if(compareHash === undefined)
+                                if(compareHash === undefined || compareHash.original.length !== refHash.original.length)
                                 {
+                                    if(compareHash !== undefined) {
+                                        this.logger.debug(`Hash lengths were not the same! Will need to recompute compare hash to match reference.\n\nReference: ${referenceImage.basePath} has is ${refHash.original.length} char long | Comparing: ${imgData.basePath} has is ${compareHash} ${compareHash.original.length} long`);
+                                    }
                                     compareHash = await imgData.hash(this.imageDetection.hash.bits);
                                     if(this.imageDetection.hash.ttl !== undefined) {
-                                        await this.resources.setImageHash(imgData, compareHash, this.imageDetection.hash.ttl);
+                                        await this.resources.setImageHash(imgData, this.imageDetection.hash.ttl);
                                     }
                                 }
-                                const refHash = await referenceImage.hash(this.imageDetection.hash.bits);
-                                if(refHash.length !== compareHash.length) {
-                                    this.logger.debug(`Hash lengths were not the same! Will need to recompute compare hash to match reference.\n\nReference: ${referenceImage.baseUrl} has is ${refHash.length} char long | Comparing: ${imgData.baseUrl} has is ${compareHash} ${compareHash.length} long`);
-                                    compareHash = await imgData.hash(this.imageDetection.hash.bits)
+                                let diff: number;
+                                const odistance = leven(refHash.original, compareHash.original);
+                                diff = (odistance/refHash.original.length)*100;
+
+                                // compare flipped hash if it exists
+                                // if it has less difference than normal comparison then the image is probably flipped (or so different it doesn't matter)
+                                if(compareHash.flipped !== undefined) {
+                                    const fdistance = leven(refHash.original, compareHash.flipped);
+                                    const fdiff = (fdistance/refHash.original.length)*100;
+                                    if(fdiff < diff) {
+                                        diff = fdiff;
+                                    }
                                 }
-                                const distance = leven(refHash, compareHash);
-                                const diff = (distance/refHash.length)*100;
 
 
                                 // return image if hard is defined and diff is less
@@ -498,6 +509,7 @@ export class RecentActivityRule extends Rule {
                 testValue,
                 karmaThreshold,
                 combinedKarma,
+                subredditBreakdown: getSubredditBreakdownByActivityType(activities)
             }
         };
     }

src/Schema/Action.json

@@ -46,6 +46,9 @@
         {
             "$ref": "#/definitions/ModNoteActionJson"
         },
+        {
+            "$ref": "#/definitions/SubmissionActionJson"
+        },
         {
             "type": "string"
         }
@@ -442,7 +445,6 @@
                     "examples": [
                         "Sock puppet for u/AnotherUser"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 },
                 "reason": {
@@ -450,7 +452,6 @@
                     "examples": [
                         "repeat spam"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 }
             },
@@ -706,6 +707,20 @@
                 "sticky": {
                     "description": "Stick the comment after creation?",
                     "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this comment should be made\n\nValid values: 'self' | 'parent' | [reddit permalink]\n\n'self' and 'parent' are special targets that are relative to the Activity being processed:\n* When Activity is Submission => 'parent' does nothing\n* When Activity is Comment\n   * 'self' => reply to Activity\n   * 'parent' => make a top-level comment in the Submission the Comment is in\n\nIf target is not self/parent then CM assumes the value is a reddit permalink and will attempt to make a comment to that Activity"
                 }
             },
             "required": [
@@ -1649,13 +1664,13 @@
                     "type": "string"
                 },
                 "to": {
-                    "description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
+                    "description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
                     "examples": [
                         "aUserName",
                         "u/aUserName",
-                        "r/aSubreddit"
+                        "r/aSubreddit",
+                        "r/{{item.subreddit}}"
                     ],
-                    "pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
                     "type": "string"
                 }
             },
@@ -2175,7 +2190,16 @@
                     "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
                     "type": "string"
                 },
+                "note": {
+                    "description": "(Optional) A mod-readable note added to the removal reason for this Activity. Can use Templating.\n\nThis note (and removal reasons) are only visible on New Reddit",
+                    "type": "string"
+                },
+                "reasonId": {
+                    "description": "(Optional) The ID of the Removal Reason to use\n\nRemoval reasons are only visible on New Reddit\n\nTo find IDs for removal reasons check the \"Removal Reasons\" popup located in the CM dashboard config editor for your subreddit\n\nMore info on Removal Reasons: https://mods.reddithelp.com/hc/en-us/articles/360010094892-Removal-Reasons",
+                    "type": "string"
+                },
                 "spam": {
+                    "description": "(Optional) Mark Activity as spam",
                     "type": "boolean"
                 }
             },
@@ -2285,6 +2309,170 @@
             ],
             "type": "object"
         },
+        "SubmissionActionJson": {
+            "description": "Reply to the Activity. For a submission the reply will be a top-level comment.",
+            "properties": {
+                "authorIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/AuthorCriteria"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<AuthorCriteria>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<AuthorCriteria>"
+                        }
+                    ],
+                    "description": "If present then these Author criteria are checked before running the Check. If criteria fails then the Check will fail."
+                },
+                "content": {
+                    "description": "The Content to submit for this Action. Content is interpreted as reddit-flavored Markdown.\n\nIf value starts with `wiki:` then the proceeding value will be used to get a wiki page from the current subreddit\n\n * EX `wiki:botconfig/mybot` tries to get `https://reddit.com/r/currentSubreddit/wiki/botconfig/mybot`\n\nIf the value starts with `wiki:` and ends with `|someValue` then `someValue` will be used as the base subreddit for the wiki page\n\n* EX `wiki:replytemplates/test|ContextModBot` tries to get `https://reddit.com/r/ContextModBot/wiki/replytemplates/test`\n\nIf the value starts with `url:` then the value is fetched as an external url and expects raw text returned\n\n* EX `url:https://pastebin.com/raw/38qfL7mL` tries to get the text response of `https://pastebin.com/raw/38qfL7mL`\n\nIf none of the above is used the value is treated as the raw context\n\n * EX `this is **bold** markdown text` => \"this is **bold** markdown text\"\n\nAll Content is rendered using [mustache](https://github.com/janl/mustache.js/#templates) to enable [Action Templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nThe following properties are always available in the template (view individual Rules to see rule-specific template data):\n```\nitem.kind      => The type of Activity that was checked (comment/submission)\nitem.author    => The name of the Author of the Activity EX FoxxMD\nitem.permalink => A permalink URL to the Activity EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nitem.url       => If the Activity is Link Sumbission then the external URL\nitem.title     => If the Activity is a Submission then the title of that Submission\nrules          => An object containing RuleResults of all the rules run for this check. See Action Templating for more details on naming\n```",
+                    "examples": [
+                        "This is the content of a comment/report/usernote",
+                        "this is **bold** markdown text",
+                        "wiki:botconfig/acomment"
+                    ],
+                    "type": "string"
+                },
+                "distinguish": {
+                    "description": "Distinguish as Mod after creation?",
+                    "type": "boolean"
+                },
+                "dryRun": {
+                    "default": false,
+                    "description": "If `true` the Action will not make the API request to Reddit to perform its action.",
+                    "examples": [
+                        false,
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "enable": {
+                    "default": true,
+                    "description": "If set to `false` the Action will not be run",
+                    "examples": [
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "flairId": {
+                    "description": "Flair template to apply to this Submission",
+                    "type": "string"
+                },
+                "flairText": {
+                    "description": "Flair text to apply to this Submission",
+                    "type": "string"
+                },
+                "footer": {
+                    "anyOf": [
+                        {
+                            "enum": [
+                                false
+                            ],
+                            "type": "boolean"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Customize the footer for Actions that send replies (Comment/Ban)\n\nIf `false` no footer is appended\n\nIf `string` the value is rendered as markdown or will use `wiki:` parser the same way `content` properties on Actions are rendered with [templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nIf footer is `undefined` (not set) the default footer will be used:\n\n> *****\n> This action was performed by [a bot.] Mention a moderator or [send a modmail] if you any ideas, questions, or concerns about this action.\n\n*****\n\nThe following properties are available for [templating](https://github.com/FoxxMD/context-mod#action-templating):\n```\nsubName    => name of subreddit Action was performed in (EX 'mealtimevideos')\npermaLink  => The permalink for the Activity the Action was performed on EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nmodmaiLink => An encoded URL that will open a new message to your subreddit with the Action permalink appended to the body\nbotLink    => A permalink to the FAQ for this bot.\n```\nIf you use your own footer or no footer **please link back to the bot FAQ** using the `{{botLink}}` property in your content :)"
+                },
+                "itemIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/SubmissionState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/CommentState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<TypedActivityState>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<TypedActivityState>"
+                        }
+                    ],
+                    "description": "A list of criteria to test the state of the `Activity` against before running the check.\n\nIf any set of criteria passes the Check will be run. If the criteria fails then the Check will fail.\n\n* @examples [[{\"over_18\": true, \"removed': false}]]"
+                },
+                "kind": {
+                    "description": "The type of action that will be performed",
+                    "enum": [
+                        "submission"
+                    ],
+                    "type": "string"
+                },
+                "lock": {
+                    "description": "Lock the Submission after creation?",
+                    "type": "boolean"
+                },
+                "name": {
+                    "description": "An optional, but highly recommended, friendly name for this Action. If not present will default to `kind`.\n\nCan only contain letters, numbers, underscore, spaces, and dashes",
+                    "examples": [
+                        "myDescriptiveAction"
+                    ],
+                    "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
+                    "type": "string"
+                },
+                "nsfw": {
+                    "type": "boolean"
+                },
+                "spoiler": {
+                    "type": "boolean"
+                },
+                "sticky": {
+                    "description": "Sticky the Submission after creation?",
+                    "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
+                },
+                "title": {
+                    "description": "The title of this Submission.\n\nTemplated the same as **content**",
+                    "type": "string"
+                },
+                "url": {
+                    "description": "If Submission should be a Link, the URL to use\n\nTemplated the same as **content**\n\nPROTIP: To make a Link Submission pointing to the Activity being processed use `{{item.permalink}}` as the URL value",
+                    "type": "string"
+                }
+            },
+            "required": [
+                "kind",
+                "title"
+            ],
+            "type": "object"
+        },
         "SubmissionState": {
             "description": "Different attributes a `Submission` can be in. Only include a property if you want to check it.",
             "examples": [

src/Schema/App.json

@@ -923,7 +923,6 @@
                     "examples": [
                         "Sock puppet for u/AnotherUser"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 },
                 "reason": {
@@ -931,7 +930,6 @@
                     "examples": [
                         "repeat spam"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 }
             },
@@ -1378,6 +1376,20 @@
                 "sticky": {
                     "description": "Stick the comment after creation?",
                     "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this comment should be made\n\nValid values: 'self' | 'parent' | [reddit permalink]\n\n'self' and 'parent' are special targets that are relative to the Activity being processed:\n* When Activity is Submission => 'parent' does nothing\n* When Activity is Comment\n   * 'self' => reply to Activity\n   * 'parent' => make a top-level comment in the Submission the Comment is in\n\nIf target is not self/parent then CM assumes the value is a reddit permalink and will attempt to make a comment to that Activity"
                 }
             },
             "required": [
@@ -1449,6 +1461,9 @@
                             {
                                 "$ref": "#/definitions/ModNoteActionJson"
                             },
+                            {
+                                "$ref": "#/definitions/SubmissionActionJson"
+                            },
                             {
                                 "type": "string"
                             }
@@ -3439,13 +3454,13 @@
                     "type": "string"
                 },
                 "to": {
-                    "description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
+                    "description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
                     "examples": [
                         "aUserName",
                         "u/aUserName",
-                        "r/aSubreddit"
+                        "r/aSubreddit",
+                        "r/{{item.subreddit}}"
                     ],
-                    "pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
                     "type": "string"
                 }
             },
@@ -4680,7 +4695,16 @@
                     "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
                     "type": "string"
                 },
+                "note": {
+                    "description": "(Optional) A mod-readable note added to the removal reason for this Activity. Can use Templating.\n\nThis note (and removal reasons) are only visible on New Reddit",
+                    "type": "string"
+                },
+                "reasonId": {
+                    "description": "(Optional) The ID of the Removal Reason to use\n\nRemoval reasons are only visible on New Reddit\n\nTo find IDs for removal reasons check the \"Removal Reasons\" popup located in the CM dashboard config editor for your subreddit\n\nMore info on Removal Reasons: https://mods.reddithelp.com/hc/en-us/articles/360010094892-Removal-Reasons",
+                    "type": "string"
+                },
                 "spam": {
+                    "description": "(Optional) Mark Activity as spam",
                     "type": "boolean"
                 }
             },
@@ -5654,6 +5678,170 @@
             ],
             "type": "object"
         },
+        "SubmissionActionJson": {
+            "description": "Reply to the Activity. For a submission the reply will be a top-level comment.",
+            "properties": {
+                "authorIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/AuthorCriteria"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<AuthorCriteria>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<AuthorCriteria>"
+                        }
+                    ],
+                    "description": "If present then these Author criteria are checked before running the Check. If criteria fails then the Check will fail."
+                },
+                "content": {
+                    "description": "The Content to submit for this Action. Content is interpreted as reddit-flavored Markdown.\n\nIf value starts with `wiki:` then the proceeding value will be used to get a wiki page from the current subreddit\n\n * EX `wiki:botconfig/mybot` tries to get `https://reddit.com/r/currentSubreddit/wiki/botconfig/mybot`\n\nIf the value starts with `wiki:` and ends with `|someValue` then `someValue` will be used as the base subreddit for the wiki page\n\n* EX `wiki:replytemplates/test|ContextModBot` tries to get `https://reddit.com/r/ContextModBot/wiki/replytemplates/test`\n\nIf the value starts with `url:` then the value is fetched as an external url and expects raw text returned\n\n* EX `url:https://pastebin.com/raw/38qfL7mL` tries to get the text response of `https://pastebin.com/raw/38qfL7mL`\n\nIf none of the above is used the value is treated as the raw context\n\n * EX `this is **bold** markdown text` => \"this is **bold** markdown text\"\n\nAll Content is rendered using [mustache](https://github.com/janl/mustache.js/#templates) to enable [Action Templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nThe following properties are always available in the template (view individual Rules to see rule-specific template data):\n```\nitem.kind      => The type of Activity that was checked (comment/submission)\nitem.author    => The name of the Author of the Activity EX FoxxMD\nitem.permalink => A permalink URL to the Activity EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nitem.url       => If the Activity is Link Sumbission then the external URL\nitem.title     => If the Activity is a Submission then the title of that Submission\nrules          => An object containing RuleResults of all the rules run for this check. See Action Templating for more details on naming\n```",
+                    "examples": [
+                        "This is the content of a comment/report/usernote",
+                        "this is **bold** markdown text",
+                        "wiki:botconfig/acomment"
+                    ],
+                    "type": "string"
+                },
+                "distinguish": {
+                    "description": "Distinguish as Mod after creation?",
+                    "type": "boolean"
+                },
+                "dryRun": {
+                    "default": false,
+                    "description": "If `true` the Action will not make the API request to Reddit to perform its action.",
+                    "examples": [
+                        false,
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "enable": {
+                    "default": true,
+                    "description": "If set to `false` the Action will not be run",
+                    "examples": [
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "flairId": {
+                    "description": "Flair template to apply to this Submission",
+                    "type": "string"
+                },
+                "flairText": {
+                    "description": "Flair text to apply to this Submission",
+                    "type": "string"
+                },
+                "footer": {
+                    "anyOf": [
+                        {
+                            "enum": [
+                                false
+                            ],
+                            "type": "boolean"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Customize the footer for Actions that send replies (Comment/Ban)\n\nIf `false` no footer is appended\n\nIf `string` the value is rendered as markdown or will use `wiki:` parser the same way `content` properties on Actions are rendered with [templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nIf footer is `undefined` (not set) the default footer will be used:\n\n> *****\n> This action was performed by [a bot.] Mention a moderator or [send a modmail] if you any ideas, questions, or concerns about this action.\n\n*****\n\nThe following properties are available for [templating](https://github.com/FoxxMD/context-mod#action-templating):\n```\nsubName    => name of subreddit Action was performed in (EX 'mealtimevideos')\npermaLink  => The permalink for the Activity the Action was performed on EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nmodmaiLink => An encoded URL that will open a new message to your subreddit with the Action permalink appended to the body\nbotLink    => A permalink to the FAQ for this bot.\n```\nIf you use your own footer or no footer **please link back to the bot FAQ** using the `{{botLink}}` property in your content :)"
+                },
+                "itemIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/SubmissionState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/CommentState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<TypedActivityState>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<TypedActivityState>"
+                        }
+                    ],
+                    "description": "A list of criteria to test the state of the `Activity` against before running the check.\n\nIf any set of criteria passes the Check will be run. If the criteria fails then the Check will fail.\n\n* @examples [[{\"over_18\": true, \"removed': false}]]"
+                },
+                "kind": {
+                    "description": "The type of action that will be performed",
+                    "enum": [
+                        "submission"
+                    ],
+                    "type": "string"
+                },
+                "lock": {
+                    "description": "Lock the Submission after creation?",
+                    "type": "boolean"
+                },
+                "name": {
+                    "description": "An optional, but highly recommended, friendly name for this Action. If not present will default to `kind`.\n\nCan only contain letters, numbers, underscore, spaces, and dashes",
+                    "examples": [
+                        "myDescriptiveAction"
+                    ],
+                    "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
+                    "type": "string"
+                },
+                "nsfw": {
+                    "type": "boolean"
+                },
+                "spoiler": {
+                    "type": "boolean"
+                },
+                "sticky": {
+                    "description": "Sticky the Submission after creation?",
+                    "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
+                },
+                "title": {
+                    "description": "The title of this Submission.\n\nTemplated the same as **content**",
+                    "type": "string"
+                },
+                "url": {
+                    "description": "If Submission should be a Link, the URL to use\n\nTemplated the same as **content**\n\nPROTIP: To make a Link Submission pointing to the Activity being processed use `{{item.permalink}}` as the URL value",
+                    "type": "string"
+                }
+            },
+            "required": [
+                "kind",
+                "title"
+            ],
+            "type": "object"
+        },
         "SubmissionCheckConfigData": {
             "properties": {
                 "actions": {
@@ -5717,6 +5905,9 @@
                             {
                                 "$ref": "#/definitions/ModNoteActionJson"
                             },
+                            {
+                                "$ref": "#/definitions/SubmissionActionJson"
+                            },
                             {
                                 "type": "string"
                             }

src/Schema/Check.json

@@ -937,7 +937,6 @@
                     "examples": [
                         "Sock puppet for u/AnotherUser"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 },
                 "reason": {
@@ -945,7 +944,6 @@
                     "examples": [
                         "repeat spam"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 }
             },
@@ -1201,6 +1199,20 @@
                 "sticky": {
                     "description": "Stick the comment after creation?",
                     "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this comment should be made\n\nValid values: 'self' | 'parent' | [reddit permalink]\n\n'self' and 'parent' are special targets that are relative to the Activity being processed:\n* When Activity is Submission => 'parent' does nothing\n* When Activity is Comment\n   * 'self' => reply to Activity\n   * 'parent' => make a top-level comment in the Submission the Comment is in\n\nIf target is not self/parent then CM assumes the value is a reddit permalink and will attempt to make a comment to that Activity"
                 }
             },
             "required": [
@@ -1272,6 +1284,9 @@
                             {
                                 "$ref": "#/definitions/ModNoteActionJson"
                             },
+                            {
+                                "$ref": "#/definitions/SubmissionActionJson"
+                            },
                             {
                                 "type": "string"
                             }
@@ -3034,13 +3049,13 @@
                     "type": "string"
                 },
                 "to": {
-                    "description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
+                    "description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
                     "examples": [
                         "aUserName",
                         "u/aUserName",
-                        "r/aSubreddit"
+                        "r/aSubreddit",
+                        "r/{{item.subreddit}}"
                     ],
-                    "pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
                     "type": "string"
                 }
             },
@@ -4135,7 +4150,16 @@
                     "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
                     "type": "string"
                 },
+                "note": {
+                    "description": "(Optional) A mod-readable note added to the removal reason for this Activity. Can use Templating.\n\nThis note (and removal reasons) are only visible on New Reddit",
+                    "type": "string"
+                },
+                "reasonId": {
+                    "description": "(Optional) The ID of the Removal Reason to use\n\nRemoval reasons are only visible on New Reddit\n\nTo find IDs for removal reasons check the \"Removal Reasons\" popup located in the CM dashboard config editor for your subreddit\n\nMore info on Removal Reasons: https://mods.reddithelp.com/hc/en-us/articles/360010094892-Removal-Reasons",
+                    "type": "string"
+                },
                 "spam": {
+                    "description": "(Optional) Mark Activity as spam",
                     "type": "boolean"
                 }
             },
@@ -4979,6 +5003,170 @@
             ],
             "type": "object"
         },
+        "SubmissionActionJson": {
+            "description": "Reply to the Activity. For a submission the reply will be a top-level comment.",
+            "properties": {
+                "authorIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/AuthorCriteria"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<AuthorCriteria>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<AuthorCriteria>"
+                        }
+                    ],
+                    "description": "If present then these Author criteria are checked before running the Check. If criteria fails then the Check will fail."
+                },
+                "content": {
+                    "description": "The Content to submit for this Action. Content is interpreted as reddit-flavored Markdown.\n\nIf value starts with `wiki:` then the proceeding value will be used to get a wiki page from the current subreddit\n\n * EX `wiki:botconfig/mybot` tries to get `https://reddit.com/r/currentSubreddit/wiki/botconfig/mybot`\n\nIf the value starts with `wiki:` and ends with `|someValue` then `someValue` will be used as the base subreddit for the wiki page\n\n* EX `wiki:replytemplates/test|ContextModBot` tries to get `https://reddit.com/r/ContextModBot/wiki/replytemplates/test`\n\nIf the value starts with `url:` then the value is fetched as an external url and expects raw text returned\n\n* EX `url:https://pastebin.com/raw/38qfL7mL` tries to get the text response of `https://pastebin.com/raw/38qfL7mL`\n\nIf none of the above is used the value is treated as the raw context\n\n * EX `this is **bold** markdown text` => \"this is **bold** markdown text\"\n\nAll Content is rendered using [mustache](https://github.com/janl/mustache.js/#templates) to enable [Action Templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nThe following properties are always available in the template (view individual Rules to see rule-specific template data):\n```\nitem.kind      => The type of Activity that was checked (comment/submission)\nitem.author    => The name of the Author of the Activity EX FoxxMD\nitem.permalink => A permalink URL to the Activity EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nitem.url       => If the Activity is Link Sumbission then the external URL\nitem.title     => If the Activity is a Submission then the title of that Submission\nrules          => An object containing RuleResults of all the rules run for this check. See Action Templating for more details on naming\n```",
+                    "examples": [
+                        "This is the content of a comment/report/usernote",
+                        "this is **bold** markdown text",
+                        "wiki:botconfig/acomment"
+                    ],
+                    "type": "string"
+                },
+                "distinguish": {
+                    "description": "Distinguish as Mod after creation?",
+                    "type": "boolean"
+                },
+                "dryRun": {
+                    "default": false,
+                    "description": "If `true` the Action will not make the API request to Reddit to perform its action.",
+                    "examples": [
+                        false,
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "enable": {
+                    "default": true,
+                    "description": "If set to `false` the Action will not be run",
+                    "examples": [
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "flairId": {
+                    "description": "Flair template to apply to this Submission",
+                    "type": "string"
+                },
+                "flairText": {
+                    "description": "Flair text to apply to this Submission",
+                    "type": "string"
+                },
+                "footer": {
+                    "anyOf": [
+                        {
+                            "enum": [
+                                false
+                            ],
+                            "type": "boolean"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Customize the footer for Actions that send replies (Comment/Ban)\n\nIf `false` no footer is appended\n\nIf `string` the value is rendered as markdown or will use `wiki:` parser the same way `content` properties on Actions are rendered with [templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nIf footer is `undefined` (not set) the default footer will be used:\n\n> *****\n> This action was performed by [a bot.] Mention a moderator or [send a modmail] if you any ideas, questions, or concerns about this action.\n\n*****\n\nThe following properties are available for [templating](https://github.com/FoxxMD/context-mod#action-templating):\n```\nsubName    => name of subreddit Action was performed in (EX 'mealtimevideos')\npermaLink  => The permalink for the Activity the Action was performed on EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nmodmaiLink => An encoded URL that will open a new message to your subreddit with the Action permalink appended to the body\nbotLink    => A permalink to the FAQ for this bot.\n```\nIf you use your own footer or no footer **please link back to the bot FAQ** using the `{{botLink}}` property in your content :)"
+                },
+                "itemIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/SubmissionState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/CommentState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<TypedActivityState>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<TypedActivityState>"
+                        }
+                    ],
+                    "description": "A list of criteria to test the state of the `Activity` against before running the check.\n\nIf any set of criteria passes the Check will be run. If the criteria fails then the Check will fail.\n\n* @examples [[{\"over_18\": true, \"removed': false}]]"
+                },
+                "kind": {
+                    "description": "The type of action that will be performed",
+                    "enum": [
+                        "submission"
+                    ],
+                    "type": "string"
+                },
+                "lock": {
+                    "description": "Lock the Submission after creation?",
+                    "type": "boolean"
+                },
+                "name": {
+                    "description": "An optional, but highly recommended, friendly name for this Action. If not present will default to `kind`.\n\nCan only contain letters, numbers, underscore, spaces, and dashes",
+                    "examples": [
+                        "myDescriptiveAction"
+                    ],
+                    "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
+                    "type": "string"
+                },
+                "nsfw": {
+                    "type": "boolean"
+                },
+                "spoiler": {
+                    "type": "boolean"
+                },
+                "sticky": {
+                    "description": "Sticky the Submission after creation?",
+                    "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
+                },
+                "title": {
+                    "description": "The title of this Submission.\n\nTemplated the same as **content**",
+                    "type": "string"
+                },
+                "url": {
+                    "description": "If Submission should be a Link, the URL to use\n\nTemplated the same as **content**\n\nPROTIP: To make a Link Submission pointing to the Activity being processed use `{{item.permalink}}` as the URL value",
+                    "type": "string"
+                }
+            },
+            "required": [
+                "kind",
+                "title"
+            ],
+            "type": "object"
+        },
         "SubmissionCheckConfigData": {
             "properties": {
                 "actions": {
@@ -5042,6 +5230,9 @@
                             {
                                 "$ref": "#/definitions/ModNoteActionJson"
                             },
+                            {
+                                "$ref": "#/definitions/SubmissionActionJson"
+                            },
                             {
                                 "type": "string"
                             }

src/Schema/OperatorConfig.json

@@ -2030,6 +2030,14 @@
                 },
                 "name": {
                     "type": "string"
+                },
+                "wikiConfig": {
+                    "default": "botconfig/contextbot",
+                    "description": "The relative URL to the ContextMod wiki page EX `https://reddit.com/r/subreddit/wiki/<path>`\n\nThis will override the default relative URL as well as any URL set at the bot-level",
+                    "examples": [
+                        "botconfig/contextbot"
+                    ],
+                    "type": "string"
                 }
             },
             "required": [
@@ -2132,7 +2140,7 @@
             "description": "Configuration for the **Server** application. See [Architecture Documentation](https://github.com/FoxxMD/context-mod/blob/master/docs/serverClientArchitecture.md) for more info",
             "properties": {
                 "friendly": {
-                    "description": "A friendly name for this server. This will override `friendly` in `BotConnection` if specified.",
+                    "description": "A friendly name for this server. This will override `friendly` in `BotConnection` if specified.\n\nIf none is set one is randomly generated.",
                     "type": "string"
                 },
                 "port": {
@@ -2243,6 +2251,20 @@
             "$ref": "#/definitions/DatabaseStatisticsOperatorJsonConfig",
             "description": "Set defaults for the frequency time series stats are collected"
         },
+        "dev": {
+            "properties": {
+                "monitorMemory": {
+                    "description": "Invoke `process.memoryUsage()` on an interval and send metrics to Influx\n\nOnly works if Influx config is provided",
+                    "type": "boolean"
+                },
+                "monitorMemoryInterval": {
+                    "default": 15,
+                    "description": "Interval, in seconds, to invoke `process.memoryUsage()` at\n\nDefaults to 15 seconds",
+                    "type": "number"
+                }
+            },
+            "type": "object"
+        },
         "influxConfig": {
             "$ref": "#/definitions/InfluxConfig"
         },
@@ -2411,20 +2433,6 @@
                     },
                     "type": "object"
                 },
-                "invites": {
-                    "description": "Settings related to oauth flow invites",
-                    "properties": {
-                        "maxAge": {
-                            "default": 0,
-                            "description": "Number of seconds an invite should be valid for\n\n If `0` or not specified (default) invites do not expire",
-                            "examples": [
-                                0
-                            ],
-                            "type": "number"
-                        }
-                    },
-                    "type": "object"
-                },
                 "logLevel": {
                     "description": "The default log level to filter to in the web interface\n\nIf not specified or `null` will be same as global `logLevel`",
                     "enum": [

src/Schema/Run.json

@@ -934,7 +934,6 @@
                     "examples": [
                         "Sock puppet for u/AnotherUser"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 },
                 "reason": {
@@ -942,7 +941,6 @@
                     "examples": [
                         "repeat spam"
                     ],
-                    "maxLength": 100,
                     "type": "string"
                 }
             },
@@ -1198,6 +1196,20 @@
                 "sticky": {
                     "description": "Stick the comment after creation?",
                     "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this comment should be made\n\nValid values: 'self' | 'parent' | [reddit permalink]\n\n'self' and 'parent' are special targets that are relative to the Activity being processed:\n* When Activity is Submission => 'parent' does nothing\n* When Activity is Comment\n   * 'self' => reply to Activity\n   * 'parent' => make a top-level comment in the Submission the Comment is in\n\nIf target is not self/parent then CM assumes the value is a reddit permalink and will attempt to make a comment to that Activity"
                 }
             },
             "required": [
@@ -1269,6 +1281,9 @@
                             {
                                 "$ref": "#/definitions/ModNoteActionJson"
                             },
+                            {
+                                "$ref": "#/definitions/SubmissionActionJson"
+                            },
                             {
                                 "type": "string"
                             }
@@ -3150,13 +3165,13 @@
                     "type": "string"
                 },
                 "to": {
-                    "description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
+                    "description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
                     "examples": [
                         "aUserName",
                         "u/aUserName",
-                        "r/aSubreddit"
+                        "r/aSubreddit",
+                        "r/{{item.subreddit}}"
                     ],
-                    "pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
                     "type": "string"
                 }
             },
@@ -4251,7 +4266,16 @@
                     "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
                     "type": "string"
                 },
+                "note": {
+                    "description": "(Optional) A mod-readable note added to the removal reason for this Activity. Can use Templating.\n\nThis note (and removal reasons) are only visible on New Reddit",
+                    "type": "string"
+                },
+                "reasonId": {
+                    "description": "(Optional) The ID of the Removal Reason to use\n\nRemoval reasons are only visible on New Reddit\n\nTo find IDs for removal reasons check the \"Removal Reasons\" popup located in the CM dashboard config editor for your subreddit\n\nMore info on Removal Reasons: https://mods.reddithelp.com/hc/en-us/articles/360010094892-Removal-Reasons",
+                    "type": "string"
+                },
                 "spam": {
+                    "description": "(Optional) Mark Activity as spam",
                     "type": "boolean"
                 }
             },
@@ -5225,6 +5249,170 @@
             ],
             "type": "object"
         },
+        "SubmissionActionJson": {
+            "description": "Reply to the Activity. For a submission the reply will be a top-level comment.",
+            "properties": {
+                "authorIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/AuthorCriteria"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<AuthorCriteria>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<AuthorCriteria>"
+                        }
+                    ],
+                    "description": "If present then these Author criteria are checked before running the Check. If criteria fails then the Check will fail."
+                },
+                "content": {
+                    "description": "The Content to submit for this Action. Content is interpreted as reddit-flavored Markdown.\n\nIf value starts with `wiki:` then the proceeding value will be used to get a wiki page from the current subreddit\n\n * EX `wiki:botconfig/mybot` tries to get `https://reddit.com/r/currentSubreddit/wiki/botconfig/mybot`\n\nIf the value starts with `wiki:` and ends with `|someValue` then `someValue` will be used as the base subreddit for the wiki page\n\n* EX `wiki:replytemplates/test|ContextModBot` tries to get `https://reddit.com/r/ContextModBot/wiki/replytemplates/test`\n\nIf the value starts with `url:` then the value is fetched as an external url and expects raw text returned\n\n* EX `url:https://pastebin.com/raw/38qfL7mL` tries to get the text response of `https://pastebin.com/raw/38qfL7mL`\n\nIf none of the above is used the value is treated as the raw context\n\n * EX `this is **bold** markdown text` => \"this is **bold** markdown text\"\n\nAll Content is rendered using [mustache](https://github.com/janl/mustache.js/#templates) to enable [Action Templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nThe following properties are always available in the template (view individual Rules to see rule-specific template data):\n```\nitem.kind      => The type of Activity that was checked (comment/submission)\nitem.author    => The name of the Author of the Activity EX FoxxMD\nitem.permalink => A permalink URL to the Activity EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nitem.url       => If the Activity is Link Sumbission then the external URL\nitem.title     => If the Activity is a Submission then the title of that Submission\nrules          => An object containing RuleResults of all the rules run for this check. See Action Templating for more details on naming\n```",
+                    "examples": [
+                        "This is the content of a comment/report/usernote",
+                        "this is **bold** markdown text",
+                        "wiki:botconfig/acomment"
+                    ],
+                    "type": "string"
+                },
+                "distinguish": {
+                    "description": "Distinguish as Mod after creation?",
+                    "type": "boolean"
+                },
+                "dryRun": {
+                    "default": false,
+                    "description": "If `true` the Action will not make the API request to Reddit to perform its action.",
+                    "examples": [
+                        false,
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "enable": {
+                    "default": true,
+                    "description": "If set to `false` the Action will not be run",
+                    "examples": [
+                        true
+                    ],
+                    "type": "boolean"
+                },
+                "flairId": {
+                    "description": "Flair template to apply to this Submission",
+                    "type": "string"
+                },
+                "flairText": {
+                    "description": "Flair text to apply to this Submission",
+                    "type": "string"
+                },
+                "footer": {
+                    "anyOf": [
+                        {
+                            "enum": [
+                                false
+                            ],
+                            "type": "boolean"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Customize the footer for Actions that send replies (Comment/Ban)\n\nIf `false` no footer is appended\n\nIf `string` the value is rendered as markdown or will use `wiki:` parser the same way `content` properties on Actions are rendered with [templating](https://github.com/FoxxMD/context-mod#action-templating).\n\nIf footer is `undefined` (not set) the default footer will be used:\n\n> *****\n> This action was performed by [a bot.] Mention a moderator or [send a modmail] if you any ideas, questions, or concerns about this action.\n\n*****\n\nThe following properties are available for [templating](https://github.com/FoxxMD/context-mod#action-templating):\n```\nsubName    => name of subreddit Action was performed in (EX 'mealtimevideos')\npermaLink  => The permalink for the Activity the Action was performed on EX https://reddit.com/r/yourSub/comments/o1h0i0/title_name/1v3b7x\nmodmaiLink => An encoded URL that will open a new message to your subreddit with the Action permalink appended to the body\nbotLink    => A permalink to the FAQ for this bot.\n```\nIf you use your own footer or no footer **please link back to the bot FAQ** using the `{{botLink}}` property in your content :)"
+                },
+                "itemIs": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/definitions/SubmissionState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/CommentState"
+                                    },
+                                    {
+                                        "$ref": "#/definitions/NamedCriteria<TypedActivityState>"
+                                    },
+                                    {
+                                        "type": "string"
+                                    }
+                                ]
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "$ref": "#/definitions/FilterOptionsJson<TypedActivityState>"
+                        }
+                    ],
+                    "description": "A list of criteria to test the state of the `Activity` against before running the check.\n\nIf any set of criteria passes the Check will be run. If the criteria fails then the Check will fail.\n\n* @examples [[{\"over_18\": true, \"removed': false}]]"
+                },
+                "kind": {
+                    "description": "The type of action that will be performed",
+                    "enum": [
+                        "submission"
+                    ],
+                    "type": "string"
+                },
+                "lock": {
+                    "description": "Lock the Submission after creation?",
+                    "type": "boolean"
+                },
+                "name": {
+                    "description": "An optional, but highly recommended, friendly name for this Action. If not present will default to `kind`.\n\nCan only contain letters, numbers, underscore, spaces, and dashes",
+                    "examples": [
+                        "myDescriptiveAction"
+                    ],
+                    "pattern": "^[a-zA-Z]([\\w -]*[\\w])?$",
+                    "type": "string"
+                },
+                "nsfw": {
+                    "type": "boolean"
+                },
+                "spoiler": {
+                    "type": "boolean"
+                },
+                "sticky": {
+                    "description": "Sticky the Submission after creation?",
+                    "type": "boolean"
+                },
+                "targets": {
+                    "anyOf": [
+                        {
+                            "items": {
+                                "type": "string"
+                            },
+                            "type": "array"
+                        },
+                        {
+                            "type": "string"
+                        }
+                    ],
+                    "description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
+                },
+                "title": {
+                    "description": "The title of this Submission.\n\nTemplated the same as **content**",
+                    "type": "string"
+                },
+                "url": {
+                    "description": "If Submission should be a Link, the URL to use\n\nTemplated the same as **content**\n\nPROTIP: To make a Link Submission pointing to the Activity being processed use `{{item.permalink}}` as the URL value",
+                    "type": "string"
+                }
+            },
+            "required": [
+                "kind",
+                "title"
+            ],
+            "type": "object"
+        },
         "SubmissionCheckConfigData": {
             "properties": {
                 "actions": {
@@ -5288,6 +5476,9 @@
                             {
                                 "$ref": "#/definitions/ModNoteActionJson"
                             },
+                            {
+                                "$ref": "#/definitions/SubmissionActionJson"
+                            },
                             {
                                 "type": "string"
                             }

src/Subreddit/Manager.ts

@@ -99,6 +99,8 @@ import {parseFromJsonOrYamlToObject} from "../Common/Config/ConfigUtil";
 import {FilterCriteriaDefaults} from "../Common/Infrastructure/Filters/FilterShapes";
 import {InfluxClient} from "../Common/Influx/InfluxClient";
 import { Point } from "@influxdata/influxdb-client";
+import {NormalizedManagerResponse} from "../Web/Common/interfaces";
+import {guestEntityToApiGuest} from "../Common/Entities/Guest/GuestEntity";
 
 export interface RunningState {
     state: RunState,
@@ -1493,6 +1495,11 @@ export class Manager extends EventEmitter implements RunningStates {
 
     async startQueue(causedBy: Invokee = 'system', options?: ManagerStateChangeOption) {
 
+        if(!this.validConfigLoaded) {
+            this.logger.warn('Cannot start queue while manager has an invalid configuration');
+            return;
+        }
+
         if(this.activityRepo === undefined) {
             this.activityRepo = this.resources.database.getRepository(Activity);
         }
@@ -1763,4 +1770,41 @@ export class Manager extends EventEmitter implements RunningStates {
 
         await this.cacheManager.defaultDatabase.getRepository(ManagerEntity).save(this.managerEntity);
     }
+
+    async writeHealthMetrics(time?: number) {
+        if (this.influxClients.length > 0) {
+            const metric = new Point('managerHealth')
+                .intField('delayedActivities', this.resources !== undefined ? this.resources.delayedItems.length : 0)
+                .intField('processing', this.queue.running())
+                .intField('queued', this.queue.length())
+                .booleanField('eventsRunning', this.eventsState.state === RUNNING)
+                .booleanField('queueRunning', this.queueState.state === RUNNING)
+                .booleanField('running', this.managerState.state === RUNNING)
+                .intField('uptime', this.startedAt !== undefined ? dayjs().diff(this.startedAt, 'seconds') : 0)
+                .intField('configAge', this.lastWikiRevision === undefined ? 0 : dayjs().diff(this.lastWikiRevision, 'seconds'));
+
+            if (this.resources !== undefined) {
+                const {req, miss} = this.resources.getCacheTotals();
+                metric.intField('cacheRequests', req)
+                    .intField('cacheMisses', miss);
+            }
+
+            if (time !== undefined) {
+                metric.timestamp(time);
+            }
+
+            for (const client of this.influxClients) {
+                await client.writePoint(metric);
+            }
+        }
+    }
+
+    toNormalizedManager(): NormalizedManagerResponse {
+        return {
+            name: this.displayLabel,
+            subreddit: this.subreddit.display_name,
+            subredditNormal: parseRedditEntity(this.subreddit.display_name).name,
+            guests: this.managerEntity.getGuests().map(x => guestEntityToApiGuest(x))
+        }
+    }
 }

src/Subreddit/SubredditResources.ts

@@ -3,8 +3,8 @@ import objectHash from 'object-hash';
 import {
     activityIsDeleted, activityIsFiltered,
     activityIsRemoved,
-    AuthorTypedActivitiesOptions, BOT_LINK,
-    getAuthorHistoryAPIOptions
+    AuthorTypedActivitiesOptions, BOT_LINK, TemplateContext,
+    getAuthorHistoryAPIOptions, renderContent
 } from "../Utils/SnoowrapUtils";
 import {map as mapAsync} from 'async';
 import winston, {Logger} from "winston";
@@ -58,7 +58,7 @@ import {
     filterByTimeRequirement,
     asSubreddit,
     modActionCriteriaSummary,
-    parseRedditFullname
+    parseRedditFullname, asStrongImageHashCache
 } from "../util";
 import LoggedError from "../Utils/LoggedError";
 import {
@@ -96,7 +96,7 @@ import {CMEvent as ActionedEventEntity, CMEvent } from "../Common/Entities/CMEve
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import globrex from 'globrex';
 import {runMigrations} from "../Common/Migrations/CacheMigrationUtils";
-import {isStatusError, MaybeSeriousErrorWithCause, SimpleError} from "../Utils/Errors";
+import {CMError, isStatusError, MaybeSeriousErrorWithCause, SimpleError} from "../Utils/Errors";
 import {ErrorWithCause} from "pony-cause";
 import {ManagerEntity} from "../Common/Entities/ManagerEntity";
 import {Bot} from "../Common/Entities/Bot";
@@ -120,7 +120,7 @@ import {
 } from "../Common/Infrastructure/Filters/FilterCriteria";
 import {
     ActivitySource, ConfigFragmentValidationFunc, DurationVal,
-    EventRetentionPolicyRange,
+    EventRetentionPolicyRange, ImageHashCacheData,
     JoinOperands,
     ModActionType,
     ModeratorNameCriteria, ModUserNoteLabel, statFrequencies, StatisticFrequency,
@@ -146,7 +146,7 @@ import {
     ActivityType,
     AuthorHistorySort,
     CachedFetchedActivitiesResult, FetchedActivitiesResult,
-    SnoowrapActivity
+    SnoowrapActivity, SubredditRemovalReason
 } from "../Common/Infrastructure/Reddit";
 import {AuthorCritPropHelper} from "../Common/Infrastructure/Filters/AuthorCritPropHelper";
 import {NoopLogger} from "../Utils/loggerFactory";
@@ -161,6 +161,7 @@ import {IncludesData} from "../Common/Infrastructure/Includes";
 import {parseFromJsonOrYamlToObject} from "../Common/Config/ConfigUtil";
 import ConfigParseError from "../Utils/ConfigParseError";
 import {ActivityReport} from "../Common/Entities/ActivityReport";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export const DEFAULT_FOOTER = '\r\n*****\r\nThis action was performed by [a bot.]({{botLink}}) Mention a moderator or [send a modmail]({{modmailLink}}) if you any ideas, questions, or concerns about this action.';
 
@@ -510,9 +511,15 @@ export class SubredditResources {
         this.delayedItems.push(data);
     }
 
-    async removeDelayedActivity(id: string) {
-        await this.dispatchedActivityRepo.delete(id);
-        this.delayedItems = this.delayedItems.filter(x => x.id !== id);
+    async removeDelayedActivity(val?: string | string[]) {
+        if(val === undefined) {
+            await this.dispatchedActivityRepo.delete({manager: {id: this.managerEntity.id}});
+            this.delayedItems = [];
+        } else {
+            const ids = typeof val === 'string' ? [val] : val;
+            await this.dispatchedActivityRepo.delete(ids);
+            this.delayedItems = this.delayedItems.filter(x => !ids.includes(x.id));
+        }
     }
 
     async initStats() {
@@ -773,11 +780,15 @@ export class SubredditResources {
         }
     }
 
-    async getStats() {
-        const totals = Object.values(this.stats.cache).reduce((acc, curr) => ({
+    getCacheTotals() {
+        return Object.values(this.stats.cache).reduce((acc, curr) => ({
             miss: acc.miss + curr.miss,
             req: acc.req + curr.requests,
         }), {miss: 0, req: 0});
+    }
+
+    async getStats() {
+        const totals = this.getCacheTotals();
         const cacheKeys = Object.keys(this.stats.cache);
         const res = {
             cache: {
@@ -984,7 +995,7 @@ export class SubredditResources {
                 hash = `sub-${item.name}`;
                 if (tryToFetch && item instanceof Submission) {
                     // @ts-ignore
-                    const itemToCache = await item.fetch();
+                    const itemToCache = await item.refresh();
                     await this.cache.set(hash, itemToCache, {ttl: this.submissionTTL});
                     return itemToCache;
                 } else {
@@ -996,7 +1007,7 @@ export class SubredditResources {
                 hash = `comm-${item.name}`;
                 if (tryToFetch && item instanceof Comment) {
                     // @ts-ignore
-                    const itemToCache = await item.fetch();
+                    const itemToCache = await item.refresh();
                     await this.cache.set(hash, itemToCache, {ttl: this.commentTTL});
                     return itemToCache;
                 } else {
@@ -1006,8 +1017,12 @@ export class SubredditResources {
                 }
             }
             return item;
-        } catch (e) {
-            throw new ErrorWithCause('Error occurred while trying to add Activity to cache', {cause: e});
+        } catch (e: any) {
+            if(e.message !== undefined && e.message.includes('Cannot read properties of undefined (reading \'constructor\')')) {
+                throw new ErrorWithCause('Error occurred while trying to add Activity to cache (Comment likely does not exist)', {cause: e});
+            } else {
+                throw new ErrorWithCause('Error occurred while trying to add Activity to cache', {cause: e});
+            }
         }
     }
 
@@ -1092,8 +1107,9 @@ export class SubredditResources {
                 return subreddit as Subreddit;
             }
         } catch (err: any) {
-            this.logger.error('Error while trying to fetch a cached subreddit', err);
-            throw err.logged;
+            const cmError = new CMError('Error while trying to fetch a cached subreddit', {cause: err, logged: true});
+            this.logger.error(cmError);
+            throw cmError;
         }
     }
 
@@ -1751,6 +1767,29 @@ export class SubredditResources {
         return wikiContent;
     }
 
+    /**
+     * Convenience method for using getContent and SnoowrapUtils@renderContent in one method
+     * */
+    async renderContent(contentStr: string, activity: SnoowrapActivity, ruleResults: RuleResultEntity[] = [], actionResults: ActionResultEntity[] = [], templateData: TemplateContext = {}) {
+        const content = await this.getContent(contentStr);
+
+        const {usernotes = this.userNotes, ...restData} = templateData;
+        return await renderContent(content, {
+            ...restData,
+            activity,
+            usernotes,
+            ruleResults,
+            actionResults,
+        });
+    }
+
+    async renderFooter(item: Submission | Comment, footer: false | string | undefined = this.footer) {
+        if (footer === false) {
+            return '';
+        }
+        return this.renderContent(footer, item);
+    }
+
     async getConfigFragment<T>(includesData: IncludesData, validateFunc?: ConfigFragmentValidationFunc): Promise<T> {
 
         const {
@@ -2268,7 +2307,7 @@ export class SubredditResources {
                             const requestedSourcesVal: string[] = !Array.isArray(itemOptVal) ? [itemOptVal] as string[] : itemOptVal as string[];
                             const requestedSources = requestedSourcesVal.map(x => strToActivitySource(x).toLowerCase());
 
-                            propResultsMap.source!.passed = criteriaPassWithIncludeBehavior(requestedSources.some(x => source.toLowerCase().includes(x)), include);
+                            propResultsMap.source!.passed = criteriaPassWithIncludeBehavior(requestedSources.some(x => source.toLowerCase().trim() === x.toLowerCase().trim()), include);
                             break;
                         }
                     case 'score':
@@ -3322,27 +3361,26 @@ export class SubredditResources {
         this.logger.debug(`Cached check result '${result.check.name}' for User ${userName} on Submission ${item.link_id} for ${ttl} seconds (Hash ${hash})`);
     }
 
-    async generateFooter(item: Submission | Comment, actionFooter?: false | string) {
-        let footer = actionFooter !== undefined ? actionFooter : this.footer;
-        if (footer === false) {
-            return '';
+    async getImageHash(img: ImageData): Promise<Required<ImageHashCacheData>|undefined> {
+
+        if(img.hashResult !== undefined && img.hashResultFlipped !== undefined) {
+            return img.toHashCache() as Required<ImageHashCacheData>;
         }
-        const subName = await item.subreddit.display_name;
-        const permaLink = `https://reddit.com${await item.permalink}`
-        const modmailLink = `https://www.reddit.com/message/compose?to=%2Fr%2F${subName}&message=${encodeURIComponent(permaLink)}`
 
-        const footerRawContent = await this.getContent(footer, item.subreddit);
-        return he.decode(Mustache.render(footerRawContent, {subName, permaLink, modmailLink, botLink: BOT_LINK}));
-    }
-
-    async getImageHash(img: ImageData): Promise<string|undefined> {
-        const hash = `imgHash-${img.baseUrl}`;
+        const hash = `imgHash-${img.basePath}`;
         const result = await this.cache.get(hash) as string | undefined | null;
         this.stats.cache.imageHash.requests++
         this.stats.cache.imageHash.requestTimestamps.push(Date.now());
         await this.stats.cache.imageHash.identifierRequestCount.set(hash, (await this.stats.cache.imageHash.identifierRequestCount.wrap(hash, () => 0) as number) + 1);
         if(result !== undefined && result !== null) {
-            return result;
+            try {
+                const data =  JSON.parse(result);
+                if(asStrongImageHashCache(data)) {
+                    return data;
+                }
+            } catch (e) {
+                // had old values, just drop it
+            }
         }
         this.stats.cache.commentCheck.miss++;
         return undefined;
@@ -3353,8 +3391,8 @@ export class SubredditResources {
         // return hash;
     }
 
-    async setImageHash(img: ImageData, hash: string, ttl: number): Promise<void> {
-        await this.cache.set(`imgHash-${img.baseUrl}`, hash, {ttl});
+    async setImageHash(img: ImageData, ttl: number): Promise<void> {
+        await this.cache.set(`imgHash-${img.basePath}`, img.toHashCache() as Required<ImageHashCacheData>, {ttl});
         // const hash = await this.cache.wrap(img.baseUrl, async () => await img.hash(true), { ttl }) as string;
         // if(img.hashResult === undefined) {
         //     img.hashResult = hash;
@@ -3368,6 +3406,21 @@ export class SubredditResources {
         }
         return undefined;
     }
+
+    async getSubredditRemovalReasons(): Promise<SubredditRemovalReason[]> {
+        if(this.wikiTTL !== false) {
+            return await this.cache.wrap(`removalReasons`, async () => {
+                const res = await this.client.getSubredditRemovalReasons(this.subreddit.display_name);
+                return Object.values(res.data);
+            }, { ttl: this.wikiTTL }) as SubredditRemovalReason[];
+        }
+        const res = await this.client.getSubredditRemovalReasons(this.subreddit.display_name);
+        return Object.values(res.data);
+    }
+
+    async getSubredditRemovalReasonById(id: string): Promise<SubredditRemovalReason | undefined> {
+        return (await this.getSubredditRemovalReasons()).find(x => x.id === id);
+    }
 }
 
 export class BotResourcesManager {
@@ -3568,11 +3621,19 @@ export class BotResourcesManager {
     }
 
     async addPendingSubredditInvite(subreddit: string): Promise<void> {
+        if(subreddit === null || subreddit === undefined || subreddit == '') {
+            throw new CMError('Subreddit name cannot be empty');
+        }
         let subredditNames = await this.defaultCache.get(`modInvites`) as (string[] | undefined | null);
         if (subredditNames === undefined || subredditNames === null) {
             subredditNames = [];
         }
-        subredditNames.push(subreddit);
+        const cleanName = subreddit.trim();
+
+        if(subredditNames.some(x => x.trim().toLowerCase() === cleanName.toLowerCase())) {
+            throw new CMError(`An invite for the Subreddit '${subreddit}' already exists`);
+        }
+        subredditNames.push(cleanName);
         await this.defaultCache.set(`modInvites`, subredditNames, {ttl: 0});
         return;
     }
@@ -3582,7 +3643,7 @@ export class BotResourcesManager {
         if (subredditNames === undefined || subredditNames === null) {
             subredditNames = [];
         }
-        subredditNames = subredditNames.filter(x => x !== subreddit);
+        subredditNames = subredditNames.filter(x => x.toLowerCase() !== subreddit.trim().toLowerCase());
         await this.defaultCache.set(`modInvites`, subredditNames, {ttl: 0});
         return;
     }

src/Utils/SnoowrapClients.ts

@@ -3,7 +3,8 @@ import {Submission, Subreddit, Comment} from "snoowrap/dist/objects";
 import {parseSubredditName} from "../util";
 import {ModUserNoteLabel} from "../Common/Infrastructure/Atomic";
 import {CreateModNoteData, ModNote, ModNoteRaw, ModNoteSnoowrapPopulated} from "../Subreddit/ModNotes/ModNote";
-import {SimpleError} from "./Errors";
+import {CMError, SimpleError} from "./Errors";
+import {RawSubredditRemovalReasonData, SnoowrapActivity} from "../Common/Infrastructure/Reddit";
 
 // const proxyFactory = (endpoint: string) => {
 //     return class ProxiedSnoowrap extends Snoowrap {
@@ -140,6 +141,47 @@ export class ExtendedSnoowrap extends Snoowrap {
         }) as { created: ModNoteRaw };
         return new ModNote(response.created, this);
     }
+
+    /**
+     * Add a removal reason and/or mod note to a REMOVED Activity
+     *
+     * The activity must already be removed for this call to succeed. This is an UNDOCUMENTED endpoint.
+     *
+     * @see https://github.com/praw-dev/praw/blob/b22e1f514d68d36545daf62e8a8d6c6c8caf782b/praw/endpoints.py#L149 for endpoint
+     * @see https://github.com/praw-dev/praw/blob/b22e1f514d68d36545daf62e8a8d6c6c8caf782b/praw/models/reddit/mixins/__init__.py#L28 for usage
+     * */
+    async addRemovalReason(item: SnoowrapActivity, note?: string, reason?: string) {
+        try {
+            if(note === undefined && reason === undefined) {
+                throw new CMError(`Must provide either a note or reason in order to add removal reason on Activity ${item.name}`, {isSerious: false});
+            }
+            await this.oauthRequest({
+                uri: 'api/v1/modactions/removal_reasons',
+                method: 'post',
+                body: {
+                    item_ids: [item.name],
+                    mod_note: note ?? null,
+                    reason_id: reason ?? null,
+                },
+            });
+        } catch(e: any) {
+            throw e;
+        }
+    }
+
+    /**
+     * Get a list of New Reddit removal reasons for a Subreddit
+     *
+     * This is an UNDOCUMENTED endpoint.
+     *
+     * @see https://github.com/praw-dev/praw/blob/b22e1f514d68d36545daf62e8a8d6c6c8caf782b/praw/endpoints.py#L151 for endpoint
+     * */
+    async getSubredditRemovalReasons(sub: Subreddit | string): Promise<RawSubredditRemovalReasonData> {
+        return await this.oauthRequest({
+            uri: `api/v1/${typeof sub === 'string' ? sub : sub.display_name}/removal_reasons`,
+            method: 'get'
+        }) as RawSubredditRemovalReasonData;
+    }
 }
 
 export class RequestTrackingSnoowrap extends ExtendedSnoowrap {

src/Utils/SnoowrapUtils.ts

@@ -15,6 +15,7 @@ import {
     asStrongSubredditState,
     asSubmission,
     convertSubredditsRawToStrong,
+    formatNumber,
     getActivityAuthorName,
     getActivitySubredditName,
     isStrongSubredditState, isSubmission,
@@ -22,7 +23,7 @@ import {
     normalizeName,
     parseDurationValToDuration,
     parseRedditEntity,
-    parseRuleResultsToMarkdownSummary, removeUndefinedKeys,
+    parseResultsToMarkdownSummary, removeUndefinedKeys,
     subredditStateIsNameOnly,
     toStrongSubredditState,
     truncateStringToLength,
@@ -34,8 +35,14 @@ import {URL} from "url";
 import {isStatusError, MaybeSeriousErrorWithCause, SimpleError} from "./Errors";
 import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
 import {StrongSubredditCriteria, SubredditCriteria} from "../Common/Infrastructure/Filters/FilterCriteria";
-import {DurationVal} from "../Common/Infrastructure/Atomic";
+import {DurationVal, GenericContentTemplateData} from "../Common/Infrastructure/Atomic";
 import {ActivityWindowCriteria} from "../Common/Infrastructure/ActivityWindow";
+import {
+    SnoowrapActivity,
+    SubredditActivityAbsoluteBreakdown,
+    SubredditActivityBreakdown, SubredditActivityBreakdownByType
+} from "../Common/Infrastructure/Reddit";
+import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
 
 export const BOT_LINK = 'https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot';
 
@@ -116,67 +123,187 @@ export const isSubreddit = async (subreddit: Subreddit, stateCriteria: Subreddit
     })() as boolean;
 }
 
-export const renderContent = async (template: string, data: (Submission | Comment), ruleResults: RuleResultEntity[] = [], usernotes: UserNotes) => {
-    const conditional: any = {};
-    if(data.can_mod_post) {
-        conditional.reports = data.num_reports;
-        conditional.modReports = data.mod_reports.length;
-        conditional.userReports = data.user_reports.length;
-    }
-    if(asSubmission(data)) {
-        conditional.nsfw = data.over_18;
-        conditional.spoiler = data.spoiler;
-        conditional.op = true;
-        conditional.upvoteRatio = `${data.upvote_ratio * 100}%`;
-    } else {
-        conditional.op = data.is_submitter;
-    }
-    const templateData: any = {
-        kind: data instanceof Submission ? 'submission' : 'comment',
-        author: await data.author.name,
-        votes: data.score,
-        age: dayjs.duration(dayjs().diff(dayjs.unix(data.created))).humanize(),
-        permalink: `https://reddit.com${data.permalink}`,
+const renderContentCommentTruncate = truncateStringToLength(50);
+const shortTitleTruncate = truncateStringToLength(15);
+
+export interface TemplateContext {
+    usernotes?: UserNotes
+    check?: string
+    manager?: string
+    ruleResults?: RuleResultEntity[]
+    actionResults?: ActionResultEntity[]
+    activity?: SnoowrapActivity
+    [key: string]: any
+}
+
+export const renderContent = async (template: string, data: TemplateContext = {}) => {
+    const {
+        usernotes,
+        ruleResults,
+        actionResults,
+        activity,
+        ...restContext
+    } = data;
+
+    let view: GenericContentTemplateData = {
         botLink: BOT_LINK,
-        ...conditional
+        ...restContext
+    };
+
+    if(activity !== undefined) {
+        const conditional: any = {};
+        if (activity.can_mod_post) {
+            conditional.reports = activity.num_reports;
+            conditional.modReports = activity.mod_reports.length;
+            conditional.userReports = activity.user_reports.length;
+        }
+        if (asSubmission(activity)) {
+            conditional.nsfw = activity.over_18;
+            conditional.spoiler = activity.spoiler;
+            conditional.op = true;
+            conditional.upvoteRatio = `${activity.upvote_ratio * 100}%`;
+        } else {
+            conditional.op = activity.is_submitter;
+        }
+
+        const subreddit = activity.subreddit.display_name;
+        const permalink =  `https://reddit.com${activity.permalink}`;
+
+        view.modmailLink = `https://www.reddit.com/message/compose?to=%2Fr%2F${subreddit}&message=${encodeURIComponent(permalink)}`;
+
+        const templateData: any = {
+            kind: activity instanceof Submission ? 'submission' : 'comment',
+            // @ts-ignore
+            author: getActivityAuthorName(await activity.author),
+            votes: activity.score,
+            age: dayjs.duration(dayjs().diff(dayjs.unix(activity.created))).humanize(),
+            permalink,
+            id: activity.name,
+            subreddit,
+            ...conditional
+        }
+        if (template.includes('{{item.notes') && usernotes !== undefined) {
+            // we need to get notes
+            const notesData = await usernotes.getUserNotes(activity.author);
+            // return usable notes data with some stats
+            const current = notesData.length > 0 ? notesData[notesData.length - 1] : undefined;
+            // group by type
+            const grouped = notesData.reduce((acc: any, x) => {
+                const {[x.noteType]: nt = []} = acc;
+                return Object.assign(acc, {[x.noteType]: nt.concat(x)});
+            }, {});
+            templateData.notes = {
+                data: notesData,
+                current,
+                ...grouped,
+            };
+        }
+        if (activity instanceof Submission) {
+            templateData.url = activity.url;
+            templateData.title = activity.title;
+            templateData.shortTitle = shortTitleTruncate(activity.title);
+        } else {
+            templateData.title = renderContentCommentTruncate(activity.body);
+            templateData.shortTitle = shortTitleTruncate(activity.body);
+        }
+
+        view.item = templateData;
     }
-    if (template.includes('{{item.notes')) {
-        // we need to get notes
-        const notesData = await usernotes.getUserNotes(data.author);
-        // return usable notes data with some stats
-        const current = notesData.length > 0 ? notesData[notesData.length - 1] : undefined;
-        // group by type
-        const grouped = notesData.reduce((acc: any, x) => {
-            const {[x.noteType]: nt = []} = acc;
-            return Object.assign(acc, {[x.noteType]: nt.concat(x)});
-        }, {});
-        templateData.notes = {
-            data: notesData,
-            current,
-            ...grouped,
+
+
+    if(ruleResults !== undefined) {
+        view = {
+            ...view,
+            ...parseRuleResultForTemplate(ruleResults)
+        }
+    }
+
+    if(actionResults !== undefined) {
+        view = {
+            ...view,
+            ...parseActionResultForTemplate(actionResults)
+        }
+    }
+
+    const rendered = Mustache.render(template, view) as string;
+    return he.decode(rendered);
+}
+
+export const parseActionResultForTemplate = (actionResults: ActionResultEntity[] = []) => {
+    // normalize rule names and map context data
+    // NOTE: we are relying on users to use unique names for action. If they don't only the last action run of kind X will have its results here
+    const normalizedActionResults = actionResults.reduce((acc: object, actionResult) => {
+        const {
+            success,
+            data:{
+                ...restData
+            } = {},
+            result,
+        } = actionResult;
+        let name = actionResult.premise.name;
+        const kind = actionResult.premise.kind.name;
+        if(name === undefined || name === null) {
+            name = kind;
+        }
+        let formattedData: any = {};
+        // remove all non-alphanumeric characters (spaces, dashes, underscore) and set to lowercase
+        // we will set this as the rule property name to make it easy to access results from mustache template
+        const normalName = normalizeName(name);
+        return {
+            ...acc, [normalName]: {
+                kind,
+                success,
+                result,
+                ...restData,
+                ...formattedData,
+            }
         };
-    }
-    if (data instanceof Submission) {
-        templateData.url = data.url;
-        templateData.title = data.title;
-    }
+    }, {});
+
+    return {
+        actionSummary: parseResultsToMarkdownSummary(actionResults),
+        actions: normalizedActionResults
+    };
+}
+
+export const parseRuleResultForTemplate = (ruleResults: RuleResultEntity[] = []) => {
+
     // normalize rule names and map context data
     // NOTE: we are relying on users to use unique names for rules. If they don't only the last rule run of kind X will have its results here
     const normalizedRuleResults = ruleResults.reduce((acc: object, ruleResult) => {
         const {
-            //name,
             triggered,
-            data = {},
+            data:{
+                subredditBreakdown,
+                ...restData
+            } = {},
             result,
-            // premise: {
-            //     kind
-            // }
         } = ruleResult;
         let name = ruleResult.premise.name;
         const kind = ruleResult.premise.kind.name;
         if(name === undefined || name === null) {
             name = kind;
         }
+        let formattedData: any = {};
+        if (subredditBreakdown !== undefined) {
+            // format breakdown for markdown
+            if (Array.isArray(subredditBreakdown)) {
+                const bdArr = subredditBreakdown as SubredditActivityBreakdown[];
+                formattedData.subredditBreakdownFormatted = formatSubredditBreakdownAsMarkdownList(bdArr);
+            } else {
+                const bd = subredditBreakdown as SubredditActivityBreakdownByType;
+
+                // default to total
+                formattedData.subredditBreakdownFormatted = formatSubredditBreakdownAsMarkdownList(bd.total);
+
+                const formatted = Object.entries((bd)).reduce((acc: { [key: string]: string }, curr) => {
+                    const [name, breakdownData] = curr;
+                    acc[`${name}Formatted`] = formatSubredditBreakdownAsMarkdownList(breakdownData);
+                    return acc;
+                }, {});
+                formattedData.subredditBreakdown = {...bd, ...formatted};
+            }
+        }
         // remove all non-alphanumeric characters (spaces, dashes, underscore) and set to lowercase
         // we will set this as the rule property name to make it easy to access results from mustache template
         const normalName = normalizeName(name);
@@ -185,14 +312,16 @@ export const renderContent = async (template: string, data: (Submission | Commen
                 kind,
                 triggered,
                 result,
-                ...data,
+                ...restData,
+                ...formattedData,
             }
         };
     }, {});
 
-    const view = {item: templateData, ruleSummary: parseRuleResultsToMarkdownSummary(ruleResults), rules: normalizedRuleResults};
-    const rendered = Mustache.render(template, view) as string;
-    return he.decode(rendered);
+    return {
+        ruleSummary: parseResultsToMarkdownSummary(ruleResults),
+        rules: normalizedRuleResults
+    };
 }
 
 export interface ItemContent {
@@ -382,3 +511,58 @@ export const getAuthorHistoryAPIOptions = (val: any) => {
 
     return opts;
 }
+
+export const getSubredditBreakdown = (activities: SnoowrapActivity[] = []): SubredditActivityBreakdown[] => {
+    if(activities.length === 0) {
+        return [];
+    }
+
+    const total = activities.length;
+
+    const countBd = activities.reduce((acc: { [key: string]: number }, curr) => {
+        const subName = curr.subreddit.display_name;
+        if (acc[subName] === undefined) {
+            acc[subName] = 0;
+        }
+        acc[subName]++;
+
+        return acc;
+    }, {});
+
+    const breakdown: SubredditActivityBreakdown[] = Object.entries(countBd).reduce((acc, curr) => {
+        const [name, count] = curr;
+        return acc.concat(
+            {
+                name,
+                count,
+                percent: Number.parseFloat(formatNumber((count / total) * 100))
+            }
+        );
+    }, ([] as SubredditActivityBreakdown[]));
+
+    return breakdown;
+}
+
+export const getSubredditBreakdownByActivityType = (activities: SnoowrapActivity[]): SubredditActivityBreakdownByType => {
+
+    return {
+        total: getSubredditBreakdown(activities),
+        submission: getSubredditBreakdown(activities.filter(x => x instanceof Submission)),
+        comment: getSubredditBreakdown(activities.filter(x => x instanceof Comment)),
+    }
+}
+
+export const formatSubredditBreakdownAsMarkdownList = (data: SubredditActivityBreakdown[] = []): string => {
+    if(data.length === 0) {
+        return '';
+    }
+    data.sort((a, b) => b.count - a.count);
+
+    const bd = data.map(x => {
+        const entity = parseRedditEntity(x.name);
+        const prefixedName = entity.type === 'subreddit' ? `r/${entity.name}` : `u/${entity.name}`;
+        return `* ${prefixedName} - ${x.count} (${x.percent}%)`
+    }).join('\n');
+
+    return `${bd}\n`;
+}

src/Web/Client/CMInstance.ts

@@ -1,13 +1,18 @@
 import {URL} from "url";
 import {Logger} from "winston";
-import {BotInstance, CMInstanceInterface, CMInstanceInterface as CMInterface} from "../interfaces";
 import dayjs from 'dayjs';
 import {BotConnection, LogInfo} from "../../Common/interfaces";
 import normalizeUrl from "normalize-url";
-import {HeartbeatResponse} from "../Common/interfaces";
+import {
+    BotInstance,
+    CMInstanceInterface as CMInterface,
+    CMInstanceInterface,
+    HeartbeatResponse
+} from "../Common/interfaces";
 import jwt from "jsonwebtoken";
 import got from "got";
 import {ErrorWithCause} from "pony-cause";
+import ClientBotInstance from "./ClientBotInstance";
 
 export class CMInstance implements CMInterface {
     friendly?: string;
@@ -24,6 +29,7 @@ export class CMInstance implements CMInterface {
     migrationBlocker?: string
     host: string;
     secret: string;
+    invites: string[] = [];
 
     logger: Logger;
     logs: LogInfo[] = [];
@@ -72,6 +78,7 @@ export class CMInstance implements CMInterface {
             secret: this.secret,
             ranMigrations: this.ranMigrations,
             migrationBlocker: this.migrationBlocker,
+            invites: this.invites,
         }
     }
 
@@ -86,6 +93,16 @@ export class CMInstance implements CMInterface {
         return normalizeUrl(val) == this.normalUrl;
     }
 
+    getToken() {
+        return jwt.sign({
+            data: {
+                machine: true,
+            },
+        }, this.secret, {
+            expiresIn: '1m'
+        });
+    }
+
     updateFromHeartbeat = (resp: HeartbeatResponse, otherFriendlies: string[] = []) => {
         this.operators = resp.operators ?? [];
         this.operatorDisplay = resp.operatorDisplay ?? '';
@@ -101,9 +118,10 @@ export class CMInstance implements CMInterface {
             }
         }
 
-        this.subreddits = resp.subreddits;
-        //@ts-ignore
-        this.bots = resp.bots.map(x => ({...x, instance: this}));
+        this.bots = resp.bots.map(x => new ClientBotInstance(x, this));
+        this.subreddits = this.bots.map(x => x.getSubreddits()).flat(3);
+        this.invites = resp.invites;
+
     }
 
     checkHeartbeat = async (force = false, otherFriendlies: string[] = []) => {
@@ -125,13 +143,7 @@ export class CMInstance implements CMInterface {
         if (shouldCheck) {
             this.logger.debug('Starting Heartbeat check');
             this.lastCheck = dayjs().unix();
-            const machineToken = jwt.sign({
-                data: {
-                    machine: true,
-                },
-            }, this.secret, {
-                expiresIn: '1m'
-            });
+            const machineToken = this.getToken();
 
             try {
                 const resp = await got.get(`${this.normalUrl}/heartbeat`, {

src/Web/Client/ClientBotInstance.ts

@@ -0,0 +1,61 @@
+import {
+    BotInstance,
+    BotInstanceResponse,
+    CMInstanceInterface,
+    ManagerResponse,
+    NormalizedManagerResponse
+} from '../Common/interfaces';
+import {intersect, parseRedditEntity} from "../../util";
+
+export class ClientBotInstance implements BotInstance {
+    instance: CMInstanceInterface;
+    botName: string;
+   // botLink: string;
+    error?: string | undefined;
+    managers: NormalizedManagerResponse[];
+    nanny?: string | undefined;
+    running: boolean;
+
+    constructor(data: BotInstanceResponse, instance: CMInstanceInterface) {
+        this.instance = instance;
+        this.botName = data.botName;
+        //this.botLink = data.botLink;
+        this.error = data.error;
+        this.managers = data.managers.map(x => ({...x, subredditNormal: parseRedditEntity(x.subreddit).name}));
+        this.nanny = data.nanny;
+        this.running = data.running;
+    }
+
+    getManagerNames(): string[] {
+        return this.managers.map(x => x.name);
+    }
+
+    getSubreddits(normalized = true): string[] {
+        return normalized ? this.managers.map(x => x.subredditNormal) : this.managers.map(x => x.subreddit);
+    }
+
+    getAccessibleSubreddits(user: string, subreddits: string[] = []): string[] {
+        const normalSubs = subreddits.map(x => parseRedditEntity(x).name);
+        return Array.from(new Set([...this.getGuestSubreddits(user), ...intersect(normalSubs, this.getSubreddits())]));
+    }
+
+    getGuestManagers(user: string): NormalizedManagerResponse[] {
+        const louser = user.toLowerCase();
+        return this.managers.filter(x => x.guests.map(y => y.name.toLowerCase()).includes(louser));
+    }
+
+    getGuestSubreddits(user: string): string[] {
+        return this.getGuestManagers(user).map(x => x.subredditNormal);
+    }
+
+    canUserAccessBot(user: string, subreddits: string[] = []) {
+        return this.getAccessibleSubreddits(user, subreddits).length > 0;
+    }
+
+    canUserAccessSubreddit(subreddit: string, user: string, subreddits: string[] = []): boolean {
+       return this.getAccessibleSubreddits(user, subreddits).includes(parseRedditEntity(subreddit).name);
+    }
+
+}
+
+export default ClientBotInstance;

src/Web/Client/StorageProvider.ts

@@ -7,11 +7,8 @@ import {Cache} from "cache-manager";
 import CacheManagerStore from 'express-session-cache-manager'
 import {CacheOptions} from "../../Common/interfaces";
 import {Brackets, DataSource, IsNull, LessThanOrEqual, Repository} from "typeorm";
-import {DateUtils} from 'typeorm/util/DateUtils';
 import {ClientSession} from "../../Common/WebEntities/ClientSession";
-import dayjs from "dayjs";
 import {Logger} from "winston";
-import {Invite} from "../../Common/WebEntities/Invite";
 import {WebSetting} from "../../Common/WebEntities/WebSetting";
 import {ErrorWithCause} from "pony-cause";
 
@@ -29,12 +26,6 @@ export type TypeormStoreOptions = Partial<SessionOptions & {
 interface IWebStorageProvider {
     createSessionStore(options?: CacheManagerStoreOptions | TypeormStoreOptions): Store
 
-    inviteGet(id: string): Promise<InviteData | undefined>
-
-    inviteDelete(id: string): Promise<void>
-
-    inviteCreate(id: string, data: InviteData): Promise<InviteData>
-
     getSessionSecret(): Promise<string | undefined>
 
     setSessionSecret(secret: string): Promise<void>
@@ -42,43 +33,25 @@ interface IWebStorageProvider {
 
 interface StorageProviderOptions {
     logger: Logger
-    invitesMaxAge?: number
     loggerLabels?: string[]
 }
 
 abstract class StorageProvider implements IWebStorageProvider {
 
-    invitesMaxAge?: number
     logger: Logger;
 
     protected constructor(data: StorageProviderOptions) {
         const {
             logger,
-            invitesMaxAge,
             loggerLabels = [],
         } = data;
-        this.invitesMaxAge = invitesMaxAge;
         this.logger = logger.child({labels: ['Web', 'Storage', ...loggerLabels]}, mergeArr);
     }
 
-    protected abstract getInvite(id: string): Promise<InviteData | undefined | null>;
-
-    async inviteGet(id: string) {
-        const data = await this.getInvite(id);
-        if (data === undefined || data === null) {
-            return undefined;
-        }
-        return data;
-    }
-
     abstract createSessionStore(options?: CacheManagerStoreOptions | TypeormStoreOptions): Store;
 
     abstract getSessionSecret(): Promise<string | undefined>;
 
-    abstract inviteCreate(id: string, data: InviteData): Promise<InviteData>;
-
-    abstract inviteDelete(id: string): Promise<void>;
-
     abstract setSessionSecret(secret: string): Promise<void>;
 }
 
@@ -100,19 +73,6 @@ export class CacheStorageProvider extends StorageProvider {
         return new CacheManagerStore(this.cache, {prefix: 'sess:'});
     }
 
-    protected async getInvite(id: string) {
-        return await this.cache.get(`invite:${id}`) as InviteData | undefined | null;
-    }
-
-    async inviteCreate(id: string, data: InviteData): Promise<InviteData> {
-        await this.cache.set(`invite:${id}`, data, {ttl: (this.invitesMaxAge ?? 0) * 1000});
-        return data;
-    }
-
-    async inviteDelete(id: string): Promise<void> {
-        return await this.cache.del(`invite:${id}`);
-    }
-
     async getSessionSecret() {
         const val = await this.cache.get(`sessionSecret`);
         if (val === null || val === undefined) {
@@ -130,14 +90,12 @@ export class CacheStorageProvider extends StorageProvider {
 export class DatabaseStorageProvider extends StorageProvider {
 
     database: DataSource;
-    inviteRepo: Repository<Invite>;
     webSettingRepo: Repository<WebSetting>;
     clientSessionRepo: Repository<ClientSession>
 
     constructor(data: { database: DataSource } & StorageProviderOptions) {
         super(data);
         this.database = data.database;
-        this.inviteRepo = this.database.getRepository(Invite);
         this.webSettingRepo = this.database.getRepository(WebSetting);
         this.clientSessionRepo = this.database.getRepository(ClientSession);
         this.logger.debug('Using DATABASE');
@@ -147,26 +105,6 @@ export class DatabaseStorageProvider extends StorageProvider {
         return new TypeormStore(options).connect(this.clientSessionRepo)
     }
 
-    protected async getInvite(id: string): Promise<InviteData | undefined | null> {
-        const qb = this.inviteRepo.createQueryBuilder('invite');
-        return await qb
-            .andWhere({id})
-            .andWhere(new Brackets((qb) => {
-                    qb.where({_expiresAt: LessThanOrEqual(DateUtils.mixedDateToDatetimeString(dayjs().toDate()))})
-                        .orWhere({_expiresAt: IsNull()})
-                })
-            ).getOne();
-    }
-
-    async inviteCreate(id: string, data: InviteData): Promise<InviteData> {
-        await this.inviteRepo.save(new Invite({...data, id}));
-        return data;
-    }
-
-    async inviteDelete(id: string): Promise<void> {
-        await this.inviteRepo.delete(id);
-    }
-
     async getSessionSecret(): Promise<string | undefined> {
         try {
             const dbSessionSecret = await this.webSettingRepo.findOneBy({name: 'sessionSecret'});

src/Web/Client/index.ts

@@ -13,7 +13,7 @@ import {
     CheckSummary,
     RunResult,
     ActionedEvent,
-    ActionResult, RuleResult, EventActivity
+    ActionResult, RuleResult, EventActivity, OperatorConfigWithFileContext
 } from "../../Common/interfaces";
 import {
     buildCachePrefix,
@@ -38,7 +38,6 @@ import sharedSession from "express-socket.io-session";
 import dayjs from "dayjs";
 import httpProxy from 'http-proxy';
 import {arrayMiddle, booleanMiddle} from "../Common/middleware";
-import {BotInstance, CMInstanceInterface} from "../interfaces";
 import { URL } from "url";
 import {MESSAGE} from "triple-beam";
 import Autolinker from "autolinker";
@@ -57,6 +56,8 @@ import {MigrationService} from "../../Common/MigrationService";
 import {RuleResultEntity} from "../../Common/Entities/RuleResultEntity";
 import {RuleSetResultEntity} from "../../Common/Entities/RuleSetResultEntity";
 import { PaginationAwareObject } from "../Common/util";
+import {BotInstance, BotStatusResponse, CMInstanceInterface, InviteData} from "../Common/interfaces";
+import {open} from "fs/promises";
 
 const emitter = new EventEmitter();
 
@@ -151,10 +152,10 @@ const availableLevels = ['error', 'warn', 'info', 'verbose', 'debug'];
 
 let webLogs: LogInfo[] = [];
 
-const webClient = async (options: OperatorConfig) => {
+const webClient = async (options: OperatorConfigWithFileContext) => {
     const {
         operator: {
-            name,
+            name: operatorName,
             display,
         },
         userAgent: uaFragment,
@@ -169,9 +170,6 @@ const webClient = async (options: OperatorConfig) => {
             port,
             storage: webStorage = 'database',
             caching,
-            invites: {
-              maxAge: invitesMaxAge,
-            },
             session: {
                 secret: sessionSecretFromConfig,
                 maxAge: sessionMaxAge,
@@ -179,16 +177,14 @@ const webClient = async (options: OperatorConfig) => {
             },
             maxLogs,
             clients,
-            credentials: {
-                clientId,
-                clientSecret,
-                redirectUri
-            },
+            credentials,
             operators = [],
         },
         //database
     } = options;
 
+    let clientCredentials = credentials;
+
     let sessionSecretSynced = false;
 
     const userAgent = getUserAgent(`web:contextBot:{VERSION}{FRAG}:dashboard`, uaFragment);
@@ -237,7 +233,7 @@ const webClient = async (options: OperatorConfig) => {
         }
     });
 
-    const storage = webStorage === 'database' ? new DatabaseStorageProvider({database, invitesMaxAge, logger}) : new CacheStorageProvider({...caching, invitesMaxAge, logger});
+    const storage = webStorage === 'database' ? new DatabaseStorageProvider({database, logger}) : new CacheStorageProvider({...caching, logger});
 
     let sessionSecret: string;
     if (sessionSecretFromConfig !== undefined) {
@@ -297,9 +293,9 @@ const webClient = async (options: OperatorConfig) => {
             }
             const client = await ExtendedSnoowrap.fromAuthCode({
                 userAgent,
-                clientId,
-                clientSecret,
-                redirectUri: redirectUri as string,
+                clientId: clientCredentials.clientId,
+                clientSecret: clientCredentials.clientSecret,
+                redirectUri: clientCredentials.redirectUri as string,
                 code: code as string,
             });
             const user = await client.getMe().name as string;
@@ -315,7 +311,7 @@ const webClient = async (options: OperatorConfig) => {
 
     let sessionStoreProvider = storage;
     if(sessionStorage !== webStorage) {
-        sessionStoreProvider = sessionStorage === 'database' ? new DatabaseStorageProvider({database, invitesMaxAge, logger, loggerLabels: ['Session']}) : new CacheStorageProvider({...caching, invitesMaxAge, logger, loggerLabels: ['Session']});
+        sessionStoreProvider = sessionStorage === 'database' ? new DatabaseStorageProvider({database, logger, loggerLabels: ['Session']}) : new CacheStorageProvider({...caching, logger, loggerLabels: ['Session']});
     }
     const sessionObj = session({
         cookie: {
@@ -349,10 +345,50 @@ const webClient = async (options: OperatorConfig) => {
         }
     }
 
+    app.postAsync('/init', async (req, res, next) => {
+        if (clientCredentials.clientId === undefined || clientCredentials.clientSecret === undefined) {
+            const {
+                redirect = '',
+                clientId = '',
+                clientSecret = '',
+                operator = '',
+            } = req.body as any;
+            if (redirect === null || redirect.trim() === '') {
+                return res.status(400).send('redirect cannot be empty');
+            }
+            if (clientId === null || clientId.trim() === '') {
+                return res.status(400).send('clientId cannot be empty');
+            }
+            if (clientSecret === null || clientSecret.trim() === '') {
+                return res.status(400).send('clientSecret cannot be empty');
+            }
+            if(operatorName === undefined) {
+                return res.status(400).send('operator cannot be empty');
+            }
+            options.fileConfig.document.setWebCredentials({redirectUri: redirect.trim(), clientId: clientId.trim(), clientSecret: clientSecret.trim()});
+            if(operators.length === 0 && operator !== '') {
+                options.fileConfig.document.setOperator(parseRedditEntity(operator, 'user').name);
+            }
+            const handle = await open(options.fileConfig.document.location as string, 'w');
+            await handle.writeFile(options.fileConfig.document.toString());
+            await handle.close();
+
+            clientCredentials = {
+                clientId,
+                clientSecret,
+                redirectUri: redirect
+            }
+
+            return res.status(200).send();
+        } else {
+            return res.status(400).send('Can only do init setup when client credentials do not already exist.');
+        }
+    });
+
     const scopeMiddle = arrayMiddle(['scope']);
     const successMiddle = booleanMiddle([{name: 'closeOnSuccess', defaultVal: undefined, required: false}]);
     app.getAsync('/login', scopeMiddle, successMiddle, async (req, res, next) => {
-        if (redirectUri === undefined) {
+        if (clientCredentials.redirectUri === undefined) {
             return res.render('error', {error: `No <b>redirectUri</b> was specified through environmental variables or program argument. This must be provided in order to use the web interface.`});
         }
         const {query: { scope: reqScopes = [], closeOnSuccess } } = req;
@@ -364,10 +400,13 @@ const webClient = async (options: OperatorConfig) => {
             // @ts-ignore
             req.session.closeOnSuccess = closeOnSuccess;
         }
+        if(clientCredentials.clientId === undefined) {
+            return res.render('init', { operators: operators.join(',') });
+        }
         const authUrl = Snoowrap.getAuthUrl({
-            clientId,
+            clientId: clientCredentials.clientId,
             scope: scope,
-            redirectUri: redirectUri as string,
+            redirectUri: clientCredentials.redirectUri as string,
             permanent: false,
             state: req.session.state,
         });
@@ -394,10 +433,10 @@ const webClient = async (options: OperatorConfig) => {
                 return res.render('error', {error: errContent});
             }
             // @ts-ignore
-            const invite = await storage.inviteGet(req.session.inviteId);
+            const invite = req.session.invite as InviteData; //await storage.inviteGet(req.session.inviteId);
             if(invite === undefined) {
                 // @ts-ignore
-                return res.render('error', {error: `Could not find invite with id ${req.session.inviteId}?? This should happen!`});
+                return res.render('error', {error: `Could not find invite in session?? This should happen!`});
             }
             const client = await Snoowrap.fromAuthCode({
                 userAgent,
@@ -409,40 +448,36 @@ const webClient = async (options: OperatorConfig) => {
             // @ts-ignore
             const user = await client.getMe();
             const userName = `u/${user.name}`;
+
             // @ts-ignore
-            await storage.inviteDelete(req.session.inviteId);
+            //await storage.inviteDelete(req.session.inviteId);
             let data: any = {
                 accessToken: client.accessToken,
                 refreshToken: client.refreshToken,
                 userName,
             };
-            if(invite.instance !== undefined) {
-                const bot = cmInstances.find(x => x.getName() === invite.instance);
-                if(bot !== undefined) {
-                    const botPayload: any = {
-                        overwrite: invite.overwrite === true,
-                        name: userName,
-                        credentials: {
-                            reddit: {
-                                accessToken: client.accessToken,
-                                refreshToken: client.refreshToken,
-                                clientId: invite.clientId,
-                                clientSecret: invite.clientSecret,
-                            }
-                        }
-                    };
-                    if(invite.subreddits !== undefined && invite.subreddits.length > 0) {
-                        botPayload.subreddits =  {names: invite.subreddits};
+
+            // @ts-ignore
+            const inviteId = invite.id as string;
+
+            // @ts-ignore
+            const botAddResult: any = await addBot(inviteId, {
+                invite: inviteId,
+                credentials: {
+                    reddit: {
+                        accessToken: client.accessToken,
+                        refreshToken: client.refreshToken,
+                        clientId: invite.clientId,
+                        clientSecret: invite.clientSecret,
                     }
-                    const botAddResult: any = await addBot(bot, {name: invite.creator}, botPayload);
-                    // stored
-                    // success
-                    data = {...data, ...botAddResult};
-                    // @ts-ignore
-                    req.session.destroy();
-                    req.logout();
-                }
-            }
+                },
+                name: userName,
+            });
+            data = {...data, ...botAddResult};
+
+            // @ts-ignore
+            req.session.destroy();
+            req.logout();
             return res.render('callback', data);
         } else {
             return next();
@@ -503,31 +538,100 @@ const webClient = async (options: OperatorConfig) => {
         }
     }
 
-    app.getAsync('/auth/helper', helperAuthed, (req, res) => {
+    const createUserToken = async (req: express.Request, res: express.Response, next: Function) => {
+        req.token = createToken(req.instance as CMInstanceInterface, req.user);
+        next();
+    }
+
+    const instanceWithPermissions = async (req: express.Request, res: express.Response, next: Function) => {
+        delete req.session.botId;
+        delete req.session.authBotId;
+
+        const msg = 'Bot does not exist or you do not have permission to access it';
+        const instance = cmInstances.find(x => x.getName() === req.query.instance);
+        if (instance === undefined) {
+            return res.status(404).render('error', {error: msg});
+        }
+
+        if (!req.user?.clientData?.webOperator && !req.user?.canAccessInstance(instance)) {
+            return res.status(404).render('error', {error: msg});
+        }
+
+        if (req.params.subreddit !== undefined && !req.user?.canAccessSubreddit(instance,req.params.subreddit)) {
+            return res.status(404).render('error', {error: msg});
+        }
+        req.instance = instance;
+        req.session.botId = instance.getName();
+        req.session.authBotId = instance.getName();
+        return next();
+    }
+
+    const instancesViewData = async (req: express.Request, res: express.Response, next: Function) => {
+
+        const user = req.user as Express.User;
+        const instance = req.instance as CMInstance;
+
+        const shownInstances = cmInstances.reduce((acc: CMInstance[], curr) => {
+            const isBotOperator = user?.isInstanceOperator(curr);
+            if(user?.clientData?.webOperator) {
+                // @ts-ignore
+                return acc.concat({...curr.getData(), canAccessLocation: true, isOperator: isBotOperator});
+            }
+            if(!isBotOperator && !req.user?.canAccessInstance(curr)) {
+                return acc;
+            }
+            // @ts-ignore
+            return acc.concat({...curr.getData(), canAccessLocation: isBotOperator, isOperator: isBotOperator, botId: curr.getName()});
+        },[]);
+
+        // @ts-ignore
+        req.instancesViewData = {
+            instances: shownInstances,
+            instanceId: instance.getName()
+        };
+
+        next();
+    }
+
+    app.getAsync('/auth/helper', helperAuthed, instanceWithPermissions, instancesViewData, (req, res) => {
         return res.render('helper', {
-            redirectUri,
-            clientId,
-            clientSecret,
+            redirectUri: clientCredentials.redirectUri,
+            clientId: clientCredentials.clientId,
+            clientSecret: clientCredentials.clientSecret,
             token: req.isAuthenticated() && req.user?.clientData?.webOperator ? token : undefined,
-            instances: cmInstances.filter(x => req.user?.isInstanceOperator(x)).map(x => x.getName()),
+            // @ts-ignore
+            ...req.instancesViewData,
         });
     });
 
-    app.getAsync('/auth/invite', async (req, res) => {
-        const {invite: inviteId} = req.query;
+    app.getAsync('/auth/invite/:inviteId', async (req, res) => {
+        const {inviteId} = req.params;
 
-        if(inviteId === undefined) {
+        if (inviteId === undefined) {
             return res.render('error', {error: '`invite` param is missing from URL'});
         }
-        const invite = await storage.inviteGet(inviteId as string);
-        if(invite === undefined || invite === null) {
+
+        const cmInstance = cmInstances.find(x => x.invites.includes(inviteId));
+        if (cmInstance === undefined) {
             return res.render('error', {error: 'Invite with the given id does not exist'});
         }
 
-        return res.render('invite', {
-            permissions: JSON.stringify(invite.permissions || []),
-            invite: inviteId,
-        });
+        try {
+            const invite = await got.get(`${cmInstance.normalUrl}/invites/${inviteId}`, {
+                headers: {
+                    'Authorization': `Bearer ${cmInstance.getToken()}`,
+                }
+            }).json() as InviteData;
+
+            return res.render('invite', {
+                guests: invite.guests !== undefined && invite.guests !== null && invite.guests.length > 0 ? invite.guests.join(',') : '',
+                permissions: JSON.stringify(invite.permissions || []),
+                invite: inviteId,
+            });
+        } catch (err: any) {
+            cmInstance.logger.error(new ErrorWithCause(`Retrieving invite failed`, {cause: err}));
+            return res.render('error', {error: 'An error occurred while validating your invite and has been logged. Let the person who gave you this invite know! Sorry about that.'})
+        }
     });
 
     app.postAsync('/auth/create', helperAuthed, async (req: express.Request, res: express.Response) => {
@@ -538,15 +642,15 @@ const webClient = async (options: OperatorConfig) => {
             redirect: redir,
             instance,
             subreddits,
-            code,
+            guests: guestsVal,
         } = req.body as any;
 
-        const cid = ci || clientId;
+        const cid = ci || clientCredentials.clientId;
         if(cid === undefined || cid.trim() === '') {
             return res.status(400).send('clientId is required');
         }
 
-        const ced = ce || clientSecret;
+        const ced = ce || clientCredentials.clientSecret;
         if(ced === undefined || ced.trim() === '') {
             return res.status(400).send('clientSecret is required');
         }
@@ -555,32 +659,74 @@ const webClient = async (options: OperatorConfig) => {
             return res.status(400).send('redirectUrl is required');
         }
 
-        const inviteId = code || nanoid(20);
-        await storage.inviteCreate(inviteId, {
+        let guestArr = [];
+        if(typeof guestsVal === 'string') {
+            guestArr = guestsVal.split(',');
+        } else if(Array.isArray(guestsVal)) {
+            guestArr = guestsVal;
+        }
+        guestArr = guestArr.filter(x => x.trim() !== '').map(x => parseRedditEntity(x, 'user').name);
+
+        const inviteData = {
             permissions,
-            clientId: (ci || clientId).trim(),
-            clientSecret: (ce || clientSecret).trim(),
+            clientId: (ci || clientCredentials.clientId).trim(),
+            clientSecret: (ce || clientCredentials.clientSecret).trim(),
             redirectUri: redir.trim(),
             instance,
             subreddits: subreddits.trim() === '' ? [] : subreddits.split(',').map((x: string) => parseRedditEntity(x).name),
             creator: (req.user as Express.User).name,
-        });
-        return res.send(inviteId);
+            guests: guestArr.length > 0 ? guestArr : undefined
+        };
+        const cmInstance = cmInstances.find(x => x.friendly === instance);
+        if(cmInstance === undefined) {
+            return res.status(400).send(`No instance found with name "${instance}"`);
+        }
+
+        const token = createToken(cmInstance, req.user);
+
+        try {
+            const resp = await got.post(`${cmInstance.normalUrl}/invites`, {
+                headers: {
+                    'Authorization': `Bearer ${token}`,
+                },
+                json: inviteData,
+            }).json() as any;
+            cmInstance.invites.push(resp.id);
+            return res.send(resp.id);
+        } catch (err: any) {
+            cmInstance.logger.error(new ErrorWithCause(`Could not create bot invite.`, {cause: err}));
+            return res.status(400).send(`Error while creating invite: ${err.message}`);
+        }
     });
 
-    app.getAsync('/auth/init', async (req: express.Request, res: express.Response) => {
-        const {invite: inviteId} = req.query;
+    app.getAsync('/auth/init/:inviteId', async (req: express.Request, res: express.Response) => {
+        const { inviteId } = req.params;
         if(inviteId === undefined) {
             return res.render('error', {error: '`invite` param is missing from URL'});
         }
-        const invite = await storage.inviteGet(inviteId as string);
-        if(invite === undefined || invite === null) {
+
+        const cmInstance = cmInstances.find(x => x.invites.includes(inviteId));
+        if (cmInstance === undefined) {
             return res.render('error', {error: 'Invite with the given id does not exist'});
         }
 
+        let invite: InviteData;
+
+        try {
+            invite = await got.get(`${cmInstance.normalUrl}/invites/${inviteId}`, {
+                headers: {
+                    'Authorization': `Bearer ${cmInstance.getToken()}`,
+                }
+            }).json() as InviteData;
+
+        } catch (err: any) {
+            cmInstance.logger.error(new ErrorWithCause(`Retrieving invite failed`, {cause: err}));
+            return res.render('error', {error: 'An error occurred while validating your invite and has been logged. Let the person who gave you this invite know! Sorry about that.'})
+        }
+
         req.session.state = `bot_${randomId()}`;
         // @ts-ignore
-        req.session.inviteId = inviteId;
+        req.session.invite = invite;
 
         const scope = Object.entries(invite.permissions).reduce((acc: string[], curr) => {
             const [k, v] = curr as unknown as [string, boolean];
@@ -620,31 +766,6 @@ const webClient = async (options: OperatorConfig) => {
     }
     logger.info(`Web UI started: http://localhost:${port}`, {label: ['Web']});
 
-    const instanceWithPermissions = async (req: express.Request, res: express.Response, next: Function) => {
-        delete req.session.botId;
-        delete req.session.authBotId;
-
-        const msg = 'Bot does not exist or you do not have permission to access it';
-        const instance = cmInstances.find(x => x.getName() === req.query.instance);
-        if (instance === undefined) {
-            return res.status(404).render('error', {error: msg});
-        }
-
-        if (!req.user?.clientData?.webOperator && !req.user?.canAccessInstance(instance)) {
-            return res.status(404).render('error', {error: msg});
-        }
-
-        if (req.params.subreddit !== undefined && !req.user?.isInstanceOperator(instance) && !req.user?.subreddits.includes(req.params.subreddit)) {
-            return res.status(404).render('error', {error: msg});
-        }
-        req.instance = instance;
-        req.session.botId = instance.getName();
-        if(req.user?.canAccessInstance(instance)) {
-            req.session.authBotId = instance.getName();
-        }
-        return next();
-    }
-
 
     const botWithPermissions = (required: boolean = false, setDefault: boolean = false) => async (req: express.Request, res: express.Response, next: Function) => {
 
@@ -680,7 +801,7 @@ const webClient = async (options: OperatorConfig) => {
                 return res.status(404).render('error', {error: msg});
             }
 
-            if (req.params.subreddit !== undefined && !req.user?.isInstanceOperator(instance) && !req.user?.subreddits.includes(req.params.subreddit)) {
+            if (req.params.subreddit !== undefined && !req.user?.canAccessSubreddit(instance,req.params.subreddit)) {
                 return res.status(404).render('error', {error: msg});
             }
             req.bot = botInstance;
@@ -689,11 +810,6 @@ const webClient = async (options: OperatorConfig) => {
         next();
     }
 
-    const createUserToken = async (req: express.Request, res: express.Response, next: Function) => {
-        req.token = createToken(req.instance as CMInstanceInterface, req.user);
-        next();
-    }
-
     const defaultSession = (req: express.Request, res: express.Response, next: Function) => {
         if(req.session.limit === undefined) {
             req.session.limit = 200;
@@ -714,7 +830,14 @@ const webClient = async (options: OperatorConfig) => {
     // botUserRouter.use([ensureAuthenticated, defaultSession, botWithPermissions, createUserToken]);
     // app.use(botUserRouter);
 
-    app.useAsync('/api/', [ensureAuthenticated, defaultSession, instanceWithPermissions, botWithPermissions(false), createUserToken], (req: express.Request, res: express.Response) => {
+    // proxy.on('proxyReq', (req) => {
+    //    logger.debug(`Got proxy request: ${req.path}`);
+    // });
+    // proxy.on('proxyRes', (proxyRes, req, res) => {
+    //     logger.debug(`Got proxy response: ${res.statusCode} for ${req.url}`);
+    // });
+
+    app.useAsync('/api/', [ensureAuthenticatedApi, defaultSession, instanceWithPermissions, botWithPermissions(false), createUserToken], (req: express.Request, res: express.Response) => {
         req.headers.Authorization = `Bearer ${req.token}`
 
         const instance = req.instance as CMInstanceInterface;
@@ -725,6 +848,10 @@ const webClient = async (options: OperatorConfig) => {
                 port: instance.url.port,
             },
             prependPath: false,
+            proxyTimeout: 11000,
+        }, (e: any) => {
+            logger.error(e);
+            res.status(500).send();
         });
     });
 
@@ -739,7 +866,7 @@ const webClient = async (options: OperatorConfig) => {
                 if(x.operators.includes(user.name)) {
                     return true;
                 }
-                return intersect(user.subreddits, x.subreddits).length > 0;
+                return x.bots.some(y => y.canUserAccessBot(user.name, user.subreddits));
             });
 
             if(accessibleInstance === undefined) {
@@ -754,13 +881,13 @@ const webClient = async (options: OperatorConfig) => {
         next();
     }
 
-    const defaultSubreddit = async (req: express.Request, res: express.Response, next: Function) => {
+/*    const defaultSubreddit = async (req: express.Request, res: express.Response, next: Function) => {
         if(req.bot !== undefined && req.query.subreddit === undefined) {
-            const firstAccessibleSub = req.bot.subreddits.find(x => req.user?.isInstanceOperator(req.instance) || req.user?.subreddits.includes(x));
+            const firstAccessibleSub = req.bot.managers.find(x => req.user?.isInstanceOperator(req.instance) || req.user?.subreddits.includes(x));
             req.query.subreddit = firstAccessibleSub;
         }
         next();
-    }
+    }*/
 
     const initHeartbeat = async (req: express.Request, res: express.Response, next: Function) => {
         if(!init) {
@@ -782,19 +909,41 @@ const webClient = async (options: OperatorConfig) => {
     }
 
     const migrationRedirect = async (req: express.Request, res: express.Response, next: Function) => {
+        const user = req.user as Express.User;
         const instance = req.instance as CMInstance;
+
         if(instance.bots.length === 0 && instance?.ranMigrations === false && instance?.migrationBlocker !== undefined) {
+
+            if(!user.isInstanceOperator(instance)) {
+                return res.render('error-authenticated', {
+                    error: `A database migration, which requires manual confirmation by its <strong>Operator</strong>, is required before this CM instance can finish starting up.`,
+                    // @ts-ignore
+                    ...req.instancesViewData
+                })
+            }
+
             return res.render('migrations', {
                 type: 'app',
                 ranMigrations: instance.ranMigrations,
                 migrationBlocker: instance.migrationBlocker,
-                instance: instance.friendly
+                instance: instance.friendly,
+                // @ts-ignore
+                ...req.instancesViewData
             });
         }
         return next();
     };
 
-    app.getAsync('/', [initHeartbeat, redirectBotsNotAuthed, ensureAuthenticated, defaultSession, defaultInstance, instanceWithPermissions, migrationRedirect, botWithPermissions(false, true), createUserToken], async (req: express.Request, res: express.Response) => {
+    const redirectNoBots = async (req: express.Request, res: express.Response, next: Function) => {
+        const i = req.instance as CMInstance;
+        if (i.bots.length === 0) {
+            // assuming user is doing first-time setup and this is the default localhost bot
+            return res.redirect(`/auth/helper?instance=${i.getName()}`);
+        }
+        next();
+    }
+
+    app.getAsync('/', [initHeartbeat, redirectBotsNotAuthed, ensureAuthenticated, defaultSession, defaultInstance, instanceWithPermissions, instancesViewData, migrationRedirect, redirectNoBots, botWithPermissions(false, true), createUserToken], async (req: express.Request, res: express.Response) => {
 
         const user = req.user as Express.User;
         const instance = req.instance as CMInstance;
@@ -803,19 +952,6 @@ const webClient = async (options: OperatorConfig) => {
         const sort = req.session.sort;
         const level = req.session.level;
 
-        const shownInstances = cmInstances.reduce((acc: CMInstance[], curr) => {
-            const isBotOperator = req.user?.isInstanceOperator(curr);
-            if(user?.clientData?.webOperator) {
-                // @ts-ignore
-                return acc.concat({...curr.getData(), canAccessLocation: true, isOperator: isBotOperator});
-            }
-            if(!isBotOperator && !req.user?.canAccessInstance(curr)) {
-                return acc;
-            }
-            // @ts-ignore
-            return acc.concat({...curr.getData(), canAccessLocation: isBotOperator, isOperator: isBotOperator, botId: curr.getName()});
-        },[]);
-
         let resp;
         try {
             resp = await got.get(`${instance.normalUrl}/status`, {
@@ -837,8 +973,8 @@ const webClient = async (options: OperatorConfig) => {
             refreshClient({host: instance.host, secret: instance.secret});
             const isOp = req.user?.isInstanceOperator(instance);
             return res.render('offline', {
-                instances: shownInstances,
-                instanceId: (req.instance as CMInstance).getName(),
+                // @ts-ignore
+                ...req.instancesViewData,
                 isOperator: isOp,
                 // @ts-ignore
                 logs: filterLogs((isOp ? instance.logs : instance.logs.filter(x => x.user === undefined || x.user.includes(req.user.name))), {limit, sort, level}),
@@ -869,11 +1005,31 @@ const webClient = async (options: OperatorConfig) => {
 
         const isOp = req.user?.isInstanceOperator(instance);
 
+        // const bots = resp.bots.map((x: BotStatusResponse) => {
+        //     return {
+        //         ...x,
+        //         subreddits: x.subreddits.map(y => {
+        //            return {
+        //                ...y,
+        //                guests: y.guests.map(z => {
+        //                    const d = z.expiresAt === undefined ? undefined : dayjs(z.expiresAt);
+        //                    return {
+        //                        ...z,
+        //                        relative: d === undefined ? 'Never' : dayjs.duration(d.diff(dayjs())).humanize(),
+        //                        date: d === undefined ? 'Never' : d.format('YYYY-MM-DD HH:mm:ssZ')
+        //                    }
+        //                })
+        //            }
+        //         })
+        //     }
+        // });
+
         res.render('status', {
-            instances: shownInstances,
+            // @ts-ignore
+            ...req.instancesViewData,
             bots: resp.bots,
+            now: dayjs().add(1, 'minute').format('YYYY-MM-DDTHH:mm'),
             botId: (req.instance as CMInstance).getName(),
-            instanceId: (req.instance as CMInstance).getName(),
             isOperator: isOp,
             system: isOp ? {
                 // @ts-ignore
@@ -904,14 +1060,19 @@ const webClient = async (options: OperatorConfig) => {
         });
     });
 
+    app.getAsync('/guest', [ensureAuthenticatedApi, defaultSession, instanceWithPermissions, botWithPermissions(true)], async (req: express.Request, res: express.Response) => {
+        const {subreddit} = req.query as any;
+        return res.status(req.user?.isSubredditGuest(req.bot, subreddit) ? 200 : 403).send();
+    });
+
     app.postAsync('/config', [ensureAuthenticatedApi, defaultSession, instanceWithPermissions, botWithPermissions(true)], async (req: express.Request, res: express.Response) => {
         const {subreddit} = req.query as any;
         const {location, data, reason = 'Updated through CM Web', create = false} = req.body as any;
 
         const client = new ExtendedSnoowrap({
             userAgent,
-            clientId,
-            clientSecret,
+            clientId: clientCredentials.clientId,
+            clientSecret: clientCredentials.clientSecret,
             accessToken: req.user?.clientData?.token
         });
 
@@ -1342,14 +1503,18 @@ const webClient = async (options: OperatorConfig) => {
         }
     }
 
-    const addBot = async (bot: CMInstanceInterface, userPayload: any, botPayload: any) => {
+    const addBot = async (inviteId: string, botPayload: any) => {
+
+        const cmInstance = cmInstances.find(x => x.invites.includes(inviteId));
+        if(cmInstance === undefined) {
+            return {success: false, error: 'Could not determine CM instance to add bot to based on invite id (invite id was not found)'};
+        }
+
         try {
-            const token = createToken(bot, userPayload);
-            const resp = await got.post(`${bot.normalUrl}/bot`, {
-                body: JSON.stringify(botPayload),
+            const resp = await got.post(`${cmInstance.normalUrl}/bot`, {
+                json: botPayload,
                 headers: {
-                    'Content-Type': 'application/json',
-                    'Authorization': `Bearer ${token}`,
+                    'Authorization': `Bearer ${cmInstance.getToken()}`,
                 }
             }).json() as object;
             return {success: true, ...resp};

src/Web/Common/User/CMUser.ts

@@ -18,6 +18,9 @@ abstract class CMUser<Instance, Bot, SubredditEntity> implements IUser {
     public abstract accessibleBots(bots: Bot[]): Bot[]
     public abstract canAccessSubreddit(val: Bot, name: string): boolean;
     public abstract accessibleSubreddits(bot: Bot): SubredditEntity[]
+    public abstract isSubredditGuest(val: Bot, name: string): boolean;
+    public abstract isSubredditMod(val: Bot, name: string): boolean;
+    public abstract getModeratedSubreddits(val: Bot): SubredditEntity[]
 }
 
 export default CMUser;

src/Web/Common/User/ClientUser.ts

@@ -1,6 +1,6 @@
-import {BotInstance, CMInstanceInterface} from "../../interfaces";
 import CMUser from "./CMUser";
 import {intersect, parseRedditEntity} from "../../../util";
+import {BotInstance, CMInstanceInterface} from "../interfaces";
 
 class ClientUser extends CMUser<CMInstanceInterface, BotInstance, string> {
 
@@ -9,15 +9,15 @@ class ClientUser extends CMUser<CMInstanceInterface, BotInstance, string> {
     }
 
     canAccessInstance(val: CMInstanceInterface): boolean {
-        return this.isInstanceOperator(val) || intersect(this.subreddits, val.subreddits.map(x => parseRedditEntity(x).name)).length > 0;
+        return this.isInstanceOperator(val) || val.bots.filter(x => x.canUserAccessBot(this.name, this.subreddits)).length > 0;
     }
 
     canAccessBot(val: BotInstance): boolean {
-        return this.isInstanceOperator(val.instance) || intersect(this.subreddits, val.subreddits.map(x => parseRedditEntity(x).name)).length > 0;
+        return this.isInstanceOperator(val.instance) || val.canUserAccessBot(this.name, this.subreddits);
     }
 
     canAccessSubreddit(val: BotInstance, name: string): boolean {
-        return this.isInstanceOperator(val.instance) || this.subreddits.map(x => x.toLowerCase()).includes(parseRedditEntity(name).name.toLowerCase());
+        return this.isInstanceOperator(val.instance) || val.canUserAccessSubreddit(name, this.name, this.subreddits);
     }
 
     accessibleBots(bots: BotInstance[]): BotInstance[] {
@@ -28,12 +28,32 @@ class ClientUser extends CMUser<CMInstanceInterface, BotInstance, string> {
             if (this.isInstanceOperator(x.instance)) {
                 return true;
             }
-            return intersect(this.subreddits, x.subreddits.map(y => parseRedditEntity(y).name)).length > 0
+            return x.canUserAccessBot(this.name, this.subreddits);
+            //return intersect(this.subreddits, x.managers.map(y => parseRedditEntity(y).name)).length > 0
         });
     }
 
     accessibleSubreddits(bot: BotInstance): string[] {
-        return this.isInstanceOperator(bot.instance) ? bot.subreddits.map(x => parseRedditEntity(x).name) : intersect(this.subreddits, bot.subreddits.map(x => parseRedditEntity(x).name));
+        return this.isInstanceOperator(bot.instance) ? bot.getSubreddits() :  bot.getAccessibleSubreddits(this.name, this.subreddits);
+    }
+
+    isSubredditGuest(val: BotInstance, name: string): boolean {
+        const normalName = parseRedditEntity(name).name;
+        const manager = val.managers.find(x => x.subredditNormal === normalName);
+        if(manager !== undefined) {
+            return manager.guests.some(y => y.name.toLowerCase() === this.name.toLowerCase());
+        }
+        return false;
+    }
+
+    isSubredditMod(val: BotInstance, name: string): boolean {
+        const normalName = parseRedditEntity(name).name;
+        return this.canAccessSubreddit(val, name) && this.subreddits.map(x => parseRedditEntity(name).name).includes(normalName);
+    }
+
+    getModeratedSubreddits(val: BotInstance): string[] {
+        const normalSubs = this.subreddits.map(x => parseRedditEntity(x).name);
+        return val.managers.filter(x => normalSubs.includes(x.subredditNormal)).map(x => x.subredditNormal);
     }
 
 }

src/Web/Common/User/ServerUser.ts

@@ -1,9 +1,9 @@
-import {BotInstance, CMInstanceInterface} from "../../interfaces";
 import CMUser from "./CMUser";
 import {intersect, parseRedditEntity} from "../../../util";
 import {App} from "../../../App";
 import Bot from "../../../Bot";
 import {Manager} from "../../../Subreddit/Manager";
+import {BotInstance, CMInstanceInterface} from "../interfaces";
 
 class ServerUser extends CMUser<App, Bot, Manager> {
 
@@ -16,23 +16,49 @@ class ServerUser extends CMUser<App, Bot, Manager> {
     }
 
     canAccessInstance(val: App): boolean {
-        return this.isOperator || val.bots.filter(x => intersect(this.subreddits, x.subManagers.map(y => y.subreddit.display_name))).length > 0;
+        return this.isOperator || val.bots.filter(x => x.canUserAccessBot(this.name, this.subreddits)).length > 0;
     }
 
     canAccessBot(val: Bot): boolean {
-        return this.isOperator || intersect(this.subreddits, val.subManagers.map(y => y.subreddit.display_name)).length > 0;
+        return this.isOperator || val.canUserAccessBot(this.name, this.subreddits);
     }
 
     accessibleBots(bots: Bot[]): Bot[] {
-        return this.isOperator ? bots : bots.filter(x => intersect(this.subreddits, x.subManagers.map(y => y.subreddit.display_name)).length > 0);
+        return this.isOperator ? bots : bots.filter(x => x.canUserAccessBot(this.name, this.subreddits));
     }
 
     canAccessSubreddit(val: Bot, name: string): boolean {
-        return this.isOperator || this.subreddits.includes(parseRedditEntity(name).name) && val.subManagers.some(y => y.subreddit.display_name.toLowerCase() === parseRedditEntity(name).name.toLowerCase());
+        const normalName = parseRedditEntity(name).name;
+        return this.isOperator || this.accessibleSubreddits(val).some(x => x.toNormalizedManager().subredditNormal === normalName);
     }
 
     accessibleSubreddits(bot: Bot): Manager[] {
-        return this.isOperator ? bot.subManagers : bot.subManagers.filter(x => intersect(this.subreddits, [x.subreddit.display_name]).length > 0);
+        if(this.isOperator) {
+            return bot.subManagers;
+        }
+
+        const subs = bot.getAccessibleSubreddits(this.name, this.subreddits);
+        return bot.subManagers.filter(x => subs.includes(x.toNormalizedManager().subredditNormal));
+    }
+
+    isSubredditGuest(val: Bot, name: string): boolean {
+        const normalName = parseRedditEntity(name).name;
+        const manager = val.subManagers.find(x => parseRedditEntity(x.subreddit.display_name).name === normalName);
+        if(manager !== undefined) {
+            return manager.toNormalizedManager().guests.some(x => x.name === this.name);
+        }
+        return false;
+    }
+
+    isSubredditMod(val: Bot, name: string): boolean {
+        const normalName = parseRedditEntity(name).name;
+        return val.subManagers.some(x => parseRedditEntity(x.subreddit.display_name).name === normalName) && this.subreddits.map(x => parseRedditEntity(x).name).some(x => x === normalName);
+    }
+
+    getModeratedSubreddits(val: Bot): Manager[] {
+        const normalSubs = this.subreddits.map(x => parseRedditEntity(x).name);
+
+        return val.subManagers.filter(x => normalSubs.includes(x.subreddit.display_name));
     }
 }
 

src/Web/Common/defaults.ts

@@ -49,6 +49,7 @@ const sub: SubredditDataResponse = {
     heartbeatHuman: "-",
     indicator: "-",
     logs: [],
+    guests: [],
     maxWorkers: 0,
     name: "-",
     pollingInfo: [],

src/Web/Common/interfaces.ts

@@ -1,6 +1,8 @@
 import {RunningState} from "../../Subreddit/Manager";
-import {LogInfo, ManagerStats} from "../../Common/interfaces";
-import {BotInstance} from "../interfaces";
+import {BotConnection, LogInfo, ManagerStats} from "../../Common/interfaces";
+import {Guest, GuestAll} from "../../Common/Entities/Guest/GuestInterfaces";
+import {URL} from "url";
+import {Dayjs} from "dayjs";
 
 export interface BotStats {
     startedAtHuman: string,
@@ -50,6 +52,7 @@ export interface SubredditDataResponse {
     heartbeatHuman?: string
     heartbeat: number
     retention: string
+    guests: (Guest | GuestAll)[]
 }
 
 export interface BotStatusResponse {
@@ -77,14 +80,65 @@ export interface IUser {
     tokenExpiresAt?: number
 }
 
+export interface ManagerResponse {
+    name: string,
+    subreddit: string,
+    guests: Guest[]
+}
+
+export interface NormalizedManagerResponse extends ManagerResponse {
+    subredditNormal: string
+}
+
+
+export interface BotInstanceResponse {
+    botName: string
+    //botLink: string
+    error?: string
+    managers: ManagerResponse[]
+    nanny?: string
+    running: boolean
+}
+
+export interface BotInstanceFunctions {
+    getSubreddits: (normalized?: boolean) => string[]
+    getAccessibleSubreddits: (user: string, subreddits: string[]) => string[]
+    getManagerNames: () => string[]
+    getGuestManagers: (user: string) => NormalizedManagerResponse[]
+    getGuestSubreddits: (user: string) => string[]
+    canUserAccessBot: (user: string, subreddits: string[]) => boolean
+    canUserAccessSubreddit: (subreddit: string, user: string, subreddits: string[]) => boolean
+}
+
+export interface BotInstance extends BotInstanceResponse, BotInstanceFunctions {
+    managers: NormalizedManagerResponse[]
+    instance: CMInstanceInterface
+}
+
+export interface CMInstanceInterface extends BotConnection {
+    friendly?: string
+    operators: string[]
+    operatorDisplay: string
+    url: URL,
+    normalUrl: string,
+    lastCheck?: number
+    online: boolean
+    subreddits: string[]
+    bots: BotInstance[]
+    error?: string
+    ranMigrations: boolean
+    migrationBlocker?: string
+    invites: string[]
+}
+
 export interface HeartbeatResponse {
     ranMigrations: boolean
     migrationBlocker?: string
-    subreddits: string[]
     operators: string[]
     operatorDisplay?: string
     friendly?: string
-    bots: BotInstance[]
+    bots: BotInstanceResponse[]
+    invites: string[]
 }
 
 
@@ -97,4 +151,14 @@ export interface InviteData {
     redirectUri: string
     creator: string
     overwrite?: boolean
+    initialConfig?: string
+    expiresAt?: number | Dayjs
+    guests?: string[]
+}
+
+export interface SubredditInviteData {
+    subreddit: string
+    guests?: string[]
+    initialConfig?: string
+    expiresAt?: number | Dayjs
 }

src/Web/Server/middleware.ts

@@ -2,12 +2,26 @@ import {Request, Response, NextFunction} from "express";
 import Bot from "../../Bot";
 import ServerUser from "../Common/User/ServerUser";
 
-export const authUserCheck = (userRequired: boolean = true) => async (req: Request, res: Response, next: Function) => {
+export type AuthEntityType = 'user' | 'operator' | 'machine';
+
+export const authUserCheck = (allowedEntityTypes: AuthEntityType | AuthEntityType[] = ['user']) => async (req: Request, res: Response, next: Function) => {
+    const types = Array.isArray(allowedEntityTypes) ? allowedEntityTypes : [allowedEntityTypes];
+
     if (req.isAuthenticated()) {
-        if (userRequired && (req.user as ServerUser).machine) {
-            return res.status(403).send('Must be authenticated as a user to access this route');
+        if(types.length === 0) {
+            return next();
         }
-        return next();
+        if(types.includes('machine') && (req.user as ServerUser).machine) {
+            return next();
+        }
+        if(types.includes('operator') && req.user.isInstanceOperator(req.botApp)) {
+            return next();
+        }
+        if(types.includes('user') && !(req.user as ServerUser).machine) {
+            return next();
+        }
+        req.logger.error(`User is authenticated but does not sufficient permissions. Required: ${types.join(', ')} | User: ${req.user.name}`);
+        return res.status(403).send('Must be authenticated to access this route');
     } else {
         return res.status(401).send('Must be authenticated to access this route');
     }
@@ -38,7 +52,7 @@ export const botRoute = (required = true) => async (req: Request, res: Response,
     return next();
 }
 
-export const subredditRoute = (required = true) => async (req: Request, res: Response, next: Function) => {
+export const subredditRoute = (required = true, modRequired = false, guestRequired = false) => async (req: Request, res: Response, next: Function) => {
 
     const bot = req.serverBot;
 
@@ -57,7 +71,7 @@ export const subredditRoute = (required = true) => async (req: Request, res: Res
                 return res.status(400).send('Cannot access route for subreddit you do not manage or is not run by the bot')
             }
 
-            if (!req.user?.canAccessSubreddit(bot, subreddit)) {
+            if (!req.user?.canAccessSubreddit(bot, subreddit) || (modRequired && !req.user?.isSubredditMod(bot, subreddit)) || (guestRequired && !req.user?.isSubredditGuest(bot, subreddit))) {
                 return res.status(400).send('Cannot access route for subreddit you do not manage or is not run by the bot')
             }
 

src/Web/Server/routes/authenticated/applicationRoutes.ts

@@ -2,9 +2,10 @@ import {Router} from '@awaitjs/express';
 import {Request, Response} from 'express';
 import {authUserCheck} from "../../middleware";
 import {HeartbeatResponse} from "../../../Common/interfaces";
+import {guestEntityToApiGuest} from "../../../../Common/Entities/Guest/GuestEntity";
 
-const router = Router();
-router.use(authUserCheck(false));
+/*const router = Router();
+router.use(authUserCheck(['machine']));*/
 
 interface OperatorData {
     name: string[]
@@ -17,24 +18,26 @@ export const heartbeat = (opData: OperatorData) => {
         if(req.botApp === undefined) {
             return res.status(500).send('Application is initializing, try again in a few seconds');
         }
-        req.botApp.migrationBlocker
+        //req.botApp.migrationBlocker
         const heartbeatData: HeartbeatResponse = {
-            subreddits: req.botApp.bots.map(y => y.subManagers.map(x => x.subreddit.display_name)).flat(),
-            // @ts-ignore
-            bots: req.botApp.bots.map(x => ({botName: x.botName, subreddits: x.subManagers.map(y => y.displayLabel), running: x.running})),
+            //subreddits: req.botApp.bots.map(y => y.subManagers.map(x => x.subreddit.display_name)).flat(),
+            bots: req.botApp.bots.map(x => ({
+                botName: x.botName as string,
+                managers: x.subManagers.map(y => ({
+                    name: y.displayLabel,
+                    subreddit: y.subreddit.display_name,
+                    guests: y.managerEntity.getGuests().map(x => guestEntityToApiGuest(x)),
+                })),
+                running: x.running,
+            })),
             operators: opData.name,
             operatorDisplay: opData.display,
-            friendly: opData.friendly,
+            friendly: req.botApp.friendly,
             ranMigrations: req.botApp.ranMigrations,
             migrationBlocker: req.botApp.migrationBlocker,
-            //friendly: req.botApp !== undefined ? req.botApp.botName : undefined,
-            //running: req.botApp !== undefined ? req.botApp.heartBeating : false,
-            //nanny: req.botApp !== undefined ? req.botApp.nannyMode : undefined,
-            //botName: req.botApp !== undefined ? req.botApp.botName : undefined,
-            //botLink: req.botApp !== undefined ? req.botApp.botLink : undefined,
-            //error: req.botApp.error,
+            invites: await req.botApp.getInviteIds()
         };
         return res.json(heartbeatData);
     };
-    return [authUserCheck(false), response];
+    return [authUserCheck(['machine']), response];
 }

src/Web/Server/routes/authenticated/user/addBot.ts

@@ -5,24 +5,29 @@ import Bot from "../../../../../Bot";
 import LoggedError from "../../../../../Utils/LoggedError";
 import {open} from 'fs/promises';
 import {buildBotConfig} from "../../../../../ConfigBuilder";
+import {BotInvite} from "../../../../../Common/Entities/BotInvite";
+import dayjs from 'dayjs';
 
 const addBot = () => {
 
     const middleware = [
-        authUserCheck(),
+        authUserCheck(['machine','operator']),
     ];
 
     const response = async (req: Request, res: Response) => {
 
-        if (!req.user?.isInstanceOperator(req.app)) {
-            return res.status(401).send("Must be an Operator to use this route");
-        }
-
         if (!req.botApp.fileConfig.isWriteable) {
             return res.status(409).send('Operator config is not writeable');
         }
 
-        const {overwrite = false, ...botData} = req.body;
+        const {overwrite = false, invite: inviteId, ...botData} = req.body;
+
+        // see if we are adding from invite
+        let invite: BotInvite | undefined;
+
+        if(inviteId !== undefined) {
+            invite = await req.botApp.getInviteById(inviteId);
+        }
 
         // check if bot is new or overwriting
         let existingBot = req.botApp.bots.find(x => x.botAccount === botData.name);
@@ -40,12 +45,20 @@ const addBot = () => {
             req.botApp.bots.splice(existingBotIndex, 1);
         }
 
+        if(invite !== undefined && invite.subreddits !== undefined) {
+            botData.subreddits = {names: invite.subreddits};
+        }
+
         req.botApp.fileConfig.document.addBot(botData);
 
         const handle = await open(req.botApp.fileConfig.document.location as string, 'w');
         await handle.writeFile(req.botApp.fileConfig.document.toString());
         await handle.close();
 
+        if(invite !== undefined) {
+            await req.botApp.deleteInvite(inviteId);
+        }
+
         const newBot = new Bot(buildBotConfig(botData, req.botApp.config), req.botApp.logger);
         req.botApp.bots.push(newBot);
         let result: any = {stored: true, success: true};
@@ -77,6 +90,9 @@ const addBot = () => {
                     req.botApp.logger.error(err);
                 }
             });
+            if(invite !== undefined && invite.guests !== undefined && invite.guests !== null && invite.guests.length > 0) {
+                await newBot.addGuest(invite.guests, dayjs().add(1, 'day'));
+            }
         } catch (err: any) {
             req.botApp.logger.error(`Bot ${newBot.botName} cannot recover from this error and must be re-built`);
             if (!err.logged || !(err instanceof LoggedError)) {

src/Web/Server/routes/authenticated/user/index.ts

@@ -1,12 +1,16 @@
 import {Request, Response} from 'express';
 import {authUserCheck, botRoute, subredditRoute} from "../../../middleware";
-import Submission from "snoowrap/dist/objects/Submission";
 import winston from 'winston';
-import {COMMENT_URL_ID, parseLinkIdentifier, parseRedditThingsFromLink, SUBMISSION_URL_ID} from "../../../../../util";
+import {
+    COMMENT_URL_ID,
+    parseLinkIdentifier,
+    parseRedditEntity,
+    parseRedditThingsFromLink,
+    SUBMISSION_URL_ID
+} from "../../../../../util";
 import {booleanMiddle} from "../../../../Common/middleware";
 import {Manager} from "../../../../../Subreddit/Manager";
-import {ActionedEvent} from "../../../../../Common/interfaces";
-import {CMEvent, CMEvent as ActionedEventEntity} from "../../../../../Common/Entities/CMEvent";
+import {CMEvent} from "../../../../../Common/Entities/CMEvent";
 import {nanoid} from "nanoid";
 import dayjs from "dayjs";
 import {
@@ -16,10 +20,11 @@ import {
     getFullEventsById,
     paginateRequest
 } from "../../../../Common/util";
-import {filterResultsBuilder} from "../../../../../Utils/typeormUtils";
-import {Brackets} from "typeorm";
 import {Activity} from "../../../../../Common/Entities/Activity";
-import {RedditThing} from "../../../../../Common/Infrastructure/Reddit";
+import {Guest} from "../../../../../Common/Entities/Guest/GuestInterfaces";
+import {guestEntitiesToAll, guestEntityToApiGuest} from "../../../../../Common/Entities/Guest/GuestEntity";
+import {ManagerEntity} from "../../../../../Common/Entities/ManagerEntity";
+import {AuthorEntity} from "../../../../../Common/Entities/AuthorEntity";
 
 const commentReg = parseLinkIdentifier([COMMENT_URL_ID]);
 const submissionReg = parseLinkIdentifier([SUBMISSION_URL_ID]);
@@ -41,36 +46,13 @@ const configLocation = async (req: Request, res: Response) => {
 
 export const configLocationRoute = [authUserCheck(), botRoute(), subredditRoute(), configLocation];
 
-const getInvites = async (req: Request, res: Response) => {
-
-    return res.json(await req.serverBot.cacheManager.getPendingSubredditInvites());
+const removalReasons = async (req: Request, res: Response) => {
+    const manager = req.manager as Manager;
+    const reasons = await manager.resources.getSubredditRemovalReasons()
+    return res.json(reasons);
 };
 
-export const getInvitesRoute = [authUserCheck(), botRoute(), getInvites];
-
-const addInvite = async (req: Request, res: Response) => {
-
-    const {subreddit} = req.body as any;
-    if (subreddit === undefined || subreddit === null || subreddit === '') {
-        return res.status(400).send('subreddit must be defined');
-    }
-    await req.serverBot.cacheManager.addPendingSubredditInvite(subreddit);
-    return res.status(200).send();
-};
-
-export const addInviteRoute = [authUserCheck(), botRoute(), addInvite];
-
-const deleteInvite = async (req: Request, res: Response) => {
-
-    const {subreddit} = req.query as any;
-    if (subreddit === undefined || subreddit === null || subreddit === '') {
-        return res.status(400).send('subreddit must be defined');
-    }
-    await req.serverBot.cacheManager.deletePendingSubredditInvite(subreddit);
-    return res.status(200).send();
-};
-
-export const deleteInviteRoute = [authUserCheck(), botRoute(), deleteInvite];
+export const removalReasonsRoute = [authUserCheck(), botRoute(), subredditRoute(), removalReasons];
 
 const actionedEvents = async (req: Request, res: Response) => {
 
@@ -194,20 +176,94 @@ const cancelDelayed = async (req: Request, res: Response) => {
     const {id} = req.query as any;
     const {name: userName} = req.user as Express.User;
 
-    if(req.manager?.resources === undefined) {
+    if (req.manager?.resources === undefined) {
         req.manager?.logger.error('Subreddit does not have delayed items!', {user: userName});
         return res.status(400).send();
     }
 
-    const delayedItem = req.manager.resources.delayedItems.find(x => x.id === id);
-    if(delayedItem === undefined) {
-        req.manager?.logger.error(`No delayed items exists with the id ${id}`, {user: userName});
-        return res.status(400).send();
+    if (id === undefined) {
+        await req.manager.resources.removeDelayedActivity();
+    } else {
+        const delayedItem = req.manager.resources.delayedItems.find(x => x.id === id);
+        if (delayedItem === undefined) {
+            req.manager?.logger.error(`No delayed items exists with the id ${id}`, {user: userName});
+            return res.status(400).send();
+        }
+
+        await req.manager.resources.removeDelayedActivity(delayedItem.id);
+        req.manager?.logger.info(`Remove Delayed Item '${delayedItem.id}'`, {user: userName});
     }
 
-    req.manager.resources.delayedItems = req.manager.resources.delayedItems.filter(x => x.id !== id);
-    req.manager?.logger.info(`Remove Delayed Item '${delayedItem.id}'`, {user: userName});
     return res.send('OK');
 };
 
 export const cancelDelayedRoute = [authUserCheck(), botRoute(), subredditRoute(true), cancelDelayed];
+
+const removeGuestMod = async (req: Request, res: Response) => {
+
+    const {name} = req.query as any;
+
+    const isAll = req.manager === undefined;
+    const managers = (isAll ? req.user?.getModeratedSubreddits(req.serverBot) : [req.manager as Manager]) as Manager[];
+
+    const newGuests = await req.serverBot.removeGuest(name, managers.map(x => x.subreddit.display_name));
+
+    const guests = isAll ? guestEntitiesToAll(newGuests) : Array.from(newGuests.values()).flat(3);
+
+    return res.json(guests);
+};
+
+export const removeGuestModRoute = [authUserCheck(), botRoute(), subredditRoute(true, true), removeGuestMod];
+
+const addGuestMod = async (req: Request, res: Response) => {
+
+    const {name, time} = req.query as any;
+
+    const isAll = req.manager === undefined;
+    const managers = (isAll ? req.user?.getModeratedSubreddits(req.serverBot) : [req.manager as Manager]) as Manager[];
+
+    const expiresAt = dayjs(Number.parseInt(time));
+
+    const newGuests = await req.serverBot.addGuest(name, expiresAt, managers.map(x => x.subreddit.display_name));
+
+    const guests = isAll ? guestEntitiesToAll(newGuests) : Array.from(newGuests.values()).flat(3);
+
+    return res.status(200).json(guests);
+};
+
+export const addGuestModRoute = [authUserCheck(), botRoute(), subredditRoute(true, true), addGuestMod];
+
+const saveGuestWikiEdit = async (req: Request, res: Response) => {
+    const {location, data, reason = 'Updated through CM Web', create = false} = req.body as any;
+
+    try {
+        // @ts-ignore
+        const wiki = await req.manager?.subreddit.getWikiPage(location) as WikiPage;
+        await wiki.edit({
+            text: data,
+            reason: `${reason} by Guest Mod ${req.user?.name}`,
+        });
+    } catch (err: any) {
+        res.status(500);
+        return res.send(err.message);
+    }
+
+    if(create) {
+        try {
+            // @ts-ignore
+            await req.manager.subreddit.getWikiPage(location).editSettings({
+                permissionLevel: 2,
+                // don't list this page on r/[subreddit]/wiki/pages
+                listed: false,
+            });
+        } catch (err: any) {
+            res.status(500);
+            return res.send(`Successfully created wiki page for configuration but encountered error while setting visibility. You should manually set the wiki page visibility on reddit. \r\n Error: ${err.message}`);
+        }
+    }
+
+    res.status(200);
+    return res.send();
+}
+
+export const saveGuestWikiEditRoute = [authUserCheck(), botRoute(), subredditRoute(true, false, true), saveGuestWikiEdit];

src/Web/Server/routes/authenticated/user/invites.ts

@@ -0,0 +1,54 @@
+import {authUserCheck, botRoute} from "../../../middleware";
+import {Request, Response} from "express";
+import {CMError} from "../../../../../Utils/Errors";
+
+const getSubredditInvites = async (req: Request, res: Response) => {
+
+    return res.json(await req.serverBot.cacheManager.getPendingSubredditInvites());
+};
+export const getSubredditInvitesRoute = [authUserCheck(), botRoute(), getSubredditInvites];
+const addSubredditInvite = async (req: Request, res: Response) => {
+
+    const {subreddit} = req.body as any;
+    if (subreddit === undefined || subreddit === null || subreddit === '') {
+        return res.status(400).send('subreddit must be defined');
+    }
+    try {
+        await req.serverBot.cacheManager.addPendingSubredditInvite(subreddit);
+    } catch (e: any) {
+        if (e instanceof CMError) {
+            req.logger.warn(e);
+            return res.status(400).send(e.message);
+        } else {
+            req.logger.error(e);
+            return res.status(500).send(e.message);
+        }
+    }
+    return res.status(200).send();
+};
+export const addSubredditInviteRoute = [authUserCheck(), botRoute(), addSubredditInvite];
+const deleteSubredditInvite = async (req: Request, res: Response) => {
+
+    const {subreddit} = req.query as any;
+    if (subreddit === undefined || subreddit === null || subreddit === '') {
+        return res.status(400).send('subreddit must be defined');
+    }
+    await req.serverBot.cacheManager.deletePendingSubredditInvite(subreddit);
+    return res.status(200).send();
+};
+export const deleteSubredditInviteRoute = [authUserCheck(), botRoute(), deleteSubredditInvite];
+
+const getBotInvite = async (req: Request, res: Response) => {
+    const invite = await req.botApp.getInviteById(req.params.id as any);
+    if(invite === undefined) {
+        return res.status(404).send(`Invite with ID ${req.params.id} does not exist`);
+    }
+    return res.json(invite);
+}
+export const getBotInviteRoute = [authUserCheck(['machine']), getBotInvite];
+
+const addBotInvite = async (req: Request, res: Response) => {
+    const invite = await req.botApp.addInvite(req.body);
+    return res.json(invite);
+}
+export const addBotInviteRoute = [authUserCheck(['operator']), addBotInvite];

src/Web/Server/routes/authenticated/user/liveStats.ts

@@ -1,13 +1,108 @@
 import {authUserCheck, botRoute, subredditRoute} from "../../../middleware";
 import {Request, Response} from "express";
 import Bot from "../../../../../Bot";
-import {boolToString, cacheStats, filterLogs, formatNumber, logSortFunc, pollingInfo} from "../../../../../util";
+import {
+    boolToString,
+    cacheStats,
+    difference,
+    filterLogs,
+    formatNumber,
+    logSortFunc, parseRedditEntity,
+    pollingInfo,
+    symmetricalDifference
+} from "../../../../../util";
 import dayjs from "dayjs";
 import {LogInfo, ResourceStats, RUNNING, STOPPED, SYSTEM} from "../../../../../Common/interfaces";
 import {Manager} from "../../../../../Subreddit/Manager";
 import winston from "winston";
 import {opStats} from "../../../../Common/util";
 import {BotStatusResponse} from "../../../../Common/interfaces";
+import deepEqual from "fast-deep-equal";
+import {DispatchedEntity} from "../../../../../Common/Entities/DispatchedEntity";
+import {
+    guestEntitiesToAll,
+    guestEntityToApiGuest,
+    ManagerGuestEntity
+} from "../../../../../Common/Entities/Guest/GuestEntity";
+import {Guest} from "../../../../../Common/Entities/Guest/GuestInterfaces";
+
+const lastFullResponse: Map<string, Record<string, any>> = new Map();
+
+const mergeDeepEqual = (a: Record<any, any>, b: Record<any, any>): Record<any, any> => {
+    const delta: Record<any, any> = {};
+    for(const [k,v] of Object.entries(a)) {
+        if(typeof v === 'object' && v !== null && typeof b[k] === 'object' && b[k] !== null) {
+            const objDelta = mergeDeepEqual(v, b[k]);
+            if(Object.keys(objDelta).length > 0) {
+                delta[k] = objDelta;
+            }
+        } else if(!deepEqual(v, b[k])) {
+            delta[k] = v;
+        }
+    }
+    return delta;
+}
+
+const generateDeltaResponse = (data: Record<string, any>, hash: string, responseType: 'full' | 'delta') => {
+    let resp = data;
+    if(responseType === 'delta') {
+        const reference = lastFullResponse.get(hash);
+        if(reference === undefined) {
+            // shouldn't happen...
+            return data;
+        }
+        const delta: Record<string, any> = {};
+        for(const [k,v] of Object.entries(data)) {
+            switch(k) {
+                case 'delayedItems':
+                    // on delayed items delta we will send a different data structure back with just remove/new(add)
+                    const refIds = reference[k].map((x: DispatchedEntity) => x.id);
+                    const latestIds = v.map((x: DispatchedEntity) => x.id);
+
+                    if(symmetricalDifference(refIds, latestIds).length === 0) {
+                        continue;
+                    }
+
+                    const newIds = Array.from(difference(latestIds, refIds));
+                    const newItems = v.filter((x: DispatchedEntity) => newIds.includes(x.id));
+
+                    // just need ids that should be removed on frontend
+                    const removedItems = Array.from(difference(refIds, latestIds));
+                    delta[k] = {new: newItems, removed: removedItems};
+                    break;
+                case 'guests':
+                    const refNames = reference[k].map((x: Guest) => `${x.name}-${x.expiresAt}`);
+                    const latestNames = v.map((x: Guest) => `${x.name}-${x.expiresAt}`);
+
+                    if(symmetricalDifference(refNames, latestNames).length === 0) {
+                        continue;
+                    }
+
+                    // const newNames = Array.from(difference(latestNames, refNames));
+                    // const newGuestItems = v.filter((x: Guest) => newNames.includes(x.name));
+                    //
+                    // // just need ids that should be removed on frontend
+                    // const removedGuestItems = Array.from(difference(refNames, latestNames));
+                    // delta[k] = {new: newGuestItems, removed: removedGuestItems};
+                    delta[k] = v;
+                    break;
+                default:
+                    if(!deepEqual(v, reference[k])) {
+                        if(v !== null && typeof v === 'object' && reference[k] !== null && typeof reference[k] === 'object') {
+                            // for things like cache/stats we only want to delta changed properties, not the entire object
+                            delta[k] = mergeDeepEqual(v, reference[k]);
+                        } else {
+                            delta[k] = v;
+                        }
+                    }
+                    break;
+            }
+        }
+        resp = delta;
+    }
+    lastFullResponse.set(hash, data);
+    return resp;
+}
 
 const liveStats = () => {
     const middleware = [
@@ -20,18 +115,35 @@ const liveStats = () => {
     {
         const bot = req.serverBot as Bot;
         const manager = req.manager;
-        
+        const responseType = req.query.type === 'delta' ? 'delta' : 'full';
+        const hash = `${bot.botName}${manager !== undefined ? `-${manager.getDisplay()}` : ''}`;
+        const isOperator = req.user?.isInstanceOperator(bot);
+
+        const userModerated: string[] = (req.user as Express.User).subreddits.map(x => parseRedditEntity(x).name);
+
         if(manager === undefined) {
             // getting all
             const subManagerData: any[] = [];
+            //let managerGuests: ManagerGuestEntity[] = [];
             for (const m of req.user?.accessibleSubreddits(bot) as Manager[]) {
+
+                const isMod = userModerated.some(x => parseRedditEntity(m.subreddit.display_name).name === x);
+                const isGuest = m.managerEntity.getGuests().some(y => y.author.name === req.user?.name);
+
+                //const guests = await m.managerEntity.getGuests();
+                //managerGuests = managerGuests.concat(guests);
+
                 const sd = {
+                    name: m.displayLabel,
+                    guests: isOperator || isMod ? m.managerEntity.getGuests().map(x => guestEntityToApiGuest(x)) : [],
                     queuedActivities: m.queue.length(),
                     runningActivities: m.queue.running(),
                     delayedItems: m.getDelayedSummary(),
                     maxWorkers: m.queue.concurrency,
                     subMaxWorkers: m.subMaxWorkers || bot.maxWorkers,
                     globalMaxWorkers: bot.maxWorkers,
+                    isMod,
+                    isGuest,
                     checks: {
                         submissions: m.submissionChecks === undefined ? 0 : m.submissionChecks.length,
                         comments: m.commentChecks === undefined ? 0 : m.commentChecks.length,
@@ -155,6 +267,11 @@ const liveStats = () => {
                 scopes: scopes === null || !Array.isArray(scopes) ? [] : scopes,
                 subMaxWorkers,
                 runningActivities,
+                guests: guestEntitiesToAll(subManagerData.reduce((acc, curr) => {
+                    acc.set(curr.name, curr.guests);
+                    return acc;
+                }, new Map<string, Guest[]>())),
+                isMod: subManagerData.some(x => x.isMod),
                 queuedActivities,
                 delayedItems,
                 botState: {
@@ -213,14 +330,21 @@ const liveStats = () => {
                 },
                 ...allManagerData,
             };
-            return res.json(data);
+            const respData = generateDeltaResponse(data, hash, responseType);
+            if(Object.keys(respData).length === 0) {
+                return res.status(304).send();
+            }
+            return res.json(respData);
         } else {
+            const isGuest = manager.managerEntity.getGuests().some(y => y.author.name === req.user?.name);
+            const isMod = userModerated.some(x => parseRedditEntity(manager.subreddit.display_name).name === x);
             // getting specific subreddit stats
             const sd = {
                 name: manager.displayLabel,
                 botState: manager.managerState,
                 eventsState: manager.eventsState,
                 queueState: manager.queueState,
+                guests: isOperator || isMod ? manager.managerEntity.getGuests().map(x => guestEntityToApiGuest(x)) : [],
                 indicator: 'gray',
                 permissions: await manager.getModPermissions(),
                 queuedActivities: manager.queue.length(),
@@ -231,6 +355,8 @@ const liveStats = () => {
                 globalMaxWorkers: bot.maxWorkers,
                 validConfig: boolToString(manager.validConfigLoaded),
                 configFormat: manager.wikiFormat,
+                isGuest,
+                isMod,
                 dryRun: boolToString(manager.dryRun === true),
                 pollingInfo: manager.pollOptions.length === 0 ? ['nothing :('] : manager.pollOptions.map(pollingInfo),
                 checks: {
@@ -282,7 +408,11 @@ const liveStats = () => {
                 }
             }
 
-            return res.json(sd);
+            const respData = generateDeltaResponse(sd, hash, responseType);
+            if(Object.keys(respData).length === 0) {
+                return res.status(304).send();
+            }
+            return res.json(respData);
         }
     }
     return [...middleware, response];

src/Web/Server/routes/authenticated/user/logs.ts

@@ -45,7 +45,6 @@ const logs = () => {
 
         const userName = req.user?.name as string;
         const isOperator = req.user?.isInstanceOperator(req.botApp);
-        const realManagers = req.botApp.bots.map(x => req.user?.accessibleSubreddits(x).map(x => x.displayLabel)).flat() as string[];
         const {level = 'verbose', stream, limit = 200, sort = 'descending', streamObjects = false, formatted: formattedVal = true, transports: transportsVal = false} = req.query;
 
         const formatted = formattedVal as boolean;
@@ -68,8 +67,6 @@ const logs = () => {
             }
         }
 
-        //const allReq = req.query.subreddit !== undefined && (req.query.subreddit as string).toLowerCase() === 'all';
-
         if (stream) {
 
             const requestedManagers = managers.map(x => x.displayLabel);

src/Web/Server/routes/authenticated/user/status.ts

@@ -5,7 +5,7 @@ import {
     filterLogBySubreddit, filterLogs,
     formatNumber,
     intersect,
-    LogEntry, logSortFunc, parseDurationValToDuration,
+    LogEntry, logSortFunc, parseDurationValToDuration, parseRedditEntity,
     pollingInfo
 } from "../../../../../util";
 import {Manager} from "../../../../../Subreddit/Manager";
@@ -17,6 +17,11 @@ import {opStats} from "../../../../Common/util";
 import {authUserCheck, botRoute, subredditRoute} from "../../../middleware";
 import Bot from "../../../../../Bot";
 import {DurationVal} from "../../../../../Common/Infrastructure/Atomic";
+import {
+    guestEntitiesToAll,
+    guestEntityToApiGuest,
+    ManagerGuestEntity
+} from "../../../../../Common/Entities/Guest/GuestEntity";
 
 const status = () => {
 
@@ -63,6 +68,7 @@ const status = () => {
         } = req.query;
 
         const allReq = req.query.subreddit !== undefined && (req.query.subreddit as string).toLowerCase() === 'all';
+        const userModerated: string[] = (req.user as Express.User).subreddits.map(x => parseRedditEntity(x).name);
 
         const subManagerData = [];
         for (const m of req.user?.accessibleSubreddits(bot) as Manager[]) {
@@ -120,6 +126,8 @@ const status = () => {
                 startedAtHuman: 'Not Started',
                 delayBy: m.delayBy === undefined ? 'No' : `Delayed by ${m.delayBy} sec`,
                 retention,
+                isGuest: m.managerEntity.getGuests().some(y => y.author.name === req.user?.name),
+                isMod: userModerated.some(x => parseRedditEntity(m.subreddit.display_name).name === x)
             };
             // TODO replace indicator data with js on client page
             let indicator;
@@ -255,6 +263,7 @@ const status = () => {
             subMaxWorkers,
             runningActivities,
             queuedActivities,
+            isMod: subManagerData.some(x => x.isMod),
             delayedItems,
             botState: {
                 state: RUNNING,

src/Web/Server/server.ts

@@ -16,26 +16,32 @@ import status from './routes/authenticated/user/status';
 import liveStats from './routes/authenticated/user/liveStats';
 import {
     actionedEventsRoute,
-    actionRoute,
-    addInviteRoute,
+    actionRoute, addGuestModRoute,
     cancelDelayedRoute,
     configLocationRoute,
     configRoute,
-    deleteInviteRoute,
-    getInvitesRoute
+    removeGuestModRoute, saveGuestWikiEditRoute, removalReasonsRoute
 } from "./routes/authenticated/user";
 import action from "./routes/authenticated/user/action";
 import {authUserCheck, botRoute} from "./middleware";
 import Bot from "../../Bot";
 import addBot from "./routes/authenticated/user/addBot";
 import ServerUser from "../Common/User/ServerUser";
-import {SimpleError} from "../../Utils/Errors";
+import {CMError, SimpleError} from "../../Utils/Errors";
 import {ErrorWithCause} from "pony-cause";
 import {Manager} from "../../Subreddit/Manager";
 import {MESSAGE} from "triple-beam";
 import dayjs from "dayjs";
 import { sleep } from '../../util';
 import {Invokee} from "../../Common/Infrastructure/Atomic";
+import {Point} from "@influxdata/influxdb-client";
+import {
+    addBotInviteRoute,
+    addSubredditInviteRoute,
+    deleteSubredditInviteRoute,
+    getBotInviteRoute,
+    getSubredditInvitesRoute
+} from "./routes/authenticated/user/invites";
 
 const server = addAsync(express());
 server.use(bodyParser.json());
@@ -147,6 +153,7 @@ const rcbServer = async function (options: OperatorConfigWithFileContext) {
     server.use(passport.authenticate('jwt', {session: false}));
     server.use((req, res, next) => {
         req.botApp = app;
+        req.logger = logger;
         next();
     });
 
@@ -202,6 +209,10 @@ const rcbServer = async function (options: OperatorConfigWithFileContext) {
 
     server.getAsync('/config/location', ...configLocationRoute);
 
+    server.postAsync('/config', ...saveGuestWikiEditRoute);
+
+    server.getAsync('/reasons', ...removalReasonsRoute);
+
     server.getAsync('/events', ...actionedEventsRoute);
 
     server.getAsync('/action', ...action);
@@ -210,14 +221,22 @@ const rcbServer = async function (options: OperatorConfigWithFileContext) {
 
     server.postAsync('/bot', ...addBot());
 
-    server.getAsync('/bot/invite', ...getInvitesRoute);
+    server.getAsync('/bot/invite', ...getSubredditInvitesRoute);
 
-    server.postAsync('/bot/invite', ...addInviteRoute);
+    server.postAsync('/bot/invite', ...addSubredditInviteRoute);
 
-    server.deleteAsync('/bot/invite', ...deleteInviteRoute);
+    server.deleteAsync('/bot/invite', ...deleteSubredditInviteRoute);
 
     server.deleteAsync('/delayed', ...cancelDelayedRoute);
 
+    server.deleteAsync('/guests', ...removeGuestModRoute);
+
+    server.postAsync('/guests', ...addGuestModRoute);
+
+    server.getAsync('/invites/:id', ...getBotInviteRoute);
+
+    server.postAsync('/invites', ...addBotInviteRoute);
+
     app = new App(options);
 
     const initBot = async (causedBy: Invokee = 'system') => {
@@ -232,6 +251,33 @@ const rcbServer = async function (options: OperatorConfigWithFileContext) {
         });
     }
 
+    // would like to use node-memwatch for more stats but doesn't work with docker (alpine gclib?) and requires more gyp bindings, yuck
+    // https://github.com/airbnb/node-memwatch
+    const writeMemoryMetrics = async () => {
+        if (options.dev.monitorMemory) {
+            if (options.influx !== undefined) {
+                const influx = options.influx;
+                while (true) {
+                    await sleep(options.dev.monitorMemoryInterval);
+                    try {
+                        const memUsage = process.memoryUsage();
+                        await influx.writePoint(new Point('serverMemory')
+                            .intField('external', memUsage.external)
+                            .intField('rss', memUsage.rss)
+                            .intField('arrayBuffers', memUsage.arrayBuffers)
+                            .intField('heapTotal', memUsage.heapTotal)
+                            .intField('heapUsed', memUsage.heapUsed)
+                        );
+                    } catch (e: any) {
+                        logger.warn(new CMError('Error occurred while trying to collect memory metrics', {cause: e}));
+                    }
+                }
+            } else {
+                logger.warn('Cannot monitor memory because influx config was not set');
+            }
+        }
+    }
+
     server.postAsync('/init', authUserCheck(), async (req, res) => {
         logger.info(`${(req.user as Express.User).name} requested the app to be re-built. Starting rebuild now...`, {subreddit: (req.user as Express.User).name});
         await initBot('user');
@@ -285,6 +331,7 @@ const rcbServer = async function (options: OperatorConfigWithFileContext) {
 
     logger.info('Initializing database...');
     try {
+        writeMemoryMetrics();
         const dbReady = await app.initDatabase();
         if(dbReady) {
             logger.info('Initializing application...');

src/Web/assets/public/app.css

@@ -176,3 +176,17 @@ a {
 li > ul {
     padding: revert;
 }
+
+.cancellable {
+    display:inline;
+/*    margin-left: 10px*/
+}
+
+.smallLi:before {
+    margin-left: -10px;
+    content: ""
+}
+
+.introjs-tooltip-title,.introjs-tooltiptext {
+    color: black;
+}

src/Web/assets/public/statusTour.js

@@ -0,0 +1,255 @@
+let steps = [];
+
+steps = [
+    {
+        title: 'Welcome to the ContextMod (CM) Dashboard',
+        intro: `
+<div class="space-y-3"><div>The dashboard allows you to monitor and configure your Bot's behavior for each Subreddit it runs on.</div>
+        <ul class="list-inside list-disc">
+        <li><a href="https://github.com/FoxxMD/context-mod/blob/master/docs/webInterface.md" target="_blank">Dashboard Tips</a></li>
+        <li><a href="https://github.com/FoxxMD/context-mod/tree/master/docs/subreddit/components" target="_blank">Config Docs</a></li>
+        <li><a href="https://github.com/FoxxMD/context-mod/issues" target="_blank">Report Issue</a></li>
+        <li><a href="https://www.reddit.com/r/ContextModBot/" target="_blank">CM Subreddit</a></li>
+        <li><a href="https://discord.gg/YgehbC8pXW" target="_blank">CM Discord</a></li>
+        </ul>
+</div>`
+    },
+    {
+        element: document.querySelector('#help'),
+        intro: 'If you need a refresher of this guide you can click here to re-run it.'
+    },
+    {
+        element: document.querySelector('#botTabs'),
+        title: 'Bots List',
+        intro: 'All of the bot accounts that moderate a subreddit you also moderate are listed here'
+    }
+];
+
+const bot = document.querySelector('#botTabs li span:not([data-bot="system"])');
+if (bot !== null) {
+    steps.push({
+        element: bot,
+        intro: `
+        <div class="space-y-3">
+            <div>Click on a Bot tab to view all of the Subreddits it is running.</div>
+        </div>`
+    });
+} else {
+    steps.push({
+        element: document.querySelector('#botTabs'),
+        intro: `
+        <div class="space-y-3">
+            <div>Once a Bot account has been added it will be visible here.</div>
+        </div>`
+    });
+}
+
+if(window.isOperator) {
+    steps.push({
+        element: document.querySelector('#botTabs li:last-child'),
+        title: 'Add A Bot',
+        intro: `
+        <div class="space-y-3">
+            <div>Start the invite process for adding a new Bot</div>
+        </div>`
+    });
+}
+
+const nonSystemSub = document.querySelector('.sub:not([data-bot="system"])');
+
+if(nonSystemSub === null) {
+    steps.push({
+        element: document.querySelector('.sub'),
+        intro: `
+        <div class="space-y-3">
+            <div>After you have added a Bot with a moderated Subreddit re-run this tour to finish!</div>
+        </div>`
+    });
+} else {
+    const subTab = document.querySelector('#subredditsTab ul');
+
+    steps.push({
+        element: subTab,
+        title: 'Subreddits List',
+        intro: `
+        <div class="space-y-3">
+            <div>Displays all of the Subreddits run by the selected Bot</div>
+            <div>${window.isOperator ? 'As an operator you can see all Subreddits even if you are not a moderator. Otherwise you would only be able to see Subreddits you moderate.' : 'You can only view Subreddits that you are also a moderator of.'}</div>
+        </div>`
+    });
+
+    const allSub = document.querySelector('#subredditsTab li span[data-subreddit="All"]');
+    steps.push({
+        element: allSub !== null ? allSub : subTab,
+        intro: `
+        <div class="space-y-3">
+            <div><strong>All Subreddits</strong> displays an Overview of all Subreddits you have access to as well as some basic Bot information.</div>
+        </div>`
+    });
+
+    const notAllSub = document.querySelector('#subredditsTab li span:not([data-subreddit="All"])');
+    steps.push({
+        element: notAllSub !== null ? notAllSub : subTab,
+        title: 'Subreddit',
+        intro: `
+        <div class="space-y-3">
+            <div>Clicking on an individual Subreddit will switch to its overview/logs.</div>
+            <div><strong>Please click on this Subreddit now before continuing the tour!</strong></div>
+        </div>`
+    });
+
+    const activeSub = document.querySelector('.sub:not([data-subreddit="All"])');
+
+    steps.push({
+        element: activeSub,
+        position: 'top',
+        title: 'Subreddit View',
+        intro: `
+        <div class="space-y-3">
+            <div>Information for the currently selected subreddit from the <strong>Subreddit List</strong> is displayed here.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.overviewContainer'),
+        title: 'Overview',
+        intro: `
+        <div class="space-y-3">
+            <div><strong>Overview</strong> displays the current state of the Bot on this Subreddit.</div>
+            <div>You may also start/stop/pause the Bot, for this Subreddit from here.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.overviewContainer .pollingInfo'),
+        intro: `
+        <div class="space-y-3">
+            <div>When <strong>Events</strong> is <strong>running</strong> the Bot is watching these sources, defined in your configuration, for new Activities in your subreddit.</div>
+            <div>When it sees a new Activity it automatically processes it using the Runs/Checks from its configuration.</div>
+            <div>This is a list of the sources the Bot is watching. Abbreviations:</div>
+            <ul class="list-inside list-disc">
+                <li>UNMODERATED - unmoderated mod queue</li>
+                <li>MODQUEUE - modqueue</li>
+                <li>NEWCOMM - new comments</li>
+                <li>NEWSUB - new submissions</li>
+            </ul>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.configContainer'),
+        title: 'Config',
+        intro: `
+        <div class="space-y-3">
+            <div><strong>Config</strong> displays information about this Subreddit's configuration.</div>
+            <div>A Subreddit's configuration is what determines how the Bot behaves. The Bot <strong>will not run</strong> if its configuration is empty or invalid.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.configContainer .dryRunLabel'),
+        title: 'Dry Run',
+        intro: `
+        <div class="space-y-3">
+            <div><strong>Dry Run</strong> status determines if the Bot is running in "pretend" mode or not.</div>
+            <div>In Dry Run mode the Bot will check Activities normally but <strong>will not</strong> run any Actions when triggered.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.configContainer .openConfig'),
+        title: 'Config Editor',
+        intro: `
+        <div class="space-y-3">
+            <div><strong>View</strong> opens the <strong>Configuration Editor</strong> for this subreddit.</div>
+            <div>You can view/create/edit your Subreddit's configuration from the Editor.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.usageContainer'),
+        title: 'Usage',
+        intro: `
+        <div class="space-y-3">
+            <div>Displays statistics about what the Bot has done on your Subreddit.</div>
+            <div><strong>Events</strong> are the number of Comments/Submissions the Bot has checked, in total.</div>
+            <div><strong>Actions</strong> are individual actions the Bot has taken in response to triggered Checks. This is usually things like removals, reporting, commenting, etc...</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.usageContainer .openActioned'),
+        intro: `
+        <div class="space-y-3">
+            <div>Opens a new page where you can see past Actions the Bot has taken, as well as search by permalink. This is equivalent to <strong>Mod Log</strong> on Reddit.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.runBotOnThing'),
+        title: 'Manually Running the Bot',
+        intro: `
+        <div class="space-y-3">
+            <div>You may <strong>manually run</strong> the Bot on any Activity (Submission/Comment) using its permalink.</div>
+            <div>To be clear -- the Bot automatically runs on new Activities from the Subreddit. This is for when you want to re-run or manually run on an arbitrary Activity.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.runBotOnThing input'),
+        intro: `
+        <div class="space-y-3">
+            <div>Copy the permalink (URL) for a Submission/Comment and paste it here</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.runBotOnThing a.dryRunCheck'),
+        intro: `
+        <div class="space-y-3">
+            <div><strong>Dry Run</strong> means the bot will check the Activity normally but <strong>will not run Actions.</strong></div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.runBotOnThing a.runCheck'),
+        intro: `
+        <div class="space-y-3">
+            <div>Otherwise use <strong>Run</strong> to run the Bot normally.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('.logs'),
+        intro: `
+        <div class="space-y-3">
+            <div>A <strong>real-time</strong> stream of logs for this Subreddit.</div>
+            <div>This shows a detailed stream of events and internal details for what the bot is doing.</div>
+        </div>`
+    });
+
+    steps.push({
+        element: activeSub.querySelector('span.has-tooltip'),
+        title: 'More Help',
+        intro: `
+        <div class="space-y-3">
+            <div>Make sure to hover over any <strong>?</strong> symbols you see as these contain more helpful information!</div>
+        </div>`
+    });
+
+    steps.push({
+        title: 'Good Luck!',
+        intro: `This concludes the tour. Remember you can always click <strong>Tour</strong> at any time to replay this guide. Happy botting!`
+    });
+
+}
+
+
+let intro = introJs().setOptions({
+    steps,
+})
+
+document.querySelector('#helpStart').addEventListener('click', (e) => {
+    e.preventDefault();
+    intro.start();
+});

src/Web/assets/views/config.ejs

@@ -56,6 +56,31 @@
                         </span>
                     </span>
                 </span>
+                <span id="reasonsWrapper" style="display: none;">
+                    |
+                    <span class="has-tooltip">
+                    <span style="z-index:999; margin-top: 30px;" class='tooltip rounded shadow-lg p-3 bg-gray-100 text-black space-y-2'>
+                       <strong>Subreddit Removal Reasons Helper</strong>
+                        <div>Copy the <b>ID</b> for use in <span class="font-mono">remove</span> action's <span class="font-mono">removalId</span> field</div>
+                        <ul style="user-select: text;" class="list-inside list-disc">
+                        </ul>
+                    </span>
+                    <span class="cursor-help">
+                            Removal Reasons
+                        <span>
+                            <svg xmlns="http://www.w3.org/2000/svg"
+                                 class="h-4 w-4 inline-block cursor-help"
+                                 fill="none"
+                                 viewBox="0 0 24 24" stroke="currentColor">
+                                <path stroke-linecap="round"
+                                      stroke-linejoin="round"
+                                      stroke-width="2"
+                                      d="M8.228 9c.549-1.165 2.03-2 3.772-2 2.21 0 4 1.343 4 3 0 1.4-1.278 2.575-3.006 2.907-.542.104-.994.54-.994 1.093m0 3h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/>
+                            </svg>
+                        </span>
+                    </span>
+                </span>
+                </span>
                 | <input id="configUrl" class="text-black placeholder-gray-500 rounded mx-2" style="min-width:400px;" placeholder="URL of a config to load"/> <a href="#" id="loadConfig">Load</a>
                 <div id="editWrapper" class="my-2">
                     <label style="display: none" for="reason">Edit Reason</label><input id="reason" class="text-black placeholder-gray-500 rounded mr-2" style="min-width:400px;" placeholder="Edit Reason: Updated through CM Web"/>
@@ -127,6 +152,7 @@
     }
 
     window.canSave = <%= canSave %>;
+    window.isGuest = false;
 
     if (searchParams.get('subreddit') === null) {
         document.querySelector('#saveTip').style.display = 'none';
@@ -141,6 +167,9 @@
             const saveLink = document.querySelector('#doSave');
             saveLink.classList.remove('isDisabled');
             saveLink.href = '#';
+            if(window.isGuest && !saveLink.innerHTML.includes('guest')) {
+                saveLink.innerHTML = `${saveLink.innerHTML} as Guest`;
+            }
             document.querySelector('#reason').style.display = 'initial';
         } else {
             document.querySelector('#saveTip').classList.add('has-tooltip');
@@ -212,7 +241,7 @@
             payload.reason = reasonVal;
         }
 
-        fetch(`${document.location.origin}/config${document.location.search}`, {
+        fetch(window.isGuest ? `${document.location.origin}/api/config${document.location.search}` : `${document.location.origin}/config${document.location.search}`, {
             method: 'POST',
             headers: {
                 'Accept': 'application/json',
@@ -381,6 +410,50 @@
                     } else {
                         resp.text().then(data => {
                             window.wikiLocation = data;
+                        });
+                        fetch(`${document.location.origin}/guest${document.location.search}`).then((resp) => {
+                            if(resp.ok) {
+                                window.canSave = true;
+                                window.isGuest = true;
+                                window.setSaveStatus();
+                            }
+                        }).catch((e) => {
+                            // do nothing, not a guest
+                        });
+
+                        // Since we are getting config for a subreddit and (assuming) user is authorized to see config then get subreddit removal reasons and populate helper
+                        fetch(`${document.location.origin}/api/reasons${document.location.search}`).then((resp) => {
+                            if(resp.ok) {
+                                resp.json().then((data) => {
+                                    document.querySelector('#reasonsWrapper').style.display = 'initial';
+
+                                    const reasonsList = document.querySelector('#reasonsWrapper ul');
+                                    if(data.length === 0) {
+                                        const node = document.createElement("LI");
+                                        node.appendChild(document.createTextNode('None'));
+                                        reasonsList.appendChild(node);
+                                    } else {
+                                        for(const reason of data) {
+                                            const node = document.createElement("LI");
+                                            node.appendChild(document.createTextNode(reason.title));
+
+                                            const copy = document.createElement('span');
+                                            copy.classList.add('cursor-pointer', 'float-right');
+                                            copy.insertAdjacentHTML('beforeend', `<a class="hover:bg-gray-400 no-underline rounded-md py-1 px-3 border" href="">Copy ID <span style="display:inline" class="iconify" data-icon="clarity:copy-to-clipboard-line"></span></a>`);
+                                            copy.addEventListener('click', e => {
+                                                e.preventDefault();
+                                                navigator.clipboard.writeText(reason.id);
+                                            });
+                                            node.appendChild(copy);
+                                            reasonsList.appendChild(node);
+                                        }
+                                    }
+                                })
+                            }
+                        }).catch((e) => {
+                            // just log it
+                            console.error('Error occurred while trying to fetch subreddit removal reasons');
+                            console.error(e);
                         })
                     }
                 });

src/Web/assets/views/error-authenticated.ejs

@@ -0,0 +1,41 @@
+<html lang="en">
+<%- include('partials/head', {title: 'CM'}) %>
+<body class="bg-gray-900 text-white font-sans">
+<div class="min-w-screen min-h-screen">
+    <%- include('partials/header') %>
+    <div class="container mx-auto">
+        <div class="grid">
+            <div class="bg-gray-600">
+                <div class="p-6 md:px-10 md:py-6">
+                    <div class="text-xl mb-4">Oops 😬</div>
+                    <div class="space-y-3">
+                        <div>Something went wrong while processing that last request:</div>
+                        <div class="space-y-3"><%- error %></div>
+                        <% if(locals.operatorDisplay !== undefined && locals.operatorDisplay !== 'Anonymous') { %>
+                            <div>Operated By: <%= operatorDisplay %></div>
+                        <% } %>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+    <%- include('partials/footer') %>
+</div>
+<script>
+    const instanceSearchParams = new URLSearchParams(window.location.search);
+
+    const instance = instanceSearchParams.get('instance');
+
+    document.querySelectorAll(`[data-instance].instanceSelectWrapper`).forEach((el) => {
+        if(el.dataset.instance === instance) {
+            el.classList.add('border-2');
+            el.querySelector('a.instanceSelect').classList.add('pointer-events-none','no-underline','font-bold');
+        } else {
+            el.classList.add('border');
+            el.querySelector('a.instanceSelect').classList.add('font-normal','pointer');
+        }
+
+    });
+</script>
+</body>
+</html>

src/Web/assets/views/helper.ejs

@@ -2,7 +2,7 @@
 <%- include('partials/head', {title: 'CM OAuth Helper'}) %>
 <body class="bg-gray-900 text-white font-sans">
 <div class="min-w-screen min-h-screen">
-    <%- include('partials/title', {title: ' OAuth Helper'}) %>
+    <%- include('partials/header') %>
     <div class="container mx-auto">
         <div class="grid">
             <div class="bg-gray-600">
@@ -65,24 +65,34 @@
       ease-in-out
       m-0
       focus:text-gray-700 focus:bg-white focus:border-blue-600 focus:outline-none" aria-label="Default select example">
-                                <% instances.forEach(function (name, index){ %>
-                                    <option selected="<%= index === 0 ? 'true' : 'false' %>" value="<%= name %>"><%= name %></option>
-                                    <%= name %>
-                                <% }) %>
+                                <option selected="true" value="<%= instanceId %>"><%= instanceId %></option>
+                                <%= instanceId %>
                             </select>
                         </div>
                     </div>
                     <div class="text-lg text-semibold my-3">4. Optionally, restrict to Subreddits</div>
                     <div class="my-2 ml-5">
                         <div class="space-y-3">
-                        <div>Specify which subreddits, out of all the subreddits the bot moderates, CM should run on.</div>
-                        <div>Subreddits should be seperated with a comma. Leave blank to run on all moderated subreddits</div>
+                            <div>Specify which subreddits, out of all the subreddits the bot account already moderates, CM should run on.</div>
+                            <div>Subreddits should be seperated with a comma. Leave blank to run on all moderated subreddits.</div>
                         <input id="subreddits" style="max-width:800px; display: block;"
                                class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2 w-full"
                                placeholder="aSubreddit,aSecondSubreddit,aThirdSubreddit">
                         </div>
                     </div>
-                    <div class="text-lg text-semibold my-3">5. Select permissions</div>
+                    <div class="text-lg text-semibold my-3">5. Optionally, specify initial Guest Access</div>
+                    <div class="my-2 ml-5">
+                        <div class="space-y-3">
+                            <div>Specify Reddit users who should be automatically added as <b>Guest Mods</b> to any subreddits found by the bot once authorization is complete.</div>
+                            <div>If you are already a moderator on all of the subreddits being added you can skip this step.</div>
+                            <div>Adding initial Guest Mods is useful when you (the operator) want to setup configs for subreddits you are not a moderator of. This step reduces friction for onboarding as it eliminates the need for moderators to login to the dashboard and manually add you as a Guest Mod.</div>
+                            <input id="guestMods" style="max-width:800px; display: block;"
+                                   class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2 w-full"
+                                   placeholder="RedditUser1,RedditUser2">
+                            <div><strong>Note:</strong> The user completing authorization will be able to opt-out of initial Guest Mods <strong>and</strong> change which users will be used. This step is a convenience for autofilling this information for the authorization process.</div>
+                        </div>
+                    </div>
+                    <div class="text-lg text-semibold my-3">6. Select permissions</div>
                     <div class="my-2 ml-5">
                         <div class="space-y-3">
                             <div>These are permissions to allow the bot account to perform these actions, <b>in
@@ -222,11 +232,9 @@
                             </div>
                         </div>
                     </div>
-                    <div class="text-lg text-semibold my-3">4. <a id="doAuth" href="">Create Authorization Invite</a>
+                    <div class="text-lg text-semibold my-3">7. <a id="doAuth" href="">Create Authorization Invite</a>
                     </div>
                     <div class="ml-5 mb-4">
-                        <input id="inviteCode" style="min-width:500px;"
-                               class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2" placeholder="Invite code value to use. Leave blank to generate a random one."/>
                         <div class="space-y-3">
                             <div>A unique link will be generated that you (or someone) will use to authorize a Reddit account with this application.</div>
                             <div id="inviteLink"></div>
@@ -238,6 +246,7 @@
         </div>
     </div>
 </div>
+<%- include('partials/footer') %>
 <script>
     if (document.querySelector('#redirectUri').value === '') {
         document.querySelector('#redirectUri').value = `${document.location.origin}/callback`;
@@ -259,10 +268,10 @@
                 redirect: document.querySelector('#redirectUri').value,
                 clientId: document.querySelector('#clientId').value,
                 clientSecret: document.querySelector('#clientSecret').value,
-                code: document.querySelector("#inviteCode").value === '' ? undefined : document.querySelector("#inviteCode").value,
                 permissions,
                 instance: document.querySelector('#instanceSelect').value,
-                subreddits: document.querySelector('#subreddits').value
+                subreddits: document.querySelector('#subreddits').value,
+                guests: document.querySelector('#guestMods').value.split(',')
             })
         }).then((resp) => {
             if(!resp.ok) {
@@ -272,9 +281,9 @@
                 });
             } else {
                 document.querySelector("#errorWrapper").classList.add('hidden');
-                document.querySelector("#inviteCode").value = '';
+                document.querySelector('#subreddits').value = '';
                 resp.text().then(t => {
-                    document.querySelector("#inviteLink").innerHTML = `Invite Link: <a class="font-semibold" href="${document.location.origin}/auth/invite?invite=${t}">${document.location.origin}/auth/invite?invite=${t}</a>`;
+                    document.querySelector("#inviteLink").innerHTML = `Invite Link: <a class="font-semibold" href="${document.location.origin}/auth/invite/${t}">${document.location.origin}/auth/invite/${t}</a>`;
                 });
             }
         });

src/Web/assets/views/init.ejs

@@ -0,0 +1,103 @@
+<html lang="en">
+<%- include('partials/head', {title: 'CM'}) %>
+<body class="bg-gray-900 text-white font-sans">
+<div class="min-w-screen min-h-screen">
+    <%- include('partials/title', {title: 'First Time Setup'}) %>
+    <div class="container mx-auto">
+        <div class="grid">
+            <div class="bg-gray-600">
+                <div class="p-6 md:px-10 md:py-6">
+                    <div class="text-xl mb-4">Hi! Looks like you are setting up ContextMod.</div>
+                    <div class="space-y-3">
+                        <div>
+                            It looks like you are setting up ContextMod because either CM could not find a configuration
+                            or your configuration does not include the <a target="_blank"
+                                                                          href="https://github.com/FoxxMD/context-mod/blob/master/docs/operator/configuration.md#minimum-config">minimum
+                                configuration</a> needed to login to the dashboard. <br/>
+                            If you are sure you already have a configuration then make sure it is in a <a
+                                    target="_blank"
+                                    href="https://github.com/FoxxMD/context-mod/blob/master/docs/operator/configuration.md#specify-file-location">default
+                                location or you have specified where to find it.</a>
+                        </div>
+                        <div>
+                            If this is your first time setting up CM and you do not have a configuration then proceed to
+                            generate your minimum configuration.
+                        </div>
+                        <div>
+                            <strong>Note:</strong> If this is a <a target="_blank"
+                                                                   href="https://github.com/FoxxMD/context-mod/blob/master/docs/operator/installation.md#dockerhub">docker
+                                installation</a> then verify you have <strong>bound the config directory</strong> or
+                            else your configuration will be lost the next time you update CM!
+                        </div>
+                    </div>
+                    <div class="text-lg text-semibold my-3">Set the information you got from <a target="_blank"
+                                                                                                href="https://github.com/FoxxMD/context-mod/tree/master/docs/operator#provisioning-a-reddit-client">creating
+                            a Reddit client</a>
+                    </div>
+                    <div class="ml-5 stats" style="max-width: fit-content">
+                        <label for="redirectUri" style="margin:auto">Redirect URI</label>
+                        <input id="redirectUri" style="min-width:500px;"
+                               class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2"
+                               placeholder="http://localhost:8085/callback"/>
+                        <label for="clientId" style="margin:auto">Client ID</label>
+                        <input id="clientId" style="min-width:500px;"
+                               class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2"
+                               placeholder="Client ID">
+                        <label for="clientSecret" style="margin:auto">Client Secret</label>
+                        <input id="clientSecret" style="min-width:500px; display: block;"
+                               class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2"
+                               placeholder="Client Secret">
+                    </div>
+                    <% if(operators === '') { %>
+                        <div class="text-lg text-semibold my-3">Set an Operator</div>
+                        <div class="space-y-3">
+                            This should be <strong>your Reddit username.</strong> CM will use this to determine who can see the "admin" view for CM once you login with your reddit account.
+                            <input id="operator" style="min-width:500px; display: block;"
+                                   class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2"
+                                   placeholder="MyUserName">
+                        </div>
+                    <% } %>
+                    <div class="text-lg text-semibold my-3">7. <a id="doConfig" href="">Write to Config</a></div>
+                    <div id="errorWrapper" class="font-semibold hidden">Error: <span id="error"></span></div>
+                </div>
+            </div>
+        </div>
+        <%- include('partials/footer') %>
+    </div>
+    <script>
+        const operators = '<%= operators %>';
+        if (document.querySelector('#redirectUri').value === '') {
+            document.querySelector('#redirectUri').value = `${document.location.origin}/callback`;
+        }
+        document.querySelector('#doConfig').addEventListener('click', e => {
+            e.preventDefault();
+            fetch(`${document.location.origin}/init`, {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json'
+                },
+                body: JSON.stringify({
+                    redirect: document.querySelector('#redirectUri').value,
+                    clientId: document.querySelector('#clientId').value,
+                    clientSecret: document.querySelector('#clientSecret').value,
+                    operator: document.querySelector('#operator').value
+                })
+            }).then((resp) => {
+                if(!resp.ok) {
+                    document.querySelector("#errorWrapper").classList.remove('hidden');
+                    resp.text().then(t => {
+                        document.querySelector("#error").innerHTML = t;
+                    });
+                } else {
+                    if(operators === '') {
+                        document.querySelector("#errorWrapper").classList.remove('hidden');
+                        document.querySelector('#errorWrapper').innerHTML = 'Success! Because you have set an Operator you must RESTART CM before changes take affect.';
+                    } else {
+                        window.location.href = `${document.location.origin}/login`;
+                    }
+                }
+            });
+        });
+    </script>
+</body>
+</html>

src/Web/assets/views/invite.ejs

@@ -8,7 +8,14 @@
             <div class="bg-gray-600">
                 <div class="p-6 md:px-10 md:py-6">
                     <div class="text-xl mb-4">Hi! Looks like you're accepting an invite to authorize an account to run on this ContextMod instance:</div>
-                    <div class="text-lg text-semibold my-3">1. Review permissions</div>
+                    <div class="text-lg text-semibold my-3">1. Visit this page while <strong>logged in to the Reddit account that will be the bot</strong>
+                    </div>
+                    <div class="ml-5">
+                        <div class="space-y-3">
+                            <div>Protip: Login to Reddit in an Incognito session, then open this URL in a new tab.</div>
+                        </div>
+                    </div>
+                    <div class="text-lg text-semibold my-3">2. Review permissions</div>
                     <div class="my-2 ml-5">
                         <div class="space-y-3">
                             <div>These are permissions to allow the bot account to perform these actions, <b>in
@@ -140,14 +147,20 @@
                             </div>
                         </div>
                     </div>
-                    <div class="text-lg text-semibold my-3">2. Login to Reddit with the account that will be the bot
-                    </div>
-                    <div class="ml-5">
+                    <div class="text-lg text-semibold my-3">3. Choose initial Guest Access</div>
+                    <div class="my-2 ml-5">
                         <div class="space-y-3">
-                            <div>Protip: Login to Reddit in an Incognito session, then open this URL in a new tab.</div>
+                            <div><b>Guests</b> are Reddit users who are NOT moderators of your Subreddits but can still access the Bot's dashboard and modify its configuration. They <strong>only</strong> have access to ContextMod -- not your subreddit in general.</div>
+                            <div>Guest are useful when a reddit user who is not a moderator of your subreddits is helping setup configurations for your bot. Usually this is the Operator of this CM server.</div>
+                            <div>Users specified below will be added to all Subreddits for <strong>24 hours</strong> after which they will be automatically removed. You can manually remove Guest from any/all subreddits at any time from the dashboard.</div>
+                            <input id="guestMods" disabled style="max-width:800px; display: block;"
+                                   class="text-black placeholder-gray-500 rounded mt-2 mb-3 p-2 w-full" value="<%- guests %>">
+                            <% if(guests.length > 0) { %>
+                                <div><strong>Note:</strong> The user(s) above have been pre-filled by the Operator of this server.</div>
+                            <% } %>
                         </div>
                     </div>
-                    <div class="text-lg text-semibold my-3">3. <a id="doAuth" href="">Authorize the account</a>
+                    <div class="text-lg text-semibold my-3">4. <a id="doAuth" href="">Authorize the account</a>
                     </div>
                 </div>
             </div>
@@ -162,7 +175,7 @@
     }
     document.querySelector('#doAuth').addEventListener('click', e => {
         e.preventDefault()
-        const url = `${document.location.origin}/auth/init?invite=<%= invite %>`;
+        const url = `${document.location.origin}/auth/init/<%= invite %>`;
         window.location.href = url;
     })
 </script>

src/Web/assets/views/migrations.ejs

@@ -2,7 +2,7 @@
 <%- include('partials/head', {title: 'CM'}) %>
 <body class="bg-gray-900 text-white font-sans">
 <div class="min-w-screen min-h-screen">
-    <%- include('partials/title', {title: 'Migrations Required'}) %>
+    <%- include('partials/header') %>
     <div class="container mx-auto">
         <div class="grid">
             <div class="bg-gray-600">
@@ -42,6 +42,22 @@
     </div>
     <%- include('partials/footer') %>
 </div>
+<script>
+    const instanceSearchParams = new URLSearchParams(window.location.search);
+
+    const instance = instanceSearchParams.get('instance');
+
+    document.querySelectorAll(`[data-instance].instanceSelectWrapper`).forEach((el) => {
+        if(el.dataset.instance === instance) {
+            el.classList.add('border-2');
+            el.querySelector('a.instanceSelect').classList.add('pointer-events-none','no-underline','font-bold');
+        } else {
+            el.classList.add('border');
+            el.querySelector('a.instanceSelect').classList.add('font-normal','pointer');
+        }
+
+    });
+</script>
 </body>
 <script>
     document.querySelector('#run').addEventListener('click', e => {

src/Web/assets/views/modInvites.ejs

@@ -37,7 +37,10 @@
             } else {
                 resp.json().then(data => {
                     if (data.length > 0) {
-                        document.querySelector('#noSubs').style = 'display: none;';
+                        const ns = document.querySelector('#noSubs');
+                        if(ns !== null) {
+                            document.querySelector('#noSubs').style = 'display: none;';
+                        }
                         sl.removeChild(sl.childNodes[1]);
                     }
                     for (const sub of data) {
@@ -69,7 +72,10 @@
                         document.querySelector("#error").innerHTML = t;
                     });
                 } else {
-                    document.querySelector('#noSubs').style = 'display: none;';
+                    const ns = document.querySelector('#noSubs');
+                    if(ns !== null) {
+                        document.querySelector('#noSubs').style = 'display: none;';
+                    }
                     addSubredditElement(subName);
                     subNameElm.value = '';
                 }
@@ -81,7 +87,7 @@
             node.id = `subInvite-${sub}`;
             var textNode = document.createTextNode(sub);
             node.appendChild(textNode);
-            node.insertAdjacentHTML('beforeend', `<a href="" class="removeSub" id="removeSub-${sub}" data-subreddit="${sub}"><span style="display:inline; margin-left: 10px" class="iconify-inline" data-icon="icons8:cancel"></span><a/>`);
+            node.insertAdjacentHTML('beforeend', `<a href="" class="removeSub" id="removeSub-${sub}" data-subreddit="${sub}"><span style="display:inline; margin-left: 10px" class="iconify-inline" data-icon="icons8:cancel"></span></a>`);
             sl.appendChild(node);
             document.querySelector(`#removeSub-${sub}`).addEventListener('click', e => {
                 e.preventDefault();

src/Web/assets/views/partials/botsTab.ejs

@@ -27,7 +27,7 @@
             <% if(locals.isOperator === true && locals.instanceId !== undefined) { %>
                     <li class="my-3 px-3">
                         <span class="rounded-md py-2 px-3 border">
-                        <a class="font-normal pointer hover:font-bold" href="/auth/helper">
+                        <a class="font-normal pointer hover:font-bold" href="/auth/helper?instance=<%= locals.instanceId %>">
                             Add Bot +
                         </a>
                         </span>

src/Web/assets/views/partials/head.ejs

@@ -12,4 +12,5 @@
     <!-- https://developers.google.com/search/docs/advanced/crawling/block-indexing#meta-tag -->
     <meta name="robots" content="noindex">
     <!--icons from https://heroicons.com -->
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/intro.js/6.0.0/introjs.min.css" crossorigin="anonymous" referrerpolicy="no-referrer" />
 </head>

src/Web/assets/views/partials/header.ejs

@@ -33,6 +33,13 @@
                 </ul>
                 <% } %>
             </div>
+            <% if(locals.showHelp === true) { %>
+            <div id="help" class="flex items-center mr-8 text-sm">
+                <a id="helpStart" href="" >
+                    Help
+                </a>
+            </div>
+            <% } %>
             <div class="flex items-center mr-8 text-sm">
                 <a href="https://redditstatus.com" target="_blank">
                     <span>Reddit Status: <span id="redditStatus" class="ml-2"><span class="iconify-inline" data-icon="ep:question-filled"></span></span></span>

src/Web/assets/views/partials/subredditsTab.ejs

@@ -1,5 +1,5 @@
 <div class="space-x-4 pt-2 md:px-5 leading-6 font-semibold bg-gray-800">
-    <div class="container mx-auto">
+    <div id="subredditsTab" class="container mx-auto">
         <% if(locals.bots !== undefined) { %>
         <% bots.forEach(function (botData){ %>
             <ul data-bot="<%= botData.system.name %>" class="inline-flex flex-wrap subreddit nestedTabs">

src/Web/assets/views/status.ejs

@@ -2,7 +2,7 @@
 <%- include('partials/head', {title: undefined}) %>
 <body class="bg-gray-900 text-white">
 <div class="min-w-screen min-h-screen font-sans">
-    <%- include('partials/header') %>
+    <%- include('partials/header', {showHelp: true}) %>
     <%- include('partials/botsTab') %>
     <%- include('partials/subredditsTab') %>
     <div class="container mx-auto">
@@ -26,7 +26,7 @@
                         <% bot.subreddits.forEach(function (data){ %>
                         <div class="sub <%= bot.system.running ? '' : 'offline' %>" data-subreddit="<%= data.name %>" data-bot="<%= bot.system.name %>">
                             <div class="grid grid-cols-1 lg:grid-cols-2 xl:grid-cols-3 2xl:grid-cols-3 gap-5">
-                                <div class="bg-white shadow-md rounded my-3 bg-gray-600 ">
+                                <div class="bg-white shadow-md rounded my-3 bg-gray-600 overviewContainer">
                                     <div class="space-x-4 px-4 p-2 leading-2 font-semibold bg-gray-300 bg-gray-700 ">
                                         <div class="flex items-center justify-between">
                                             <h4>Overview</h4>
@@ -261,6 +261,42 @@
                                                     <span class="cursor-help underline" style="text-decoration-style: dotted"><%= data.scopes.length %></span>
                                                 </span>
                                             <% } else %>
+                                            <label class="guestsLabel <%= (!isOperator && !data.isMod ? 'hidden' : '')%>">
+                                                <span class="has-tooltip">
+                                                       <span style="margin-top:55px" class='tooltip rounded shadow-lg p-1 bg-gray-100 text-black space-y-3 p-2 text-left'>
+                                                            <p>Reddit users who are allowed to access your bot even though they are not moderators.</p>
+                                                            <p>Guest can do everything a regular mod can except view/add/remove Guest.</p>
+                                                            <p>Additionally, they can <b>edit the subreddit's config using the bot.</b> If a Guest edits your config their username will be mentioned in the wiki page edit reason.</p>
+                                                        </span>
+                                                        <span>
+                                                            Guests<svg xmlns="http://www.w3.org/2000/svg"
+                                                                          class="h-4 w-4 inline-block cursor-help"
+                                                                          fill="none"
+                                                                          viewBox="0 0 24 24" stroke="currentColor">
+                                                                <use xlink:href="public/questionsymbol.svg#q" />
+                                                            </svg>
+                                                        </span>
+                                                    </span>
+                                            </label>
+                                            <span class="guests <%= (!isOperator && !data.isMod ? 'hidden' : '')%>" style="margin-left: 5px;">
+                                                <ul class="list-disc list-inside guestList">
+                                                    <li class="smallLi">None</li>
+                                                </ul>
+                                                <div class="guestAdd <%= (!data.isMod ? 'hidden' : '')%> inline-flex items-center mt-1">
+                                                    <div>
+                                                        <input
+                                                                               style="width:200px;"
+                                                                               class="guestAddName border-gray-50 placeholder-gray-500 rounded mr-1 p-1 text-black"
+                                                        placeholder="userName"/>
+                                                        <input type="datetime-local"
+                                                                                         class="guestAddTime border-gray-50 placeholder-gray-500 mt-2 mr-2 rounded text-black"
+                                                                                         value="<%= now %>"
+                                                                                         min="<%= now %>"/>
+
+                                                    </div>
+                                                    <a href="" class="addGuest">Add</a>
+                                                </div>
+                                            </span>
                                         </div>
                                         <% if (data.name !== 'All') { %>
                                             <ul class="list-disc list-inside mt-4 pollingInfo">
@@ -272,7 +308,7 @@
                                     </div>
                                 </div>
                                 <% if (data.name === 'All') { %>
-                                    <div class="bg-white shadow-md rounded my-3 bg-gray-600 ">
+                                    <div class="bg-white shadow-md rounded my-3 bg-gray-600 configContainer">
                                         <div class="space-x-4 px-4 p-2 leading-2 font-semibold bg-gray-300 bg-gray-700 ">
                                             <h4>API</h4>
                                         </div>
@@ -332,7 +368,7 @@
                                     </div>
                                 <% } %>
                                 <% if (data.name !== 'All') { %>
-                                    <div class="bg-white shadow-md rounded my-3 bg-gray-600 ">
+                                    <div class="bg-white shadow-md rounded my-3 bg-gray-600 configContainer">
                                         <div class="space-x-4 px-4 p-2 leading-2 font-semibold bg-gray-700 ">
                                             <h4>Config
                                                     <span class="has-tooltip">
@@ -349,7 +385,7 @@
                                                 <span class="font-semibold validConfig"><%= data.validConfig %></span>
                                                 <label>Checks</label>
                                                 <span><span class="submissionCheckCount"><%= data.checks.submissions %></span> Submission | <span class="commentCheckCount"><%= data.checks.comments %></span> Comment </span>
-                                                <label>Dry Run</label>
+                                                <label class="dryRunLabel">Dry Run</label>
                                                 <span><%= data.dryRun %></span>
                                                 <label>Updated</label>
                                                 <span class="has-tooltip">
@@ -375,14 +411,14 @@
                                                 <label>Location</label>
                                                 <span>
                                                     <a style="display: inline"
-                                                       href="<%= data.wikiHref %>"><%= data.wikiLocation %></a> | <a style="display: inline" target="_blank"
+                                                       href="<%= data.wikiHref %>"><%= data.wikiLocation %></a> | <a class="openConfig" style="display: inline" target="_blank"
                                                                                                                      href="/config?format=<%= data.configFormat %>&instance=<%= instanceId %>&bot=<%= bot.system.name %>&subreddit=<%= data.name %>">View</a>
                                                 </span>
                                             </div>
                                         </div>
                                     </div>
                                 <% } %>
-                                <div class="bg-white shadow-md rounded my-3 bg-gray-600 ">
+                                <div class="bg-white shadow-md rounded my-3 bg-gray-600 usageContainer">
                                     <div class="space-x-4 px-4 p-2 leading-2 font-semibold bg-gray-700 ">
                                         <h4>Usage</h4>
                                     </div>
@@ -417,9 +453,9 @@
                                                     </span>
                                                 </div>
                                                 <% if (data.name !== 'All') { %>
-                                                    <a target="_blank" href="/events?instance=<%= instanceId %>&bot=<%= bot.system.name %>&subreddit=<%= data.name %>" style="text-decoration-style: dotted">Actioned Events</a>
+                                                    <a class="openActioned" target="_blank" href="/events?instance=<%= instanceId %>&bot=<%= bot.system.name %>&subreddit=<%= data.name %>" style="text-decoration-style: dotted">Actioned Events</a>
                                                 <% } else { %>
-                                                    <a target="_blank" href="/events?instance=<%= instanceId %>&bot=<%= bot.system.name %>">Actioned Events</a>
+                                                    <a class="openActioned" target="_blank" href="/events?instance=<%= instanceId %>&bot=<%= bot.system.name %>">Actioned Events</a>
                                                 <% } %>
                                             </div>
                                             <div>
@@ -620,7 +656,7 @@
                             </div>
                             <br/>
                             <div class="flex items-center justify-between flex-wrap">
-                                <div class="inline-flex items-center">
+                                <div class="inline-flex items-center runBotOnThing">
                                     <div class="relative" style="width:550px; display: inline-block;">
                                     <input data-subreddit="<%= data.name %>"
                                            style="width: 100%;"
@@ -679,6 +715,9 @@
 <script>
     window.sort = 'desc';
 
+    const isOperator = <%= isOperator %>;
+    window.isOperator = <%= isOperator %>;
+
     document.querySelectorAll('.action').forEach(el => {
         el.addEventListener('click', e => {
             e.preventDefault();
@@ -726,10 +765,10 @@
             const isDryun = e.target.classList.contains('dryRunCheck');
 
             const subSection = e.target.closest('div.sub');
-            bot = subSection.dataset.bot;
+            const botSection = subSection.dataset.bot;
             const url = urlInput.value;
 
-            const fetchUrl = `/api/check?instance=<%= instanceId %>&bot=${bot}&url=${url}&dryRun=${isDryun ? 1 : 0}&subreddit=${subreddit}&delayOption=${delayOpt}`;
+            const fetchUrl = `/api/check?instance=<%= instanceId %>&bot=${botSection}&url=${url}&dryRun=${isDryun ? 1 : 0}&subreddit=${subreddit}&delayOption=${delayOpt}`;
             fetch(fetchUrl);
 
             urlInput.value = '';
@@ -999,13 +1038,17 @@
             });
 
         fetchPromise.then(async res => {
+            // can potentially happen if request is aborted (by immediate log cancel) before logs begin to be read
+            if(res === undefined) {
+                return;
+            }
             const reader = res.getReader();
             let keepReading = true;
             while(keepReading) {
-                const {done, value} = await reader.read();
+                const {done, value, ...rest} = await reader.read();
                 if(done) {
                     keepReading = false;
-                    console.debug('done');
+                    console.debug(`${bot}.${sub} log stream reader signalled it is done`);
                 }
                 if(value) {
                     //console.log(`((Logged For ${bot} ${sub})) ${value.message}`);
@@ -1039,178 +1082,474 @@
                             read();*/
         }).catch((e) => {
             if(e.name !== 'AbortError') {
+                console.debug(`Non-abort error occurred while streaming logs for ${bot} ${sub}`);
                 console.error(e);
+            } else {
+                console.debug(`Log streaming for ${bot} ${sub} aborted`);
             }
         });
 
         const existing = recentlySeen.get(`${bot}.${sub}`) || {};
-        recentlySeen.set(`${bot}.${sub}`, {...existing, fetch: fetchPromise, controller});
+        recentlySeen.set(`${bot}.${sub}`, {...existing, fetch: fetchPromise, controller, streamStart: Date.now()});
     }
 
-    function updateLiveStats(resp) {
+    const delayedItemsMap = new Map();
+    let lastSeenIdentifier = null;
+
+    function updateLiveStats(resp, sub, bot, responseType) {
         let el;
-        let isAll = resp.name.toLowerCase() === 'all';
+        let isAll = sub.toLowerCase() === 'all';
         if(isAll) {
             // got all
-            el = document.querySelector(`[data-subreddit="All"][data-bot="${resp.bot}"].sub`);
+            el = document.querySelector(`[data-subreddit="All"][data-bot="${bot}"].sub`);
         } else {
             // got subreddit
-            el = document.querySelector(`[data-subreddit="${resp.name}"].sub`);
+            el = document.querySelector(`[data-subreddit="${sub}"].sub`);
         }
 
-        if(resp.system.running && el.classList.contains('offline')) {
-            el.classList.remove('offline');
-        } else if(!resp.system.running && !el.classList.contains('offline')) {
-            el.classList.add('offline');
+        const {
+            system: {
+                running
+            } = {},
+            delayedItems,
+            guests,
+            runningActivities,
+            queuedActivities,
+            permissions,
+            stats: {
+                historical: {
+                    eventsCheckedTotal,
+                    checksTriggeredTotal,
+                    rulesTriggeredTotal,
+                    actionsRunTotal,
+                } = {},
+            } = {},
+            checks: {
+              comments,
+              submissions
+            } = {},
+            pollingInfo,
+        } = resp;
+
+        if(running !== undefined) {
+            if(resp.system.running && el.classList.contains('offline')) {
+                el.classList.remove('offline');
+            } else if(!resp.system.running && !el.classList.contains('offline')) {
+                el.classList.add('offline');
+            }
         }
 
-        el.querySelector('.runningActivities').innerHTML = resp.runningActivities;
-        el.querySelector('.queuedActivities').innerHTML = resp.queuedActivities;
-        el.querySelector('.delayedItemsCount').innerHTML = resp.delayedItems.length;
-        el.querySelector('.delayedItemsList').innerHTML = 'No delayed Items!';
-        if(resp.delayedItems.length > 0) {
-            el.querySelector('.delayedItemsList').innerHTML = '';
-            const now = dayjs();
-            const sorted = resp.delayedItems.map(x => ({...x, queuedAtUnix: x.queuedAt, queuedAt: dayjs.unix(x.queuedAt), dispatchAt: dayjs.unix(x.queuedAt + x.duration)}));
-            sorted.sort((a, b) => {
-                return a.dispatchAt.isSameOrAfter(b.dispatchAt) ? 1 : -1
-            });
-            const delayedItemDivs = sorted.map(x => {
-                const diffUntilNow = x.dispatchAt.diff(now);
-                const durationUntilNow = dayjs.duration(diffUntilNow, 'ms');
-                const queuedAtDisplay = x.queuedAt.format('HH:mm:ss z');
-                const durationDayjs = dayjs.duration(x.duration, 'seconds');
-                const durationDisplay = durationDayjs.humanize();
-                const cancelLink = `<a href="#" data-id="${x.id}" data-subreddit="${x.subreddit}" class="delayCancel">CANCEL</a>`;
-                return `<div>A <a href="https://reddit.com${x.permalink}">${x.submissionId !== undefined ? 'Comment' : 'Submssion'}</a>${isAll ? ` in <a href="https://reddit.com${x.subreddit}">${x.subreddit}</a> ` : ''} by <a href="https://reddit.com/u/${x.author}">${x.author}</a> queued by ${x.source} at ${queuedAtDisplay} for ${durationDisplay} (dispatches ${durationUntilNow.humanize(true)}) -- ${cancelLink}</div>`;
-            });
-            el.querySelector('.delayedItemsList').insertAdjacentHTML('afterbegin', delayedItemDivs.join(''));
-            el.querySelectorAll('.delayedItemsList .delayCancel').forEach(elm => {
-                elm.addEventListener('click', e => {
-                    e.preventDefault();
-                    const id = e.target.dataset.id;
-                    const subreddit = e.target.dataset.subreddit;
-                    fetch(`/api/delayed?instance=<%= instanceId %>&bot=${resp.bot}&subreddit=${subreddit}&id=${id}`, {
+        if(runningActivities !== undefined) {
+            el.querySelector('.runningActivities').innerHTML = resp.runningActivities;
+        }
+        if(queuedActivities !== undefined) {
+            el.querySelector('.queuedActivities').innerHTML = resp.queuedActivities;
+        }
+
+        if(delayedItems !== undefined) {
+            let items = [];
+            if(responseType === 'full') {
+                delayedItemsMap.clear();
+                for(const i of delayedItems) {
+                    delayedItemsMap.set(i.id, i);
+                }
+                items = delayedItems;
+            } else {
+                for(const n of delayedItems.new) {
+                    delayedItemsMap.set(n.id, n);
+                }
+                for(const n of delayedItems.removed) {
+                    delayedItemsMap.delete(n);
+                }
+                items = Array.from(delayedItemsMap.values());
+            }
+            el.querySelector('.delayedItemsCount').innerHTML = items.length;
+            el.querySelector('.delayedItemsList').innerHTML = 'No delayed Items!';
+            if(items.length > 0) {
+                el.querySelector('.delayedItemsList').innerHTML = '';
+                const now = dayjs();
+                const sorted = items.map(x => ({...x, queuedAtUnix: x.queuedAt, queuedAt: dayjs.unix(x.queuedAt), dispatchAt: dayjs.unix(x.queuedAt + x.duration)}));
+                sorted.sort((a, b) => {
+                    return a.dispatchAt.isSameOrAfter(b.dispatchAt) ? 1 : -1
+                });
+                const delayedItemDivs = sorted.map(x => {
+                    const diffUntilNow = x.dispatchAt.diff(now);
+                    const durationUntilNow = dayjs.duration(diffUntilNow, 'ms');
+                    const queuedAtDisplay = x.queuedAt.format('HH:mm:ss z');
+                    const durationDayjs = dayjs.duration(x.duration, 'seconds');
+                    const durationDisplay = durationDayjs.humanize();
+                    const cancelLink = `<a href="#" data-id="${x.id}" data-subreddit="${x.subreddit}" class="delayCancel">CANCEL</a>`;
+                    return `<div>A <a href="https://reddit.com${x.permalink}">${x.submissionId !== undefined ? 'Comment' : 'Submission'}</a>${isAll ? ` in <a href="https://reddit.com${x.subreddit}">${x.subreddit}</a> ` : ''} by <a href="https://reddit.com/u/${x.author}">${x.author}</a> queued by ${x.source} at ${queuedAtDisplay} for ${durationDisplay} (dispatches ${durationUntilNow.humanize(true)}) -- ${cancelLink}</div>`;
+                });
+                //let sub = resp.name;
+                if(sub === 'All') {
+                    sub = sorted.reduce((acc, curr) => {
+                        if(!acc.includes(curr.subreddit)) {
+                            return acc.concat(curr.subreddit);
+                        }
+                        return acc;
+                    },[]).join(',');
+                }
+                delayedItemDivs.unshift(`<div><a href="#"data-subreddit="${sub}" class="delayCancelAll">Cancel ALL</a></div>`)
+                el.querySelector('.delayedItemsList').insertAdjacentHTML('afterbegin', delayedItemDivs.join(''));
+                el.querySelectorAll('.delayedItemsList .delayCancel').forEach(elm => {
+                    elm.addEventListener('click', e => {
+                        e.preventDefault();
+                        const id = e.target.dataset.id;
+                        const subreddit = e.target.dataset.subreddit;
+                        fetch(`/api/delayed?instance=<%= instanceId %>&bot=${bot}&subreddit=${subreddit}&id=${id}`, {
+                            method: 'DELETE'
+                        }).then((resp) => {
+                            if (!resp.ok) {
+                                console.error('Response was not OK from delay cancel');
+                            } else {
+                                console.log('Removed ok');
+                            }
+                        });
+                    });
+                });
+                el.querySelectorAll('.delayedItemsList .delayCancelAll').forEach(elm => {
+                    elm.addEventListener('click', e => {
+                        e.preventDefault();
+                        const subreddit = e.target.dataset.subreddit;
+                        deleteDelayedActivities(bot, subreddit);
+
+                        /*fetch(`/api/delayed?instance=<%= instanceId %>&bot=${bot}&subreddit=${subreddit}`, {
                         method: 'DELETE'
                     }).then((resp) => {
                         if (!resp.ok) {
-                            console.error('Response was not OK from delay cancel');
+                            console.error('Response was not OK from delay cancel ALL');
                         } else {
-                            console.log('Removed ok');
+                            console.log('Removed ALL ok');
                         }
+                    });*/
                     });
                 });
-            });
+            }
         }
 
-        el.querySelector('.allStats .eventsCount').innerHTML = resp.stats.historical.eventsCheckedTotal;
-        el.querySelector('.allStats .checksCount').innerHTML = resp.stats.historical.checksTriggeredTotal;
-        el.querySelector('.allStats .rulesCount').innerHTML = resp.stats.historical.rulesTriggeredTotal;
-        el.querySelector('.allStats .actionsCount').innerHTML = resp.stats.historical.actionsRunTotal;
+        if(guests !== undefined) {
+            renderGuestMods(bot, sub, guests);
+        }
+
+        if(eventsCheckedTotal !== undefined) {
+            el.querySelector('.allStats .eventsCount').innerHTML = resp.stats.historical.eventsCheckedTotal;
+        }
+        if(checksTriggeredTotal !== undefined) {
+            el.querySelector('.allStats .checksCount').innerHTML = resp.stats.historical.checksTriggeredTotal;
+        }
+        if(rulesTriggeredTotal !== undefined) {
+            el.querySelector('.allStats .rulesCount').innerHTML = resp.stats.historical.rulesTriggeredTotal;
+        }
+        if(actionsRunTotal !== undefined) {
+            el.querySelector('.allStats .actionsCount').innerHTML = resp.stats.historical.actionsRunTotal;
+        }
 
         if(isAll) {
             for(const elm of ['apiAvg','apiLimit','apiDepletion','nextHeartbeat', 'nextHeartbeatHuman', 'limitReset', 'limitResetHuman', 'nannyMode', 'startedAtHuman']) {
-                el.querySelector(`#${elm}`).innerHTML = resp[elm];
+                if(resp[elm] !== undefined) {
+                    el.querySelector(`#${elm}`).innerHTML = resp[elm];
+                }
+            }
+
+            if(running !== undefined) {
+                el.querySelector(`.botStatus`).innerHTML = resp.system.running ? 'ONLINE' : 'OFFLINE';
             }
-            el.querySelector(`.botStatus`).innerHTML = resp.system.running ? 'ONLINE' : 'OFFLINE';
         } else {
-            if(el.querySelector('.modPermissionsCount').innerHTML != resp.permissions.length) {
-                el.querySelector('.modPermissionsCount').innerHTML = resp.permissions.length;
-                el.querySelector('.modPermissionsList').innerHTML = '';
-                el.querySelector('.modPermissionsList').insertAdjacentHTML('afterbegin', resp.permissions.map(x => `<li class="font-mono">${x}</li>`).join(''));
+            if(permissions !== undefined) {
+                if(el.querySelector('.modPermissionsCount').innerHTML != resp.permissions.length) {
+                    el.querySelector('.modPermissionsCount').innerHTML = resp.permissions.length;
+                    el.querySelector('.modPermissionsList').innerHTML = '';
+                    el.querySelector('.modPermissionsList').insertAdjacentHTML('afterbegin', resp.permissions.map(x => `<li class="font-mono">${x}</li>`).join(''));
+                }
             }
 
             for(const elm of ['botState', 'queueState', 'eventsState']) {
-                const state = resp[elm];
-                el.querySelector(`.${elm}`).innerHTML = `${state.state}${state.causedBy === 'system' ? '' : ' (user)'}`;
+                if(resp[elm] !== undefined) {
+                    const state = resp[elm];
+                    el.querySelector(`.${elm}`).innerHTML = `${state.state}${state.causedBy === 'system' ? '' : ' (user)'}`;
+                }
             }
             for(const elm of ['startedAt', 'startedAtHuman', 'wikiLastCheck', 'wikiLastCheckHuman', 'wikiRevision', 'wikiRevisionHuman', 'validConfig', 'delayBy']) {
-                el.querySelector(`.${elm}`).innerHTML = resp[elm];
+                if(resp[elm] !== undefined) {
+                    el.querySelector(`.${elm}`).innerHTML = resp[elm];
+                }
+            }
+            if(comments !== undefined) {
+                el.querySelector(`.commentCheckCount`).innerHTML = resp.checks.comments;
+            }
+            if(submissions !== undefined) {
+                el.querySelector(`.submissionCheckCount`).innerHTML = resp.checks.submissions;
             }
-            el.querySelector(`.commentCheckCount`).innerHTML = resp.checks.comments;
-            el.querySelector(`.submissionCheckCount`).innerHTML = resp.checks.submissions;
 
-            const newInner = resp.pollingInfo.map(x => `<li>${x}</li>`).join('');
-            if(el.querySelector(`.pollingInfo`).innerHTML !== newInner) {
-                el.querySelector(`.pollingInfo`).innerHTML = newInner;
+            if(pollingInfo !== undefined) {
+                const newInner = resp.pollingInfo.map(x => `<li>${x}</li>`).join('');
+                if(el.querySelector(`.pollingInfo`).innerHTML !== newInner) {
+                    el.querySelector(`.pollingInfo`).innerHTML = newInner;
+                }
             }
         }
     }
 
-    function getLiveStats(bot, sub) {
+    function deleteDelayedActivities(bot, subredditStr, id) {
+        const subs = subredditStr.split(',');
+        fetch(`/api/delayed?instance=<%= instanceId %>&bot=${bot}&subreddit=${subs[0]}${id !== undefined ? `&id=${id}` : ''}`, {
+            method: 'DELETE'
+        }).then((resp) => {
+            if (!resp.ok) {
+                if(id === undefined) {
+                    console.error(`Response was not OK from ${subs[0]} delay ALL cancel`);
+                } else {
+                    console.error(`Response was not OK from ${subs[0]} delay cancel ${id}`);
+                }
+            } else {
+                if(id === undefined) {
+                    console.log(`Removed ALL for ${subs[0]} ok`);
+                } else {
+                    console.log(`Removed ${id} for ${subs[0]} ok`);
+                }
+                if(subs.length > 1) {
+                    deleteDelayedActivities(bot, subs.slice(1).join(','));
+                }
+            }
+        });
+    }
+
+    function removeGuestMod(bot, subredditStr, name) {
+        const subs = subredditStr.split(',');
+        fetch(`/api/guests?instance=<%= instanceId %>&bot=${bot}&subreddit=${subs[0]}&name=${name}`, {
+            method: 'DELETE'
+        }).then((resp) => {
+            if (!resp.ok) {
+                throw new Error(`Response was not OK from ${subs[0]} remove guest ${id}${name}`)
+            } else {
+                return resp.json();
+            }
+        }).then((data) => {
+            renderGuestMods(bot, subs[0], data);
+        });
+    }
+
+    function addGuestMod(bot, subredditStr, name, time) {
+        const subs = subredditStr.split(',');
+        fetch(`/api/guests?instance=<%= instanceId %>&bot=${bot}&subreddit=${subs[0]}&time=${time}&name=${name}`, {
+            method: 'POST'
+        }).then((resp) => {
+            if (!resp.ok) {
+                throw new Error(`Response was not OK from ${subs[0]} add guest ${name}`);
+            } else {
+                return resp.json();
+            }
+        }).then((data) => {
+            renderGuestMods(bot, subs[0], data);
+            document.querySelector(`[data-bot="${bot}"][data-subreddit="${subs[0]}"] .guestAddName`).value = '';
+            //document.querySelector(`[data-bot="${bot}"][data-subreddit="${subs[0]}"] .guestAddTime`).value = dayjs().add(1, 'minutes').format('YYYY-MM-DDTHH:mm');
+        });
+    }
+
+    function renderGuestMods(bot, sub, data) {
+        let el;
+        let isAll = sub.toLowerCase() === 'all';
+        if(isAll) {
+            // got all
+            el = document.querySelector(`[data-subreddit="All"][data-bot="${bot}"] .guestList`);
+        } else {
+            // got subreddit
+            el = document.querySelector(`[data-bot="${bot}"][data-subreddit="${sub}"] .guestList`);
+        }
+
+        const now = dayjs();
+
+        el.innerHTML = '';
+        if(data.length === 0) {
+            const node = document.createElement("LI");
+            node.classList.add('smallLi');
+            node.appendChild(document.createTextNode('None'));
+            el.appendChild(node);
+        } else {
+            for(const g of data) {
+                const node = document.createElement("LI");
+                node.classList.add('smallLi');
+                let relTime = g.expiresAt === undefined ? 'Never' : dayjs.duration(dayjs(g.expiresAt).diff(now)).humanize();
+                let guestText = g.name;
+                if(isAll) {
+                    guestText += ` (${g.subreddits.length} Subs, at least ${relTime})`;
+                } else {
+                    guestText += ` (${relTime})`;
+                }
+                node.appendChild(document.createTextNode(guestText));
+                node.insertAdjacentHTML('beforeend', `<a href="" class="remove ml-1" data-name="${g.name}"><span class="cancellable iconify-inline" data-icon="icons8:cancel"></span></a>`);
+                node.addEventListener('click', e => {
+                    e.preventDefault();
+                    removeGuestMod(bot, sub, g.name);
+                });
+                el.appendChild(node);
+            }
+        }
+    }
+
+    document.querySelectorAll('.addGuest').forEach(elm => {
+        elm.addEventListener('click', (e) => {
+            e.preventDefault();
+            const el = e.target;
+            const parent = el.closest('.sub');
+            const sub = parent.dataset.subreddit;
+            const bot = parent.dataset.bot;
+
+            const userEl = el.parentElement.querySelector('input.guestAddName');
+            const timeEl = el.parentElement.querySelector('input.guestAddTime');
+
+            const d = dayjs(timeEl.value);
+            // don't allow users to set a time before now
+            const time = d.isBefore(dayjs()) ? dayjs().add(1, 'minute').valueOf() : d.valueOf();
+            const user = userEl.value;
+
+            console.log(`Adding ${user} expiring at ${time} to ${bot}.${sub}`);
+
+            addGuestMod(bot, sub, user, time)
+        });
+    });
+
+    function getLiveStats(bot, sub, responseType = 'full') {
         console.debug(`Getting live stats for ${bot} ${sub}`)
-        return fetch(`/api/liveStats?instance=<%= instanceId %>&bot=${bot}&subreddit=${sub}`)
-            .then(response => response.json())
-            .then(resp => updateLiveStats(resp));
+        return fetch(`/api/liveStats?instance=<%= instanceId %>&bot=${bot}&subreddit=${sub}&type=${responseType}`)
+            .then(response => {
+                if(response.status === 304) {
+                    return Promise.resolve(false);
+                }
+                return response.json();
+            })
+            .then(resp => {
+                if(resp === false) {
+                    return;
+                }
+                updateLiveStats(resp, sub, bot, responseType);
+            });
+    }
+
+    function onSubVisible (bot, sub) {
+        const identifier = `${bot}.${sub}`
+        lastSeenIdentifier = identifier;
+
+        console.debug(`Focused on ${identifier}`);
+
+        let immediateCancel = [];
+        const notNew = Array.from(recentlySeen.entries()).filter(([k,v]) => k !== identifier);
+        // browsers have a default limit for number of concurrent connections
+        // which INCLUDES streaming responses (logs)
+        // so we need to keep number of idle streaming logs low to prevent browser from hanging on new requests
+        if(notNew.length > 2) {
+            notNew.sort((a, b) =>  a[1].streamStart - b[1].streamStart);
+            immediateCancel = notNew.slice(2).map(x => x[0]);
+            console.debug(`More than 2 other views are still streaming logs! Will immediately stop the oldest (skipping two earliest): ${immediateCancel.join(' , ')}`);
+        }
+
+        recentlySeen.forEach((value, key) => {
+            const {timeout, liveStatsInt,...rest} = value;
+            if(key === identifier && timeout !== undefined) {
+
+                console.debug(`${key} Clearing unfocused timeout on own already set`);
+                clearTimeout(timeout);
+                recentlySeen.set(key, rest);
+
+            } else if(key !== identifier) {
+
+                // stop live stats for tabs we are not viewing
+                clearInterval(liveStatsInt);
+
+                if(immediateCancel.includes(key)) {
+                    const {controller} = value;
+                    if(controller !== undefined) {
+                        console.debug(`${key} Stopping logs IMMEDIATELY`);
+                        controller.abort();
+                        recentlySeen.delete(key);
+                    }
+                } else
+                    // set timeout for logs we are not viewing
+                if(timeout === undefined) {
+                    const t = setTimeout(() => {
+                        const k = key;
+                        const val = recentlySeen.get(k);
+                        if(val !== undefined) {
+                            const {controller} = val;
+                            console.debug(`${k} 15 second unfocused timeout expired, stopping log streaming`);
+                            if(controller !== undefined) {
+                                console.debug(`${k} Stopping logs`);
+                                controller.abort();
+                            }
+                            recentlySeen.delete(k);
+                        }
+                    },15000);
+                    recentlySeen.set(key, {timeout: t, liveStatsInt, ...rest});
+                }
+            }
+        });
+
+        if(!recentlySeen.has(identifier)) {
+            getLogBlock(bot, sub).then(() => {
+                getStreamingLogs(sub, bot);
+            });
+        }
+
+        delayedItemsMap.clear();
+        // always get live stats for tab we just started viewing
+        getLiveStats(bot, sub).then(() => {
+            let liveStatsInt;
+            const liveStatFunc = () => {
+                // after initial live stats "full frame" only request deltas to reduce data usage
+                getLiveStats(bot, sub, 'delta').catch((err) => {
+                    console.error(err);
+                    // stop interval if live stat encounters an error
+                    clearInterval(liveStatsInt);
+                })
+            };
+            liveStatsInt = setInterval(liveStatFunc, 5000);
+            const existing = recentlySeen.get(identifier) ?? {};
+            recentlySeen.set(identifier, {...existing, bot, sub, liveStatsInt});
+        });
     }
 
     document.querySelectorAll('.sub').forEach(el => {
         const sub = el.dataset.subreddit;
         const bot = el.dataset.bot;
-        //console.log(`Focused on ${bot} ${sub}`);
-        onVisible(el, () => {
-            console.debug(`Focused on ${bot} ${sub}`);
-
-            const identifier = `${bot}.${sub}`;
-
-            recentlySeen.forEach((value, key) => {
-                const {timeout, liveStatsInt, ...rest} = value;
-                if(key === identifier && timeout !== undefined) {
-
-                    console.debug('Clearing timeout on own already set');
-                    clearTimeout(timeout);
-                    recentlySeen.set(key, rest);
-
-                } else if(key !== identifier) {
-
-                    // stop live stats for tabs we are not viewing
-                    clearInterval(liveStatsInt);
-
-                    // set timeout for logs we are not viewing
-                    if(timeout === undefined) {
-                        const t = setTimeout(() => {
-                            const k = key;
-                            const val = recentlySeen.get(k);
-                            if(val !== undefined) {
-                                const {controller} = val;
-                                console.debug(`timeout expired, stopping live data for ${k}`);
-                                if(controller !== undefined) {
-                                    console.debug('Stopping logs');
-                                    controller.abort();
-                                }
-                                // if(liveStatInt !== undefined) {
-                                //     console.log('Stopping live stats');
-                                //     clearInterval(liveStatInt);
-                                // }
-                                recentlySeen.delete(k);
-                            }
-                        },15000);
-                        recentlySeen.set(key, {timeout: t, liveStatsInt, ...rest});
-                    }
-                }
-            });
-
-            if(!recentlySeen.has(identifier)) {
-                getLogBlock(bot, sub).then(() => {
-                    getStreamingLogs(sub, bot);
-                });
-            }
-
-            // always get live stats for tab we just started viewing
-            getLiveStats(bot, sub).then(() => {
-                let liveStatsInt;
-                const liveStatFunc = () => {
-                    getLiveStats(bot, sub).catch(() => {
-                        // stop interval if live stat encounters an error
-                        clearInterval(liveStatsInt);
-                    })
-                };
-                liveStatsInt = setInterval(liveStatFunc, 5000);
-                recentlySeen.set(identifier, {liveStatsInt});
-            });
-        });
+        onVisible(el, () => onSubVisible(bot, sub));
     });
 
+    let backgroundTimeout = null;
+
+    document.addEventListener("visibilitychange", (e) => {
+        if (document.visibilityState === "hidden") {
+            console.debug(`Set 15 seconds timeout for ${lastSeenIdentifier} live data due to page not being visible`);
+            backgroundTimeout = setTimeout(() => {
+                console.debug(`Stopping live data for ${lastSeenIdentifier} due to page not being visible`);
+
+                const {liveStatsInt, controller} = recentlySeen.get(lastSeenIdentifier) ?? {};
+                if(liveStatsInt !== undefined && liveStatsInt !== null) {
+                    clearInterval(liveStatsInt);
+                }
+                if(controller !== undefined && controller !== null) {
+                    controller.abort();
+                }
+                backgroundTimeout = null;
+            }, 15000);
+        } else {
+            // cancel real-time data timeout because page is visible again
+            if(backgroundTimeout !== null) {
+                console.debug(`Cancelled live-data timeout for ${lastSeenIdentifier}`);
+                clearTimeout(backgroundTimeout);
+                backgroundTimeout = null;
+            } else if(lastSeenIdentifier !== null) {
+                // if timeout is null then it was hit
+                // and since we have a last seen this is what is visible to the user so restart live data for it
+                const {bot, sub} = recentlySeen.get(lastSeenIdentifier) ?? {};
+                if(bot !== undefined && sub !== undefined) {
+                    console.debug(`Restarting live-data for ${lastSeenIdentifier} due to page being visible`);
+                    recentlySeen.delete(lastSeenIdentifier);
+                    onSubVisible(bot, sub);
+                }
+            }
+        }
+    });
 
     var searchParams = new URLSearchParams(window.location.search);
     const shownSub = searchParams.get('sub') || 'All'
@@ -1323,5 +1662,7 @@
         document.body.classList.remove('connected');
     });
 </script>
+<script src="https://cdnjs.cloudflare.com/ajax/libs/intro.js/6.0.0/intro.min.js" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
+<script src="/public/statusTour.js"></script>
 </body>
 </html>

src/Web/interfaces.ts

@@ -1,28 +0,0 @@
-import { URL } from "url";
-import {BotConnection} from "../Common/interfaces";
-import {Logger} from "winston";
-
-export interface BotInstance {
-    botName: string
-    botLink: string
-    error?: string
-    subreddits: string[]
-    nanny?: string
-    running: boolean
-    instance: CMInstanceInterface
-}
-
-export interface CMInstanceInterface extends BotConnection {
-    friendly?: string
-    operators: string[]
-    operatorDisplay: string
-    url: URL,
-    normalUrl: string,
-    lastCheck?: number
-    online: boolean
-    subreddits: string[]
-    bots: BotInstance[]
-    error?: string
-    ranMigrations: boolean
-    migrationBlocker?: string
-}