Rewrite docs to explain server-client architecture

2026-04-19 03:00:07 -04:00 · 2021-08-20 13:47:55 -04:00
parent 7b00e1c54b
commit 2fd1ffed19
7 changed files with 167 additions and 34 deletions
--- a/README.md
+++ b/README.md
@@ -15,7 +15,10 @@ An example of the above that Context Bot can do now:

 Some feature highlights:
 * Simple rule-action behavior can be combined to create any level of complexity in behavior
-* One instance can manage all moderated subreddits for the authenticated account
+* Server/client architecture
+  * Default/no configuration runs "All In One" behavior
+  * Additional configuration allows web interface to connect to multiple servers
+  * Each server instance can run multiple reddit accounts as bots
 * **Per-subreddit configuration** is handled by JSON stored in the subreddit wiki
 * Any text-based actions (comment, submission, message, usernotes, ban, etc...) can be configured via a wiki page or raw text in JSON and support [mustache](https://mustache.github.io) [templating](/docs/actionTemplating.md)
 * History-based rules support multiple "valid window" types -- [ISO 8601 Durations](https://en.wikipedia.org/wiki/ISO_8601#Durations), [Day.js Durations](https://day.js.org/docs/en/durations/creating), and submission/comment count limits.
@@ -27,7 +30,7 @@ Some feature highlights:
 * Support for [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes) as criteria or Actions (writing notes)
 * Docker container support
 * Event notification via Discord
-* **Web interface** for monitoring and administration
+* **Web interface** for monitoring, administration, and oauth bot authentication

 # Table of Contents

@@ -89,7 +92,6 @@ Context Bot's configuration can be written in JSON, [JSON5](https://json5.org/)

 ## Web UI and Screenshots

-
 ### Dashboard

 CM comes equipped with a dashboard designed for use by both moderators and bot operators.
--- a/docs/operatorConfiguration.md
+++ b/docs/operatorConfiguration.md
@@ -14,13 +14,13 @@ activities the Bot runs on.

 # Minimum Required Configuration

-| property       | Bot Authentication | API And Web        | API Only           | Web Only           |
-|:--------------:|:------------------:|:------------------:|:------------------:|:------------------:|
-| `clientId`     | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
-| `clientSecret` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
-| `redirectUri`  | :x:                | :heavy_check_mark: | :x:                | :heavy_check_mark: |
-| `refreshToken` | :x:                | :heavy_check_mark: | :heavy_check_mark: | :x:                |
-| `accessToken`  | :x:                | :heavy_check_mark: | :heavy_check_mark: | :x:                |
+| property       | Server And Web     | Server Only        | Web/Bot-Auth Only  |
+|:--------------:|:------------------:|:------------------:|:------------------:|
+| `clientId`     | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| `clientSecret` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| `redirectUri`  | :heavy_check_mark: | :x:                | :heavy_check_mark: |
+| `refreshToken` | :heavy_check_mark: | :heavy_check_mark: | :x:                |
+| `accessToken`  | :heavy_check_mark: | :heavy_check_mark: | :x:                |

 Refer to the **[Bot Authentication guide](/docs/botAuthentication.md)** to retrieve credentials.

@@ -46,6 +46,17 @@ noted with the same symbol as above. The value shown is the default.

 [**See the Operator Config Schema here**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)

+## Defining Multiple Bots or CM Instances
+
+One ContextMod instance can
+
+* Run multiple bots (multiple reddit accounts -- each as a bot)
+* Connect to many other, independent, ContextMod instances
+
+However, the default configuration (using **ENV/ARG**) assumes your intention is to run one bot (one reddit account) on one CM instance without these additional features. This is to make this mode of operation easier for users with this intention.
+
+To take advantage of this additional features you **must** use a **FILE** configuration. Learn about how this works and how to configure this scenario in the [Architecture Documentation.](/docs/serverClientArchitecture.md)
+
 ## CLI Usage

 Running CM from the command line is accomplished with the following command:
@@ -110,14 +121,18 @@ Below are examples of the minimum required config to run the application using a
 Using **FILE**
 <details>

-```json
+```json5
 {
-  "credentials": {
-    "clientId": "f4b4df1c7b2",
-    "clientSecret": "34v5q1c56ub",
-    "refreshToken": "34_f1w1v4",
-    "accessToken": "p75_1c467b2"
-  }
+  "bots": [
+    {
+      "credentials": {
+        "clientId": "f4b4df1c7b2",
+        "clientSecret": "34v5q1c56ub",
+        "refreshToken": "34_f1w1v4",
+        "accessToken": "p75_1c467b2"
+      }
+    }
+  ]
 }
 ```

@@ -155,10 +170,8 @@ An example of using multiple configuration levels together IE all are provided t

 ```json
 {
-  "credentials": {
-    "clientId": "f4b4df1c7b2",
-    "refreshToken": "34_f1w1v4",
-    "accessToken": "p75_1c467b2"
+  "logging": {
+    "level": "debug"
  }
 }
 ```
@@ -171,9 +184,10 @@ An example of using multiple configuration levels together IE all are provided t

 ```
 CLIENT_SECRET=34v5q1c56ub
+REFRESH_TOKEN=34_f1w1v4
+ACCESS_TOKEN=p75_1c467b2
 SUBREDDITS=sub1,sub2,sub3
 PORT=9008
-LOG_LEVEL=DEBUG
 ```

 </details>
@@ -183,7 +197,7 @@ LOG_LEVEL=DEBUG
 <details>

 ```
-node src/index.js run --subreddits=sub1
+node src/index.js run --subreddits=sub1 --clientId=34v5q1c56ub
 ```

 </details>
@@ -200,6 +214,52 @@ port: 9008
 log level: debug
 ```

+## Configuring Client for Many Instances
+
+See the [Architecture Docs](/docs/serverClientArchitecture.md) for more information.
+
+<details>
+
+```json5
+{
+  "bots": [
+    {
+      "credentials": {
+        "clientId": "f4b4df1c7b2",
+        "clientSecret": "34v5q1c56ub",
+        "refreshToken": "34_f1w1v4",
+        "accessToken": "p75_1c467b2"
+      }
+    }
+  ],
+  "web": {
+    "credentials": {
+      "clientId": "f4b4df1c7b2",
+      "clientSecret": "34v5q1c56ub",
+      "redirectUri": "http://localhost:8085/callback"
+    },
+    "clients": [
+      // server application running on this same CM instance
+      {
+        "host": "localhost:8095",
+        "secret": "localSecret"
+      },
+      // a server application running somewhere else
+      {
+        // api endpoint and port
+        "host": "mySecondContextMod.com:8095",
+        "secret": "anotherSecret"
+      }
+    ]
+  },
+  "api": {
+    "secret": "localSecret",
+  }
+}
+```
+
+</details>
+
 # Cache Configuration

 CM implements two caching backend **providers**. By default all providers use `memory`:
--- a/docs/serverClientArchitecture.md
+++ b/docs/serverClientArchitecture.md
@@ -0,0 +1,71 @@
+
+# Overview
+
+ContextMod's high-level functionality is separated into two **independently run** applications. 
+
+Each application consists of an [Express](https://expressjs.com/) web server that executes the core logic for that application and communicates via HTTP API calls:
+
+Applications:
+
+* **Server** -- Responsible for **running the bots** and providing an API to retrieve information on and interact with them EX start/stop bot, reload config, retrieve operational status, etc.
+* **Client** -- Responsible for serving the **web interface** and handling the bot oauth authentication flow between operators and moderators.
+
+Both applications operate independently and can be run individually. The determination for which is run is made by environmental variables, operator config, or cli arguments.
+
+# Authentication
+
+Communication between the applications is secured using [Json Web Tokens](https://github.com/mikenicholson/passport-jwt) signed/encoded by a **shared secret** (HMAC algorithm). The secret is defined in the operator configuration.
+
+# Configuration
+
+## Default Mode
+
+**ContextMod is designed to operate in a "monolith" mode by default.** 
+
+This is done by assuming that when configuration is provided by **environmental variables or CLI arguments** the user's intention is to run the client/server together with only one bot, as if ContextMod is a monolith application. When using these configuration types the same values are parsed to both the server/client to ensure interoperability/transparent usage for the operator. Some examples of this in the **operator configuration**:
+
+* The **shared secret** for both client/secret cannot be defined using env/cli -- at runtime a random string is generated that is set for the value `secret` on both the `api` and `web` properties.
+* The `bots` array cannot be defined using env/cli -- a single entry is generated by the configuration parser using the combined values provided from env/cli
+* The `PORT` env/cli argument only applies to the `client` wev server to guarantee the default port for the `server` web server is used (so the `client` can connect to `server`)
+
+**The end result of this default behavior is that an operator who does not care about running multiple CM instances does not need to know or understand anything about the client/server architecture.**
+
+## Server
+
+To run a ContextMod instance as **sever only (headless):**
+
+* Config file -- define top-level `"mode":"server"`
+* ENV -- `MODE=server`
+* CLI - `node src/index.js run server`
+
+The relevant sections of the **operator configuration** for the **Server** are:
+
+* [`operator.name`](https://json-schema.app/view/%23/%23%2Fproperties%2Foperator?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json) -- Define the reddit users who will be able to have full access to this server regardless of moderator status
+* `api`
+
+### [`api`](https://json-schema.app/view/%23/%23%2Fproperties%2Fapi?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
+
+* `port` - The port the Server will listen on for incoming api requests. Cannot be the same as the Client (when running on the same host)
+* `secret` - The **shared secret** that will be used to verify incoming api requests coming from an authenticated Client.
+* `friendly` - An optional string to identify this **Server** on the client. It is recommended to provide this otherwise it will default to `host:port`
+
+## Client
+
+To run a ContextMod instance as **client only:**
+
+* Config file -- define top-level `"mode":"client"`
+* ENV -- `MODE=client`
+* CLI - `node src/index.js run client`
+
+### [`web`](https://json-schema.app/view/%23/%23%2Fproperties%2Fweb?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
+
+In the **operator configuration** the top-level `web` property defines the configuration for the **Client** application.
+
+* `web.credentials` -- Defines the reddit oauth credentials used to authenticate users for the web interface
+  * Must contain a `redirectUri` property to work
+  * Credentials are parsed from ENV/CLI credentials when not specified (IE will be same as default bot)
+* `web.operators` -- Parsed from `operator.name` if not specified IE will use same users as defined for the bot operators
+* `port` -- the port the web interface will be served from, defaults to `8085`
+* `clients` -- An array of `BotConnection` objects that specify what **Server** instances the web interface should connect to. Each object should have:
+  * `host` -- The URL specifying where the server api is listening ie `localhost:8085`
+  * `secret` -- The **shared secret** used to sign api calls. **This should be the same as `api.secret` on the server being connected to.**
--- a/src/Common/interfaces.ts
+++ b/src/Common/interfaces.ts
@@ -1123,12 +1123,12 @@ export interface OperatorJsonConfig {
     * Mode to run ContextMod in
     *
     * * `all` (default) - Run the api and the web interface
-     * * `web` - Run web interface only
-     * * `api` - Run the api only
+     * * `client` - Run web interface only
+     * * `server` - Run the api/bots only
     *
     * @default "all"
     * */
-    mode?: 'bot' | 'web' | 'all',
+    mode?: 'server' | 'client' | 'all',
    /**
     * Settings related to the user(s) running this ContextMod instance and information on the bot
     * */
@@ -1314,7 +1314,7 @@ export interface BotInstanceConfig extends BotInstanceJsonConfig {
 }

 export interface OperatorConfig extends OperatorJsonConfig {
-    mode: 'all' | 'web' | 'bot',
+    mode: 'all' | 'server' | 'client',
    operator: {
        name: string[]
        display?: string,
--- a/src/ConfigBuilder.ts
+++ b/src/ConfigBuilder.ts
@@ -404,7 +404,7 @@ export const parseDefaultBotInstanceFromEnv = (): BotInstanceJsonConfig => {

 export const parseOpConfigFromEnv = (): OperatorJsonConfig => {
    const data = {
-        mode: process.env.MODE !== undefined ? process.env.MODE as ('all' | 'bot' | 'web') : undefined,
+        mode: process.env.MODE !== undefined ? process.env.MODE as ('all' | 'server' | 'client') : undefined,
            operator: {
            name: parseListFromEnv(process.env.OPERATOR),
            display: process.env.OPERATOR_DISPLAY
--- a/src/Schema/OperatorConfig.json
+++ b/src/Schema/OperatorConfig.json
@@ -546,11 +546,11 @@
        },
        "mode": {
            "default": "all",
-            "description": "Mode to run ContextMod in\n\n* `all` (default) - Run the api and the web interface\n* `web` - Run web interface only\n* `api` - Run the api only",
+            "description": "Mode to run ContextMod in\n\n* `all` (default) - Run the api and the web interface\n* `client` - Run web interface only\n* `server` - Run the api/bots only",
            "enum": [
                "all",
-                "bot",
-                "web"
+                "client",
+                "server"
            ],
            "type": "string"
        },
--- a/src/index.ts
+++ b/src/index.ts
@@ -54,7 +54,7 @@ const program = new Command();

        let runCommand = program
            .command('run')
-            .addArgument(new Argument('[interface]', 'Which interface to start the bot with').choices(['web', 'bot', 'all']).default(undefined, 'process.env.MODE || all'))
+            .addArgument(new Argument('[interface]', 'Which interface to start the bot with').choices(['client', 'server', 'all']).default(undefined, 'process.env.MODE || all'))
            .description('Monitor new activities from configured subreddits.')
            .allowUnknownOption();
        runCommand = addOptions(runCommand, getUniversalWebOptions());
@@ -64,10 +64,10 @@ const program = new Command();
                mode,
            } = config;
            try {
-                if(mode === 'all' || mode === 'web') {
+                if(mode === 'all' || mode === 'client') {
                    await clientServer(config);
                }
-                if(mode === 'all' || mode === 'bot') {
+                if(mode === 'all' || mode === 'server') {
                    await apiServer(config);
                }
            } catch (err) {