mirror of
https://github.com/FoxxMD/context-mod.git
synced 2026-05-11 03:00:42 -04:00
docs: Rewrite and improve operator-related documentation
* Break out documentation into more standalone docs and reorganize into an operator folder * Remove outdated information on adding bot * Add additional information on docker install * Make configuration more opinionated for "recommend" approach * Add docs on database and caching * Rewrite operator getting started guide to be more concise
This commit is contained in:
FoxxMD
19
README.md
19
README.md
@@ -7,7 +7,7 @@ alt="ContextMod logo" width="180" height="176">
|
||||
|
||||
It is designed to help fill in the gaps for [automoderator](https://www.reddit.com/wiki/automoderator/full-documentation) in regard to more complex behavior with a focus on **user-history based moderation.**
|
||||
|
||||
An example of the above that Context Bot can do now:
|
||||
An example of the above that Context Bot can do:
|
||||
|
||||
> * On a new submission, check if the user has also posted the same link in **N** number of other subreddits within a timeframe/# of posts
|
||||
> * On a new submission or comment, check if the user has had any activity (sub/comment) in **N** set of subreddits within a timeframe/# of posts
|
||||
@@ -23,13 +23,18 @@ Some feature highlights:
|
||||
* **Per-subreddit configuration** is handled by YAML (**like automoderator!**) or JSON stored in the subreddit wiki
|
||||
* Any text-based actions (comment, submission, message, usernotes, ban, etc...) can be configured via a wiki page or raw text and supports [mustache](https://mustache.github.io) [templating](/docs/actionTemplating.md)
|
||||
* History-based rules support multiple "valid window" types -- [ISO 8601 Durations](https://en.wikipedia.org/wiki/ISO_8601#Durations), [Day.js Durations](https://day.js.org/docs/en/durations/creating), and submission/comment count limits.
|
||||
* Support Activity skipping based on:
|
||||
* Author criteria (name, css flair/text, age, karma, moderator status, and [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes))
|
||||
* Activity state (removed, locked, distinguished, etc.)
|
||||
* Support Activity filtering based on:
|
||||
* Author criteria (name, css flair/text, age, karma, moderator status, [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes), and more!)
|
||||
* Activity state (removed, locked, distinguished, etc...)
|
||||
* Rules and Actions support named references (write once, reference anywhere)
|
||||
* Powerful state-machine-esque logic control (if, then, goto)
|
||||
* Delay/re-process activities using arbitrary rules
|
||||
* [**Image Comparisons**](/docs/imageComparison.md) via fingerprinting and/or pixel differences
|
||||
* [**Repost detection**](/docs/examples/repost) with support for external services (youtube, etc...)
|
||||
* Global/subreddit-level **API caching**
|
||||
* Database Persistence using SQLite, MySql, or Postgres
|
||||
* Audit trails for bot activity
|
||||
* Historical statistics
|
||||
* Support for [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes) as criteria or Actions (writing notes)
|
||||
* Docker container support
|
||||
* Event notification via Discord
|
||||
@@ -76,7 +81,7 @@ ___
|
||||
|
||||
This guide is for users who want to **run their own bot on a ContextMod instance.**
|
||||
|
||||
See the [Operator's Getting Started Guide](/docs/gettingStartedOperator.md)
|
||||
See the [Operator's Getting Started Guide](/docs/operator/gettingStarted.md)
|
||||
|
||||
#### Moderators
|
||||
|
||||
@@ -88,7 +93,7 @@ See the [Moderator's Getting Started Guide](/docs/gettingStartedMod.md)
|
||||
|
||||
Context Bot's configuration can be written in YAML (like automoderator) or [JSON5](https://json5.org/). Its schema conforms to [JSON Schema Draft 7](https://json-schema.org/). Additionally, many **operator** settings can be passed via command line or environmental variables.
|
||||
|
||||
* For **operators** (running the bot instance) see the [Operator Configuration](/docs/operatorConfiguration.md) guide
|
||||
* For **operators** (running the bot instance) see the [Operator Configuration](/docs/operator/configuration.md) guide
|
||||
* For **moderators** consult the [app schema and examples folder](/docs/#configuration-and-usage)
|
||||
|
||||
[**Check the full docs for in-depth explanations of all concepts and examples**](/docs)
|
||||
@@ -113,7 +118,7 @@ CM comes equipped with a dashboard designed for use by both moderators and bot o
|
||||
|
||||
### Bot Setup/Authentication
|
||||
|
||||
A bot oauth helper allows operators to define oauth credentials/permissions and then generate unique, one-time invite links that allow moderators to authenticate their own bots without operator assistance. [Learn more about using the oauth helper.](docs/botAuthentication.md#cm-oauth-helper-recommended)
|
||||
A bot oauth helper allows operators to define oauth credentials/permissions and then generate unique, one-time invite links that allow moderators to authenticate their own bots without operator assistance. [Learn more about using the oauth helper.](docs/operator/addingBot.md#cm-oauth-helper-recommended)
|
||||
|
||||
Operator view/invite link generation:
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
|
||||
Review **at least** the **How It Works** and **Concepts** below, then:
|
||||
|
||||
* For **Operators** (running a bot instance) refer to [**Operator Getting Started**](/docs/gettingStartedOperator.md) guide
|
||||
* For **Operators** (running a bot instance) refer to [**Operator Getting Started**](/docs/operator/gettingStarted.md) guide
|
||||
* For **Moderators** (configuring an existing bot for your subreddit) refer to the [**Moderator Getting Started**](/docs/gettingStartedMod.md) guide
|
||||
|
||||
## How It Works
|
||||
@@ -186,7 +186,9 @@ An **Action** is some action the bot can take against the checked Activity (comm
|
||||
* Comment (Reply to Comment/Submission)
|
||||
* Lock (Comment/Submission)
|
||||
* Report (Comment/Submission)
|
||||
* Contributor -- add or remove Author of Comment/Submission as approved contributor
|
||||
* [UserNote](/docs/examples/userNotes) (User, when /r/Toolbox is used)
|
||||
* Dispatch/Delay Activity processing
|
||||
|
||||
For detailed explanation and options of what individual Actions can do [see the links in the `actions` property in the schema.](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubmissionCheckJson?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json)
|
||||
|
||||
@@ -242,8 +244,8 @@ Some examples of using `authorIs` can be found in the [Author examples.](/docs/e
|
||||
|
||||
## Configuration And Usage
|
||||
|
||||
* For **Operator/Bot maintainers** see **[Operation Configuration](/docs/operatorConfiguration.md)**
|
||||
* [CLI Usage](docs/operatorConfiguration.md#cli-usage)
|
||||
* For **Operator/Bot maintainers** see **[Operation Configuration](/docs/operator/configuration.md)**
|
||||
* [CLI Usage](operator/configuration.md#cli-usage)
|
||||
* For **Moderators**
|
||||
* Refer to the [examples folder](/docs/examples) or the [subreddit-ready examples](/docs/examples/subredditReady)
|
||||
* as well as the [schema editor](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) which has
|
||||
|
||||
@@ -1,109 +0,0 @@
|
||||
**Note:** This is for **bot operators.** If you are a subreddit moderator check out the **[Getting Started Guide](/docs/gettingStartedMod.md)**
|
||||
|
||||
Before you can start using your bot on reddit there are a few steps you must take:
|
||||
|
||||
* Create your bot account IE the reddit account that will be the "bot"
|
||||
* Create a Reddit application
|
||||
* Authenticate your bot account with the application
|
||||
|
||||
At the end of this process you will have this info:
|
||||
|
||||
* clientId
|
||||
* clientSecret
|
||||
* refreshToken
|
||||
* accessToken
|
||||
* redirectUri
|
||||
|
||||
**Note:** If you already have this information you can skip this guide **but make sure your redirect uri is correct if you plan on using the web interface.**
|
||||
|
||||
# Table Of Contents
|
||||
|
||||
* [Creating an Application](#create-application)
|
||||
* [Authenticate Your Bot](#authenticate-your-bot-account)
|
||||
* [Using CM OAuth Helper](#cm-oauth-helper-recommended)
|
||||
* [Using Aardvark OAuth Helper](#aardvark-oauth-helper)
|
||||
* [Provide Credentials to CM](#provide-credentials-to-cm)
|
||||
|
||||
# Create Application
|
||||
|
||||
Visit [your reddit preferences](https://www.reddit.com/prefs/apps) and at the bottom of the page go through the **create an(other) app** process.
|
||||
* Give it a **name**
|
||||
* Choose **web app**
|
||||
* If you know what you will use for **redirect uri** go ahead and use it, otherwise use **http://localhost:8085/callback**
|
||||
|
||||
Click **create app**.
|
||||
|
||||
Then write down your **Client ID, Client Secret, and Redirect Uri** somewhere (or keep this webpage open)
|
||||
|
||||
# Authenticate Your Bot Account
|
||||
|
||||
There are **two ways** you can authenticate your bot account. It is recommended to use the CM oauth helper.
|
||||
|
||||
## CM OAuth Helper (Recommended)
|
||||
|
||||
This method will use CM's built in oauth flow. It is recommended because it will ensure your bot is authenticated with the correct oauth permissions.
|
||||
|
||||
### Start CM with Client ID/Secret and Operator
|
||||
|
||||
Start the application and provide these to your configuration:
|
||||
|
||||
* **Client ID**
|
||||
* **Client Secret**
|
||||
* **Redirect URI**
|
||||
* **Operator**
|
||||
|
||||
It is important you define **Operator** because the auth route is **protected.** You must login to the application in order to access the route.
|
||||
|
||||
Refer to the [operator config guide](/docs/operatorConfiguration.md) if you need help with this.
|
||||
|
||||
Examples:
|
||||
|
||||
* CLI - `node src/index.js --clientId=myId --clientSecret=mySecret --redirectUri="http://localhost:8085/callback" --operator=FoxxMD`
|
||||
* Docker - `docker run -e "CLIENT_ID=myId" -e "CLIENT_SECRET=mySecret" -e "OPERATOR=FoxxMD" -e "REDIRECT_URI=http://localhost:8085/callback" foxxmd/context-mod`
|
||||
|
||||
### Create An Auth Invite
|
||||
|
||||
Then open the CM web interface (default is [http://localhost:8085](http://localhost:8085)) and login.
|
||||
|
||||
After logging in you should be automatically redirected the auth page. If you are not then visit [http://localhost:8085/auth/helper](http://localhost:8085/auth/helper))
|
||||
|
||||
Follow the directions in the helper to create an **auth invite link.** Open this link and then follow the directions to authenticate your bot. At the end of the process you will receive an **Access Token** and **Refresh Token**
|
||||
|
||||
## Aardvark OAuth Helper
|
||||
|
||||
This method should only be used if you cannot use the [CM OAuth Helper method](#cm-oauth-helper-recommended) because you cannot access the CM web interface.
|
||||
|
||||
* Visit [https://not-an-aardvark.github.io/reddit-oauth-helper/](https://not-an-aardvark.github.io/reddit-oauth-helper/) and follow the instructions given.
|
||||
* **Note:** You will need to update your **redirect uri.**
|
||||
* Input your **Client ID** and **Client Secret** in the text boxes with those names.
|
||||
* Choose scopes. **It is very important you check everything on this list or CM may not work correctly**
|
||||
* edit
|
||||
* flair
|
||||
* history
|
||||
* identity
|
||||
* modcontributors
|
||||
* modflair
|
||||
* modposts
|
||||
* modself
|
||||
* mysubreddits
|
||||
* read
|
||||
* report
|
||||
* submit
|
||||
* wikiread
|
||||
* wikiedit (if you are using Toolbox User Notes)
|
||||
* Click **Generate tokens**, you will get a popup asking you to approve access (or login) -- **the account you approve access with is the account that Bot will control.**
|
||||
* After approving an **Access Token** and **Refresh Token** will be shown at the bottom of the page. Save these to use with CM.
|
||||
|
||||
# Provide Credentials to CM
|
||||
|
||||
At the end of the last step you chose you should now have this information saved somewhere:
|
||||
|
||||
* clientId
|
||||
* clientSecret
|
||||
* refreshToken
|
||||
* accessToken
|
||||
* redirectUri
|
||||
|
||||
This is all the information you need to run your bot with CM.
|
||||
|
||||
Using these credentials follow the [operator config guide](/docs/operatorConfiguration.md) to finish setting up your CM instance.
|
||||
@@ -16,7 +16,7 @@ Map port `1080:1080` -- acts as both the proxy port and the UI endpoint with the
|
||||
http(s)://localhost:1080/mockserver/dashboard
|
||||
```
|
||||
|
||||
In your [operator configuration](/docs/operatorConfiguration.md) define a proxy for snoowrap at the top-level:
|
||||
In your [operator configuration](/docs/operator/operatorConfiguration.md) define a proxy for snoowrap at the top-level:
|
||||
|
||||
```yaml
|
||||
snoowrap:
|
||||
|
||||
@@ -104,7 +104,7 @@ The **Goto** behavior is an **advanced** behavior that allows you to specify tha
|
||||
|
||||
**Goto should be use with care.** If you do not fully understand how this mechanism works you should avoid using it as **most** behaviors can be accomplished using the other behaviors.
|
||||
|
||||
As an additional protection **goto depth is limited to 1 by default** which means if a `goto` would be executed more than once during an Activity's lifecycle CM will automatically stop processing that Activity. The `maxGotoDepth` can be raised by the [**Bot Operator**](/docs/gettingStartedOperator.md) per subreddit.
|
||||
As an additional protection **goto depth is limited to 1 by default** which means if a `goto` would be executed more than once during an Activity's lifecycle CM will automatically stop processing that Activity. The `maxGotoDepth` can be raised by the [**Bot Operator**](/docs/operator/gettingStarted.md) per subreddit.
|
||||
|
||||
#### Goto Syntax
|
||||
|
||||
|
||||
@@ -111,7 +111,7 @@ criteria:
|
||||
* **external** -- get items from the Submission's link source that may be reposted (currently implemented for **Comment Checks** only)
|
||||
* When the Submission link is for...
|
||||
* **Youtube** -- get top comments on video by replies/like count
|
||||
* **NOTE:** An **API Key** for the [Youtube Data API](https://developers.google.com/youtube/v3) must be provided for this facet to work. This can be provided by the operator alongside [bot credentials](/docs/operatorConfiguration.md) or in the top-level `credentials` property for a [subreddit configuration.](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)
|
||||
* **NOTE:** An **API Key** for the [Youtube Data API](https://developers.google.com/youtube/v3) must be provided for this facet to work. This can be provided by the operator alongside [bot credentials](/docs/operator/configuration.md) or in the top-level `credentials` property for a [subreddit configuration.](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Freddit-context-bot%2Fmaster%2Fsrc%2FSchema%2FApp.json)
|
||||
|
||||
### Facet Modifiers
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
This getting started guide is for **reddit moderators** -- that is, someone who wants **an existing ContextMod bot to run on their subreddit.** If you are trying to run a ContextMod
|
||||
instance (software) please refer to the [operator getting started](/docs/gettingStartedOperator.md) guide.
|
||||
instance (software) please refer to the [operator getting started](/docs/operator/gettingStartedOperator.md) guide.
|
||||
|
||||
# Table of Contents
|
||||
|
||||
@@ -34,7 +34,7 @@ If the Operator has communicated that **you should add a bot they control as a m
|
||||
|
||||
* Shared api quota among other moderated subreddits (not great for high-volume subreddits)
|
||||
|
||||
___
|
||||
---
|
||||
|
||||
Ensure that you are in communication with the **operator** of this bot. The bot **will only accept a moderator invitation if your subreddit has been whitelisted by the operator.** This is an intentional barrier to ensure moderators and the operator are familiar with their respective needs and have some form of trust.
|
||||
|
||||
@@ -55,7 +55,7 @@ If the operator has communicated that **they want to use a bot you control** thi
|
||||
|
||||
* **Dedicated API quota**
|
||||
* This is basically a requirement if your subreddit has high-volume activity and you plan on running checks on comments
|
||||
* More security guarantees since you control the account
|
||||
* More security guarantees since you control the account
|
||||
* **Note:** authenticating an account does NOT give the operator access to view or change the email/password for the account
|
||||
* Established history in your subreddit
|
||||
|
||||
@@ -63,7 +63,7 @@ If the operator has communicated that **they want to use a bot you control** thi
|
||||
|
||||
* You must have access to the credentials for the reddit account (bot)
|
||||
|
||||
___
|
||||
---
|
||||
|
||||
The **operator** will send you an **invite link** that you will use to authenticate your bot with the operator's application. Example link: `https://operatorsUrl.com/auth/invite?invite=4kf9n3o03ncd4nd`
|
||||
|
||||
@@ -117,11 +117,13 @@ After you have found a configuration to use as a starting point:
|
||||
CM comes equipped with a [configuration explorer](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FApp.json) to help you see all available options, with descriptions and examples, that can be used in your configuration.
|
||||
|
||||
To create or edit a configuration you should use **CM's buit-in editor** which features:
|
||||
|
||||
* syntax validation and formatting
|
||||
* full configuration validation with error highlighting, hints, and fixes
|
||||
* hover over properties to see documentation and examples
|
||||
|
||||
To use the editor either:
|
||||
|
||||
* [use your subreddit's built-in editor](#using-the-built-in-editor)
|
||||
* or use the public editor at https://cm.foxxmd.dev/config
|
||||
|
||||
@@ -148,7 +150,7 @@ In the web interface each subreddit's tab has access to the built-in editor. Use
|
||||
|
||||
</details>
|
||||
|
||||
___
|
||||
---
|
||||
|
||||
The bot automatically checks for new configurations on your wiki page every 5 minutes. If your operator has the web interface accessible you may login there and force the config to update on your subreddit.
|
||||
|
||||
|
||||
@@ -1,90 +0,0 @@
|
||||
This getting started guide is for **Operators** -- that is, someone who wants to run the actual software for a ContentMod bot. If you are a **Moderator** check out the [moderator getting started](/docs/gettingStartedMod.md) guide instead.
|
||||
|
||||
# Table of Contents
|
||||
|
||||
* [Installation](#installation)
|
||||
* [Docker](#docker-recommended)
|
||||
* [Locally](#locally)
|
||||
* [Heroku](#heroku-quick-deployhttpsherokucomabout)
|
||||
* [Bot Authentication](#bot-authentication)
|
||||
* [Instance Configuration](#instance-configuration)
|
||||
* [Run Your Bot and Start Moderating](#run-your-bot-and-start-moderating)
|
||||
|
||||
# Installation
|
||||
|
||||
In order to run a ContextMod instance you must first you must install it somewhere.
|
||||
|
||||
ContextMod can be run on almost any operating system but it is recommended to use Docker due to ease of deployment.
|
||||
|
||||
## Docker (Recommended)
|
||||
|
||||
PROTIP: Using a container management tool like [Portainer.io CE](https://www.portainer.io/products/community-edition) will help with setup/configuration tremendously.
|
||||
|
||||
### [Dockerhub](https://hub.docker.com/r/foxxmd/context-mod)
|
||||
|
||||
An example of starting the container using the [minimum configuration](/docs/operatorConfiguration.md#minimum-config) with a [configuration file](/docs/operatorConfiguration.md#defining-configuration-via-file):
|
||||
|
||||
* Bind the folder where your config, logs, and database are located on your host machine into the container `-v /host/path/folder:/config`
|
||||
* Expose the web interface using the container port `8085`
|
||||
* Specify the user or user:group to run the container with -- this is important to keep your host files usable
|
||||
|
||||
```
|
||||
docker run -d -e "OPERATOR_CONFIG=/config/myConfig.yaml" -v /host/path/folder:/config -p 8085:8085 --user=myHostUsername foxxmd/context-mod
|
||||
```
|
||||
|
||||
### Locally
|
||||
|
||||
Requirements:
|
||||
|
||||
* Typescript >=4.3.5
|
||||
* Node >=15
|
||||
|
||||
Clone this repository somewhere and then install from the working directory
|
||||
|
||||
```bash
|
||||
git clone https://github.com/FoxxMD/context-mod.git .
|
||||
cd context-mod
|
||||
npm install
|
||||
tsc -p .
|
||||
```
|
||||
|
||||
An example of running CM using the [minimum configuration](/docs/operatorConfiguration.md#minimum-config) with a [configuration file](/docs/operatorConfiguration.md#defining-configuration-via-file):
|
||||
|
||||
```bash
|
||||
node src/index.js run
|
||||
```
|
||||
|
||||
### [Heroku Quick Deploy](https://heroku.com/about)
|
||||
[](https://dashboard.heroku.com/new?template=https://github.com/FoxxMD/context-mod)
|
||||
|
||||
This template provides a **web** and **worker** dyno for heroku.
|
||||
|
||||
* **Web** -- Will run the bot **and** the web interface for ContextMod.
|
||||
* **Worker** -- Will run **just** the bot.
|
||||
|
||||
Be aware that Heroku's [free dyno plan](https://devcenter.heroku.com/articles/free-dyno-hours#dyno-sleeping) enacts some limits:
|
||||
|
||||
* A **Web** dyno will go to sleep (pause) after 30 minutes without web activity -- so your bot will ALSO go to sleep at this time
|
||||
* The **Worker** dyno **will not** go to sleep but you will NOT be able to access the web interface. You can, however, still see how Cm is running by reading the logs for the dyno.
|
||||
|
||||
If you want to use a free dyno it is recommended you perform first-time setup (bot authentication and configuration, testing, etc...) with the **Web** dyno, then SWITCH to a **Worker** dyno so it can run 24/7.
|
||||
|
||||
# Bot Authentication
|
||||
|
||||
Next you need to create a bot and authenticate it with Reddit. Follow the [bot authentication guide](/docs/botAuthentication.md) to complete this step.
|
||||
|
||||
# Instance Configuration
|
||||
|
||||
Finally, you must provide the credentials you received from the **Bot Authentication** step to the ContextMod instance you installed earlier. Refer to the [Operator Configuration](/docs/operatorConfiguration.md) guide to learn how this can be done as there are multiple approaches depending on how you installed the software.
|
||||
|
||||
Additionally, at this step you can also tweak many more settings and behavior concerning how your CM bot will operate.
|
||||
|
||||
# Run Your Bot and Start Moderating
|
||||
|
||||
Congratulations! You should now have a fully authenticated bot running on ContextMod software.
|
||||
|
||||
In order for your Bot to operate on reddit though it **must be a moderator in the subreddit you want it to run in.** This may be your own subreddit or someone else's.
|
||||
|
||||
**Note: ContextMod does not currently handle moderation invites automatically** and may never have this functionality. Due to the fact that many of its behaviors are api-heavy and that subreddits can control their own configuration the api and resource (cpu/memory) usage of a ContextMod instance can be highly variable. It therefore does not make sense to allow any/all subreddits to automatically have access to an instance through automatically accepting moderator invites. So...if you are planning to run a ContextMod instance for subreddits other than those you moderate you should establish solid trust with moderators of that subreddit as well as a solid line of communication in order to ensure their configurations can be tailored to best fit their needs and your resources.
|
||||
|
||||
Once you have logged in as your bot and manually accepted the moderator invite you will need to restart your ContextMod instance in order for these changes to take effect.
|
||||
78
docs/operator/README.md
Normal file
78
docs/operator/README.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# Operator Guide
|
||||
|
||||
An **Operator** is the user **running the ContextMod software.**
|
||||
|
||||
They are responsible for configuring the software at a high-level and managing associated infrastructure such as:
|
||||
|
||||
* Creating cache/database servers and configuring their connections in CM
|
||||
* Provisioning the [Reddit Clients](/botAutHere) needed to run bots and the CM UI
|
||||
* Providing [global-level configuration](/operatorConfiguration.md) that affects general bot/subreddit behavior
|
||||
* Onboarding new bots/subreddits
|
||||
|
||||
# Table of Contents
|
||||
|
||||
* [Overview](#overview)
|
||||
* [Client-Server Architecture](/docs/serverClientArchitecture.md)
|
||||
* [Getting Started](#getting-started)
|
||||
* [Installation](#installation)
|
||||
* [Provisioning a Reddit Client](#provisioning-a-reddit-client)
|
||||
* [Configuration](/docs/operator/configuration.md)
|
||||
* [Adding A Bot](/docs/operator/addingBot.md)
|
||||
|
||||
# Overview
|
||||
|
||||
CM is composed of two applications that operate indepedently but are packaged together such that they act as one piece of software:
|
||||
|
||||
* **Server** -- Responsible for **running the bot(s)** and providing an API to retrieve information on and interact with them EX start/stop bot, reload config, retrieve operational status, etc.
|
||||
* **Client** -- Responsible for serving the **web interface** and handling the bot oauth authentication flow between operators and subreddits/bots.
|
||||
|
||||
Both applications authenticate, and are primarily operated, by using [Reddit's API through OAuth.](https://github.com/reddit-archive/reddit/wiki/OAuth2) The **Client** uses OAuth to verify the identity of moderators logging into the web interface. The **Server** uses oauth tokens to interact with Reddit's API and operate all the configured bots.
|
||||
|
||||
In its default mode of operation CM takes care of all the interaction between **Server** and **Client** for you so that you can effectively treat it as a monolithic application. Learn more about CM's architecture and other operation modes in the [Server-Client Architecture documentation.](/docs/serverClientArchitecture.md)
|
||||
|
||||
# [Getting Started](/doc1s/operator/gettingStarted.md)
|
||||
|
||||
The [Getting Started](/doc1s/operator/gettingStarted.md) guide serves as a straight-forward "how-to" for standing up a CM server from scratch with minimal explanation.
|
||||
|
||||
# [Installation](/docs/operator/installation.md)
|
||||
|
||||
CM has many installation options:
|
||||
|
||||
* Locally, from source, as a typescript project
|
||||
* Built/pulled from a Docker image hosted on Dockerhub
|
||||
* Deployed to Heroku with a Quick Deploy template (experimental)
|
||||
|
||||
Refer to the [Installation](/docs/oeprator/installation) docs for more information.
|
||||
|
||||
# Provisioning A Reddit Client
|
||||
|
||||
As mentioning in the [Overview](#overview), CM operates primarily using Reddit's API through OAuth. You must create a [Reddit Client](https://github.com/reddit-archive/reddit/wiki/OAuth2#getting-started) in order to interact with the API.
|
||||
|
||||
## Create Application
|
||||
|
||||
Visit [your reddit preferences](https://www.reddit.com/prefs/apps) and at the bottom of the page go through the **create an(other) app** process.
|
||||
|
||||
* Give it a **name**
|
||||
* Choose **web app**
|
||||
* If you know what you will use for **redirect uri** go ahead and use it, otherwise use `http://localhost:8085/callback`
|
||||
|
||||
Click **create app**.
|
||||
|
||||
Then write down your **Client ID, Client Secret, and Redirect Uri** somewhere
|
||||
|
||||
# [Configuration](/docs/operator/configuration.md)
|
||||
|
||||
The [Configuration](/docs/operator/configuration.md) documentation covers:
|
||||
|
||||
* How CM's configuration can be defined
|
||||
* How to create and define location for a config file
|
||||
* Running CM from the command line
|
||||
* Documentation for configuration on Bots, the web client, API, and more...
|
||||
|
||||
# [Adding A Bot](/docs/operator/addingBot.md)
|
||||
|
||||
The [Adding A Bot](/docs/operator/addingBot.md) documentation covers:
|
||||
|
||||
* What is a Bot?
|
||||
* What is needed to add a Bot to CM?
|
||||
* Different approaches to authenticating and adding a Bot to CM
|
||||
86
docs/operator/addingBot.md
Normal file
86
docs/operator/addingBot.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Table of Contents
|
||||
|
||||
* [What is a Bot?](#what-is-a-bot)
|
||||
* [Prequisites](#prerequisties)
|
||||
* [Adding a Bot to CM](#adding-a-bot-tocm)
|
||||
* [Using CM OAuth Helper (Recommended)](#cm-oauth-helper-recommended)
|
||||
* [Using Aardvark OAuth Helper](#aarvark-oauth-helper)
|
||||
|
||||
# What is a Bot?
|
||||
|
||||
A **reddit bot** is composed of two components:
|
||||
|
||||
* A normal **reddit account** like `u/MyRedditAccount`
|
||||
* Software that performs actions **on behalf of that reddit account** using Reddit's API
|
||||
|
||||
There is nothing special about the account! What's special is how its used -- through the API *with bot software* like ContextMod.
|
||||
|
||||
# Prerequistes
|
||||
|
||||
These things need to be done before a Bot can be added to CM:
|
||||
|
||||
* [Provisioned a Reddit Client](/docs/operator/README.md#provising-a-reddit-client)
|
||||
* You or the person who controls the Bot account must have account credentials (username/password). Logging in to reddit is part of the setup process.
|
||||
* If the bot does not exist **create a reddit account for it.**
|
||||
* If the bot does exist make sure you are in communication with the owner of the account.
|
||||
|
||||
# Adding A Bot to CM
|
||||
|
||||
## CM OAuth Helper (Recommended)
|
||||
|
||||
This method will use CM's built in oauth flow. It is recommended because:
|
||||
|
||||
* It's easy!
|
||||
* Will ensure your bot is authenticated with the correct oauth permissions
|
||||
|
||||
### Start CM with the Minimum Configuration (Initial Setup)
|
||||
|
||||
If this is your **first time adding a bot** you must make sure you have
|
||||
|
||||
* done the [prerequisties](#prerequisites)
|
||||
* created a [minimum operator configuration](/docs/operator/configuration.md#minimum-config)
|
||||
* that specifies the client id/secret from provisioning your reddit client
|
||||
* specified **Operator Name** in the configuration
|
||||
|
||||
It is important you define **Operator Name** because the auth route is **protected.** You must login to CM's web interface in order to access the route.
|
||||
|
||||
### Create A Bot Invite
|
||||
|
||||
Open the CM web interface (default is [http://localhost:8085](http://localhost:8085)) and login with the reddit account specified in **Operator Name.**
|
||||
|
||||
If this is your first time setting up a bot you should be automatically redirected to the auth page. Otherwise, visit [http://localhost:8085/auth/helper](http://localhost:8085/auth/helper)
|
||||
|
||||
Follow the directions in the helper to create a **Bot Invite Link.**
|
||||
|
||||
### Onboard the Bot
|
||||
|
||||
Visit the **Bot Invite Link** while **logged in to reddit as the bot account** to begin the onboarding process. Refer to the [Onboarding Your Bot]() subreddit documentation for more information on this process.
|
||||
|
||||
At the end of the onboarding process the bot should be automatically added to your operator configuration. If there is an issue with automatically adding it then the oauth credentials will be displayed at the end of onboarding and can be [manually added to the configuration.](/docs/operator/configuration.md#manually-adding-a-bot)
|
||||
|
||||
## Aardvark OAuth Helper
|
||||
|
||||
This method should only be used if you cannot use the [CM OAuth Helper method.](#cm-oauth-helper-recommended)
|
||||
|
||||
* Visit [https://not-an-aardvark.github.io/reddit-oauth-helper/](https://not-an-aardvark.github.io/reddit-oauth-helper/) and follow the instructions given.
|
||||
* **Note:** You will need to update the **redirect uri** you set when [provisioning your reddit client.](/docs/operator/README.md#provisioning-a-reddit-client)
|
||||
* Input your **Client ID** and **Client Secret** in the text boxes with those names.
|
||||
* Choose scopes. **It is very important you check everything on this list or CM may not work correctly**
|
||||
* edit
|
||||
* flair
|
||||
* history
|
||||
* identity
|
||||
* modcontributors
|
||||
* modflair
|
||||
* modposts
|
||||
* modself
|
||||
* modnote
|
||||
* mysubreddits
|
||||
* read
|
||||
* report
|
||||
* submit
|
||||
* wikiread
|
||||
* wikiedit (if you are using Toolbox User Notes)
|
||||
* Click **Generate tokens**, you will get a popup asking you to approve access (or login) -- **the account you approve access with is the account that Bot will control.**
|
||||
* After approving an **Access Token** and **Refresh Token** will be shown at the bottom of the page. Use these to [manually add a bot to your operator configuration.](/docs/operator/configuration.md#manually-adding-a-bot)
|
||||
* After adding the bot you will need to restart CM.
|
||||
161
docs/operator/caching.md
Normal file
161
docs/operator/caching.md
Normal file
@@ -0,0 +1,161 @@
|
||||
# Table of Contents
|
||||
|
||||
* [Overview](#overview)
|
||||
* [What Is Cache?](#what-is-cache)
|
||||
* [How CM Uses Caching](#how-cm-uses-caching)
|
||||
* [Reddit API Calls](#reddit-api-calls)
|
||||
* [Rules and Filters](#rules-and-filters)
|
||||
* [Configuration](#configuration)
|
||||
* [Cache Provider](#cache-provider)
|
||||
|
||||
# Overview
|
||||
|
||||
**Caching** is a major factor in CM's performance and optimization of Reddit API usage. Leveraging caching effectively in your operator configuration and in individual subreddit configurations can make or break your CM instance.
|
||||
|
||||
### What Is Cache?
|
||||
|
||||
A **Cache** is a storage medium with high **write** and **read** speed that is generally used to store **temporary, but frequently accessed data.**
|
||||
|
||||
## How CM Uses Caching
|
||||
|
||||
CM primarily **caches** two types of data:
|
||||
|
||||
### Reddit API Calls
|
||||
|
||||
#### How Reddit's API Works
|
||||
|
||||
In order to communicate with Reddit to retrieve posts, comments, user information, etc... CM uses API calls. Each API call is composed of a
|
||||
|
||||
* **request** -- CM asks Reddit for certain information
|
||||
* **response** -- Reddit responds with the request information
|
||||
|
||||
[Reddit imposes an **api quota**](https://github.com/reddit-archive/reddit/wiki/API#rules) on every **individual account** using the API through an application. This quota is **600 requests per 10 minutes.** At the end of the 10 minutes period the quota is reset.
|
||||
|
||||
Additionally, some API calls have limits on how much data they can return. The most relevant of these is **user history can only be returned 100 activities (submission/comments) per API call**. EX if you want to get **500** activities from a user's history you will need to make **5** api calls.
|
||||
|
||||
#### Caching API Responses
|
||||
|
||||
In order to most effectively use the limited quota of API calls CM will **automatically cache API responses based on the "fingerprint" of the request sent.**
|
||||
|
||||
On an individual "item" basis that means these resources are always cached:
|
||||
|
||||
* General user information (name, karma, age, profile description, etc..)
|
||||
* General subreddit information (name, nsfw, quarantined, etc...)
|
||||
* Individually processed activities (comment body, is comment author op, submission title, reports, link, etc...)
|
||||
|
||||
Additionally (and most importantly), responses for **user history** are cached **based on what was requested**. Example "fingerprint":
|
||||
|
||||
* username
|
||||
* type of activities to retrieve (overview, only submissions, only comments)
|
||||
* range of activities to retrieve (last 100, last 6 months, etc...)
|
||||
|
||||
If the above "fingerprint" is used in three different Rules then
|
||||
|
||||
* First fingerprint appearance -> CM make API call and caches response
|
||||
* Second fingerprint appearance -> CM uses cached response
|
||||
* Third fingerprint appearance -> CM uses cached response
|
||||
|
||||
So only **one** API call is made even though the history is used three times.
|
||||
|
||||
It is therefore **important to re-use window criteria** wherever possible to take advantage of this caching.
|
||||
|
||||
### Rules and Filters
|
||||
|
||||
Once CM has processed a Rule or Filter (`itemIs` or `authorIs`) the results of that component is stored in cache. For Rules the result is stored for the lifecycle of the Activity being processed and then discarded. For Filters the result is stored for a short time in cache and can be re-used by other Activities.
|
||||
|
||||
Re-using Rules and Filters by either using the exact same configuration or by using **names** provides:
|
||||
|
||||
* A major performance benefit since these do not need to be re-calculated
|
||||
* A low-to-medium optimization on API caching, depending on what criteria are being tested.
|
||||
|
||||
In general re-use should always be a goal.
|
||||
|
||||
# Configuration
|
||||
|
||||
## Cache Provider
|
||||
|
||||
CM supports two cache **providers**. By default all providers use `memory`:
|
||||
|
||||
* `memory` -- in-memory (non-persistent) backend
|
||||
* Cache will be lost when CM is restarted/exits
|
||||
* `redis` -- [Redis](https://redis.io/) backend
|
||||
|
||||
Each `provider` object in configuration can be specified as:
|
||||
|
||||
* one of the above **strings** to use the **defaults settings** or
|
||||
* an **object** with keys to override default settings
|
||||
|
||||
[Refer to full documentation on cache providers in the schema](https://json-schema.app/view/%23/%23%2Fdefinitions%2FOperatorCacheConfig/%23%2Fdefinitions%2FCacheOptions?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
|
||||
Some examples:
|
||||
|
||||
```json5
|
||||
{
|
||||
"provider": {
|
||||
"store": "memory", // one of "memory" or "redis"
|
||||
"ttl": 60, // the default max age of a key in seconds
|
||||
"max": 500, // the maximum number of keys in the cache (for "memory" only)
|
||||
|
||||
// the below properties only apply to 'redis' provider
|
||||
"host": 'localhost',
|
||||
"port": 6379,
|
||||
"auth_pass": null,
|
||||
"db": 0,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
YAML
|
||||
|
||||
```yaml
|
||||
provider:
|
||||
store: redis
|
||||
ttl: 60
|
||||
max: 500
|
||||
host: localhost
|
||||
port: 6379
|
||||
auth_pass: null
|
||||
db: 0
|
||||
```
|
||||
|
||||
Providers can be specified in multiple locations, with each more specific location overriding the parent-level config:
|
||||
|
||||
* top-level config
|
||||
* in individual bot configurations
|
||||
* in the web config
|
||||
|
||||
```yaml
|
||||
operator:
|
||||
name: example
|
||||
# top level
|
||||
caching:
|
||||
provider:
|
||||
...
|
||||
bots:
|
||||
- name: u/MyBot
|
||||
# overrides top level
|
||||
caching:
|
||||
provider:
|
||||
...
|
||||
web:
|
||||
# overrides top level
|
||||
caching:
|
||||
provider:
|
||||
...
|
||||
```
|
||||
|
||||
## Cache TTL
|
||||
|
||||
The **Time To Live (TTL)** -- how long data may live in the cache before "expiring" -- can be controlled indepedently for different data types. Sane defaults are provided for all types but tweaking these can improve API caching optimization depending on the subreddit's configuration (use case).
|
||||
|
||||
Each of these can be specified in the `caching` property. TTL is measured in seconds.
|
||||
|
||||
* `authorTTL` (default 60) -- user activity (overview, submissions, comments)
|
||||
* `commentTTL` (default 60) -- individually fetched comments
|
||||
* `submissionTTL` (default 60) -- individually fetched submissions
|
||||
* `filterCriteriaTTL` (default 60) -- filter results (`itemIs` and `authorIs`)
|
||||
* `selfTTL` (default 60) -- actions performed by the bot (creating comment reply, report). If action is in cache the bot ignores it if found during polling.
|
||||
* This helps prevent bot from reacting to things it did itself IE you don't want it to remove a comment because it reported the comment itself
|
||||
* `subredditTTL` (default 60) -- general information on fetched subreddit
|
||||
* `userNotesTTL` (default 300) -- Amount of time [Toolbox User Notes](https://www.reddit.com/r/toolbox/wiki/docs/usernotes) are cached
|
||||
* `wikiTTL` (default 300) -- Wiki pages used for content (in messages, reports, bans, etc...) are cached for this amount of time
|
||||
@@ -2,32 +2,23 @@ The **Operator** configuration refers to configuration used configure to the act
|
||||
from the **Subreddit** configuration that is defined in each Subreddit's wiki and determines the rules/actions for
|
||||
activities the Bot runs on.
|
||||
|
||||
**The full documentation** for all options in the operator configuration can be found [**here in the operator schema.**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
|
||||
# Table of Contents
|
||||
|
||||
* [Minimum Required Configuration](#minimum-required-configuration)
|
||||
* [Defining Configuration](#defining-configuration)
|
||||
* [CLI Usage](#cli-usage)
|
||||
* [Minimum Configuration](#minimum-configuration)
|
||||
* [Bots](#bots)
|
||||
* [Examples](#example-configurations)
|
||||
* [Minimum Config](#minimum-config)
|
||||
* [Using Config Overrides](#using-config-overrides)
|
||||
* [Cache Configuration](#cache-configuration)
|
||||
|
||||
# Minimum Required Configuration
|
||||
|
||||
| property | Server And Web | Server Only | Web/Bot-Auth Only |
|
||||
|:--------------:|:------------------:|:------------------:|:------------------:|
|
||||
| `clientId` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
|
||||
| `clientSecret` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
|
||||
| `redirectUri` | :heavy_check_mark: | :x: | :heavy_check_mark: |
|
||||
| `refreshToken` | :heavy_check_mark: | :heavy_check_mark: | :x: |
|
||||
| `accessToken` | :heavy_check_mark: | :heavy_check_mark: | :x: |
|
||||
|
||||
Refer to the **[Bot Authentication guide](/docs/botAuthentication.md)** to retrieve credentials.
|
||||
* [Database Configuration](#database-configuration)
|
||||
|
||||
# Defining Configuration
|
||||
|
||||
CM can be configured using **any or all** of the approaches below. Note that **at each level ALL configuration values are
|
||||
optional** but the "required configuration" mentioned above must be available when all levels are combined.
|
||||
CM can be configured using **any or all** of the approaches below. **It is recommended to use FILE ([File Configuration](#file-configuration-recommended))**
|
||||
|
||||
Any values defined at a **lower-listed** level of configuration will override any values from a higher-listed
|
||||
configuration.
|
||||
@@ -35,20 +26,37 @@ configuration.
|
||||
* **ENV** -- Environment variables loaded from an [`.env`](https://github.com/toddbluhm/env-cmd) file (path may be
|
||||
specified with `--file` cli argument)
|
||||
* **ENV** -- Any already existing environment variables (exported on command line/terminal profile/etc.)
|
||||
* **FILE** -- Values specified in a JSON configuration file using the structure [in the schema](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
* **FILE** -- Values specified in a YAML/JSON configuration file using the structure [in the schema](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
* When reading the **schema** if the variable is available at a level of configuration other than **FILE** it will be
|
||||
noted with the same symbol as above. The value shown is the default.
|
||||
* **ARG** -- Values specified as CLI arguments to the program (see [ClI Usage](#cli-usage) below)
|
||||
|
||||
**Note:** When reading the **schema** if the variable is available at a level of configuration other than **FILE** it will be
|
||||
noted with the same symbol as above. The value shown is the default.
|
||||
## File Configuration (Recommended)
|
||||
|
||||
## Defining Configuration Via File
|
||||
Using a file has many benefits over using ARG or ENV:
|
||||
|
||||
* **from the command line** use the `-c` cli argument EX: `node src/index.js -c /path/to/JSON/config.json`
|
||||
* **using an environmental variable** use `OPERATOR_CONFIG` EX: `OPERATOR_CONFIG=/path/to/JSON/config.json`
|
||||
* CM can automatically update your configuration
|
||||
* CM can automatically add bots via the [CM OAuth Helper](/docs/operator/addingBot.md#using-cm-oauth-helper-recommended)
|
||||
* CM has a built-in configuration editor that can help you build and validate your configuration file
|
||||
* File config is **required** if adding multiple bots to CM
|
||||
|
||||
[**See the Operator Config Schema here**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
### Specify File Location
|
||||
|
||||
## Defining Multiple Bots or CM Instances
|
||||
By default CM will look for `config.yaml` or `config.json` in the `DATA_DIR` directory:
|
||||
|
||||
* [Local installation](/docs/operator/installation.md#locally) -- `DATA_DIR` is the root of your installation directory (same folder as `package.json`)
|
||||
* [Docker](/docs/operator/installation.md#docker-recommended) -- `DATA_DIR` is at `/config` in the container
|
||||
|
||||
The `DATA_DIR` directory can be changed by passing `DATA_DIR` as an environmental variable EX `DATA_DIR=/path/to/directory`
|
||||
|
||||
The name of the config file can be changed by passing `OPERATOR_CONFIG` as an environmental variable:
|
||||
|
||||
* As filename -- `OPERATOR_CONFIG=myConfig.yaml` -> CM looks for `/path/to/directory/myConfig.yaml`
|
||||
* As absolute path -- `OPERATOR_CONFIG=/a/path/myConfig.yaml` -> CM looks for `/a/path/myConfig.yaml`
|
||||
|
||||
[**Refer to the Operator Config File Schema for full documentation**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
|
||||
### Defining Multiple Bots or CM Instances
|
||||
|
||||
One ContextMod instance can
|
||||
|
||||
@@ -114,6 +122,85 @@ Options:
|
||||
|
||||
</details>
|
||||
|
||||
# Minimum Configuration
|
||||
|
||||
The minimum configuration required to run CM assumes you have no bots and want to use CM to [add your first bot.](/docs/operator/addingBot.md#cm-oauth-helper-recommended)
|
||||
|
||||
You will need have this information available:
|
||||
|
||||
* From [provision a reddit client](/docs/operator/README.md#provisioning-a-reddit-client)
|
||||
* Client ID
|
||||
* Client Secret
|
||||
* Redirect URI (if different from default `http://localhost:8085/callback`)
|
||||
* Operator Name -- username of the reddit account you want to use to administer CM with
|
||||
|
||||
See the [**example minimum configuration** below.](#minimum-config)
|
||||
|
||||
# Bots
|
||||
|
||||
Configured using the `bots` top-level property. Bot configuration can override and specify many more options than are available at the operator-level. Many of these can also set the defaults for each subreddit the bot runs:
|
||||
|
||||
* Of the subreddits this bot moderates, specify a subset of subreddits to run or exclude from running
|
||||
* default caching behavior
|
||||
* control the soft/hard api usage limits
|
||||
* Flow Control defaults
|
||||
* Filter Criteria defaults
|
||||
* default Polling behavior
|
||||
|
||||
[Full documentation for all bot instance options can be found in the schema.](https://json-schema.app/view/%23/%23%2Fdefinitions%2FBotInstanceJsonConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
|
||||
## Adding A Bot
|
||||
|
||||
If you use the [CM OAuth Helper](/docs/operator/addingBot.md#cm-oauth-helper-recommended) and it works successfully then the configuration for the Bot will be automatically added.
|
||||
|
||||
### Manually Adding a Bot
|
||||
|
||||
Add a new *object* to the `bots` property at the top-level of your configuration. If `bots` does not exist create it now.
|
||||
|
||||
Minimum information required for a valid bot:
|
||||
|
||||
* Client Id
|
||||
* Client Secret
|
||||
* Refresh Token
|
||||
* Access Token
|
||||
|
||||
<details>
|
||||
<summary>Example</summary>
|
||||
|
||||
```yaml
|
||||
operator:
|
||||
name: YourRedditUsername
|
||||
|
||||
bots:
|
||||
- name: u/MyRedditBot # name is optional but highly recommend for readability in both config and web interface
|
||||
credentials:
|
||||
reddit:
|
||||
clientId: f4b4df1c7b2
|
||||
clientSecret: 34v5q1c56ub
|
||||
accessToken: 34_f1w1v4
|
||||
refreshToken: p75_1c467b2
|
||||
|
||||
web:
|
||||
credentials:
|
||||
clientId: f4b4df1c7b2
|
||||
clientSecret: 34v5q1c56ub
|
||||
redirectUri: 'http://localhost:8085/callback'
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
# Web Client
|
||||
|
||||
Configured using the `web` top-level property. Allows specifying settings related to:
|
||||
|
||||
* UI port
|
||||
* Database and caching connection, if different from global settings
|
||||
* Session max age and secret
|
||||
* Invite max age
|
||||
* Connections to CM API instances (if using multiple)
|
||||
|
||||
[Full documentation for all web settings can be found in the schema.](https://json-schema.app/view/%23/%23%2Fproperties%2Fweb?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
|
||||
# Example Configurations
|
||||
|
||||
## Minimum Config
|
||||
@@ -121,41 +208,35 @@ Options:
|
||||
Below are examples of the minimum required config to run the application using all three config approaches independently.
|
||||
|
||||
Using **FILE**
|
||||
|
||||
<details>
|
||||
|
||||
CM will look for a file configuration at `PROJECT_DIR/config.yaml` by default [or you can specify your own location.](#defining-configuration-via-file)
|
||||
See [Specify File Location](#specify-file-location) for where this file would be located.
|
||||
|
||||
YAML (`config.yaml`)
|
||||
|
||||
YAML
|
||||
```yaml
|
||||
operator:
|
||||
name: YourRedditUsername
|
||||
bots:
|
||||
- credentials:
|
||||
clientId: f4b4df1c7b2
|
||||
clientSecret: 34v5q1c56ub
|
||||
web:
|
||||
credentials:
|
||||
clientId: f4b4df1c7b2
|
||||
clientSecret: 34v5q1c56ub
|
||||
redirectUri: 'http://localhost:8085/callback'
|
||||
```
|
||||
JSON
|
||||
|
||||
JSON (`config.json5`)
|
||||
|
||||
```json5
|
||||
{
|
||||
"operator": {
|
||||
"name": "YourRedditUsername"
|
||||
},
|
||||
"bots": [
|
||||
{
|
||||
"credentials": {
|
||||
"clientId": "f4b4df1c7b2",
|
||||
"clientSecret": "34v5q1c56ub"
|
||||
}
|
||||
}
|
||||
],
|
||||
"web": {
|
||||
"credentials": {
|
||||
"clientId": "f4b4df1c7b2",
|
||||
"clientSecret": "34v5q1c56ub"
|
||||
"clientSecret": "34v5q1c56ub",
|
||||
"redirectUri": "http://localhost:8085/callback"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -171,6 +252,7 @@ Using **ENV** (`.env`)
|
||||
OPERATOR=YourRedditUsername
|
||||
CLIENT_ID=f4b4df1c7b2
|
||||
CLIENT_SECRET=34v5q1c56ub
|
||||
REDIRECT_URI=http://localhost:8085/callback
|
||||
```
|
||||
|
||||
</details>
|
||||
@@ -180,7 +262,7 @@ Using **ARG**
|
||||
<details>
|
||||
|
||||
```
|
||||
node src/index.js run --clientId=f4b4df1c7b2 --clientSecret=34v5q1c56ub --refreshToken=34_f1w1v4 --accessToken=p75_1c467b2
|
||||
node src/index.js run --clientId=f4b4df1c7b2 --clientSecret=34v5q1c56ub --redirectUri=http://localhost:8085/callback
|
||||
```
|
||||
|
||||
</details>
|
||||
@@ -190,6 +272,7 @@ node src/index.js run --clientId=f4b4df1c7b2 --clientSecret=34v5q1c56ub --refres
|
||||
An example of using multiple configuration levels together IE all are provided to the application:
|
||||
|
||||
**FILE**
|
||||
|
||||
<details>
|
||||
|
||||
```json
|
||||
@@ -199,7 +282,9 @@ An example of using multiple configuration levels together IE all are provided t
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
YAML
|
||||
|
||||
```yaml
|
||||
logging:
|
||||
level: debug
|
||||
@@ -213,8 +298,6 @@ logging:
|
||||
|
||||
```
|
||||
CLIENT_SECRET=34v5q1c56ub
|
||||
REFRESH_TOKEN=34_f1w1v4
|
||||
ACCESS_TOKEN=p75_1c467b2
|
||||
SUBREDDITS=sub1,sub2,sub3
|
||||
PORT=9008
|
||||
```
|
||||
@@ -236,8 +319,6 @@ When all three are used together they produce these variables at runtime for the
|
||||
```
|
||||
clientId: f4b4df1c7b2
|
||||
clientSecret: 34v5q1c56ub
|
||||
refreshToken: 34_f1w1v4
|
||||
accessToken: accessToken
|
||||
subreddits: sub1
|
||||
port: 9008
|
||||
log level: debug
|
||||
@@ -250,6 +331,7 @@ See the [Architecture Docs](/docs/serverClientArchitecture.md) for more informat
|
||||
<details>
|
||||
|
||||
YAML
|
||||
|
||||
```yaml
|
||||
bots:
|
||||
- credentials:
|
||||
@@ -272,7 +354,9 @@ web:
|
||||
api:
|
||||
secret: localSecret
|
||||
```
|
||||
|
||||
JSON
|
||||
|
||||
```json5
|
||||
{
|
||||
"bots": [
|
||||
@@ -315,41 +399,8 @@ JSON
|
||||
|
||||
# Cache Configuration
|
||||
|
||||
CM implements two caching backend **providers**. By default all providers use `memory`:
|
||||
See the [Cache Configuration](/docs/operator/caching.md) documentation.
|
||||
|
||||
* `memory` -- in-memory (non-persistent) backend
|
||||
* `redis` -- [Redis](https://redis.io/) backend
|
||||
# Database Configuration
|
||||
|
||||
Each `provider` object in configuration can be specified as:
|
||||
|
||||
* one of the above **strings** to use the **defaults settings** or
|
||||
* an **object** with keys to override default settings
|
||||
|
||||
A caching object in the json configuration:
|
||||
|
||||
```json5
|
||||
{
|
||||
"provider": {
|
||||
"store": "memory", // one of "memory" or "redis"
|
||||
"ttl": 60, // the default max age of a key in seconds
|
||||
"max": 500, // the maximum number of keys in the cache (for "memory" only)
|
||||
|
||||
// the below properties only apply to 'redis' provider
|
||||
"host": 'localhost',
|
||||
"port": 6379,
|
||||
"auth_pass": null,
|
||||
"db": 0,
|
||||
}
|
||||
}
|
||||
```
|
||||
YAML
|
||||
```yaml
|
||||
provider:
|
||||
store: memory
|
||||
ttl: 60
|
||||
max: 500
|
||||
host: localhost
|
||||
port: 6379
|
||||
auth_pass: null
|
||||
db: 0
|
||||
```
|
||||
See the [Database Configuration](/docs/operator/database.md) documentation.
|
||||
131
docs/operator/database.md
Normal file
131
docs/operator/database.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Overview
|
||||
|
||||
CM uses a database to store three types of data:
|
||||
|
||||
* **Recorded Events** -- an "audit" of how CM processed an Activity (Comment/Submission) and what actions it took based on the result of processing it (comment, report, remove, etc.)
|
||||
* Persistent Settings/Data
|
||||
* Settings like the last known state of a Subreddit's bot before the application exited
|
||||
* Web Client sessions and invites -- stuff that should survive a restart
|
||||
* Statistics
|
||||
* All-time and time-series high-level statistics like # of events, # of checks run, etc...
|
||||
|
||||
CM does NOT store subreddit configurations or any runtime alterations of these configurations. This is to keep configurations **portable** -- on principle, if you (a moderator) choose to use a different CM instance to run your subreddit's bot then it should not function differently.
|
||||
|
||||
# Providers
|
||||
|
||||
CM uses [TypeORM](https://typeorm.io/) as the database access layer and specifically supports three database types:
|
||||
|
||||
* SQLite -- using either [SQL.js](https://sql.js.org) or native SQLite through [better-sqlite3](https://github.com/JoshuaWise/better-sqlite3)
|
||||
* MySQL/MariaDB
|
||||
* Postgres
|
||||
|
||||
The database configuration is specified in the top-level `databaseConfig.connection` property in the operator configuration. EX:
|
||||
|
||||
```yaml
|
||||
operator:
|
||||
name: u/MyRedditAccount
|
||||
databaseConfig:
|
||||
connection:
|
||||
...
|
||||
```
|
||||
|
||||
## SQLite
|
||||
|
||||
When using a [local installation](/docs/installation.md#locally) the default database is `sqljs`, which requires no binary dependencies. When using [docker](/docs/operator/installation.md#docker-recommended) the default is `better-sqlite3`.
|
||||
|
||||
**NOTE:** It is **NOT RECOMMENDED** to use `sqljs` in a production environment for performance reasons. You should at least switch to `better-sqlite3` or preferably MySql/Postgres.
|
||||
|
||||
* [`sqljs` connection options](https://typeorm.io/data-source-options#sqljs-data-source-options)
|
||||
* [`better-sqlite3` connection options](https://typeorm.io/data-source-options#better-sqlite3-data-source-options)
|
||||
|
||||
For both sqlite types, if no database/location is specified, it will be created in the [`DATA_DIR` directory.](/docs/operator/configuration.md#specify-file-location)
|
||||
|
||||
If CM detects it cannot **read and write** to the database files, or directory if no files exist, it will fallback to using an in-memory database that will be lost when CM restarts. If you have trouble with r/w permissions and are using docker make sure [file permissions are correct for your mounted volume.](/docs/operator/installation.md#linux-host)
|
||||
|
||||
## MySQL/MariaDB
|
||||
|
||||
[MySQL/MariaDB connection options](https://typeorm.io/data-source-options#mysql--mariadb-data-source-options)
|
||||
|
||||
The `database` you specify should exist before using CM.
|
||||
|
||||
## Postgres
|
||||
|
||||
[Postgres connection options](https://typeorm.io/data-source-options#postgres--cockroachdb-data-source-options)
|
||||
|
||||
The `database` and `schema` you specify should exist before using CM.
|
||||
|
||||
# Migrations
|
||||
|
||||
CM implements database migrations. On startup it will check for any pending migrations. If the database doesn't exist (sqlite) or is empty or no tables conflict it will automatically execute migrations.
|
||||
|
||||
If there is any kind of conflict it will **pause startup** and display a prompt in the user interface to confirm migration execution. **You should always backup your database before running migrations.**
|
||||
|
||||
To force CM to always run migrations without confirmation set `force` to `true` in the `migrations` property within `databaseConfig`:
|
||||
|
||||
```yaml
|
||||
databaseConfig:
|
||||
migrations:
|
||||
force: true # always run migrations
|
||||
```
|
||||
|
||||
### SQLite
|
||||
|
||||
When using a SQLite driver CM can create automatic backups for you. Another prompt will be displayed on the migrations page in the web interface to make a copy of your database. You can make CM automatically backup and continue with migrations like so:
|
||||
|
||||
```yaml
|
||||
databaseConfig:
|
||||
migrations:
|
||||
continueOnAutomatedBackup: true # try to backup sqlite files automatically and continue with migrations if successful
|
||||
```
|
||||
|
||||
# Recorded Event Retention
|
||||
|
||||
The **Recorded Events** CM stores in the database can be controlled per subreddit. By default events will be stored indefinitely.
|
||||
|
||||
A **Retention Policy** is a metric to determine what "range" of Recorded Events CM should keep in the database. It can be either:
|
||||
|
||||
* A **number** -- The last X number of Recorded Events will be kept
|
||||
* EX `1000` -> Keep the last 1000 events and discard any others.
|
||||
* A **duration** -- A time period, starting from now until X `duration` in the past, for which events will be kept
|
||||
* EX `3 months` -> Keep all events created between now and 3 months ago. Anything older than 3 months ago will be discarded.
|
||||
|
||||
The Retention Policy can be specified at operator level, bot, subreddit *override*, and subreddit configuration level:
|
||||
|
||||
```yaml
|
||||
operator: name: u/MyRedditAccount
|
||||
databaseConfig:
|
||||
retention: '3 months' # each subreddit will retain 3 more of recorded events
|
||||
bots:
|
||||
# all subreddits this bot moderates will have 3 month retention
|
||||
- name: u/OneBotAccount
|
||||
credentials:
|
||||
...
|
||||
subreddits:
|
||||
overrides:
|
||||
- name: aSpecialSubredit
|
||||
databaseConfig:
|
||||
# overrides retention for THIS SUBBREDIT ONLY, will retain last 1000 events
|
||||
# -- also overrides any retention set in the subreddit's actual configuration
|
||||
retention: 1000
|
||||
|
||||
- name: u/TwoBotAccount
|
||||
credentials:
|
||||
...
|
||||
databaseConfig:
|
||||
retention: '1 month' # overrides top-level rentention for all subeddits this bot moderates
|
||||
```
|
||||
|
||||
In a subreddit's config:
|
||||
|
||||
```yaml
|
||||
polling:
|
||||
- unmoderated
|
||||
|
||||
# will retain last 2000 events
|
||||
# -- will override top-level operator retention or bot-specific retention
|
||||
# -- will NOT override a subreddit override specified in bot coniguration
|
||||
retention: 2000
|
||||
|
||||
runs:
|
||||
...
|
||||
```
|
||||
60
docs/operator/gettingStarted.md
Normal file
60
docs/operator/gettingStarted.md
Normal file
@@ -0,0 +1,60 @@
|
||||
This getting started guide is for **Operators** -- that is, someone who wants to run the actual software for a ContentMod bot. If you are a **Moderator** check out the [moderator getting started](/docs/gettingStartedMod.md) guide instead.
|
||||
|
||||
# Table of Contents
|
||||
|
||||
* [Installation](#installation)
|
||||
* [Create a Reddit Client](#create-a-reddit-client)
|
||||
* [Create a Minimum Configuration](#create-a-minimum-configuration)
|
||||
* [Local Installation](#local-installation)
|
||||
* [Docker Installation](#docker-installation)
|
||||
* [Add a Bot to CM](#add-a-bot-to-cm)
|
||||
* [Access The Dashboard](#access-the-dashboard)
|
||||
* [What's Next?](#whats-next)
|
||||
|
||||
# Installation
|
||||
|
||||
Follow the [installation](/docs/operator/installation) documentation. It is recommended to use **Docker** since it is self-contained.
|
||||
|
||||
# Create a Reddit Client
|
||||
|
||||
[Create a reddit client](/docs/operator/README.md#provisioning-a-reddit-client)
|
||||
|
||||
# Create a Minimum Configuration
|
||||
|
||||
Using the information you received in the previous step [create a minimum file configuration](/docs/operator/configuration.md#minimum-configuration) save it as `config.yaml` somewhere.
|
||||
|
||||
# Start ContextMod With Configuration
|
||||
|
||||
## Local Installation
|
||||
|
||||
If you [installed CM locally](/docs/installation#locally) move your configuration file `config.yaml` to the root of the project directory (where `package.json`) is located.
|
||||
|
||||
From the root directory run this command to start CM
|
||||
|
||||
```
|
||||
node src/index.js run
|
||||
```
|
||||
|
||||
## Docker Installation
|
||||
|
||||
If you [installed CM using Docker](/docs/installation#docker-recommended) make note of the directory you saved your minimum configuration to and substitute its full path for `host/path/folder` in the docker command show in the [docker install directions](/docs/operator/installation.md#docker-recommended)
|
||||
|
||||
# Add A Bot to CM
|
||||
|
||||
Once CM is up and running use the [CM OAuth Helper](/docs/operator/addingBot.md#cm-oauth-helper-recommended) to add authorize and add a Bot to your CM instance.
|
||||
|
||||
# Access The Dashboard
|
||||
|
||||
Congratulations! You should now have a fully authenticated bot running on a ContextMod instance.
|
||||
|
||||
In order for your Bot to operate in a subreddit it **must be a moderator in that subreddit.** This may be your own subreddit or someone else's.
|
||||
|
||||
To monitor the behavior of bots running on your instance visit http://localhost:8085.
|
||||
|
||||
# What's Next?
|
||||
|
||||
As an operator you should familiarize yourself with how the [operator configuration](/docs/operator/configuration) you made works. This will help you understand how to get the most of your CM instance by leveraging the [Cache](/docs/oeprator/caching.md) and [Database](/docs/operator/database.md) effectively as well as provide you will all possible options for configuring CM using the [schema.](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fcontext-mod%2Fmaster%2Fsrc%2FSchema%2FOperatorConfig.json)
|
||||
|
||||
If you are also the moderator of the subreddit the bot will be running you should check out the [moderator getting started guide.](/docs/gettingStartedMod.md#setup-wiki-page)
|
||||
|
||||
You might also be interested in these [quick tips for using the web interface](/docs/webInterface.md)
|
||||
78
docs/operator/installation.md
Normal file
78
docs/operator/installation.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# Installation
|
||||
|
||||
In order to run a ContextMod instance you must first you must install it somewhere.
|
||||
|
||||
ContextMod can be run on almost any operating system but it is recommended to use Docker due to ease of deployment.
|
||||
|
||||
## Docker (Recommended)
|
||||
|
||||
PROTIP: Using a container management tool like [Portainer.io CE](https://www.portainer.io/products/community-edition) will help with setup/configuration tremendously.
|
||||
|
||||
### [Dockerhub](https://hub.docker.com/r/foxxmd/context-mod)
|
||||
|
||||
An example of starting the container using the [minimum configuration](/docs/operator/operatorConfiguration.md#minimum-config) with a [configuration file](/docs/operator/operatorConfiguration.md#defining-configuration-via-file):
|
||||
|
||||
* Bind the directory where your config file, logs, and database are located on your host machine into the container's default `DATA_DIR` by using `-v /host/path/folder:/config`
|
||||
* Expose the web interface using the container port `8085`
|
||||
|
||||
```
|
||||
docker run -d -v /host/path/folder:/config -p 8085:8085 foxxmd/context-mod
|
||||
```
|
||||
|
||||
The location of `DATA_DIR` in the container can be changed by passing it as an environmental variable EX `-e "DATA_DIR=/home/abc/config`
|
||||
|
||||
### Linux Host
|
||||
|
||||
**NOTE:** If you are using [rootless containers with Podman](https://developers.redhat.com/blog/2020/09/25/rootless-containers-with-podman-the-basics#why_podman_) this DOES NOT apply to you.
|
||||
|
||||
If you are running Docker on a **Linux Host** you must specify `user:group` permissions of the user who owns the **configuration directory** on the host to avoid [docker file permission problems.](https://ikriv.com/blog/?p=4698) These can be specified using the [environmental variables **PUID** and **PGID**.](https://docs.linuxserver.io/general/understanding-puid-and-pgid)
|
||||
|
||||
To get the UID and GID for the current user run these commands from a terminal:
|
||||
|
||||
* `id -u` -- prints UID
|
||||
* `id -g` -- prints GID
|
||||
|
||||
```
|
||||
docker run -d -v /host/path/folder:/config -p 8085:8085 -e PUID=1000 -e PGID=1000 foxxmd/context-mod
|
||||
```
|
||||
|
||||
## Locally
|
||||
|
||||
Requirements:
|
||||
|
||||
* [Typescript](https://www.typescriptlang.org/) >=4.3.5
|
||||
* [Node](https://nodejs.org) >=16
|
||||
* [NPM](https://www.npmjs.com/) >=8 (usually bundled with Node)
|
||||
|
||||
Clone this repository somewhere and then install from the working directory
|
||||
|
||||
```bash
|
||||
git clone https://github.com/FoxxMD/context-mod.git .
|
||||
cd context-mod
|
||||
npm install
|
||||
tsc -p .
|
||||
```
|
||||
|
||||
An example of running CM using the [minimum configuration](/docs/operator/operatorConfiguration.md#minimum-config) with a [configuration file](/docs/operator/operatorConfiguration.md#defining-configuration-via-file):
|
||||
|
||||
```bash
|
||||
node src/index.js run
|
||||
```
|
||||
|
||||
## [Heroku Quick Deploy](https://heroku.com/about)
|
||||
|
||||
**NOTE:** This is still experimental and requires more testing.
|
||||
|
||||
[](https://dashboard.heroku.com/new?template=https://github.com/FoxxMD/context-mod)
|
||||
|
||||
This template provides a **web** and **worker** dyno for heroku.
|
||||
|
||||
* **Web** -- Will run the bot **and** the web interface for ContextMod.
|
||||
* **Worker** -- Will run **just** the bot.
|
||||
|
||||
Be aware that Heroku's [free dyno plan](https://devcenter.heroku.com/articles/free-dyno-hours#dyno-sleeping) enacts some limits:
|
||||
|
||||
* A **Web** dyno will go to sleep (pause) after 30 minutes without web activity -- so your bot will ALSO go to sleep at this time
|
||||
* The **Worker** dyno **will not** go to sleep but you will NOT be able to access the web interface. You can, however, still see how Cm is running by reading the logs for the dyno.
|
||||
|
||||
If you want to use a free dyno it is recommended you perform first-time setup (bot authentication and configuration, testing, etc...) with the **Web** dyno, then SWITCH to a **Worker** dyno so it can run 24/7.
|
||||
@@ -1,7 +1,6 @@
|
||||
|
||||
# Overview
|
||||
|
||||
ContextMod's high-level functionality is separated into two **independently run** applications.
|
||||
ContextMod's high-level functionality is separated into two **independently run** applications.
|
||||
|
||||
Each application consists of an [Express](https://expressjs.com/) web server that executes the core logic for that application and communicates via HTTP API calls:
|
||||
|
||||
@@ -20,7 +19,7 @@ Communication between the applications is secured using [Json Web Tokens](https:
|
||||
|
||||
## Default Mode
|
||||
|
||||
**ContextMod is designed to operate in a "monolith" mode by default.**
|
||||
**ContextMod is designed to operate in a "monolith" mode by default.**
|
||||
|
||||
This is done by assuming that when configuration is provided by **environmental variables or CLI arguments** the user's intention is to run the client/server together with only one bot, as if ContextMod is a monolith application. When using these configuration types the same values are parsed to both the server/client to ensure interoperability/transparent usage for the operator. Some examples of this in the **operator configuration**:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user