mirror of
https://github.com/FoxxMD/context-mod.git
synced 2026-01-14 16:08:02 -05:00
Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
ea875e392f | ||
|
|
b89d98e42b |
12
Dockerfile
12
Dockerfile
@@ -81,8 +81,9 @@ RUN apk add --no-cache \
|
||||
#
|
||||
|
||||
# vips required to run sharp library for image comparison
|
||||
# opencv required for other image processing
|
||||
RUN echo "http://dl-4.alpinelinux.org/alpine/v3.14/community" >> /etc/apk/repositories \
|
||||
&& apk --no-cache add vips
|
||||
&& apk --no-cache add vips opencv opencv-dev
|
||||
|
||||
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
|
||||
|
||||
@@ -115,6 +116,15 @@ RUN npm install --production \
|
||||
&& rm -rf node_modules/ts-node \
|
||||
&& rm -rf node_modules/typescript
|
||||
|
||||
# build bindings for opencv
|
||||
RUN apk add --no-cache --virtual .build-deps \
|
||||
make \
|
||||
g++ \
|
||||
gcc \
|
||||
libgcc \
|
||||
&& npm run cv-install-docker-prebuild \
|
||||
&& apk del .build-deps
|
||||
|
||||
ENV NPM_CONFIG_LOGLEVEL debug
|
||||
|
||||
# can set database to use more performant better-sqlite3 since we control everything
|
||||
|
||||
@@ -60,6 +60,65 @@ An example of running CM using the [minimum configuration](/docs/operator/config
|
||||
node src/index.js run
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
Note: All below dependencies are automatically included in the [Docker](#docker-recommended) image.
|
||||
|
||||
#### Sharp
|
||||
|
||||
For basic [Image Comparisons](/docs/imageComparison.md) and image data CM uses [sharp](https://sharp.pixelplumbing.com/) which depends on [libvips](https://www.libvips.org/)
|
||||
|
||||
Binaries for Sharp and libvips ship with the `sharp` npm package for all major operating systems and should require no additional steps to use -- installing CM with the above script should be sufficient.
|
||||
|
||||
See more about Sharp dependencies in the [image comparison prerequisites.](/docs/imageComparison.md#prerequisites)
|
||||
|
||||
|
||||
#### OpenCV
|
||||
|
||||
For advanced image comparison CM uses [OpenCV.](https://opencv.org/) OpenCV is an **optional** dependency that is only utilized if CM is configured to run these advanced image operations so if you are NOT doing any image-related operations you can safely ignore this section/dependency.
|
||||
|
||||
**NOTE:** Depending on the image being compared (resolution) and operations being performed this can be a **CPU heavy resource.** TODO: Add rules that are cpu heavy...
|
||||
|
||||
##### Installation
|
||||
|
||||
Installation is not an automatic process. The below instructions are a summary of "easy" paths for installation but are not exhaustive. DO reference the detailed instructions (including additional details for windows installs) at [opencv4nodejs How to Install](https://github.com/UrielCh/opencv4nodejs#how-to-install).
|
||||
|
||||
###### Build From Source
|
||||
|
||||
This may take **some time** since openCV will be built from scratch.
|
||||
|
||||
On windows you must first install build tools: `npm install --global windows-build-tools`
|
||||
|
||||
Otherwise, run one of the following commands from the CM project directory:
|
||||
|
||||
* For CUDA (Nvidia GPU acceleration): `npm run cv-autoinstall-cuda`
|
||||
* Normal: `npm run cv-autoinstall`
|
||||
|
||||
###### Build from Prebuilt
|
||||
|
||||
In this use-case you already have openCV built OR are using a distro prebuilt package. This method is much faster than building from source as only bindings need to be built.
|
||||
|
||||
[More information on prebuild installation](https://github.com/UrielCh/opencv4nodejs#installing-opencv-manually)
|
||||
|
||||
Prerequisites:
|
||||
|
||||
* Windows `choco install OpenCV -y -version 4.1.0`
|
||||
* MacOS `brew install opencv@4; brew link --force opencv@4`
|
||||
* Linux -- varies, check your package manager for `opencv` and `opencv-dev`
|
||||
|
||||
A script for building on **Ubuntu** is already included in CM:
|
||||
|
||||
* `sudo apt install opencv-dev`
|
||||
* `npm run cv-install-ubuntu-prebuild`
|
||||
|
||||
Otherwise, you will need to modify `scripts` in CM's `package.json`, use the script `cv-install-ubuntu-prebuild` as an example. Your command must include:
|
||||
|
||||
* `--incDir [path/to/opencv/dev-files]` (on linux, usually `/usr/include/opencv4/`)
|
||||
* `--libDir [path/to/opencv/shared-files]` (on linux usually `/lib/x86_64-linux-gnu/` or `/usr/lib/`)
|
||||
* `--binDir=[path/to/openv/binaries]` (on linux usually `/usr/bin/`)
|
||||
|
||||
After you have modified/added a script for your operating system run it with `npm run yourScriptName`
|
||||
|
||||
## [Heroku Quick Deploy](https://heroku.com/about)
|
||||
|
||||
**NOTE:** This is still experimental and requires more testing.
|
||||
|
||||
@@ -1,32 +1,12 @@
|
||||
Actions that can submit text (Report, Comment, UserNote, Message, Ban, Submission) will have their `content` values run through a [Mustache Template](https://mustache.github.io/). This means you can insert data generated by Rules into your text before the Action is performed.
|
||||
Actions that can submit text (Report, Comment, UserNote) will have their `content` values run through a [Mustache Template](https://mustache.github.io/). This means you can insert data generated by Rules into your text before the Action is performed.
|
||||
|
||||
See here for a [cheatsheet](https://gist.github.com/FoxxMD/d365707cf99fdb526a504b8b833a5b78) and [here](https://www.tsmean.com/articles/mustache/the-ultimate-mustache-tutorial/) for a more thorough tutorial.
|
||||
|
||||
# Template Data
|
||||
|
||||
Some data can always be accessed at the top-level. Example
|
||||
|
||||
```
|
||||
This action was run from {{manager}} in Check {{check}}.
|
||||
|
||||
The bot intro post is {{botLink}}
|
||||
|
||||
Message the moderators of this subreddit using this [compose link]({{modmailLink}})
|
||||
```
|
||||
|
||||
|
||||
|
||||
| Name | Description | Example |
|
||||
|---------------|---------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `manager` | The name of the subreddit the bot is running in | mealtimevideos |
|
||||
| `check` | The name of the Check that was triggered | myCheck |
|
||||
| `botLink` | A link to the bot introduction | https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot |
|
||||
| `modmailLink` | A link that opens reddit's DM compose with the subject line as the Activity being processed | https://www.reddit.com/message/compose?to=/r/mealtimevideos&message=https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot |
|
||||
|
||||
|
||||
## Activity Data
|
||||
|
||||
**Activity data can be accessed using the `item` variable.** Example
|
||||
Activity data can be accessed using the `item` variable. Example
|
||||
|
||||
```
|
||||
This activity is a {{item.kind}} with {{item.votes}} votes, created {{item.age}} ago.
|
||||
@@ -39,17 +19,14 @@ Produces:
|
||||
|
||||
All Actions with `content` have access to this data:
|
||||
|
||||
| Name | Description | Example |
|
||||
|--------------|-----------------------------------------------------------------------------------------------------|----------------------------------------------------------|
|
||||
| `kind` | The Activity type (submission or comment) | submission |
|
||||
| `author` | Name of the Author of the Activity being processed | FoxxMD |
|
||||
| `permalink` | URL to the Activity | https://reddit.com/r/mySuibreddit/comments/ab23f/my_post |
|
||||
| `votes` | Number of upvotes | 69 |
|
||||
| `age` | The age of the Activity in a [human friendly format](https://day.js.org/docs/en/durations/humanize) | 5 minutes |
|
||||
| `subreddit` | The name of the subreddit the Activity is from | mealtimevideos |
|
||||
| `id` | The `Reddit Thing` ID for the Activity | t3_0tin1 |
|
||||
| `title` | As comments => the body of the comment. As Submission => title | Test post please ignore |
|
||||
| `shortTitle` | The same as `title` but truncated to 15 characters | test post pleas... |
|
||||
| Name | Description | Example |
|
||||
|-------------|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
|
||||
| `kind` | The Activity type (submission or comment) | submission |
|
||||
| `author` | Name of the Author of the Activity being processed | FoxxMD |
|
||||
| `permalink` | URL to the Activity | https://reddit.com/r/mySuibreddit/comments/ab23f/my_post |
|
||||
| `votes` | Number of upvotes | 69 |
|
||||
| `age` | The age of the Activity in a [human friendly format](https://day.js.org/docs/en/durations/humanize) | 5 minutes |
|
||||
| `botLink` | A URL to CM's introduction thread | https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot |
|
||||
|
||||
### Submissions
|
||||
|
||||
@@ -112,39 +89,7 @@ Produces
|
||||
|
||||
> Submission was repeated 7 times
|
||||
|
||||
## Action Data
|
||||
|
||||
### Summary
|
||||
|
||||
A summary of what actions have already been run **when the template is rendered** is available using the `actionSummary` variable. It is therefore important that the Action you want to produce the summary is run **after** any other Actions you want to get a summary for.
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
A summary of actions processed for this activity, so far:
|
||||
|
||||
{{actionSummary}}
|
||||
```
|
||||
|
||||
Would produce:
|
||||
> A summary of actions processed for this activity, so far:
|
||||
>
|
||||
> * approve - ✘ - Item is already approved??
|
||||
> * lock - ✓
|
||||
> * modnote - ✓ - (SOLID_CONTRIBUTOR) User is good
|
||||
|
||||
### Individual
|
||||
|
||||
Individual **Actions** can be accessed using the name of the action, **lower-cased, with all spaces/dashes/underscores.** Example:
|
||||
|
||||
```
|
||||
User was banned for {{actions.exampleban.duration}} for {{actions.exampleban.reason}}
|
||||
```
|
||||
Produces
|
||||
|
||||
> User was banned for 4 days for toxic behavior
|
||||
|
||||
# Quick Templating Tutorial
|
||||
#### Quick Templating Tutorial
|
||||
|
||||
As a quick example for how you will most likely be using templating -- wrapping a variable in curly brackets, `{{variable}}`, will cause the variable value to be rendered instead of the brackets:
|
||||
|
||||
|
||||
@@ -692,16 +692,15 @@ Some other things to note:
|
||||
* If the `to` property is not specified then the message is sent to the Author of the Activity being processed
|
||||
* `to` may be a **User** (u/aUser) or a **Subreddit** (r/aSubreddit)
|
||||
* `to` **cannot** be a Subreddit when `asSubreddit: true` -- IE cannot send subreddit-to-subreddit messages
|
||||
* TIP: `to` can be templated -- to send a message to the subreddit the Activity being processed is in use `'r/{{item.subreddit}}'`
|
||||
* `content` and `title` can be [templated](#templating) and use [URL Tokens](#url-tokens)
|
||||
* `content` can be [templated](#templating) and use [URL Tokens](#url-tokens)
|
||||
|
||||
```yaml
|
||||
actions:
|
||||
- kind: message
|
||||
asSubreddit: true
|
||||
content: 'A message sent as the subreddit' # can be templated
|
||||
title: 'Title of the message' # can be templated
|
||||
to: 'u/aUser' # do not specify 'to' in order default to sending to Author of Activity being processed. Can also be templated
|
||||
content: 'A message sent as the subreddit'
|
||||
title: 'Title of the message'
|
||||
to: 'u/aUser' # do not specify 'to' in order default to sending to Author of Activity being processed
|
||||
```
|
||||
|
||||
### Remove
|
||||
|
||||
@@ -40,7 +40,7 @@
|
||||
// for this to pass the Author of the Submission must not have the flair "Supreme Memer" and have the name "user1" or "user2"
|
||||
{
|
||||
"flairText": ["Supreme Memer"],
|
||||
"name": ["user1","user2"]
|
||||
"names": ["user1","user2"]
|
||||
},
|
||||
{
|
||||
// for this to pass the Author of the Submission must not have the flair "Decent Memer"
|
||||
|
||||
@@ -30,7 +30,7 @@ runs:
|
||||
# for this to pass the Author of the Submission must not have the flair "Supreme Memer" and have the name "user1" or "user2"
|
||||
- flairText:
|
||||
- Supreme Memer
|
||||
name:
|
||||
names:
|
||||
- user1
|
||||
- user2
|
||||
# for this to pass the Author of the Submission must not have the flair "Decent Memer"
|
||||
|
||||
1944
package-lock.json
generated
1944
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@@ -21,7 +21,11 @@
|
||||
"circular-graph": "madge --image graph.svg --circular --extensions ts src/index.ts",
|
||||
"postinstall": "patch-package",
|
||||
"typeorm": "node --require ts-node/register ./node_modules/typeorm/cli.js",
|
||||
"initMigration": "npm run typeorm -- migration:generate -t 1642180264563 -d ormconfig.js \"src/Common/Migrations/Database/init\""
|
||||
"initMigration": "npm run typeorm -- migration:generate -t 1642180264563 -d ormconfig.js \"src/Common/Migrations/Database/init\"",
|
||||
"cv-install-ubuntu-prebuild": "build-opencv --incDir /usr/include/opencv4/ --libDir /lib/x86_64-linux-gnu/ --binDir=/usr/bin/ --nobuild rebuild",
|
||||
"cv-install-docker-prebuild": "build-opencv --incDir /usr/include/opencv4/ --libDir /usr/lib/ --binDir=/usr/bin/ --nobuild rebuild",
|
||||
"cv-autoinstall-cuda": "build-opencv --version 4.5.5 --flags=\"-DWITH_CUDA=ON -DWITH_CUDNN=ON -DOPENCV_DNN_CUDA=ON -DCUDA_FAST_MATH=ON\" build",
|
||||
"cv-autoinstall": "build-opencv build"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16"
|
||||
@@ -42,6 +46,7 @@
|
||||
"@nlpjs/language": "^4.22.7",
|
||||
"@nlpjs/nlp": "^4.23.5",
|
||||
"@stdlib/regexp-regexp": "^0.0.6",
|
||||
"@u4/opencv4nodejs": "^6.2.1",
|
||||
"ajv": "^7.2.4",
|
||||
"ansi-regex": ">=5.0.1",
|
||||
"async": "^3.2.0",
|
||||
@@ -160,6 +165,7 @@
|
||||
"better-sqlite3": "^7.5.0",
|
||||
"mongo": "^3.6.0",
|
||||
"mysql": "^2.18.1",
|
||||
"opencv-build": "^0.1.9",
|
||||
"pg": "^8.7.1",
|
||||
"sharp": "^0.29.1"
|
||||
}
|
||||
|
||||
@@ -3,7 +3,7 @@ import LockAction, {LockActionJson} from "./LockAction";
|
||||
import {RemoveAction, RemoveActionJson} from "./RemoveAction";
|
||||
import {ReportAction, ReportActionJson} from "./ReportAction";
|
||||
import {FlairAction, FlairActionJson} from "./SubmissionAction/FlairAction";
|
||||
import Action, {ActionJson, ActionRuntimeOptions, StructuredActionJson} from "./index";
|
||||
import Action, {ActionJson, StructuredActionJson} from "./index";
|
||||
import {Logger} from "winston";
|
||||
import {UserNoteAction, UserNoteActionJson} from "./UserNoteAction";
|
||||
import ApproveAction, {ApproveActionConfig} from "./ApproveAction";
|
||||
@@ -21,38 +21,38 @@ import {ModNoteAction, ModNoteActionJson} from "./ModNoteAction";
|
||||
import {SubmissionAction, SubmissionActionJson} from "./SubmissionAction";
|
||||
|
||||
export function actionFactory
|
||||
(config: StructuredActionJson, runtimeOptions: ActionRuntimeOptions): Action {
|
||||
(config: StructuredActionJson, logger: Logger, subredditName: string, resources: SubredditResources, client: ExtendedSnoowrap, emitter: EventEmitter): Action {
|
||||
switch (config.kind) {
|
||||
case 'comment':
|
||||
return new CommentAction({...config as StructuredFilter<CommentActionJson>, ...runtimeOptions});
|
||||
return new CommentAction({...config as StructuredFilter<CommentActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'submission':
|
||||
return new SubmissionAction({...config as StructuredFilter<SubmissionActionJson>, ...runtimeOptions});
|
||||
return new SubmissionAction({...config as StructuredFilter<SubmissionActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'lock':
|
||||
return new LockAction({...config as StructuredFilter<LockActionJson>, ...runtimeOptions});
|
||||
return new LockAction({...config as StructuredFilter<LockActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'remove':
|
||||
return new RemoveAction({...config as StructuredFilter<RemoveActionJson>, ...runtimeOptions});
|
||||
return new RemoveAction({...config as StructuredFilter<RemoveActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'report':
|
||||
return new ReportAction({...config as StructuredFilter<ReportActionJson>, ...runtimeOptions});
|
||||
return new ReportAction({...config as StructuredFilter<ReportActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'flair':
|
||||
return new FlairAction({...config as StructuredFilter<FlairActionJson>, ...runtimeOptions});
|
||||
return new FlairAction({...config as StructuredFilter<FlairActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'userflair':
|
||||
return new UserFlairAction({...config as StructuredFilter<UserFlairActionJson>, ...runtimeOptions});
|
||||
return new UserFlairAction({...config as StructuredFilter<UserFlairActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'approve':
|
||||
return new ApproveAction({...config as StructuredFilter<ApproveActionConfig>, ...runtimeOptions});
|
||||
return new ApproveAction({...config as StructuredFilter<ApproveActionConfig>, logger, subredditName, resources, client, emitter});
|
||||
case 'usernote':
|
||||
return new UserNoteAction({...config as StructuredFilter<UserNoteActionJson>, ...runtimeOptions});
|
||||
return new UserNoteAction({...config as StructuredFilter<UserNoteActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'ban':
|
||||
return new BanAction({...config as StructuredFilter<BanActionJson>, ...runtimeOptions});
|
||||
return new BanAction({...config as StructuredFilter<BanActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'message':
|
||||
return new MessageAction({...config as StructuredFilter<MessageActionJson>, ...runtimeOptions});
|
||||
return new MessageAction({...config as StructuredFilter<MessageActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'dispatch':
|
||||
return new DispatchAction({...config as StructuredFilter<DispatchActionJson>, ...runtimeOptions});
|
||||
return new DispatchAction({...config as StructuredFilter<DispatchActionJson>, logger, subredditName, resources, client, emitter});
|
||||
case 'cancelDispatch':
|
||||
return new CancelDispatchAction({...config as StructuredFilter<CancelDispatchActionJson>, ...runtimeOptions})
|
||||
return new CancelDispatchAction({...config as StructuredFilter<CancelDispatchActionJson>, logger, subredditName, resources, client, emitter})
|
||||
case 'contributor':
|
||||
return new ContributorAction({...config as StructuredFilter<ContributorActionJson>, ...runtimeOptions})
|
||||
return new ContributorAction({...config as StructuredFilter<ContributorActionJson>, logger, subredditName, resources, client, emitter})
|
||||
case 'modnote':
|
||||
return new ModNoteAction({...config as StructuredFilter<ModNoteActionJson>, ...runtimeOptions})
|
||||
return new ModNoteAction({...config as StructuredFilter<ModNoteActionJson>, logger, subredditName, resources, client, emitter})
|
||||
default:
|
||||
throw new Error('rule "kind" was not recognized.');
|
||||
}
|
||||
|
||||
@@ -8,7 +8,6 @@ import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTarget, ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {asComment, asSubmission} from "../util";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class ApproveAction extends Action {
|
||||
|
||||
@@ -27,7 +26,7 @@ export class ApproveAction extends Action {
|
||||
this.targets = targets;
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const touchedEntities = [];
|
||||
|
||||
|
||||
@@ -7,18 +7,10 @@ import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {truncateStringToLength} from "../util";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
const truncate = truncateStringToLength(100);
|
||||
const truncateLongMessage = truncateStringToLength(200);
|
||||
|
||||
const truncateIfNotUndefined = (val: string | undefined) => {
|
||||
if(val === undefined) {
|
||||
return undefined;
|
||||
}
|
||||
return truncate(val);
|
||||
}
|
||||
|
||||
export class BanAction extends Action {
|
||||
|
||||
message?: string;
|
||||
@@ -47,13 +39,13 @@ export class BanAction extends Action {
|
||||
return 'ban';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const renderedBody = await this.renderContent(this.message, item, ruleResults, actionResults);
|
||||
const renderedContent = renderedBody === undefined ? undefined : `${renderedBody}${await this.resources.renderFooter(item, this.footer)}`;
|
||||
const renderedBody = this.message === undefined ? undefined : await this.resources.renderContent(this.message, item, ruleResults);
|
||||
const renderedContent = renderedBody === undefined ? undefined : `${renderedBody}${await this.resources.generateFooter(item, this.footer)}`;
|
||||
|
||||
const renderedReason = truncateIfNotUndefined(await this.renderContent(this.reason, item, ruleResults, actionResults) as string);
|
||||
const renderedNote = truncateIfNotUndefined(await this.renderContent(this.note, item, ruleResults, actionResults) as string);
|
||||
const renderedReason = this.reason === undefined ? undefined : truncate(await this.resources.renderContent(this.reason, item, ruleResults));
|
||||
const renderedNote = this.note === undefined ? undefined : truncate(await this.resources.renderContent(this.note, item, ruleResults));
|
||||
|
||||
const touchedEntities = [];
|
||||
let banPieces = [];
|
||||
@@ -80,13 +72,7 @@ export class BanAction extends Action {
|
||||
dryRun,
|
||||
success: true,
|
||||
result: `Banned ${item.author.name} ${durText}${renderedReason !== undefined ? ` (${renderedReason})` : ''}`,
|
||||
touchedEntities,
|
||||
data: {
|
||||
message: renderedContent === undefined ? undefined : renderedContent,
|
||||
reason: renderedReason,
|
||||
note: renderedNote,
|
||||
duration: durText
|
||||
}
|
||||
touchedEntities
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -12,7 +12,6 @@ import {isSubmission, parseDurationValToDuration} from "../util";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTarget, ActionTypes, InclusiveActionTarget} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class CancelDispatchAction extends Action {
|
||||
identifiers?: (string | null)[];
|
||||
@@ -36,7 +35,7 @@ export class CancelDispatchAction extends Action {
|
||||
this.targets = !Array.isArray(target) ? [target] : target;
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
// see note in DispatchAction about missing runtimeDryrun
|
||||
const dryRun = this.dryRun;
|
||||
|
||||
|
||||
@@ -9,7 +9,6 @@ import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTarget, ActionTypes, ArbitraryActionTarget} from "../Common/Infrastructure/Atomic";
|
||||
import {CMError} from "../Utils/Errors";
|
||||
import {SnoowrapActivity} from "../Common/Infrastructure/Reddit";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class CommentAction extends Action {
|
||||
content: string;
|
||||
@@ -45,11 +44,12 @@ export class CommentAction extends Action {
|
||||
return 'comment';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const body = await this.renderContent(this.content, item, ruleResults, actionResults) as string;
|
||||
const content = await this.resources.getContent(this.content, item.subreddit);
|
||||
const body = await renderContent(content, item, ruleResults, this.resources.userNotes);
|
||||
|
||||
const footer = await this.resources.renderFooter(item, this.footer);
|
||||
const footer = await this.resources.generateFooter(item, this.footer);
|
||||
|
||||
const renderedContent = `${body}${footer}`;
|
||||
this.logger.verbose(`Contents:\r\n${renderedContent.length > 100 ? `\r\n${renderedContent}` : renderedContent}`);
|
||||
@@ -154,12 +154,6 @@ export class CommentAction extends Action {
|
||||
success: !allErrors,
|
||||
result: `${targetResults.join('\n')}${truncateStringToLength(100)(body)}`,
|
||||
touchedEntities,
|
||||
data: {
|
||||
body,
|
||||
bodyShort: truncateStringToLength(100)(body),
|
||||
comments: targetResults,
|
||||
commentsFormatted: targetResults.map(x => `* ${x}`).join('\n')
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ import Comment from "snoowrap/dist/objects/Comment";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTarget, ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class ContributorAction extends Action {
|
||||
|
||||
@@ -26,7 +25,7 @@ export class ContributorAction extends Action {
|
||||
this.actionType = action;
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
|
||||
const contributors = await this.resources.getSubredditContributors();
|
||||
|
||||
@@ -8,7 +8,6 @@ import {activityDispatchConfigToDispatch, isSubmission, parseDurationValToDurati
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTarget, ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class DispatchAction extends Action {
|
||||
dispatchData: ActivityDispatchConfig;
|
||||
@@ -40,7 +39,7 @@ export class DispatchAction extends Action {
|
||||
this.targets = !Array.isArray(target) ? [target] : target;
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
// ignore runtimeDryrun here because "real run" isn't causing any reddit api calls to happen
|
||||
// -- basically if bot is in dryrun this should still run since we want the "full effect" of the bot
|
||||
// BUT if the action explicitly sets 'dryRun: true' then do not dispatch as they probably don't want to it actually going (intention?)
|
||||
|
||||
@@ -5,14 +5,13 @@ import {ActionProcessResult, RuleResult} from "../Common/interfaces";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class LockAction extends Action {
|
||||
getKind(): ActionTypes {
|
||||
return 'lock';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const touchedEntities = [];
|
||||
//snoowrap typing issue, thinks comments can't be locked
|
||||
|
||||
@@ -16,7 +16,6 @@ import {ErrorWithCause} from "pony-cause";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class MessageAction extends Action {
|
||||
content: string;
|
||||
@@ -49,30 +48,28 @@ export class MessageAction extends Action {
|
||||
return 'message';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
|
||||
const body = await this.renderContent(this.content, item, ruleResults, actionResults);
|
||||
const titleTemplate = this.title ?? `Concerning your ${isSubmission(item) ? 'Submission' : 'Comment'}`;
|
||||
const subject = await this.renderContent(titleTemplate, item, ruleResults, actionResults) as string;
|
||||
const body = await this.resources.renderContent(this.content, item, ruleResults);
|
||||
const subject = this.title === undefined ? `Concerning your ${isSubmission(item) ? 'Submission' : 'Comment'}` : await this.resources.renderContent(this.title, item, ruleResults);
|
||||
|
||||
const footer = await this.resources.renderFooter(item, this.footer);
|
||||
const footer = await this.resources.generateFooter(item, this.footer);
|
||||
|
||||
const renderedContent = `${body}${footer}`;
|
||||
|
||||
let recipient = item.author.name;
|
||||
if(this.to !== undefined) {
|
||||
const renderedTo = await this.renderContent(this.to, item, ruleResults, actionResults) as string;
|
||||
// parse to value
|
||||
try {
|
||||
const entityData = parseRedditEntity(renderedTo, 'user');
|
||||
const entityData = parseRedditEntity(this.to, 'user');
|
||||
if(entityData.type === 'user') {
|
||||
recipient = entityData.name;
|
||||
} else {
|
||||
recipient = `/r/${entityData.name}`;
|
||||
}
|
||||
} catch (err: any) {
|
||||
throw new ErrorWithCause(`'to' field for message was not in a valid format, given value after templating: ${renderedTo} -- See ${REDDIT_ENTITY_REGEX_URL} for valid examples`, {cause: err});
|
||||
throw new ErrorWithCause(`'to' field for message was not in a valid format. See ${REDDIT_ENTITY_REGEX_URL} for valid examples`, {cause: err});
|
||||
}
|
||||
if(recipient.includes('/r/') && this.asSubreddit) {
|
||||
throw new SimpleError(`Cannot send a message as a subreddit to another subreddit. Requested recipient: ${recipient}`);
|
||||
@@ -126,7 +123,7 @@ export interface MessageActionConfig extends RequiredRichContent, Footer {
|
||||
asSubreddit: boolean
|
||||
|
||||
/**
|
||||
* Entity to send message to. It can be templated.
|
||||
* Entity to send message to.
|
||||
*
|
||||
* If not present Message be will sent to the Author of the Activity being checked.
|
||||
*
|
||||
@@ -138,9 +135,8 @@ export interface MessageActionConfig extends RequiredRichContent, Footer {
|
||||
*
|
||||
* **Note:** Reddit does not support sending a message AS a subreddit TO another subreddit
|
||||
*
|
||||
* **Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`
|
||||
*
|
||||
* @examples ["aUserName","u/aUserName","r/aSubreddit", "r/{{item.subreddit}}"]
|
||||
* @pattern ^\s*(\/[ru]\/|[ru]\/)*(\w+)*\s*$
|
||||
* @examples ["aUserName","u/aUserName","r/aSubreddit"]
|
||||
* */
|
||||
to?: string
|
||||
|
||||
|
||||
@@ -9,7 +9,6 @@ import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes, ModUserNoteLabel} from "../Common/Infrastructure/Atomic";
|
||||
import {ModNote} from "../Subreddit/ModNotes/ModNote";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
|
||||
export class ModNoteAction extends Action {
|
||||
@@ -40,12 +39,13 @@ export class ModNoteAction extends Action {
|
||||
}
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
|
||||
const modLabel = this.type !== undefined ? toModNoteLabel(this.type) : undefined;
|
||||
|
||||
const renderedContent = await this.renderContent(this.content, item, ruleResults, actionResults);
|
||||
const content = await this.resources.getContent(this.content, item.subreddit);
|
||||
const renderedContent = await renderContent(content, item, ruleResults, this.resources.userNotes);
|
||||
this.logger.verbose(`Note:\r\n(${this.type}) ${renderedContent}`);
|
||||
|
||||
// TODO see what changes are made for bulk fetch of notes before implementing this
|
||||
|
||||
@@ -8,7 +8,6 @@ import {isSubmission, truncateStringToLength} from "../util";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
const truncate = truncateStringToLength(100);
|
||||
export class RemoveAction extends Action {
|
||||
@@ -32,7 +31,7 @@ export class RemoveAction extends Action {
|
||||
this.reasonId = reasonId;
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const touchedEntities = [];
|
||||
let removeSummary = [];
|
||||
@@ -45,7 +44,7 @@ export class RemoveAction extends Action {
|
||||
removeSummary.push('Marked as SPAM');
|
||||
this.logger.verbose('Marking as spam on removal');
|
||||
}
|
||||
const renderedNote = await this.renderContent(this.note, item, ruleResults, actionResults);
|
||||
const renderedNote = this.note === undefined ? undefined : await this.resources.renderContent(this.note, item, ruleResults);
|
||||
let foundReasonId: string | undefined;
|
||||
let foundReason: string | undefined;
|
||||
|
||||
@@ -100,8 +99,7 @@ export class RemoveAction extends Action {
|
||||
return {
|
||||
dryRun,
|
||||
success: true,
|
||||
touchedEntities,
|
||||
result: removeSummary.join(' | ')
|
||||
touchedEntities
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ import {ActionProcessResult, RichContent, RuleResult} from "../Common/interfaces
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
// https://www.reddit.com/dev/api/oauth#POST_api_report
|
||||
// denotes 100 characters maximum
|
||||
@@ -26,9 +25,10 @@ export class ReportAction extends Action {
|
||||
return 'report';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const renderedContent = (await this.renderContent(this.content, item, ruleResults, actionResults) as string);
|
||||
const content = await this.resources.getContent(this.content, item.subreddit);
|
||||
const renderedContent = await renderContent(content, item, ruleResults, this.resources.userNotes);
|
||||
this.logger.verbose(`Contents:\r\n${renderedContent}`);
|
||||
const truncatedContent = reportTrunc(renderedContent);
|
||||
const touchedEntities = [];
|
||||
|
||||
@@ -10,7 +10,6 @@ import {ActionTarget, ActionTypes, ArbitraryActionTarget} from "../Common/Infras
|
||||
import {CMError} from "../Utils/Errors";
|
||||
import {SnoowrapActivity} from "../Common/Infrastructure/Reddit";
|
||||
import Subreddit from "snoowrap/dist/objects/Subreddit";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class SubmissionAction extends Action {
|
||||
content?: string;
|
||||
@@ -68,21 +67,21 @@ export class SubmissionAction extends Action {
|
||||
return 'submission';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
|
||||
const title = await this.renderContent(this.title, item, ruleResults, actionResults) as string;
|
||||
const title = await this.resources.renderContent(this.title, item, ruleResults);
|
||||
this.logger.verbose(`Title: ${title}`);
|
||||
|
||||
const url = await this.renderContent(this.url, item, ruleResults, actionResults);
|
||||
const url = this.url !== undefined ? await this.resources.renderContent(this.url, item, ruleResults) : undefined;
|
||||
|
||||
this.logger.verbose(`URL: ${url !== undefined ? url : '[No URL]'}`);
|
||||
|
||||
const body = await this.renderContent(this.content, item, ruleResults, actionResults);
|
||||
const body = this.content !== undefined ? await this.resources.renderContent(this.content, item, ruleResults) : undefined;
|
||||
|
||||
let renderedContent: string | undefined = undefined;
|
||||
if(body !== undefined) {
|
||||
const footer = await this.resources.renderFooter(item, this.footer);
|
||||
const footer = await this.resources.generateFooter(item, this.footer);
|
||||
renderedContent = `${body}${footer}`;
|
||||
this.logger.verbose(`Contents:\r\n${renderedContent.length > 100 ? `\r\n${renderedContent}` : renderedContent}`);
|
||||
} else {
|
||||
@@ -205,11 +204,6 @@ export class SubmissionAction extends Action {
|
||||
success: !allErrors,
|
||||
result: `${targetResults.join('\n')}${this.url !== undefined ? `\nURL: ${this.url}` : ''}${body !== undefined ? truncateStringToLength(100)(body) : ''}`,
|
||||
touchedEntities,
|
||||
data: {
|
||||
body,
|
||||
bodyShort: body !== undefined ? truncateStringToLength(100)(body) : '',
|
||||
submissions: targetResults.map(x => `* ${x}`).join('\n')
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
@@ -315,7 +309,7 @@ export interface SubmissionActionConfig extends RichContent, Footer {
|
||||
* * 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed
|
||||
* * [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos
|
||||
* */
|
||||
targets?: ('self' | string) | ('self' | string)[]
|
||||
targets?: 'self' | string
|
||||
}
|
||||
|
||||
export interface SubmissionActionOptions extends SubmissionActionConfig, ActionOptions {
|
||||
|
||||
@@ -6,7 +6,6 @@ import Comment from 'snoowrap/dist/objects/Comment';
|
||||
import {RuleResultEntity} from "../../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../../Subreddit/Manager";
|
||||
import {ActionTypes} from "../../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class FlairAction extends Action {
|
||||
text: string;
|
||||
@@ -27,7 +26,7 @@ export class FlairAction extends Action {
|
||||
return 'flair';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
let flairParts = [];
|
||||
if(this.text !== '') {
|
||||
|
||||
@@ -4,7 +4,6 @@ import {ActionProcessResult, RuleResult} from '../Common/interfaces';
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export class UserFlairAction extends Action {
|
||||
text?: string;
|
||||
@@ -23,7 +22,7 @@ export class UserFlairAction extends Action {
|
||||
return 'userflair';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
let flairParts = [];
|
||||
|
||||
|
||||
@@ -8,7 +8,6 @@ import {ActionProcessResult, RuleResult} from "../Common/interfaces";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {runCheckOptions} from "../Subreddit/Manager";
|
||||
import {ActionTypes, UserNoteType} from "../Common/Infrastructure/Atomic";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
|
||||
export class UserNoteAction extends Action {
|
||||
@@ -28,9 +27,10 @@ export class UserNoteAction extends Action {
|
||||
return 'usernote';
|
||||
}
|
||||
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
async process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult> {
|
||||
const dryRun = this.getRuntimeAwareDryrun(options);
|
||||
const renderedContent = (await this.renderContent(this.content, item, ruleResults, actionResults) as string);
|
||||
const content = await this.resources.getContent(this.content, item.subreddit);
|
||||
const renderedContent = await renderContent(content, item, ruleResults, this.resources.userNotes);
|
||||
this.logger.verbose(`Note:\r\n(${this.type}) ${renderedContent}`);
|
||||
|
||||
if (!this.allowDuplicate) {
|
||||
|
||||
@@ -19,8 +19,6 @@ import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
import {FindOptionsWhere} from "typeorm/find-options/FindOptionsWhere";
|
||||
import {ActionTypes} from "../Common/Infrastructure/Atomic";
|
||||
import {RunnableBaseJson, RunnableBaseOptions, StructuredRunnableBase} from "../Common/Infrastructure/Runnable";
|
||||
import { SubredditResources } from "../Subreddit/SubredditResources";
|
||||
import {SnoowrapActivity} from "../Common/Infrastructure/Reddit";
|
||||
|
||||
export abstract class Action extends RunnableBase {
|
||||
name?: string;
|
||||
@@ -31,8 +29,6 @@ export abstract class Action extends RunnableBase {
|
||||
managerEmitter: EventEmitter;
|
||||
// actionEntity: ActionEntity | null = null;
|
||||
actionPremiseEntity: ActionPremise | null = null;
|
||||
checkName: string;
|
||||
subredditName: string;
|
||||
|
||||
constructor(options: ActionOptions) {
|
||||
super(options);
|
||||
@@ -44,7 +40,6 @@ export abstract class Action extends RunnableBase {
|
||||
subredditName,
|
||||
dryRun = false,
|
||||
emitter,
|
||||
checkName,
|
||||
} = options;
|
||||
|
||||
this.name = name;
|
||||
@@ -53,8 +48,6 @@ export abstract class Action extends RunnableBase {
|
||||
this.client = client;
|
||||
this.logger = logger.child({labels: [`Action ${this.getActionUniqueName()}`]}, mergeArr);
|
||||
this.managerEmitter = emitter;
|
||||
this.checkName = checkName;
|
||||
this.subredditName = subredditName;
|
||||
}
|
||||
|
||||
abstract getKind(): ActionTypes;
|
||||
@@ -119,7 +112,7 @@ export abstract class Action extends RunnableBase {
|
||||
}
|
||||
}
|
||||
|
||||
async handle(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionResultEntity> {
|
||||
async handle(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionResultEntity> {
|
||||
const {dryRun: runtimeDryrun} = options;
|
||||
const dryRun = runtimeDryrun || this.dryRun;
|
||||
|
||||
@@ -155,11 +148,10 @@ export abstract class Action extends RunnableBase {
|
||||
actRes.runReason = runReason;
|
||||
return actRes;
|
||||
}
|
||||
const results = await this.process(item, ruleResults, actionResults, options);
|
||||
const results = await this.process(item, ruleResults, options);
|
||||
actRes.success = results.success;
|
||||
actRes.dryRun = results.dryRun;
|
||||
actRes.result = results.result;
|
||||
actRes.data = results.data;
|
||||
actRes.touchedEntities = results.touchedEntities ?? [];
|
||||
|
||||
return actRes;
|
||||
@@ -174,31 +166,20 @@ export abstract class Action extends RunnableBase {
|
||||
}
|
||||
}
|
||||
|
||||
abstract process(item: Comment | Submission, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[], options: runCheckOptions): Promise<ActionProcessResult>;
|
||||
abstract process(item: Comment | Submission, ruleResults: RuleResultEntity[], options: runCheckOptions): Promise<ActionProcessResult>;
|
||||
|
||||
getRuntimeAwareDryrun(options: runCheckOptions): boolean {
|
||||
const {dryRun: runtimeDryrun} = options;
|
||||
return runtimeDryrun || this.dryRun;
|
||||
}
|
||||
|
||||
async renderContent(template: string | undefined, item: SnoowrapActivity, ruleResults: RuleResultEntity[], actionResults: ActionResultEntity[]): Promise<string | undefined> {
|
||||
if(template === undefined) {
|
||||
return undefined;
|
||||
}
|
||||
return await this.resources.renderContent(template, item, ruleResults, actionResults, {manager: this.subredditName, check: this.checkName});
|
||||
}
|
||||
}
|
||||
|
||||
export interface ActionRuntimeOptions {
|
||||
checkName: string
|
||||
subredditName: string
|
||||
export interface ActionOptions extends Omit<ActionConfig, 'authorIs' | 'itemIs'>, RunnableBaseOptions {
|
||||
//logger: Logger;
|
||||
subredditName: string;
|
||||
//resources: SubredditResources;
|
||||
client: ExtendedSnoowrap;
|
||||
emitter: EventEmitter;
|
||||
resources: SubredditResources;
|
||||
logger: Logger;
|
||||
}
|
||||
|
||||
export interface ActionOptions extends Omit<ActionConfig, 'authorIs' | 'itemIs'>, RunnableBaseOptions, ActionRuntimeOptions {
|
||||
emitter: EventEmitter
|
||||
}
|
||||
|
||||
export interface ActionConfig extends RunnableBaseJson {
|
||||
|
||||
@@ -386,7 +386,7 @@ class Bot implements BotInstanceFunctions {
|
||||
async testClient(initial = true) {
|
||||
try {
|
||||
// @ts-ignore
|
||||
const user = await this.client.getMe().fetch();
|
||||
const user = this.client.getMe().fetch();
|
||||
this.logger.info('Test API call successful');
|
||||
return user;
|
||||
} catch (err: any) {
|
||||
|
||||
@@ -222,14 +222,7 @@ export abstract class Check extends RunnableBase implements Omit<ICheck, 'postTr
|
||||
this.actions.push(actionFactory({
|
||||
...aj,
|
||||
dryRun: this.dryRun || aj.dryRun
|
||||
}, {
|
||||
logger: this.logger,
|
||||
subredditName,
|
||||
resources: this.resources,
|
||||
client: this.client,
|
||||
emitter: this.emitter,
|
||||
checkName: this.name
|
||||
}));
|
||||
}, this.logger, subredditName, this.resources, this.client, this.emitter));
|
||||
// @ts-ignore
|
||||
a.logger = this.logger;
|
||||
} else {
|
||||
@@ -571,7 +564,7 @@ export abstract class Check extends RunnableBase implements Omit<ICheck, 'postTr
|
||||
const dr = dryRun || this.dryRun;
|
||||
this.logger.debug(`${dr ? 'DRYRUN - ' : ''}Running Actions`);
|
||||
for (const a of this.actions) {
|
||||
const res = await a.handle(item, ruleResults, runActions, options);
|
||||
const res = await a.handle(item, ruleResults, options);
|
||||
runActions.push(res);
|
||||
}
|
||||
this.logger.info(`${dr ? 'DRYRUN - ' : ''}Ran Actions: ${runActions.map(x => x.premise.getFriendlyIdentifier()).join(' | ')}`);
|
||||
|
||||
@@ -56,11 +56,6 @@ export class ActionResultEntity extends TimeAwareRandomBaseEntity {
|
||||
@JoinColumn({name: 'premiseId'})
|
||||
premise!: ActionPremise;
|
||||
|
||||
/**
|
||||
* Ephemeral -- only added during actual run time and used for action templating. Is not available after loading from DB.
|
||||
* */
|
||||
data?: any;
|
||||
|
||||
touchedEntities: (Submission | Comment | RedditUser | string)[] = []
|
||||
|
||||
set itemIs(data: ActivityStateFilterResult | IFilterResult<TypedActivityState> | undefined) {
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import {ActivityType} from "./Reddit";
|
||||
|
||||
/**
|
||||
* A duration and how to compare it against a value
|
||||
*
|
||||
@@ -286,84 +284,3 @@ export interface ImageHashCacheData {
|
||||
original?: string
|
||||
flipped?: string
|
||||
}
|
||||
|
||||
// https://www.reddit.com/message/compose?to=/r/mealtimevideos&message=https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot
|
||||
|
||||
export interface BaseTemplateData {
|
||||
botLink: string
|
||||
modmailLink?: string
|
||||
manager?: string
|
||||
check?: string
|
||||
//[key: string]: any
|
||||
}
|
||||
|
||||
export interface ActivityTemplateData {
|
||||
kind: ActivityType
|
||||
author: string
|
||||
votes: number
|
||||
age: string
|
||||
permalink: string
|
||||
id: string
|
||||
subreddit: string
|
||||
title: string
|
||||
shortTitle: string
|
||||
}
|
||||
|
||||
export interface ModdedActivityTemplateData {
|
||||
reports: number
|
||||
modReports: number
|
||||
userReports: number
|
||||
}
|
||||
|
||||
export interface SubmissionTemplateData extends ActivityTemplateData, Partial<ModdedActivityTemplateData> {
|
||||
nsfw: boolean
|
||||
spoiler: boolean
|
||||
op: boolean
|
||||
upvoteRatio: string
|
||||
url: string
|
||||
}
|
||||
|
||||
export interface CommentTemplateData extends ActivityTemplateData, Partial<ModdedActivityTemplateData> {
|
||||
op: boolean
|
||||
}
|
||||
|
||||
export interface SubredditTemplateData {
|
||||
subredditBreakdownFormatted: string
|
||||
subredditBreakdown?: {
|
||||
totalFormatted: string
|
||||
submissionFormatted: string
|
||||
commentFormatted: string
|
||||
}
|
||||
}
|
||||
|
||||
export interface RuleResultTemplateData {
|
||||
kind: string
|
||||
triggered: boolean
|
||||
result: string
|
||||
[key: string]: any
|
||||
}
|
||||
|
||||
export interface ActionResultTemplateData {
|
||||
kind: string
|
||||
success: boolean
|
||||
result: string
|
||||
[key: string]: any
|
||||
}
|
||||
|
||||
export interface ActionResultsTemplateData {
|
||||
actionSummary: string
|
||||
actions: {
|
||||
[key: string]: ActionResultTemplateData
|
||||
}
|
||||
}
|
||||
|
||||
export interface RuleResultsTemplateData {
|
||||
ruleSummary: string
|
||||
rules: {
|
||||
[key: string]: RuleResultTemplateData
|
||||
}
|
||||
}
|
||||
|
||||
export interface GenericContentTemplateData extends BaseTemplateData, Partial<RuleResultsTemplateData>, Partial<ActionResultsTemplateData> {
|
||||
item?: (SubmissionTemplateData | CommentTemplateData)
|
||||
}
|
||||
|
||||
@@ -88,18 +88,3 @@ export interface SubredditRemovalReason {
|
||||
id: string,
|
||||
title: string
|
||||
}
|
||||
|
||||
export interface SubredditActivityAbsoluteBreakdown {
|
||||
count: number
|
||||
name: string
|
||||
}
|
||||
|
||||
export interface SubredditActivityBreakdown extends SubredditActivityAbsoluteBreakdown {
|
||||
percent: number
|
||||
}
|
||||
|
||||
export interface SubredditActivityBreakdownByType {
|
||||
total: SubredditActivityBreakdown[]
|
||||
submission: SubredditActivityBreakdown[]
|
||||
comment: SubredditActivityBreakdown[]
|
||||
}
|
||||
|
||||
242
src/Common/OpenCVService.ts
Normal file
242
src/Common/OpenCVService.ts
Normal file
@@ -0,0 +1,242 @@
|
||||
import winston, {Logger} from "winston";
|
||||
import {CMError} from "../Utils/Errors";
|
||||
import {formatNumber, mergeArr, resolvePath} from "../util";
|
||||
import * as cvTypes from '@u4/opencv4nodejs'
|
||||
import ImageData from "./ImageData";
|
||||
import {pathToFileURL} from "url";
|
||||
|
||||
let cv: any;
|
||||
|
||||
export const getCV = async (): Promise<typeof cvTypes.cv> => {
|
||||
if (cv === undefined) {
|
||||
try {
|
||||
const cvImport = await import('@u4/opencv4nodejs');
|
||||
if (cvImport === undefined) {
|
||||
throw new CMError('Could not initialize openCV because opencv4nodejs is not installed');
|
||||
}
|
||||
cv = cvImport.default;
|
||||
} catch (e: any) {
|
||||
throw new CMError('Could not initialize openCV', {cause: e});
|
||||
}
|
||||
}
|
||||
return cv as typeof cvTypes.cv;
|
||||
}
|
||||
|
||||
export class OpenCVService {
|
||||
|
||||
logger: Logger;
|
||||
|
||||
constructor(logger?: Logger) {
|
||||
const parentLogger = logger ?? winston.loggers.get('app');
|
||||
this.logger = parentLogger.child({labels: ['OpenCV']}, mergeArr)
|
||||
}
|
||||
|
||||
async cv() {
|
||||
if (cv === undefined) {
|
||||
try {
|
||||
const cvImport = await import('@u4/opencv4nodejs');
|
||||
if (cvImport === undefined) {
|
||||
throw new CMError('Could not initialize openCV because opencv4nodejs is not installed');
|
||||
}
|
||||
cv = cvImport.default;
|
||||
} catch (e: any) {
|
||||
throw new CMError('Could not initialize openCV', {cause: e});
|
||||
}
|
||||
}
|
||||
return cv as typeof cvTypes.cv;
|
||||
}
|
||||
}
|
||||
|
||||
interface CurrentMaxData {
|
||||
confidence: number,
|
||||
loc: cvTypes.Point2,
|
||||
ratio?: number
|
||||
}
|
||||
|
||||
export interface MatchResult {matchRec?: cvTypes.Rect, matchedConfidence?: number}
|
||||
|
||||
|
||||
/**
|
||||
* Use openCV matchTemplate() to find images within images
|
||||
*
|
||||
* The majority of these code concepts are based on https://pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/
|
||||
* and examples/usage of opencv.js is from https://github.com/UrielCh/opencv4nodejs/tree/master/examples/src/templateMatch
|
||||
*
|
||||
* */
|
||||
export class TemplateCompare {
|
||||
cv: typeof cvTypes.cv;
|
||||
logger: Logger;
|
||||
|
||||
template?: cvTypes.Mat;
|
||||
downscaledTemplates: cvTypes.Mat[] = [];
|
||||
|
||||
constructor(cv: typeof cvTypes.cv, logger: Logger) {
|
||||
this.cv = cv;
|
||||
this.logger = logger.child({labels: ['OpenCV', 'Template Match']}, mergeArr)
|
||||
}
|
||||
|
||||
protected async normalizeImage(image: ImageData) {
|
||||
return this.cv.imdecode(await ((await image.sharp()).clone().greyscale().toBuffer()));
|
||||
}
|
||||
|
||||
async setTemplate(image: ImageData) {
|
||||
this.template = await this.normalizeImage(image);
|
||||
}
|
||||
|
||||
protected getTemplate() {
|
||||
if (this.template === undefined) {
|
||||
throw new Error('Template is not defined, use setTemplate() first');
|
||||
}
|
||||
return this.template.copy().canny(50, 200);
|
||||
}
|
||||
|
||||
downscaleTemplates() {
|
||||
if (this.template === undefined) {
|
||||
throw new Error('Template is not defined, use setTemplate() first');
|
||||
}
|
||||
|
||||
const [tH, tW] = this.template.sizes;
|
||||
|
||||
for (let i = 10; i <= 80; i += 10) {
|
||||
const templateRatio = (100 - i) / 100;
|
||||
|
||||
// for debugging
|
||||
// const scaled = this.template.copy().resize(new cv.Size(Math.floor(templateRatio * tW), Math.floor(templateRatio * tH))).canny(50, 200);
|
||||
// const path = pathToFileURL(resolvePath(`./tests/assets/star/starTemplateScaled-${Math.floor(templateRatio * 100)}.jpg`, './')).pathname;
|
||||
// cv.imwrite(path, scaled);
|
||||
this.downscaledTemplates.push(this.template.copy().resize(new cv.Size(Math.floor(templateRatio * tW), Math.floor(templateRatio * tH))).canny(50, 200))
|
||||
}
|
||||
}
|
||||
|
||||
async matchImage(sourceImageData: ImageData, downscaleWhich: 'template' | 'image', confidence = 0.5): Promise<[boolean, MatchResult]> {
|
||||
if (this.template === undefined) {
|
||||
throw new Error('Template is not defined, use setTemplate() first');
|
||||
}
|
||||
|
||||
let currMax: CurrentMaxData | undefined;
|
||||
|
||||
let matchRec: cvTypes.Rect | undefined;
|
||||
let matchedConfidence: number | undefined;
|
||||
|
||||
if (downscaleWhich === 'template') {
|
||||
// in this scenario we assume our template is a significant fraction of the size of the source
|
||||
// so we want to scale down the template size incrementally
|
||||
// because we are assuming the template in the image is smaller than our source template
|
||||
|
||||
// generate scaled templates and save for later use!
|
||||
// its likely this class is in use in Recent/Repeat rules which means we will probably be comparing this template against many images
|
||||
if (this.downscaledTemplates.length === 0) {
|
||||
this.downscaleTemplates();
|
||||
}
|
||||
|
||||
let currMaxTemplateSize: number[] | undefined;
|
||||
|
||||
const src = (await this.normalizeImage(sourceImageData)).canny(50, 200);
|
||||
|
||||
const edgedTemplate = await this.getTemplate();
|
||||
|
||||
for (const scaledTemplate of [edgedTemplate].concat(this.downscaledTemplates)) {
|
||||
|
||||
// more information on methods...
|
||||
// https://docs.opencv.org/4.x/d4/dc6/tutorial_py_template_matching.html
|
||||
// https://stackoverflow.com/questions/58158129/understanding-and-evaluating-template-matching-methods
|
||||
// https://stackoverflow.com/questions/48799711/explain-difference-between-opencvs-template-matching-methods-in-non-mathematica
|
||||
// https://datahacker.rs/014-template-matching-using-opencv-in-python/
|
||||
// ...may want to try with TM_SQDIFF but will need to use minimum values instead of max
|
||||
const result = src.matchTemplate(scaledTemplate, cv.TM_CCOEFF_NORMED);
|
||||
|
||||
const minMax = result.minMaxLoc();
|
||||
const {maxVal, maxLoc} = minMax;
|
||||
|
||||
if (currMax === undefined || maxVal > currMax.confidence) {
|
||||
currMaxTemplateSize = scaledTemplate.sizes;
|
||||
currMax = {confidence: maxVal, loc: maxLoc};
|
||||
console.log(`New Best Max Confidence: ${formatNumber(maxVal, {toFixed: 4})}`)
|
||||
}
|
||||
if (maxVal >= confidence) {
|
||||
this.logger.verbose(`Match with confidence ${formatNumber(maxVal, {toFixed: 4})} met threshold of ${confidence}`);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (currMax !== undefined) {
|
||||
matchedConfidence = currMax.confidence;
|
||||
|
||||
if (currMaxTemplateSize !== undefined) {
|
||||
const startX = currMax.loc.x;
|
||||
const startY = currMax.loc.y;
|
||||
|
||||
matchRec = new cv.Rect(startX, startY, currMaxTemplateSize[1], currMaxTemplateSize[0]);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
} else {
|
||||
// in this scenario we assume our template is small, compared to the source image
|
||||
// and the template found in the source is likely larger than the template
|
||||
// so we scale down the source incrementally to try to get them to match
|
||||
|
||||
const normalSrc = (await this.normalizeImage(sourceImageData));
|
||||
let src = normalSrc.copy();
|
||||
const [width, height] = src.sizes;
|
||||
|
||||
const edgedTemplate = await this.getTemplate();
|
||||
const [tH, tW] = edgedTemplate.sizes;
|
||||
|
||||
let ratio = 1;
|
||||
|
||||
for (let i = 0; i <= 80; i += 5) {
|
||||
ratio = (100 - i) / 100;
|
||||
|
||||
if (i !== 100) {
|
||||
const resizedWidth = Math.floor(width * ratio);
|
||||
const resizedHeight = Math.floor(height * ratio);
|
||||
src = src.resize(new cv.Size(resizedWidth, resizedHeight));
|
||||
}
|
||||
|
||||
const [sH, sW] = src.sizes;
|
||||
if (sH < tH || sW < tW) {
|
||||
// scaled source is smaller than template
|
||||
this.logger.debug(`Template matching ended early due to downscaled image being smaller than template`);
|
||||
break;
|
||||
}
|
||||
|
||||
const edged = src.canny(50, 200);
|
||||
const result = edged.matchTemplate(edgedTemplate, cv.TM_CCOEFF_NORMED);
|
||||
|
||||
const minMax = result.minMaxLoc();
|
||||
const {maxVal, maxLoc} = minMax;
|
||||
|
||||
if (currMax === undefined || maxVal > currMax.confidence) {
|
||||
currMax = {confidence: maxVal, loc: maxLoc, ratio};
|
||||
console.log(`New Best Confidence: ${formatNumber(maxVal, {toFixed: 4})}`)
|
||||
}
|
||||
if (maxVal >= confidence) {
|
||||
this.logger.verbose(`Match with confidence ${formatNumber(maxVal, {toFixed: 4})} met threshold of ${confidence}`);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (currMax === undefined) {
|
||||
// template was larger than source
|
||||
this.logger.debug('No local max found');
|
||||
} else {
|
||||
const maxRatio = currMax.ratio as number;
|
||||
|
||||
const startX = currMax.loc.x * (1 / maxRatio);
|
||||
const startY = currMax.loc.y * (1 / maxRatio);
|
||||
|
||||
const endWidth = tW * (1 / maxRatio);
|
||||
const endHeight = tH * (1 / maxRatio);
|
||||
|
||||
matchRec = new cv.Rect(startX, startY, endWidth, endHeight);
|
||||
matchedConfidence = currMax.confidence;
|
||||
}
|
||||
}
|
||||
|
||||
if (currMax !== undefined) {
|
||||
return [currMax.confidence >= confidence, {matchRec, matchedConfidence}]
|
||||
}
|
||||
return [false, {matchRec, matchedConfidence}]
|
||||
}
|
||||
}
|
||||
@@ -42,4 +42,4 @@ export const filterCriteriaDefault: FilterCriteriaDefaults = {
|
||||
export const defaultDataDir = path.resolve(__dirname, '../..');
|
||||
export const defaultConfigFilenames = ['config.json', 'config.yaml'];
|
||||
|
||||
export const VERSION = '0.12.1';
|
||||
export const VERSION = '0.12.0';
|
||||
|
||||
@@ -1730,7 +1730,6 @@ export interface ActionProcessResult {
|
||||
dryRun: boolean,
|
||||
result?: string
|
||||
touchedEntities?: (Submission | Comment | RedditUser | string)[]
|
||||
data?: any
|
||||
}
|
||||
|
||||
export interface EventActivity {
|
||||
|
||||
@@ -7,10 +7,9 @@ import {Rule, RuleJSONConfig, RuleOptions} from "./index";
|
||||
import Submission from "snoowrap/dist/objects/Submission";
|
||||
import dayjs from "dayjs";
|
||||
import {
|
||||
asComment,
|
||||
asSubmission,
|
||||
FAIL,
|
||||
formatNumber, getActivitySubredditName, historyFilterConfigToOptions, isComment, isSubmission,
|
||||
formatNumber, getActivitySubredditName, historyFilterConfigToOptions, isSubmission,
|
||||
parseSubredditName,
|
||||
PASS,
|
||||
percentFromString, removeUndefinedKeys, toStrongSubredditState, windowConfigToWindowCriteria
|
||||
@@ -21,7 +20,6 @@ import {CompareValueOrPercent} from "../Common/Infrastructure/Atomic";
|
||||
import {ActivityWindowConfig, ActivityWindowCriteria} from "../Common/Infrastructure/ActivityWindow";
|
||||
import {ErrorWithCause} from "pony-cause";
|
||||
import {comparisonTextOp, parseGenericValueOrPercentComparison} from "../Common/Infrastructure/Comparisons";
|
||||
import {getSubredditBreakdownByActivityType} from "../Utils/SnoowrapUtils";
|
||||
|
||||
export interface CommentThresholdCriteria extends ThresholdCriteria {
|
||||
/**
|
||||
@@ -208,11 +206,10 @@ export class HistoryRule extends Rule {
|
||||
fOpTotal = filteredCounts.opTotal;
|
||||
}
|
||||
|
||||
let asOp = false;
|
||||
let commentTrigger = undefined;
|
||||
if(comment !== undefined) {
|
||||
const {operator, value, isPercent, extra = ''} = parseGenericValueOrPercentComparison(comment);
|
||||
asOp = extra.toLowerCase().includes('op');
|
||||
const asOp = extra.toLowerCase().includes('op');
|
||||
if(isPercent) {
|
||||
const per = value / 100;
|
||||
if(asOp) {
|
||||
@@ -267,8 +264,7 @@ export class HistoryRule extends Rule {
|
||||
submissionTrigger,
|
||||
commentTrigger,
|
||||
totalTrigger,
|
||||
triggered: (submissionTrigger === undefined || submissionTrigger === true) && (commentTrigger === undefined || commentTrigger === true) && (totalTrigger === undefined || totalTrigger === true),
|
||||
subredditBreakdown: getSubredditBreakdownByActivityType(!asOp ? filteredActivities : filteredActivities.filter(x => asSubmission(x) || x.is_submitter))
|
||||
triggered: (submissionTrigger === undefined || submissionTrigger === true) && (commentTrigger === undefined || commentTrigger === true) && (totalTrigger === undefined || totalTrigger === true)
|
||||
});
|
||||
}
|
||||
|
||||
@@ -324,7 +320,6 @@ export class HistoryRule extends Rule {
|
||||
submissionTrigger,
|
||||
commentTrigger,
|
||||
totalTrigger,
|
||||
subredditBreakdown,
|
||||
} = results;
|
||||
|
||||
const data: any = {
|
||||
@@ -343,7 +338,6 @@ export class HistoryRule extends Rule {
|
||||
submissionTrigger,
|
||||
commentTrigger,
|
||||
totalTrigger,
|
||||
subredditBreakdown
|
||||
};
|
||||
|
||||
let thresholdSummary = [];
|
||||
|
||||
@@ -43,7 +43,6 @@ import {
|
||||
import {ActivityWindow, ActivityWindowConfig} from "../Common/Infrastructure/ActivityWindow";
|
||||
import {comparisonTextOp, parseGenericValueOrPercentComparison} from "../Common/Infrastructure/Comparisons";
|
||||
import {ImageHashCacheData} from "../Common/Infrastructure/Atomic";
|
||||
import {getSubredditBreakdownByActivityType} from "../Utils/SnoowrapUtils";
|
||||
|
||||
const parseLink = parseUsableLinkIdentifier();
|
||||
|
||||
@@ -509,7 +508,6 @@ export class RecentActivityRule extends Rule {
|
||||
testValue,
|
||||
karmaThreshold,
|
||||
combinedKarma,
|
||||
subredditBreakdown: getSubredditBreakdownByActivityType(activities)
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1664,13 +1664,13 @@
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
|
||||
"description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
|
||||
"examples": [
|
||||
"aUserName",
|
||||
"u/aUserName",
|
||||
"r/aSubreddit",
|
||||
"r/{{item.subreddit}}"
|
||||
"r/aSubreddit"
|
||||
],
|
||||
"pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
@@ -2445,18 +2445,8 @@
|
||||
"type": "boolean"
|
||||
},
|
||||
"targets": {
|
||||
"anyOf": [
|
||||
{
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"type": "string"
|
||||
}
|
||||
],
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos",
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"description": "The title of this Submission.\n\nTemplated the same as **content**",
|
||||
|
||||
@@ -3454,13 +3454,13 @@
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
|
||||
"description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
|
||||
"examples": [
|
||||
"aUserName",
|
||||
"u/aUserName",
|
||||
"r/aSubreddit",
|
||||
"r/{{item.subreddit}}"
|
||||
"r/aSubreddit"
|
||||
],
|
||||
"pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
@@ -5814,18 +5814,8 @@
|
||||
"type": "boolean"
|
||||
},
|
||||
"targets": {
|
||||
"anyOf": [
|
||||
{
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"type": "string"
|
||||
}
|
||||
],
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos",
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"description": "The title of this Submission.\n\nTemplated the same as **content**",
|
||||
|
||||
@@ -3049,13 +3049,13 @@
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
|
||||
"description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
|
||||
"examples": [
|
||||
"aUserName",
|
||||
"u/aUserName",
|
||||
"r/aSubreddit",
|
||||
"r/{{item.subreddit}}"
|
||||
"r/aSubreddit"
|
||||
],
|
||||
"pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
@@ -5139,18 +5139,8 @@
|
||||
"type": "boolean"
|
||||
},
|
||||
"targets": {
|
||||
"anyOf": [
|
||||
{
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"type": "string"
|
||||
}
|
||||
],
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos",
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"description": "The title of this Submission.\n\nTemplated the same as **content**",
|
||||
|
||||
@@ -3165,13 +3165,13 @@
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"description": "Entity to send message to. It can be templated.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit\n\n**Tip:** To send a message to the subreddit of the Activity us `to: 'r/{{item.subreddit}}'`",
|
||||
"description": "Entity to send message to.\n\nIf not present Message be will sent to the Author of the Activity being checked.\n\nValid formats:\n\n* `aUserName` -- send to /u/aUserName\n* `u/aUserName` -- send to /u/aUserName\n* `r/aSubreddit` -- sent to modmail of /r/aSubreddit\n\n**Note:** Reddit does not support sending a message AS a subreddit TO another subreddit",
|
||||
"examples": [
|
||||
"aUserName",
|
||||
"u/aUserName",
|
||||
"r/aSubreddit",
|
||||
"r/{{item.subreddit}}"
|
||||
"r/aSubreddit"
|
||||
],
|
||||
"pattern": "^\\s*(\\/[ru]\\/|[ru]\\/)*(\\w+)*\\s*$",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
@@ -5385,18 +5385,8 @@
|
||||
"type": "boolean"
|
||||
},
|
||||
"targets": {
|
||||
"anyOf": [
|
||||
{
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"type": "string"
|
||||
}
|
||||
],
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos"
|
||||
"description": "Specify where this Submission should be made\n\nValid values: 'self' | [subreddit]\n\n* 'self' -- DEFAULT. Post Submission to same subreddit of Activity being processed\n* [subreddit] -- The name of a subreddit to post Submission to. EX mealtimevideos",
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"description": "The title of this Submission.\n\nTemplated the same as **content**",
|
||||
|
||||
@@ -3,7 +3,7 @@ import objectHash from 'object-hash';
|
||||
import {
|
||||
activityIsDeleted, activityIsFiltered,
|
||||
activityIsRemoved,
|
||||
AuthorTypedActivitiesOptions, BOT_LINK, TemplateContext,
|
||||
AuthorTypedActivitiesOptions, BOT_LINK,
|
||||
getAuthorHistoryAPIOptions, renderContent
|
||||
} from "../Utils/SnoowrapUtils";
|
||||
import {map as mapAsync} from 'async';
|
||||
@@ -161,7 +161,6 @@ import {IncludesData} from "../Common/Infrastructure/Includes";
|
||||
import {parseFromJsonOrYamlToObject} from "../Common/Config/ConfigUtil";
|
||||
import ConfigParseError from "../Utils/ConfigParseError";
|
||||
import {ActivityReport} from "../Common/Entities/ActivityReport";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export const DEFAULT_FOOTER = '\r\n*****\r\nThis action was performed by [a bot.]({{botLink}}) Mention a moderator or [send a modmail]({{modmailLink}}) if you any ideas, questions, or concerns about this action.';
|
||||
|
||||
@@ -1770,24 +1769,9 @@ export class SubredditResources {
|
||||
/**
|
||||
* Convenience method for using getContent and SnoowrapUtils@renderContent in one method
|
||||
* */
|
||||
async renderContent(contentStr: string, activity: SnoowrapActivity, ruleResults: RuleResultEntity[] = [], actionResults: ActionResultEntity[] = [], templateData: TemplateContext = {}) {
|
||||
async renderContent(contentStr: string, data: SnoowrapActivity, ruleResults: RuleResultEntity[] = [], usernotes?: UserNotes) {
|
||||
const content = await this.getContent(contentStr);
|
||||
|
||||
const {usernotes = this.userNotes, ...restData} = templateData;
|
||||
return await renderContent(content, {
|
||||
...restData,
|
||||
activity,
|
||||
usernotes,
|
||||
ruleResults,
|
||||
actionResults,
|
||||
});
|
||||
}
|
||||
|
||||
async renderFooter(item: Submission | Comment, footer: false | string | undefined = this.footer) {
|
||||
if (footer === false) {
|
||||
return '';
|
||||
}
|
||||
return this.renderContent(footer, item);
|
||||
return await renderContent(content, data, ruleResults, usernotes ?? this.userNotes);
|
||||
}
|
||||
|
||||
async getConfigFragment<T>(includesData: IncludesData, validateFunc?: ConfigFragmentValidationFunc): Promise<T> {
|
||||
@@ -3361,6 +3345,19 @@ export class SubredditResources {
|
||||
this.logger.debug(`Cached check result '${result.check.name}' for User ${userName} on Submission ${item.link_id} for ${ttl} seconds (Hash ${hash})`);
|
||||
}
|
||||
|
||||
async generateFooter(item: Submission | Comment, actionFooter?: false | string) {
|
||||
let footer = actionFooter !== undefined ? actionFooter : this.footer;
|
||||
if (footer === false) {
|
||||
return '';
|
||||
}
|
||||
const subName = await item.subreddit.display_name;
|
||||
const permaLink = `https://reddit.com${await item.permalink}`
|
||||
const modmailLink = `https://www.reddit.com/message/compose?to=%2Fr%2F${subName}&message=${encodeURIComponent(permaLink)}`
|
||||
|
||||
const footerRawContent = await this.getContent(footer, item.subreddit);
|
||||
return he.decode(Mustache.render(footerRawContent, {subName, permaLink, modmailLink, botLink: BOT_LINK}));
|
||||
}
|
||||
|
||||
async getImageHash(img: ImageData): Promise<Required<ImageHashCacheData>|undefined> {
|
||||
|
||||
if(img.hashResult !== undefined && img.hashResultFlipped !== undefined) {
|
||||
|
||||
@@ -15,7 +15,6 @@ import {
|
||||
asStrongSubredditState,
|
||||
asSubmission,
|
||||
convertSubredditsRawToStrong,
|
||||
formatNumber,
|
||||
getActivityAuthorName,
|
||||
getActivitySubredditName,
|
||||
isStrongSubredditState, isSubmission,
|
||||
@@ -23,7 +22,7 @@ import {
|
||||
normalizeName,
|
||||
parseDurationValToDuration,
|
||||
parseRedditEntity,
|
||||
parseResultsToMarkdownSummary, removeUndefinedKeys,
|
||||
parseRuleResultsToMarkdownSummary, removeUndefinedKeys,
|
||||
subredditStateIsNameOnly,
|
||||
toStrongSubredditState,
|
||||
truncateStringToLength,
|
||||
@@ -35,14 +34,8 @@ import {URL} from "url";
|
||||
import {isStatusError, MaybeSeriousErrorWithCause, SimpleError} from "./Errors";
|
||||
import {RuleResultEntity} from "../Common/Entities/RuleResultEntity";
|
||||
import {StrongSubredditCriteria, SubredditCriteria} from "../Common/Infrastructure/Filters/FilterCriteria";
|
||||
import {DurationVal, GenericContentTemplateData} from "../Common/Infrastructure/Atomic";
|
||||
import {DurationVal} from "../Common/Infrastructure/Atomic";
|
||||
import {ActivityWindowCriteria} from "../Common/Infrastructure/ActivityWindow";
|
||||
import {
|
||||
SnoowrapActivity,
|
||||
SubredditActivityAbsoluteBreakdown,
|
||||
SubredditActivityBreakdown, SubredditActivityBreakdownByType
|
||||
} from "../Common/Infrastructure/Reddit";
|
||||
import {ActionResultEntity} from "../Common/Entities/ActionResultEntity";
|
||||
|
||||
export const BOT_LINK = 'https://www.reddit.com/r/ContextModBot/comments/otz396/introduction_to_contextmodbot';
|
||||
|
||||
@@ -126,184 +119,73 @@ export const isSubreddit = async (subreddit: Subreddit, stateCriteria: Subreddit
|
||||
const renderContentCommentTruncate = truncateStringToLength(50);
|
||||
const shortTitleTruncate = truncateStringToLength(15);
|
||||
|
||||
export interface TemplateContext {
|
||||
usernotes?: UserNotes
|
||||
check?: string
|
||||
manager?: string
|
||||
ruleResults?: RuleResultEntity[]
|
||||
actionResults?: ActionResultEntity[]
|
||||
activity?: SnoowrapActivity
|
||||
[key: string]: any
|
||||
}
|
||||
|
||||
export const renderContent = async (template: string, data: TemplateContext = {}) => {
|
||||
const {
|
||||
usernotes,
|
||||
ruleResults,
|
||||
actionResults,
|
||||
activity,
|
||||
...restContext
|
||||
} = data;
|
||||
|
||||
let view: GenericContentTemplateData = {
|
||||
export const renderContent = async (template: string, data: (Submission | Comment), ruleResults: RuleResultEntity[] = [], usernotes: UserNotes) => {
|
||||
const conditional: any = {};
|
||||
if(data.can_mod_post) {
|
||||
conditional.reports = data.num_reports;
|
||||
conditional.modReports = data.mod_reports.length;
|
||||
conditional.userReports = data.user_reports.length;
|
||||
}
|
||||
if(asSubmission(data)) {
|
||||
conditional.nsfw = data.over_18;
|
||||
conditional.spoiler = data.spoiler;
|
||||
conditional.op = true;
|
||||
conditional.upvoteRatio = `${data.upvote_ratio * 100}%`;
|
||||
} else {
|
||||
conditional.op = data.is_submitter;
|
||||
}
|
||||
const templateData: any = {
|
||||
kind: data instanceof Submission ? 'submission' : 'comment',
|
||||
// @ts-ignore
|
||||
author: getActivityAuthorName(await data.author),
|
||||
votes: data.score,
|
||||
age: dayjs.duration(dayjs().diff(dayjs.unix(data.created))).humanize(),
|
||||
permalink: `https://reddit.com${data.permalink}`,
|
||||
botLink: BOT_LINK,
|
||||
...restContext
|
||||
};
|
||||
|
||||
if(activity !== undefined) {
|
||||
const conditional: any = {};
|
||||
if (activity.can_mod_post) {
|
||||
conditional.reports = activity.num_reports;
|
||||
conditional.modReports = activity.mod_reports.length;
|
||||
conditional.userReports = activity.user_reports.length;
|
||||
}
|
||||
if (asSubmission(activity)) {
|
||||
conditional.nsfw = activity.over_18;
|
||||
conditional.spoiler = activity.spoiler;
|
||||
conditional.op = true;
|
||||
conditional.upvoteRatio = `${activity.upvote_ratio * 100}%`;
|
||||
} else {
|
||||
conditional.op = activity.is_submitter;
|
||||
}
|
||||
|
||||
const subreddit = activity.subreddit.display_name;
|
||||
const permalink = `https://reddit.com${activity.permalink}`;
|
||||
|
||||
view.modmailLink = `https://www.reddit.com/message/compose?to=%2Fr%2F${subreddit}&message=${encodeURIComponent(permalink)}`;
|
||||
|
||||
const templateData: any = {
|
||||
kind: activity instanceof Submission ? 'submission' : 'comment',
|
||||
// @ts-ignore
|
||||
author: getActivityAuthorName(await activity.author),
|
||||
votes: activity.score,
|
||||
age: dayjs.duration(dayjs().diff(dayjs.unix(activity.created))).humanize(),
|
||||
permalink,
|
||||
id: activity.name,
|
||||
subreddit,
|
||||
...conditional
|
||||
}
|
||||
if (template.includes('{{item.notes') && usernotes !== undefined) {
|
||||
// we need to get notes
|
||||
const notesData = await usernotes.getUserNotes(activity.author);
|
||||
// return usable notes data with some stats
|
||||
const current = notesData.length > 0 ? notesData[notesData.length - 1] : undefined;
|
||||
// group by type
|
||||
const grouped = notesData.reduce((acc: any, x) => {
|
||||
const {[x.noteType]: nt = []} = acc;
|
||||
return Object.assign(acc, {[x.noteType]: nt.concat(x)});
|
||||
}, {});
|
||||
templateData.notes = {
|
||||
data: notesData,
|
||||
current,
|
||||
...grouped,
|
||||
};
|
||||
}
|
||||
if (activity instanceof Submission) {
|
||||
templateData.url = activity.url;
|
||||
templateData.title = activity.title;
|
||||
templateData.shortTitle = shortTitleTruncate(activity.title);
|
||||
} else {
|
||||
templateData.title = renderContentCommentTruncate(activity.body);
|
||||
templateData.shortTitle = shortTitleTruncate(activity.body);
|
||||
}
|
||||
|
||||
view.item = templateData;
|
||||
id: data.name,
|
||||
...conditional
|
||||
}
|
||||
|
||||
|
||||
if(ruleResults !== undefined) {
|
||||
view = {
|
||||
...view,
|
||||
...parseRuleResultForTemplate(ruleResults)
|
||||
}
|
||||
}
|
||||
|
||||
if(actionResults !== undefined) {
|
||||
view = {
|
||||
...view,
|
||||
...parseActionResultForTemplate(actionResults)
|
||||
}
|
||||
}
|
||||
|
||||
const rendered = Mustache.render(template, view) as string;
|
||||
return he.decode(rendered);
|
||||
}
|
||||
|
||||
export const parseActionResultForTemplate = (actionResults: ActionResultEntity[] = []) => {
|
||||
// normalize rule names and map context data
|
||||
// NOTE: we are relying on users to use unique names for action. If they don't only the last action run of kind X will have its results here
|
||||
const normalizedActionResults = actionResults.reduce((acc: object, actionResult) => {
|
||||
const {
|
||||
success,
|
||||
data:{
|
||||
...restData
|
||||
} = {},
|
||||
result,
|
||||
} = actionResult;
|
||||
let name = actionResult.premise.name;
|
||||
const kind = actionResult.premise.kind.name;
|
||||
if(name === undefined || name === null) {
|
||||
name = kind;
|
||||
}
|
||||
let formattedData: any = {};
|
||||
// remove all non-alphanumeric characters (spaces, dashes, underscore) and set to lowercase
|
||||
// we will set this as the rule property name to make it easy to access results from mustache template
|
||||
const normalName = normalizeName(name);
|
||||
return {
|
||||
...acc, [normalName]: {
|
||||
kind,
|
||||
success,
|
||||
result,
|
||||
...restData,
|
||||
...formattedData,
|
||||
}
|
||||
if (template.includes('{{item.notes')) {
|
||||
// we need to get notes
|
||||
const notesData = await usernotes.getUserNotes(data.author);
|
||||
// return usable notes data with some stats
|
||||
const current = notesData.length > 0 ? notesData[notesData.length - 1] : undefined;
|
||||
// group by type
|
||||
const grouped = notesData.reduce((acc: any, x) => {
|
||||
const {[x.noteType]: nt = []} = acc;
|
||||
return Object.assign(acc, {[x.noteType]: nt.concat(x)});
|
||||
}, {});
|
||||
templateData.notes = {
|
||||
data: notesData,
|
||||
current,
|
||||
...grouped,
|
||||
};
|
||||
}, {});
|
||||
|
||||
return {
|
||||
actionSummary: parseResultsToMarkdownSummary(actionResults),
|
||||
actions: normalizedActionResults
|
||||
};
|
||||
}
|
||||
|
||||
export const parseRuleResultForTemplate = (ruleResults: RuleResultEntity[] = []) => {
|
||||
|
||||
}
|
||||
if (data instanceof Submission) {
|
||||
templateData.url = data.url;
|
||||
templateData.title = data.title;
|
||||
templateData.shortTitle = shortTitleTruncate(data.title);
|
||||
} else {
|
||||
templateData.title = renderContentCommentTruncate(data.body);
|
||||
templateData.shortTitle = shortTitleTruncate(data.body);
|
||||
}
|
||||
// normalize rule names and map context data
|
||||
// NOTE: we are relying on users to use unique names for rules. If they don't only the last rule run of kind X will have its results here
|
||||
const normalizedRuleResults = ruleResults.reduce((acc: object, ruleResult) => {
|
||||
const {
|
||||
//name,
|
||||
triggered,
|
||||
data:{
|
||||
subredditBreakdown,
|
||||
...restData
|
||||
} = {},
|
||||
data = {},
|
||||
result,
|
||||
// premise: {
|
||||
// kind
|
||||
// }
|
||||
} = ruleResult;
|
||||
let name = ruleResult.premise.name;
|
||||
const kind = ruleResult.premise.kind.name;
|
||||
if(name === undefined || name === null) {
|
||||
name = kind;
|
||||
}
|
||||
let formattedData: any = {};
|
||||
if (subredditBreakdown !== undefined) {
|
||||
// format breakdown for markdown
|
||||
if (Array.isArray(subredditBreakdown)) {
|
||||
const bdArr = subredditBreakdown as SubredditActivityBreakdown[];
|
||||
formattedData.subredditBreakdownFormatted = formatSubredditBreakdownAsMarkdownList(bdArr);
|
||||
} else {
|
||||
const bd = subredditBreakdown as SubredditActivityBreakdownByType;
|
||||
|
||||
// default to total
|
||||
formattedData.subredditBreakdownFormatted = formatSubredditBreakdownAsMarkdownList(bd.total);
|
||||
|
||||
const formatted = Object.entries((bd)).reduce((acc: { [key: string]: string }, curr) => {
|
||||
const [name, breakdownData] = curr;
|
||||
acc[`${name}Formatted`] = formatSubredditBreakdownAsMarkdownList(breakdownData);
|
||||
return acc;
|
||||
}, {});
|
||||
formattedData.subredditBreakdown = {...bd, ...formatted};
|
||||
}
|
||||
}
|
||||
// remove all non-alphanumeric characters (spaces, dashes, underscore) and set to lowercase
|
||||
// we will set this as the rule property name to make it easy to access results from mustache template
|
||||
const normalName = normalizeName(name);
|
||||
@@ -312,16 +194,14 @@ export const parseRuleResultForTemplate = (ruleResults: RuleResultEntity[] = [])
|
||||
kind,
|
||||
triggered,
|
||||
result,
|
||||
...restData,
|
||||
...formattedData,
|
||||
...data,
|
||||
}
|
||||
};
|
||||
}, {});
|
||||
|
||||
return {
|
||||
ruleSummary: parseResultsToMarkdownSummary(ruleResults),
|
||||
rules: normalizedRuleResults
|
||||
};
|
||||
const view = {item: templateData, ruleSummary: parseRuleResultsToMarkdownSummary(ruleResults), rules: normalizedRuleResults};
|
||||
const rendered = Mustache.render(template, view) as string;
|
||||
return he.decode(rendered);
|
||||
}
|
||||
|
||||
export interface ItemContent {
|
||||
@@ -511,58 +391,3 @@ export const getAuthorHistoryAPIOptions = (val: any) => {
|
||||
|
||||
return opts;
|
||||
}
|
||||
|
||||
export const getSubredditBreakdown = (activities: SnoowrapActivity[] = []): SubredditActivityBreakdown[] => {
|
||||
if(activities.length === 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const total = activities.length;
|
||||
|
||||
const countBd = activities.reduce((acc: { [key: string]: number }, curr) => {
|
||||
const subName = curr.subreddit.display_name;
|
||||
if (acc[subName] === undefined) {
|
||||
acc[subName] = 0;
|
||||
}
|
||||
acc[subName]++;
|
||||
|
||||
return acc;
|
||||
}, {});
|
||||
|
||||
const breakdown: SubredditActivityBreakdown[] = Object.entries(countBd).reduce((acc, curr) => {
|
||||
const [name, count] = curr;
|
||||
return acc.concat(
|
||||
{
|
||||
name,
|
||||
count,
|
||||
percent: Number.parseFloat(formatNumber((count / total) * 100))
|
||||
}
|
||||
);
|
||||
}, ([] as SubredditActivityBreakdown[]));
|
||||
|
||||
return breakdown;
|
||||
}
|
||||
|
||||
export const getSubredditBreakdownByActivityType = (activities: SnoowrapActivity[]): SubredditActivityBreakdownByType => {
|
||||
|
||||
return {
|
||||
total: getSubredditBreakdown(activities),
|
||||
submission: getSubredditBreakdown(activities.filter(x => x instanceof Submission)),
|
||||
comment: getSubredditBreakdown(activities.filter(x => x instanceof Comment)),
|
||||
}
|
||||
}
|
||||
|
||||
export const formatSubredditBreakdownAsMarkdownList = (data: SubredditActivityBreakdown[] = []): string => {
|
||||
if(data.length === 0) {
|
||||
return '';
|
||||
}
|
||||
data.sort((a, b) => b.count - a.count);
|
||||
|
||||
const bd = data.map(x => {
|
||||
const entity = parseRedditEntity(x.name);
|
||||
const prefixedName = entity.type === 'subreddit' ? `r/${entity.name}` : `u/${entity.name}`;
|
||||
return `* ${prefixedName} - ${x.count} (${x.percent}%)`
|
||||
}).join('\n');
|
||||
|
||||
return `${bd}\n`;
|
||||
}
|
||||
|
||||
18
src/util.ts
18
src/util.ts
@@ -117,7 +117,6 @@ import {
|
||||
import {RunnableBaseJson} from "./Common/Infrastructure/Runnable";
|
||||
import Snoowrap from "snoowrap";
|
||||
import { uniqueNamesGenerator, adjectives, colors, animals } from 'unique-names-generator';
|
||||
import {ActionResultEntity} from "./Common/Entities/ActionResultEntity";
|
||||
|
||||
|
||||
//import {ResembleSingleCallbackComparisonResult} from "resemblejs";
|
||||
@@ -1935,28 +1934,21 @@ export function findLastIndex<T>(array: Array<T>, predicate: (value: T, index: n
|
||||
return -1;
|
||||
}
|
||||
|
||||
export const parseResultsToMarkdownSummary = (ruleResults: (RuleResultEntity | ActionResultEntity)[]): string => {
|
||||
export const parseRuleResultsToMarkdownSummary = (ruleResults: RuleResultEntity[]): string => {
|
||||
const results = ruleResults.map((y) => {
|
||||
let name = y.premise.name;
|
||||
const kind = y.premise.kind.name;
|
||||
if(name === undefined) {
|
||||
name = kind;
|
||||
}
|
||||
let runIndicator = null;
|
||||
if(y instanceof RuleResultEntity) {
|
||||
runIndicator = y.triggered;
|
||||
} else {
|
||||
runIndicator = y.success;
|
||||
}
|
||||
const {result, ...restY} = y;
|
||||
|
||||
const {triggered, result, ...restY} = y;
|
||||
let t = triggeredIndicator(false);
|
||||
if(runIndicator === null) {
|
||||
if(triggered === null) {
|
||||
t = 'Skipped';
|
||||
} else if(runIndicator === true) {
|
||||
} else if(triggered === true) {
|
||||
t = triggeredIndicator(true);
|
||||
}
|
||||
return `* ${name} - ${t}${result !== undefined ? ` - ${result}` : ''}`;
|
||||
return `* ${name} - ${t} - ${result || '-'}`;
|
||||
});
|
||||
return results.join('\r\n');
|
||||
}
|
||||
|
||||
BIN
tests/assets/star-inside.png
Normal file
BIN
tests/assets/star-inside.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 809 KiB |
BIN
tests/assets/star-transparent.png
Normal file
BIN
tests/assets/star-transparent.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 75 KiB |
BIN
tests/assets/tran-selection.jpg
Normal file
BIN
tests/assets/tran-selection.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 45 KiB |
BIN
tests/assets/tran.jpg
Normal file
BIN
tests/assets/tran.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 90 KiB |
@@ -103,8 +103,8 @@ describe('Hash Comparisons', function () {
|
||||
});
|
||||
await compareImg.hash(32);
|
||||
|
||||
const distanceNormal = leven(original.hashResult as string, compareImg.hashResult as string);
|
||||
const diffNormal = (distanceNormal/(original.hashResult as string).length)*100;
|
||||
const distanceNormal = leven(original.hashResult, compareImg.hashResult);
|
||||
const diffNormal = (distanceNormal/original.hashResult.length)*100;
|
||||
|
||||
assert.equal(diffNormal, 0);
|
||||
});
|
||||
@@ -116,8 +116,8 @@ describe('Hash Comparisons', function () {
|
||||
});
|
||||
await compareImg.hash(32);
|
||||
|
||||
const distanceNormal = leven(original.hashResult as string, compareImg.hashResult as string);
|
||||
const diffNormal = (distanceNormal/(original.hashResult as string).length)*100;
|
||||
const distanceNormal = leven(original.hashResult, compareImg.hashResult);
|
||||
const diffNormal = (distanceNormal/original.hashResult.length)*100;
|
||||
|
||||
assert.isAtMost(diffNormal, 4);
|
||||
});
|
||||
@@ -129,8 +129,8 @@ describe('Hash Comparisons', function () {
|
||||
});
|
||||
await compareImg.hash(32);
|
||||
|
||||
const distanceNormal = leven(original.hashResult as string, compareImg.hashResult as string);
|
||||
const diffNormal = (distanceNormal/(original.hashResult as string).length)*100;
|
||||
const distanceNormal = leven(original.hashResult, compareImg.hashResult);
|
||||
const diffNormal = (distanceNormal/original.hashResult.length)*100;
|
||||
|
||||
assert.equal(diffNormal, 0);
|
||||
});
|
||||
@@ -142,13 +142,13 @@ describe('Hash Comparisons', function () {
|
||||
});
|
||||
await flipped.hash(32);
|
||||
|
||||
const distanceNormal = leven(original.hashResult as string, flipped.hashResult as string);
|
||||
const diffNormal = (distanceNormal/(original.hashResult as string).length)*100;
|
||||
const distanceNormal = leven(original.hashResult, flipped.hashResult);
|
||||
const diffNormal = (distanceNormal/original.hashResult.length)*100;
|
||||
|
||||
assert.isAtLeast(diffNormal, 50);
|
||||
|
||||
const distanceFlipped = leven(original.hashResult as string, flipped.hashResultFlipped as string);
|
||||
const diffFlipped = (distanceFlipped/(original.hashResult as string).length)*100;
|
||||
const distanceFlipped = leven(original.hashResult, flipped.hashResultFlipped);
|
||||
const diffFlipped = (distanceFlipped/original.hashResult.length)*100;
|
||||
|
||||
assert.isAtMost(diffFlipped, 4);
|
||||
});
|
||||
@@ -160,8 +160,8 @@ describe('Hash Comparisons', function () {
|
||||
});
|
||||
await compareImg.hash(32);
|
||||
|
||||
const distanceNormal = leven(original.hashResult as string, compareImg.hashResult as string);
|
||||
const diffNormal = (distanceNormal/(original.hashResult as string).length)*100;
|
||||
const distanceNormal = leven(original.hashResult, compareImg.hashResult);
|
||||
const diffNormal = (distanceNormal/original.hashResult.length)*100;
|
||||
|
||||
assert.isAtMost(diffNormal, 10);
|
||||
});
|
||||
@@ -173,8 +173,8 @@ describe('Hash Comparisons', function () {
|
||||
});
|
||||
await compareImg.hash(32);
|
||||
|
||||
const distanceNormal = leven(original.hashResult as string, compareImg.hashResult as string);
|
||||
const diffNormal = (distanceNormal/(original.hashResult as string).length)*100;
|
||||
const distanceNormal = leven(original.hashResult, compareImg.hashResult);
|
||||
const diffNormal = (distanceNormal/original.hashResult.length)*100;
|
||||
|
||||
assert.isAtLeast(diffNormal, 50);
|
||||
});
|
||||
|
||||
75
tests/opencv.test.ts
Normal file
75
tests/opencv.test.ts
Normal file
@@ -0,0 +1,75 @@
|
||||
import {describe, it} from 'mocha';
|
||||
import chai from 'chai';
|
||||
import chaiAsPromised from 'chai-as-promised';
|
||||
import express, {Request, Response} from "express";
|
||||
import {formatNumber, resolvePath, sleep} from "../src/util";
|
||||
import {pathToFileURL, URL} from "url";
|
||||
import ImageData from "../src/Common/ImageData";
|
||||
import * as cvTypes from '@u4/opencv4nodejs'
|
||||
import {getCV, TemplateCompare} from "../src/Common/OpenCVService";
|
||||
import winston from 'winston';
|
||||
|
||||
chai.use(chaiAsPromised);
|
||||
|
||||
const assert = chai.assert;
|
||||
|
||||
const star = pathToFileURL(resolvePath('./tests/assets/star-transparent.png', './'));
|
||||
const starInside = pathToFileURL(resolvePath('./tests/assets/star-inside.png', './'));
|
||||
const tran = pathToFileURL(resolvePath('./tests/assets/tran.jpg', './'));
|
||||
const tranSel = pathToFileURL(resolvePath('./tests/assets/tran-selection.jpg', './'));
|
||||
|
||||
describe('Template Matching', function () {
|
||||
|
||||
let cv: typeof cvTypes.cv;
|
||||
|
||||
before(async () => {
|
||||
cv = await getCV();
|
||||
});
|
||||
|
||||
it('matches a standard example', async function () {
|
||||
|
||||
const templateMatch = new TemplateCompare(cv, winston.loggers.get('app'));
|
||||
|
||||
await templateMatch.setTemplate(new ImageData({path: tranSel}));
|
||||
|
||||
const [passed, results] = await templateMatch.matchImage(new ImageData({
|
||||
path: tran
|
||||
}), 'template');
|
||||
|
||||
if(results.matchRec !== undefined) {
|
||||
const src = cv.imread(tran.pathname);
|
||||
src.drawRectangle(
|
||||
results.matchRec,
|
||||
new cv.Vec3(0, 255, 0),
|
||||
2,
|
||||
cv.LINE_8
|
||||
);
|
||||
// TODO mask is not drawn correctly (its above?)
|
||||
cv.imwrite(pathToFileURL(resolvePath(`./tests/assets/tran-masked.jpg`, './')).pathname, src);
|
||||
}
|
||||
|
||||
assert.isTrue(passed);
|
||||
});
|
||||
|
||||
it('matches a template using service', async function () {
|
||||
|
||||
const templateMatch = new TemplateCompare(cv, winston.loggers.get('app'));
|
||||
|
||||
await templateMatch.setTemplate(new ImageData({path: star}));
|
||||
|
||||
const [passed, results] = await templateMatch.matchImage(new ImageData({
|
||||
path: starInside
|
||||
}), 'template', 0.2);
|
||||
|
||||
if(results.matchRec !== undefined) {
|
||||
const src = cv.imread(starInside.pathname);
|
||||
src.drawRectangle(
|
||||
results.matchRec,
|
||||
new cv.Vec3(0, 255, 0),
|
||||
2,
|
||||
cv.LINE_8
|
||||
);
|
||||
cv.imwrite(pathToFileURL(resolvePath(`./tests/assets/star-masked.jpg`, './')).pathname, src);
|
||||
}
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user