github/foam
1
1
Fork 0
mirror of https://github.com/foambubble/foam.git synced 2026-01-10 22:48:09 -05:00
Code Issues Packages Projects Releases Wiki Activity

added support for graph middleware via local plugin (#261)

Browse Source 
* added support for graph middleware via local plugin

* added support for parsing extension points / plugins

* improved parser plugin and added docs

* improved name of parameter

* need to enable local plugins, and improved configuration system

For security reasons local plugins are off by default.
The feature can be enabled via a flag in the foam configuration, which has been expanded to support this case.
The configuration system now reads a `config.json` file inside the `.foam` directory as well as a `~/.foam/config.json` file to configure the system.
Only the user specific configuration file can be used to enable local plugins, as a security measure against malicious repos.

* added prettier configuration file

This ensures consistency across machines as well as an explicit source of truth
This commit is contained in:
Riccardo
2020-10-07 22:19:04 +02:00
committed by GitHub
parent 39854277a9
commit 74dbf485df
31 changed files with 740 additions and 202 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
docs/foam-local-plugins.md Normal file
View File

@@ -0,0 +1,54 @@
# Foam Local Plugins
Foam can use workspace plugins to provide customization for users.
## ATTENTION
This feature is experimental and its API subject to change.
**Local plugins can execute arbitrary code on your machine** - ensure you trust the content of the repo.
## Goal
Here are some of the things that we could enable with local plugins in Foam:
- extend the document syntax to support roam style attributes (e.g. `stage:: seedling`)
- automatically add tags to my notes based on the location in the repo (e.g. notes in `/areas/finance` will automatically get the `#finance` tag)
- add a new CLI command to support some internal use case or automate import/export
- extend the VSCode experience to support one's own workflow, e.g. weekly note, templates, extra panels, foam model derived TOC, ... all without having to write/deploy a VSCode extension
## How to enable local plugins
Plugins can execute arbitrary code on the client's machine.
For this reason this feature is disabled by default, and needs to be explicitly enabled.
To enable the feature:
- create a `~/.foam/config.json` file
- add the following content to the file
```
{
"experimental": {
"localPlugins": {
"enabled": true
}
}
}
```
For security reasons this setting can only be defined in the user settings file.
(otherwise a malicious repo could set it via its `./foam/config.json`)
- [[todo]] an additional security mechanism would involve having an explicit list of whitelisted repo paths where plugins are allowed. This would provide finer grain control over when to enable or disable the feature.
## Technical approach
When Foam is loaded it will check whether the experimental local plugin feature is enabled, and in such case it will:
- check `.foam/plugins` directory.
- each directory in there is considered a plugin
- the layout of each directory is
- `index.js` contains the main info about the plugin, specifically it exports:
- `name: string` the name of the plugin
- `description?: string` the description of the plugin
- `graphMiddleware?: Middleware` an object that can intercept calls to the Foam graph
- `parser?: ParserPlugin` an object that interacts with the markdown parsing phase
Currently for simplicity we keep everything in one file. We might in the future split the plugin by domain (e.g. vscode, cli, core, ...)

19
packages/foam-cli/src/commands/janitor.ts
View File

@@ -1,10 +1,11 @@
import { Command, flags } from '@oclif/command';
import * as ora from 'ora';
import ora from 'ora';
import {
initializeNoteGraph,
bootstrap,
createConfigFromFolders,
generateLinkReferences,
generateHeading,
applyTextEdit
applyTextEdit,
} from 'foam-core';
import { writeFileToDisk } from '../utils/write-file-to-disk';
import { isValidDirectory } from '../utils';
@@ -21,7 +22,8 @@ export default class Janitor extends Command {
static flags = {
'without-extensions': flags.boolean({
char: 'w',
description: 'generate link reference definitions without extensions (for legacy support)'
description:
'generate link reference definitions without extensions (for legacy support)',
}),
help: flags.help({ char: 'h' }),
};
@@ -36,7 +38,8 @@ export default class Janitor extends Command {
const { workspacePath = './' } = args;
if (isValidDirectory(workspacePath)) {
const graph = await initializeNoteGraph(workspacePath);
const graph = (await bootstrap(createConfigFromFolders(workspacePath)))
.notes;
const notes = graph.getNotes().filter(Boolean); // removes undefined notes
@@ -54,7 +57,11 @@ export default class Janitor extends Command {
const fileWritePromises = notes.map(note => {
// Get edits
const heading = generateHeading(note);
const definitions = generateLinkReferences(note, graph, !flags['without-extensions']);
const definitions = generateLinkReferences(
note,
graph,
!flags['without-extensions']
);
// apply Edits
let file = note.source.text;

21
packages/foam-cli/src/commands/migrate.ts
View File

@@ -1,11 +1,12 @@
import { Command, flags } from '@oclif/command';
import * as ora from 'ora';
import ora from 'ora';
import {
initializeNoteGraph,
bootstrap,
createConfigFromFolders,
generateLinkReferences,
generateHeading,
getKebabCaseFileName,
applyTextEdit
applyTextEdit,
} from 'foam-core';
import { writeFileToDisk } from '../utils/write-file-to-disk';
import { renameFile } from '../utils/rename-file';
@@ -25,7 +26,8 @@ Successfully generated link references and heading!
static flags = {
'without-extensions': flags.boolean({
char: 'w',
description: 'generate link reference definitions without extensions (for legacy support)'
description:
'generate link reference definitions without extensions (for legacy support)',
}),
help: flags.help({ char: 'h' }),
};
@@ -38,9 +40,10 @@ Successfully generated link references and heading!
const { args, flags } = this.parse(Migrate);
const { workspacePath = './' } = args;
const config = createConfigFromFolders(workspacePath);
if (isValidDirectory(workspacePath)) {
let graph = await initializeNoteGraph(workspacePath);
let graph = (await bootstrap(config)).notes;
let notes = graph.getNotes().filter(Boolean); // removes undefined notes
@@ -69,7 +72,7 @@ Successfully generated link references and heading!
spinner.text = 'Renaming files';
// Reinitialize the graph after renaming files
graph = await initializeNoteGraph(workspacePath);
graph = (await bootstrap(config)).notes;
notes = graph.getNotes().filter(Boolean); // remove undefined notes
@@ -80,7 +83,11 @@ Successfully generated link references and heading!
notes.map(note => {
// Get edits
const heading = generateHeading(note);
const definitions = generateLinkReferences(note, graph, !flags['without-extensions']);
const definitions = generateLinkReferences(
note,
graph,
!flags['without-extensions']
);
// apply Edits
let file = note.source.text;

1
packages/foam-cli/tsconfig.json
View File

@@ -6,6 +6,7 @@
"outDir": "lib",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"target": "es2017"
},
"include": [

59
packages/foam-core/src/bootstrap.ts Normal file
View File

@@ -0,0 +1,59 @@
import glob from 'glob';
import { promisify } from 'util';
import fs from 'fs';
import os from 'os';
import detectNewline from 'detect-newline';
import { createGraph, NoteGraphAPI } from './note-graph';
import { createMarkdownParser } from './markdown-provider';
import { FoamConfig, Foam } from './index';
import { loadPlugins } from './plugins';
import { isNotNull } from './utils';
import { NoteParser } from './types';
const findAllFiles = promisify(glob);
const loadNoteGraph = (
graph: NoteGraphAPI,
parser: NoteParser,
files: string[]
) => {
return Promise.all(
files.map(f => {
return fs.promises.readFile(f).then(data => {
const markdown = (data || '').toString();
const eol = detectNewline(markdown) || os.EOL;
graph.setNote(parser.parse(f, markdown, eol));
});
})
).then(() => graph);
};
export const bootstrap = async (config: FoamConfig) => {
const plugins = await loadPlugins(config);
const middlewares = plugins
.map(p => p.graphMiddleware || null)
.filter(isNotNull);
const parserPlugins = plugins.map(p => p.parser || null).filter(isNotNull);
const parser = createMarkdownParser(parserPlugins);
const files = await Promise.all(
config.workspaceFolders.map(folder => {
if (folder.substr(-1) === '/') {
folder = folder.slice(0, -1);
}
return findAllFiles(`${folder}/**/*.md`, {});
})
);
const graph = await loadNoteGraph(
createGraph(middlewares),
parser,
([] as string[]).concat(...files)
);
return {
notes: graph,
config: config,
parse: parser.parse,
} as Foam;
};

51
packages/foam-core/src/config.ts Normal file
View File

@@ -0,0 +1,51 @@
import { readFileSync } from 'fs';
import { merge } from 'lodash';
export interface FoamConfig {
workspaceFolders: string[];
get<T>(path: string): T | undefined;
get<T>(path: string, defaultValue: T): T;
}
export const createConfigFromObject = (
workspaceFolders: string[],
settings: any
) => {
const config: FoamConfig = {
workspaceFolders: workspaceFolders,
get: <T>(path: string, defaultValue?: T) => {
const tokens = path.split('.');
const value = tokens.reduce((acc, t) => acc?.[t], settings);
return value ?? defaultValue;
},
};
return config;
};
export const createConfigFromFolders = (
workspaceFolders: string[]
): FoamConfig => {
const workspaceConfig: any = workspaceFolders.reduce(
(acc, f) => merge(acc, parseConfig(`${f}/config.json`)),
{}
);
// For security reasons local plugins can only be
// activated via user config
if ('experimental' in workspaceConfig) {
delete workspaceConfig['experimental']['localPlugins'];
}
const userConfig = parseConfig(`~/.foam/config.json`);
const settings = merge(workspaceConfig, userConfig);
return createConfigFromObject(workspaceFolders, settings);
};
const parseConfig = (path: string) => {
try {
return JSON.parse(readFileSync(path, 'utf8'));
} catch {
console.warn('Could not read configuration from ' + path);
}
};

27
packages/foam-core/src/index.ts
View File

@@ -1,7 +1,10 @@
import { NoteGraph, Note, NoteLink } from './note-graph';
import { Note, NoteLink } from './types';
import { NoteGraph, NoteGraphAPI } from './note-graph';
import { FoamConfig } from './config';
export { FoamConfig };
export {
createNoteFromMarkdown,
createMarkdownReferences,
stringifyMarkdownLinkReferenceDefinition,
} from './markdown-provider';
@@ -15,25 +18,19 @@ export {
export { applyTextEdit } from './janitor/apply-text-edit';
export { initializeNoteGraph } from './initialize-note-graph';
export { createConfigFromFolders } from './config';
export { NoteGraph, Note, NoteLink };
export { bootstrap } from './bootstrap';
export { NoteGraph, NoteGraphAPI, Note, NoteLink };
export {
LINK_REFERENCE_DEFINITION_HEADER,
LINK_REFERENCE_DEFINITION_FOOTER,
} from './definitions';
export interface FoamConfig {
// TODO
}
export interface Foam {
notes: NoteGraph;
// config: FoamConfig
notes: NoteGraphAPI;
config: FoamConfig;
parse: (uri: string, text: string, eol: string) => Note;
}
export const createFoam = (config: FoamConfig) => ({
notes: new NoteGraph(),
config: config,
});

30
packages/foam-core/src/initialize-note-graph.ts
View File

@@ -1,30 +0,0 @@
import glob from 'glob';
import { promisify } from 'util';
import fs from 'fs';
import os from 'os';
import detectNewline from 'detect-newline';
import { NoteGraph } from './note-graph';
import { createNoteFromMarkdown } from './markdown-provider';
const findAllFiles = promisify(glob);
export const initializeNoteGraph = async (workspacePath: string) => {
// remove trailing slash from workspacePath if exists
if (workspacePath.substr(-1) === '/')
workspacePath = workspacePath.slice(0, -1);
const files = await findAllFiles(`${workspacePath}/**/*.md`, {});
const graph = new NoteGraph();
await Promise.all(
(await files).map(f => {
return fs.promises.readFile(f).then(data => {
const markdown = (data || '').toString();
const eol = detectNewline(markdown) || os.EOL;
graph.setNote(createNoteFromMarkdown(f, markdown, eol));
});
})
);
return graph;
};

5
packages/foam-core/src/janitor/index.ts
View File

@@ -1,6 +1,7 @@
import { Position } from 'unist';
import GithubSlugger from 'github-slugger';
import { Note, GraphNote, NoteGraph } from '../note-graph';
import { GraphNote, NoteGraphAPI } from '../note-graph';
import { Note } from '../types';
import {
LINK_REFERENCE_DEFINITION_HEADER,
LINK_REFERENCE_DEFINITION_FOOTER,
@@ -20,7 +21,7 @@ export interface TextEdit {
export const generateLinkReferences = (
note: GraphNote,
ng: NoteGraph,
ng: NoteGraphAPI,
includeExtensions: boolean
): TextEdit | null => {
if (!note) {

135
packages/foam-core/src/markdown-provider.ts
View File

@@ -6,88 +6,115 @@ import { parse as parseYAML } from 'yaml';
import visit, { CONTINUE, EXIT } from 'unist-util-visit';
import { Node, Parent, Point } from 'unist';
import * as path from 'path';
import { NoteLink, NoteLinkDefinition, NoteGraph, Note } from './note-graph';
import { NoteGraphAPI } from './note-graph';
import { NoteLink, NoteLinkDefinition, Note, NoteParser } from './types';
import { dropExtension, uriToSlug } from './utils';
import { ID } from './types';
import { ParserPlugin } from './plugins';
let processor: unified.Processor | null = null;
function parse(markdown: string): Node {
processor =
processor ||
unified()
.use(markdownParse, { gfm: true })
.use(frontmatterPlugin, ['yaml'])
.use(wikiLinkPlugin);
return processor.parse(markdown);
}
export function createNoteFromMarkdown(
uri: string,
markdown: string,
eol: string
): Note {
const tree = parse(markdown);
let title: string | null = null;
visit(tree, node => {
if (node.type === 'heading' && node.depth === 1) {
title = ((node as Parent)!.children[0].value as string) || title;
}
return title === null ? CONTINUE : EXIT;
});
const links: NoteLink[] = [];
const linkDefinitions: NoteLinkDefinition[] = [];
let frontmatter: any = {};
let start: Point = { line: 1, column: 1, offset: 0 }; // start position of the note
visit(tree, node => {
const yamlPlugin: ParserPlugin = {
visit: (node, note) => {
if (node.type === 'yaml') {
frontmatter = parseYAML(node.value as string) ?? {}; // parseYAML returns null if the frontmatter is empty
note.properties = {
...note.properties,
...(parseYAML(node.value as string) ?? {}),
};
// Give precendence to the title from the frontmatter if it exists
note.title = note.properties.title ?? note.title;
// Update the start position of the note by exluding the metadata
start = {
note.source.contentStart = {
line: node.position!.end.line! + 1,
column: 1,
offset: node.position!.end.offset! + 1,
};
}
},
};
const titlePlugin: ParserPlugin = {
visit: (node, note) => {
if (note.title == null && node.type === 'heading' && node.depth === 1) {
note.title =
((node as Parent)!.children[0].value as string) || note.title;
}
},
};
const wikilinkPlugin: ParserPlugin = {
visit: (node, note) => {
if (node.type === 'wikiLink') {
links.push({
note.links.push({
type: 'wikilink',
slug: node.value as string,
position: node.position!,
});
}
},
};
const definitionsPlugin: ParserPlugin = {
visit: (node, note) => {
if (node.type === 'definition') {
linkDefinitions.push({
note.definitions.push({
label: node.label as string,
url: node.url as string,
title: node.title as string,
position: node.position,
});
}
});
},
onDidVisitTree: (tree, note) => {
note.definitions = getFoamDefinitions(note.definitions, note.source.end);
},
};
// Give precendence to the title from the frontmatter if it exists
title = frontmatter.title ?? title;
export function createMarkdownParser(extraPlugins: ParserPlugin[]): NoteParser {
const parser = unified()
.use(markdownParse, { gfm: true })
.use(frontmatterPlugin, ['yaml'])
.use(wikiLinkPlugin);
const end = tree.position!.end;
const definitions = getFoamDefinitions(linkDefinitions, end);
const plugins = [
yamlPlugin,
titlePlugin,
wikilinkPlugin,
definitionsPlugin,
...extraPlugins,
];
plugins.forEach(plugin => plugin.onDidInitializeParser?.(parser));
return {
properties: frontmatter,
slug: uriToSlug(uri),
title: title,
links: links,
definitions: definitions,
source: {
uri: uri,
text: markdown,
contentStart: start,
end: end,
eol: eol,
parse: (uri: string, markdown: string, eol: string): Note => {
markdown = plugins.reduce((acc, plugin) => {
return plugin.onWillParseMarkdown?.(acc) || acc;
}, markdown);
const tree = parser.parse(markdown);
var note: Note = {
slug: uriToSlug(uri),
properties: {},
title: null,
links: [],
definitions: [],
source: {
uri: uri,
text: markdown,
contentStart: tree.position!.start,
end: tree.position!.end,
eol: eol,
},
};
plugins.forEach(plugin => plugin.onWillVisitTree?.(tree, note));
visit(tree, node => {
for (let i = 0, len = plugins.length; i < len; i++) {
plugins[i].visit?.(node, note);
}
});
plugins.forEach(plugin => plugin.onDidVisitTree?.(tree, note));
return note;
},
};
}
@@ -128,7 +155,7 @@ export function stringifyMarkdownLinkReferenceDefinition(
return text;
}
export function createMarkdownReferences(
graph: NoteGraph,
graph: NoteGraphAPI,
noteId: ID,
includeExtension: boolean
): NoteLinkDefinition[] {

75
packages/foam-core/src/note-graph.ts
View File

@@ -1,43 +1,8 @@
import { Graph } from 'graphlib';
import { EventEmitter } from 'events';
import { Position, Point, URI, ID } from './types';
import { URI, ID, Note, NoteLink } from './types';
import { hashURI, computeRelativeURI } from './utils';
export interface NoteSource {
uri: URI;
text: string;
contentStart: Point;
end: Point;
eol: string;
}
export interface WikiLink {
type: 'wikilink';
slug: string;
position: Position;
}
// at the moment we only model wikilink
export type NoteLink = WikiLink;
export interface NoteLinkDefinition {
label: string;
url: string;
title?: string;
position?: Position;
}
export interface Note {
title: string | null;
slug: string; // note: this slug is not necessarily unique
properties: object;
// sections: NoteSection[]
// tags: NoteTag[]
links: NoteLink[];
definitions: NoteLinkDefinition[];
source: NoteSource;
}
export type GraphNote = Note & {
id: ID;
};
@@ -52,7 +17,27 @@ export type NoteGraphEventHandler = (e: { note: GraphNote }) => void;
export type NotesQuery = { slug: string } | { title: string };
export class NoteGraph {
export interface NoteGraphAPI {
setNote(note: Note): GraphNote;
getNotes(query?: NotesQuery): GraphNote[];
getNote(noteId: ID): GraphNote | null;
getNoteByURI(uri: URI): GraphNote | null;
getAllLinks(noteId: ID): GraphConnection[];
getForwardLinks(noteId: ID): GraphConnection[];
getBacklinks(noteId: ID): GraphConnection[];
unstable_onNoteAdded(callback: NoteGraphEventHandler): void;
unstable_onNoteUpdated(callback: NoteGraphEventHandler): void;
unstable_removeEventListener(callback: NoteGraphEventHandler): void;
}
export type Middleware = (next: NoteGraphAPI) => Partial<NoteGraphAPI>;
export const createGraph = (middlewares: Middleware[]): NoteGraphAPI => {
const graph: NoteGraphAPI = new NoteGraph();
return middlewares.reduce((acc, m) => backfill(acc, m), graph);
};
export class NoteGraph implements NoteGraphAPI {
private graph: Graph;
private events: EventEmitter;
private createIdFromURI: (uri: URI) => ID;
@@ -149,3 +134,19 @@ export class NoteGraph {
this.events.removeAllListeners();
}
}
const backfill = (next: NoteGraphAPI, middleware: Middleware): NoteGraphAPI => {
const m = middleware(next);
return {
setNote: m.setNote || next.setNote,
getNotes: m.getNotes || next.getNotes,
getNote: m.getNote || next.getNote,
getNoteByURI: m.getNoteByURI || next.getNoteByURI,
getAllLinks: m.getAllLinks || next.getAllLinks,
getForwardLinks: m.getForwardLinks || next.getForwardLinks,
getBacklinks: m.getBacklinks || next.getBacklinks,
unstable_onNoteAdded: next.unstable_onNoteAdded.bind(next),
unstable_onNoteUpdated: next.unstable_onNoteUpdated.bind(next),
unstable_removeEventListener: next.unstable_removeEventListener.bind(next),
};
};

79
packages/foam-core/src/plugins/index.ts Normal file
View File

@@ -0,0 +1,79 @@
import * as fs from 'fs';
import path from 'path';
import { Node } from 'unist';
import { isNotNull } from '../utils';
import { Middleware } from '../note-graph';
import { Note } from '../types';
import unified from 'unified';
import { FoamConfig } from '../config';
export interface FoamPlugin {
name: string;
description?: string;
graphMiddleware?: Middleware;
parser?: ParserPlugin;
}
export interface ParserPlugin {
visit?: (node: Node, note: Note) => void;
onDidInitializeParser?: (parser: unified.Processor) => void;
onWillParseMarkdown?: (markdown: string) => string;
onWillVisitTree?: (tree: Node, note: Note) => void;
onDidVisitTree?: (tree: Node, note: Note) => void;
}
export interface PluginConfig {
enabled?: boolean;
pluginFolders?: string[];
}
export const SETTINGS_PATH = 'experimental.localPlugins';
export async function loadPlugins(config: FoamConfig): Promise<FoamPlugin[]> {
const pluginConfig = config.get<PluginConfig>(SETTINGS_PATH, {});
const isFeatureEnabled = pluginConfig.enabled ?? false;
if (!isFeatureEnabled) {
return [];
}
const pluginDirs: string[] =
pluginConfig.pluginFolders ?? findPluginDirs(config.workspaceFolders);
const plugins = await Promise.all(
pluginDirs
.filter(dir => fs.statSync(dir).isDirectory)
.map(async dir => {
try {
const pluginFile = path.join(dir, 'index.js');
fs.accessSync(pluginFile);
const plugin = validate(await import(pluginFile));
return plugin;
} catch (e) {
console.error(`Error while loading plugin at [${dir}] - skipping`, e);
return null;
}
})
);
return plugins.filter(isNotNull);
}
function findPluginDirs(workspaceFolders: string[]) {
return workspaceFolders
.map(root => path.join(root, '.foam', 'plugins'))
.reduce((acc, pluginDir) => {
try {
const content = fs
.readdirSync(pluginDir)
.map(dir => path.join(pluginDir, dir));
return [...acc, ...content.filter(c => fs.statSync(c).isDirectory())];
} catch {
return acc;
}
}, [] as string[]);
}
function validate(plugin: any): FoamPlugin {
if (!plugin.name) {
throw new Error('Plugin must export `name` string property');
}
return plugin;
}

4
packages/foam-core/src/types.d.ts vendored
View File

@@ -1,4 +0,0 @@
export { Position, Point } from 'unist';
export type URI = string;
export type ID = string;

46
packages/foam-core/src/types.ts Normal file
View File

@@ -0,0 +1,46 @@
// this file can't simply be .d.ts because the TS compiler wouldn't copy it to the dist directory
// see https://stackoverflow.com/questions/56018167/typescript-does-not-copy-d-ts-files-to-build
import { Position, Point } from 'unist';
export { Position, Point };
export type URI = string;
export type ID = string;
export interface NoteSource {
uri: URI;
text: string;
contentStart: Point;
end: Point;
eol: string;
}
export interface WikiLink {
type: 'wikilink';
slug: string;
position: Position;
}
// at the moment we only model wikilink
export type NoteLink = WikiLink;
export interface NoteLinkDefinition {
label: string;
url: string;
title?: string;
position?: Position;
}
export interface Note {
title: string | null;
slug: string; // note: this slug is not necessarily unique
properties: any;
// sections: NoteSection[]
// tags: NoteTag[]
links: NoteLink[];
definitions: NoteLinkDefinition[];
source: NoteSource;
}
export interface NoteParser {
parse: (uri: string, text: string, eol: string) => Note;
}

6
packages/foam-core/src/utils.ts
View File

@@ -2,7 +2,11 @@ import path from 'path';
import crypto from 'crypto';
import { titleCase } from 'title-case';
import GithubSlugger from 'github-slugger';
import { URI, ID } from 'types';
import { URI, ID } from './types';
export function isNotNull<T>(value: T | null): value is T {
return value != null;
}
export const hash = (text: string) =>
crypto

38
packages/foam-core/test/config.test.ts Normal file
View File

@@ -0,0 +1,38 @@
import * as path from 'path';
import { createConfigFromFolders } from '../src/config';
const testFolder = path.join(__dirname, 'test-config');
describe('Foam configuration', () => {
it('can read settings from config.json', () => {
const config = createConfigFromFolders([path.join(testFolder, 'folder1')]);
expect(config.get('feature1.setting1.value')).toBeTruthy();
expect(config.get('feature2.value')).toEqual(12);
const section = config.get<{ value: boolean }>('feature1.setting1');
expect(section!.value).toBeTruthy();
});
it('can merge settings from multiple foam folders', () => {
const config = createConfigFromFolders([
path.join(testFolder, 'folder1'),
path.join(testFolder, 'folder2'),
]);
// override value
expect(config.get('feature1.setting1.value')).toBe(false);
// this was not overridden
expect(config.get('feature1.setting1.extraValue')).toEqual('go foam');
// new value from second config file
expect(config.get('feature1.setting1.value2')).toBe('hello');
// this whole section doesn't exist in second file
expect(config.get('feature2.value')).toEqual(12);
});
it('cannot activate local plugins from workspace config', () => {
const config = createConfigFromFolders([
path.join(testFolder, 'enable-plugins'),
]);
expect(config.get('experimental.localPlugins.enabled')).toBeUndefined();
});
});

26
packages/foam-core/test/core.test.ts
View File

@@ -1,4 +1,5 @@
import { NoteGraph, NoteLinkDefinition, Note } from '../src/note-graph';
import { NoteGraph, createGraph } from '../src/note-graph';
import { NoteLinkDefinition, Note } from '../src/types';
import { uriToSlug } from '../src/utils';
const position = {
@@ -10,7 +11,7 @@ const documentStart = position.start;
const documentEnd = position.end;
const eol = '\n';
const createTestNote = (params: {
export const createTestNote = (params: {
uri: string;
title?: string;
definitions?: NoteLinkDefinition[];
@@ -168,7 +169,7 @@ describe('Graph querying', () => {
it('finds a note by title', () => {
const graph = new NoteGraph();
const note = graph.setNote(
graph.setNote(
createTestNote({ uri: '/dir1/page-a.md', title: 'My Title' })
);
expect(graph.getNotes({ title: 'My Title' }).length).toEqual(1);
@@ -185,3 +186,22 @@ describe('Graph querying', () => {
expect(graph.getNotes({ title: 'My Title' }).length).toEqual(2);
});
});
describe('graph middleware', () => {
it('can intercept calls to the graph', async () => {
const graph = createGraph([
next => ({
setNote: note => {
note.properties = {
injected: true,
};
return next.setNote(note);
},
}),
]);
const note = createTestNote({ uri: '/dir1/page-a.md', title: 'My Title' });
expect(note.properties['injected']).toBeUndefined();
const res = graph.setNote(note);
expect(res.properties['injected']).toBeTruthy();
});
});

12
packages/foam-core/test/janitor/generateHeadings.test.ts
View File

@@ -1,13 +1,17 @@
import * as path from 'path';
import { NoteGraph } from '../../src/note-graph';
import { NoteGraphAPI } from '../../src/note-graph';
import { generateHeading } from '../../src/janitor';
import { initializeNoteGraph } from '../../src/initialize-note-graph';
import { bootstrap } from '../../src/bootstrap';
import { createConfigFromFolders } from '../../src/config';
describe('generateHeadings', () => {
let _graph: NoteGraph;
let _graph: NoteGraphAPI;
beforeAll(async () => {
_graph = await initializeNoteGraph(path.join(__dirname, '../__scaffold__'));
const foam = await bootstrap(
createConfigFromFolders([path.join(__dirname, '../__scaffold__')])
);
_graph = foam.notes;
});
it('should add heading to a file that does not have them', () => {

11
packages/foam-core/test/janitor/generateLinkReferences.test.ts
View File

@@ -1,13 +1,16 @@
import * as path from 'path';
import { NoteGraph } from '../../src/note-graph';
import { NoteGraphAPI } from '../../src/note-graph';
import { generateLinkReferences } from '../../src/janitor';
import { initializeNoteGraph } from '../../src/initialize-note-graph';
import { bootstrap } from '../../src/bootstrap';
import { createConfigFromFolders } from '../../src/config';
describe('generateLinkReferences', () => {
let _graph: NoteGraph;
let _graph: NoteGraphAPI;
beforeAll(async () => {
_graph = await initializeNoteGraph(path.join(__dirname, '../__scaffold__'));
_graph = await bootstrap(
createConfigFromFolders([path.join(__dirname, '../__scaffold__')])
).then(foam => foam.notes);
});
it('initialised test graph correctly', () => {

37
packages/foam-core/test/markdown-provider.test.ts
View File

@@ -1,8 +1,9 @@
import {
createNoteFromMarkdown,
createMarkdownParser,
createMarkdownReferences,
} from '../src/markdown-provider';
import { NoteGraph } from '../src/note-graph';
import { ParserPlugin } from '../src/plugins';
const pageA = `
# Page A
@@ -41,6 +42,8 @@ const pageF = `
# Empty Frontmatter
`;
const createNoteFromMarkdown = createMarkdownParser([]).parse;
describe('Markdown loader', () => {
it('Converts markdown to notes', () => {
const graph = new NoteGraph();
@@ -179,3 +182,35 @@ describe('wikilinks definitions', () => {
]);
});
});
describe('parser plugins', () => {
const testPlugin: ParserPlugin = {
visit: (node, note) => {
if (node.type === 'heading') {
note.properties.hasHeading = true;
}
},
};
const parser = createMarkdownParser([testPlugin]);
it('can augment the parsing of the file', async () => {
const note1 = parser.parse(
'/path/to/a',
`
This is a test note without headings.
But with some content.
`,
'\n'
);
expect(note1.properties.hasHeading).toBeUndefined();
const note2 = parser.parse(
'/path/to/a',
`
# This is a note with header
and some content`,
'\n'
);
expect(note2.properties.hasHeading).toBeTruthy();
});
});

71
packages/foam-core/test/plugin.test.ts Normal file
View File

@@ -0,0 +1,71 @@
import path from 'path';
import { loadPlugins } from '../src/plugins';
import { createMarkdownParser } from '../src/markdown-provider';
import { createGraph } from '../src/note-graph';
import { createTestNote } from './core.test';
import { FoamConfig, createConfigFromObject } from '../src/config';
const config: FoamConfig = createConfigFromObject([], {
experimental: {
localPlugins: {
enabled: true,
pluginFolders: [path.join(__dirname, 'test-plugin')],
},
},
});
describe('Foam plugins', () => {
it('will not load if feature is not explicitly enabled', async () => {
let plugins = await loadPlugins(createConfigFromObject([], {}));
expect(plugins.length).toEqual(0);
plugins = await loadPlugins(
createConfigFromObject([], {
experimental: {
localPlugins: {},
},
})
);
expect(plugins.length).toEqual(0);
plugins = await loadPlugins(
createConfigFromObject([], {
experimental: {
localPlugins: {
enabled: false,
},
},
})
);
expect(plugins.length).toEqual(0);
});
it('can load', async () => {
const plugins = await loadPlugins(config);
expect(plugins.length).toEqual(1);
expect(plugins[0].name).toEqual('Test Plugin');
});
it('supports graph middleware', async () => {
const plugins = await loadPlugins(config);
const middleware = plugins[0].graphMiddleware;
expect(middleware).not.toBeUndefined();
const graph = createGraph([middleware!]);
const note = graph.setNote(createTestNote({ uri: '/path/to/note.md' }));
expect(note.properties['injectedByMiddleware']).toBeTruthy();
});
it('supports parser extension', async () => {
const plugins = await loadPlugins(config);
const parserPlugin = plugins[0].parser;
expect(parserPlugin).not.toBeUndefined();
const parser = createMarkdownParser([parserPlugin!]);
const note = parser.parse(
'/path/to/a',
`
# This is a note with header
and some content`,
'\n'
);
expect(note.properties.hasHeading).toBeTruthy();
});
});

7
packages/foam-core/test/test-config/enable-plugins/config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"experimental": {
"localPlugins": {
"enabled": true
}
}
}

11
packages/foam-core/test/test-config/folder1/config.json Normal file
View File

@@ -0,0 +1,11 @@
{
"feature1": {
"setting1": {
"value": true,
"extraValue": "go foam"
}
},
"feature2": {
"value": 12
}
}

8
packages/foam-core/test/test-config/folder2/config.json Normal file
View File

@@ -0,0 +1,8 @@
{
"feature1": {
"setting1": {
"value": false,
"value2": "hello"
}
}
}

20
packages/foam-core/test/test-plugin/index.js Normal file
View File

@@ -0,0 +1,20 @@
const middleware = next => ({
setNote: note => {
note.properties['injectedByMiddleware'] = true;
return next.setNote(note);
},
});
const parser = {
visit: (node, note) => {
if (node.type === 'heading') {
note.properties.hasHeading = true;
}
},
};
module.exports = {
name: 'Test Plugin',
graphMiddleware: middleware,
parser: parser,
};

4
packages/foam-core/tsconfig.build.json
View File

@@ -1,6 +1,6 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"module": "es6"
"module": "ESNext"
}
}
}

39
packages/foam-vscode/src/extension.ts
View File

@@ -4,6 +4,7 @@
*/
"use strict";
import path from "path";
import * as fs from "fs";
import {
workspace,
@@ -15,10 +16,10 @@ import {
} from "vscode";
import {
createNoteFromMarkdown,
createFoam,
bootstrap as foamBootstrap,
FoamConfig,
Foam
Foam,
createConfigFromFolders
} from "foam-core";
import { features } from "./features";
@@ -26,10 +27,14 @@ import { features } from "./features";
let workspaceWatcher: FileSystemWatcher | null = null;
export function activate(context: ExtensionContext) {
const foamPromise = bootstrap(getConfig());
features.forEach(f => {
f.activate(context, foamPromise);
});
try {
const foamPromise = bootstrap();
features.forEach(f => {
f.activate(context, foamPromise);
});
} catch (e) {
console.log("An error occurred while bootstrapping Foam", e);
}
}
export function deactivate() {
@@ -49,16 +54,17 @@ async function registerFile(foam: Foam, localUri: Uri) {
// create note
const eol =
window.activeTextEditor?.document?.eol === EndOfLine.CRLF ? "\r\n" : "\n";
const note = createNoteFromMarkdown(path, markdown, eol);
const note = foam.parse(path, markdown, eol);
// add to graph
foam.notes.setNote(note);
return note;
}
const bootstrap = async (config: FoamConfig) => {
const bootstrap = async () => {
const files = await workspace.findFiles("**/*");
const foam = createFoam(config);
const config: FoamConfig = getConfig();
const foam = await foamBootstrap(config);
const addFile = (uri: Uri) => registerFile(foam, uri);
await Promise.all(files.filter(isLocalMarkdownFile).map(addFile));
@@ -69,7 +75,7 @@ const bootstrap = async (config: FoamConfig) => {
true,
true
);
workspaceWatcher.onDidCreate(uri => {
if (isLocalMarkdownFile(uri)) {
addFile(uri).then(() => {
@@ -81,6 +87,13 @@ const bootstrap = async (config: FoamConfig) => {
return foam;
};
export const getConfig = () => {
return {};
export const getConfig = (): FoamConfig => {
const workspaceFolders = workspace
.workspaceFolders!.filter(dir => {
const foamPath = path.join(dir.uri.fsPath, ".foam");
return fs.existsSync(foamPath) && fs.statSync(foamPath).isDirectory();
})
.map(dir => dir.uri.fsPath);
return createConfigFromFolders(workspaceFolders);
};

10
packages/foam-vscode/src/features/janitor.ts
View File

@@ -6,7 +6,7 @@ import {
Range,
ProgressLocation
} from "vscode";
import fs = require("fs");
import * as fs from "fs";
import { FoamFeature } from "../types";
import {
applyTextEdit,
@@ -56,9 +56,11 @@ async function janitor(foam: Foam) {
);
}
} catch (e) {
window.showErrorMessage(`Foam Janitor attempted to clean your workspace but ran into an error. Please check that we didn't break anything before committing any changes to version control, and pass the following error message to the Foam team on GitHub issues:
window.showErrorMessage(
`Foam Janitor attempted to clean your workspace but ran into an error. Please check that we didn't break anything before committing any changes to version control, and pass the following error message to the Foam team on GitHub issues:
${e.message}
${e.stack}`);
${e.stack}`
);
}
}
@@ -126,7 +128,7 @@ async function runJanitor(foam: Foam) {
const editor = await window.showTextDocument(doc);
const note = dirtyNotes.find(
n => n.source.uri === editor.document.fileName
);
)!;
// Get edits
const heading = generateHeading(note);

6
packages/foam-vscode/src/features/open-daily-note.ts
View File

@@ -5,11 +5,11 @@ import {
WorkspaceConfiguration,
ExtensionContext,
commands,
Selection,
Selection
} from "vscode";
import { dirname, join } from "path";
import dateFormat = require("dateformat");
import fs = require("fs");
import dateFormat from "dateformat";
import * as fs from "fs";
import { FoamFeature } from "../types";
import { docConfig } from '../utils';

24
packages/foam-vscode/src/features/wikilink-reference-generation.ts
View File

@@ -16,21 +16,18 @@ import {
import {
createMarkdownReferences,
stringifyMarkdownLinkReferenceDefinition,
createNoteFromMarkdown,
NoteGraph,
NoteGraphAPI,
Foam,
LINK_REFERENCE_DEFINITION_HEADER,
LINK_REFERENCE_DEFINITION_FOOTER
} from "foam-core";
import { basename } from "path";
import {
hasEmptyTrailing,
docConfig,
loadDocConfig,
isMdEditor,
mdDocSelector,
getText,
dropExtension
getText
} from "../utils";
import { FoamFeature } from "../types";
import { includeExtensions } from "../settings";
@@ -72,11 +69,11 @@ const feature: FoamFeature = {
function updateDocumentInNoteGraph(foam: Foam, document: TextDocument) {
foam.notes.setNote(
createNoteFromMarkdown(document.fileName, document.getText(), docConfig.eol)
foam.parse(document.fileName, document.getText(), docConfig.eol)
);
}
async function createReferenceList(foam: NoteGraph) {
async function createReferenceList(foam: NoteGraphAPI) {
let editor = window.activeTextEditor;
if (!editor || !isMdEditor(editor)) {
@@ -100,7 +97,7 @@ async function createReferenceList(foam: NoteGraph) {
}
}
async function updateReferenceList(foam: NoteGraph) {
async function updateReferenceList(foam: NoteGraphAPI) {
const editor = window.activeTextEditor;
if (!editor || !isMdEditor(editor)) {
@@ -128,7 +125,10 @@ async function updateReferenceList(foam: NoteGraph) {
}
}
function generateReferenceList(foam: NoteGraph, doc: TextDocument): string[] {
function generateReferenceList(
foam: NoteGraphAPI,
doc: TextDocument
): string[] {
const filePath = doc.fileName;
const note = foam.getNoteByURI(filePath);
@@ -163,7 +163,7 @@ function generateReferenceList(foam: NoteGraph, doc: TextDocument): string[] {
* Find the range of existing reference list
* @param doc
*/
function detectReferenceListRange(doc: TextDocument): Range {
function detectReferenceListRange(doc: TextDocument): Range | null {
const fullText = doc.getText();
const headerIndex = fullText.indexOf(LINK_REFERENCE_DEFINITION_HEADER);
@@ -190,9 +190,9 @@ function detectReferenceListRange(doc: TextDocument): Range {
}
class WikilinkReferenceCodeLensProvider implements CodeLensProvider {
private foam: NoteGraph;
private foam: NoteGraphAPI;
constructor(foam: NoteGraph) {
constructor(foam: NoteGraphAPI) {
this.foam = foam;
}

6
prettier.config.js Normal file
View File

@@ -0,0 +1,6 @@
module.exports = {
trailingComma: 'es5',
tabWidth: 2,
semi: true,
singleQuote: true,
};