Commit Graph

142 Commits

Author SHA1 Message Date
Kevin Lewis
e67ed8acf7 Add docs for themes extension type (#20561)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-11-28 19:58:51 +00:00
Pascal Jufer
59d40451ba Fix "Format" check in CI (#20482) 2023-11-20 16:22:08 +01:00
Rijk van Zanten
258621981e All incoming code is through PRs
See the irony of this being a direct commit? hihi
2023-11-08 16:12:50 -05:00
Rijk van Zanten
421e3e8f9f Only run CI against files that changed (#20365)
* Only run eslint against the files that have changed

* Don't build for eslint?

* Only run unit tests against changed files

* Fix unit test runner

* Run CodeQL daily, not on PR

* Checkout base ref for vitest to be able to check

* Inching closer

* Trial and error is my favorite

* With origin?

* Is this still needed?

* Does this work?

* It needs the whole fetch depth

Without it, the current HEAD doesn't have a shared base which causes a "no merge base" error

* Add comment to ensure we don't remove this later
2023-11-08 15:57:46 -05:00
Rijk van Zanten
6509e06a54 Update assign-next-release-milestone.yml 2023-11-06 10:02:14 -05:00
Rijk van Zanten
46d5a8ddde Temporarily remove the on-PR blackbox tests (#20208)
* Temporarily remove the on-PR blackbox tests

* Fix formatting, rename file
2023-10-26 16:44:56 -04:00
Pascal Jufer
f29b5a167a Enable npm provenance (#20182) 2023-10-25 13:13:46 -04:00
Pascal Jufer
61b7432950 Use fixed tag for prereleases (#20180) 2023-10-25 11:21:07 +02:00
Pascal Jufer
4fa20578f5 Optimize MSSQL test setup (#19979) 2023-10-10 13:27:36 +02:00
Rijk van Zanten
8dc68730ff Update name of milestone 2023-10-05 11:59:35 -04:00
Pascal Jufer
1ec1f98431 Bye-bye, Jest (#19886)
* Bye-bye, Jest

* Clean-up non-null assertions for PORT

* Test all vendors

* Consistent file names

* Try with MSSQL 2022

* Revert "Try with MSSQL 2022"

This reverts commit da807c6ec8.

* Try to run on Windows for better MSSQL perf

* Worth a try (hopefully runs MSSQL faster this way)

* Start docker services via bash

* Quick attempt with local installation of MSSQL

* Test with MSSQL 2019

* Back to ubuntu-latest for other tests

* Add typecheck cmd

* Revert "Test all vendors"

This reverts commit d6fbdd76c9.

* Use TS for Vitest config file

* Simplify vendors typecasting

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-10-04 14:53:36 -04:00
Brainslug
0deadb9062 Improve stale github action (#19551)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-08-29 18:23:26 +02:00
Pascal Jufer
96bb544c56 Support for prereleases in release workflow (#19510)
* Make release workflow compatible for prereleases

* Support prereleases in release-notes-generator

* Update readme

* Create metal-dodos-return.md

* Update .changeset/metal-dodos-return.md

* Override version of main package

* Rework & enable usage with changesets's prerelease mode

* Fix conditional DIRECTUS_PRERELEASE_ID output

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add hint to beforeExit event

* Add basic test for getInfo

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-08-25 16:48:50 +02:00
ian
66a5e60fbe Enable more checks for branch (#19514) 2023-08-23 20:24:29 +02:00
Pascal Jufer
921ac972d0 Enable checks for next branch (#19511) 2023-08-23 13:22:36 -04:00
Connor
92621429ee Linting and Formatting Upgrade (#18892)
* add docs to eslint

* update prettier ignore

* fix vitepress linting

* eslint ignore fixes

* prettier run

* update prettier ignore

* fix formatting

* enable linting of markdown files

* revert format command change

* fix irregular whitespace

* update dictionary

* (Changelog) Create four-boxes-shake.md

* Rework ESLint / Prettier setup

- Disable js/ts/vue files for Prettier to ensure linting/formatting is
  only happening via ESLint
- Rework formatting of code blocks in md files
  - Disable formatting of code blocks in md files under '/docs' by Prettier
  - Instead use "eslint-plugin-markdown" to format & __lint__ js*/ts*/vue such code blocks
  - Replace unmaintained "eslint-plugin-md" plugin by official "eslint-plugin-markdown" plugin
  - I'll check whether we can use this to format other code blocks
    (json, html, ...) as well
- Restructure, clean-up and apply some fixes to the ESLint config
  (Note: Not ready for flat config yet since not supported by
  vscode-eslint)
- Enable cache for ESLint / Prettier in scripts
- Clean-up ignore file
  - Explicit folder declaration (.../)
  - Don't ignore all 'extensions' folders in ESLint (only
    '/api/extensions/')
  - Enable formatting in '/.github' folder

* Fix all formatting issues with Prettier

* Update md files under /docs/.typedocs

* Fix lint issues in vue/js files

* ESLint / Prettier config revision v2

Enable Prettier for md code blocks, but only as warnings since it can
get into the way with Vitepress md extensions like '[!code ...]'
comments

* Remove prettier-ignore comments

* Make spellchecker happy

* Remove changeset

* Revert lint setup for code blocks

There are many cases in the docs where linting / formatting of code
blocks doesn't make
sense:
- Code block is only an excerpt - linter fails
- Code block contains special comments (e.g. markdown extensions) which
  needs to remain at the same place - formatting would break it
- ...

* Apply lint issues / formatting from temp lint setup

* Run formatter

* Fix merge failure

* Simplify & modernize ESLint / Prettier setup

No longer run Prettier via ESLint. Nowadays, this is the recommended
setup. There's no real need to run it this way, it's just an additional
layer.

Add VS Code settings to make the work with the new setup easier.

* Remove unused eslint disable directives

* Make editorconfig more useful

* Fix formatting issues reported by editorconfig

* Format files with Prettier

* Enable formatting of source translations file

* Format source translations file

* Remove unnecessary console error

* Remove unnecessary line

* Only ignore md files under .changeset

* Add CI reporter for Prettier

* Fail job on wrongly formatted files

* Fix format

* Test Prettier action on changed/added file

* Use simple CI format check for now & no cache

* Revert "Test Prettier action on changed/added file"

This reverts commit 4f7d8826ad.

* Introduce code blocks check for docs

* Fix code block issues

* Ignore auto-generated packages dir

* Fix comment position

* Also lint `/app/.storybook`

* Reformat modified files

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-06-29 11:54:01 +02:00
Pascal Jufer
82aff75443 Fix "Assign Next Release Milestone" workflow for PR's from forks (#19009)
* Make "Assign Next Release Milestone" workflow work on external PR's

* Reduce external factors, replacing actions by gh api calls
2023-06-27 11:49:41 -04:00
ian
4249bc7734 Fix existing milestone error (#18828) 2023-06-06 15:46:46 -04:00
ian
79a915301e Automatically assign next release milestone (#18827)
* Automatically assign next release milestone

* Automatically create milestone

* Rename workflow job

* Update to latest version
2023-06-06 15:31:19 -04:00
Rijk van Zanten
d6c0d31b39 Add auto-closer for stale issues (#18780)
* Add auto-closer for stale issues

* Tweak name

* Tweak language
2023-05-30 16:23:43 -04:00
ian
101ec6649d Specify oracledb version for blackbox tests (#18720) 2023-05-25 17:02:15 +08:00
Rijk van Zanten
8401cd5a8e Delete linear-auto.yml 2023-05-24 14:08:35 -04:00
Rijk van Zanten
3603b34c9a Delete slash-commands.yaml 2023-05-24 14:08:23 -04:00
Rijk van Zanten
86527cde84 Delete linear-dispatch.yaml 2023-05-24 14:08:13 -04:00
ian
eb82110b7e Update gh-action-linear to fix workflow failure (#18699) 2023-05-23 12:19:26 +02:00
Pascal Jufer
2e647d80e6 Patch Tuesday 🐸 (#18643)
* Update `directus-monorepo` deps

* Update `@directus/api` deps

* Update `@directus/app` deps

* Update `@directus/composables` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* Update `@directus/data` deps

* Update `@directus/data-driver-postgres` deps

* Update `@directus/extensions-sdk` deps

* Update `@directus/pressure` deps

* Update `@directus/random` deps

* Update `@directus/release-notes-generator` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* Update `docs` deps

* Update `tests-blackbox` deps

* Bump pnpm to 8.5.1

* Bump build-push-action in release workflow to v4

* Assign new spy instances before each test

See https://github.com/vitest-dev/vitest/pull/3386

* Create clean-needles-nail.md

* Remove leftover 'beforeAll' imports

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-19 10:48:21 -04:00
Pascal Jufer
cfe5227798 Clarify SMTP configs (client / server) (#18152) 2023-04-12 12:36:12 +00:00
Pascal Jufer
15b91dee34 Blackbox tests restructuring (#18122)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-11 18:28:37 +02:00
Pascal Jufer
f7c2f14181 Check if there are any missing contributors (#18099) 2023-04-07 07:58:47 -04:00
ian
6bf5327614 Remove double bracket from CLA message (#18086) 2023-04-06 16:58:39 +02:00
Rijk van Zanten
535edd7d7e Update cla.yml 2023-04-05 17:42:21 -04:00
Rijk van Zanten
8a654433e1 Update cla.yml 2023-04-05 14:01:27 -04:00
Rijk van Zanten
a9d45fc4a5 Add CLA + automation (#18071) 2023-04-05 13:59:50 -04:00
Pascal Jufer
8c0ac96358 Add workflow for docs spellchecker (#18063)
* Ignore node_modules for docs spellchecking

* Add workflows for docs spellchecking

* Update command to new structure

* Skip core checks on pure docs changes

* Don't build packages for spellcheck workflow

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-05 09:54:11 -04:00
rijkvanzanten
8ea179248e No emoji please 2023-04-04 17:49:08 -04:00
Pascal Jufer
5b256f64c6 Use pnpm 8.1.0, define version via packageManager field, clean-up other version definitions (#17993)
* Use `packageManager` field to define pnpm version in one place

* Update to pnpm 8.1.0 (see also #17990)

* Enforce pnpm >= 8 via `engines`

* Clean-up `engines` & remove `volta`

* Add dedup workaround, consolidate test env prepare steps

* No checkout, no action :-)

* Bump minimal pnpm version to 8.1.0

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-04 15:20:25 -04:00
Pascal Jufer
5d6468b477 Fix publishing of packages to NPM in release workflow (#17900)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-03-23 14:02:55 -04:00
Rijk van Zanten
59f965cc8d Update release.yml 2023-03-23 13:40:26 -04:00
Rijk van Zanten
0d8b3a52c7 Simplify / improve release flow (#17858)
* Remove git-hook

This is already running in GH Actions as well

* Start new Dockerfile

* Remove unused files

* Close, but not quite

* Line break between environment specific commands / operational ones

* Cleanup dockerfile

* Update github release automation

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Improve based on review comments

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-03-23 13:20:14 -04:00
ian
cce33436d1 Rework schema caching to use memory (#17763)
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Jan Arends <jaads@users.noreply.github.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-03-23 12:52:06 +01:00
Rijk van Zanten
9526b4e5b2 Improve S3 asset read performance (#17835)
* Create new s3 client for each read

* Temp disable ts while debugging

* Add concurrency test

* Add minio to other tests

* Reduce unavailable count

* Trigger blackbox tests whenever packages are updated

* Prevent minio-mc from exiting

* Decrease requests and increase test timeout

* Spam more requests over longer period

* Increase request timeout

* Run autocannon directly with larger image

* Fix tests

* Lock version

* My favorite file

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-03-20 18:16:30 -04:00
Pascal Jufer
4b2edb05c3 Update gh-action-linear to v0.3.1 (#17693) 2023-03-03 12:17:14 +01:00
Pascal Jufer
d5c39fd087 Use dedicated PAT for slash command action (#17671) 2023-03-01 17:06:35 -05:00
Pascal Jufer
59dcd30e6e Use npm auth set-up from 'setup-node' action (#17656) 2023-03-01 08:07:16 +01:00
Rijk van Zanten
b98b67b8e2 Create slash-commands.yaml 2023-02-28 10:55:59 -05:00
Rijk van Zanten
451e48bd93 Create linear-dispatch.yaml 2023-02-28 10:55:42 -05:00
Rijk van Zanten
c4b185942a Update and rename linear.yml to linear-auto.yml 2023-02-28 10:54:52 -05:00
Rijk van Zanten
d2fae39372 Delete add-to-project.yml 2022-12-21 15:09:43 -05:00
ian
f1a8e0446f Fix duplicated results and functions in nested filters (#14798)
* Speed query up by reusing existing aliases which reduces table joins

* Use subquery in top level m2o to remove duplicates

* Fix linting

* Apply distinct on primary key field in subqueries

* Use distinct instead as there are only primary keys

* Apply subquery on top level

* Try remove sub sub query

* Test if working for all vendors

* Add support for _none and _some

* Use subquery only when field depth > 1

* Add tests

* Use original table names for columns with functions (#14690)

* Use original table names for columns with functions

* Extract filter function path parsing as shared util

* Fix filter function path when adding node

* Pass the originalCollectionName into filter functions

* Update unit test

* Replace functions within deep GraphQL

* Fix invalid operator error for _none and _some

* Add filter function tests

* Revert triggering for all vendors

* Simplify aliasMap

* Replace functions in filter within GraphQL aggregate query

* Add API support for filtering of alias field

* Mark schema as optional

* Shift logical operators upwards

* Separate recursive parseFilter

* Rework shifting of logical operators

* Error on invalid usage of _none and _some

* Use inner join to preserve sort order

* Run tests for all vendors

* Reuse aliasMap for sort and filter

* Sort on top level query

* Remove unnecessary limit on wrapper query

* Refactor applyQuery options

* Remove duplicates from nested multi relational sort

* Fix offset in MSSQL requiring OrderBy

* Disable schema cache

* Use inner query only for nested sort or multi relational filter

* Fix MSSQL duplicate order column

* Use inner query only for multi relational

* Additional integration tests

* Order within partition for multi relational sorts

* Rename to directus_row_number

* Fix unit test

* Add base sort and filter tests

* Fix Oracle uppercased rowNumber column

* Fix unit test

* Fix top level query sort with function

* Parse functions in inner query

* Increase clarity with knex.ref()

* Remove sort filter for top level primary key

* Fix unit test

* Bypass queries with groupBy

* Add collection to aliasMap to fix functions in nested sort

* Fix multi relational sort with functions

* Add tests for filter and sort with functions

* Fix accidental deletion of brackets

* Fix top level alias filter node interface

* Update M2M sort tests

* Add M2A tests

* Cast m2a primary key as varchar2 for oracle

* Enable filtering tests for M2A

* Fix prototype polluting assignment in aliasMap

* Remove unnecessary currentKey

* Simplify code to increase readability

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Fix linting and missing 'this' error

* Revert optional chaining

* Add mysql5 to tests

* Fix mysql5 missing rowNumber()

* Overcome indexing delays in MySQL5

* Verify MySQL5 sorting is in order as the result count varies between runs

* Skip joining when sorting field already exists

* Simplify variable assignment

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Fix linting

* Reduce duplicate logic with vars

* Transform _func fields in GraphQL only for valid functions

* Fix unit test

* Fix unsupported date_part() in CrDB

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-12-21 11:56:18 -05:00
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00