Commit Graph

223 Commits

Author SHA1 Message Date
Pascal Jufer
f7c2f14181 Check if there are any missing contributors (#18099) 2023-04-07 07:58:47 -04:00
ian
6bf5327614 Remove double bracket from CLA message (#18086) 2023-04-06 16:58:39 +02:00
Rijk van Zanten
535edd7d7e Update cla.yml 2023-04-05 17:42:21 -04:00
Rijk van Zanten
7db025b95c Create CODEOWNERS 2023-04-05 17:37:42 -04:00
Rijk van Zanten
1505a2cc8c Update pull_request_template.md (#18075) 2023-04-05 17:34:52 -04:00
Rijk van Zanten
8a654433e1 Update cla.yml 2023-04-05 14:01:27 -04:00
Rijk van Zanten
a9d45fc4a5 Add CLA + automation (#18071) 2023-04-05 13:59:50 -04:00
Pascal Jufer
8c0ac96358 Add workflow for docs spellchecker (#18063)
* Ignore node_modules for docs spellchecking

* Add workflows for docs spellchecking

* Update command to new structure

* Skip core checks on pure docs changes

* Don't build packages for spellcheck workflow

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-05 09:54:11 -04:00
Rijk van Zanten
31e575202c Update docs_problem.yml 2023-04-05 09:19:47 -04:00
Pascal Jufer
aeb26bdbaa Update pull request template after move of docs (#18058) 2023-04-05 11:07:45 +00:00
rijkvanzanten
11e10742a4 Update issue templates 2023-04-04 18:57:00 -04:00
rijkvanzanten
8ea179248e No emoji please 2023-04-04 17:49:08 -04:00
Pascal Jufer
5b256f64c6 Use pnpm 8.1.0, define version via packageManager field, clean-up other version definitions (#17993)
* Use `packageManager` field to define pnpm version in one place

* Update to pnpm 8.1.0 (see also #17990)

* Enforce pnpm >= 8 via `engines`

* Clean-up `engines` & remove `volta`

* Add dedup workaround, consolidate test env prepare steps

* No checkout, no action :-)

* Bump minimal pnpm version to 8.1.0

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-04 15:20:25 -04:00
Pascal Jufer
5d6468b477 Fix publishing of packages to NPM in release workflow (#17900)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-03-23 14:02:55 -04:00
Rijk van Zanten
59f965cc8d Update release.yml 2023-03-23 13:40:26 -04:00
Rijk van Zanten
0d8b3a52c7 Simplify / improve release flow (#17858)
* Remove git-hook

This is already running in GH Actions as well

* Start new Dockerfile

* Remove unused files

* Close, but not quite

* Line break between environment specific commands / operational ones

* Cleanup dockerfile

* Update github release automation

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Improve based on review comments

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update Dockerfile

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-03-23 13:20:14 -04:00
Rijk van Zanten
5ea0ecaa2d Update bug_report.yml 2023-03-23 10:13:27 -04:00
ian
cce33436d1 Rework schema caching to use memory (#17763)
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Jan Arends <jaads@users.noreply.github.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-03-23 12:52:06 +01:00
Rijk van Zanten
2d49526a6f Update FUNDING.yml
It's a team effort <3
2023-03-22 10:47:02 -04:00
Rijk van Zanten
9526b4e5b2 Improve S3 asset read performance (#17835)
* Create new s3 client for each read

* Temp disable ts while debugging

* Add concurrency test

* Add minio to other tests

* Reduce unavailable count

* Trigger blackbox tests whenever packages are updated

* Prevent minio-mc from exiting

* Decrease requests and increase test timeout

* Spam more requests over longer period

* Increase request timeout

* Run autocannon directly with larger image

* Fix tests

* Lock version

* My favorite file

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-03-20 18:16:30 -04:00
Pascal Jufer
4d7e81f295 Switch back to latest pnpm version in workflows (#17809) 2023-03-16 11:52:24 +01:00
Pascal Jufer
1c7320e68c Use pnpm 7.29.1 for now to circument dep regression (#17787) 2023-03-14 21:30:15 +08:00
Pascal Jufer
4b2edb05c3 Update gh-action-linear to v0.3.1 (#17693) 2023-03-03 12:17:14 +01:00
Pascal Jufer
d5c39fd087 Use dedicated PAT for slash command action (#17671) 2023-03-01 17:06:35 -05:00
Pascal Jufer
59dcd30e6e Use npm auth set-up from 'setup-node' action (#17656) 2023-03-01 08:07:16 +01:00
Rijk van Zanten
b98b67b8e2 Create slash-commands.yaml 2023-02-28 10:55:59 -05:00
Rijk van Zanten
451e48bd93 Create linear-dispatch.yaml 2023-02-28 10:55:42 -05:00
Rijk van Zanten
c4b185942a Update and rename linear.yml to linear-auto.yml 2023-02-28 10:54:52 -05:00
Rijk van Zanten
3c69a14021 Update bug_report.yml 2023-02-15 15:32:09 -05:00
Rijk van Zanten
3e20b18bdb Update bug_report.yml 2023-02-15 15:30:04 -05:00
Rijk van Zanten
fea0a8a20f Update feature-requests.yml 2023-02-08 12:57:23 -05:00
Rijk van Zanten
43f55bb76c Update feature-requests.yml 2023-02-08 12:56:40 -05:00
Rijk van Zanten
b3c57eab54 Add discussion template for feature requests (#17451) 2023-02-08 12:56:01 -05:00
Rijk van Zanten
d2fae39372 Delete add-to-project.yml 2022-12-21 15:09:43 -05:00
ian
f1a8e0446f Fix duplicated results and functions in nested filters (#14798)
* Speed query up by reusing existing aliases which reduces table joins

* Use subquery in top level m2o to remove duplicates

* Fix linting

* Apply distinct on primary key field in subqueries

* Use distinct instead as there are only primary keys

* Apply subquery on top level

* Try remove sub sub query

* Test if working for all vendors

* Add support for _none and _some

* Use subquery only when field depth > 1

* Add tests

* Use original table names for columns with functions (#14690)

* Use original table names for columns with functions

* Extract filter function path parsing as shared util

* Fix filter function path when adding node

* Pass the originalCollectionName into filter functions

* Update unit test

* Replace functions within deep GraphQL

* Fix invalid operator error for _none and _some

* Add filter function tests

* Revert triggering for all vendors

* Simplify aliasMap

* Replace functions in filter within GraphQL aggregate query

* Add API support for filtering of alias field

* Mark schema as optional

* Shift logical operators upwards

* Separate recursive parseFilter

* Rework shifting of logical operators

* Error on invalid usage of _none and _some

* Use inner join to preserve sort order

* Run tests for all vendors

* Reuse aliasMap for sort and filter

* Sort on top level query

* Remove unnecessary limit on wrapper query

* Refactor applyQuery options

* Remove duplicates from nested multi relational sort

* Fix offset in MSSQL requiring OrderBy

* Disable schema cache

* Use inner query only for nested sort or multi relational filter

* Fix MSSQL duplicate order column

* Use inner query only for multi relational

* Additional integration tests

* Order within partition for multi relational sorts

* Rename to directus_row_number

* Fix unit test

* Add base sort and filter tests

* Fix Oracle uppercased rowNumber column

* Fix unit test

* Fix top level query sort with function

* Parse functions in inner query

* Increase clarity with knex.ref()

* Remove sort filter for top level primary key

* Fix unit test

* Bypass queries with groupBy

* Add collection to aliasMap to fix functions in nested sort

* Fix multi relational sort with functions

* Add tests for filter and sort with functions

* Fix accidental deletion of brackets

* Fix top level alias filter node interface

* Update M2M sort tests

* Add M2A tests

* Cast m2a primary key as varchar2 for oracle

* Enable filtering tests for M2A

* Fix prototype polluting assignment in aliasMap

* Remove unnecessary currentKey

* Simplify code to increase readability

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Fix linting and missing 'this' error

* Revert optional chaining

* Add mysql5 to tests

* Fix mysql5 missing rowNumber()

* Overcome indexing delays in MySQL5

* Verify MySQL5 sorting is in order as the result count varies between runs

* Skip joining when sorting field already exists

* Simplify variable assignment

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Fix linting

* Reduce duplicate logic with vars

* Transform _func fields in GraphQL only for valid functions

* Fix unit test

* Fix unsupported date_part() in CrDB

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-12-21 11:56:18 -05:00
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00
Rijk van Zanten
1df81a0826 Update linear.yml 2022-12-16 11:14:39 -05:00
Rijk van Zanten
88312eb830 Create linear.yml 2022-12-15 17:35:14 -05:00
Pascal Jufer
20a06a6c0e Clean-up lint / format set-up (#16380)
* Format json & all yaml files and update ignore files

* Fix resulting formatting issues
2022-11-09 07:26:54 -08:00
ian
7899f6a267 Remove dev dependencies when running blackbox tests (#16271)
* Remove dev dependencies when running blackbox tests

* Install dev dependencies in workspace root to run tests

* Shift dependency to see if tests fail

* Revert dependency shifting test

* Remove specifying of shell
2022-11-03 09:15:48 -04:00
Pascal Jufer
f5c564ddfd Fix "Create release" job (#16261) 2022-11-01 21:11:51 -04:00
Roger Stringer
53a3194d5c Add support for SAML Part 2 (#16145)
* new saml branch

* put saml info back in

* put saml info back in

* clean up code

* validate saml config

* validate schema

* Add saml auth flow tests

* use RelayState for redirects

* Update tests for RelayState

* Fix linting

* remove validateMeta as samlify does it already

* Fix linting

* change catch on login

* Update api/src/auth/drivers/saml.ts

Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>

* remove login since not needed here

* clear cookie if set on logout

* empty login method

* invalidate logout in db

* if relayState and login failed, redirect back with a reason

* Cleanup linter warnings

* Remove range from packages

* Opinions opinions opinions opinions

Just a couple personal opinion cleanup pieces

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-11-01 22:09:31 +00:00
Pascal Jufer
92466dd2a9 Replace depreacted "create-release" action (#16144)
* Replace depreacted "create-release" action

* Fix indentation

* Target is not required as the tag will already exist

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-10-24 12:26:47 -04:00
Pascal Jufer
5507ca3f41 Fix path for packages in package.json used for Docker image (#16141)
* Use basename to get relative path for packages

* Revert "Revert "Small `Release` workflow clean-up (#16083)" (#16120)"

This reverts commit c2ee8c24ef.
2022-10-24 09:47:03 -04:00
Rijk van Zanten
c2ee8c24ef Revert "Small Release workflow clean-up (#16083)" (#16120)
This reverts commit dbfedb56fe.
2022-10-22 13:05:22 -04:00
Pascal Jufer
dbfedb56fe Small Release workflow clean-up (#16083)
* Consistent usage of `if` statements

Brackets are not required in `if` statements, see https://docs.github.com/en/actions/learn-github-actions/expressions#about-expressions

* Use prepare action to setup pnpm in publish-npm job
2022-10-20 12:34:19 -04:00
ian
98daa7dbac Update upload-artifact action to v3 (#16081) 2022-10-20 11:29:53 -04:00
Pascal Jufer
2939eb1d96 Prepare Action: Set output via newer file method (#16080) 2022-10-20 23:13:50 +08:00
ian
83b6027cc3 Run blackbox tests in parallel (#16048)
* Run blackbox tests in parallel

* Add slight delay for updates in timezone tests

* Revert trigger on PR
2022-10-19 10:04:32 -04:00
ian
6cdd0eead4 Fix CodeQL threadflow steps (#15873)
* Upload SARIF artifact

* Test removal of empty object

* Fix polynomial-redos

* Revert fail-fast change

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-10-12 16:01:13 -04:00