Commit Graph

2409 Commits

Author SHA1 Message Date
José Varela
5e555f7756 Graphql: Prevent store response with errors in cache (#16926)
* Prevent cache response with errors

* Fix failing test

* mock env in notifications service test

* Use res.locals.cache = false instead

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-01-12 17:06:24 -05:00
Francisco Marques
0f9f0d8fc4 Fix url encoding query parameters and added tests (#17120)
* fix(utils): query parameters encoding for Url

* improvement(utils): added tests to Url

Also, made trailing slash behaviour consistent

* fix: updated nvmrc to the recommended node version
2023-01-12 12:31:51 -05:00
Brainslug
d312bd172d Fix GraphQL server health output (#17113)
* fix graphql server health

* Add tests

Co-authored-by: ian <licitdev@gmail.com>
2023-01-12 13:12:06 +01:00
rijkvanzanten
065c77dc1d v9.22.4 2023-01-11 17:05:16 -05:00
Azri Kahar
d2a6621153 Add tests for Flows operations (#16580)
* Add tests for Flows operations

* fix notifications test env

* tweaks

* fix env mock
2023-01-11 16:12:16 -05:00
Roger Stringer
102b965abe offset should be 0 not 1 by default (#17101) 2023-01-11 20:57:18 +00:00
rijkvanzanten
9f708d67bc v9.22.3 2023-01-11 12:16:11 -05:00
rijkvanzanten
b9a5005859 v9.22.2 2023-01-11 11:27:56 -05:00
Azri Kahar
32603cbf2f use setCacheValue in get-permissions util (#17044) 2023-01-10 16:45:59 -05:00
Brainslug
dda1afcf16 Collection export limit 0 leads to 500 error (#16722)
* prevent 500 error when limit=0

* short circuit parser instead

* removed unneeded comment

* setting a sensible minimum and step for export limit

* fixed notif message when exporting limit=-1

* re-added !exportSettings.limit fallback behavior

* guard against negative limit values lower than -1

* add unit test for sanitizeQuery

* remove unnecessary test cases

because offset and page are validated in validateQuery

* delegate validation to validateQuery instead

* use sanitizeQuery util for export

* Added check for integers

Co-authored-by: ian <licitdev@gmail.com>

* validate instead of sanitize

* removed obsolete test

* added integer checks for limit, offset, page and sensible minimums

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-01-10 09:35:31 +01:00
Rijk van Zanten
c249d58a8c Revert "Speed-up api dev scripts by using swc (#16609)" (#17064)
This reverts commit e9a2ce5a9e.
2023-01-09 16:57:03 -05:00
Azri Kahar
7616b25988 Pass emitEvents to query methods in item-read & item-delete operations (#16579)
* fix item-read's readByQuery emitEvents

* fix item-delete's deleteByQuery emitEvents

* tests
2023-01-09 15:50:55 -05:00
Pascal Jufer
e9a2ce5a9e Speed-up api dev scripts by using swc (#16609) 2023-01-09 15:47:41 -05:00
ian
1d6e77d5ab Fix last admin check for alterations type inputs (#16595)
* Fix last admin check for alterations type inputs

* Add unit tests

* fix mocks causing side-effects

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-01-09 20:33:13 +08:00
ian
f0e8f0bef0 Remove TZ conversion for timestamps in MySQL (#16679) 2023-01-06 16:06:09 -05:00
José Varela
c1887bee86 Cache / Permissions: Prevent server crash when command times out (#16927)
* Catch get permissions from cache when command times out

* Update api/src/utils/get-permissions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Use get/set cache for permissions

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-01-06 18:20:14 +00:00
Roger Stringer
c429ecd57c healthcheck crashes with local file storage (#16944)
* Update api/src/services/server.ts
* remove try catch in write
* Update api/src/services/server.ts

Co-authored-by: Freekrai <freekrai@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-01-06 07:17:04 -08:00
Nitwel
2ac022d286 Extension Improvements (#16822)
* add link command and small improvements

* put local bundles into own folder on link

* get rid of packs and add bundle support for local extensions

* make bundle type extensions work locally and remove traces of pack

* fix hot reloading of bundles

* fix app.js not refreshing

* fixed linter errors

* add endpoint to install extensions

* update package.json validation and support top level extensions

* update endpoints

* added some URL escapes and ran linter

* remove installation part

* readd endpoint

* update dependencies

* fix types and validation in extension-sdk

* run linter

* fix linter

* add defaults to manifest

* Added missing constant export

* ensure all the extension folders

* ignore unneeded vite error

* update linking process

* run parser separate

* add await

* fixed linter errors

Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-01-04 15:20:33 +01:00
ian
802dfa0703 Fix inner query sort limit (#16922)
* Add sort within inner query

* Run tests for all vendors

* Apply limit to all queries

* Enable sorting in MSSQL subquery by setting a limit

* Add tests

* Parse function field names for a2o

* Improve order string spacing

* Update unit test

* Fix mysql5 tests

* Revert testing for all vendors

* Extract original collection name

* Add comment for MSSQL applyLimit helper

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-01-04 19:43:17 +08:00
ian
d01c4cbedf Allow env access in Flows Run Script operation (#16111)
* Allow env access in Flows Run Script operation

* Add env variable

* Expose env for consistency

* Remove global variable from run script operation

* Rename env to FLOWS_ENV_ALLOW_LIST

* Use env within data for consistency

* Remove env notice in run script operation

* Remove translation and tests

* Fix unit test

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-01-04 10:39:52 +01:00
Pascal Jufer
c3ecbed0b4 Handle plain object body in "Send Email" operation (#16859)
* Handle plain object body in "Send Email" operation

* Update api/src/operations/mail/index.ts

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-30 14:08:18 +01:00
Azri Kahar
e899244ef3 Add unit tests to several API utility functions (#16662)
* add unit tests to several API util functions

* fix timezone tests to account for daylight saving

* add a note for future reference

* Update api/src/utils/get-date-formatted.test.ts

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* remove unnecessary note as it is not an issue

* fix getEnv mock in validate-env test

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-12-23 16:51:49 +00:00
Azri Kahar
ddb873e09b Fix getCacheKey path matching for graphql (#16647)
* fix getCacheKey path matching for graphql

* tweak  test
2022-12-23 11:48:45 -05:00
Rijk van Zanten
40f94e6907 Patch Tuesday 🐸 (#16875)
* Update versions

* Fix usage of fullcalendar

* Patch Tuesday Additions 🐸🐸 (#16878)

* Interop default in vitest

* Update exif-reader

* remove deprecated @types/pino

* remove old "grant" package type shim

* Small clean-up of dependencies (#16576)

* Remove outdated type deps & update concurrently

* fix lock file

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2022-12-23 10:45:15 -05:00
rijkvanzanten
602f5db4f7 v9.22.1 2022-12-21 17:12:11 -05:00
rijkvanzanten
8f0fb8b40f v9.22.0 2022-12-21 14:33:04 -05:00
ian
f1a8e0446f Fix duplicated results and functions in nested filters (#14798)
* Speed query up by reusing existing aliases which reduces table joins

* Use subquery in top level m2o to remove duplicates

* Fix linting

* Apply distinct on primary key field in subqueries

* Use distinct instead as there are only primary keys

* Apply subquery on top level

* Try remove sub sub query

* Test if working for all vendors

* Add support for _none and _some

* Use subquery only when field depth > 1

* Add tests

* Use original table names for columns with functions (#14690)

* Use original table names for columns with functions

* Extract filter function path parsing as shared util

* Fix filter function path when adding node

* Pass the originalCollectionName into filter functions

* Update unit test

* Replace functions within deep GraphQL

* Fix invalid operator error for _none and _some

* Add filter function tests

* Revert triggering for all vendors

* Simplify aliasMap

* Replace functions in filter within GraphQL aggregate query

* Add API support for filtering of alias field

* Mark schema as optional

* Shift logical operators upwards

* Separate recursive parseFilter

* Rework shifting of logical operators

* Error on invalid usage of _none and _some

* Use inner join to preserve sort order

* Run tests for all vendors

* Reuse aliasMap for sort and filter

* Sort on top level query

* Remove unnecessary limit on wrapper query

* Refactor applyQuery options

* Remove duplicates from nested multi relational sort

* Fix offset in MSSQL requiring OrderBy

* Disable schema cache

* Use inner query only for nested sort or multi relational filter

* Fix MSSQL duplicate order column

* Use inner query only for multi relational

* Additional integration tests

* Order within partition for multi relational sorts

* Rename to directus_row_number

* Fix unit test

* Add base sort and filter tests

* Fix Oracle uppercased rowNumber column

* Fix unit test

* Fix top level query sort with function

* Parse functions in inner query

* Increase clarity with knex.ref()

* Remove sort filter for top level primary key

* Fix unit test

* Bypass queries with groupBy

* Add collection to aliasMap to fix functions in nested sort

* Fix multi relational sort with functions

* Add tests for filter and sort with functions

* Fix accidental deletion of brackets

* Fix top level alias filter node interface

* Update M2M sort tests

* Add M2A tests

* Cast m2a primary key as varchar2 for oracle

* Enable filtering tests for M2A

* Fix prototype polluting assignment in aliasMap

* Remove unnecessary currentKey

* Simplify code to increase readability

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Fix linting and missing 'this' error

* Revert optional chaining

* Add mysql5 to tests

* Fix mysql5 missing rowNumber()

* Overcome indexing delays in MySQL5

* Verify MySQL5 sorting is in order as the result count varies between runs

* Skip joining when sorting field already exists

* Simplify variable assignment

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Fix linting

* Reduce duplicate logic with vars

* Transform _func fields in GraphQL only for valid functions

* Fix unit test

* Fix unsupported date_part() in CrDB

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-12-21 11:56:18 -05:00
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00
Brainslug
4bb8a463f6 Added extra environment setting for sharp processing of invalid images (#16811)
* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`
2022-12-21 11:55:12 +01:00
Brainslug
ca93f5cb1d Add support for custom JS embeds in the App (#16650)
* add embed hook definitions

* inject embeds in the App html

* fixed typo

* removed unnecessary env parameter

* Added comment marking the custom embeds

* attempt to add test for createApp

* mock db in app test

* temporarily set log style to raw in test

* one more round

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2022-12-17 20:14:30 -05:00
Brainslug
bbcf76e030 enforce uppercase UUIDs for MS SQL (#16691)
* add forced uppercase for UUIDs in MS SQL

* test whether mssql returned uuid is uppercase

* use once just for slight better correctness

* once is actually not enough

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
2022-12-15 13:34:31 +08:00
Azri Kahar
431f95210a align ASSETS_CACHE_TTL default value in .env stub (#16755) 2022-12-15 00:34:02 +08:00
Brainslug
807bda3dcb Follow up user Roles not loading (#16278)
* applied the same role fix too users invite

* added same fix when saving permissions

* Prevent loading users unecessarily on the permission page

* Added proper `deep` query typing

* fixed typing for tests

* apply similar fix for navigation folder
2022-12-13 15:05:14 +01:00
Nick Ufer
9405e87a34 fix: adds missing Query and x-metadata to /components/schemas (#16524)
* fix: adds missing Query and x-metadata to /components/schemas

* test: adds Query and x-metadata to specifications.test.ts

* refactor: moves oas schemas to be always included to constants.ts

* refactor: renames OAS_INCLUDE_SCHEMAS to OAS_REQUIRED_SCHEMAS

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-06 09:17:23 +01:00
Piotr
018452d141 Fix typo in env stub (#16717) 2022-12-05 14:45:41 +01:00
Rijk van Zanten
86dd593f91 Use node16 module-resolution (#16696)
* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use CJS require in hook import

* Fix type export in schema

* Clean up defaults usage

* Use require instead of import for migrations

* Use a vitest compatible dynamic import

* Uno mas

* Cleanup type export
2022-12-02 15:53:15 -05:00
Azri Kahar
a4de019ead Fix repeated logic caused by updateOne & deleteOne overrides (#16433)
* fix updateOne & deleteOne overrides repeated logic

* replace remaining jest usages

* fix notifications service env mock

* simplify mockImplementation with mockResolvedValue

* fix types in test

Co-authored-by: ian <licitdev@gmail.com>
2022-11-30 21:51:11 +08:00
Brainslug
1eea9d8b0f Missing logs with Pino asynchronous logging (#16299)
* force logger in sync mode when running on the CLI

* using the sync logger for all cli commands

* using cli logger for app initialization

* using pino pretty for log formatting

* installed deps

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2022-11-29 14:26:37 -05:00
rijkvanzanten
d6b8e5963d v9.21.2 2022-11-28 15:44:12 -05:00
rijkvanzanten
51b1daec58 v9.21.1 2022-11-28 15:09:00 -05:00
ian
8a3dc4b68b Propagate mutation options for schema apply (#16562)
* Propagate mutation options for schema apply

* Fix unit test

* Add bypassCache flag and remove flushCaches calls

* remove accountability option from getSchema

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2022-11-22 14:57:17 -05:00
José Varela
f3e4ae0b6f Add nonce to request (#16563) 2022-11-22 10:02:38 -05:00
Pascal Jufer
47db8e5d2f Clean-up dependencies (#16481)
* Clean-up api

- Remove unnecessary deps
- rimraf -> del-cli

* Clean-up app

- bytes -> pretty-bytes
- Remove ms as we already have pretty-ms
- Remove other unnecessary deps
- Add storybook build to gitignore

* Clean-up drive packages

- Simplify cleanup scripts
- rimraf -> del-cli
- Remove unnecessary deps

* Clean-up schema

- Remove unnecessary dep (lodash)
- Replace 'npm-watch' with native watch mode

* Clean-up shared

- Remove unnecessary dep (c8, adding @vitest/coverage-c8 will be addressed in
  other PR)
- rimraf -> del-cli
- npm-run-all -> concurrently

* Clean-up root

- Add missing eslint-plugin-jest dep
- listr -> listr2

* Make build output a bit friendlier

* Remove cleanup scripts

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-18 17:02:41 +00:00
Azri Kahar
f0da901af4 Set auth_data to null when updating user (#16501) 2022-11-18 11:46:55 -05:00
ian
06aca3329c Remove named timezone usage on MySQL (#16499) 2022-11-18 11:46:29 -05:00
rijkvanzanten
ba00c28bf3 v9.21.0 2022-11-17 16:31:59 -05:00
rijkvanzanten
4ab04f67f3 Add getEnv wrapper 2022-11-17 12:20:57 -05:00
Azri Kahar
5e4d7972b1 use pnpm shell-emulator instead of cross-env (#16486)
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-11-17 11:21:22 -05:00
ian
0ac08da7ed Fix legacy permissions for M2O fields in GraphQL (#16430)
* Remove relation for legacy permission without allowed field

* Remove deprecated formatError and improve error handling

* Add unit test

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-17 11:04:56 -05:00
Azri Kahar
ed53673b0f Fix condition operation passing even when the checked field isn't present in the payload (#16483)
* use requireAll in validatePayload for condition

* add/update tests
2022-11-17 10:55:43 -05:00