Commit Graph

10 Commits

Author SHA1 Message Date
ian
802dfa0703 Fix inner query sort limit (#16922)
* Add sort within inner query

* Run tests for all vendors

* Apply limit to all queries

* Enable sorting in MSSQL subquery by setting a limit

* Add tests

* Parse function field names for a2o

* Improve order string spacing

* Update unit test

* Fix mysql5 tests

* Revert testing for all vendors

* Extract original collection name

* Add comment for MSSQL applyLimit helper

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-01-04 19:43:17 +08:00
ian
f1a8e0446f Fix duplicated results and functions in nested filters (#14798)
* Speed query up by reusing existing aliases which reduces table joins

* Use subquery in top level m2o to remove duplicates

* Fix linting

* Apply distinct on primary key field in subqueries

* Use distinct instead as there are only primary keys

* Apply subquery on top level

* Try remove sub sub query

* Test if working for all vendors

* Add support for _none and _some

* Use subquery only when field depth > 1

* Add tests

* Use original table names for columns with functions (#14690)

* Use original table names for columns with functions

* Extract filter function path parsing as shared util

* Fix filter function path when adding node

* Pass the originalCollectionName into filter functions

* Update unit test

* Replace functions within deep GraphQL

* Fix invalid operator error for _none and _some

* Add filter function tests

* Revert triggering for all vendors

* Simplify aliasMap

* Replace functions in filter within GraphQL aggregate query

* Add API support for filtering of alias field

* Mark schema as optional

* Shift logical operators upwards

* Separate recursive parseFilter

* Rework shifting of logical operators

* Error on invalid usage of _none and _some

* Use inner join to preserve sort order

* Run tests for all vendors

* Reuse aliasMap for sort and filter

* Sort on top level query

* Remove unnecessary limit on wrapper query

* Refactor applyQuery options

* Remove duplicates from nested multi relational sort

* Fix offset in MSSQL requiring OrderBy

* Disable schema cache

* Use inner query only for nested sort or multi relational filter

* Fix MSSQL duplicate order column

* Use inner query only for multi relational

* Additional integration tests

* Order within partition for multi relational sorts

* Rename to directus_row_number

* Fix unit test

* Add base sort and filter tests

* Fix Oracle uppercased rowNumber column

* Fix unit test

* Fix top level query sort with function

* Parse functions in inner query

* Increase clarity with knex.ref()

* Remove sort filter for top level primary key

* Fix unit test

* Bypass queries with groupBy

* Add collection to aliasMap to fix functions in nested sort

* Fix multi relational sort with functions

* Add tests for filter and sort with functions

* Fix accidental deletion of brackets

* Fix top level alias filter node interface

* Update M2M sort tests

* Add M2A tests

* Cast m2a primary key as varchar2 for oracle

* Enable filtering tests for M2A

* Fix prototype polluting assignment in aliasMap

* Remove unnecessary currentKey

* Simplify code to increase readability

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Fix linting and missing 'this' error

* Revert optional chaining

* Add mysql5 to tests

* Fix mysql5 missing rowNumber()

* Overcome indexing delays in MySQL5

* Verify MySQL5 sorting is in order as the result count varies between runs

* Skip joining when sorting field already exists

* Simplify variable assignment

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Fix linting

* Reduce duplicate logic with vars

* Transform _func fields in GraphQL only for valid functions

* Fix unit test

* Fix unsupported date_part() in CrDB

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-12-21 11:56:18 -05:00
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00
Brainslug
bbcf76e030 enforce uppercase UUIDs for MS SQL (#16691)
* add forced uppercase for UUIDs in MS SQL

* test whether mssql returned uuid is uppercase

* use once just for slight better correctness

* once is actually not enough

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
2022-12-15 13:34:31 +08:00
Brainslug
807bda3dcb Follow up user Roles not loading (#16278)
* applied the same role fix too users invite

* added same fix when saving permissions

* Prevent loading users unecessarily on the permission page

* Added proper `deep` query typing

* fixed typing for tests

* apply similar fix for navigation folder
2022-12-13 15:05:14 +01:00
Pascal Jufer
45eb4941fc Small clean-up after Jest to Vitest switch in API (#16462)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-15 17:20:45 -05:00
Azri Kahar
b0529d4951 Optimize number of times cache is being cleared in ItemsService updateBatch (#16453)
* improve cache clearing for updateBatch

* add service level data-is-array check

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-15 10:14:35 -05:00
Pascal Jufer
c303bdcf10 Finish switch from Jest to Vitest in API (#16263)
* build:  add vitest and update test scripts

* build: 🔧 add vitest config

* build:  Migrate tests to vitest

Remove jest references from test api test files and replace with vitest equivalents.

Tests: 13 tests are failing.

* build: 🚚 move vite.config.ts to api/src folder

* build: 🔥 remove unused vitest.config from api root

* build:  import vitest modules for tests

* build:  add type conversion for actual object

* Finish switch from Jest to Vitest in API

* Replace some leftovers

* Load "sharp" before tests

* Try with cjs

* Temporary enable verbose reporter

* Try with globalSetup

* Fix path to globalSetup

* Provide default export in globalSetup

* Final clean-up

* Remove @vitest/ui & update vitest to 0.25.0

* Add vitest c8 coverage dependency

* Update vitest to v0.25.1

* Replace unnecessary Vitest workaround

* Rework new tests

* Resolve build errors

Co-authored-by: Dorian C Brown <brown.3794@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-14 16:09:47 -05:00
ian
73fa8d66ac Emit nested action events after the transaction completes (#14981)
* Emit nested action events after the transaction completes

* Propagate mutation options for the disabling of nested emits

* Disable emit in test

* Shift emitting
2022-08-18 16:07:33 -04:00
Rijk van Zanten
19ddd89d8d Move unit tests to files they apply to (#15130)
* Move unit tests to files they apply to

* Remove tests root
2022-08-17 16:01:16 -04:00