* Step 1
* Step 2
* False sense of confidence
* Couple more before dinner
* Update schema package
* Update format-title
* Upgrade specs file
* Close
* Replace ts-node-dev with tsx, and various others
* Replace lodash with lodash-es
* Add lodash-es types
* Update knex import
* More fun is had
* FSE
* Consolidate repos
* Various tweaks and fixes
* Fix specs
* Remove dependency on knex-schema-inspector
* Fix wrong imports of inspector
* Move shared exceptions to new package
* Move constants to separate module
* Move types to new types package
* Use directus/types
* I believe this is no longer needed
* [WIP] Start moving utils to esm
* ESMify Shared
* Move shared utils to @directus/utils
* Use @directus/utils instead of @directus/shared/utils
* It runs!
* Use correct schemaoverview type
* Fix imports
* Fix the thing
* Start on new update-checker lib
* Use new update-check package
* Swap out directus/shared in app
* Pushing through the last bits now
* Dangerously make extensions SDK ESM
* Use @directus/types in tests
* Copy util function to test
* Fix linter config
* Add missing import
* Hot takes
* Fix build
* Curse these default exports
* No tests in constants
* Add tests
* Remove tests from types
* Add tests for exceptions
* Fix test
* Fix app tests
* Fix import in test
* Fix various tests
* Fix specs export
* Some more tests
* Remove broken integration tests
These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these
* Regenerate lockfile
* Fix imports from merge
* I create my own problems
* Make sharp play nice
* Add vitest config
* Install missing blackbox dep
* Consts shouldn't be in types
tsk tsk tsk tsk
* Fix type/const usage in extensions-sdk
* cursed.default
* Reduce circular deps
* Fix circular dep in items service
* vvv
* Trigger testing for all vendors
* Add workaround for rollup
* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"
* Fix postgres
* Schema package updates
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
* Resolve cjs/mjs extensions
* Clean-up eslint config
* fixed extension concatination
* using string interpolation for consistency
* Revert MySQL optimisation
* Revert testing for all vendors
* Replace tsx with esbuild-kit/esm-loader
Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js
Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467
* Use exact version for esbuild-kit/esm-loader
* Fix import
---------
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Use `packageManager` field to define pnpm version in one place
* Update to pnpm 8.1.0 (see also #17990)
* Enforce pnpm >= 8 via `engines`
* Clean-up `engines` & remove `volta`
* Add dedup workaround, consolidate test env prepare steps
* No checkout, no action :-)
* Bump minimal pnpm version to 8.1.0
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
---------
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Create new s3 client for each read
* Temp disable ts while debugging
* Add concurrency test
* Add minio to other tests
* Reduce unavailable count
* Trigger blackbox tests whenever packages are updated
* Prevent minio-mc from exiting
* Decrease requests and increase test timeout
* Spam more requests over longer period
* Increase request timeout
* Run autocannon directly with larger image
* Fix tests
* Lock version
* My favorite file
---------
Co-authored-by: ian <licitdev@gmail.com>
* Expose schema endpoints
* respond with 204
* fix payload for export flag
* allow export as YAML
* use SchemaService
* move getStringFromStream to utils
* clearer exceptions with logs
* check empty request body
* specs
* add bypassCache flag and remove flushCaches calls
* Identify integer typed fields in Oracle
* Fix uuid foreign keys type for Oracle
* Detect autoincrement for CockroachDB
* Bypass cache for fields and relations services
* Add same DB tests
* Test if working for all vendors
* Add schema snapshot testing into sequential flow
* Fix schema re-apply attempts when it's a different database vendor (#14816)
* prevent diffing on certain properties
* fix apply-snapshot test
* prevent updates to autoincrement fields' schema
* sanitize relations
* remove `colleciton.schema.sql` parsing in test
It is now being sanitized by `sanitizeCollection()` function in get-snapshot-diff.ts
* Change concurrency group to not cancel test running on main
* remove multipart for apply & add hash checking
* check incoming version & add force query param
* refine error message & stub for nested hash
* add `vendor` top-level property
as an additional safety net for cross db vendor applies
* sanitize generated schema snapshots
* snapshot joi validation
* diff joi validation
* minor cleanup
* extract applyDiff & use deep-diff applyChange
* use applyDiff in schema service
* Mark vendor as optional
* Update tests to apply diffs
* move force flag check into service
* Patch mssql vendor typo
* Set relation no action in cyclic relations
* Update mysql timestamp default value
* Oracle cannot define no action
* Update oracle timestamp type default values
* add hash checking for outdated diffs
* fix diff service & endpoint
* Add hashes when returning after
* Fix self referencing relations for Oracle
* Add temp fix for CURRENT_TIMESTAMP defaults in Oracle
* clean up driver and database client types
* only require diff hash if not kind N
* update hash comparison logic for create/delete
* Set no action constraint for self referencing M2M in MSSQL
* Add basic hash check tests
* omit default value when hashing auto increments
Specifically for CockroachDB with differing sequences
* add vendor check
* update specs
* Validate vendors with type definition
* Spread the vendors input array
* re-add Export component
* re-add js-yaml to root
* Propagate mutation options for schema apply
* Verify that snapshots differ and clear cache before hash tests
* Fix unit test
* Revert temp fix for CURRENT_TIMESTAMP defaults in Oracle
* Define and reuse type for export format
* Define and reuse list of database clients
* change `were` to `was`
* change `where` to `were`
* add some empty lines for readability
* tweak exception message
* fix test
* use object-hash
* use boolean to check whether file is included
* simplify request content type check
* throw error when multiple files were uploaded
* use nullish coalesce instead of short circuit
* Update api/src/services/schema.ts
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Remove unnecessary `Omit` on `SnapshotWithHash` type
* Revert "Remove unnecessary `Omit` on `SnapshotWithHash` type"
This reverts commit d22ac771ec.
* check empty snapshot earlier
* use allow-list logic via pick instead of omit
* Update api/src/services/schema.ts
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Move to own /schema endpoint controller
* Fix refs to schema endpoint
* move streamToString to utils package
* move get-versioned-hash and add test
* extract kind into an enum
* Fix mysql5 timestamp value
* Fix test collection existing on rerun
* resolve TODO comment in blackbox test
* Drop deep level hashes in diff
These hashes are used only for more accurate error reporting
but are otherwise superfluous, since changes are already
detected by the top level hash.
Therefore we remove them in favor of a simpler diff format and
implementation.
* Revert schema "fix" for createItem, add comment
* Strict diff schema validation
* Revert CrDB auto-increment detection patch in 816c998
* Clear systemCache to expose newly added fields
* Use DiffKind constants
* Extract diff & snapshot validation into own tested utils
* Apply suggestions from @azrikahar
* Update knex-schema-inspector to 3.0.1
Includes the fix for CrDB auto-increment detection (knex/knex-schema-inspector#135)
* Update knex-schema-inspector in packages
* Update lock file
* add test for schema service
* add test for export service
* add relevant tests to modified util functions
* fix csv test to account for os end of line
* fix files controller test
* dedupe test data for schema service
* Align schema specs with docs
* Update api/src/controllers/schema.ts
* Revert testing for all vendors
---------
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Speed query up by reusing existing aliases which reduces table joins
* Use subquery in top level m2o to remove duplicates
* Fix linting
* Apply distinct on primary key field in subqueries
* Use distinct instead as there are only primary keys
* Apply subquery on top level
* Try remove sub sub query
* Test if working for all vendors
* Add support for _none and _some
* Use subquery only when field depth > 1
* Add tests
* Use original table names for columns with functions (#14690)
* Use original table names for columns with functions
* Extract filter function path parsing as shared util
* Fix filter function path when adding node
* Pass the originalCollectionName into filter functions
* Update unit test
* Replace functions within deep GraphQL
* Fix invalid operator error for _none and _some
* Add filter function tests
* Revert triggering for all vendors
* Simplify aliasMap
* Replace functions in filter within GraphQL aggregate query
* Add API support for filtering of alias field
* Mark schema as optional
* Shift logical operators upwards
* Separate recursive parseFilter
* Rework shifting of logical operators
* Error on invalid usage of _none and _some
* Use inner join to preserve sort order
* Run tests for all vendors
* Reuse aliasMap for sort and filter
* Sort on top level query
* Remove unnecessary limit on wrapper query
* Refactor applyQuery options
* Remove duplicates from nested multi relational sort
* Fix offset in MSSQL requiring OrderBy
* Disable schema cache
* Use inner query only for nested sort or multi relational filter
* Fix MSSQL duplicate order column
* Use inner query only for multi relational
* Additional integration tests
* Order within partition for multi relational sorts
* Rename to directus_row_number
* Fix unit test
* Add base sort and filter tests
* Fix Oracle uppercased rowNumber column
* Fix unit test
* Fix top level query sort with function
* Parse functions in inner query
* Increase clarity with knex.ref()
* Remove sort filter for top level primary key
* Fix unit test
* Bypass queries with groupBy
* Add collection to aliasMap to fix functions in nested sort
* Fix multi relational sort with functions
* Add tests for filter and sort with functions
* Fix accidental deletion of brackets
* Fix top level alias filter node interface
* Update M2M sort tests
* Add M2A tests
* Cast m2a primary key as varchar2 for oracle
* Enable filtering tests for M2A
* Fix prototype polluting assignment in aliasMap
* Remove unnecessary currentKey
* Simplify code to increase readability
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
* Fix linting and missing 'this' error
* Revert optional chaining
* Add mysql5 to tests
* Fix mysql5 missing rowNumber()
* Overcome indexing delays in MySQL5
* Verify MySQL5 sorting is in order as the result count varies between runs
* Skip joining when sorting field already exists
* Simplify variable assignment
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
* Fix linting
* Reduce duplicate logic with vars
* Transform _func fields in GraphQL only for valid functions
* Fix unit test
* Fix unsupported date_part() in CrDB
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Update monorepo root
* Update API dependencies
* Use unranged version
* Update app deps
* Upgrade packages dependencies
* Fix shared build
* Fix type warning in api
* Fix test runner
* Get rekt by ya boy comma-,
* Fix missing unit deprecation for sass color.mix()
Co-authored-by: ian <licitdev@gmail.com>
* Update API non-majors
* Update major changes
* Temp fix for type warning in sdk
* Resolve missing new keyword in GraphQLNonNull use
* Lock versions in api
* Update and lock versions in shared
* Update lockfile
* Fix build type errors in api
* Update app patches/minors
* Upgrade major versions
* Lock versions
* Upgrade root deps
* Update packages
* Switch from pino-colada to pino-pretty
* Update snapshots
* Fix globby sync usage
* Remove globals from jest config
* No ranges
* Revert "Remove globals from jest config"
This reverts commit e249e2c8fe.
* Revert "Upgrade root deps"
This reverts commit 62add28a85.
* Update non-jest stuff
* We'll keep trying
* Update globby to latest minor only
* Test on all vendors
* Revert test on all vendors
* Fix linting
* Preserve request log format
* Preserve existing timestamp format and allow configuration
* Use lodash merge to allow for nested objects
Co-authored-by: ian <licitdev@gmail.com>
* Use pnpm instead of npm
* Setup workspace
* Replace scripts / lerna root stuff
* Add todo file
* Update lock, start updating packages
* Update todo
* [WIP] Keep going on module resolution in pnpm
* Fix final missing deps/types for complete build
* [WIP] Replace npm commands with pnpm alternatives
* Update jest config for v28
* Fix missing package under schema
* Update workflow
* Run CI on fork
* Use local reference
* Remove version from local workflow
* Add build to prepare
* Add Node's max old space size flag to linter
* Idemdito for codeql
* Hyphens? Underscores?
* Bang
* Only build one thing at a time
* Underscores after all?
* Match 7gb of GH
* Set env on prepare
* Jest is driving me nuts
* Downgrade jest
* Add root jest
* We'll get there, eventually.
* 🥳
* Het houdt niet op, niet vanzelf
* attempt to fix e2e
* fix unit test & shared import
* add `debug: true` to CodeQL action
* Fix dev call in shared
* Add missing cross-env
* Add missing geo-json dependency for app
* add flag to prepare action to skip build
* remove debug flag from codeql action
* fix invalid env syntax
* add tslib to app dependencies for tests
* another attempt
* Added missing rimraf dependency for the api prebuild/cleanup scripts
* Added missing ts-node dependency for the api cli script
* update E2E tests readme to use pnpm
* Undo move of extensions sdk
* Fixes nested groups in accordions not rendering fields (#14369)
* initial fix for nested groups in accordions
* removed debug code
* Fix strict relative dates showing "incorrect" (#14390)
* Set rounding method of relative formatted date to floor when strict option is selected
* Add round fn as an option
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Don't use locales in generated camelCased values from env (#14401)
Ref https://github.com/directus/directus/discussions/14122
* Fix list panel descending sort (#14396)
* fix list panel descending sort
* Handle missing or explicit desc value
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Fix query primary field for system tables (#14402)
* New Crowdin updates (#14403)
* Update source file en-US.yaml
* New translations en-US.yaml (Spanish, Chile)
* New translations en-US.yaml (Spanish, Latin America)
* New translations en-US.yaml (Greek)
* New translations en-US.yaml (Italian)
* New translations en-US.yaml (French, Canada)
* Fix/wysiwyg context menu (#14404)
* New translations en-US.yaml (Polish) (#14407)
* Encoded the url using encodeURIComponent, so that the url gets sanitzed and so, we did not get 404 error. (#14418)
* Encoded the url using encodeURIComponent, so that the url gets sanitized and we did not get 404 error.
* Added required changes as stated by the reviewer
* Used encodeURI instead of encodeURIComponent to encode the params only
* Solved linting error
* Solved linting errors
* checked with npm run lint, no linting errors found
* fix generateJoi error due to empty permissions when creating new role (#14416)
* fix empty permissions when creating new role
* basic test
* move logic up
* additional test
* Update api/tests/utils/filter-items.test.ts
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: ian <licitdev@gmail.com>
* Recreate lockfile
* Fix build of extensions-sdk
* Add missing dependency
* Add some more missing deps
* Update blackbox to use pnpm
* Update workflow to use main
* Update pack to work with pnpm
* Simplify commands
* Remove todo file (completed)
* ADd missing types dep
* Use local test version
* Remove version from local reference
* Call super with context
* Add missing dep
* Simplify workflows
Ref https://github.com/directus/organization/issues/135
* Simplify some more
* Linter is on root
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Gerard Lamusse <gerardlamo@gmail.com>
Co-authored-by: José Varela <joselcvarela@gmail.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Zeel Pathak <60271095+zeel-pathak@users.noreply.github.com>
* Add black box tests
* Revert docker compose file
* Update workflow
* Try use workflow from dev repo
* Increase seedDB() timeout
* Disable other checks for now
* Change DB sequence
* Update jest moduleNameMapper
* Update workflow's docker-compose.yml path
* Slice array first
* Remove differentiation of status code
* Delete field only after foreign key constraints are removed
* Add checks for different types of primary key
* Test global query filter for all field types
* Increase timeout for m2o seeding
* Add case insensitive string operators
* Update filter check to run on relational fields
* Enable time field checks
* Add seeded random and fix relational seeding
* Add casting for integer and bigInteger
* Minor fixes
* Reduce bigInt values
* Separate seeding of DB structure from values
* Add primaryKey seeding function
* Use automatic IDs except for string pk
* Try fix ci
* Update package-lock.json
* Update common.test for concealed user tokens
* Use dynamic field type for m2o.test relational fields
* Temporary disable missing nicontains for string type
* Add support for alias type filtering
* Fix relational filter operator checks
* Add initial o2m test
* Remove integer pk limit
* Add empty checks for string and uuid null
* Limit generated integer value to 4 bytes
* Patch timezone tests for MSSQL
* Remove sample query filter test
* Fix timezone test for sqlite
* Fix MSSQL uuids
* Fix MSSQL timestamp inaccuracy
* Cast datetime schema to milliseconds for comparison
* Fix MySQL / Maria timestamp inaccuracy
* Fix MySQL / Maria between operator inconsistency for float type
* Fix missing time datatype in Oracle
* Skip filter testing on Oracle
* Enable o2m filter tests for other collections
* Run tests only on SQLite for PRs unless the Full Tests label exists
* Try fix actions
* Refactor github actions
* Update tests flow setup to use getURL()
* Start postgres docker
* Reinstate package-lock
* Fix geometry test
* Remove .gitkeep files
* Add todo.md
* Rename black box to blackbox
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Encoded the url using encodeURIComponent, so that the url gets sanitized and we did not get 404 error.
* Added required changes as stated by the reviewer
* Used encodeURI instead of encodeURIComponent to encode the params only
* Solved linting error
* Solved linting errors
* checked with npm run lint, no linting errors found
* Remove npmrc files which prevent lockfile creation in workspaces
Since `v8.5.0` npm will detect that it is running inside a workspace and issue commands at the root package.
* Require a minimum npm version of 8.5.0
* Update knex to version 2.0.0
Fixes#10842
* Remove unused optional dependencies
* Update sqlite3 to version 5.0.6
* Fix returning primary key when inserting items into database
* Add migration to remove default value from project_color
* Upgrade the default theme
* Do a pass over the header buttons
* Do another pass over --warning uses
* Little things
* Tweak smoke
* Use updated map
* Use dark system elements in dark mode
* Remove outdated files, update toasts
* Use correct delete button style for role
* Use primary for created point
* Tweak spacing in revisions section
* Use primary for sidebar sections
* Various color tweaks and changes
* Update base color palette
* Update "css reset" code
* Remove splashscreens, add default favicon
* Add primary highlight to datamodel page
* dark mode color update
* fix sign out hover color
* Does this help?!
* This fixes some loading issues
Fixes https://github.com/directus/directus/issues/10707
* Calculate default pretty background?
* Fix public view with logo
* Fix responsiveness of login page
* adjust notification group width to be equal
* Do an absolute pointless task that doesn't do anything
It was suggested that it could fix the stylelint error in tests, but alas
* Remove stylelint
Doesn't seem to play nice with script setup and other changes
Co-authored-by: Ben Haynes <ben@rngr.org>
* Add "authenticate" filter hook that allows custom auth check
* Start on test
* Update Jest, restructure API tests, start implementing authenticate test
* Move access token verify to util function
* Ensure jest can show inline warnings on correct lines
* Update is-directus-jwt to use jsonwebtoken decode + add tests
* Remove unused package
* Tweak and finish + test authenticate
* Tweak test
* Add authenticate filter to docs
* Don't scan tests for codeql
* No seriously, ignore tests
* .gitignored package-lock.json that was created in the docs subfolder
* tweaked color names, except the ones in in vue files
* converted terminal screencaps into markdown
* updated images in 'Content'
* updated user-directory images
* updated media in file-library
* moved picsto cloud in reference>files
* updated images in config > datamodels > relationships
* updated primary colors in app-overview.svg
* updated insights images
* swapped quickstart-guide media and touched up copy
* Update last colors, remove linting from docs temporarily
* Fix theme color
* Update homepage header image
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* tsconfig loosened to match api
* integration test folder and script
* .env mock isn't needed
* mock-env, locations of tests
* I was wrong.
* Updated progress
* schemas => getSchema(), exported getColumnPreprocessor()
* change imports to allow build
* add MockClient as DB Client (should mock getDBClient())
* move authservice into. not used elsewhere
* test passing
* remove unneeded test for this PR
* removed unneded fields from rawItem
* formatting
* createOne test passing
* typo
* removed int specific jest.config
* use systemSchema over getSchema()
* this.knex over getDatabase()
* mock getDatabaseClient
* removed unused mock-env
* formatting
* removed unused collections from schema
* response => responseOnce
* more direct
* unused var
* more tests please!
* test for: createOne no permissions
* removed broken test for now
* clean up
* readOne tests what does into db and returned
* added bindings check
* added into db sql check
* moved tests outside src
* changed ts config
* cli test location changed
* removed looking for tests in src
* readding it for e2e tests? Seems fishy
* Integration tests for ItemsService user tables (#11208)
* renamed ItemsService test to match src file name
* add user table schema
* system/user tables use .each, formatting needed
* format system and user better in title
* tables/table=> schemas/schema
* filter _eq test passing
* comment for later and formatting
* everything in it's right place
* future proofing as more fields are added
* removed duplicate test
* formatting
* readMany() filter: _or
Co-authored-by: Jay Cammarano <jaycammarano@gmail.com>
* added knex check before getDatabase()
* Move util test to tests folder
Co-authored-by: Jay Cammarano <jaycammarano@gmail.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Enable end-to-end tests for Oracle and SQLite
* Add Oracle back to allVendors
* Use yum
* Use dnf
* Actually use yum
* Actually use dnf
* Actually use yum
* Try again
* Try manual install
* Fix env declaration
* Bump knex version (because https://github.com/knex/knex/issues/4844)
* Set max pool size for Oracle
* Add awaitDatabaseConnection
* Cache install and build step
* Run different tests sequentially
* Fix workflow name
* Show test results
* Fix names
* Fix success check
* Fix outputs
* Add oracle driver install
* Fix env
* Revert to previous structure to benchmark performance
* Only build specs and drive packages for unit tests
* Don't install everything to run linters
* Use this branch
* Fix missing lint dep
* Revert "Don't install everything to run linters", also build shared package
* Skip app build for tests
* Don't serve app for e2e tests
* Change time fields to timestamp becaues of inconsistencies between vendors
* Make npm ci faster
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* mock-knex
* test on the migrations run started.
* test passing for run.up()
* reorganize /tests/ to allow integration tests
* e2e setup changes
* e2e jest.config moved
* e2e paths fixed, integration config
* add nonadmin role and user seed+migration
* auth/login w/ documentation (docs will be moved)
* update user seed
* add postgres10 to the ci?
* argon2 saves the day
* items tests passing with postgres10 support
* removed comments
* move generateHash out of directus_users
Co-authored-by: Jay Cammarano <jaycammarano@gmail.com>
