* Step 1
* Step 2
* False sense of confidence
* Couple more before dinner
* Update schema package
* Update format-title
* Upgrade specs file
* Close
* Replace ts-node-dev with tsx, and various others
* Replace lodash with lodash-es
* Add lodash-es types
* Update knex import
* More fun is had
* FSE
* Consolidate repos
* Various tweaks and fixes
* Fix specs
* Remove dependency on knex-schema-inspector
* Fix wrong imports of inspector
* Move shared exceptions to new package
* Move constants to separate module
* Move types to new types package
* Use directus/types
* I believe this is no longer needed
* [WIP] Start moving utils to esm
* ESMify Shared
* Move shared utils to @directus/utils
* Use @directus/utils instead of @directus/shared/utils
* It runs!
* Use correct schemaoverview type
* Fix imports
* Fix the thing
* Start on new update-checker lib
* Use new update-check package
* Swap out directus/shared in app
* Pushing through the last bits now
* Dangerously make extensions SDK ESM
* Use @directus/types in tests
* Copy util function to test
* Fix linter config
* Add missing import
* Hot takes
* Fix build
* Curse these default exports
* No tests in constants
* Add tests
* Remove tests from types
* Add tests for exceptions
* Fix test
* Fix app tests
* Fix import in test
* Fix various tests
* Fix specs export
* Some more tests
* Remove broken integration tests
These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these
* Regenerate lockfile
* Fix imports from merge
* I create my own problems
* Make sharp play nice
* Add vitest config
* Install missing blackbox dep
* Consts shouldn't be in types
tsk tsk tsk tsk
* Fix type/const usage in extensions-sdk
* cursed.default
* Reduce circular deps
* Fix circular dep in items service
* vvv
* Trigger testing for all vendors
* Add workaround for rollup
* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"
* Fix postgres
* Schema package updates
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
* Resolve cjs/mjs extensions
* Clean-up eslint config
* fixed extension concatination
* using string interpolation for consistency
* Revert MySQL optimisation
* Revert testing for all vendors
* Replace tsx with esbuild-kit/esm-loader
Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js
Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467
* Use exact version for esbuild-kit/esm-loader
* Fix import
---------
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Add `getMilliseconds` util for safer value interpretation
* Test more data types
* Remove remnant
* Customizable fallback with default of undefined
* Clean-up
* Transform getMilliseconds to named export
---------
Co-authored-by: ian <licitdev@gmail.com>
* EXTENSIONS_CACHE_TTL: Add to allow list & decouple from other cache envs
* Remove unnecessary import
* Broaden getCacheControlHeader function, use it for assets & extensions
* Add unit tests
* Apply suggestions from code review
Consistent lowercase "cache-control" in test description
---------
Co-authored-by: ian <licitdev@gmail.com>
* Expose schema endpoints
* respond with 204
* fix payload for export flag
* allow export as YAML
* use SchemaService
* move getStringFromStream to utils
* clearer exceptions with logs
* check empty request body
* specs
* add bypassCache flag and remove flushCaches calls
* Identify integer typed fields in Oracle
* Fix uuid foreign keys type for Oracle
* Detect autoincrement for CockroachDB
* Bypass cache for fields and relations services
* Add same DB tests
* Test if working for all vendors
* Add schema snapshot testing into sequential flow
* Fix schema re-apply attempts when it's a different database vendor (#14816)
* prevent diffing on certain properties
* fix apply-snapshot test
* prevent updates to autoincrement fields' schema
* sanitize relations
* remove `colleciton.schema.sql` parsing in test
It is now being sanitized by `sanitizeCollection()` function in get-snapshot-diff.ts
* Change concurrency group to not cancel test running on main
* remove multipart for apply & add hash checking
* check incoming version & add force query param
* refine error message & stub for nested hash
* add `vendor` top-level property
as an additional safety net for cross db vendor applies
* sanitize generated schema snapshots
* snapshot joi validation
* diff joi validation
* minor cleanup
* extract applyDiff & use deep-diff applyChange
* use applyDiff in schema service
* Mark vendor as optional
* Update tests to apply diffs
* move force flag check into service
* Patch mssql vendor typo
* Set relation no action in cyclic relations
* Update mysql timestamp default value
* Oracle cannot define no action
* Update oracle timestamp type default values
* add hash checking for outdated diffs
* fix diff service & endpoint
* Add hashes when returning after
* Fix self referencing relations for Oracle
* Add temp fix for CURRENT_TIMESTAMP defaults in Oracle
* clean up driver and database client types
* only require diff hash if not kind N
* update hash comparison logic for create/delete
* Set no action constraint for self referencing M2M in MSSQL
* Add basic hash check tests
* omit default value when hashing auto increments
Specifically for CockroachDB with differing sequences
* add vendor check
* update specs
* Validate vendors with type definition
* Spread the vendors input array
* re-add Export component
* re-add js-yaml to root
* Propagate mutation options for schema apply
* Verify that snapshots differ and clear cache before hash tests
* Fix unit test
* Revert temp fix for CURRENT_TIMESTAMP defaults in Oracle
* Define and reuse type for export format
* Define and reuse list of database clients
* change `were` to `was`
* change `where` to `were`
* add some empty lines for readability
* tweak exception message
* fix test
* use object-hash
* use boolean to check whether file is included
* simplify request content type check
* throw error when multiple files were uploaded
* use nullish coalesce instead of short circuit
* Update api/src/services/schema.ts
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Remove unnecessary `Omit` on `SnapshotWithHash` type
* Revert "Remove unnecessary `Omit` on `SnapshotWithHash` type"
This reverts commit d22ac771ec.
* check empty snapshot earlier
* use allow-list logic via pick instead of omit
* Update api/src/services/schema.ts
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Move to own /schema endpoint controller
* Fix refs to schema endpoint
* move streamToString to utils package
* move get-versioned-hash and add test
* extract kind into an enum
* Fix mysql5 timestamp value
* Fix test collection existing on rerun
* resolve TODO comment in blackbox test
* Drop deep level hashes in diff
These hashes are used only for more accurate error reporting
but are otherwise superfluous, since changes are already
detected by the top level hash.
Therefore we remove them in favor of a simpler diff format and
implementation.
* Revert schema "fix" for createItem, add comment
* Strict diff schema validation
* Revert CrDB auto-increment detection patch in 816c998
* Clear systemCache to expose newly added fields
* Use DiffKind constants
* Extract diff & snapshot validation into own tested utils
* Apply suggestions from @azrikahar
* Update knex-schema-inspector to 3.0.1
Includes the fix for CrDB auto-increment detection (knex/knex-schema-inspector#135)
* Update knex-schema-inspector in packages
* Update lock file
* add test for schema service
* add test for export service
* add relevant tests to modified util functions
* fix csv test to account for os end of line
* fix files controller test
* dedupe test data for schema service
* Align schema specs with docs
* Update api/src/controllers/schema.ts
* Revert testing for all vendors
---------
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Refactor storage logic to testable setup
* Add tests to get storage config
* Add tests for register drivers
* Tests for index
* Organize imports
* Add missing getStorage
* Setup boilerplate for cloudinary drive
* Add build script
* Add cloudinary configuration
* WIP tweaks for R&D
* Start storage abstraction v2
* Make storage manager single file
* Add test coverage
* Organize imports
* Setup local driver boilerplate
* [WIP] Start on local driver 2.0
* Add more methods
* Lunchtime
* Add put method
* Add list method
* [WIP] Try using storage in api
* Use node16 module-resolution
* Lets not mess with apply-query rn
* I love CJS. Death to CJS.
* Use dynamic imports
* Make things work
* Add path normalizer
* Add azure storage driver
* Update lock
* Start on tests
* Add getBuffer test
* Add getStat tests
* Add tests for exists
* Add tests for move
* Add tests for copy
* Add tests for put
* Add tests for delete
* Add test coverage for list
* Add removeLeading option to normalizePath
* Use removeLeading option
* Start on gcs
* Add fullpath test
* Add getStream
* Add getBuffer
* Add getStat
* Add exists
* Add move
* Add copy
* Add put
* Add delete
* Finish DriverGCS
* Cleanup tests a bit
* Start s3
* Add getStream
* Add getBuffer
* Please Wryn
* Add #exists
* Use randomize data
* No more hardcoded values 🙅♀️
* Add tests for copy
* Add tests for put
* Add put/copy/delete
* Add tests for delete
* WIP list
* Remove duplicate fullPath check
* Finish AWS tests
* Listen to wryn
* Mission critical tweak
* Add randomization, cleanup tests
* Check src vs dest full path
* Start on Cloudinary
* Add parameter signature helper
* Fix ESM building of shared
* Fix ESM building of shared
* Improve tests
* Update pnpm
* Remove old build commands
* Generated d.ts files
* Fix export naming
* Move ESM only utils to new @directus/utils
* Update lockfile
* Fix type exports
* Implement getStream
* Cleanup tests
* Simplify api
* Simplify API
* Simplify api
* Simplify API
* Add read/stat
* Cleanup / add exists
* Add move
* Add write
* Move uploadChunk to separate method
* Add test for #uploadChunk
* Add tests for write
* Add copy
* Add delete
* Add list
* Add list error handling
* Remove old drive packages
* Start updating API usage of storage
* Use Readable instead of NodeJS.ReadableStream
* Use readable instead of buffer
* Restore shared to main
* Update lockfile
* Use a streaming pipeline for sharp metadata
* Add basic e2e test for local upload and delete
* Fix integer filesize in SQLite
* fixed environment mocking in unit tests
* applied the same env mocking to other tests
* Update api/src/storage/register-drivers.ts
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Use sha256 by default
* Add base read test for /assets
* Replace exifr with exif-reader
* Fix tests for storage/index
* Install faking lib for tests
* Add test for register-drivers
* Add tests for register-locations
* Finish tests
* Organize imports
* Resolve Azris comments
* Fix outdated tests
Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..
* Test for sha256
* Attempt 1 at fixing toString errorr
I'm not seeing this issue locally, so we'll spam a couple commits here to get things going
* Use node 18 in tests?!
* Fix localhost resolution with 127.0.0.1
* Mock getEnv()
* Use @directus/tsconfig instead of duplicated conf
* Does this fix it?
* OK fun detour
* Recreate lockfile
* Update config files
* Use multipart uploads in S3
* Cleanup imports
* File Storage 2.0: Make metadata extraction backward-compatible (#16868)
* Reinstall packages using pnpm instead of manually removing them (#16871)
* Added extra environment setting for sharp processing of invalid images (#16811)
* Added extra environment setting for sharp processing of invalid images
* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`
* Remove unused excludes from tsconfig
* Remove copy/paste leftover
* Update packages/utils/readme.md
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Update packages/utils/package.json
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
* Propagate mutation options for schema apply
* Fix unit test
* Add bypassCache flag and remove flushCaches calls
* remove accountability option from getSchema
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
* build: ➕ add vitest and update test scripts
* build: 🔧 add vitest config
* build: ✅ Migrate tests to vitest
Remove jest references from test api test files and replace with vitest equivalents.
Tests: 13 tests are failing.
* build: 🚚 move vite.config.ts to api/src folder
* build: 🔥 remove unused vitest.config from api root
* build: ✅ import vitest modules for tests
* build: ✅ add type conversion for actual object
* Finish switch from Jest to Vitest in API
* Replace some leftovers
* Load "sharp" before tests
* Try with cjs
* Temporary enable verbose reporter
* Try with globalSetup
* Fix path to globalSetup
* Provide default export in globalSetup
* Final clean-up
* Remove @vitest/ui & update vitest to 0.25.0
* Add vitest c8 coverage dependency
* Update vitest to v0.25.1
* Replace unnecessary Vitest workaround
* Rework new tests
* Resolve build errors
Co-authored-by: Dorian C Brown <brown.3794@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Utils to compress/decompress data
Gzip was chosen because we want smaller data but quick algorithm since this will be ran for every request
* Compress system cache
* Decompress system cache
* Set/Get compressed cache for individual requests
* Switch from gzip to snappy, use json compression too
* Fix cache exp set/get
* Remove unused import
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* query function added to list
* dashboard reading query, adding to object
* typecasting of filter vals needed still
* numbers accepting strings too
* json-to-graphql-query => devD
* fixed unneeded return in list index.ts
* stitching and calling but not actually calling
* calls on panel change
* query object += new panel before dashboard save
* uuid generated in app not api
* fixed panel ids in query
* fixed the tests I just wrote
* passing the query data down!
* list showing data
* objDiff test moved to test
* metric bug fixes + data
* dashboard logic
* time series conversion started
* timeseries GQL query almost there
* query querying
* chart loading
* aggregate handling improved
* error handling for aggregate+filter errors
* removed query on empty queryObj
* maybe more error handling
* more error handling working
* improvements to erorr handling
* stitchGQL() error return type corrected
* added string fields to COUNT
* pushing up but needs work
* not an endless recursion
* its not pretty but it works.
* throws an error
* system collections supported
* refactor to solve some errors
* loading correct
* metric function fixed
* data loading but not blocking rendering
* removed redundant code.
* relational fields
* deep nesting relations
* options.precision has a default
* relational fields fix. (thanks azri)
* the limit
* limit and time series
* range has a default
* datat to workspace
* v-if
* panels loading
* workspaces dont get data anymore
* package.json
* requested changes
* loading
* get groups util
* timeseries => script setup
* list => script setup
* metric => script setup
* label => script setup
* declare optional props
* loadingPanels: only loading spinner on loading panels
* remove unneeded parseDate!!
* applyDataToPanels tests
* -.only
* remove unneeded steps
* processQuery tests
* tests
* removed unused var
* jest.config and some queryCaller tests
* one more test
* query tests
* typo
* clean up
* fix some but not all bugs
* bugs from merge fixed
* Start cleaning up 🧹
* Refactor custom input type
* Small tweaks in list index
* Cleanup imports
* Require Query object to be returned from query prop
* Tweak return statement
* Fix imports
* Cleanup metric watch effect
* Tweaks tweaks tweaks
* Don't rely on options, simplify fetch logic
* Add paths to validation errors
* [WIP] Start handling things in the store
* Rework query fetching logic into store
* Clean up data passing
* Use composition setup for insights store
* Remove outdated
* Fix missing return
* Allow batch updating in REST API
Allows sending an array of partial items to the endpoints, updating all to their own values
* Add batch update to graphql
* Start integrating edits
* Readd clear
* Add deletion
* Add duplication
* Finish create flow
* Resolve cache refresh on panel config
* Prevent warnings about component name
* Improve loading state
* Finalize dashboard overhaul
* Add auto-refresh sidebar detail
* Add efficient panel reloading
* Set/remove errors on succeeded requests
* Move options rendering to shared
* Fix wrong imports, render options in app
* Selectively reload panels with changed variables
* Ensure newly added panels don't lose data
* Only refresh panel if data query changed
* Never use empty filter object in metric query
* Add default value support to variable panel
* Centralize no-data state
* Only reload data on var change when query is altered
* Fix build
* Fix time series order
* Remove unused utils
* Remove no-longer-used logic
* Mark batch update result as non-nullable in GraphQL schema
* Interim flows fix
* Skip parsing undefined keys
* Refresh insights dashboard when discarding changes
* Don't submit primary key when updating batch
* Handle null prop field better
* Tweak panel padding
Co-authored-by: jaycammarano <jay.cammarano@gmail.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
* Adds x-directus-cache response header with HIT value
* Added ENV var for cache header and MISS, UNCACHEABLE
* Rename CACHE_HEADER_KEY -> CACHE_STATUS_HEADER
Tad opinionated, but I feel like it describes the functionality slightly better
* Return miss consistently
Uncacheable doesn't quite describe what's happening. The user actively requests the cache to be skipped, which should result in an expected miss. The value isn't un-cacheable, the cache is skipped on purpose
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Use script setup
* Start on export dialog
* Use new system field interface, replace limit with numeric input
* Set placeholder
* Add sort config
* Use folder picker, correct layoutQuery use
* Add local download button
* Allow writing exports to file
* Add notification after export
* Fix sort config, use new export endpoint
* Setup notification hints
* Add information notice
* Fix local limit, cancel button
* Add (basic) docs for export functionality
* Fix json export file format
* Implement xml batch stitching
* Resolve review points
* Add "authenticate" filter hook that allows custom auth check
* Start on test
* Update Jest, restructure API tests, start implementing authenticate test
* Move access token verify to util function
* Ensure jest can show inline warnings on correct lines
* Update is-directus-jwt to use jsonwebtoken decode + add tests
* Remove unused package
* Tweak and finish + test authenticate
* Tweak test
* Add authenticate filter to docs
* Don't scan tests for codeql
* No seriously, ignore tests
* tsconfig loosened to match api
* integration test folder and script
* .env mock isn't needed
* mock-env, locations of tests
* I was wrong.
* Updated progress
* schemas => getSchema(), exported getColumnPreprocessor()
* change imports to allow build
* add MockClient as DB Client (should mock getDBClient())
* move authservice into. not used elsewhere
* test passing
* remove unneeded test for this PR
* removed unneded fields from rawItem
* formatting
* createOne test passing
* typo
* removed int specific jest.config
* use systemSchema over getSchema()
* this.knex over getDatabase()
* mock getDatabaseClient
* removed unused mock-env
* formatting
* removed unused collections from schema
* response => responseOnce
* more direct
* unused var
* more tests please!
* test for: createOne no permissions
* removed broken test for now
* clean up
* readOne tests what does into db and returned
* added bindings check
* added into db sql check
* moved tests outside src
* changed ts config
* cli test location changed
* removed looking for tests in src
* readding it for e2e tests? Seems fishy
* Integration tests for ItemsService user tables (#11208)
* renamed ItemsService test to match src file name
* add user table schema
* system/user tables use .each, formatting needed
* format system and user better in title
* tables/table=> schemas/schema
* filter _eq test passing
* comment for later and formatting
* everything in it's right place
* future proofing as more fields are added
* removed duplicate test
* formatting
* readMany() filter: _or
Co-authored-by: Jay Cammarano <jaycammarano@gmail.com>
* added knex check before getDatabase()
* Move util test to tests folder
Co-authored-by: Jay Cammarano <jaycammarano@gmail.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* set statuscode to 204 if no content is send
this is currently wrong for the DELETE Http Method in directus, which sends 200 status code although no content is served
* make linter happy
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Moved over oauth impl to new interface
* Fixed most build issues and started addind schema to auth drivers
* Finished up OAuth2 and OpenID drivers
* Removed unused migration and utils
* Fixed minor todos
* Removed old oauth flow
* Changed oauth flow to re-use refresh token
* Added new oauth frontend
* Added font awesome social icons
* Updated authentication documentation
* Update api/src/auth/drivers/oauth2.ts
* Tested implementation and fixed incorrect validation
* Updated docs
* Improved OAuth error handling and re-enabled creating users with provider/identifier
* Removed Session config from docs
* Update app/src/components/v-icon/v-icon.vue
* Removed oauth need to define default roleID
* Added FormatTitle to SSO links
* Prevent local auth without password
* Store OAuth access token in session data
* Update docs/guides/api-config.md
* Fixed copy and removed fontawesome-vue dependency
* More docs fixes
* Crucialy importend type fiks
* Update package-lock
* Remove is-email-allowed check
In favor of more advanced version based on filtering coming later
* Fix JSON type casting
* Delete unused util
* Update type signature to include name
* Add warning when code isn't found in oauth url
and remove obsolete imports
* Auto-continue on successful SSO login
* Tweak type signature
* More type casting shenanigans
* Please the TS gods
* Check for missing token before crashing
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
This fixes not being able to build the repo due to type issues
introduced by the Typescript 4.4 option "useUnknownInCatchVariables",
which is enabled by default in strict mode.
