* Step 1
* Step 2
* False sense of confidence
* Couple more before dinner
* Update schema package
* Update format-title
* Upgrade specs file
* Close
* Replace ts-node-dev with tsx, and various others
* Replace lodash with lodash-es
* Add lodash-es types
* Update knex import
* More fun is had
* FSE
* Consolidate repos
* Various tweaks and fixes
* Fix specs
* Remove dependency on knex-schema-inspector
* Fix wrong imports of inspector
* Move shared exceptions to new package
* Move constants to separate module
* Move types to new types package
* Use directus/types
* I believe this is no longer needed
* [WIP] Start moving utils to esm
* ESMify Shared
* Move shared utils to @directus/utils
* Use @directus/utils instead of @directus/shared/utils
* It runs!
* Use correct schemaoverview type
* Fix imports
* Fix the thing
* Start on new update-checker lib
* Use new update-check package
* Swap out directus/shared in app
* Pushing through the last bits now
* Dangerously make extensions SDK ESM
* Use @directus/types in tests
* Copy util function to test
* Fix linter config
* Add missing import
* Hot takes
* Fix build
* Curse these default exports
* No tests in constants
* Add tests
* Remove tests from types
* Add tests for exceptions
* Fix test
* Fix app tests
* Fix import in test
* Fix various tests
* Fix specs export
* Some more tests
* Remove broken integration tests
These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these
* Regenerate lockfile
* Fix imports from merge
* I create my own problems
* Make sharp play nice
* Add vitest config
* Install missing blackbox dep
* Consts shouldn't be in types
tsk tsk tsk tsk
* Fix type/const usage in extensions-sdk
* cursed.default
* Reduce circular deps
* Fix circular dep in items service
* vvv
* Trigger testing for all vendors
* Add workaround for rollup
* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"
* Fix postgres
* Schema package updates
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
* Resolve cjs/mjs extensions
* Clean-up eslint config
* fixed extension concatination
* using string interpolation for consistency
* Revert MySQL optimisation
* Revert testing for all vendors
* Replace tsx with esbuild-kit/esm-loader
Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js
Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467
* Use exact version for esbuild-kit/esm-loader
* Fix import
---------
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Expose schema endpoints
* respond with 204
* fix payload for export flag
* allow export as YAML
* use SchemaService
* move getStringFromStream to utils
* clearer exceptions with logs
* check empty request body
* specs
* add bypassCache flag and remove flushCaches calls
* Identify integer typed fields in Oracle
* Fix uuid foreign keys type for Oracle
* Detect autoincrement for CockroachDB
* Bypass cache for fields and relations services
* Add same DB tests
* Test if working for all vendors
* Add schema snapshot testing into sequential flow
* Fix schema re-apply attempts when it's a different database vendor (#14816)
* prevent diffing on certain properties
* fix apply-snapshot test
* prevent updates to autoincrement fields' schema
* sanitize relations
* remove `colleciton.schema.sql` parsing in test
It is now being sanitized by `sanitizeCollection()` function in get-snapshot-diff.ts
* Change concurrency group to not cancel test running on main
* remove multipart for apply & add hash checking
* check incoming version & add force query param
* refine error message & stub for nested hash
* add `vendor` top-level property
as an additional safety net for cross db vendor applies
* sanitize generated schema snapshots
* snapshot joi validation
* diff joi validation
* minor cleanup
* extract applyDiff & use deep-diff applyChange
* use applyDiff in schema service
* Mark vendor as optional
* Update tests to apply diffs
* move force flag check into service
* Patch mssql vendor typo
* Set relation no action in cyclic relations
* Update mysql timestamp default value
* Oracle cannot define no action
* Update oracle timestamp type default values
* add hash checking for outdated diffs
* fix diff service & endpoint
* Add hashes when returning after
* Fix self referencing relations for Oracle
* Add temp fix for CURRENT_TIMESTAMP defaults in Oracle
* clean up driver and database client types
* only require diff hash if not kind N
* update hash comparison logic for create/delete
* Set no action constraint for self referencing M2M in MSSQL
* Add basic hash check tests
* omit default value when hashing auto increments
Specifically for CockroachDB with differing sequences
* add vendor check
* update specs
* Validate vendors with type definition
* Spread the vendors input array
* re-add Export component
* re-add js-yaml to root
* Propagate mutation options for schema apply
* Verify that snapshots differ and clear cache before hash tests
* Fix unit test
* Revert temp fix for CURRENT_TIMESTAMP defaults in Oracle
* Define and reuse type for export format
* Define and reuse list of database clients
* change `were` to `was`
* change `where` to `were`
* add some empty lines for readability
* tweak exception message
* fix test
* use object-hash
* use boolean to check whether file is included
* simplify request content type check
* throw error when multiple files were uploaded
* use nullish coalesce instead of short circuit
* Update api/src/services/schema.ts
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Remove unnecessary `Omit` on `SnapshotWithHash` type
* Revert "Remove unnecessary `Omit` on `SnapshotWithHash` type"
This reverts commit d22ac771ec.
* check empty snapshot earlier
* use allow-list logic via pick instead of omit
* Update api/src/services/schema.ts
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Move to own /schema endpoint controller
* Fix refs to schema endpoint
* move streamToString to utils package
* move get-versioned-hash and add test
* extract kind into an enum
* Fix mysql5 timestamp value
* Fix test collection existing on rerun
* resolve TODO comment in blackbox test
* Drop deep level hashes in diff
These hashes are used only for more accurate error reporting
but are otherwise superfluous, since changes are already
detected by the top level hash.
Therefore we remove them in favor of a simpler diff format and
implementation.
* Revert schema "fix" for createItem, add comment
* Strict diff schema validation
* Revert CrDB auto-increment detection patch in 816c998
* Clear systemCache to expose newly added fields
* Use DiffKind constants
* Extract diff & snapshot validation into own tested utils
* Apply suggestions from @azrikahar
* Update knex-schema-inspector to 3.0.1
Includes the fix for CrDB auto-increment detection (knex/knex-schema-inspector#135)
* Update knex-schema-inspector in packages
* Update lock file
* add test for schema service
* add test for export service
* add relevant tests to modified util functions
* fix csv test to account for os end of line
* fix files controller test
* dedupe test data for schema service
* Align schema specs with docs
* Update api/src/controllers/schema.ts
* Revert testing for all vendors
---------
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
* Add sort within inner query
* Run tests for all vendors
* Apply limit to all queries
* Enable sorting in MSSQL subquery by setting a limit
* Add tests
* Parse function field names for a2o
* Improve order string spacing
* Update unit test
* Fix mysql5 tests
* Revert testing for all vendors
* Extract original collection name
* Add comment for MSSQL applyLimit helper
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
* Speed query up by reusing existing aliases which reduces table joins
* Use subquery in top level m2o to remove duplicates
* Fix linting
* Apply distinct on primary key field in subqueries
* Use distinct instead as there are only primary keys
* Apply subquery on top level
* Try remove sub sub query
* Test if working for all vendors
* Add support for _none and _some
* Use subquery only when field depth > 1
* Add tests
* Use original table names for columns with functions (#14690)
* Use original table names for columns with functions
* Extract filter function path parsing as shared util
* Fix filter function path when adding node
* Pass the originalCollectionName into filter functions
* Update unit test
* Replace functions within deep GraphQL
* Fix invalid operator error for _none and _some
* Add filter function tests
* Revert triggering for all vendors
* Simplify aliasMap
* Replace functions in filter within GraphQL aggregate query
* Add API support for filtering of alias field
* Mark schema as optional
* Shift logical operators upwards
* Separate recursive parseFilter
* Rework shifting of logical operators
* Error on invalid usage of _none and _some
* Use inner join to preserve sort order
* Run tests for all vendors
* Reuse aliasMap for sort and filter
* Sort on top level query
* Remove unnecessary limit on wrapper query
* Refactor applyQuery options
* Remove duplicates from nested multi relational sort
* Fix offset in MSSQL requiring OrderBy
* Disable schema cache
* Use inner query only for nested sort or multi relational filter
* Fix MSSQL duplicate order column
* Use inner query only for multi relational
* Additional integration tests
* Order within partition for multi relational sorts
* Rename to directus_row_number
* Fix unit test
* Add base sort and filter tests
* Fix Oracle uppercased rowNumber column
* Fix unit test
* Fix top level query sort with function
* Parse functions in inner query
* Increase clarity with knex.ref()
* Remove sort filter for top level primary key
* Fix unit test
* Bypass queries with groupBy
* Add collection to aliasMap to fix functions in nested sort
* Fix multi relational sort with functions
* Add tests for filter and sort with functions
* Fix accidental deletion of brackets
* Fix top level alias filter node interface
* Update M2M sort tests
* Add M2A tests
* Cast m2a primary key as varchar2 for oracle
* Enable filtering tests for M2A
* Fix prototype polluting assignment in aliasMap
* Remove unnecessary currentKey
* Simplify code to increase readability
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
* Fix linting and missing 'this' error
* Revert optional chaining
* Add mysql5 to tests
* Fix mysql5 missing rowNumber()
* Overcome indexing delays in MySQL5
* Verify MySQL5 sorting is in order as the result count varies between runs
* Skip joining when sorting field already exists
* Simplify variable assignment
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
* Fix linting
* Reduce duplicate logic with vars
* Transform _func fields in GraphQL only for valid functions
* Fix unit test
* Fix unsupported date_part() in CrDB
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* add forced uppercase for UUIDs in MS SQL
* test whether mssql returned uuid is uppercase
* use once just for slight better correctness
* once is actually not enough
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
* Fix date functions for databases not in UTC timezone
* Fix passing raw date objects to DB
* Add unit test
* Use global timezone for mysql and maria
* Fix unit test timezone inconsistency as date object is in local timezone
* Date and datetime fields should be in local timezone
* Fix missing padStart in local string
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* use a different constraint name whyen recreating constraints for cockroachdb
* implement cockroach specific logic in the appropriate helper
Co-authored-by: ian <licitdev@gmail.com>
* Disable foreign check outside the trx in SQLite
* Update test
* Test if working for all vendors
* Fix sequential tests flow
* Revert triggering for all vendors
* Change notifications timestamp to nullable
* check special flags that generate data
* create and use new updateType helper
* update migration identifier
* use changeToType() where applicable
* fix method name used in new migration
* bump identifier again
* Remove changeToString
* update migrations
* add conditional text/string type for compatibility
* bump migration identifier
* bump
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
* Process relational collection's permissions in functions
* Fix deep usage of functions in GraphQL
* Add unit test for updated SQL query with permissions
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Fix range to be based on current time
* Parse using localtime for fields without timezone data on SQLite
* Add tests
* Parse test response to int
* Refactor to options object
* Read timestamps as UTC for Oracle
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
* Remove UTC conversion from date, time and datetime fields
* Fix mysql / maria timestamps when not in UTC timezone
* Add sequential tests with node timezone change
* Increase connection attempt check
* Add error catching in tests flow setup
* Check for server connectivity
* Fix promises
* Fix timestamp inconsistencies
* Revert to previously used parseISO
* Ensure database and directus connection
* Add another timezone to test positive and negative GMT
* Set local server hostname to localhost
* Add tests for SQLite
* Use notNullable primary key
* Revert connection testing sleep duration
* Fix nested transactions on SQLite
* Increase MSSQL request timeout
* Add type override flag for SQLite
* Remove commented code
* Add type override flags for Oracle
* Updated test file path
* Increase test servers launch timeout
* Increase test servers launch timeout
* Update format of tests
* Fix typo
* Increase test timeout for CockroachDB
* Add type overrides when creating fields through collections service
* Remove time field type conversion for Oracle
* Update collections list in test
* Remove check for time field in Oracle
* Add missing continue...
* Remove database override
* Add note for SQLite
* Rename flags and extract shared util
* Abstract remaining DB specific checks
* Revert flags renaming except renaming of test files
* Use date helper to add field flag
* Move field date typecasting upstream
* Use timestamp helper for date-created and date-updated
* Fix tests
* Remove unused vars
* Add tests for date-created
* Increase connection attempt count
* Fix test for mariadb
* Increase allowable difference to account for delays
* Add tests for date-updated
* Fix tests again for mariadb
* Add date helpers post merge
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Rename date functions to fn, add json_array_length for pg
* Add json count to mssql
* Add json array count support to other vendors
* Add UI for selecting API functions
* Make it not break
* Render functions in filter preview better
* Include functions in field altering
* Add schema access to database helper
* Allow filtering against o2m/m2m/m2a count
* Add data function execution helpers in utils
* Fix type issue
* Inject function results in validate step
* Render field keys with function names translated
* Allow selecting nested/functions in field validation step
* Make sure number comparisons are treated as numbers
* Add check if instanceof date when casting to a Number
* Prevent selecting foreign keys for junction sort (#12463)
* [SDK] Add further request options to `items` functions (#12503)
* add possibility to set further options to the request
* fix options type
* add typings to interface
* add test if headers are passed thourght
* create reusable options param
* set higher priority to options param
* Small stylistic cleanup
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Jürg Hunziker <juerg.hunziker@gmail.com>
* base changes for cockroachdb
* allow creating of tables
* allow deleting of fields
* allow deleting of tables
* rebase
* fix migrations
* bump knex-schema-inspector to 1.7.0
* Update package-lock
* Add cockroach to debugging docker-compose file
* Remove unused import
* Tweak name in example.env
* Force nullable primary keys in cockroach
* Tweak shares migration to run on cockroach
* Rename var for clarification
* suggestion for migration helper
* change to schema and update remaining migrations
* Remove custom cockroach schema in favor of sharing with pg
* Fix migrations for CockroachDB
* Hopefully fix Oracle migrations 🤞🏻
* Make ~~aiden~~ oracle happy
* Resolve branching paths in migrations
* Enable tests for cockroach
* Fix test config
* One more config change for good measure
* Adjust test to match cockroach's bigint auto-int structure
* Increase request timeout for mssql
* Update api/src/database/helpers/schema/types.ts
Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Oreille <33065839+Oreilles@users.noreply.github.com>
Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>
* Update geometric types and patch new field flow.
* Add migration
* Fixed migrations
* Also fixed migrations
* Update migration ID
* Cleanup type selector a bit
* Add missing fallback interface/display for new types
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
* Remove advanced filter sidebar detail
So long, and thanks for all the fish.
* Remove filter conversion logic
* Start replacing/removing old skool filters
* Add inline mode for usages in search bar
* Make filter work in header bar
* Emit empty string as null in filter
* Move shared filter types to shared
* Upgrade use-items
* Fix manual sort on tabular
* Cleanup styling in search bar usage
* Tweak styling
* Fix filtering issues
* Update cards
* Remove activeFilterCount from tabular
* Update maps to work with new filters
* Update calendar to new filter/sort structure
* Fix activity module nav/search
* Fix no-results message
* Update file library filtering
* Finalize user search
* Allow filtering in drawer-collection
* Handle cancelled responses semi-gracefully
* Add loading start state timeout
* Replace sort type in api
* Last commit before redoing a bunch
* Finish new visual style
* Remove unused rounded prop from v-menu
* Tweak sizing
* Enough size tweaking for now
* Count all filter operators instead of top
* Fix archive casting
* Fix api build
* Add merge filters util
* Split filter in user vs system
* Fix export sidebar detail
* Show field label on permissions configuration
* Add migration for filter/sort
* Use filters in insights
* Added map layout
* Cleanup and bug fixes
* Removed package-lock
* Cleanup and fixes
* Small fix
* Added back package-lock
* Saved camera, autofitting option, bug fixes
* Refactor and ui improvements
* Improvements
* Added seled mode
* Removed unused dependency
* Changed selection behaviour, cleanup.
* update import and dependencies
* make custom style into drawer
* remove unused imports
* use lodash functions
* add popups
* allow header to become small
* reorganize settings
* add styling to popup
* change default template
* add projection option
* add basic map interface
* finish simple map
* add mapbox style
* support more mapbox layouts
* add api key option
* add mapbox backgrounds to layout
* warn when no api key is set
* fix for latest version
* Improved map layout and interface, bug fixes, refactoring.
.
.
* Added postgis geometry format, added marker icon shadow
* Made map buttons bigger and their icons thinner. Added transition to header bar.
* Bug fixes and error handling in map interface.
* Moved box-select control out of the map component. Removed material icons sprite and use addImage for marker support.
* Handle MultiGeometry -> Geometry interface error.
* Removed hardcoded styles. Added migrations for basemap column. Lots of refactoring.
Removed hardcoded styles. Added migrations for basemap column. Lots of refactoring.
* Fixed style reloading error. Added translations.
* Moved worker code to lib.
* Removed worker code. Prevent Mapbox from removing access_token from the URL.
* Refactoring.
* Change basemap selection to in-map dropdown for layout and interface.
* Touchscreen selection support and small fixes.
* Small change.
* Fixed unused imports.
* Added support for PostgreSQL identity column
* Renamed migration. Added crs translation.
* Only show fields using the map interface in the map layout.
* Removed logging.
* Reverted Dockerfile change.
* Improved crs support.
* Fixed translations.
* Check for schema identity before updating it.
* Fixed popup not updating on feature hover.
* Added feature hover styling. Fixed layer customization input. Added out of bounds error handling.
* Added geometry type and support for database native geometries.
* Fixed linting.
* Fixed layout.
* Fixed layout.
* Actually fixed linting
* Full support for native geometries
Fixed basemap input
Improved feature popup on hover
Locked interfaced support
* Fixed geometryType option not updating
* Bug fixes in interface
* Fixed crash when empty basemap settings. Fixed fitBounds option not updating.
* Added back storage type option. Improved interface behaviour.
* Dropped wkb because of vendor inconsistency with binary data
* Updated layout to match new geometry type. Fixed geojson payload transform.
* Added missing geometry_format attributes to local types.
* Fixed typos & refactoring
* Removed dependency on proj4
* Fix error when empty map interface options
* Set geometry SRID to 4326 when inserting into the database
* Add support for selectMode
* Fix error on initial source load
* Added geocoder, use GeoJSON for api i/o, removed geometry_format option, refactoring
* Added geometry intersects filter. Created geometry helper class.
* Fix error when null geometryOptions, added mapbox_key setting.
* Moved all geometry parsing/serializing into processGeometries in `payload.ts`. Fixed type errors.
* Migrate to Vue 3
* Use wellknown instead of wkx
* Fixed basemap selection.
* Added available operator for geometry type
* Added nintersects filter, fixed map interface for filter input
* Added intersects_bbox filter & bug fixes.
* Fixed icons rendering
* Fixed cursor icon in select mode
* Added geometry aggregate function
* Fixed geometry processing bug when imported from relational field.
* Fixed error with geocoder instanciation
* Removed @types/maplibre-gl dependency
* Removed fitViewToData options
* Merge remote-tracking branch 'upstream/main' into map-layout
* Fixed style and geometryType in map interface options
* Fixed style change on map interface.
* Improved fitViewToData behaviour
* Fixed type imports and previous merge conflict
* Fixed linting
* Added available operators
* Fix and merge migrations
* Remove outdated p-queue dep
* Fix get-schema column extract
* Replace pg with postgis for local debugging
* Re-add missing import
* Add mapbox as a basemap when key exists
* Remove unused tz flag
* Process delta in payloadservice
* Set default map, add limit number styling
* Default display template to just PK
* Tweak styling of error dialog
* Fix method usage in helpers
* Move sdo_geo to oracle section
* Remove extensions from ts config exclude
* Move geo types to shared, remove _Geometry
* Remove unused type
* Tiny Tweaks
* Remove fit to bounds option in favor of on
* Validate incoming intersects query
* Deepmap filter values
* Add GraphQL support
* No defaultValue for geometryType
* Resolve c
* Fix translations
Co-authored-by: Nitwel <nitwel@arcor.de>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
