Commit Graph

110 Commits

Author SHA1 Message Date
Samuele Zolfanelli
8d064fc646 Allow creating big integer auto-incremented primary keys in MySQL and PostgreSQL (#13301) 2022-05-16 11:12:17 -04:00
ian
e004b0d596 Delete field only after foreign key constraints are removed (#13222) 2022-05-11 10:39:18 -04:00
Azri Kahar
90b5883f49 fix api for non existent fields & collections (#12952) 2022-04-22 10:49:03 -04:00
Azri Kahar
3fb81db1c9 Fix schema apply for CURRENT_TIMESTAMP default value (#12716)
* Fix schema apply for CURRENT_TIMESTAMP default val

* add typeof string check
2022-04-13 11:18:58 -04:00
ian
68066eccb7 Remove UTC conversion from date, time and datetime fields (#10956)
* Remove UTC conversion from date, time and datetime fields

* Fix mysql / maria timestamps when not in UTC timezone

* Add sequential tests with node timezone change

* Increase connection attempt check

* Add error catching in tests flow setup

* Check for server connectivity

* Fix promises

* Fix timestamp inconsistencies

* Revert to previously used parseISO

* Ensure database and directus connection

* Add another timezone to test positive and negative GMT

* Set local server hostname to localhost

* Add tests for SQLite

* Use notNullable primary key

* Revert connection testing sleep duration

* Fix nested transactions on SQLite

* Increase MSSQL request timeout

* Add type override flag for SQLite

* Remove commented code

* Add type override flags for Oracle

* Updated test file path

* Increase test servers launch timeout

* Increase test servers launch timeout

* Update format of tests

* Fix typo

* Increase test timeout for CockroachDB

* Add type overrides when creating fields through collections service

* Remove time field type conversion for Oracle

* Update collections list in test

* Remove check for time field in Oracle

* Add missing continue...

* Remove database override

* Add note for SQLite

* Rename flags and extract shared util

* Abstract remaining DB specific checks

* Revert flags renaming except renaming of test files

* Use date helper to add field flag

* Move field date typecasting upstream

* Use timestamp helper for date-created and date-updated

* Fix tests

* Remove unused vars

* Add tests for date-created

* Increase connection attempt count

* Fix test for mariadb

* Increase allowable difference to account for delays

* Add tests for date-updated

* Fix tests again for mariadb

* Add date helpers post merge

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-04-06 16:15:22 -04:00
Rijk van Zanten
3307bed5fd Improve cache reliability in DDL operations (#12400)
* Add TTL to schema cache

* Clear caches on unexpected errors in DDL

* Consistent return value use

* Don't set a default value for schema ttl
2022-04-01 13:24:20 -04:00
ian
5068ca096b Add lock for system cache (#12017)
* Add lock for system cache

* Add lock when forcing a flush

* Simplify code

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-03-18 15:54:02 -04:00
Oreille
c0fab454fd Explicitly set column nullable when unspecified (#11572) 2022-02-10 11:41:55 -05:00
Rijk van Zanten
eee8725140 Resolve transaction unreliability issues in schema alterations (#11279)
* Start with test setup

* Reorder operations in transaction

Prevent dead roaches

* Fix test post payload

* Fix tests for real now?
2022-02-03 20:34:00 -05:00
Aiden Foxx
4e0c27db73 Fixed issue when updating columns in Oracle (#11359)
* Fixed issue when updating columns in Oracle

* Fix lint error

* Removed optional chaining
2022-01-31 13:47:06 -05:00
ian
191d72c156 Fix nested transactions on SQLite (#11227)
* Fix nested transactions on SQLite

* Increase MSSQL request timeout
2022-01-24 10:16:32 -05:00
Michael Schramm
75b5f33727 Implement CockroachDB support (#10113)
* base changes for cockroachdb

* allow creating of tables

* allow deleting of fields

* allow deleting of tables

* rebase

* fix migrations

* bump knex-schema-inspector to 1.7.0

* Update package-lock

* Add cockroach to debugging docker-compose file

* Remove unused import

* Tweak name in example.env

* Force nullable primary keys in cockroach

* Tweak shares migration to run on cockroach

* Rename var for clarification

* suggestion for migration helper

* change to schema and update remaining migrations

* Remove custom cockroach schema in favor of sharing with pg

* Fix migrations for CockroachDB

* Hopefully fix Oracle migrations 🤞🏻

* Make ~~aiden~~ oracle happy

* Resolve branching paths in migrations

* Enable tests for cockroach

* Fix test config

* One more config change for good measure

* Adjust test to match cockroach's bigint auto-int structure

* Increase request timeout for mssql

* Update api/src/database/helpers/schema/types.ts

Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Oreille <33065839+Oreilles@users.noreply.github.com>
Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>
2022-01-20 14:14:38 -05:00
John Huffsmith
e7ada1f173 Moving schema and relation types into shared package (#11179)
* moving schema and relation types into shared package

* updating SchemaOverview imports

* removing duplicate import
2022-01-20 13:36:19 -05:00
ian
5bd41bcabb Shift hook emits to field service (#9351)
* Shift hook emits to field service

* Update field hooks to use filter and action

* Allow updating filter payload in hook

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-12-01 20:14:37 +00:00
Oreille
bc864d1f51 Improve helpers structure (#10052)
* Improve helpers structure

* Added DatabaseHelper base class

* Refactor index.ts
2021-12-01 15:08:24 -05:00
Oreille
5f43b20ebf Update geometric types and patch new field flow. (#9397)
* Update geometric types and patch new field flow.

* Add migration

* Fixed migrations

* Also fixed migrations

* Update migration ID

* Cleanup type selector a bit

* Add missing fallback interface/display for new types

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-11-04 21:47:54 -04:00
Nicola Krumschmidt
3d0e086c55 Rework hook registration (#8027)
* Rework hook registration

* Remove event and action fields from hook payloads

* Move "error" action to "request.error" filter

* Emit meta and context objects in filters and actions

* Run filters sequentially

* Update hook templates

* Fix CLI hook test

* Also emit `<collection>.items.crud` when emitting `items.crud`.

* Update hook docs

Co-authored-by: Oreilles <oreilles.github@nitoref.io>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-11-03 17:18:56 -04:00
Rijk van Zanten
65291b95c4 Allow dynamic user variables to be used with filter rules (cont.) (#9376)
* Move permissions extraction to accountability

* Fix permissions retrieval for public user

* Fetch user / role context in permissions middleware

* Remove unnecessary parseFilter

* Rename schemaCache to systemCache

* Add permissions caching

* Add system cache invalidation on permission changes

* Improve caching perf by reducing scope

* Add note to docs

* Clarify compatibility with conditional fields/filters

* Fix lint warning

* Allow nested vars in system-filter-input

* Add custom getter function that resolves arrays

* Add is-dynamic-variable util

* Export new util

* Cleanup parse filter

* Fix build

* Move debounce up to use-items

* Remove unused prop

* 🧹

* Fix input pattern usage w/ vars

* Remove debounce from search-input, increase throttle
2021-11-03 16:56:16 -04:00
Nitwel
894b4975f2 Fix issues with fields config (#9324)
* Fix issues with fields config

* Fix /fields endpoint return type for alias fields

* Fix display selection in relational interfaces
2021-11-01 20:55:45 +00:00
Azri Kahar
68869474b5 fix directus_fields group cleanup on delete (#8707) 2021-10-12 09:55:40 -04:00
Alejandro Castaño González
829e996574 Exclude creation of primary and unique indexes (#8640) 2021-10-08 10:29:23 -04:00
Rijk van Zanten
ce104b6a9c Add native schema migration capabilities (#7939)
* Add snapshot creation command

* Read and start diffing snapshot

* Add apply snapshot functionality

* Fix cli invocation

* Add log messages

* Fix duplicated if check

* Add (minimal) docs on schema migrations

* Fix missing import

* Update api/src/utils/apply-snapshot.ts

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>

* Appease to Nicola's programming professor

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
2021-09-13 17:15:04 -04:00
Nicola Krumschmidt
d64ca14348 Explicitly set catch parameters to any type (#7654)
This fixes not being able to build the repo due to type issues
introduced by the Typescript 4.4 option "useUnknownInCatchVariables",
which is enabled by default in strict mode.
2021-08-27 10:33:30 -04:00
Oreille
83e8814b2d Add support for Geometry type, add Map Layout & Interface (#5684)
* Added map layout

* Cleanup and bug fixes

* Removed package-lock

* Cleanup and fixes

* Small fix

* Added back package-lock

* Saved camera, autofitting option, bug fixes

* Refactor and ui improvements

* Improvements

* Added seled mode

* Removed unused dependency

* Changed selection behaviour, cleanup.

* update import and dependencies

* make custom style into drawer

* remove unused imports

* use lodash functions

* add popups

* allow header to become small

* reorganize settings

* add styling to popup

* change default template

* add projection option

* add basic map interface

* finish simple map

* add mapbox style

* support more mapbox layouts

* add api key option

* add mapbox backgrounds to layout

* warn when no api key is set

* fix for latest version

* Improved map layout and interface, bug fixes, refactoring.

.

.

* Added postgis geometry format, added marker icon shadow

* Made map buttons bigger and their icons thinner. Added transition to header bar.

* Bug fixes and error handling in map interface.

* Moved box-select control out of the map component. Removed material icons sprite and use addImage for marker support.

* Handle MultiGeometry -> Geometry interface error.

* Removed hardcoded styles. Added migrations for basemap column. Lots of refactoring.

Removed hardcoded styles. Added migrations for basemap column. Lots of refactoring.

* Fixed style reloading error. Added translations.

* Moved worker code to lib.

* Removed worker code. Prevent Mapbox from removing access_token from the URL.

* Refactoring.

* Change basemap selection to in-map dropdown for layout and interface.

* Touchscreen selection support and small fixes.

* Small change.

* Fixed unused imports.

* Added support for PostgreSQL identity column

* Renamed migration. Added crs translation.

* Only show fields using the map interface in the map layout.

* Removed logging.

* Reverted Dockerfile change.

* Improved crs support.

* Fixed translations.

* Check for schema identity before updating it.

* Fixed popup not updating on feature hover.

* Added feature hover styling. Fixed layer customization input. Added out of bounds error handling.

* Added geometry type and support for database native geometries.

* Fixed linting.

* Fixed layout.

* Fixed layout.

* Actually fixed linting

* Full support for native geometries
Fixed basemap input
Improved feature popup on hover
Locked interfaced support

* Fixed geometryType option not updating

* Bug fixes in interface

* Fixed crash when empty basemap settings. Fixed fitBounds option not updating.

* Added back storage type option. Improved interface behaviour.

* Dropped wkb because of vendor inconsistency with binary data

* Updated layout to match new geometry type. Fixed geojson payload transform.

* Added missing geometry_format attributes to local types.

* Fixed typos & refactoring

* Removed dependency on proj4

* Fix error when empty map interface options

* Set geometry SRID to 4326 when inserting into the database

* Add support for selectMode

* Fix error on initial source load

* Added geocoder, use GeoJSON for api i/o, removed geometry_format option, refactoring

* Added geometry intersects filter. Created geometry helper class.

* Fix error when null geometryOptions, added mapbox_key setting.

* Moved all geometry parsing/serializing into processGeometries in `payload.ts`. Fixed type errors.

* Migrate to Vue 3

* Use wellknown instead of wkx

* Fixed basemap selection.

* Added available operator for geometry type

* Added nintersects filter, fixed map interface for filter input

* Added intersects_bbox filter & bug fixes.

* Fixed icons rendering

* Fixed cursor icon in select mode

* Added geometry aggregate function

* Fixed geometry processing bug when imported from relational field.

* Fixed error with geocoder instanciation

* Removed @types/maplibre-gl dependency

* Removed fitViewToData options

* Merge remote-tracking branch 'upstream/main' into map-layout

* Fixed style and geometryType in map interface options

* Fixed style change on map interface.

* Improved fitViewToData behaviour

* Fixed type imports and previous merge conflict

* Fixed linting

* Added available operators

* Fix and merge migrations

* Remove outdated p-queue dep

* Fix get-schema column extract

* Replace pg with postgis for local debugging

* Re-add missing import

* Add mapbox as a basemap when key exists

* Remove unused tz flag

* Process delta in payloadservice

* Set default map, add limit number styling

* Default display template to just PK

* Tweak styling of error dialog

* Fix method usage in helpers

* Move sdo_geo to oracle section

* Remove extensions from ts config exclude

* Move geo types to shared, remove _Geometry

* Remove unused type

* Tiny Tweaks

* Remove fit to bounds option in favor of on

* Validate incoming intersects query

* Deepmap filter values

* Add GraphQL support

* No defaultValue for geometryType

* Resolve c

* Fix translations

Co-authored-by: Nitwel <nitwel@arcor.de>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2021-08-12 22:01:34 +02:00
Rijk van Zanten
d0f6cb35e3 Support dynamic variables in conditional fields (#7202)
* Move accountability type to shared

* Rely on shared for parse-filter

* Install date-fns

* Support dynamic variables in conditional fields

Closes #7157
2021-08-05 01:51:15 +02:00
Rijk van Zanten
ce234f3165 Fix timezone problems in dateTime type (#7200)
* Disable useTZ option from MS SQL by default

* Fix recognition of OracleDB timestamp

* Smoothen out knex default in useTz for datatypes

* Fix timezone parsing for dateTime fields

* Remove unused import
2021-08-05 01:19:24 +02:00
Rijk van Zanten
9eace1c9ff Prevent duplicate alias fields from being created (#7142)
Fixes #7107
2021-08-02 22:16:13 +02:00
Rijk van Zanten
14254bae62 Fix top level perm check on nested m2a records (#7060)
Fixes #6317
2021-07-28 23:11:54 +02:00
Rijk van Zanten
92e1ee77bd Add support for Conditional Fields (#6864)
* Add conditions field to directus_fields

* Add conditions configuration

* Apply conditional overrides

* Handle conditions in nested groups

* Fix reverse mutating conditions

* Start on filter setup interface

* Move field types/constants to shared

* [WIP] Updated client side filter validation

* Support logical operators in client validation step

* Use new validation util in conditions check

* Add nesting in filter seutp

* Add filter rule setup configurator

* Fixes that should've been done in the merge

* Strip out filter-settings interface

TBD in a new PR

* Move browser to index
2021-07-26 18:02:24 -04:00
Nicola Krumschmidt
b61543c0e9 Move DeepPartial to shared (#6886)
Also cleans it up and adds support for readonly arrays.
2021-07-20 11:40:59 -04:00
Rijk van Zanten
44082c60e1 Add schema caching (#6456)
* Rework cache handler to be function export

* Add default schema caching

* Add schema cache

* Auto purge schema cache on schema change from api

* Only set last_access value on login

* Add note on schema cache setting
2021-06-22 20:50:20 -04:00
Pascal Jufer
927b9f8cc2 Fix DB column update for data types with length or boolean as default value (#6163)
* Fix DB column update for types with length or boolean as default value

* Add docs/index.json to gitignore

* Remove specificType support

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-06-09 18:30:59 -04:00
Rijk van Zanten
77e00b7db4 Don't initialize database on file require (#6003) 2021-06-02 11:41:42 -04:00
Rijk van Zanten
ba2e56e43c Fix collection delete for collections w/ m2o to them (#5809)
* Fix new-collection relational field creation

* Fix collection delete for collections w/ m2o to them

Fixes #5793
2021-05-21 13:57:42 -04:00
Rijk van Zanten
9335372400 Foreign Key Constraints (#5615)
* Bump knex-schema-inspector

* Fix cli role name attr

* Update relation type

* Restructure relations

* Restructure relations table

* Update api type for relation record

* Fetch relations in new structure

* Update schema-inspector

* Use new relations schema structure in api

* Update relations GETters

* Add default value to one deselect

* Add create relationship on existing field

* Add updating existing relationship

* Add delete relations

* Add relations query resolver

* Add graphql mutations for relations

* Fix reading from wrong name

* Fix wrong method name

* No idea why this flip flops every install

* Update relation type

* Accept null in use-collection composable

* Use new relations structure in translations

* Use new relations structure in new-collection

* Start updating field detail store

* Renames for new relations structure

* Silently ignore passed collection/field in relation update

* Fix setting pk field in m2o relational setup

* Small tweaks in o2m setup

* Fix m2m setup

* Tweak m2o setup

* Fix m2a setup

* Allow null for related collection (m2a)

* Fix languages code name

* Fix migration default value

* Fix relational cleanup in collections/fields

* Fix transaction problem in field delete

* Fix inserting relational o2m items

* Don't execute updateByQuery on empty item set

Fixes #5710, fixes #5070

* Show referential action input on m2o

* Finish language for m2o

* Show triggers config on o2m

* Delete items on one_deselect_item delete

* Fix naming, show relational trigger config on m2m

* Tweak language, add setup to m2a

* Fix linter warnings

* Add trigger setup for translations

* fix Edit non-schema relationship issue

* Sync existing on_delete triggers in o2m setup

* Add migration to setup foreign key constraints

* Update illegal FK values before setting constraint

* Fix MySQL unsigned vs not-unsigned in FK creation

* Use pretty names for labels in relational triggers

* Prefix auto-junction when system table

Fixes #5493

* Add system foreign key triggers

Fixes #5749

* Update docs
2021-05-19 12:29:16 -04:00
Matteo Gazzoni
c84c002c7b Makes primary key always not nullable (#5731) 2021-05-17 19:11:39 +00:00
Oreille
316ac9076b Check for schema identity before updating it. (#5608) 2021-05-11 14:36:33 -04:00
rijkvanzanten
ae1e22d116 Enforce new collections to have PK field
Fixes #5512
2021-05-06 11:19:51 -04:00
Rijk van Zanten
f3574deae0 Ask for value when changing nullable to not-nullable (#5400)
* Add ContainsNullValues exception abstraction

* Add dialog for null values when disabling non-null

Fixes #2934

* Add translation for CONTAINS_NULL_VALUE error

* Make dialog title translated
2021-04-30 15:27:18 -04:00
rijkvanzanten
74861e6f81 Add default display template for directus_files
Fixes #5364
2021-04-29 20:40:52 -04:00
rijkvanzanten
ba98f23d3d Fix TS errors 2021-04-29 16:18:09 -04:00
rijkvanzanten
801e868554 Fix remaining eslint errors
h/t @paescuj
2021-04-29 15:55:12 -04:00
Pascal Jufer
acd41eb0be Syntax fixes (#5367)
* Declare return types on functions

And a very few other type related minor fixes

* Minor syntax fixes

* Remove unnecessary escape chars in regexes
* Remove unnecessary awaits
* Replace deprecated req.connection with req.socket
* Replace deprecated upload with uploadOne
* Remove unnecessary eslint-disable-next-line comments
* Comment empty functions / catch or finally clauses
* Fix irregular whitespaces
* Add missing returns (null)
* Remove unreachable code
* A few logical fixes
* Remove / Handle non-null assertions which are certainly unnecessary (e.g. in
tests)
2021-04-29 12:11:43 -04:00
Rijk van Zanten
3b05372713 Finalize services API (#5170)
* Start the xOne/xMany work

* Update update/upsert

* Finish updating itemsservice

* Add comments, add nested revisions on update

* Use new internal api

* Update collectionService to one/many structure

* Move files to one/many structure

* Move permissions to one/many structure

* Move relations service to one/many structure

* Move roles to one/many structure

* Move users service over

* Move webhooks to updated structure

* Move deprecated methods to the bottom

* Replace deprecated uses

* Use new methods in controllers, add batch update/delete by query

* Use updated methods in API

* Fix email being required

* Remove unnecessary DB call

* Fix batch update/delete validation
2021-04-20 18:06:27 -04:00
rijkvanzanten
f6e0b2cc81 Clear archive_field / sort_field of directus_collections on field delete
Fixes #3737
2021-04-08 17:11:16 -04:00
Joonas Palosuo
9c621c6e20 feat: add delete event emitting to fields service (#4840)
* feat: add delete event emitting to fields service

* Inline event scope

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-04-08 16:39:31 -04:00
rijkvanzanten
eb79c00756 Don't delete columns for alias fields
Fixes #4802
2021-04-05 17:01:19 -04:00
Rijk van Zanten
f90c31b798 GraphQL 2.0 (#4625)
* Start on GraphQL "2.0", add methodnotallowed exceptoin

* Fix relative file pointer in peer dep

* [WIP] Add pre-filtered schema to SchemaOverview

* Use root schema as is, add reduce-schema util

* Use reduceSchema in the wild

* Base schema on local reduced schema

* Remove todo

* Use graphql-compose to build out schema

* Start restructuring resolvers

* Add create mutation

* Return boolean true for empty create mutation selections

* Add update mutation

* Add delete mutation

* Add system/items scoping

* Fix merge conflicts for real now

* Use system services, rename ids->keys

* Start on docs on mutations

* Updates to match main

* Add fetch-by-id

* Add one/many resolvers for mutations

* Check system collection rows for singleton

* Fix resolver extraction for single read

* Share delete return type

* Add comments

* Use collection root name for readable type

* Add specs endpoint for GraphQL SDL

* Update docs

* Add note on SDL spec

* Fix delete single example

* Remove package-lock

* Fix collection read scoping in non-read
2021-03-30 17:06:35 -04:00
rijkvanzanten
ef9f281c3c Fix unique constraint error on field create/update
Fixes #4481
2021-03-11 18:11:58 -05:00
Rijk van Zanten
4248b187bb Implement unique constraint support (#4467)
* Allow creating unique fields

* Allow removing unique constriant

* Show unique constraint error as validation error in app
2021-03-10 17:35:21 -05:00