Commit Graph

63 Commits

Author SHA1 Message Date
ian
fdf0fa2fb8 Add max batch mutation (#17535)
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-04-07 18:25:25 +02:00
Rijk van Zanten
2983e61870 The Great TypeScript Modernization Program Season 3 Episode 6: The Big One (#18014)
* Step 1

* Step 2

* False sense of confidence

* Couple more before dinner

* Update schema package

* Update format-title

* Upgrade specs file

* Close

* Replace ts-node-dev with tsx, and various others

* Replace lodash with lodash-es

* Add lodash-es types

* Update knex import

* More fun is had

* FSE

* Consolidate repos

* Various tweaks and fixes

* Fix specs

* Remove dependency on knex-schema-inspector

* Fix wrong imports of inspector

* Move shared exceptions to new package

* Move constants to separate module

* Move types to new types package

* Use directus/types

* I believe this is no longer needed

* [WIP] Start moving utils to esm

* ESMify Shared

* Move shared utils to  @directus/utils

* Use @directus/utils instead of @directus/shared/utils

* It runs!

* Use correct schemaoverview type

* Fix imports

* Fix the thing

* Start on new update-checker lib

* Use new update-check package

* Swap out directus/shared in app

* Pushing through the last bits now

* Dangerously make extensions SDK ESM

* Use @directus/types in tests

* Copy util function to test

* Fix linter config

* Add missing import

* Hot takes

* Fix build

* Curse these default exports

* No tests in constants

* Add tests

* Remove tests from types

* Add tests for exceptions

* Fix test

* Fix app tests

* Fix import in test

* Fix various tests

* Fix specs export

* Some more tests

* Remove broken integration tests

These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these

* Regenerate lockfile

* Fix imports from merge

* I create my own problems

* Make sharp play nice

* Add vitest config

* Install missing blackbox dep

* Consts shouldn't be in types

tsk tsk tsk tsk

* Fix type/const usage in extensions-sdk

* cursed.default

* Reduce circular deps

* Fix circular dep in items service

* vvv

* Trigger testing for all vendors

* Add workaround for rollup

* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"

* Fix postgres

* Schema package updates

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Resolve cjs/mjs extensions

* Clean-up eslint config

* fixed extension concatination

* using string interpolation for consistency

* Revert MySQL optimisation

* Revert testing for all vendors

* Replace tsx with esbuild-kit/esm-loader

Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js

Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467

* Use exact version for esbuild-kit/esm-loader

* Fix import

---------

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-04-04 17:41:56 -04:00
Rijk van Zanten
80f4807a09 TS Config Modernization Program Part 3 of many (#17904)
* noImplicitOverride: true

* noImplicitReturns: true

* noPropertyAccessFromIndexSignature: true
2023-03-23 16:47:55 -04:00
rijkvanzanten
5247b68afa Updates 6 of 6 2023-03-22 17:23:30 -04:00
Rijk van Zanten
1a5f06c791 TSConfig Modernization: Set importsNotUsedAsValues (#17887)
* Set importsNotUsedAsValues

* Fix import
2023-03-22 14:51:27 -04:00
Azri Kahar
89fcb19a16 Expose schema endpoints (#14713)
* Expose schema endpoints

* respond with 204

* fix payload for export flag

* allow export as YAML

* use SchemaService

* move getStringFromStream to utils

* clearer exceptions with logs

* check empty request body

* specs

* add bypassCache flag and remove flushCaches calls

* Identify integer typed fields in Oracle

* Fix uuid foreign keys type for Oracle

* Detect autoincrement for CockroachDB

* Bypass cache for fields and relations services

* Add same DB tests

* Test if working for all vendors

* Add schema snapshot testing into sequential flow

* Fix schema re-apply attempts when it's a different database vendor (#14816)

* prevent diffing on certain properties

* fix apply-snapshot test

* prevent updates to autoincrement fields' schema

* sanitize relations

* remove `colleciton.schema.sql` parsing in test

It is now being sanitized by `sanitizeCollection()` function in get-snapshot-diff.ts

* Change concurrency group to not cancel test running on main

* remove multipart for apply & add hash checking

* check incoming version & add force query param

* refine error message & stub for nested hash

* add `vendor` top-level property

as an additional safety net for cross db vendor applies

* sanitize generated schema snapshots

* snapshot joi validation

* diff joi validation

* minor cleanup

* extract applyDiff & use deep-diff applyChange

* use applyDiff in schema service

* Mark vendor as optional

* Update tests to apply diffs

* move force flag check into service

* Patch mssql vendor typo

* Set relation no action in cyclic relations

* Update mysql timestamp default value

* Oracle cannot define no action

* Update oracle timestamp type default values

* add hash checking for outdated diffs

* fix diff service & endpoint

* Add hashes when returning after

* Fix self referencing relations for Oracle

* Add temp fix for CURRENT_TIMESTAMP defaults in Oracle

* clean up driver and database client types

* only require diff hash if not kind N

* update hash comparison logic for create/delete

* Set no action constraint for self referencing M2M in MSSQL

* Add basic hash check tests

* omit default value when hashing auto increments

Specifically for CockroachDB with differing sequences

* add vendor check

* update specs

* Validate vendors with type definition

* Spread the vendors input array

* re-add Export component

* re-add js-yaml to root

* Propagate mutation options for schema apply

* Verify that snapshots differ and clear cache before hash tests

* Fix unit test

* Revert temp fix for CURRENT_TIMESTAMP defaults in Oracle

* Define and reuse type for export format

* Define and reuse list of database clients

* change `were` to `was`

* change `where` to `were`

* add some empty lines for readability

* tweak exception message

* fix test

* use object-hash

* use boolean to check whether file is included

* simplify request content type check

* throw error when multiple files were uploaded

* use nullish coalesce instead of short circuit

* Update api/src/services/schema.ts

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Remove unnecessary `Omit` on `SnapshotWithHash` type

* Revert "Remove unnecessary `Omit` on `SnapshotWithHash` type"

This reverts commit d22ac771ec.

* check empty snapshot earlier

* use allow-list logic via pick instead of omit

* Update api/src/services/schema.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Move to own /schema endpoint controller

* Fix refs to schema endpoint

* move streamToString to utils package

* move get-versioned-hash and add test

* extract kind into an enum

* Fix mysql5 timestamp value

* Fix test collection existing on rerun

* resolve TODO comment in blackbox test

* Drop deep level hashes in diff

These hashes are used only for more accurate error reporting
but are otherwise superfluous, since changes are already
detected by the top level hash.
Therefore we remove them in favor of a simpler diff format and
implementation.

* Revert schema "fix" for createItem, add comment

* Strict diff schema validation

* Revert CrDB auto-increment detection patch in 816c998

* Clear systemCache to expose newly added fields

* Use DiffKind constants

* Extract diff & snapshot validation into own tested utils

* Apply suggestions from @azrikahar

* Update knex-schema-inspector to 3.0.1

Includes the fix for CrDB auto-increment detection (knex/knex-schema-inspector#135)

* Update knex-schema-inspector in packages

* Update lock file

* add test for schema service

* add test for export service

* add relevant tests to modified util functions

* fix csv test to account for os end of line

* fix files controller test

* dedupe test data for schema service

* Align schema specs with docs

* Update api/src/controllers/schema.ts

* Revert testing for all vendors

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-01-31 17:36:49 +00:00
John Flow
6793d9fd40 Added https:// (#17171) 2023-01-15 11:45:45 -05:00
Azri Kahar
431f95210a align ASSETS_CACHE_TTL default value in .env stub (#16755) 2022-12-15 00:34:02 +08:00
Piotr
018452d141 Fix typo in env stub (#16717) 2022-12-05 14:45:41 +01:00
Rijk van Zanten
86dd593f91 Use node16 module-resolution (#16696)
* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use CJS require in hook import

* Fix type export in schema

* Clean up defaults usage

* Use require instead of import for migrations

* Use a vitest compatible dynamic import

* Uno mas

* Cleanup type export
2022-12-02 15:53:15 -05:00
ian
193844c106 Update MAX_PAYLOAD_SIZE env stub to 1mb (#16057) 2022-10-19 13:14:01 -04:00
José Varela
da2998a7e7 Allow custom label for auth provider (#15673)
* Allow to pass custom label for auth provider

* Add tests

* Update api/src/cli/utils/create-env/env-stub.liquid

* Fix tests

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2022-10-14 23:14:00 -04:00
Azri Kahar
f940b6a1fc correct the documented default type for argon2 (#15594) 2022-09-15 22:15:52 +08:00
Mehdi Nassim KHODJA
31a6fd208f Adding SendGrid email transport (#15384)
* Adding SendGrid email transport

* Formatting

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-09-05 10:14:41 -04:00
ian
f64e1d36d3 Expose login stall time environment variable (#15215)
* Expose login stall time environment variable

* Update env stub template

* Update description

* Increase default stall time to 500ms
2022-08-24 16:52:23 -04:00
Bjørnar Hagen
3757167d4b Fix duplicate env keys (#14845) 2022-08-05 16:34:25 -04:00
Rijk van Zanten
27cc3e1bec Move repo to pnpm based workflow (#14350)
* Use pnpm instead of npm

* Setup workspace

* Replace scripts / lerna root stuff

* Add todo file

* Update lock, start updating packages

* Update todo

* [WIP] Keep going on module resolution in pnpm

* Fix final missing deps/types for complete build

* [WIP] Replace npm commands with pnpm alternatives

* Update jest config for v28

* Fix missing package under schema

* Update workflow

* Run CI on fork

* Use local reference

* Remove version from local workflow

* Add build to prepare

* Add Node's max old space size flag to linter

* Idemdito for codeql

* Hyphens? Underscores?

* Bang

* Only build one thing at a time

* Underscores after all?

* Match 7gb of GH

* Set env on prepare

* Jest is driving me nuts

* Downgrade jest

* Add root jest

* We'll get there, eventually.

* 🥳

* Het houdt niet op, niet vanzelf

* attempt to fix e2e

* fix unit test & shared import

* add `debug: true` to CodeQL action

* Fix dev call in shared

* Add missing cross-env

* Add missing geo-json dependency for app

* add flag to prepare action to skip build

* remove debug flag from codeql action

* fix invalid env syntax

* add tslib to app dependencies for tests

* another attempt

* Added missing rimraf dependency for the api prebuild/cleanup scripts

* Added missing ts-node dependency for the api cli script

* update E2E tests readme to use pnpm

* Undo move of extensions sdk

* Fixes nested groups in accordions not rendering fields (#14369)

* initial fix for nested groups in accordions

* removed debug code

* Fix strict relative dates showing "incorrect" (#14390)

* Set rounding method of relative formatted date to floor when strict option is selected

* Add round fn as an option

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>

* Don't use locales in generated camelCased values from env (#14401)

Ref https://github.com/directus/directus/discussions/14122

* Fix list panel descending sort (#14396)

* fix list panel descending sort

* Handle missing or explicit desc value

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>

* Fix query primary field for system tables (#14402)

* New Crowdin updates (#14403)

* Update source file en-US.yaml

* New translations en-US.yaml (Spanish, Chile)

* New translations en-US.yaml (Spanish, Latin America)

* New translations en-US.yaml (Greek)

* New translations en-US.yaml (Italian)

* New translations en-US.yaml (French, Canada)

* Fix/wysiwyg context menu (#14404)

* New translations en-US.yaml (Polish) (#14407)

* Encoded the url using encodeURIComponent, so that the url gets sanitzed and so, we did not get 404 error. (#14418)

* Encoded the url using encodeURIComponent, so that the url gets sanitized and we did not get 404 error.

* Added required changes as stated by the reviewer

* Used encodeURI instead of encodeURIComponent to encode the params only

* Solved linting error

* Solved linting errors

* checked with npm run lint, no linting errors found

* fix generateJoi error due to empty permissions when creating new role (#14416)

* fix empty permissions when creating new role

* basic test

* move logic up

* additional test

* Update api/tests/utils/filter-items.test.ts

Co-authored-by: ian <licitdev@gmail.com>

Co-authored-by: ian <licitdev@gmail.com>

* Recreate lockfile

* Fix build of extensions-sdk

* Add missing dependency

* Add some more missing deps

* Update blackbox to use pnpm

* Update workflow to use main

* Update pack to work with pnpm

* Simplify commands

* Remove todo file (completed)

* ADd missing types dep

* Use local test version

* Remove version from local reference

* Call super with context

* Add missing dep

* Simplify workflows

Ref https://github.com/directus/organization/issues/135

* Simplify some more

* Linter is on root

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Gerard Lamusse <gerardlamo@gmail.com>
Co-authored-by: José Varela <joselcvarela@gmail.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Zeel Pathak <60271095+zeel-pathak@users.noreply.github.com>
2022-07-15 17:41:49 -04:00
Jay Cammarano
8a461b2ef7 add comment (#14029) 2022-06-21 17:06:57 -04:00
Eron Donevan Powell
3fa81b6b9c Docs config refactor (#13945)
* moved config-options.md to self-hosted

* added non-docs files that link to config-options

* moved SSO and fixed links

* moved upgrades-migrations into self-hosted

* moved filter-rules into reference docs and fixed broken links
2022-06-17 11:51:38 -04:00
sxsws
c42522d16b Add comments to the .env file stub (#13352)
* adding comments to env-stub 

to improve for autogenerated .env file during install - is this too verbose?

* Completing adding comments to vars

* Update api/src/cli/utils/create-env/env-stub.liquid

Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>

* Thanks to @aidenfoxx

* header changes

* updating db ports to default

* header change again

* Added meaning of values in [ ]

* remove quotes on string bools

* remove in favour of generated env stub

* remove example.env

* use online help for poss env keys

* grammar, needs a comma!

* remove trailing comma

Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>
2022-05-31 12:13:05 -04:00
Michael Lopez
c561826572 Support listen parameter (#11465)
* Support listen option for allowing server to listen on wanted IP address or host

* Change default listen value from localhost to 0.0.0.0

* Update LISTEN variable to HOST

* fix formatting

* Update docs/configuration/config-options.md

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-02-28 21:09:27 +00:00
Rijk van Zanten
88c87f3920 Set CRDB options to avoid inconsistencies between vendors (#11193)
* Set correct CRDB options

* Add missing SETs to pool config for e2e

* How about now

* Better solution for the same problem
2022-01-20 20:18:22 -05:00
rijkvanzanten
3cbe86d3f1 Mark CRDB as beta, mark oracle as ga 2022-01-20 18:29:08 -05:00
Michael Schramm
75b5f33727 Implement CockroachDB support (#10113)
* base changes for cockroachdb

* allow creating of tables

* allow deleting of fields

* allow deleting of tables

* rebase

* fix migrations

* bump knex-schema-inspector to 1.7.0

* Update package-lock

* Add cockroach to debugging docker-compose file

* Remove unused import

* Tweak name in example.env

* Force nullable primary keys in cockroach

* Tweak shares migration to run on cockroach

* Rename var for clarification

* suggestion for migration helper

* change to schema and update remaining migrations

* Remove custom cockroach schema in favor of sharing with pg

* Fix migrations for CockroachDB

* Hopefully fix Oracle migrations 🤞🏻

* Make ~~aiden~~ oracle happy

* Resolve branching paths in migrations

* Enable tests for cockroach

* Fix test config

* One more config change for good measure

* Adjust test to match cockroach's bigint auto-int structure

* Increase request timeout for mssql

* Update api/src/database/helpers/schema/types.ts

Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Oreille <33065839+Oreilles@users.noreply.github.com>
Co-authored-by: Aiden Foxx <aiden.foxx.mail@gmail.com>
2022-01-20 14:14:38 -05:00
Azri Kahar
31bb890496 Unify & translate initial role/user detail in init & bootstrap command (#10493)
* Unify initial role/user detail in init & bootstrap

* translate role descriptions

* translate data from system collections
2021-12-14 09:33:44 -05:00
Aiden Foxx
fa3b1171e8 New OpenID and OAuth2 drivers (#8660)
* Moved over oauth impl to new interface

* Fixed most build issues and started addind schema to auth drivers

* Finished up OAuth2 and OpenID drivers

* Removed unused migration and utils

* Fixed minor todos

* Removed old oauth flow

* Changed oauth flow to re-use refresh token

* Added new oauth frontend

* Added font awesome social icons

* Updated authentication documentation

* Update api/src/auth/drivers/oauth2.ts

* Tested implementation and fixed incorrect validation

* Updated docs

* Improved OAuth error handling and re-enabled creating users with provider/identifier

* Removed Session config from docs

* Update app/src/components/v-icon/v-icon.vue

* Removed oauth need to define default roleID

* Added FormatTitle to SSO links

* Prevent local auth without password

* Store OAuth access token in session data

* Update docs/guides/api-config.md

* Fixed copy and removed fontawesome-vue dependency

* More docs fixes

* Crucialy importend type fiks

* Update package-lock

* Remove is-email-allowed check

In favor of more advanced version based on filtering coming later

* Fix JSON type casting

* Delete unused util

* Update type signature to include name

* Add warning when code isn't found in oauth url

and remove obsolete imports

* Auto-continue on successful SSO login

* Tweak type signature

* More type casting shenanigans

* Please the TS gods

* Check for missing token before crashing

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-10-21 17:45:01 -04:00
j3n57h0m45
79c91ed50d enable custom name for refresh token (#6890) 2021-07-20 15:29:11 +02:00
Rijk van Zanten
ea19390f11 Add encrypt option to MS SQL questions (#6746) 2021-07-14 00:02:00 +02:00
rijkvanzanten
801e868554 Fix remaining eslint errors
h/t @paescuj
2021-04-29 15:55:12 -04:00
Pascal Jufer
acd41eb0be Syntax fixes (#5367)
* Declare return types on functions

And a very few other type related minor fixes

* Minor syntax fixes

* Remove unnecessary escape chars in regexes
* Remove unnecessary awaits
* Replace deprecated req.connection with req.socket
* Replace deprecated upload with uploadOne
* Remove unnecessary eslint-disable-next-line comments
* Comment empty functions / catch or finally clauses
* Fix irregular whitespaces
* Add missing returns (null)
* Remove unreachable code
* A few logical fixes
* Remove / Handle non-null assertions which are certainly unnecessary (e.g. in
tests)
2021-04-29 12:11:43 -04:00
Ben Haynes
d65469c974 Update CLI database support labels 2021-03-17 12:08:38 -04:00
rijkvanzanten
37b2ec8a27 Cast port to number on cli init
Fixes #4536
2021-03-15 11:20:50 -04:00
rijkvanzanten
a12b148b17 Default generated .env permissions to -rw-r----- 2021-03-08 13:32:13 -05:00
rijkvanzanten
ada85b20ce Fix build on rc 45
Fixes #4418
2021-03-08 11:18:14 -05:00
rijkvanzanten
acc8b4fd08 Use mssql instead of tedious for client name
Fixes #4427
2021-03-08 07:39:57 -05:00
Nicola Krumschmidt
08293b8570 Update Knex to version 0.95.0 (#4374)
* Update Knex to version 0.95.0

* Fix formatting in schema dialects

* Fix Knex imports

* Update package-lock

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-03-03 19:37:44 -05:00
rijkvanzanten
c2a5622cef Remove unused environment variable 2021-02-26 10:49:19 -05:00
Igor Savin
7cddb458cb Run prettier across app and api 2020-12-08 02:22:00 +02:00
rijkvanzanten
03b00548b8 Add beta flag to init for db drivers 2020-10-16 19:10:50 -04:00
rijkvanzanten
23458b5b9f Last port change, promise 2020-10-02 16:26:40 -04:00
rijkvanzanten
1524c3e398 Change default port to 8805 2020-10-02 14:56:39 -04:00
rijkvanzanten
67a55ad228 Disable rate limiter by default
Fixes #323
2020-09-18 12:30:15 -04:00
Ben Haynes
3c7a38ca6e cleaner install options 2020-09-10 16:14:27 -04:00
rijkvanzanten
e81572c0a9 Add cache section to installed stub 2020-09-09 10:39:01 -04:00
rijkvanzanten
a8552abc23 Treat sanitizedquery as frozen 2020-09-08 18:54:18 -04:00
rijkvanzanten
9f3e12f8f2 Run prettier 2020-09-08 16:03:38 -04:00
rijkvanzanten
a4758d6d63 Cleanup install env setup 2020-09-08 15:57:03 -04:00
Tanya Byrne
9c67dc8d14 adding defaults 2020-08-27 17:02:50 +01:00
Tanya Byrne
a3e9131833 enable SSL for pg 2020-08-27 14:10:38 +01:00
Tanya Byrne
c8f771c807 Only use rate limiter if rate_limit enabled set to true 2020-08-25 20:21:06 +01:00