95 Commits

Author SHA1 Message Date
Jan Arends
5c8d3d22c6 Reset auto increment sequence (#19328)
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-09-08 11:19:14 +02:00
Pascal Jufer
96bb544c56 Support for prereleases in release workflow (#19510)
* Make release workflow compatible for prereleases

* Support prereleases in release-notes-generator

* Update readme

* Create metal-dodos-return.md

* Update .changeset/metal-dodos-return.md

* Override version of main package

* Rework & enable usage with changesets's prerelease mode

* Fix conditional DIRECTUS_PRERELEASE_ID output

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add hint to beforeExit event

* Add basic test for getInfo

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-08-25 16:48:50 +02:00
Pascal Jufer
29d06bc7d1 Simplify delayedSleep in blackbox tests (#19239) 2023-07-25 19:53:56 +02:00
Pascal Jufer
80dbdb6a7e Make "Schedule Hook" test more stable (#19237)
Co-authored-by: ian <licitdev@gmail.com>
2023-07-25 14:38:29 +02:00
José Varela
3a54f0db56 API: Do count distinct since left join duplicates results (#19208)
* Do count distinct since `left join` duplicates results

* Add changeset

* Correctly access count property

* Update .changeset/angry-suits-compete.md

* Optimize for simple filters

* Add tests

* Workaround broken 'as' for countDistinct in SQLite3

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-07-24 13:32:28 +02:00
Connor
92621429ee Linting and Formatting Upgrade (#18892)
* add docs to eslint

* update prettier ignore

* fix vitepress linting

* eslint ignore fixes

* prettier run

* update prettier ignore

* fix formatting

* enable linting of markdown files

* revert format command change

* fix irregular whitespace

* update dictionary

* (Changelog) Create four-boxes-shake.md

* Rework ESLint / Prettier setup

- Disable js/ts/vue files for Prettier to ensure linting/formatting is
  only happening via ESLint
- Rework formatting of code blocks in md files
  - Disable formatting of code blocks in md files under '/docs' by Prettier
  - Instead use "eslint-plugin-markdown" to format & __lint__ js*/ts*/vue such code blocks
  - Replace unmaintained "eslint-plugin-md" plugin by official "eslint-plugin-markdown" plugin
  - I'll check whether we can use this to format other code blocks
    (json, html, ...) as well
- Restructure, clean-up and apply some fixes to the ESLint config
  (Note: Not ready for flat config yet since not supported by
  vscode-eslint)
- Enable cache for ESLint / Prettier in scripts
- Clean-up ignore file
  - Explicit folder declaration (.../)
  - Don't ignore all 'extensions' folders in ESLint (only
    '/api/extensions/')
  - Enable formatting in '/.github' folder

* Fix all formatting issues with Prettier

* Update md files under /docs/.typedocs

* Fix lint issues in vue/js files

* ESLint / Prettier config revision v2

Enable Prettier for md code blocks, but only as warnings since it can
get into the way with Vitepress md extensions like '[!code ...]'
comments

* Remove prettier-ignore comments

* Make spellchecker happy

* Remove changeset

* Revert lint setup for code blocks

There are many cases in the docs where linting / formatting of code
blocks doesn't make
sense:
- Code block is only an excerpt - linter fails
- Code block contains special comments (e.g. markdown extensions) which
  needs to remain at the same place - formatting would break it
- ...

* Apply lint issues / formatting from temp lint setup

* Run formatter

* Fix merge failure

* Simplify & modernize ESLint / Prettier setup

No longer run Prettier via ESLint. Nowadays, this is the recommended
setup. There's no real need to run it this way, it's just an additional
layer.

Add VS Code settings to make the work with the new setup easier.

* Remove unused eslint disable directives

* Make editorconfig more useful

* Fix formatting issues reported by editorconfig

* Format files with Prettier

* Enable formatting of source translations file

* Format source translations file

* Remove unnecessary console error

* Remove unnecessary line

* Only ignore md files under .changeset

* Add CI reporter for Prettier

* Fail job on wrongly formatted files

* Fix format

* Test Prettier action on changed/added file

* Use simple CI format check for now & no cache

* Revert "Test Prettier action on changed/added file"

This reverts commit 4f7d8826ad.

* Introduce code blocks check for docs

* Fix code block issues

* Ignore auto-generated packages dir

* Fix comment position

* Also lint `/app/.storybook`

* Reformat modified files

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-06-29 11:54:01 +02:00
Rijk van Zanten
ca3e7f521f Upgrade previous "Extensions" system to new "Errors" model (#18797)
* Typecheck across packages that are built with esbuild

* Boilerplate new Errors package

* No need, tsup checks with --dts

* Switch to tsup

* Setup dev script

* Add readme

* More boilerplaty things

* Finish createError function

* Install @directus/random

* Downgrade node types

* Add utility function to check if an error is a DirectusError

* Use new is-error check

* Install errors package

* Add failed validation common error

* Export common errors

* Move joi convertion to utils

* Export failed validation

* Use new failed validation error in validate-batch

* Enhance typing output of createError

* Remove outdir (handled by tsup now)

* Replace Exception with Error

* Replace exception in test

* Remove exceptions from app

* Remove exceptions from app

* Remove failed validation exception from users service

* Remove old failed validation exception from shared

* Remove exceptions package in favor of errors

* Uninstall exceptions

* Replace baseexception check

* Migrate content too large error

* Critical detail

* Replace ForbiddenException

* WIP remove exceptions

* Add ForbiddenError to errors

* HitRateLimitError

* Move validation related error/helper to new validation package

* Add index

* Add docs

* Install random

* Convert TokenExpired

* Convert user-suspended

* Convert invalid-credentials

* Move UnsupportedMediaType

* Replace wrong imports for forbidden

* Convert invalid-ip

* Move invalid provider

* Move InvalidOtp

* Convert InvalidToken

* Move MethodNotAllowed

* Convert range not satisfiable

* Move unexpect response

* Move UnprocessableContent

* Move IllegalAssetTransformation

* Move RouteNotFound

* Finalize not found

* Various db errors

* Move value too long

* Move not null

* Move record-not-unique

* Move value out of range

* Finish db errors

* Service unavailable

* GQL errors

* Update packages/validation/src/errors/failed-validation.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update packages/validation/src/errors/failed-validation.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* InvalidQuery

* Add test for invalid query message constructor

* Invalid Payload

* Finalize exceptions move

* Improve type of isDirectusError

* Various fixes

* Fix build in api

* Update websocket exceptions use

* Allow optional reason for invalid config

* Update errors usage in utils

* Remove unused package from errors

* Update lockfile

* Update api/src/auth/drivers/ldap.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update packages/validation/src/utils/joi-to-error-extensions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Put error codes in shared enum

* Replace instanceof checks in api

* Fix tests I think

* Tweak override names

* Fix linter warnings

* Set snapshots

* Start fixing BB tests

* Fix blackbox tests

* Add changeset

* Update changeset

* Update extension docs to use new createError abstraction

* 🙄

* Fix graphql validation error name

* 🥳

* use ErrorCode.Forbidden

* fix blackbox auth login test

* Add license files

* Rename preMutationException to preMutationError

* Remove unused ms dep & sort package.json

* Remove periods from error messages for consistency

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add optional code check

* Use updated error code checker

* Rename InvalidConfigError to InvalidProviderConfigError

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-06-27 17:22:26 -04:00
Rijk van Zanten
63069bd688 Consolidate Redis environment variables (#19010)
* Remove individual redis configurations

* Use shared redis config

* Update stub

* Add changeset

* Update .changeset/grumpy-rice-film.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Remove RATE_LIMITER_REDIS_ ref in docs

* Update REDIS config in docker compose example

* Readd missing param

* Fix mention of namespace ind ocs

* Undo moving of cache store env

* Update blackbox tests & docs

* Update api/src/env.ts

Co-authored-by: ian <licitdev@gmail.com>

* Fix casing of wEbSoCkEtS

* Remove redis namespace from env check

* Add global store back in

* Tweak phrasing

* Readd supplementary note 5

* Bring back CACHE_STORE in docs / tests

* Forgot one CACHE_STORE

* Consistent order

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-06-27 13:34:49 -04:00
GitStart
16ed93c9b9 Expose case-insensitive variants of _starts_with and _ends_with (#18941)
Co-authored-by: RubensRafael <rubensrafael2@live.com>
Co-authored-by: v1b3m <vibenjamin6@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-06-27 17:48:29 +02:00
Kevin Lewis
9a0cd33266 Revising Docker Guides (#18956)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-06-23 11:18:50 +00:00
rijkvanzanten
c0cd7860b1 v10.3.0 2023-06-13 12:52:12 -04:00
Brainslug
cfddabd9ee Integrating Websockets in Directus 🕸️🧦 (#14737)
* added emitter context

* partial items tests

* updated items handler tests

* fixed test after merge

* forgot the event context

* fixed auth message parsing for graphql subscriptions

* fixed type strictness

* fixed graphql subscription bug

* bumped websocket dependencies

* touched up some dangling code

* updated itemsservice usage

* disabled overkill logs

* double checked environment type processing

* fixed missed capitalization

* fixed subscription payloads

* Added explicit string type casting

* removed obsolete "trimUpper" utility

* using the parseJSON utility consistently

* pinned dependencies

* parse environment variables

* fixed pnpm-lock

* GraphQL Subscriptions for all events

* fixed typo

* added event data to the graphql definition

* fix payload for delete events

* Added optional chaining for type to prevent fatal crashes on invalid messages

* fix failing on getting type from undefined

* Update api/src/websocket/exceptions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add proper ZodError handling

* added the zod-validation-error parser

* allow disabling the rate limiter

* Update api/src/websocket/controllers/base.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* updated starting logs

* fixed email/password expiration logic

* added tests for getMessageType

* simplified message parsing and dropped capitalization

* updated authenticate test

* switched to lower cased message.type to prevent spreading "toUpperCase" around

* cleaned up debug logs

* cast enabled config to boolean

* Update api/src/websocket/controllers/rest.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/subscribe.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/subscribe.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/items.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/controllers/base.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/heartbeat.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Suggested fixes by Azri

* removed redundant try-catch

* fixed authentication timeout
added returning the refresh token when authenticating

* updated pnpm lock after merge

* Fixed authentication modes for GraphQL according to best practices

* implement useFakeTimers in heartbeat unit test

* implement useFakeTimers in items unit test

* Update api/src/services/server.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* removed obsolete authentication.verbose toggle

* added email flag to message validation

* switched to ternary for consistency

* moved getSchema out of for loop

* added singleton logic to items handler

* close the socket after failed auth for non-public connections

* disabled system collections for rest subscriptions

* re-ran pnpm i

* allow for multiple subscripitions in the memory messenger

* - fixed system collection subscriptions
- abstracted hook message bus
- fixed graphql horizontal scaling

* remove logic from root context for tests

* fix reading created item

* fix linter

* typo and extra safe guard suggested by azri

* prevent setting long timeouts in favor of a shared interval

* prevent unsubscribing all existing subscriptions when omitting "uid"

* - extracted getService utility
- block system collections mutation in the items handler
- implemented the correct services for system collections

* allow numeric uid's to be used

* fixed the types for numeric uid's to be used

* added missing await's

* fixed type imports after merge

* removed unused imports

* Update api/src/websocket/controllers/hooks.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/controllers/hooks.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/messenger.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* improved error for graphql subscriptions

* fixed TS Modernization conflicts

* fixed TS Modernization conflicts

* fixed conflicts after merge

* removed unused name property

* abstracxted environment configuration

* respond to ping messages when heartbeat disabled

* something something merge

* moved toBoolean to it's own util file

* replaced old socket naming

* removed old exception

* fixed typo

* Update api/src/env.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/websocket/handlers/heartbeat.test.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/websocket/handlers/heartbeat.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/services/server.ts

Co-authored-by: ian <licitdev@gmail.com>

* fixed for linter

* add server_info_websocket in graphql

* Add base REST websocket tests

* do merge things

* fixing things

* fixed failing unit test

* Update dependencies

* Move tests

* Update lockfile

* Use new paths when spawning

* return websockets to opt-in

* Enable websockets for tests

* Test with ephemeral access token

* no camelcasing gql subscriptions

* use underscore for gql event

* Remove unused import

* Add base GraphQL subscription tests

* Fix accidental comment

* Add some relational tests

* Organize imports

Using VS Code's default organize import

* Run ESlint formatting

* One more opinionated formatting change

* Formatting

* Fix message sequence not in order

* Remove relational batch update tests

* Test horizontal scaling

* using toboolean util for server_info

* removed unneeded type cast

* found the gql request type

* extra usage of the toBoolean util

* merge the authentication middleware and get-account-for-token util

* updated utility test

* fixed middleware unit test

* Add return

* Remove user filtering and close conns

* Fix reused accountability

* fixed failing util test

* added subscription unit tests

* added missing mock

* trigger workflow

* Revert "trigger workflow"

This reverts commit 4f544b0c1b.

* Trigger testing for all vendors

* add unsubscription confirmation

* Wait for unsubscription confirmation

* Fix incorrect sending of unsubscription confirmation

* updated ubsubscribe logic

* Update count for unsubscription message

* Fix sequence for UUID pktype in MSSQL

* Increase auth timeout

* Add start index when getting messages

* Fix subscription retrieval and cast uid to string

* Remove nested ternary

* Revert "Increase auth timeout"

This reverts commit 10707409c4.

* Terminate connection instead of close

* fixed merge

* re-added missing packages after merge resolve

* fixed type imports

* Create lazy-cows-happen.md

Added changeset

* Minor bump for "directus" package as well

* fixed "strict" auth mode for graphql subscriptions

* removed nested ternary

* Add websocket tests to sequential flow

* Disable pressure limiter for blackbox tests

* fix merge

* WebSockets Documentation (#18254)

* Small repsonsive styling fix on Card

* REST getting started guide

* Authentication guide

* REST subscription guides

* JS Chat guide

* Sidebar websocket guides section

* Added config options

* Respoinding to brainslug's review

* Fixed incorrect header on guides/rt/subs

* Fixed spellchecker

* Correct full code example on guides/rt/chat/js

* Fixed JS chat tut

* Order of steps in js chat guide updated for easier following-along

* Realtime chat Vue Guide

* feat: create react.js file

* feat: add set up for directus project

* docs: create react boilder plate

* docs: initialize connection

* docs: set up submission methods

* docs: establish websocket connection

* docs: subscribe to messages

* docs: create new messages

* docs: display new messages

* docs: display historical messages

* docs: next steps

* docs: full code sample

* docs: clean up

* docs: add name to contributors

* docs: add react card

* docs: updates to react chat

* Added live poll result guide

* docs: intro

* docs: before you begin

* docs: install packages

* docs: authenticate connection

* docs: query and mutation

* docs: utilize hooks

* docs: subscribe to changes

* docs: create helper functions

* docs: display messages

* docs: summary

* docs: full sample code

* chore: add card for webscockets with graphql

* docs: intro

* docs: subscribe to changes

* docs: handling changes

* docs: crud operations

* docs: unsubscribing from changes

* docs: updates

* chore: add card

* chore: updates to graphql docs

* chore: updates to getting started

* chore: updates to subscription

* chore: updates to real chat guide

* Added WebSockets Operations Guide

* Consistent titles

* Contributors component for docs

* Triggering Netlify

* Add operations to sidebar

* Fix operations link

* Small formatting changes

* Clarity around property values

* Removed unused values in Contributors component

* Prompt for default choice

* Tabs & lowercase doctypes

* Semicolons

* Event overwerites -> event listeners

* Spacing

* Flipped order of websockets guide to match GQL

---------

Co-authored-by: Esther Agbaje <folasadeagbaje@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* fixed typo

* removed unused import

* added tests for "to-boolean" and "exceptions"

* added websocket service tests

* quote environment variable to satisfy dictionary

* GraphQL Subscriptions update (#18804)

* updated graphql subscription structure

* updated graphql examples

* Create hungry-points-rescue.md

* using `key` instead of `ID` on the toplevel

* removed changeset

* fixed the graphql type after the rename to `key`

* retrun data null for delete events to prevent non-nullable gql error

* updated missed ID reference in the docs

* updated missed ID reference in the docs

* renamed "payload" to "data" in the REST Subscription response

* fixed missed reference to payload

* added optional event filter for REST subscriptions

* updated docs for event filter

* Update docs/guides/real-time/subscriptions/websockets.md

Co-authored-by: ian <licitdev@gmail.com>

---------

Co-authored-by: ian <licitdev@gmail.com>

* added messenger unit test

* always send subscription confirmation

* Add event to subscription options

* Update tests

* Add tests for event filtering

* Revert testing for all vendors

* Remove obsolete console comment

* Update comment

* Correct event in JS WS guide

* Fix collection name to match name used in subscription

* Fix collection name in other guides

* Fix diffs in doc & enhance chart example

* Complete sentence in GraphQL guide

* Small update to config description

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Kevin Lewis <kvn@lws.io>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Esther Agbaje <folasadeagbaje@gmail.com>
2023-06-08 18:54:50 +00:00
ian
c48405f0c3 Reuse S3 client and increase maxSockets (#18799)
* Reuse S3 client and increase maxSockets

* Update test to check for timeouts and non-2xx responses

* Create beige-olives-beam.md

* Fix unit tests

* Update packages/storage-driver-s3/src/index.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update .changeset/beige-olives-beam.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-06-06 16:44:12 +00:00
ian
0b4542fd25 Fix schedule hook test (#18719) 2023-05-25 16:35:39 +08:00
ian
80cca54d18 Enable caching for requests from data studio (#18484)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-05-25 08:37:54 +02:00
Nicola Krumschmidt
60be3c2b40 Add synchronization across horizontally scaled instances to schedule flows and hooks (#18584)
* Add synchronization to schedule flows and hooks

Fixes #15052

* Add changeset

* Add test

* Add to sequential list

* Fix spelling in changeset

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-05-24 17:16:42 -04:00
Pascal Jufer
7ecda9b2fa Clean-up package.json files (#18663)
* Remove publishConfig from all packages

* Remove invalid version field

* Move @directus/composables to devDependencies in app

* Resort all package.json files

* Align meta fields

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-19 17:54:10 -04:00
Pascal Jufer
2e647d80e6 Patch Tuesday 🐸 (#18643)
* Update `directus-monorepo` deps

* Update `@directus/api` deps

* Update `@directus/app` deps

* Update `@directus/composables` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* Update `@directus/data` deps

* Update `@directus/data-driver-postgres` deps

* Update `@directus/extensions-sdk` deps

* Update `@directus/pressure` deps

* Update `@directus/random` deps

* Update `@directus/release-notes-generator` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* Update `docs` deps

* Update `tests-blackbox` deps

* Bump pnpm to 8.5.1

* Bump build-push-action in release workflow to v4

* Assign new spy instances before each test

See https://github.com/vitest-dev/vitest/pull/3386

* Create clean-needles-nail.md

* Remove leftover 'beforeAll' imports

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-19 10:48:21 -04:00
Pascal Jufer
a708ec79d8 Redact additional keys in logs of Flows (#18534)
* Redact additional keys in logs of Flows

* Create moody-poems-pump.md

* Move REDACTED_TEXT to constants package & update tests

* Revert "Move REDACTED_TEXT to constants package & update tests"

This reverts commit 0f5b227253.

* Update redacted value in blackbox test

* Use own redact implementation

* Move REDACTED_TEXT to constants package

* Replace outdated comment

* Fix misleading return type

Since values might change (redacted), output isn't necessarily the same type anymore
2023-05-19 09:41:04 -04:00
rijkvanzanten
fa8c914978 v10.1.1 2023-05-16 14:03:02 -04:00
Pascal Jufer
b3f57f6ac2 Clean-up release-notes-generator (#18568)
* Clean-up release-notes-generator

* Remove leftover version check & more overt file names

* Explicit return types

* Add simple test for markdown generation

* Rework a bit

* Final tweaks

* Enhance comments

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-16 13:59:39 -04:00
Pascal Jufer
377b2889ec More sensible default formats for image auto conversion (#18615)
* Prevent auto conversion of png to jpg images

* Create red-swans-march.md

* Update transformation tests

* Update blackbox test

* Fix image allocation

* Fix test again :-)

* Convert formats with transparency support to png (if no accept header)

* Update tests & add final fallback

* Update changeset

* Update blackbox test

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-16 13:50:27 -04:00
ian
66d9f34c92 Disable pressure limiter for blackbox tests (#18623) 2023-05-16 08:50:12 -04:00
ian
94e8153baf Sanitize column only when applying snapshot diff (#18579)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-05-15 13:41:57 +02:00
rijkvanzanten
703c5d9c69 v10.1.0 2023-05-11 17:22:00 -04:00
Brainslug
110413e473 clear item cache when permissions change (#18526)
* clear item cache when permissions change

* Create slow-timers-fold.md

* Add tests

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-05-12 01:18:40 +08:00
Rijk van Zanten
44b4863788 Require curly brackets in if statements when they're multi-line (#18387)
* Enable Curly rule

* Fix curly misuse
2023-05-02 16:02:43 -04:00
ian
69061c1284 Add Cache option for Flows webhook trigger (#18277)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-05-02 15:17:07 +00:00
Jan Arends
8b8858da9a Support multipart/form-data to apply schema diffs (#18321)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-05-02 10:24:46 +00:00
Brainslug
87f2edc735 Prevent nested ternary expressions (#18376)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-05-01 19:07:40 +00:00
Rijk van Zanten
2bc226d16f Change license to BSL-1.1 (#18330)
* Change license to BSL1.1

* Mark major version

* Fix formatting

* Remove duplicate heading

* Update formatting

* Copy paste error

* Update license

* Update license

* Format covenants

* Update readme.md

* Update licenses for packages

* Update readme.md

* Update contributors.yml

OG... but to be clear, rijkvanzanten is the one who really built all of this ❤️

* Tweak license in openapi

* Update packages/specs/src/openapi.yaml

* Add MIT license headers

* Use v10 in examples

* Update additional examples

* Update generate-extensions-entrypoint.test.ts

* Update tests to use latest v9 and v10 versions

* use lowercase for naming consistency

* change casing for api license

* Update migrations doc

* Update dictionary

* Consistent ordering of license field in app/package.json

* Use major version in specs again (but as string)

So it's valid for all v10 versions

* Consolidate readme's

---------

Co-authored-by: Ben Haynes <ben@directus.io>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-04-26 14:10:42 -04:00
rijkvanzanten
381f1c9d8e v9.26.0 2023-04-24 16:40:53 -04:00
Pascal Jufer
17afb9a3bc Patch Tuesday 🐸 (#18260)
* Update `@directus/api` deps (minor)

 @aws-sdk/client-ses          3.292.0  →   3.316.0
 @godaddy/terminus             4.11.2  →    4.12.0
 @rollup/plugin-alias           4.0.3  →     4.0.4
 @rollup/plugin-node-resolve   15.0.1  →    15.0.2
 @types/node                  18.15.3  →  18.15.11
 @vitest/coverage-c8           0.29.3  →    0.30.1
 axios                          1.3.4  →     1.3.6
 fs-extra                      11.1.0  →    11.1.1
 helmet                         6.0.1  →     6.1.5
 ioredis                        5.3.1  →     5.3.2
 joi                           17.8.4  →    17.9.1
 liquidjs                      10.6.1  →    10.7.0
 marked                        4.2.12  →     4.3.0
 nanoid                         3.3.4  →     3.3.6
 rollup                        3.19.1  →    3.20.6
 sharp                         0.31.3  →    0.32.0
 vitest                        0.29.3  →    0.30.1
 vm2                           3.9.16  →    3.9.17

* Update `@directus/api` deps (major)

 @rollup/plugin-alias     4.0.4  →     5.0.0
 @types/inquirer          8.2.6  →     9.0.3
 @types/node           18.15.11  →  18.15.12
 camelcase                6.3.0  →     7.0.1
 chalk                    4.1.2  →     5.2.0
 commander                9.5.0  →    10.0.1
 execa                    5.1.1  →     7.1.1
 icc                      2.0.0  →     3.0.0
 inquirer                 8.2.5  →     9.1.5
 ~ldapjs                   2.3.3  →     3.0.2~
 nanoid                   3.3.6  →     4.0.2
 ~openapi3-ts              3.2.0  →     4.1.1~
 ora                      5.4.1  →     6.3.0
 strip-bom-stream         4.0.0  →     5.0.0
 tedious                 15.1.3  →    16.0.0
 typescript               4.9.5  →     5.0.4

* Update `@directus/app` deps

 @babel/core                             7.21.3  →    7.21.4
 @babel/preset-env                       7.20.2  →    7.21.4
 @fortawesome/fontawesome-svg-core        6.3.0  →     6.4.0
 @fortawesome/free-brands-svg-icons       6.3.0  →     6.4.0
 @fullcalendar/core                       6.1.4  →     6.1.5
 @fullcalendar/daygrid                    6.1.4  →     6.1.5
 @fullcalendar/interaction                6.1.4  →     6.1.5
 @fullcalendar/list                       6.1.4  →     6.1.5
 @fullcalendar/timegrid                   6.1.4  →     6.1.5
 @pinia/testing                          0.0.15  →    0.0.16
 @popperjs/core                          2.11.6  →    2.11.7
 @storybook/addon-actions            7.0.0-rc.4  →     7.0.6
 @storybook/addon-backgrounds        7.0.0-rc.4  →     7.0.6
 @storybook/addon-docs               7.0.0-rc.4  →     7.0.6
 @storybook/addon-essentials         7.0.0-rc.4  →     7.0.6
 @storybook/addon-links              7.0.0-rc.4  →     7.0.6
 @storybook/addon-mdx-gfm            7.0.0-rc.4  →     7.0.6
 @storybook/addon-measure            7.0.0-rc.4  →     7.0.6
 @storybook/addon-outline            7.0.0-rc.4  →     7.0.6
 @storybook/client-api               7.0.0-rc.4  →     7.0.6
 @storybook/client-logger            7.0.0-rc.4  →     7.0.6
 @storybook/vue3                     7.0.0-rc.4  →     7.0.6
 @storybook/vue3-vite                7.0.0-rc.4  →     7.0.6
 @types/diff                              5.0.2  →     5.0.3
 @types/dompurify                         3.0.0  →     3.0.2
 @types/lodash                         4.14.191  →  4.14.194
 @vitejs/plugin-vue                       4.0.0  →     4.1.0
 @vue/test-utils                          2.3.1  →     2.3.2
 apexcharts                              3.37.1  →    3.39.0
 axios                                    1.3.4  →     1.3.6
 dompurify                                3.0.1  →     3.0.2
 happy-dom                                8.9.0  →     9.8.4
 marked                                  4.2.12  →     4.3.0
 nanoid                                   4.0.1  →     4.0.2
 pinia                                   2.0.33  →    2.0.34
 sass                                    1.59.3  →    1.62.0
 storybook                           7.0.0-rc.4  →     7.0.6
 typescript                               4.9.5  →     5.0.4
 vite                                     4.1.4  →     4.2.2
 vitest                                  0.29.3  →    0.30.1
 webpack                                 5.76.2  →    5.80.0

* Update root deps

* Update `@directus/composables` deps

* Update `@directus/constant` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* tsconfig workaround: ignoreDeprecations

* Update `@directus/extensions-sdk` deps

* Update `@directus/schema` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* tsconfig workaround in schema: ignoreDeprecations

* tsconfig workaround in remaining packages: ignoreDeprecations

* Update `tests-blackbox` deps

* Revert "tsconfig workaround: ignoreDeprecations"

This reverts commit 5d97da55e3.

* Revert tsconfig override

* Update tsconfig

* Fix imports in @directus/utils

* Fix imports in composables

* Fix imports in extensions-sdk

* Fix imports in @directus/api

* Move RateLimiterAbstract to types import as well

* Bump pnpm to 8.3.1

* Update `docs` deps

> [...] the @vueuse/head package will be deprecated. If you're setting up this package on a new project, you should use the @unhead/vue package directly [...]

* Remove obselete dep `concurrently` from extensions-sdk

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* New day, new updates

Also forgot to include minor updates of deps in `tests-blackbox` in previous
commit

* Fix `api` tests

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-04-21 12:59:27 -04:00
rijkvanzanten
835b0de89d v9.25.2 2023-04-20 10:49:53 -04:00
Azri Kahar
1f2166ab7b Add blackbox test for assets with format=auto query parameter (#18240) 2023-04-18 23:05:04 -04:00
Rijk van Zanten
c48309ab68 Last eslint tweak (#18198)
* Should be there now

* Format
2023-04-14 17:40:50 -04:00
rijkvanzanten
37658802b7 One more 2023-04-14 17:30:56 -04:00
Rijk van Zanten
bdb1a919e2 White space rules (#18197)
* White space rules

* Auto-fix whitespace linter warnings
2023-04-14 17:21:56 -04:00
rijkvanzanten
c301cfbc82 v9.25.1 2023-04-13 16:32:58 -04:00
rijkvanzanten
4f540b03d0 v9.25.0 2023-04-13 13:32:44 -04:00
Pascal Jufer
15b91dee34 Blackbox tests restructuring (#18122)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-11 18:28:37 +02:00
ian
21f6f3d008 Port remaining tests into E2E blackbox (#17361)
* Migrate filter tests

* Migrate aggregation tests

* Remove migrated tests

* Migrate offset tests

* Migrate depth tests

* Run testing for all vendors

* Fix count returned as bigint for pg and crdb

* Calculate snapshot value counts instead of hardcoding

* Delete remnants

* Revert testing for all vendors
2023-02-01 21:47:29 +01:00
ian
f1a8e0446f Fix duplicated results and functions in nested filters (#14798)
* Speed query up by reusing existing aliases which reduces table joins

* Use subquery in top level m2o to remove duplicates

* Fix linting

* Apply distinct on primary key field in subqueries

* Use distinct instead as there are only primary keys

* Apply subquery on top level

* Try remove sub sub query

* Test if working for all vendors

* Add support for _none and _some

* Use subquery only when field depth > 1

* Add tests

* Use original table names for columns with functions (#14690)

* Use original table names for columns with functions

* Extract filter function path parsing as shared util

* Fix filter function path when adding node

* Pass the originalCollectionName into filter functions

* Update unit test

* Replace functions within deep GraphQL

* Fix invalid operator error for _none and _some

* Add filter function tests

* Revert triggering for all vendors

* Simplify aliasMap

* Replace functions in filter within GraphQL aggregate query

* Add API support for filtering of alias field

* Mark schema as optional

* Shift logical operators upwards

* Separate recursive parseFilter

* Rework shifting of logical operators

* Error on invalid usage of _none and _some

* Use inner join to preserve sort order

* Run tests for all vendors

* Reuse aliasMap for sort and filter

* Sort on top level query

* Remove unnecessary limit on wrapper query

* Refactor applyQuery options

* Remove duplicates from nested multi relational sort

* Fix offset in MSSQL requiring OrderBy

* Disable schema cache

* Use inner query only for nested sort or multi relational filter

* Fix MSSQL duplicate order column

* Use inner query only for multi relational

* Additional integration tests

* Order within partition for multi relational sorts

* Rename to directus_row_number

* Fix unit test

* Add base sort and filter tests

* Fix Oracle uppercased rowNumber column

* Fix unit test

* Fix top level query sort with function

* Parse functions in inner query

* Increase clarity with knex.ref()

* Remove sort filter for top level primary key

* Fix unit test

* Bypass queries with groupBy

* Add collection to aliasMap to fix functions in nested sort

* Fix multi relational sort with functions

* Add tests for filter and sort with functions

* Fix accidental deletion of brackets

* Fix top level alias filter node interface

* Update M2M sort tests

* Add M2A tests

* Cast m2a primary key as varchar2 for oracle

* Enable filtering tests for M2A

* Fix prototype polluting assignment in aliasMap

* Remove unnecessary currentKey

* Simplify code to increase readability

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Fix linting and missing 'this' error

* Revert optional chaining

* Add mysql5 to tests

* Fix mysql5 missing rowNumber()

* Overcome indexing delays in MySQL5

* Verify MySQL5 sorting is in order as the result count varies between runs

* Skip joining when sorting field already exists

* Simplify variable assignment

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Fix linting

* Reduce duplicate logic with vars

* Transform _func fields in GraphQL only for valid functions

* Fix unit test

* Fix unsupported date_part() in CrDB

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Roger Stringer <roger@directus.io>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-12-21 11:56:18 -05:00
Pascal Jufer
47db8e5d2f Clean-up dependencies (#16481)
* Clean-up api

- Remove unnecessary deps
- rimraf -> del-cli

* Clean-up app

- bytes -> pretty-bytes
- Remove ms as we already have pretty-ms
- Remove other unnecessary deps
- Add storybook build to gitignore

* Clean-up drive packages

- Simplify cleanup scripts
- rimraf -> del-cli
- Remove unnecessary deps

* Clean-up schema

- Remove unnecessary dep (lodash)
- Replace 'npm-watch' with native watch mode

* Clean-up shared

- Remove unnecessary dep (c8, adding @vitest/coverage-c8 will be addressed in
  other PR)
- rimraf -> del-cli
- npm-run-all -> concurrently

* Clean-up root

- Add missing eslint-plugin-jest dep
- listr -> listr2

* Make build output a bit friendlier

* Remove cleanup scripts

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-18 17:02:41 +00:00
Azri Kahar
7d62a70007 Fix GraphQL aggregate query (#14560)
* FIx aggregate query in GraphQL

* change type to Query and alter previous logic

* test
2022-07-22 09:55:51 -04:00
Rijk van Zanten
27cc3e1bec Move repo to pnpm based workflow (#14350)
* Use pnpm instead of npm

* Setup workspace

* Replace scripts / lerna root stuff

* Add todo file

* Update lock, start updating packages

* Update todo

* [WIP] Keep going on module resolution in pnpm

* Fix final missing deps/types for complete build

* [WIP] Replace npm commands with pnpm alternatives

* Update jest config for v28

* Fix missing package under schema

* Update workflow

* Run CI on fork

* Use local reference

* Remove version from local workflow

* Add build to prepare

* Add Node's max old space size flag to linter

* Idemdito for codeql

* Hyphens? Underscores?

* Bang

* Only build one thing at a time

* Underscores after all?

* Match 7gb of GH

* Set env on prepare

* Jest is driving me nuts

* Downgrade jest

* Add root jest

* We'll get there, eventually.

* 🥳

* Het houdt niet op, niet vanzelf

* attempt to fix e2e

* fix unit test & shared import

* add `debug: true` to CodeQL action

* Fix dev call in shared

* Add missing cross-env

* Add missing geo-json dependency for app

* add flag to prepare action to skip build

* remove debug flag from codeql action

* fix invalid env syntax

* add tslib to app dependencies for tests

* another attempt

* Added missing rimraf dependency for the api prebuild/cleanup scripts

* Added missing ts-node dependency for the api cli script

* update E2E tests readme to use pnpm

* Undo move of extensions sdk

* Fixes nested groups in accordions not rendering fields (#14369)

* initial fix for nested groups in accordions

* removed debug code

* Fix strict relative dates showing "incorrect" (#14390)

* Set rounding method of relative formatted date to floor when strict option is selected

* Add round fn as an option

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>

* Don't use locales in generated camelCased values from env (#14401)

Ref https://github.com/directus/directus/discussions/14122

* Fix list panel descending sort (#14396)

* fix list panel descending sort

* Handle missing or explicit desc value

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>

* Fix query primary field for system tables (#14402)

* New Crowdin updates (#14403)

* Update source file en-US.yaml

* New translations en-US.yaml (Spanish, Chile)

* New translations en-US.yaml (Spanish, Latin America)

* New translations en-US.yaml (Greek)

* New translations en-US.yaml (Italian)

* New translations en-US.yaml (French, Canada)

* Fix/wysiwyg context menu (#14404)

* New translations en-US.yaml (Polish) (#14407)

* Encoded the url using encodeURIComponent, so that the url gets sanitzed and so, we did not get 404 error. (#14418)

* Encoded the url using encodeURIComponent, so that the url gets sanitized and we did not get 404 error.

* Added required changes as stated by the reviewer

* Used encodeURI instead of encodeURIComponent to encode the params only

* Solved linting error

* Solved linting errors

* checked with npm run lint, no linting errors found

* fix generateJoi error due to empty permissions when creating new role (#14416)

* fix empty permissions when creating new role

* basic test

* move logic up

* additional test

* Update api/tests/utils/filter-items.test.ts

Co-authored-by: ian <licitdev@gmail.com>

Co-authored-by: ian <licitdev@gmail.com>

* Recreate lockfile

* Fix build of extensions-sdk

* Add missing dependency

* Add some more missing deps

* Update blackbox to use pnpm

* Update workflow to use main

* Update pack to work with pnpm

* Simplify commands

* Remove todo file (completed)

* ADd missing types dep

* Use local test version

* Remove version from local reference

* Call super with context

* Add missing dep

* Simplify workflows

Ref https://github.com/directus/organization/issues/135

* Simplify some more

* Linter is on root

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Gerard Lamusse <gerardlamo@gmail.com>
Co-authored-by: José Varela <joselcvarela@gmail.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Zeel Pathak <60271095+zeel-pathak@users.noreply.github.com>
2022-07-15 17:41:49 -04:00
ian
8d1966ab04 Blackbox testing (#13200)
* Add black box tests

* Revert docker compose file

* Update workflow

* Try use workflow from dev repo

* Increase seedDB() timeout

* Disable other checks for now

* Change DB sequence

* Update jest moduleNameMapper

* Update workflow's docker-compose.yml path

* Slice array first

* Remove differentiation of status code

* Delete field only after foreign key constraints are removed

* Add checks for different types of primary key

* Test global query filter for all field types

* Increase timeout for m2o seeding

* Add case insensitive string operators

* Update filter check to run on relational fields

* Enable time field checks

* Add seeded random and fix relational seeding

* Add casting for integer and bigInteger

* Minor fixes

* Reduce bigInt values

* Separate seeding of DB structure from values

* Add primaryKey seeding function

* Use automatic IDs except for string pk

* Try fix ci

* Update package-lock.json

* Update common.test for concealed user tokens

* Use dynamic field type for m2o.test relational fields

* Temporary disable missing nicontains for string type

* Add support for alias type filtering

* Fix relational filter operator checks

* Add initial o2m test

* Remove integer pk limit

* Add empty checks for string and uuid null

* Limit generated integer value to 4 bytes

* Patch timezone tests for MSSQL

* Remove sample query filter test

* Fix timezone test for sqlite

* Fix MSSQL uuids

* Fix MSSQL timestamp inaccuracy

* Cast datetime schema to milliseconds for comparison

* Fix MySQL / Maria timestamp inaccuracy

* Fix MySQL / Maria between operator inconsistency for float type

* Fix missing time datatype in Oracle

* Skip filter testing on Oracle

* Enable o2m filter tests for other collections

* Run tests only on SQLite for PRs unless the Full Tests label exists

* Try fix actions

* Refactor github actions

* Update tests flow setup to use getURL()

* Start postgres docker

* Reinstate package-lock

* Fix geometry test

* Remove .gitkeep files

* Add todo.md

* Rename black box to blackbox

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-07-15 15:25:32 +00:00
Azri Kahar
ea3bf3a597 Fix undefined values in filters for GraphQL (#14080)
* fix undefined values in filters for GraphQL

* add test for not passing variable in filter
2022-06-24 09:36:05 -04:00
ian
443d3f6734 Add depth limit to filtering (#11845)
* Add depth limit to filtering

* Add depth limit to GraphQL

* Add docs

* Rename environment variable

* Add simple deep filter depth calculation

* Update error message

* Shift fields depth check to base function

* Remove unused var

* Implement GraphQL filter depth

* Add check for _and & _or filters in GraphQL

* Add check for _and & _or filters in REST

* Remove commented code

* Add check for REST filter query

* Add REST tests

* Setup m2m using directus fields

* Add GraphQL tests

* Fix linter error

* Cleanup calculateDepth + add docs/tests

* Remove validator in GraphQL

* Add depth checking for nested sort

* Enable source map to display correct error lines

* Set max relational depth to be at least 2

* Update tests

* Add unit test for deep _sort

* Add minimum value in docs

* Refactor depth validation to be in validateQuery

* Add boolean parameter for calculation of _sort in deep query

* Use array of keys to parse dot notation

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-06-15 11:52:54 -04:00
ian
8f131b5156 Fix range to be based on current time (#13701)
* Fix range to be based on current time

* Parse using localtime for fields without timezone data on SQLite

* Add tests

* Parse test response to int

* Refactor to options object

* Read timestamps as UTC for Oracle

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-06-08 13:12:42 +02:00