Commit Graph

672 Commits

Author SHA1 Message Date
rijkvanzanten
c0cd7860b1 v10.3.0 2023-06-13 12:52:12 -04:00
Brainslug
cfddabd9ee Integrating Websockets in Directus 🕸️🧦 (#14737)
* added emitter context

* partial items tests

* updated items handler tests

* fixed test after merge

* forgot the event context

* fixed auth message parsing for graphql subscriptions

* fixed type strictness

* fixed graphql subscription bug

* bumped websocket dependencies

* touched up some dangling code

* updated itemsservice usage

* disabled overkill logs

* double checked environment type processing

* fixed missed capitalization

* fixed subscription payloads

* Added explicit string type casting

* removed obsolete "trimUpper" utility

* using the parseJSON utility consistently

* pinned dependencies

* parse environment variables

* fixed pnpm-lock

* GraphQL Subscriptions for all events

* fixed typo

* added event data to the graphql definition

* fix payload for delete events

* Added optional chaining for type to prevent fatal crashes on invalid messages

* fix failing on getting type from undefined

* Update api/src/websocket/exceptions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add proper ZodError handling

* added the zod-validation-error parser

* allow disabling the rate limiter

* Update api/src/websocket/controllers/base.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* updated starting logs

* fixed email/password expiration logic

* added tests for getMessageType

* simplified message parsing and dropped capitalization

* updated authenticate test

* switched to lower cased message.type to prevent spreading "toUpperCase" around

* cleaned up debug logs

* cast enabled config to boolean

* Update api/src/websocket/controllers/rest.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/subscribe.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/subscribe.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/items.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/controllers/base.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/heartbeat.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Suggested fixes by Azri

* removed redundant try-catch

* fixed authentication timeout
added returning the refresh token when authenticating

* updated pnpm lock after merge

* Fixed authentication modes for GraphQL according to best practices

* implement useFakeTimers in heartbeat unit test

* implement useFakeTimers in items unit test

* Update api/src/services/server.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* removed obsolete authentication.verbose toggle

* added email flag to message validation

* switched to ternary for consistency

* moved getSchema out of for loop

* added singleton logic to items handler

* close the socket after failed auth for non-public connections

* disabled system collections for rest subscriptions

* re-ran pnpm i

* allow for multiple subscripitions in the memory messenger

* - fixed system collection subscriptions
- abstracted hook message bus
- fixed graphql horizontal scaling

* remove logic from root context for tests

* fix reading created item

* fix linter

* typo and extra safe guard suggested by azri

* prevent setting long timeouts in favor of a shared interval

* prevent unsubscribing all existing subscriptions when omitting "uid"

* - extracted getService utility
- block system collections mutation in the items handler
- implemented the correct services for system collections

* allow numeric uid's to be used

* fixed the types for numeric uid's to be used

* added missing await's

* fixed type imports after merge

* removed unused imports

* Update api/src/websocket/controllers/hooks.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/controllers/hooks.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/messenger.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* improved error for graphql subscriptions

* fixed TS Modernization conflicts

* fixed TS Modernization conflicts

* fixed conflicts after merge

* removed unused name property

* abstracxted environment configuration

* respond to ping messages when heartbeat disabled

* something something merge

* moved toBoolean to it's own util file

* replaced old socket naming

* removed old exception

* fixed typo

* Update api/src/env.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/websocket/handlers/heartbeat.test.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/websocket/handlers/heartbeat.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/services/server.ts

Co-authored-by: ian <licitdev@gmail.com>

* fixed for linter

* add server_info_websocket in graphql

* Add base REST websocket tests

* do merge things

* fixing things

* fixed failing unit test

* Update dependencies

* Move tests

* Update lockfile

* Use new paths when spawning

* return websockets to opt-in

* Enable websockets for tests

* Test with ephemeral access token

* no camelcasing gql subscriptions

* use underscore for gql event

* Remove unused import

* Add base GraphQL subscription tests

* Fix accidental comment

* Add some relational tests

* Organize imports

Using VS Code's default organize import

* Run ESlint formatting

* One more opinionated formatting change

* Formatting

* Fix message sequence not in order

* Remove relational batch update tests

* Test horizontal scaling

* using toboolean util for server_info

* removed unneeded type cast

* found the gql request type

* extra usage of the toBoolean util

* merge the authentication middleware and get-account-for-token util

* updated utility test

* fixed middleware unit test

* Add return

* Remove user filtering and close conns

* Fix reused accountability

* fixed failing util test

* added subscription unit tests

* added missing mock

* trigger workflow

* Revert "trigger workflow"

This reverts commit 4f544b0c1b.

* Trigger testing for all vendors

* add unsubscription confirmation

* Wait for unsubscription confirmation

* Fix incorrect sending of unsubscription confirmation

* updated ubsubscribe logic

* Update count for unsubscription message

* Fix sequence for UUID pktype in MSSQL

* Increase auth timeout

* Add start index when getting messages

* Fix subscription retrieval and cast uid to string

* Remove nested ternary

* Revert "Increase auth timeout"

This reverts commit 10707409c4.

* Terminate connection instead of close

* fixed merge

* re-added missing packages after merge resolve

* fixed type imports

* Create lazy-cows-happen.md

Added changeset

* Minor bump for "directus" package as well

* fixed "strict" auth mode for graphql subscriptions

* removed nested ternary

* Add websocket tests to sequential flow

* Disable pressure limiter for blackbox tests

* fix merge

* WebSockets Documentation (#18254)

* Small repsonsive styling fix on Card

* REST getting started guide

* Authentication guide

* REST subscription guides

* JS Chat guide

* Sidebar websocket guides section

* Added config options

* Respoinding to brainslug's review

* Fixed incorrect header on guides/rt/subs

* Fixed spellchecker

* Correct full code example on guides/rt/chat/js

* Fixed JS chat tut

* Order of steps in js chat guide updated for easier following-along

* Realtime chat Vue Guide

* feat: create react.js file

* feat: add set up for directus project

* docs: create react boilder plate

* docs: initialize connection

* docs: set up submission methods

* docs: establish websocket connection

* docs: subscribe to messages

* docs: create new messages

* docs: display new messages

* docs: display historical messages

* docs: next steps

* docs: full code sample

* docs: clean up

* docs: add name to contributors

* docs: add react card

* docs: updates to react chat

* Added live poll result guide

* docs: intro

* docs: before you begin

* docs: install packages

* docs: authenticate connection

* docs: query and mutation

* docs: utilize hooks

* docs: subscribe to changes

* docs: create helper functions

* docs: display messages

* docs: summary

* docs: full sample code

* chore: add card for webscockets with graphql

* docs: intro

* docs: subscribe to changes

* docs: handling changes

* docs: crud operations

* docs: unsubscribing from changes

* docs: updates

* chore: add card

* chore: updates to graphql docs

* chore: updates to getting started

* chore: updates to subscription

* chore: updates to real chat guide

* Added WebSockets Operations Guide

* Consistent titles

* Contributors component for docs

* Triggering Netlify

* Add operations to sidebar

* Fix operations link

* Small formatting changes

* Clarity around property values

* Removed unused values in Contributors component

* Prompt for default choice

* Tabs & lowercase doctypes

* Semicolons

* Event overwerites -> event listeners

* Spacing

* Flipped order of websockets guide to match GQL

---------

Co-authored-by: Esther Agbaje <folasadeagbaje@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* fixed typo

* removed unused import

* added tests for "to-boolean" and "exceptions"

* added websocket service tests

* quote environment variable to satisfy dictionary

* GraphQL Subscriptions update (#18804)

* updated graphql subscription structure

* updated graphql examples

* Create hungry-points-rescue.md

* using `key` instead of `ID` on the toplevel

* removed changeset

* fixed the graphql type after the rename to `key`

* retrun data null for delete events to prevent non-nullable gql error

* updated missed ID reference in the docs

* updated missed ID reference in the docs

* renamed "payload" to "data" in the REST Subscription response

* fixed missed reference to payload

* added optional event filter for REST subscriptions

* updated docs for event filter

* Update docs/guides/real-time/subscriptions/websockets.md

Co-authored-by: ian <licitdev@gmail.com>

---------

Co-authored-by: ian <licitdev@gmail.com>

* added messenger unit test

* always send subscription confirmation

* Add event to subscription options

* Update tests

* Add tests for event filtering

* Revert testing for all vendors

* Remove obsolete console comment

* Update comment

* Correct event in JS WS guide

* Fix collection name to match name used in subscription

* Fix collection name in other guides

* Fix diffs in doc & enhance chart example

* Complete sentence in GraphQL guide

* Small update to config description

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Kevin Lewis <kvn@lws.io>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Esther Agbaje <folasadeagbaje@gmail.com>
2023-06-08 18:54:50 +00:00
rijkvanzanten
619d1f5726 v10.2.1 2023-05-26 14:21:39 -04:00
rijkvanzanten
ca73cc976e v10.2.0 2023-05-26 13:17:06 -04:00
Pascal Jufer
bce32d90a7 Implement update notifier in directus package (#18738)
* Include update notifier in `directus`

* Don't bundle

* Move updateCheck function to `update-check` package

* Create kind-shoes-jam.md

* Update kind-shoes-jam.md

* Remove wrong dep
2023-05-26 09:55:23 -04:00
Rijk van Zanten
2536d9e90b Add support for FILES_MIME_TYPE_ALLOW_LIST env var (#18736)
* Install minimatch

* Add support for `FILES_MIME_TYPE_ALLOW_LIST`

* Add reference

* Add changeset

* Update docs/self-hosted/config-options.md

Co-authored-by: ian <licitdev@gmail.com>

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-05-25 17:25:47 -04:00
Pascal Jufer
4683227272 Update openapi3-ts to v4 (#18722)
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-05-25 17:36:32 +02:00
Nicola Krumschmidt
60be3c2b40 Add synchronization across horizontally scaled instances to schedule flows and hooks (#18584)
* Add synchronization to schedule flows and hooks

Fixes #15052

* Add changeset

* Add test

* Add to sequential list

* Fix spelling in changeset

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-05-24 17:16:42 -04:00
Pascal Jufer
7ecda9b2fa Clean-up package.json files (#18663)
* Remove publishConfig from all packages

* Remove invalid version field

* Move @directus/composables to devDependencies in app

* Resort all package.json files

* Align meta fields

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-19 17:54:10 -04:00
Rijk van Zanten
c7d0cd0c74 Move tsconfig package to monorepo (#18659)
* Move tsconfig package to monorepo

* Use local reference to tsconfig
2023-05-19 12:30:45 -04:00
Pascal Jufer
2e647d80e6 Patch Tuesday 🐸 (#18643)
* Update `directus-monorepo` deps

* Update `@directus/api` deps

* Update `@directus/app` deps

* Update `@directus/composables` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* Update `@directus/data` deps

* Update `@directus/data-driver-postgres` deps

* Update `@directus/extensions-sdk` deps

* Update `@directus/pressure` deps

* Update `@directus/random` deps

* Update `@directus/release-notes-generator` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* Update `docs` deps

* Update `tests-blackbox` deps

* Bump pnpm to 8.5.1

* Bump build-push-action in release workflow to v4

* Assign new spy instances before each test

See https://github.com/vitest-dev/vitest/pull/3386

* Create clean-needles-nail.md

* Remove leftover 'beforeAll' imports

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-05-19 10:48:21 -04:00
Pascal Jufer
a708ec79d8 Redact additional keys in logs of Flows (#18534)
* Redact additional keys in logs of Flows

* Create moody-poems-pump.md

* Move REDACTED_TEXT to constants package & update tests

* Revert "Move REDACTED_TEXT to constants package & update tests"

This reverts commit 0f5b227253.

* Update redacted value in blackbox test

* Use own redact implementation

* Move REDACTED_TEXT to constants package

* Replace outdated comment

* Fix misleading return type

Since values might change (redacted), output isn't necessarily the same type anymore
2023-05-19 09:41:04 -04:00
rijkvanzanten
fa8c914978 v10.1.1 2023-05-16 14:03:02 -04:00
Brainslug
de89e77ad5 Update vm2 dependency (#18616) 2023-05-16 09:24:45 +00:00
rijkvanzanten
703c5d9c69 v10.1.0 2023-05-11 17:22:00 -04:00
Rijk van Zanten
20912f1645 Revert "Use vitest run instead of --watch=false (#18540)" (#18542)
This reverts commit 97f0af669e.
2023-05-10 13:33:28 -04:00
Rijk van Zanten
97f0af669e Use vitest run instead of --watch=false (#18540) 2023-05-10 13:24:30 -04:00
Rijk van Zanten
50e2c4220d Standardize TSConfig use (#18538)
* Use tsconfig.prod.json in packages

* Use prod tsconfig in api

* Install node types in storage
2023-05-10 13:22:30 -04:00
Rijk van Zanten
b56fc107a5 Add Pressure-based rate limiter (#17873)
* Start setting up @directus/pressure

* Build pressure middleware

* Add basic readme

* Install @directus/pressure

* Fix this binding

* Experiment

* Add defaults util

* Cleanup

* Fix export

* Use directus defaults

* Start tests

* Add random-utils package

* Finish testing for monitor

* Add prod deployment

* Stop building test files in prod

* My favorite

* Integrate pressure handler

* Add decent defaults

* Add retry header + custom error support

* Clean-up merge conflict & sort imports

* Fix build

* Remove default value for retry after

* Verify sampleInterval value

* ran eslint

* updated package lock

* updated vitest

* Create slimy-zebras-jam.md

* Added basic docs for config options

* updated pnpm lock and changeset

* Update & align new packages

* Update .changeset/slimy-zebras-jam.md

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
2023-05-10 14:17:53 +00:00
Brainslug
4d1d0e61dd Updated vitest (#18483)
* updated vitest to 0.31.0

* Create late-tables-call.md
2023-05-04 11:29:54 -04:00
Rijk van Zanten
2bc226d16f Change license to BSL-1.1 (#18330)
* Change license to BSL1.1

* Mark major version

* Fix formatting

* Remove duplicate heading

* Update formatting

* Copy paste error

* Update license

* Update license

* Format covenants

* Update readme.md

* Update licenses for packages

* Update readme.md

* Update contributors.yml

OG... but to be clear, rijkvanzanten is the one who really built all of this ❤️

* Tweak license in openapi

* Update packages/specs/src/openapi.yaml

* Add MIT license headers

* Use v10 in examples

* Update additional examples

* Update generate-extensions-entrypoint.test.ts

* Update tests to use latest v9 and v10 versions

* use lowercase for naming consistency

* change casing for api license

* Update migrations doc

* Update dictionary

* Consistent ordering of license field in app/package.json

* Use major version in specs again (but as string)

So it's valid for all v10 versions

* Consolidate readme's

---------

Co-authored-by: Ben Haynes <ben@directus.io>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-04-26 14:10:42 -04:00
rijkvanzanten
381f1c9d8e v9.26.0 2023-04-24 16:40:53 -04:00
Pascal Jufer
17afb9a3bc Patch Tuesday 🐸 (#18260)
* Update `@directus/api` deps (minor)

 @aws-sdk/client-ses          3.292.0  →   3.316.0
 @godaddy/terminus             4.11.2  →    4.12.0
 @rollup/plugin-alias           4.0.3  →     4.0.4
 @rollup/plugin-node-resolve   15.0.1  →    15.0.2
 @types/node                  18.15.3  →  18.15.11
 @vitest/coverage-c8           0.29.3  →    0.30.1
 axios                          1.3.4  →     1.3.6
 fs-extra                      11.1.0  →    11.1.1
 helmet                         6.0.1  →     6.1.5
 ioredis                        5.3.1  →     5.3.2
 joi                           17.8.4  →    17.9.1
 liquidjs                      10.6.1  →    10.7.0
 marked                        4.2.12  →     4.3.0
 nanoid                         3.3.4  →     3.3.6
 rollup                        3.19.1  →    3.20.6
 sharp                         0.31.3  →    0.32.0
 vitest                        0.29.3  →    0.30.1
 vm2                           3.9.16  →    3.9.17

* Update `@directus/api` deps (major)

 @rollup/plugin-alias     4.0.4  →     5.0.0
 @types/inquirer          8.2.6  →     9.0.3
 @types/node           18.15.11  →  18.15.12
 camelcase                6.3.0  →     7.0.1
 chalk                    4.1.2  →     5.2.0
 commander                9.5.0  →    10.0.1
 execa                    5.1.1  →     7.1.1
 icc                      2.0.0  →     3.0.0
 inquirer                 8.2.5  →     9.1.5
 ~ldapjs                   2.3.3  →     3.0.2~
 nanoid                   3.3.6  →     4.0.2
 ~openapi3-ts              3.2.0  →     4.1.1~
 ora                      5.4.1  →     6.3.0
 strip-bom-stream         4.0.0  →     5.0.0
 tedious                 15.1.3  →    16.0.0
 typescript               4.9.5  →     5.0.4

* Update `@directus/app` deps

 @babel/core                             7.21.3  →    7.21.4
 @babel/preset-env                       7.20.2  →    7.21.4
 @fortawesome/fontawesome-svg-core        6.3.0  →     6.4.0
 @fortawesome/free-brands-svg-icons       6.3.0  →     6.4.0
 @fullcalendar/core                       6.1.4  →     6.1.5
 @fullcalendar/daygrid                    6.1.4  →     6.1.5
 @fullcalendar/interaction                6.1.4  →     6.1.5
 @fullcalendar/list                       6.1.4  →     6.1.5
 @fullcalendar/timegrid                   6.1.4  →     6.1.5
 @pinia/testing                          0.0.15  →    0.0.16
 @popperjs/core                          2.11.6  →    2.11.7
 @storybook/addon-actions            7.0.0-rc.4  →     7.0.6
 @storybook/addon-backgrounds        7.0.0-rc.4  →     7.0.6
 @storybook/addon-docs               7.0.0-rc.4  →     7.0.6
 @storybook/addon-essentials         7.0.0-rc.4  →     7.0.6
 @storybook/addon-links              7.0.0-rc.4  →     7.0.6
 @storybook/addon-mdx-gfm            7.0.0-rc.4  →     7.0.6
 @storybook/addon-measure            7.0.0-rc.4  →     7.0.6
 @storybook/addon-outline            7.0.0-rc.4  →     7.0.6
 @storybook/client-api               7.0.0-rc.4  →     7.0.6
 @storybook/client-logger            7.0.0-rc.4  →     7.0.6
 @storybook/vue3                     7.0.0-rc.4  →     7.0.6
 @storybook/vue3-vite                7.0.0-rc.4  →     7.0.6
 @types/diff                              5.0.2  →     5.0.3
 @types/dompurify                         3.0.0  →     3.0.2
 @types/lodash                         4.14.191  →  4.14.194
 @vitejs/plugin-vue                       4.0.0  →     4.1.0
 @vue/test-utils                          2.3.1  →     2.3.2
 apexcharts                              3.37.1  →    3.39.0
 axios                                    1.3.4  →     1.3.6
 dompurify                                3.0.1  →     3.0.2
 happy-dom                                8.9.0  →     9.8.4
 marked                                  4.2.12  →     4.3.0
 nanoid                                   4.0.1  →     4.0.2
 pinia                                   2.0.33  →    2.0.34
 sass                                    1.59.3  →    1.62.0
 storybook                           7.0.0-rc.4  →     7.0.6
 typescript                               4.9.5  →     5.0.4
 vite                                     4.1.4  →     4.2.2
 vitest                                  0.29.3  →    0.30.1
 webpack                                 5.76.2  →    5.80.0

* Update root deps

* Update `@directus/composables` deps

* Update `@directus/constant` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* tsconfig workaround: ignoreDeprecations

* Update `@directus/extensions-sdk` deps

* Update `@directus/schema` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* tsconfig workaround in schema: ignoreDeprecations

* tsconfig workaround in remaining packages: ignoreDeprecations

* Update `tests-blackbox` deps

* Revert "tsconfig workaround: ignoreDeprecations"

This reverts commit 5d97da55e3.

* Revert tsconfig override

* Update tsconfig

* Fix imports in @directus/utils

* Fix imports in composables

* Fix imports in extensions-sdk

* Fix imports in @directus/api

* Move RateLimiterAbstract to types import as well

* Bump pnpm to 8.3.1

* Update `docs` deps

> [...] the @vueuse/head package will be deprecated. If you're setting up this package on a new project, you should use the @unhead/vue package directly [...]

* Remove obselete dep `concurrently` from extensions-sdk

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* New day, new updates

Also forgot to include minor updates of deps in `tests-blackbox` in previous
commit

* Fix `api` tests

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-04-21 12:59:27 -04:00
Brainslug
ac66c49a82 updated vm2 to mitigate https://github.com/patriksimek/vm2/security/advisories/GHSA-ch3r-j5x3-6q2m (#18274) 2023-04-20 17:09:54 +02:00
rijkvanzanten
835b0de89d v9.25.2 2023-04-20 10:49:53 -04:00
Nicola Krumschmidt
054188d810 Fix extension reloading (#18257)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-19 22:45:36 +02:00
rijkvanzanten
c301cfbc82 v9.25.1 2023-04-13 16:32:58 -04:00
rijkvanzanten
4f540b03d0 v9.25.0 2023-04-13 13:32:44 -04:00
Brainslug
0bb02cb78c security update in the vm2 dependency (#18162) 2023-04-13 12:47:42 +02:00
Pascal Jufer
dcc246e165 Introduce new directus wrapper package (#18134)
* Maybe?

* Maybe maybe

* ESM & version update

* Adjust Dockerfile to new wrapper package

* Clean-up api package.json file

* Update directus/package.json

* Update directus/package.json

* Update directus/package.json

* Regen pnpm lock

* Declare engines in new wrapper package

* Use wrapper version in app-settings

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-04-11 15:25:58 -04:00
Pascal Jufer
15b91dee34 Blackbox tests restructuring (#18122)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-11 18:28:37 +02:00
Pascal Jufer
b4e9afcc88 Provide filename via Content-Disposition when accessing asset (#16809)
* Provide filename via Content-Disposition when accessing asset

* Remove unused const

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-04-11 11:52:09 -04:00
bicouy0
d9d9901b02 serve rollup output chunks (#18084)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-04-11 00:09:00 +02:00
Rijk van Zanten
54abbd1970 Update keyv/redis (#18119) 2023-04-10 15:22:22 -04:00
Brainslug
e56a6715be security update in the vm2 dependency (#18112) 2023-04-08 19:32:55 +02:00
Rijk van Zanten
9ec6606694 Bro thats like just your opinion bro (#18106) 2023-04-07 16:28:57 -04:00
Brainslug
b01896314f Go back to tsx for api scripts for Windows compatibility (#18066)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-04-05 22:23:11 +00:00
Rijk van Zanten
2983e61870 The Great TypeScript Modernization Program Season 3 Episode 6: The Big One (#18014)
* Step 1

* Step 2

* False sense of confidence

* Couple more before dinner

* Update schema package

* Update format-title

* Upgrade specs file

* Close

* Replace ts-node-dev with tsx, and various others

* Replace lodash with lodash-es

* Add lodash-es types

* Update knex import

* More fun is had

* FSE

* Consolidate repos

* Various tweaks and fixes

* Fix specs

* Remove dependency on knex-schema-inspector

* Fix wrong imports of inspector

* Move shared exceptions to new package

* Move constants to separate module

* Move types to new types package

* Use directus/types

* I believe this is no longer needed

* [WIP] Start moving utils to esm

* ESMify Shared

* Move shared utils to  @directus/utils

* Use @directus/utils instead of @directus/shared/utils

* It runs!

* Use correct schemaoverview type

* Fix imports

* Fix the thing

* Start on new update-checker lib

* Use new update-check package

* Swap out directus/shared in app

* Pushing through the last bits now

* Dangerously make extensions SDK ESM

* Use @directus/types in tests

* Copy util function to test

* Fix linter config

* Add missing import

* Hot takes

* Fix build

* Curse these default exports

* No tests in constants

* Add tests

* Remove tests from types

* Add tests for exceptions

* Fix test

* Fix app tests

* Fix import in test

* Fix various tests

* Fix specs export

* Some more tests

* Remove broken integration tests

These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these

* Regenerate lockfile

* Fix imports from merge

* I create my own problems

* Make sharp play nice

* Add vitest config

* Install missing blackbox dep

* Consts shouldn't be in types

tsk tsk tsk tsk

* Fix type/const usage in extensions-sdk

* cursed.default

* Reduce circular deps

* Fix circular dep in items service

* vvv

* Trigger testing for all vendors

* Add workaround for rollup

* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"

* Fix postgres

* Schema package updates

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Resolve cjs/mjs extensions

* Clean-up eslint config

* fixed extension concatination

* using string interpolation for consistency

* Revert MySQL optimisation

* Revert testing for all vendors

* Replace tsx with esbuild-kit/esm-loader

Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js

Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467

* Use exact version for esbuild-kit/esm-loader

* Fix import

---------

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-04-04 17:41:56 -04:00
rijkvanzanten
517e7cb55c v9.24.0 2023-04-04 17:13:31 -04:00
rijkvanzanten
6b9982eae2 v9.23.4 2023-03-24 11:34:21 -04:00
rijkvanzanten
21a0b910ac v9.23.3 2023-03-23 14:11:49 -04:00
rijkvanzanten
74508c70b8 v9.23.2 2023-03-23 13:35:11 -04:00
Pascal Jufer
0293c88aa3 Patch Tuesday 🐸 (#17811)
* Update root deps

* Update `directus` deps

* Update `@directus/app` deps

* Update usage of knex-mock-client

* Update `@directus/shared` deps

* Update `extensions-sdk` deps

* Update `@directus/specs` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/utils` deps

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-03-17 15:15:03 +08:00
Pascal Jufer
5209f81879 Clean-up scripts in package.json files (#17810)
* Clean-up scripts in package.json files

* Remove superfluous comma
2023-03-16 11:09:29 +01:00
Rijk van Zanten
0154878f27 Improve assets performance (#17734)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-03-08 18:28:36 +01:00
rijkvanzanten
e468f58eec v9.23.1 2023-03-01 09:51:57 -05:00
rijkvanzanten
47b9824954 v9.23.0 2023-02-28 12:51:36 -05:00
Pascal Jufer
37faf3bd5d Patch Tuesday 🐸 (#17530)
* Update root deps

* Reformat files with new prettier version

* Update `directus` deps

* Update `@directus/app` deps

* Update `@directus/shared` deps

* Update components snapshots after updates

prop order has changed

* Update `extensions-sdk` deps

* pin version of zod in shared

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `storage-driver-cloudinary` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/utils` deps

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-02-16 17:47:39 +08:00
Pascal Jufer
26318dbe62 Patch Tuesday 🐸 (#17353) 2023-01-31 23:15:49 -05:00
Azri Kahar
89fcb19a16 Expose schema endpoints (#14713)
* Expose schema endpoints

* respond with 204

* fix payload for export flag

* allow export as YAML

* use SchemaService

* move getStringFromStream to utils

* clearer exceptions with logs

* check empty request body

* specs

* add bypassCache flag and remove flushCaches calls

* Identify integer typed fields in Oracle

* Fix uuid foreign keys type for Oracle

* Detect autoincrement for CockroachDB

* Bypass cache for fields and relations services

* Add same DB tests

* Test if working for all vendors

* Add schema snapshot testing into sequential flow

* Fix schema re-apply attempts when it's a different database vendor (#14816)

* prevent diffing on certain properties

* fix apply-snapshot test

* prevent updates to autoincrement fields' schema

* sanitize relations

* remove `colleciton.schema.sql` parsing in test

It is now being sanitized by `sanitizeCollection()` function in get-snapshot-diff.ts

* Change concurrency group to not cancel test running on main

* remove multipart for apply & add hash checking

* check incoming version & add force query param

* refine error message & stub for nested hash

* add `vendor` top-level property

as an additional safety net for cross db vendor applies

* sanitize generated schema snapshots

* snapshot joi validation

* diff joi validation

* minor cleanup

* extract applyDiff & use deep-diff applyChange

* use applyDiff in schema service

* Mark vendor as optional

* Update tests to apply diffs

* move force flag check into service

* Patch mssql vendor typo

* Set relation no action in cyclic relations

* Update mysql timestamp default value

* Oracle cannot define no action

* Update oracle timestamp type default values

* add hash checking for outdated diffs

* fix diff service & endpoint

* Add hashes when returning after

* Fix self referencing relations for Oracle

* Add temp fix for CURRENT_TIMESTAMP defaults in Oracle

* clean up driver and database client types

* only require diff hash if not kind N

* update hash comparison logic for create/delete

* Set no action constraint for self referencing M2M in MSSQL

* Add basic hash check tests

* omit default value when hashing auto increments

Specifically for CockroachDB with differing sequences

* add vendor check

* update specs

* Validate vendors with type definition

* Spread the vendors input array

* re-add Export component

* re-add js-yaml to root

* Propagate mutation options for schema apply

* Verify that snapshots differ and clear cache before hash tests

* Fix unit test

* Revert temp fix for CURRENT_TIMESTAMP defaults in Oracle

* Define and reuse type for export format

* Define and reuse list of database clients

* change `were` to `was`

* change `where` to `were`

* add some empty lines for readability

* tweak exception message

* fix test

* use object-hash

* use boolean to check whether file is included

* simplify request content type check

* throw error when multiple files were uploaded

* use nullish coalesce instead of short circuit

* Update api/src/services/schema.ts

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Remove unnecessary `Omit` on `SnapshotWithHash` type

* Revert "Remove unnecessary `Omit` on `SnapshotWithHash` type"

This reverts commit d22ac771ec.

* check empty snapshot earlier

* use allow-list logic via pick instead of omit

* Update api/src/services/schema.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Move to own /schema endpoint controller

* Fix refs to schema endpoint

* move streamToString to utils package

* move get-versioned-hash and add test

* extract kind into an enum

* Fix mysql5 timestamp value

* Fix test collection existing on rerun

* resolve TODO comment in blackbox test

* Drop deep level hashes in diff

These hashes are used only for more accurate error reporting
but are otherwise superfluous, since changes are already
detected by the top level hash.
Therefore we remove them in favor of a simpler diff format and
implementation.

* Revert schema "fix" for createItem, add comment

* Strict diff schema validation

* Revert CrDB auto-increment detection patch in 816c998

* Clear systemCache to expose newly added fields

* Use DiffKind constants

* Extract diff & snapshot validation into own tested utils

* Apply suggestions from @azrikahar

* Update knex-schema-inspector to 3.0.1

Includes the fix for CrDB auto-increment detection (knex/knex-schema-inspector#135)

* Update knex-schema-inspector in packages

* Update lock file

* add test for schema service

* add test for export service

* add relevant tests to modified util functions

* fix csv test to account for os end of line

* fix files controller test

* dedupe test data for schema service

* Align schema specs with docs

* Update api/src/controllers/schema.ts

* Revert testing for all vendors

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-01-31 17:36:49 +00:00