Commit Graph

203 Commits

Author SHA1 Message Date
Pascal Jufer
5e72ea1b08 Patch Tuesday 🐸 (#20714) 2023-12-12 04:36:50 +01:00
Pascal Jufer
ba7fac402e Update Prettier v3.1.0 (#20481) 2023-11-20 16:23:22 +01:00
Pascal Jufer
976ebf1330 Refactor env mocking (#20384) 2023-11-10 16:32:35 +01:00
Brainslug
87cc9cb9a8 Validate the query for SEARCH requests (#20253)
* validate the query used in SEARCH requests

* Create ten-onions-shop.md
2023-10-30 11:18:39 -04:00
Azri Kahar
c80f8ee247 Fix caching for Content Versioning (#20153) 2023-10-26 11:25:09 +02:00
Pascal Jufer
42d4f2d56a Merge next into main (#20107)
* Content Versioning (#19463)

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update clean-berries-do.md

* Prerelease mode

* v10.7.0-beta.0

* Ignore format of auto-generated pre.json file

* Update naming conventions for Content Versioning (#19678)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Content versioning tweaks (#19700)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Add additional system fields to Content Versioning & include preview in promote drawer (#19752)

* Use key instead of name for Content Versioning (#19813)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Reorder component tags

* Prompt user to keep or delete the version before promoting (#19829)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Fix multiple constraints to users system table from versions system table for MSSQL (#19912)

* fix multiple constraints from versions to users

* fix users unit test

* Add Sorting by Aggregated Queries (#19568)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Update API References for Content Versioning (#19911)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Add version argument in GraphQL (#19910)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Merge branch 'main' into next (#19974)

* Show installed extensions in settings pane (#19964)

* Setup boilerplate for extensions page

* Allow reading all extensions from the root endpoint

* Add basic sidebar info detail

* Render list of extensions

* Move extensions manager to folder

* Move stuff related to extensions to @directus/extensions

* Import extensions utils/types/etc from ext. package

* Resolve last imports

* Add no-extensions warning

* Start on extensions grouped

* Remove string type

* Improve naming conventions

* Couple more optimizations

* Add more structure

* Add comments

we finally understand how this works! hehe

* Remove constructor in favor of static set

* Add more comments

* Don't filter in get-extensions

TBD if this will break things horrendously

* Sprinkle on better names / comments

* Render by section, nested bundled extensions

* Fix typo

* Live-saving tip from the formatter

* Don't know where this came frmo

* Add a changeset

* 📈 Insights QoL Part 1 (#19467)

* Start adding Create Default Modal

* Add Metric List

* Number Formatter Utility

* Start on Label Formatting Options and Resizing

* Update Metric List to use New Number Formatter

* fix auto fit and styling

* fix typings for number formatter and error catching

* add conditional formatting to metric list

* Add Number Formatting to Metrics and Size Formats

* Add Option to Hide Percentage and Fix Sizing of Meter

* Add manual sizing for labels

* Add translation strings

* Add comments to insights

* Fix color errors and add icon

* update translations

* Refine Linechart

* Revert "Start adding Create Default Modal"

This reverts commit 184dd0bb73.

* Run prettier

* Remove Console Logging

* Fix Typings

* Create itchy-berries-rule.md

* Format file

* Fix Metric List for Aggre Group PR

* Remove Empty Class

* Run formatter

* Fix linter errors

* Fix import

* Update app/src/panels/label/panel-label.vue

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update app/src/panels/label/panel-label.vue

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update app/src/panels/line-chart/panel-line-chart.vue

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Remove commented css

* Remove unneeded logic

* Fix auto resizing and text alignment

* Remove Expect Error Typechecking

* Add Migration

* Fix Prettier

* Lint auto-fixes

* Fix Resizing CSS and JS

* Add Metric List Translation

* Optimize migration

* Format files

* Fix metric-list translations

* Fix type issue in metric-list/index.ts

> Default export of the module has or is using private name 'Aggregate'.ts(4082)
> Default export of the module has or is using private name 'Group'.ts(4082)

* Remove Unit Translations

* De-duplicate migration

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* SDK support for Content Versioning (#20000)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Add basic support for theming! (#20026)

* WIP start integrating theming

* Fix name

* Fix typo

* Move theme definition to json files

* Generate json from vite plugin

* Move default themes to root

* Render theme-provider in app

* Format variable names

* Move build script to separate npm run scrip
t

* Render correct var names in provider

* Render dark mode

* Update dependencies

* Setup styling rules for module bar

* Remove variables

* Don't rely on scoped var in editor

* Move light styles to theme

* Use theme vars in module bar

* Render theme styles in head

* Render alongside rather than wrapped

* Themable nav

* Allow using local devtools

* Do magic

* Fix auto theme fallback

* Update vue-i18n

* Use proper unicode symbol for multiplication

* Reorganize settings sidebar

* Use global translations for appearance options

* Add fields to migration

* Add theming group fields to system data

* Add strings to en-us for theming

* Add theming route that only renders theme group

* Add custom css to theming group

* Add system-theme interface

* Move default themes above custom css

* Move theme to global translations

* Use an array instead of object structure

* Add defaults for system fields

* Update migrations

* Pass theming information to provider

* Don't use default key in theme settings

* Use lodash.merge instead of defu

Better TypeScript type inference

* Nevermind the standalone app

It kinda sucks

* Only override rules

* Fetch theme overrides from user

* Use same field layout as settings

* Add theme overrides to db

* Classic 🤦🏻‍♂️

* Fix import

* Boilerplate overrides interface

* Update themes rules to use type def

* Boilerplate nested sections for themes

* Custom favicon (#19968)

* Add public_favicon field

* Add translation for favicon note

* Allow favicon regardless of permissions

* Return favicon as project info

* Type favicon in store

* Replace set-favicon with generate-fav util

* Render favicon in usehead

* Add changset

* Reads a little cleaner

* Semi-colons are a sign of wealth in wisdom

* Add a theme-overrides interface (#19970)

* Install typebox

* Rough in overrides interface

* Update theming settings in settings

* Upgrade dependencies

* Move theme output generation to separate composable

* Upgrade typebox

* Rename app/use-theme to use theme config

* Rework overrides interface

* Add changeset

* Allow default inheritAttrs

* Fixed handling of multiple WebSocket subscriptions on same target  (#20005)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Add a cache clear method to the UtilsService (#19990)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Change csv db type to text (#19953)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update placeholder text in file-image interface (#20004)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use current Directus version in docs (#20008)

* Use theme background instead of background-page

* Replace foreground normal with theme foreground

* Add missing translation

* Use foreground-accent and foreground-subdued from theme

* Rename --brand to --project-color

* Reuse variable color

* Replace primary/secondary/success/warning/danger

* Use theme based vars in api data

* Add header bar theming options

* Add more theming rules

* Nest icon colors

* Fix names

* Fix names in sidebar icon

* Update theming form layout

* Consolidate shades

* Consolidate primary shades

* Add primary shades

* Use shade-calculcated primary alternatives

* Use primary as mixing color

* Define warning/danger/success/secondary as shades

* Support form field label

* Add schema for interface foreground

* Allow setting input colors

* Add Google custom font loading algorithm

* Use font variables from theme

* Use unhead to load google fonts

* Install unhead to themes provider

* Use display font in title text and large dividers

* Add the dark mode default

* Run formatter

* Updating seeds is a no-no

* 😔

* Fix test

---------

Co-authored-by: Nihcep <32708027+Nihcep@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rob Lee <rob@nodeworks.com>
Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Fix build script on components package

* Fix RGB var usage

* Resolve Azri's notes

See https://github.com/directus/directus/pull/20026#pullrequestreview-1676876461

* Fix extensions icon

* Add ability to enable/disable extensions from App Settings (#20015)

* Add migrations for directus extensions table

* Prevent duplicate extensions from being loaded

* Add extension settings type

* Add settings getter with db sync

* Add commented out columns

For part 2!

* Load settings when loading extensions

* Normalize settings into extension info

* Use system-like output for extensions endpoint

* Render extensions overview from new meta type

* white         space

* Bundle shared stuff as node exports as well

* Format file

* Fix output type of nested bundle entries

* Update package lock

* Hide uninstalled extensions

* Add disable context menu

* Hide disable controls on bundles

* Add API endpoint to update enabled state in DB

* Reload extensions on enable/disable

* Add refreshing on state change for disabled

* Remove commented out code

* Remove commented fields

* Add changeset

* Image having no empty line at the end

* Update GraphQL endpoints for extensions enable/disable (#20017)

* Update type for extensions read

* Add update extension

* Add changeset

* Update docs for updates in 19988 (#20019)

* SDK methods for extension reading/updating (#20018)

* Update SDK methods for extensions

* Add changeset

* Resolve Pascal's notes

* extracted extension api output type

* updated the update command

* Update sdk/src/types/extensions.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Run formatter

* Add extensions to app deny list

* Fix input param name

* Fix typo in docs

* Azri is right, again!

* Remove extraneous fields

* Fix registration of operation and bundle extensions (#20030)

* Fix icon map

* Fix extension enable flag type on SQLite

* Fetch settings from itemservice

Fixes database inconsistency issues

* Only load extensions service when database is installed

---------

Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>

* Use theme colors in extensions settings

* 📈 Insights QoL Part 2 (#20029)

* Update Dashboard Overview Page + New Only Import Sidebar

* Update Panel Config Drawer

* Remove Old Export Function

* Add changeset

* Add Back Missing Translations

* Fix Styling

* Update Styling to use new themeing

* Fox Formatting

* Fix Last CSS Var

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Fix auto-theme issue on public pages (#20045)

* Return default appearance from server info

* Don't default to auto

* Render light/dark class based on theme appearance

* Render browser matches dynamically

* Read default appearance on public project info

* No-mo' auto

* Fix linter error

* Use unhead to set body class attr

* Content Versioning: Small tweaks & fixes (#20020)

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* fix input-rich-text-html body color css variable (#20049)

* Fix three small issues in `next` (#20051)

* Fix casting of json fields on sqlite

* Cast to json on user settings as well

* Prevent multiple versions of Vue/Pinia (#20066)

Co-authored-by: Brainslug <tim@brainslug.nl>

* Insights QoL - QA Fixes (#20063)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Add changeset for #20066 (#20071)

* Merge `main` into `next` (#20082)

* added save options and hide archive button for Content Versioning (#20095)

* Allow removal of version name (#20101)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Merge `main` into `next` (#20100)

* Add max length to `key` & `name` for Content Versioning (#20097)

* Make `next` ready for `main` (#20104)

* Durus: test for o2m (#19957)

* Fix casing in file names

* Refactor file structure

* Split up related field node

* formatter

* renamed fields-node.ts to fields.ts

* added o2m field in tdd test

* Split up tests

Co-authored-by: Nitwel <mail@nitwel.de>

* moved variables to test

* used random values in db response mock

* finalized actual test

* added some explanation

* encapsulated mock stream creation

* encapsulated receiving data from stream

* removed root prop in query

* Fix some spelling errors and variable names

* Remove mocked stream hotfix

* Reset mock after each test

* reduced test to a minimum

---------

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
Co-authored-by: Nitwel <mail@nitwel.de>

* fix styling for specific map buttons (#20047)

* fix styling for specific map buttons

* added myself to contributors.yml

* fix format

* fix location of code.

* removed extra space

* signed cla (#20058)

* Fix format of contributors.yml (#20068)

* Correct count param in BlobClient.download() (#20056)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* move api/errors to @directus/errors package (#20054)

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Fix  http range request when transformation in use (#20060)

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Fixed SDK docblock for docs build (#20075)

* Docs: Snippet Toggler Improvements (#20077)

* Enforce Vue shorthand form for `true` attribute (#20079)

* Add homepage for quick ref in root package.json (#20083)

* Apply min-width for attached menus only (#20092)

* Apply min-width for attached menus only

* Add changeset

* Content Versioning Documentation (#20070)

* docs: add content-versioning guide

* docs: add versioning to user guide items

* updates: made updates to docs

* updates: add to headless cms guide

* fix: format

* fix: typo

* Fix link to versions ref page

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Use "Content Versioning" capitalized

* Addressed comments

* Format file

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Kevin Lewis <kvn@lws.io>

* Allow disabling of field menu options (#20094)

* New Crowdin updates (#19816)

* New translations en-us.yaml (German)

* New translations en-us.yaml (Portuguese, Brazilian)

* New translations en-us.yaml (Persian)

* New translations en-us.yaml (Chinese Simplified)

* New translations en-us.yaml (Arabic)

* New translations en-us.yaml (Italian)

* New translations en-us.yaml (Dutch)

* New translations en-us.yaml (Vietnamese)

* New translations en-us.yaml (Spanish, Mexico)

* New translations en-us.yaml (Spanish)

* New translations en-us.yaml (Croatian)

* New translations en-us.yaml (Turkish)

* New translations en-us.yaml (Romanian)

* New translations en-us.yaml (French)

* New translations en-us.yaml (Afrikaans)

* New translations en-us.yaml (Bulgarian)

* New translations en-us.yaml (Catalan)

* New translations en-us.yaml (Czech)

* New translations en-us.yaml (Danish)

* New translations en-us.yaml (Greek)

* New translations en-us.yaml (Finnish)

* New translations en-us.yaml (Hebrew)

* New translations en-us.yaml (Hungarian)

* New translations en-us.yaml (Japanese)

* New translations en-us.yaml (Korean)

* New translations en-us.yaml (Lithuanian)

* New translations en-us.yaml (Norwegian)

* New translations en-us.yaml (Polish)

* New translations en-us.yaml (Portuguese)

* New translations en-us.yaml (Russian)

* New translations en-us.yaml (Slovak)

* New translations en-us.yaml (Slovenian)

* New translations en-us.yaml (Albanian)

* New translations en-us.yaml (Swedish)

* New translations en-us.yaml (Ukrainian)

* New translations en-us.yaml (Chinese Traditional)

* New translations en-us.yaml (Icelandic)

* New translations en-us.yaml (Indonesian)

* New translations en-us.yaml (Spanish, Chile)

* New translations en-us.yaml (Thai)

* New translations en-us.yaml (Estonian)

* New translations en-us.yaml (Hindi)

* New translations en-us.yaml (Malay)

* New translations en-us.yaml (English, Canada)

* New translations en-us.yaml (English, United Kingdom)

* New translations en-us.yaml (French, Canada)

* New translations en-us.yaml (Faroese)

* New translations en-us.yaml (Esperanto)

* New translations en-us.yaml (Breton)

* New translations en-us.yaml (Bosnian)

* New translations en-us.yaml (Serbian (Latin))

* New translations en-us.yaml (Nepali)

* New translations en-us.yaml (Sorani (Kurdish))

* New translations en-us.yaml (Spanish, Latin America)

* Update source file en-US.yaml

* Update source file en-US.yaml

* New translations en-us.yaml (Croatian)

* New translations en-us.yaml (Croatian)

* New translations en-us.yaml (Russian)

* New translations en-us.yaml (Italian)

* New translations en-us.yaml (Italian)

* New translations en-us.yaml (Italian)

* New translations en-us.yaml (Croatian)

* Update source file en-US.yaml

* Update source file en-US.yaml

* New translations en-us.yaml (German)

* New translations en-us.yaml (Greek)

* New translations en-us.yaml (Italian)

* New translations en-us.yaml (Greek)

* New translations en-us.yaml (Albanian)

* 10.6.4 (#20102)

* Exit prerelease mode

---------

Co-authored-by: Jan Arends <jan.arends@mailbox.org>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Boegie19 <34578426+Boegie19@users.noreply.github.com>
Co-authored-by: Dominic <dmarx@marxulm.de>
Co-authored-by: mscbpi <ms@cbpi.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Esther Agbaje <53586167+estheragbaje@users.noreply.github.com>
Co-authored-by: Kevin Lewis <kvn@lws.io>

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Connor <12714889+ConnorSimply@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Nihcep <32708027+Nihcep@users.noreply.github.com>
Co-authored-by: Rob Lee <rob@nodeworks.com>
Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
Co-authored-by: Jan Arends <jan.arends@mailbox.org>
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Boegie19 <34578426+Boegie19@users.noreply.github.com>
Co-authored-by: Dominic <dmarx@marxulm.de>
Co-authored-by: mscbpi <ms@cbpi.ch>
Co-authored-by: Esther Agbaje <53586167+estheragbaje@users.noreply.github.com>
Co-authored-by: Kevin Lewis <kvn@lws.io>
2023-10-21 00:34:57 +02:00
Boegie19
d82bb9c823 move api/errors to @directus/errors package (#20054)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-10-18 18:19:22 +02:00
Rijk van Zanten
d68ec5fda7 Cleanup imports (#19966)
* Cleanup imports

* Add changeset
2023-10-09 15:17:29 -04:00
José Varela
f96eda89e1 Import: Do task in a separate worker (#19691)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-10-04 11:39:40 +02:00
Gerard Lamusse
ad1207a2f9 Remove res.status calculation as it is set later (#19214)
* Remove res.status calculation as it is set later

* Ensure 500 is thrown on multiple errors of different types

* Add changeset

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2023-07-21 09:56:22 -04:00
Rijk van Zanten
ca3e7f521f Upgrade previous "Extensions" system to new "Errors" model (#18797)
* Typecheck across packages that are built with esbuild

* Boilerplate new Errors package

* No need, tsup checks with --dts

* Switch to tsup

* Setup dev script

* Add readme

* More boilerplaty things

* Finish createError function

* Install @directus/random

* Downgrade node types

* Add utility function to check if an error is a DirectusError

* Use new is-error check

* Install errors package

* Add failed validation common error

* Export common errors

* Move joi convertion to utils

* Export failed validation

* Use new failed validation error in validate-batch

* Enhance typing output of createError

* Remove outdir (handled by tsup now)

* Replace Exception with Error

* Replace exception in test

* Remove exceptions from app

* Remove exceptions from app

* Remove failed validation exception from users service

* Remove old failed validation exception from shared

* Remove exceptions package in favor of errors

* Uninstall exceptions

* Replace baseexception check

* Migrate content too large error

* Critical detail

* Replace ForbiddenException

* WIP remove exceptions

* Add ForbiddenError to errors

* HitRateLimitError

* Move validation related error/helper to new validation package

* Add index

* Add docs

* Install random

* Convert TokenExpired

* Convert user-suspended

* Convert invalid-credentials

* Move UnsupportedMediaType

* Replace wrong imports for forbidden

* Convert invalid-ip

* Move invalid provider

* Move InvalidOtp

* Convert InvalidToken

* Move MethodNotAllowed

* Convert range not satisfiable

* Move unexpect response

* Move UnprocessableContent

* Move IllegalAssetTransformation

* Move RouteNotFound

* Finalize not found

* Various db errors

* Move value too long

* Move not null

* Move record-not-unique

* Move value out of range

* Finish db errors

* Service unavailable

* GQL errors

* Update packages/validation/src/errors/failed-validation.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update packages/validation/src/errors/failed-validation.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* InvalidQuery

* Add test for invalid query message constructor

* Invalid Payload

* Finalize exceptions move

* Improve type of isDirectusError

* Various fixes

* Fix build in api

* Update websocket exceptions use

* Allow optional reason for invalid config

* Update errors usage in utils

* Remove unused package from errors

* Update lockfile

* Update api/src/auth/drivers/ldap.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update packages/validation/src/utils/joi-to-error-extensions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Put error codes in shared enum

* Replace instanceof checks in api

* Fix tests I think

* Tweak override names

* Fix linter warnings

* Set snapshots

* Start fixing BB tests

* Fix blackbox tests

* Add changeset

* Update changeset

* Update extension docs to use new createError abstraction

* 🙄

* Fix graphql validation error name

* 🥳

* use ErrorCode.Forbidden

* fix blackbox auth login test

* Add license files

* Rename preMutationException to preMutationError

* Remove unused ms dep & sort package.json

* Remove periods from error messages for consistency

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add optional code check

* Use updated error code checker

* Rename InvalidConfigError to InvalidProviderConfigError

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-06-27 17:22:26 -04:00
Rijk van Zanten
513a739b33 Drop support for Memcached (#18980) 2023-06-23 08:54:43 -04:00
Brainslug
cfddabd9ee Integrating Websockets in Directus 🕸️🧦 (#14737)
* added emitter context

* partial items tests

* updated items handler tests

* fixed test after merge

* forgot the event context

* fixed auth message parsing for graphql subscriptions

* fixed type strictness

* fixed graphql subscription bug

* bumped websocket dependencies

* touched up some dangling code

* updated itemsservice usage

* disabled overkill logs

* double checked environment type processing

* fixed missed capitalization

* fixed subscription payloads

* Added explicit string type casting

* removed obsolete "trimUpper" utility

* using the parseJSON utility consistently

* pinned dependencies

* parse environment variables

* fixed pnpm-lock

* GraphQL Subscriptions for all events

* fixed typo

* added event data to the graphql definition

* fix payload for delete events

* Added optional chaining for type to prevent fatal crashes on invalid messages

* fix failing on getting type from undefined

* Update api/src/websocket/exceptions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add proper ZodError handling

* added the zod-validation-error parser

* allow disabling the rate limiter

* Update api/src/websocket/controllers/base.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* updated starting logs

* fixed email/password expiration logic

* added tests for getMessageType

* simplified message parsing and dropped capitalization

* updated authenticate test

* switched to lower cased message.type to prevent spreading "toUpperCase" around

* cleaned up debug logs

* cast enabled config to boolean

* Update api/src/websocket/controllers/rest.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/subscribe.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/subscribe.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/items.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/controllers/base.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/handlers/heartbeat.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Suggested fixes by Azri

* removed redundant try-catch

* fixed authentication timeout
added returning the refresh token when authenticating

* updated pnpm lock after merge

* Fixed authentication modes for GraphQL according to best practices

* implement useFakeTimers in heartbeat unit test

* implement useFakeTimers in items unit test

* Update api/src/services/server.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* removed obsolete authentication.verbose toggle

* added email flag to message validation

* switched to ternary for consistency

* moved getSchema out of for loop

* added singleton logic to items handler

* close the socket after failed auth for non-public connections

* disabled system collections for rest subscriptions

* re-ran pnpm i

* allow for multiple subscripitions in the memory messenger

* - fixed system collection subscriptions
- abstracted hook message bus
- fixed graphql horizontal scaling

* remove logic from root context for tests

* fix reading created item

* fix linter

* typo and extra safe guard suggested by azri

* prevent setting long timeouts in favor of a shared interval

* prevent unsubscribing all existing subscriptions when omitting "uid"

* - extracted getService utility
- block system collections mutation in the items handler
- implemented the correct services for system collections

* allow numeric uid's to be used

* fixed the types for numeric uid's to be used

* added missing await's

* fixed type imports after merge

* removed unused imports

* Update api/src/websocket/controllers/hooks.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/websocket/controllers/hooks.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update api/src/messenger.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* improved error for graphql subscriptions

* fixed TS Modernization conflicts

* fixed TS Modernization conflicts

* fixed conflicts after merge

* removed unused name property

* abstracxted environment configuration

* respond to ping messages when heartbeat disabled

* something something merge

* moved toBoolean to it's own util file

* replaced old socket naming

* removed old exception

* fixed typo

* Update api/src/env.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/websocket/handlers/heartbeat.test.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/websocket/handlers/heartbeat.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/services/server.ts

Co-authored-by: ian <licitdev@gmail.com>

* fixed for linter

* add server_info_websocket in graphql

* Add base REST websocket tests

* do merge things

* fixing things

* fixed failing unit test

* Update dependencies

* Move tests

* Update lockfile

* Use new paths when spawning

* return websockets to opt-in

* Enable websockets for tests

* Test with ephemeral access token

* no camelcasing gql subscriptions

* use underscore for gql event

* Remove unused import

* Add base GraphQL subscription tests

* Fix accidental comment

* Add some relational tests

* Organize imports

Using VS Code's default organize import

* Run ESlint formatting

* One more opinionated formatting change

* Formatting

* Fix message sequence not in order

* Remove relational batch update tests

* Test horizontal scaling

* using toboolean util for server_info

* removed unneeded type cast

* found the gql request type

* extra usage of the toBoolean util

* merge the authentication middleware and get-account-for-token util

* updated utility test

* fixed middleware unit test

* Add return

* Remove user filtering and close conns

* Fix reused accountability

* fixed failing util test

* added subscription unit tests

* added missing mock

* trigger workflow

* Revert "trigger workflow"

This reverts commit 4f544b0c1b.

* Trigger testing for all vendors

* add unsubscription confirmation

* Wait for unsubscription confirmation

* Fix incorrect sending of unsubscription confirmation

* updated ubsubscribe logic

* Update count for unsubscription message

* Fix sequence for UUID pktype in MSSQL

* Increase auth timeout

* Add start index when getting messages

* Fix subscription retrieval and cast uid to string

* Remove nested ternary

* Revert "Increase auth timeout"

This reverts commit 10707409c4.

* Terminate connection instead of close

* fixed merge

* re-added missing packages after merge resolve

* fixed type imports

* Create lazy-cows-happen.md

Added changeset

* Minor bump for "directus" package as well

* fixed "strict" auth mode for graphql subscriptions

* removed nested ternary

* Add websocket tests to sequential flow

* Disable pressure limiter for blackbox tests

* fix merge

* WebSockets Documentation (#18254)

* Small repsonsive styling fix on Card

* REST getting started guide

* Authentication guide

* REST subscription guides

* JS Chat guide

* Sidebar websocket guides section

* Added config options

* Respoinding to brainslug's review

* Fixed incorrect header on guides/rt/subs

* Fixed spellchecker

* Correct full code example on guides/rt/chat/js

* Fixed JS chat tut

* Order of steps in js chat guide updated for easier following-along

* Realtime chat Vue Guide

* feat: create react.js file

* feat: add set up for directus project

* docs: create react boilder plate

* docs: initialize connection

* docs: set up submission methods

* docs: establish websocket connection

* docs: subscribe to messages

* docs: create new messages

* docs: display new messages

* docs: display historical messages

* docs: next steps

* docs: full code sample

* docs: clean up

* docs: add name to contributors

* docs: add react card

* docs: updates to react chat

* Added live poll result guide

* docs: intro

* docs: before you begin

* docs: install packages

* docs: authenticate connection

* docs: query and mutation

* docs: utilize hooks

* docs: subscribe to changes

* docs: create helper functions

* docs: display messages

* docs: summary

* docs: full sample code

* chore: add card for webscockets with graphql

* docs: intro

* docs: subscribe to changes

* docs: handling changes

* docs: crud operations

* docs: unsubscribing from changes

* docs: updates

* chore: add card

* chore: updates to graphql docs

* chore: updates to getting started

* chore: updates to subscription

* chore: updates to real chat guide

* Added WebSockets Operations Guide

* Consistent titles

* Contributors component for docs

* Triggering Netlify

* Add operations to sidebar

* Fix operations link

* Small formatting changes

* Clarity around property values

* Removed unused values in Contributors component

* Prompt for default choice

* Tabs & lowercase doctypes

* Semicolons

* Event overwerites -> event listeners

* Spacing

* Flipped order of websockets guide to match GQL

---------

Co-authored-by: Esther Agbaje <folasadeagbaje@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* fixed typo

* removed unused import

* added tests for "to-boolean" and "exceptions"

* added websocket service tests

* quote environment variable to satisfy dictionary

* GraphQL Subscriptions update (#18804)

* updated graphql subscription structure

* updated graphql examples

* Create hungry-points-rescue.md

* using `key` instead of `ID` on the toplevel

* removed changeset

* fixed the graphql type after the rename to `key`

* retrun data null for delete events to prevent non-nullable gql error

* updated missed ID reference in the docs

* updated missed ID reference in the docs

* renamed "payload" to "data" in the REST Subscription response

* fixed missed reference to payload

* added optional event filter for REST subscriptions

* updated docs for event filter

* Update docs/guides/real-time/subscriptions/websockets.md

Co-authored-by: ian <licitdev@gmail.com>

---------

Co-authored-by: ian <licitdev@gmail.com>

* added messenger unit test

* always send subscription confirmation

* Add event to subscription options

* Update tests

* Add tests for event filtering

* Revert testing for all vendors

* Remove obsolete console comment

* Update comment

* Correct event in JS WS guide

* Fix collection name to match name used in subscription

* Fix collection name in other guides

* Fix diffs in doc & enhance chart example

* Complete sentence in GraphQL guide

* Small update to config description

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Kevin Lewis <kvn@lws.io>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Esther Agbaje <folasadeagbaje@gmail.com>
2023-06-08 18:54:50 +00:00
Pascal Jufer
17afb9a3bc Patch Tuesday 🐸 (#18260)
* Update `@directus/api` deps (minor)

 @aws-sdk/client-ses          3.292.0  →   3.316.0
 @godaddy/terminus             4.11.2  →    4.12.0
 @rollup/plugin-alias           4.0.3  →     4.0.4
 @rollup/plugin-node-resolve   15.0.1  →    15.0.2
 @types/node                  18.15.3  →  18.15.11
 @vitest/coverage-c8           0.29.3  →    0.30.1
 axios                          1.3.4  →     1.3.6
 fs-extra                      11.1.0  →    11.1.1
 helmet                         6.0.1  →     6.1.5
 ioredis                        5.3.1  →     5.3.2
 joi                           17.8.4  →    17.9.1
 liquidjs                      10.6.1  →    10.7.0
 marked                        4.2.12  →     4.3.0
 nanoid                         3.3.4  →     3.3.6
 rollup                        3.19.1  →    3.20.6
 sharp                         0.31.3  →    0.32.0
 vitest                        0.29.3  →    0.30.1
 vm2                           3.9.16  →    3.9.17

* Update `@directus/api` deps (major)

 @rollup/plugin-alias     4.0.4  →     5.0.0
 @types/inquirer          8.2.6  →     9.0.3
 @types/node           18.15.11  →  18.15.12
 camelcase                6.3.0  →     7.0.1
 chalk                    4.1.2  →     5.2.0
 commander                9.5.0  →    10.0.1
 execa                    5.1.1  →     7.1.1
 icc                      2.0.0  →     3.0.0
 inquirer                 8.2.5  →     9.1.5
 ~ldapjs                   2.3.3  →     3.0.2~
 nanoid                   3.3.6  →     4.0.2
 ~openapi3-ts              3.2.0  →     4.1.1~
 ora                      5.4.1  →     6.3.0
 strip-bom-stream         4.0.0  →     5.0.0
 tedious                 15.1.3  →    16.0.0
 typescript               4.9.5  →     5.0.4

* Update `@directus/app` deps

 @babel/core                             7.21.3  →    7.21.4
 @babel/preset-env                       7.20.2  →    7.21.4
 @fortawesome/fontawesome-svg-core        6.3.0  →     6.4.0
 @fortawesome/free-brands-svg-icons       6.3.0  →     6.4.0
 @fullcalendar/core                       6.1.4  →     6.1.5
 @fullcalendar/daygrid                    6.1.4  →     6.1.5
 @fullcalendar/interaction                6.1.4  →     6.1.5
 @fullcalendar/list                       6.1.4  →     6.1.5
 @fullcalendar/timegrid                   6.1.4  →     6.1.5
 @pinia/testing                          0.0.15  →    0.0.16
 @popperjs/core                          2.11.6  →    2.11.7
 @storybook/addon-actions            7.0.0-rc.4  →     7.0.6
 @storybook/addon-backgrounds        7.0.0-rc.4  →     7.0.6
 @storybook/addon-docs               7.0.0-rc.4  →     7.0.6
 @storybook/addon-essentials         7.0.0-rc.4  →     7.0.6
 @storybook/addon-links              7.0.0-rc.4  →     7.0.6
 @storybook/addon-mdx-gfm            7.0.0-rc.4  →     7.0.6
 @storybook/addon-measure            7.0.0-rc.4  →     7.0.6
 @storybook/addon-outline            7.0.0-rc.4  →     7.0.6
 @storybook/client-api               7.0.0-rc.4  →     7.0.6
 @storybook/client-logger            7.0.0-rc.4  →     7.0.6
 @storybook/vue3                     7.0.0-rc.4  →     7.0.6
 @storybook/vue3-vite                7.0.0-rc.4  →     7.0.6
 @types/diff                              5.0.2  →     5.0.3
 @types/dompurify                         3.0.0  →     3.0.2
 @types/lodash                         4.14.191  →  4.14.194
 @vitejs/plugin-vue                       4.0.0  →     4.1.0
 @vue/test-utils                          2.3.1  →     2.3.2
 apexcharts                              3.37.1  →    3.39.0
 axios                                    1.3.4  →     1.3.6
 dompurify                                3.0.1  →     3.0.2
 happy-dom                                8.9.0  →     9.8.4
 marked                                  4.2.12  →     4.3.0
 nanoid                                   4.0.1  →     4.0.2
 pinia                                   2.0.33  →    2.0.34
 sass                                    1.59.3  →    1.62.0
 storybook                           7.0.0-rc.4  →     7.0.6
 typescript                               4.9.5  →     5.0.4
 vite                                     4.1.4  →     4.2.2
 vitest                                  0.29.3  →    0.30.1
 webpack                                 5.76.2  →    5.80.0

* Update root deps

* Update `@directus/composables` deps

* Update `@directus/constant` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* tsconfig workaround: ignoreDeprecations

* Update `@directus/extensions-sdk` deps

* Update `@directus/schema` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* tsconfig workaround in schema: ignoreDeprecations

* tsconfig workaround in remaining packages: ignoreDeprecations

* Update `tests-blackbox` deps

* Revert "tsconfig workaround: ignoreDeprecations"

This reverts commit 5d97da55e3.

* Revert tsconfig override

* Update tsconfig

* Fix imports in @directus/utils

* Fix imports in composables

* Fix imports in extensions-sdk

* Fix imports in @directus/api

* Move RateLimiterAbstract to types import as well

* Bump pnpm to 8.3.1

* Update `docs` deps

> [...] the @vueuse/head package will be deprecated. If you're setting up this package on a new project, you should use the @unhead/vue package directly [...]

* Remove obselete dep `concurrently` from extensions-sdk

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* New day, new updates

Also forgot to include minor updates of deps in `tests-blackbox` in previous
commit

* Fix `api` tests

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-04-21 12:59:27 -04:00
Rijk van Zanten
c48309ab68 Last eslint tweak (#18198)
* Should be there now

* Format
2023-04-14 17:40:50 -04:00
rijkvanzanten
37658802b7 One more 2023-04-14 17:30:56 -04:00
Rijk van Zanten
bdb1a919e2 White space rules (#18197)
* White space rules

* Auto-fix whitespace linter warnings
2023-04-14 17:21:56 -04:00
Rijk van Zanten
b7f352eec2 Fix updating payload on request.error filter (#18137)
Fixes #17002
2023-04-11 16:14:44 -04:00
Rijk van Zanten
2983e61870 The Great TypeScript Modernization Program Season 3 Episode 6: The Big One (#18014)
* Step 1

* Step 2

* False sense of confidence

* Couple more before dinner

* Update schema package

* Update format-title

* Upgrade specs file

* Close

* Replace ts-node-dev with tsx, and various others

* Replace lodash with lodash-es

* Add lodash-es types

* Update knex import

* More fun is had

* FSE

* Consolidate repos

* Various tweaks and fixes

* Fix specs

* Remove dependency on knex-schema-inspector

* Fix wrong imports of inspector

* Move shared exceptions to new package

* Move constants to separate module

* Move types to new types package

* Use directus/types

* I believe this is no longer needed

* [WIP] Start moving utils to esm

* ESMify Shared

* Move shared utils to  @directus/utils

* Use @directus/utils instead of @directus/shared/utils

* It runs!

* Use correct schemaoverview type

* Fix imports

* Fix the thing

* Start on new update-checker lib

* Use new update-check package

* Swap out directus/shared in app

* Pushing through the last bits now

* Dangerously make extensions SDK ESM

* Use @directus/types in tests

* Copy util function to test

* Fix linter config

* Add missing import

* Hot takes

* Fix build

* Curse these default exports

* No tests in constants

* Add tests

* Remove tests from types

* Add tests for exceptions

* Fix test

* Fix app tests

* Fix import in test

* Fix various tests

* Fix specs export

* Some more tests

* Remove broken integration tests

These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these

* Regenerate lockfile

* Fix imports from merge

* I create my own problems

* Make sharp play nice

* Add vitest config

* Install missing blackbox dep

* Consts shouldn't be in types

tsk tsk tsk tsk

* Fix type/const usage in extensions-sdk

* cursed.default

* Reduce circular deps

* Fix circular dep in items service

* vvv

* Trigger testing for all vendors

* Add workaround for rollup

* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"

* Fix postgres

* Schema package updates

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Resolve cjs/mjs extensions

* Clean-up eslint config

* fixed extension concatination

* using string interpolation for consistency

* Revert MySQL optimisation

* Revert testing for all vendors

* Replace tsx with esbuild-kit/esm-loader

Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js

Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467

* Use exact version for esbuild-kit/esm-loader

* Fix import

---------

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-04-04 17:41:56 -04:00
Rijk van Zanten
cb74665693 noUnusedParameters: true (#17935) 2023-03-24 16:56:18 -04:00
Rijk van Zanten
2bc039607b TS Config Modernization Program Part 4 of almost done (#17934)
* noUncheckedIndexedAccess: true !!!!!!!?!!!!!!!!!!?

* Fix duplicate non-null assertion
2023-03-24 16:43:54 -04:00
Rijk van Zanten
80f4807a09 TS Config Modernization Program Part 3 of many (#17904)
* noImplicitOverride: true

* noImplicitReturns: true

* noPropertyAccessFromIndexSignature: true
2023-03-23 16:47:55 -04:00
rijkvanzanten
fe74c43bc0 Updates 5 of many 2023-03-22 17:02:18 -04:00
Rijk van Zanten
1a5f06c791 TSConfig Modernization: Set importsNotUsedAsValues (#17887)
* Set importsNotUsedAsValues

* Fix import
2023-03-22 14:51:27 -04:00
Azri Kahar
d466204732 Add ability to limit requests from using the Cache-Control header to skip cache (#17642)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-03-02 13:19:30 +01:00
Brainslug
ee7fca3e31 Global rate limiter (#17296)
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-03-02 13:04:22 +01:00
Pascal Jufer
e3c755dcf0 Add getMilliseconds util for safer value interpretation (#17498)
* Add `getMilliseconds` util for safer value interpretation

* Test more data types

* Remove remnant

* Customizable fallback with default of undefined

* Clean-up

* Transform getMilliseconds to named export

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-02-17 11:06:26 -05:00
Pascal Jufer
45c4bc89f1 EXTENSIONS_CACHE_TTL: Add to allow list & decouple from other cache envs (#17464)
* EXTENSIONS_CACHE_TTL: Add to allow list & decouple from other cache envs

* Remove unnecessary import

* Broaden getCacheControlHeader function, use it for assets & extensions

* Add unit tests

* Apply suggestions from code review

Consistent lowercase "cache-control" in test description

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-02-14 10:28:40 -05:00
Azri Kahar
89fcb19a16 Expose schema endpoints (#14713)
* Expose schema endpoints

* respond with 204

* fix payload for export flag

* allow export as YAML

* use SchemaService

* move getStringFromStream to utils

* clearer exceptions with logs

* check empty request body

* specs

* add bypassCache flag and remove flushCaches calls

* Identify integer typed fields in Oracle

* Fix uuid foreign keys type for Oracle

* Detect autoincrement for CockroachDB

* Bypass cache for fields and relations services

* Add same DB tests

* Test if working for all vendors

* Add schema snapshot testing into sequential flow

* Fix schema re-apply attempts when it's a different database vendor (#14816)

* prevent diffing on certain properties

* fix apply-snapshot test

* prevent updates to autoincrement fields' schema

* sanitize relations

* remove `colleciton.schema.sql` parsing in test

It is now being sanitized by `sanitizeCollection()` function in get-snapshot-diff.ts

* Change concurrency group to not cancel test running on main

* remove multipart for apply & add hash checking

* check incoming version & add force query param

* refine error message & stub for nested hash

* add `vendor` top-level property

as an additional safety net for cross db vendor applies

* sanitize generated schema snapshots

* snapshot joi validation

* diff joi validation

* minor cleanup

* extract applyDiff & use deep-diff applyChange

* use applyDiff in schema service

* Mark vendor as optional

* Update tests to apply diffs

* move force flag check into service

* Patch mssql vendor typo

* Set relation no action in cyclic relations

* Update mysql timestamp default value

* Oracle cannot define no action

* Update oracle timestamp type default values

* add hash checking for outdated diffs

* fix diff service & endpoint

* Add hashes when returning after

* Fix self referencing relations for Oracle

* Add temp fix for CURRENT_TIMESTAMP defaults in Oracle

* clean up driver and database client types

* only require diff hash if not kind N

* update hash comparison logic for create/delete

* Set no action constraint for self referencing M2M in MSSQL

* Add basic hash check tests

* omit default value when hashing auto increments

Specifically for CockroachDB with differing sequences

* add vendor check

* update specs

* Validate vendors with type definition

* Spread the vendors input array

* re-add Export component

* re-add js-yaml to root

* Propagate mutation options for schema apply

* Verify that snapshots differ and clear cache before hash tests

* Fix unit test

* Revert temp fix for CURRENT_TIMESTAMP defaults in Oracle

* Define and reuse type for export format

* Define and reuse list of database clients

* change `were` to `was`

* change `where` to `were`

* add some empty lines for readability

* tweak exception message

* fix test

* use object-hash

* use boolean to check whether file is included

* simplify request content type check

* throw error when multiple files were uploaded

* use nullish coalesce instead of short circuit

* Update api/src/services/schema.ts

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Remove unnecessary `Omit` on `SnapshotWithHash` type

* Revert "Remove unnecessary `Omit` on `SnapshotWithHash` type"

This reverts commit d22ac771ec.

* check empty snapshot earlier

* use allow-list logic via pick instead of omit

* Update api/src/services/schema.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Move to own /schema endpoint controller

* Fix refs to schema endpoint

* move streamToString to utils package

* move get-versioned-hash and add test

* extract kind into an enum

* Fix mysql5 timestamp value

* Fix test collection existing on rerun

* resolve TODO comment in blackbox test

* Drop deep level hashes in diff

These hashes are used only for more accurate error reporting
but are otherwise superfluous, since changes are already
detected by the top level hash.
Therefore we remove them in favor of a simpler diff format and
implementation.

* Revert schema "fix" for createItem, add comment

* Strict diff schema validation

* Revert CrDB auto-increment detection patch in 816c998

* Clear systemCache to expose newly added fields

* Use DiffKind constants

* Extract diff & snapshot validation into own tested utils

* Apply suggestions from @azrikahar

* Update knex-schema-inspector to 3.0.1

Includes the fix for CrDB auto-increment detection (knex/knex-schema-inspector#135)

* Update knex-schema-inspector in packages

* Update lock file

* add test for schema service

* add test for export service

* add relevant tests to modified util functions

* fix csv test to account for os end of line

* fix files controller test

* dedupe test data for schema service

* Align schema specs with docs

* Update api/src/controllers/schema.ts

* Revert testing for all vendors

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-01-31 17:36:49 +00:00
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00
ian
8a3dc4b68b Propagate mutation options for schema apply (#16562)
* Propagate mutation options for schema apply

* Fix unit test

* Add bypassCache flag and remove flushCaches calls

* remove accountability option from getSchema

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2022-11-22 14:57:17 -05:00
Pascal Jufer
45eb4941fc Small clean-up after Jest to Vitest switch in API (#16462)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-15 17:20:45 -05:00
Pascal Jufer
c303bdcf10 Finish switch from Jest to Vitest in API (#16263)
* build:  add vitest and update test scripts

* build: 🔧 add vitest config

* build:  Migrate tests to vitest

Remove jest references from test api test files and replace with vitest equivalents.

Tests: 13 tests are failing.

* build: 🚚 move vite.config.ts to api/src folder

* build: 🔥 remove unused vitest.config from api root

* build:  import vitest modules for tests

* build:  add type conversion for actual object

* Finish switch from Jest to Vitest in API

* Replace some leftovers

* Load "sharp" before tests

* Try with cjs

* Temporary enable verbose reporter

* Try with globalSetup

* Fix path to globalSetup

* Provide default export in globalSetup

* Final clean-up

* Remove @vitest/ui & update vitest to 0.25.0

* Add vitest c8 coverage dependency

* Update vitest to v0.25.1

* Replace unnecessary Vitest workaround

* Rework new tests

* Resolve build errors

Co-authored-by: Dorian C Brown <brown.3794@gmail.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-11-14 16:09:47 -05:00
José Varela
54cd2d47da Cache: Fix check if endpoint is /graphql or not (#15599) 2022-09-16 09:12:00 -04:00
Rijk van Zanten
0de05b40a7 Don't reset body on singleton non-SEARCH reqs (#15369)
Fixes #15364
2022-09-02 11:11:52 -04:00
ian
d29c2dfc3f Fix lint warnings (#15358) 2022-09-02 00:08:23 -04:00
Rijk van Zanten
1f66049c42 Fix SEARCH query not functioning on singleton collections (#15339)
* Don't short-circuit on batch read singletons in search methods

* Add tests

100% babyyyyyyy

* Cleanup middleware
2022-08-31 11:59:03 -04:00
ian
a971455216 Add origin to accountability (#15273)
* Add origin to accountability

* Remove origin column from seeds
2022-08-30 14:15:44 -04:00
Rijk van Zanten
19ddd89d8d Move unit tests to files they apply to (#15130)
* Move unit tests to files they apply to

* Remove tests root
2022-08-17 16:01:16 -04:00
José Varela
cc343fdf91 Improve cache performance by compressing records (#14833)
* Utils to compress/decompress data
Gzip was chosen because we want smaller data but quick algorithm since this will be ran for every request

* Compress system cache

* Decompress system cache

* Set/Get compressed cache for individual requests

* Switch from gzip to snappy, use json compression too

* Fix cache exp set/get

* Remove unused import

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-08-04 17:35:27 -04:00
Rijk van Zanten
838770b2be Add GraphQL query POST caching (#14371)
* Add GraphQL query POST caching

Followup to #11462

* Fix condition check
2022-07-14 12:48:47 -04:00
Rijk van Zanten
32dd709778 Insights 2.0 (#14096)
* query function added to list

* dashboard reading query, adding to object

* typecasting of filter vals needed still

* numbers accepting strings too

* json-to-graphql-query => devD

* fixed unneeded return in list index.ts

* stitching and calling but not actually calling

* calls on panel change

* query object += new panel before dashboard save

* uuid generated in app not api

* fixed panel ids in query

* fixed the tests I just wrote

* passing the query data down!

* list showing data

* objDiff test moved to test

* metric bug fixes + data

* dashboard logic

* time series conversion started

* timeseries GQL query almost there

* query querying

* chart loading

* aggregate handling improved

* error handling for aggregate+filter errors

* removed query on empty queryObj

* maybe more error handling

* more error handling working

* improvements to erorr handling

* stitchGQL() error return type corrected

* added string fields to COUNT

* pushing up but needs work

* not an endless recursion

* its not pretty but it works.

* throws an error

* system collections supported

* refactor to solve some errors

* loading correct

* metric function fixed

* data loading but not blocking rendering

* removed redundant code.

* relational fields

* deep nesting relations

* options.precision has a default

* relational fields fix. (thanks azri)

* the limit

* limit and time series

* range has a default

* datat to workspace

* v-if

* panels loading

* workspaces dont get data anymore

* package.json

* requested changes

* loading

* get groups util

* timeseries => script setup

* list => script setup

* metric => script setup

* label => script setup

* declare optional props

* loadingPanels: only loading spinner on loading panels

* remove unneeded parseDate!!

* applyDataToPanels tests

* -.only

* remove unneeded steps

* processQuery tests

* tests

* removed unused var

* jest.config and some queryCaller tests

* one more test

* query tests

* typo

* clean up

* fix some but not all bugs

* bugs from merge fixed

* Start cleaning up 🧹

* Refactor custom input type

* Small tweaks in list index

* Cleanup imports

* Require Query object to be returned from query prop

* Tweak return statement

* Fix imports

* Cleanup metric watch effect

* Tweaks tweaks tweaks

* Don't rely on options, simplify fetch logic

* Add paths to validation errors

* [WIP] Start handling things in the store

* Rework query fetching logic into store

* Clean up data passing

* Use composition setup for insights store

* Remove outdated

* Fix missing return

* Allow batch updating in REST API

Allows sending an array of partial items to the endpoints, updating all to their own values

* Add batch update to graphql

* Start integrating edits

* Readd clear

* Add deletion

* Add duplication

* Finish create flow

* Resolve cache refresh on panel config

* Prevent warnings about component name

* Improve loading state

* Finalize dashboard overhaul

* Add auto-refresh sidebar detail

* Add efficient panel reloading

* Set/remove errors on succeeded requests

* Move options rendering to shared

* Fix wrong imports, render options in app

* Selectively reload panels with changed variables

* Ensure newly added panels don't lose data

* Only refresh panel if data query changed

* Never use empty filter object in metric query

* Add default value support to variable panel

* Centralize no-data state

* Only reload data on var change when query is altered

* Fix build

* Fix time series order

* Remove unused utils

* Remove no-longer-used logic

* Mark batch update result as non-nullable in GraphQL schema

* Interim flows fix

* Skip parsing undefined keys

* Refresh insights dashboard when discarding changes

* Don't submit primary key when updating batch

* Handle null prop field better

* Tweak panel padding

Co-authored-by: jaycammarano <jay.cammarano@gmail.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
2022-06-27 15:26:42 -04:00
rijkvanzanten
2ecaf0360c Don't check bytesize for empty payloads 2022-06-13 13:04:21 -04:00
Rijk van Zanten
8cee483a1d Add optional cache max value size limit configuration (#13871)
Resolves #13708
2022-06-13 13:03:41 -04:00
Rijk van Zanten
a36c7eabeb Use json parser wrapper function to prevent pollution attacks (#13191) 2022-05-09 14:57:38 -04:00
keesvanbemmel
d11c59462f Adds x-directus-cache response header with HIT value (#12732)
* Adds x-directus-cache response header with HIT value

* Added ENV var for cache header and MISS, UNCACHEABLE

* Rename CACHE_HEADER_KEY -> CACHE_STATUS_HEADER

Tad opinionated, but I feel like it describes the functionality slightly better

* Return miss consistently

Uncacheable doesn't quite describe what's happening. The user actively requests the cache to be skipped, which should result in an expected miss. The value isn't un-cacheable, the cache is skipped on purpose

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-04-14 13:42:10 -04:00
Rijk van Zanten
1c3e94d830 Add new export experience (#12201)
* Use script setup

* Start on export dialog

* Use new system field interface, replace limit with numeric input

* Set placeholder

* Add sort config

* Use folder picker, correct layoutQuery use

* Add local download button

* Allow writing exports to file

* Add notification after export

* Fix sort config, use new export endpoint

* Setup notification hints

* Add information notice

* Fix local limit, cancel button

* Add (basic) docs for export functionality

* Fix json export file format

* Implement xml batch stitching

* Resolve review points
2022-03-17 15:43:45 -04:00
Rijk van Zanten
eea9f45624 Add authenticate hook to implement custom auth checks against current request (#11942)
* Add "authenticate" filter hook that allows custom auth check

* Start on test

* Update Jest, restructure API tests, start implementing authenticate test

* Move access token verify to util function

* Ensure jest can show inline warnings on correct lines

* Update is-directus-jwt to use jsonwebtoken decode + add tests

* Remove unused package

* Tweak and finish + test authenticate

* Tweak test

* Add authenticate filter to docs

* Don't scan tests for codeql

* No seriously, ignore tests
2022-03-03 16:29:13 -05:00
Abdón Rodríguez Davila
e73cc4f773 Avoid get database when authenticate with JWT (#11617) 2022-02-28 18:43:23 +00:00
Jay Cammarano
d8c9257058 Bearer token no longer case sensitive for API authenticatiom (#11307)
* seemingly fixes issues/11300

* Add unit tests for extract-token

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-01-27 14:28:26 -05:00