29 Commits

Author SHA1 Message Date
Rijk van Zanten
63069bd688 Consolidate Redis environment variables (#19010)
* Remove individual redis configurations

* Use shared redis config

* Update stub

* Add changeset

* Update .changeset/grumpy-rice-film.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Remove RATE_LIMITER_REDIS_ ref in docs

* Update REDIS config in docker compose example

* Readd missing param

* Fix mention of namespace ind ocs

* Undo moving of cache store env

* Update blackbox tests & docs

* Update api/src/env.ts

Co-authored-by: ian <licitdev@gmail.com>

* Fix casing of wEbSoCkEtS

* Remove redis namespace from env check

* Add global store back in

* Tweak phrasing

* Readd supplementary note 5

* Bring back CACHE_STORE in docs / tests

* Forgot one CACHE_STORE

* Consistent order

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-06-27 13:34:49 -04:00
ian
757a5914af Flush cache when applying diff (#18986)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-06-27 15:13:54 +02:00
Rijk van Zanten
513a739b33 Drop support for Memcached (#18980) 2023-06-23 08:54:43 -04:00
Pascal Jufer
17afb9a3bc Patch Tuesday 🐸 (#18260)
* Update `@directus/api` deps (minor)

 @aws-sdk/client-ses          3.292.0  →   3.316.0
 @godaddy/terminus             4.11.2  →    4.12.0
 @rollup/plugin-alias           4.0.3  →     4.0.4
 @rollup/plugin-node-resolve   15.0.1  →    15.0.2
 @types/node                  18.15.3  →  18.15.11
 @vitest/coverage-c8           0.29.3  →    0.30.1
 axios                          1.3.4  →     1.3.6
 fs-extra                      11.1.0  →    11.1.1
 helmet                         6.0.1  →     6.1.5
 ioredis                        5.3.1  →     5.3.2
 joi                           17.8.4  →    17.9.1
 liquidjs                      10.6.1  →    10.7.0
 marked                        4.2.12  →     4.3.0
 nanoid                         3.3.4  →     3.3.6
 rollup                        3.19.1  →    3.20.6
 sharp                         0.31.3  →    0.32.0
 vitest                        0.29.3  →    0.30.1
 vm2                           3.9.16  →    3.9.17

* Update `@directus/api` deps (major)

 @rollup/plugin-alias     4.0.4  →     5.0.0
 @types/inquirer          8.2.6  →     9.0.3
 @types/node           18.15.11  →  18.15.12
 camelcase                6.3.0  →     7.0.1
 chalk                    4.1.2  →     5.2.0
 commander                9.5.0  →    10.0.1
 execa                    5.1.1  →     7.1.1
 icc                      2.0.0  →     3.0.0
 inquirer                 8.2.5  →     9.1.5
 ~ldapjs                   2.3.3  →     3.0.2~
 nanoid                   3.3.6  →     4.0.2
 ~openapi3-ts              3.2.0  →     4.1.1~
 ora                      5.4.1  →     6.3.0
 strip-bom-stream         4.0.0  →     5.0.0
 tedious                 15.1.3  →    16.0.0
 typescript               4.9.5  →     5.0.4

* Update `@directus/app` deps

 @babel/core                             7.21.3  →    7.21.4
 @babel/preset-env                       7.20.2  →    7.21.4
 @fortawesome/fontawesome-svg-core        6.3.0  →     6.4.0
 @fortawesome/free-brands-svg-icons       6.3.0  →     6.4.0
 @fullcalendar/core                       6.1.4  →     6.1.5
 @fullcalendar/daygrid                    6.1.4  →     6.1.5
 @fullcalendar/interaction                6.1.4  →     6.1.5
 @fullcalendar/list                       6.1.4  →     6.1.5
 @fullcalendar/timegrid                   6.1.4  →     6.1.5
 @pinia/testing                          0.0.15  →    0.0.16
 @popperjs/core                          2.11.6  →    2.11.7
 @storybook/addon-actions            7.0.0-rc.4  →     7.0.6
 @storybook/addon-backgrounds        7.0.0-rc.4  →     7.0.6
 @storybook/addon-docs               7.0.0-rc.4  →     7.0.6
 @storybook/addon-essentials         7.0.0-rc.4  →     7.0.6
 @storybook/addon-links              7.0.0-rc.4  →     7.0.6
 @storybook/addon-mdx-gfm            7.0.0-rc.4  →     7.0.6
 @storybook/addon-measure            7.0.0-rc.4  →     7.0.6
 @storybook/addon-outline            7.0.0-rc.4  →     7.0.6
 @storybook/client-api               7.0.0-rc.4  →     7.0.6
 @storybook/client-logger            7.0.0-rc.4  →     7.0.6
 @storybook/vue3                     7.0.0-rc.4  →     7.0.6
 @storybook/vue3-vite                7.0.0-rc.4  →     7.0.6
 @types/diff                              5.0.2  →     5.0.3
 @types/dompurify                         3.0.0  →     3.0.2
 @types/lodash                         4.14.191  →  4.14.194
 @vitejs/plugin-vue                       4.0.0  →     4.1.0
 @vue/test-utils                          2.3.1  →     2.3.2
 apexcharts                              3.37.1  →    3.39.0
 axios                                    1.3.4  →     1.3.6
 dompurify                                3.0.1  →     3.0.2
 happy-dom                                8.9.0  →     9.8.4
 marked                                  4.2.12  →     4.3.0
 nanoid                                   4.0.1  →     4.0.2
 pinia                                   2.0.33  →    2.0.34
 sass                                    1.59.3  →    1.62.0
 storybook                           7.0.0-rc.4  →     7.0.6
 typescript                               4.9.5  →     5.0.4
 vite                                     4.1.4  →     4.2.2
 vitest                                  0.29.3  →    0.30.1
 webpack                                 5.76.2  →    5.80.0

* Update root deps

* Update `@directus/composables` deps

* Update `@directus/constant` deps

* Update `create-directus-extension` deps

* Update `@directus/exceptions` deps

* tsconfig workaround: ignoreDeprecations

* Update `@directus/extensions-sdk` deps

* Update `@directus/schema` deps

* Update `@directus/storage` deps

* Update `@directus/storage-driver-azure` deps

* Update `@directus/storage-driver-cloudinary` deps

* Update `@directus/storage-driver-gcs` deps

* Update `@directus/storage-driver-local` deps

* Update `@directus/storage-driver-s3` deps

* Update `@directus/types` deps

* Update `@directus/update-check` deps

* Update `@directus/utils` deps

* tsconfig workaround in schema: ignoreDeprecations

* tsconfig workaround in remaining packages: ignoreDeprecations

* Update `tests-blackbox` deps

* Revert "tsconfig workaround: ignoreDeprecations"

This reverts commit 5d97da55e3.

* Revert tsconfig override

* Update tsconfig

* Fix imports in @directus/utils

* Fix imports in composables

* Fix imports in extensions-sdk

* Fix imports in @directus/api

* Move RateLimiterAbstract to types import as well

* Bump pnpm to 8.3.1

* Update `docs` deps

> [...] the @vueuse/head package will be deprecated. If you're setting up this package on a new project, you should use the @unhead/vue package directly [...]

* Remove obselete dep `concurrently` from extensions-sdk

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* New day, new updates

Also forgot to include minor updates of deps in `tests-blackbox` in previous
commit

* Fix `api` tests

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
2023-04-21 12:59:27 -04:00
Rijk van Zanten
bdb1a919e2 White space rules (#18197)
* White space rules

* Auto-fix whitespace linter warnings
2023-04-14 17:21:56 -04:00
Rijk van Zanten
2983e61870 The Great TypeScript Modernization Program Season 3 Episode 6: The Big One (#18014)
* Step 1

* Step 2

* False sense of confidence

* Couple more before dinner

* Update schema package

* Update format-title

* Upgrade specs file

* Close

* Replace ts-node-dev with tsx, and various others

* Replace lodash with lodash-es

* Add lodash-es types

* Update knex import

* More fun is had

* FSE

* Consolidate repos

* Various tweaks and fixes

* Fix specs

* Remove dependency on knex-schema-inspector

* Fix wrong imports of inspector

* Move shared exceptions to new package

* Move constants to separate module

* Move types to new types package

* Use directus/types

* I believe this is no longer needed

* [WIP] Start moving utils to esm

* ESMify Shared

* Move shared utils to  @directus/utils

* Use @directus/utils instead of @directus/shared/utils

* It runs!

* Use correct schemaoverview type

* Fix imports

* Fix the thing

* Start on new update-checker lib

* Use new update-check package

* Swap out directus/shared in app

* Pushing through the last bits now

* Dangerously make extensions SDK ESM

* Use @directus/types in tests

* Copy util function to test

* Fix linter config

* Add missing import

* Hot takes

* Fix build

* Curse these default exports

* No tests in constants

* Add tests

* Remove tests from types

* Add tests for exceptions

* Fix test

* Fix app tests

* Fix import in test

* Fix various tests

* Fix specs export

* Some more tests

* Remove broken integration tests

These were broken beyond repair.. They were also written before we really knew what we we're doing with tests, so I think it's better to say goodbye and start over with these

* Regenerate lockfile

* Fix imports from merge

* I create my own problems

* Make sharp play nice

* Add vitest config

* Install missing blackbox dep

* Consts shouldn't be in types

tsk tsk tsk tsk

* Fix type/const usage in extensions-sdk

* cursed.default

* Reduce circular deps

* Fix circular dep in items service

* vvv

* Trigger testing for all vendors

* Add workaround for rollup

* Prepend the file protocol for the ESM loader to be compatible with Windows
"WARN: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'"

* Fix postgres

* Schema package updates

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Resolve cjs/mjs extensions

* Clean-up eslint config

* fixed extension concatination

* using string interpolation for consistency

* Revert MySQL optimisation

* Revert testing for all vendors

* Replace tsx with esbuild-kit/esm-loader

Is a bit faster and we can rely on the built-in `watch` and `inspect`
functionalities of Node.js

Note: The possibility to watch other files (.env in our case) might be
added in the future, see https://github.com/nodejs/node/issues/45467

* Use exact version for esbuild-kit/esm-loader

* Fix import

---------

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-04-04 17:41:56 -04:00
Rijk van Zanten
80f4807a09 TS Config Modernization Program Part 3 of many (#17904)
* noImplicitOverride: true

* noImplicitReturns: true

* noPropertyAccessFromIndexSignature: true
2023-03-23 16:47:55 -04:00
ian
cce33436d1 Rework schema caching to use memory (#17763)
Co-authored-by: Nitwel <mail@nitwel.de>
Co-authored-by: Jan Arends <jaads@users.noreply.github.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-03-23 12:52:06 +01:00
Pascal Jufer
e3c755dcf0 Add getMilliseconds util for safer value interpretation (#17498)
* Add `getMilliseconds` util for safer value interpretation

* Test more data types

* Remove remnant

* Customizable fallback with default of undefined

* Clean-up

* Transform getMilliseconds to named export

---------

Co-authored-by: ian <licitdev@gmail.com>
2023-02-17 11:06:26 -05:00
José Varela
cc343fdf91 Improve cache performance by compressing records (#14833)
* Utils to compress/decompress data
Gzip was chosen because we want smaller data but quick algorithm since this will be ran for every request

* Compress system cache

* Decompress system cache

* Set/Get compressed cache for individual requests

* Switch from gzip to snappy, use json compression too

* Fix cache exp set/get

* Remove unused import

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-08-04 17:35:27 -04:00
Rijk van Zanten
3307bed5fd Improve cache reliability in DDL operations (#12400)
* Add TTL to schema cache

* Clear caches on unexpected errors in DDL

* Consistent return value use

* Don't set a default value for schema ttl
2022-04-01 13:24:20 -04:00
ian
5068ca096b Add lock for system cache (#12017)
* Add lock for system cache

* Add lock when forcing a flush

* Simplify code

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-03-18 15:54:02 -04:00
Rijk van Zanten
d84134c8a6 Don't force commandTimeout in redis based caches (#10773) 2021-12-30 15:32:05 -05:00
Rijk van Zanten
65291b95c4 Allow dynamic user variables to be used with filter rules (cont.) (#9376)
* Move permissions extraction to accountability

* Fix permissions retrieval for public user

* Fetch user / role context in permissions middleware

* Remove unnecessary parseFilter

* Rename schemaCache to systemCache

* Add permissions caching

* Add system cache invalidation on permission changes

* Improve caching perf by reducing scope

* Add note to docs

* Clarify compatibility with conditional fields/filters

* Fix lint warning

* Allow nested vars in system-filter-input

* Add custom getter function that resolves arrays

* Add is-dynamic-variable util

* Export new util

* Cleanup parse filter

* Fix build

* Move debounce up to use-items

* Remove unused prop

* 🧹

* Fix input pattern usage w/ vars

* Remove debounce from search-input, increase throttle
2021-11-03 16:56:16 -04:00
Rijk van Zanten
fb53ae6980 Don't flush schema cache on content update (#8056) 2021-09-15 15:46:12 -04:00
Azri Kahar
e29d803abe Align memcached configurations for multiple hosts (#7986)
* align memcached configurations for multiple hosts

* update reference
2021-09-13 12:48:26 -04:00
rijkvanzanten
3ec8d6443d Re-enable the default retrystrategy 2021-08-17 17:45:13 -04:00
Rijk van Zanten
917d575319 Flush caches on server (re)start (#7294) 2021-08-09 23:22:36 +02:00
Rijk van Zanten
faa71c7595 Add cache connection fallbacks (#7226) 2021-08-05 22:27:10 +02:00
rijkvanzanten
b9a857ccd7 Fix missing type signature 2021-06-22 21:08:28 -04:00
Rijk van Zanten
44082c60e1 Add schema caching (#6456)
* Rework cache handler to be function export

* Add default schema caching

* Add schema cache

* Auto purge schema cache on schema change from api

* Only set last_access value on login

* Add note on schema cache setting
2021-06-22 20:50:20 -04:00
rijkvanzanten
801e868554 Fix remaining eslint errors
h/t @paescuj
2021-04-29 15:55:12 -04:00
Nicola Krumschmidt
47b06dc99f Fix a typo in cache.ts 2020-12-01 18:48:26 +01:00
rijkvanzanten
5ddea5f74c Fix Redis cache 2020-11-10 16:43:40 -05:00
rijkvanzanten
9c9468798e Fix error logging in redis cache
Fixes #896
2020-11-05 14:18:42 -05:00
rijkvanzanten
85ca7b5d17 Run prettier 2020-09-22 16:11:28 -04:00
rijkvanzanten
e148583ed8 Support redis/memcached in cache 2020-09-08 19:08:46 -04:00
rijkvanzanten
a8552abc23 Treat sanitizedquery as frozen 2020-09-08 18:54:18 -04:00
rijkvanzanten
b19ffd960b Add response middleware and cache 2020-09-08 18:28:23 -04:00