Commit Graph

35 Commits

Author SHA1 Message Date
Roger Stringer
c429ecd57c healthcheck crashes with local file storage (#16944)
* Update api/src/services/server.ts
* remove try catch in write
* Update api/src/services/server.ts

Co-authored-by: Freekrai <freekrai@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-01-06 07:17:04 -08:00
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00
Rijk van Zanten
86dd593f91 Use node16 module-resolution (#16696)
* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use CJS require in hook import

* Fix type export in schema

* Clean up defaults usage

* Use require instead of import for migrations

* Use a vitest compatible dynamic import

* Uno mas

* Cleanup type export
2022-12-02 15:53:15 -05:00
nazarevrn
9c55ed6728 Make threshold checks configurable (#15284)
* Make threshold checks configurable

* Fix linter

Co-authored-by: Roman Nazarev <roman.nazarev@skyeng.ru>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2022-08-30 14:41:26 -04:00
Rijk van Zanten
a21b28b423 Add "Run Script" operation (#15101)
* [WIP] Start working on script operation

* WIP Remove npx-import experiment

* Don't break on string value in options scope

* WIP Add module whitelist

* Mission critical import cleanup work

* Add translated labels

* Use nodeVM for everything

Keeps the export function callback structure consistent

* Tweak wording

* WIP start on tests

* Improve typing in shared define types

* Add missing base reactive properties in server store

* Add tests for exec index in app

* Add tests

* Optimize error handling
2022-08-18 15:39:25 -04:00
Azri Kahar
59b62d4655 Fix macos-release error in server info (#14840) 2022-08-05 16:53:33 -04:00
Jay Cammarano
820457690f Optimize media loading across app (#10592)
* v-image and "lazy load" working

* fixed vars

* all the other img uses

* No longer require access token in url for files

* Add lazy loading and size limits

* Rename map-component source prop

* Fix lint warning

* Update app/src/views/public/public-view.vue

Co-authored-by: ian <licitdev@gmail.com>

* Fix lint

* Fix missing file type icon

* Fix null imageInfo error

* Use video.js for media playback

* Fix .js file display

* Update package-lock.json

* Update package-lock.json

* update package.json

* Update pnpm-lock.yaml

* Remove unrelated addition on VDatePicker

* Remove folder abstraction

* Use image data based aspect ratio on preview

* Base app rate throttle on API rate limit config

* Configure app rate limit throttle based on api config

* Convert v-image to script[setup]

* Convert v-media to script[setup]

* Cleanup v-media

* Remove unneeded addTokenToUrl usages

* Remove video.js

It doesn't do authorization headers for mp4/mp3, so it's pointless

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
2022-07-20 15:52:38 -04:00
Christian Rendl
0206a3d2cc Added default locale before login (#8196)
* Added default locale before login

* Use project_language as default language for new users

* Revert "Use project_language as default language for new users"

* Allow user's language field to be nullable; Use project_language whenever language is unset;

* Apply suggestions from code review

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>

* Make "Project Default" option in system-language interface configurable

* Set language only if user language is not null

* Introduced translation for project_language field; Use translation in system-language interface

* Set app language only from app.vue;
Included watcher to restore project_language if user's language is (re)set to null

* Removed .alter() in knex migration since it's not needed for changing default and breaks checks

* Reverted previous change in migration; .alter() is needed for changing column defaults

* Reverted migration to use .defaultTo(null) to set default value

* Update position of project language field in settings

* Fix cockroachdb test error in knex migration

* Fix cockroachdb test error in knex migration

* Apply migration fix also on oracle databases

* Apply changes from code review

* Rename migration to fix migration key error

* Move setLanguage calls to server.ts and hydrate.ts

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
Co-authored-by: jaycammarano <jay.cammarano@gmail.com>
Co-authored-by: Christian Rendl <cr@mutor.at>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-04-01 14:30:56 -04:00
John Huffsmith
e7ada1f173 Moving schema and relation types into shared package (#11179)
* moving schema and relation types into shared package

* updating SchemaOverview imports

* removing duplicate import
2022-01-20 13:36:19 -05:00
Ben Haynes
cbe139d7e9 style tweaks (#10760)
* style tweaks

* fix linting

* Make project descriptor configurable

* Add missing import

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-12-30 19:40:38 -05:00
Nicola Krumschmidt
d64ca14348 Explicitly set catch parameters to any type (#7654)
This fixes not being able to build the repo due to type issues
introduced by the Typescript 4.4 option "useUnknownInCatchVariables",
which is enabled by default in strict mode.
2021-08-27 10:33:30 -04:00
Rijk van Zanten
d0f6cb35e3 Support dynamic variables in conditional fields (#7202)
* Move accountability type to shared

* Rely on shared for parse-filter

* Install date-fns

* Support dynamic variables in conditional fields

Closes #7157
2021-08-05 01:51:15 +02:00
Rijk van Zanten
44082c60e1 Add schema caching (#6456)
* Rework cache handler to be function export

* Add default schema caching

* Add schema cache

* Auto purge schema cache on schema change from api

* Only set last_access value on login

* Add note on schema cache setting
2021-06-22 20:50:20 -04:00
Rijk van Zanten
76a31ca020 Don't initialize mailer on startup (#6169)
* Fix m2m relationship not showing current col/pk

* Don't initialize mailer on first boot

Another step to serverless!
2021-06-09 17:31:50 -04:00
Rijk van Zanten
77e00b7db4 Don't initialize database on file require (#6003) 2021-06-02 11:41:42 -04:00
rijkvanzanten
1bbc8535a5 Move nodemailer setup to src root 2021-05-24 13:36:12 -04:00
Aiden Foxx
7df8cf2912 Added Oracle healthcheck fix (#5461) 2021-05-04 13:22:35 -04:00
rijkvanzanten
801e868554 Fix remaining eslint errors
h/t @paescuj
2021-04-29 15:55:12 -04:00
Pascal Jufer
acd41eb0be Syntax fixes (#5367)
* Declare return types on functions

And a very few other type related minor fixes

* Minor syntax fixes

* Remove unnecessary escape chars in regexes
* Remove unnecessary awaits
* Replace deprecated req.connection with req.socket
* Replace deprecated upload with uploadOne
* Remove unnecessary eslint-disable-next-line comments
* Comment empty functions / catch or finally clauses
* Fix irregular whitespaces
* Add missing returns (null)
* Remove unreachable code
* A few logical fixes
* Remove / Handle non-null assertions which are certainly unnecessary (e.g. in
tests)
2021-04-29 12:11:43 -04:00
Rijk van Zanten
d25c35fee7 Add MailService (#5265)
* Create MailService

Fixes #5229, ref #3372, #4664, #4858, #5090, #3104, #3465, #2774, #3741

* Fix path to templates extensions

* Add mailservice example to hooks docs
2021-04-26 17:55:34 -04:00
Nicola Krumschmidt
08293b8570 Update Knex to version 0.95.0 (#4374)
* Update Knex to version 0.95.0

* Fix formatting in schema dialects

* Fix Knex imports

* Update package-lock

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-03-03 19:37:44 -05:00
Daniel Stoyanoff
54bab04b3c #3982 - Write logs as part of health check in case the status is ERROR or WARN to help investigating issues easier (#3993)
Co-authored-by: Daniel <daniel.s@appgr8.com>
2021-02-10 09:37:21 -05:00
rijkvanzanten
b6524b20eb Tweak perf values 2021-01-20 21:26:46 -05:00
rijkvanzanten
0cbf5ee418 Tweak thresholds of /health 2020-12-21 17:16:07 -05:00
rijkvanzanten
809f9bc09b Only return full data for admin users 2020-12-21 15:23:26 -05:00
rijkvanzanten
9974cb6050 Add health endpoint 2020-12-21 15:22:39 -05:00
Igor Savin
7cddb458cb Run prettier across app and api 2020-12-08 02:22:00 +02:00
rijkvanzanten
8cbdfb409b Use prefetched schema info 2020-11-09 17:13:48 -05:00
rijkvanzanten
9c07afbc86 Use settings singleton in server info endpoint 2020-10-29 16:14:50 -04:00
rijkvanzanten
9559497785 Fetch public info from /server/info endpoint
Fixes #784
2020-10-26 19:08:07 +01:00
rijkvanzanten
b4f82efcee Return project branding in /server/info endpoint at all times 2020-10-26 18:52:51 +01:00
rijkvanzanten
48fca55651 Restructure exports for services 2020-09-22 13:16:14 -04:00
rijkvanzanten
7f1f105349 Merge branch 'main' into feature-rate-limiting 2020-09-08 12:31:57 -04:00
rijkvanzanten
17c7308f35 Show new server info in app 2020-09-01 13:37:52 -04:00
rijkvanzanten
d7d3de86d2 Add server service, update returned info 2020-09-01 13:08:52 -04:00