44 Commits

Author SHA1 Message Date
Rijk van Zanten
00865fbd84 File Storage 2.0 (#16825)
* Refactor storage logic to testable setup

* Add tests to get storage config

* Add tests for register drivers

* Tests for index

* Organize imports

* Add missing getStorage

* Setup boilerplate for cloudinary drive

* Add build script

* Add cloudinary configuration

* WIP tweaks for R&D

* Start storage abstraction v2

* Make storage manager single file

* Add test coverage

* Organize imports

* Setup local driver boilerplate

* [WIP] Start on local driver 2.0

* Add more methods

* Lunchtime

* Add put method

* Add list method

* [WIP] Try using storage in api

* Use node16 module-resolution

* Lets not mess with apply-query rn

* I love CJS. Death to CJS.

* Use dynamic imports

* Make things work

* Add path normalizer

* Add azure storage driver

* Update lock

* Start on tests

* Add getBuffer test

* Add getStat tests

* Add tests for exists

* Add tests for move

* Add tests for copy

* Add tests for put

* Add tests for delete

* Add test coverage for list

* Add removeLeading option to normalizePath

* Use removeLeading option

* Start on gcs

* Add fullpath test

* Add getStream

* Add getBuffer

* Add getStat

* Add exists

* Add move

* Add copy

* Add put

* Add delete

* Finish DriverGCS

* Cleanup tests a bit

* Start s3

* Add getStream

* Add getBuffer

* Please Wryn

* Add #exists

* Use randomize data

* No more hardcoded values 🙅‍♀️

* Add tests for copy

* Add tests for put

* Add put/copy/delete

* Add tests for delete

* WIP list

* Remove duplicate fullPath check

* Finish AWS tests

* Listen to wryn

* Mission critical tweak

* Add randomization, cleanup tests

* Check src vs dest full path

* Start on Cloudinary

* Add parameter signature helper

* Fix ESM building of shared

* Fix ESM building of shared

* Improve tests

* Update pnpm

* Remove old build commands

* Generated d.ts files

* Fix export naming

* Move ESM only utils to new @directus/utils

* Update lockfile

* Fix type exports

* Implement getStream

* Cleanup tests

* Simplify api

* Simplify API

* Simplify api

* Simplify API

* Add read/stat

* Cleanup / add exists

* Add move

* Add write

* Move uploadChunk to separate method

* Add test for #uploadChunk

* Add tests for write

* Add copy

* Add delete

* Add list

* Add list error handling

* Remove old drive packages

* Start updating API usage of storage

* Use Readable instead of NodeJS.ReadableStream

* Use readable instead of buffer

* Restore shared to main

* Update lockfile

* Use a streaming pipeline for sharp metadata

* Add basic e2e test for local upload and delete

* Fix integer filesize in SQLite

* fixed environment mocking in unit tests

* applied the same env mocking to other tests

* Update api/src/storage/register-drivers.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Use sha256 by default

* Add base read test for /assets

* Replace exifr with exif-reader

* Fix tests for storage/index

* Install faking lib for tests

* Add test for register-drivers

* Add tests for register-locations

* Finish tests

* Organize imports

* Resolve Azris comments

* Fix outdated tests

Certainly not the cleanest, but I need to do a bigger pass on all these tests to get them up to date with the latest team requirements. Gonna do that in a separate PR..

* Test for sha256

* Attempt 1 at fixing toString errorr

I'm not seeing this issue locally, so we'll spam a couple commits here to get things going

* Use node 18 in tests?!

* Fix localhost resolution with 127.0.0.1

* Mock getEnv()

* Use @directus/tsconfig instead of duplicated conf

* Does this fix it?

* OK fun detour

* Recreate lockfile

* Update config files

* Use multipart uploads in S3

* Cleanup imports

* File Storage 2.0: Make metadata extraction backward-compatible (#16868)

* Reinstall packages using pnpm instead of manually removing them (#16871)

* Added extra environment setting for sharp processing of invalid images (#16811)

* Added extra environment setting for sharp processing of invalid images

* renamed environment var to `ASSETS_INVALID_IMAGE_SENSITIVITY_LEVEL`

* Remove unused excludes from tsconfig

* Remove copy/paste leftover

* Update packages/utils/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update packages/utils/package.json

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2022-12-21 10:04:03 -05:00
rijkvanzanten
701e540680 Ignore debug folder 2022-08-27 18:31:36 -04:00
Wryn (yage) Wagner
0930c2df97 Safe Metadata Upload (#12511)
* [Related to #11292] Safe Metadata Upload
- Created FILE_METADATA_ALLOWLIST
- Added test library for `FilesService`

* Rename ALLOWLIST->ALLOW_LIST to be consistent with other env vars

* Make sure it still works for files without Exif

* Prevent breaking changes in metadata formatting

* Allow * for all

* Fix md sup

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2022-04-04 20:43:56 -04:00
Paul Boudewijn
58964bd1bc Fix localstorage file deletion (#9126)
* excluded VS specific files

* new files after updating directus sources

* case insensitive string matching fix

* Remove accidental commit files

* Cleanup gitignore

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-10-25 20:43:57 -04:00
Rijk van Zanten
8f00e37daf Add improved collection organization setup (#8623)
* Add migrations, start on service

* Dont track TODO files

* Update collection types, add collection type

* Allow drag and drop sorting of collections

* Add tooltip

* Add grouping + collapsed state

* Fix nested closed state

* Tweak active drag styling

* Remove collapsed state

* Add folder creation/editing

* Render collections as nested tree in nav

* Fix open active state

* Add dense when collection count > 5

* Add visible toggle

* Add show-hidden toggle

* Fix css specificity

* Add support for query in v-list-group

* Add missing cascade

* Remove collapsed state

* Finish three-way toggle

* Add custom lock icon

* Fix icon size in non-dense

* Redirect to first & open tree on load

* Dont make prop required

* Fix search

* Only apply archive filter when enabled in settings

* Add archive view

* Add translations

* Hide organization fields

* Show system collections
2021-10-15 21:19:00 +00:00
Nicola Krumschmidt
2908063d86 Transpile docs to vue components at build time (#8743) 2021-10-12 14:22:14 -04:00
Rijk van Zanten
f5f68417cf Use field key instead of ID for group value (#8349)
* Add migration to replace group ID with group name

* Replace group IDs with field names

* Tweak group interfaces

* Ignore schema export
2021-09-27 15:06:19 -04:00
Sam Milledge
7186c4e381 Allow custom CLI commands to be added by extensions (#7675)
* Fix typescript errors in shared package

* Hooks for adding custom CLI commands

* Add CLI hooks to documentation
2021-09-07 16:18:16 -04:00
Jakob
9df7b91564 Improve e2e test watch mode (#5053)
* Improve e2e test watch mode

- run setup only once
- disable teardown

* Add container cleanup for watch mode

* Don't rely on inline exec code

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-04-14 17:54:26 -04:00
rijkvanzanten
52ee17dc33 Fix dev scripts on root 2021-04-09 16:34:52 -04:00
Juan Carlos Blanco Delgado
33f6d1870f Updating docs of the JS SDK (#3816) 2021-02-18 18:13:43 -05:00
Philleep Florence
c934dd17a6 1129.01 - Resolve links and images within the Docs Module. (#3760)
* 1129.01 - Resolve links and images within the Docs Module. Validate hint container blocks.

* Remove copy-to-app

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
2021-02-01 17:15:58 -05:00
Nicola Krumschmidt
6068ab4bef Add option to specify .env filename in development
When required from the command line, dotenv loads the .env file
specified by the DOTENV_CONFIG_PATH env var.
Files with a filename pattern of the form .env.* are ignored by git to
make it easier to work with multiple different database vendors.
2021-01-14 16:32:32 +01:00
rijkvanzanten
5db50352ab Cleanup translationsto yaml file 2020-12-08 11:59:29 -05:00
Igor Savin
87db93f88d Replace TSlint with ESLint TS plugin.
Unify Prettier configuration
Add linting Lerna job
2020-12-08 01:22:52 +02:00
rijkvanzanten
456d46f474 Get tests to 100% 2020-11-15 22:27:07 -05:00
rijkvanzanten
c578325252 Ignore sqlite db files 2020-11-13 17:31:13 -05:00
rijkvanzanten
6728f40c1d Ignore sublime settings 2020-11-02 09:58:14 -05:00
rijkvanzanten
0a0dc0198f I give up. You can have your d*mn lockfile 2020-10-08 15:18:01 -04:00
rijkvanzanten
272a2090ac Add setup command 2020-10-08 10:18:16 -04:00
rijkvanzanten
adb9254e90 Add build step to docs 2020-10-07 10:56:05 -04:00
rijkvanzanten
c36fb905c1 Setup lock structure 2020-10-02 14:17:21 -04:00
rijkvanzanten
4126445fcf Hoist all dependencies 2020-10-02 12:40:22 -04:00
rijkvanzanten
6de4567ead Ignore VS Code workspaces 2020-09-28 12:01:29 -04:00
WoLfulus
5617c29d63 small fixes 2020-09-25 11:36:25 -03:00
rijkvanzanten
faa4d594ee Ignore nova files 2020-09-16 18:34:23 +00:00
rijkvanzanten
542884e2b6 Bump versions 2020-09-03 18:53:08 -04:00
rijkvanzanten
33787c0e4c Ignore package-lock 2020-09-01 13:37:45 -04:00
rijkvanzanten
abe100adbe Ignore package lock 2020-09-01 13:32:06 -04:00
rijkvanzanten
5fa61a58cc Ignore debug logs 2020-08-03 18:27:38 -04:00
rijkvanzanten
a99a7f47f4 Use port 41201 instead of 3000 by default 2020-08-03 17:44:09 -04:00
rijkvanzanten
124368d235 Add gitignore file 2020-07-29 13:35:49 -04:00
rijkvanzanten
d1aa5d350e Move app to app folder 2020-07-29 11:24:52 -04:00
rijkvanzanten
dd7e42ebba Add app 2020-07-29 11:23:53 -04:00
rijkvanzanten
cae3baa073 Add gitignore 2020-07-29 11:23:09 -04:00
rijkvanzanten
bde7069cce Move api into api subdirectory 2020-07-29 11:22:56 -04:00
rijkvanzanten
e6dcf0d83e Add Makefile to bundle in app 2020-07-24 16:45:42 -04:00
rijkvanzanten
4bd95956b3 Rework order of seeder 2020-07-17 18:25:09 -04:00
rijkvanzanten
5620237a1c Move from yarn to npm 2020-07-03 09:39:06 -04:00
rijkvanzanten
67202260f7 Start on collections endpoint 2020-06-30 17:06:01 -04:00
rijkvanzanten
ab1296c89f Rework assets to be a service 2020-06-30 12:21:10 -04:00
Rijk van Zanten
1485a2a959 Initial commit 2020-06-16 10:29:10 -04:00
Nitwel
9fcf702b4f Add progress circular (#32)
* made spinner more fancy

* rebuild spinner to progress circular

* update readme

* cleaned code and style

* clean readme

* made spinner more fancy

* rebuild spinner to progress circular

* update readme

* cleaned code and style

* clean readme

* Register circular progress

* Fix broken import

* Fix stylelint problems in v-progress-circular

* Add some useful tests for circular progress

* Delete package-lock.json

* Update readme

* Ignore package-lock

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2020-02-17 14:28:16 -05:00
rijkvanzanten
8573e61015 Initial commit 2020-02-05 14:02:32 -05:00