14 Commits

Author SHA1 Message Date
Rijk van Zanten
3491af9de4 Make tests deterministic, remove @directus/random (#25575)
* Make tests deterministic by dropping random

* Remove random dep

* Drop random package

* Update pnpm lock

* Update app/src/composables/use-permissions/collection/lib/is-action-allowed.test.ts

---------

Co-authored-by: Alex Gaillard <alex@directus.io>
2025-07-31 17:16:40 -04:00
Matthew Rumery
17bf9be79e Add Services Type support for @directus/extensions (#25368)
* feat: add services type

* refactor: replace UserIntegrityCheckFlag from @directus/types

* refactor: replace AbstractServiceOptions from @directus/types

* refactor: replace MutationOptions from @directus/types

* refactor: replace ActionEventParams from @directus/types

* refactor: replace AbstractService from @directus/types

* refactor: replace Webhook from @directus/types

* refactor: replace Range, Stat, ReadOptions, & ChunkedUploadContent from @directus/types

* refactor: replace asset types from @directus/types

* refactor: replace LoginResult from @directus/types

* refactor: replace snapshot types from @directus/types

* refactor: replace Action & PayloadServiceProcessRelationResult from @directus/types

* refactor: replace ExportFormat from @directus/types

* refactor: replace database types from @directus/types

* feat: implement ExtensionsServices within @directus/extensions

* feat: replace QueryOptions from @directus/types

* fix: formatting issue

* fix: adjust extensions services type

* fix: generic type for each service

* fix: typescript errors for missing imports

* refactor: update ReadOptions & ChunkedUploadContext type exports to pull from @directus/types

* refactor: move DirectusError to @directus/types & add DirectusExtensionsError to @directus/types

* feat: add MailService, GraphQLService, SpecificationService, and move required exports to @directus/types

* refactor: move extension types to @directus/types

* refactor: move extension constants to @directus/constants

* refactor: adjust type imports for @directus/composables

* refactor: adjust imports for @directus/extensions-registry

* refactor: adjust import for @directus/errors

* refactor: adjust imports for @directus/themes

* refactor: adjust imports for @directus/extensions-sdk

* refactor: adjust imports for studio app

* feat: implement ExtensionService and migrate types to @directus/types

* fix: revert sdk type changes

* fix: reexport types moved from @directus/extensions to @directus/types

* fix: remove @directus/themes from @directus/extensions

* refactor: move AppField to fields file

* chore: remove @directus/themes from @directus/types

* chore: remove @directus/extensions from @directus/types

* fix: rebase issue with DirectusError generic

* chore: remove @directus/types from sdk

* fix: add missing exports to original packages

* fix: update getAppExtensionChunk return type

* chore: revert DirectusError changes

* chore: revert Collection fields key

* refactor: rename RawSchemaCollection to ApiCollection

* fix: add RawCollection from @directus/types

* fix: import GraphQLParams from @directus/types

* fix: formatting issue

* chore: add changeset

* chore: revert rename of ApiOutput

* Update .changeset/purple-crews-sink.md

* Update .changeset/purple-crews-sink.md

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>

* Update .changeset/purple-crews-sink.md

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>

* Update .changeset/purple-crews-sink.md

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>

* Update .changeset/purple-crews-sink.md

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>

* Update .changeset/purple-crews-sink.md

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>

* Update .changeset/purple-crews-sink.md

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>

* Update .changeset/purple-crews-sink.md

Co-authored-by: ian <licitdev@gmail.com>

* fix: rebase pnpm-lock error

---------

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Alex Gaillard <alex@directus.io>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2025-07-31 12:54:50 -04:00
Nitwel
3961dc0593 Add key/value information to db errors (#25112)
* Add key/value infomation to db errors

* make key/value error info consistent across DBs

* fmt and fix tests

* fix errors and clean up code

* fix tests

* fmt and fix tests

* fix linter...

* fix tests

* jeezz that formatter and linter.....

* fix fields in pg error

* rename key to value

* Update .changeset/sour-lands-sniff.md

---------

Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com>
2025-06-19 18:43:07 -04:00
Rijk van Zanten
9faac0164e Merge branch 'main' into v11-rc 2024-08-05 16:21:55 -04:00
Pascal Jufer
ce6463a4cd Clean-up error handler & fix message for unknown API errors (#22379)
Co-authored-by: Hannes Küttner <4376726+hanneskuettner@users.noreply.github.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2024-07-15 12:15:52 +02:00
Rijk van Zanten
2273480c6d Implement the new permissions policy (#22773)
* WIP start on migrations

* Add migration

* Don't insert if there's no rows

* Use service to read/write permissions

* Use payload service rather than itemsservice

* Start on downgrade command

* Update system data structure

* Update migrations to keep structure flat

* Remove icon from policies

* Drop policies table on downgrade

* Rearchitect migrations to structure v3

* Add down migration

* Update system fields

* Add policy to fields import

* Fix public role attachment

* Update packages/system-data/src/fields/index.ts

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* Update packages/system-data/src/fields/policies.yaml

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* Add nested roles

* Remove unused step

* Use two o2ms instead of m2a for attachments

* Update system data

* Implement permission policies in the API (#22384)

* Update system data structure

* Update migrations to keep structure flat

* Remove icon from policies

* Drop policies table on downgrade

* Rearchitect migrations to structure v3

* Add down migration

* Update system fields

* Add policy to fields import

* Fix public role attachment

* Update packages/system-data/src/fields/index.ts

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* Update packages/system-data/src/fields/policies.yaml

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* Add nested roles

* Remove unused step

* Use two o2ms instead of m2a for attachments

* Update system data

* [WIP] Start reorging permissions handling

* Setup field extraction

* Remove watch from vitest

* Finish fieldMap creation logic

* Add tests for utils

* Improve tests

* Improve coverage

* Split test and test:watch

* Continue on this fun

* [WIP] Setup processing

* Sort roles

* Restructure to util files for org

* Add missing util tests

* More tests

* Add cases/whencase to ast

* Start on injection logic

* Add tests for inject cases

* Add tests for process

* Add todo

* Organize run-ast

* Add clear method to kv

* Remove reliance on acc.perm

* Restructure permissions setup

* Drop perm from acc, add roles/policies

* Remove get-permissions in middleware

* Remove/comment use of acc.perm

* Add default roles/permissions

* Use knex

So we don't have to initialize the schema before we want to use the accountability system

* Use new fetching logic in get accountability

* Add new fetch global access utils

* Gotta redo based on new setup

* Replaced with new util

* Remove dropping of perm in acc

It's no longer there by default, so no need to remove here

* Temporarily comment out the enforce tfa check

* Update usage of fetch tree to use knex

* Don't store policies on accountability

* Feed in roles thru acc

* Bit of whitespace

* Rename role->policy

* Wreck some more stuff

Jk, this is splitting up the large get-ast-from-query function into smaller individual functions to make it easier to update the wildcard conversion to use permissions

* Add ability to lookup all allowed fields in col+ac

* Add note so I don't forget stuff which i will

* Handle null acc

* Introduce parseAst to itemsservice

* That cleans things up

* Replace checkAccess with validateAccess

* Remove checkaccess from service

* cleanup imports

* Whoops one more

* Leave crumbs for next time

* Implement most of the fn

* Fix various tests

* Start on test for fetch roles tree

* Add tests for fetch roles tree

* Fix process tests

* All. of. the. tests.

* Update uses of validateAccess

* Fix name in runAst

* Fix use of accountability in gql sub

* Deprecate authorization service

* Remove getPermissions use

* Drop old getpermissions

* Pass services

* Replace admin/app uses with fetch global

* Update fetch user count to pull from policies

* Remove broken admin existence checks

* Update min accountability

* Remove unused import

* Drop permissions override from controller

* Refactor reliance on acc.perm

* Replace usage of permissions in fields

* Replace usage of permissions in import/export

* Drop permissions use from relations

* Drop no longer used method

* Remove unused import

* fix type usage of pk in validate

* Fix default acc in user

* Replace use of permissions in utils

* Update reduceSchema in specs/gql

* Remove old share merging

* Remove empty file

* Remove outdated comment

* Use ctx objects for large param fns

* Add with-cache memoize util

* Add cache to fetchpermissions

* Update caching use in fetchRolesTree

* Add caching to fetchAllowedFieldMap

* Add more cache

* Refactor call signatures

* Move call signature updates

* Handle presets

* Update process call sig

* Prevent infinite recursion in roles tree lookup

* Use create util for acc

* Remove old checkIp

* Fix where equality operator

* Break EVERYTHING!

Jk just cleaning up the structure some more, and removing the dep injection in favor of mocking

* Fix build

* Add missing module tests

* Don't crash on missing parent

* Fix role lookup

* add missing type annotation

* use logical-OR assignment and avoid a memory allocation

* Attach admin policy in default admin creation

* Fix admin check

* Add todo for later

* rm code duplication

* fix test

it was missing the new `roles`

* add types and fix type error

policies dont [yet] have an icon

* move spread order to avoid potential future mishaps

new default keys would override the manually set keys, potentially leading to unintended behavior

* reduce allocations, add escape hatch to loop and type db-row

* Implement case/when

* Clean up comments

* Optimize perm fetching in allowed f

* Move apply case when to util fn

* Optimize fetch-allowed-fields

* Add fetch inconsistent util

* Allow nulls

* Remove obsolete getCacheKey

* Remove unused import

* Update getAccountabilityForRole test

* Update fetchGlobalAccess test with one more test case + fix other test case

* Type cleanup

* Fix "admin access means automatic app access" in fetchGlobalAccessForQuery

* Clean up and expand fetch-inconsistent-field-map.test.ts

* Test uncached functions

* Test uncached

* Remove cases usage in parse-current-level

* Only consider non-null rules in inject cases

* Fix parseCurrentLevel call

* Move service imports into functions to avoid circular imports

* Ensure that we test that an error is thrown in processAst test

* Add failing test case for flattenFilter

* Ensure uniqueness in extractPathsFromQuery

* Early exit in validatePath

* Add additional test case for process payload test

* Update validateCollectionAccess test

* Clean up validate-item-access.test.ts

* Remove redundant initializer

* Use createDefaultAccountability

* Fix fetch-user-count.test.ts

* Cleanup unused default initializer

* Add empty cases to subfilter in _relationCount

* Drop AccessService and PermissionsService usage from services

* Found some more PermissionsServices

* Fix a few more tests

* Add nested role relation

* Fix query invocation in aggregate and group queries

* Fix role property name in auth/refresh

* Add some missing relations for permissions, access and roles

* Add m2o relation from permissions to policy

* Add m2o relation access to role, user, policy

* Allow fetchPermissions to fetch all permissions and not just those limited by an action

* Add parent to Role type

* Make sure that admin users see all fields

* Add access and policies controller, add util methods to policies and access service

* Change name and description of public policy, update description of admin policy and add on delete trigger.

* Make sure access row uuids are auto generated

* optimize kvredis clear function and add a unit test

to be fair: unit test is also testing implementation details but thats a problem there in general and for future us

* Add minimal app permission and dynamic variable injection to the permission fetching

* Fix m2o collection name in extractFieldsFromChildren

* Make sure to clone permission before injecting dynamic variables

* Actually do the cloning in with withAppMinimalPermissions since people might missbehave with the permissions obtained from PermissionsService.readByQuery so it better to go the source of the problem

* Use knex transaction in createOne -> processPayload - otherwise deadlock

* Make sure to respect '*' field in allowed fields

* Fix extractFieldsFromChildren for o2m as well - classic

* Fix allowed field check in `FieldsService.readAll` to account for multiple permissions for collection+action

* Skip case/when if `allowedFields` includes '*'

* Restructure the way the current users permissions are returned

* add ability to clear all keys from memory cache

* add test for clear method

* add await to clear function

* Clear permissions caches on changes to policy attachments (directus_access) and policy updates (directus_policies) and permissions updates (directus_permissions)

* Make the public role a real role rather than a virtual one

* Inject the public role, we're it previously was `null`

* Revert adding a fix public role

* remove unused variable

* Ensure that a user without a role can still use the /me util endpoints

* Make sure that the /me endpoints always return minimal information, similar to /users/me

* Some fixes after merging main

* Update api/src/permissions/utils/with-cache.ts

Co-authored-by: Hannes Küttner <4376726+hanneskuettner@users.noreply.github.com>

* Avoid broken role query for now

* Skip related collection `parseFields` if user has no permissions

* Ensure same call order as in `convertWildcards`

* Create default admin policy and connect it in cli init command

* Remove obsolete middleware mock in app.test.ts

* Add validation against non-existent fields and collections to `validatePath`

* Split up permission and path existence validation and validate path existence for admin users as well

* Make applySearch not async

* Fix relation extraction and permissions for `$FOLLOW` fields

* Fix case when for related collections and query wrapping

* Rework user integrity checks for Auditus (#22737)

* Changes to user counting and integrity checks

* Ensure that user validation happens in both create one and create many

* Rename `checkType` to `flags`

* Update api/src/permissions/modules/validate-remaining-admin/validate-remaining-admin-count.ts

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* Update to enum usage

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* A few more changes to enum instead of number

* One more enum type update

* Make sure to correctly override the callback when combining options

* Clean up option type

* Update api/src/services/users.ts

Co-authored-by: ian <licitdev@gmail.com>

* Only take validation shortcut for users

We can only be sure that the deletion of users does not increase any other access types count, so in all other cases we need to verify that for example the App or API users have not increased over the limit

* Make both app and admin users count against app access limit

* Update api/src/permissions/modules/validate-remaining-admin/validate-remaining-admin-count.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* One post-merge fix, two small fixes

* Simplify flag updating and callback calling

* Changing app access in a policy only requires user limit checking, not full check

* Only the status of a created user should matter to determine if a check is neccessary

* Add count alias to count query

---------

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Add roles and permissions to the app (#22654)

* Initial app changes

* Fix getRelationsForField

* Add changeset

* Remove app-permissions from role settings

* Make sure access row uuids are auto generated

* Move a few things around, set up policies m2m properly

* Show roles as tree in sidebar
Change avatar field query for user

* Show user and role count in policy table

* Default to not adding app access for a policy, makes composability less annoying

* Correctly fall back to 0 for counts

* Change the structure of current user permissions

* Start bringing back the public role

* Make the public role a real role rather than a virtual one

* Revert public role changes

* Extend list-m2m to allow for very custom junction matching and a primary key of `null`

* Remove unused

* Fix public role policy update payload

* Fix app access for users without role (which is a thing now apparently)

* Make sure that the /me endpoints always return minimal information, similar to /users/me

* Tweak nav icons

* Pull policy id from constants

* Update permissions interface design to match

New design language in figma

* Some minor adjustments

- Make chip hover border more consistent
- Add "Remove" button to remove a full row of permissions, as in the UI mockup
- Fix table layout

* Clean up a few more things

* Fix `setFullAccess`

* Align collection view icons with navigation

* Don't query 'admin_access' for role

* Fix relation extraction and permissions for `$FOLLOW` fields

* Don't show `0 Items` for child rows, but `--` instead

* Make policy detail work in nested policy creating use case

* Remove unused v-icon override

* Move system collections to separate visual table

* Navigate before refresh

Prevents a flash of the previous value to be visible in the table

* Move composable to separate file

---------

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Optimize types

* Clone query deep

* Optimize type order

* Throw error on invalid role id

* Rename run.js -> run-ast.js

* Re-add filesizes to telemetry report collection that got lost in the merge

* Make `systemCollections` reactive

* Use one column per action to avoid unwanted shifting if some actions are not allowed at all

* Render system and custom together

* Add divider between regular and system permissions if both have elements

* Add AccessService and PoliciesService to `getService`

* Move policy global flags fetching to util

* Move collection access fetching into util

* Remove permissions for `directus_access`, `directus_permissions` and `directus_policies` from schema permissions

* use formatted-value display for name & description in roles & policies

* Rename `process.ts` to `process-ast.ts`

* Fix process-ast import after renaming

* Perform user integrity check on item deletion

* Fix first admin creation on bootstrap

* Revert "Fix first admin creation on bootstrap"

This reverts commit bf480d023c.

Will be fixed by adjusting the check in access service

* Don't perform admin integrity check if a new access row is created. Only check user limits

* Don't set an alias to the raw column value if it is wrapped in a case/when

* Correctly handle aliases when in field map and case injection

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>
Co-authored-by: Hannes Küttner <kuettner.hannes@gmail.com>
Co-authored-by: Hannes Küttner <4376726+hanneskuettner@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>

* Remove changeset of already merged PR (#22653)

* Fix multi cache subscribe call to preserve context

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Add max length to name for policy and role names

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Update app/src/modules/settings/routes/policies/collection.vue

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Organize settings sidebar for clarity

* Remove query limit override from access and policies controller

THe query limit was added previously, where the idea was to fetch all policies in the app, but since that is not required anymore we can remove the override again, which in turn fixed pagination for policies. Woop

* Rework MetaService

* Fix filtered counting (previously it did not account for left join caused by permissions filter)
* Collect permissions for current collection and dedupe access to align filter with the one used in the actual query
* Use applyFilter and an _or filter to retrieve permitted items

* Prevent certain skips in _or filters

We can only skip empty filters in `_or` if either they are not equal to the value of `cases` or if _or has exactly only exactly one empty filter

This is needed to prevent dropping joins that are required for the case/when construction. For example when having the permissions `_or: [ {}, {related_item: { id: 1} }]` all joins need to be retained

* Revert unintentional with-cache commit

* Remove check for id in children which fails on some DBs if no children are set

* Show users and roles in policy item view

* Update directus_access policy/roles + policy/users `one_deselect_action`

This ensures that the access rows are cleaned up when removing users or roles from the policy side of the relation

* Merge policy loaded from API with current edits

* Make `app_access` default to false

* Split field map into read specific and other fields

This change is necessary since process-ast is used to verify item access for actions other than `read`.
But, if a user does not have action permissions for fields used in the query filter or sort field the validation for `xByQuery` would have failed until now.

* The fields are verified separately checked against read and action specific permission.
* Updated all the tests accordingly

* Fix `hasCaseWhen` check in `getDBQuery`

Previously it was checking if `cases.length > 0` which was always true, since we always pass in at least one case (`{}`), now it checks if there are actually field nodes with a whenCase property

* Don't expose o2m fields that the user might not have access to for some items

This approach uses a flag that is introduced into the parent item db query, that uses the case/when construct to determine if a user has access to the o2m field on the specific item.
The flag is 1 if the user has access and null otherwise.
It is set in the resulting query object for all o2m fields that have a whenCase (the ones with partial access) and used when merging the nested query items into the parent items.

* Accept O2MNode as fieldNode

* Filter policies in fetchGlobalAccess by ip_access if applicable and use `withCache` util

* Filter the policies influencing the global access by ip_access filter if an ip is available
* Use `withCache` util for top level function and the two lower level functions

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>

* Fix filter in roles. Again. Almost like I didn't properly test it. huh

* Add cache key stability by only picking the props that are relevant from accountability and enforcing an order in the provided options object

* Improve `fetchAllowedFields` to only return fields that are actually in the schema

* Make a local copy of `junctionFilter` in order to prevent reloads on form value changes

* Remove debug log

* Update packages/system-data/src/fields/policies.yaml

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/permissions/utils/filter-policies-by-ip.ts

Co-authored-by: ian <licitdev@gmail.com>

* Conditionally add IP to request level cache key

Add `accountability.ip` to the request level cache key if, and only if, the IP is matched by any of the `ip_access` filters of the current users policies.

This ensures that, if the request IP influences the request result, it is path of the cache key, but also not included if there are no IP filters configured for the current user.

* Update api/src/permissions/modules/fetch-policies-ip-access/fetch-policies-ip-access.ts

Co-authored-by: ian <licitdev@gmail.com>

* Make sure that fetchAllowedFields does not remove field wildcard '*'

* Temporarily disable cache key picking

* Refine cache key picking + remove `undefined` from Accountability['ip'] type

* Rename `pick` to `prepareArg`

* Define the sort field for the `directus_access` junction table

* Verify that user has access to automatically selected sort field and default to first allowed field if not

* Sort the fetched permissions to match the order of the passed in policies

* Some clean-up of TODOs and unused code

* Take care of a special case, where no fields are requested and we are still interested the correct items being returned

This surfaced when running `validateItemAccess` with an update permission that did not include the primary key field.

* 3 less

* Update api/src/database/migrations/20240328A-permissions-policies.ts

Co-authored-by: ian <licitdev@gmail.com>

* Change /permissions/me response and add corresponding types and constants

* Fix payload validation

- Field validation needs to happen for admin users as well
- Add back injection of validation rules for non-nullable fields

Tests added/adjusted accordingly

* Mark `system-permissions` interface as `system`

* Remove special handling for public policy. It's one of us now.

- Add `icon` field to policies
- Add notice to the public role

* Rename migration to most recent date

* Clear permissions cache in `clearSystemCache`

* Make `getDBQuery` not async

* Set the sort field to `null` if the user does not have any allowed fields, not even the primary key

* Handle the case where `null` is returned as item edits by the permission detail drawer and remove the existing item, if any.

* Prevent role recursion (this is a simple check right now and I would expect it to fail on nested role updates that do funky stuff)

* Add overflow to permissions table

* Ensure fields are always passed to validation-errors

When v-form is used with the `collection` prop, instead of directly
passing fields via `fields` prop, the validation-errors component
didn't receive any field information.
This is fixed, by passing down the "finalFields" from `useForm`.

This is not directly related to 'auditus', but since it seems like this will be the
only place (so far) where we want to show validation-errors on a system
collection, I'm committing here.

* Outsource 'useSave' for roles, exactly as in policies

* Clean-up policies & roles item views

- Remove leftover styles, use clearer naming 'content'
- Fix types (policies was using role type)
- Show validation errors, handle errors on save & delete
- Use loading state of v-form (to have some indicator and less layout shift)

* Mark name field in policy & role as required

That way, an indicator is shown in the form, and value is checked when
editing via drawer

* remove role filter from public registration m2o

roles dont have access fields anymore, simply allow admin roles for now

* Remove overflow again. It broke

* Add cache purging for permission related updates.

* Add `dropForeign` in migration

Fixes migration on MySQL

* Add parsed field name to field map instead of raw field name

This fixes filtering & sorting with function as keys, e.g. `year(date_updated)`

* Account for $FOLLOW field filters earlier and don't confuse them with functions (see prev commit)

* Update migration to also work with CockroachDB

Instead of altering the `policy` column on `directus_permissions` to add the NOT NULL constraint it needs to be created with NOT NULL in the first place, as this will fail in CockroachDB.

That means we need to drop the foreign key constraint for the role column in order to update to `null` `role` to the public policy ID before we copy the values into the `policy` column

* Fix typo

* Add icon to default admin policy

* Be more clear about where the public role applies

* Update api/src/permissions/lib/fetch-policies.ts

Co-authored-by: ian <licitdev@gmail.com>

* Update api/src/permissions/lib/fetch-policies.test.ts

Co-authored-by: ian <licitdev@gmail.com>

* Enable GH workflows

* Make eslint happy, cleanup and improve role sidebar in users view

* Fix wrong suggestion application and move comments in block

* Format files

* Update isFullPermission test

* Clean up policy filter logic

* Flip order in test

* Update mocking in user/flows store tests

* Update expected results of a lot of api tests that changed during development (all stay true to the original idea)

* Update parseFilter test

* Update injectCases test

* Manually set parent to `null` if a role gets deleted and remove the `SET NULL` on delete action.

The `SET NULL` action causes problems problems on OracleDB

* Fix limit check for new users w/o "status" field

status defaults to "active", thus if the field is not in payload, the
user limit check needs to be triggered

* Fixed migration "inconsistent datatypes: expected - got CLOB" error in oracle

* Update extractFieldsFromChildren test

* Fix `count(o2m)` type queries

* Update UsersService tests

- Adapt to new user integrity logic
- Add basic ItemsService tests to ensure user integrity checks take
  place

* Move withAppMinimalPermissions to appropriate dir

* Fixed boolean logic error for graphql counting

* Make sure that relational function aliases are recognized as `functionField`

* Fix permission for relational functions

Before this functions that operated on a o2m field like `count(o2m)` did not respect permissions on the related collection.

Now function field nodes have a cases list as well and correctly get the cases injected for the related collection.
The permissions are then correctly injected in the query that is passed down to the relational count function helper.

* Fix mock in withAppMinimalPermissions test

* Update RolesService tests

Copied and commented out old checks from roles to policies, so we can
re-check later on

* Add preliminary changesets

* Reword changeset to "Policies"

* Update .changeset/strong-numbers-warn.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Policies Documentation (#22729)

Co-authored-by: Hannes Küttner <kuettner.hannes@gmail.com>
Co-authored-by: Brainslug <tim@brainslug.nl>

* Reformat docs

* SDK functions for auditus (#22795)

* Updated sdk types

* added policy commands

* prettier

* Added new and missing websocket subscription hooks

* Added missing endpoints

* Added missing graphql endpoints

* prettier

* Added changesets

* updated changesets

* Update .changeset/nine-geckos-jog.md

* Update api/src/services/graphql/index.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/create/policies.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/create/policies.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/read/policies.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/read/roles.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/read/policies.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/read/policies.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/rest/commands/read/policies.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/schema/policy.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update sdk/src/schema/policy.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com>
Co-authored-by: Hannes Küttner <kuettner.hannes@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Hannes Küttner <4376726+hanneskuettner@users.noreply.github.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Kevin Lewis <kvn@lws.io>
2024-06-20 21:21:06 +02:00
Hannes Küttner
97af4d1507 Fix primary key unique constraint error extraction for MySQL (#22434)
* Fix primary key unique constraint error extraction for MySQL

* Add changeset

* Add changeset

* Add comment about MariaDB

* Drop 5.7 special case

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2024-06-14 15:42:31 -04:00
ian
f5dcd85082 Add user limits (#22479)
* Limit users

* Add defaults

* Allow specifying of reason for limit exceeded error

* Update config options doc

* Make existing tests pass

* Update extensions limit error

* Update usage of regular expression

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Rename typo

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Rename files

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* Use first() and remove redundant +=

* Fix incorrect api access count

* Rework to account for passing of unchanged access values

* Fix increased counts in updateMany

* Consistent error message

* Simplify unnecessary find

* Add tests

* Rename UserCount to AccessTypeCount

* renamed env var

* prettier

* Add changeset

* fix limit checking for batch and status updates

* test and prettier

* removed obsolete check

* updated error

* fixed error usage in extenions service

* Use randomUUID from '@directus/random'

* Fix payload check in updateMany

* implemented RolesService.updateBatch from its parent

* resolved unit test error

* updated type

* fixed existing role query

* Temporary activation of blackbox tests

* Move to separate fn, to make skippable for non-existent role

* Revert "Temporary activation of blackbox tests"

This reverts commit 4c4ac846d6.

* Revert "Move to separate fn, to make skippable for non-existent role"

This reverts commit 1d90a82e39.

* Add user limits - extension (#22642)

* adressing existing users issue

* migrated changes from pascal

* only check the role for active users

* only count active users

* updated incorrect if

* default to count zero

* Undid abstraction to separate function

* fixed updating through user counting error

* prettier

* simplified fallback query

* prettier

* Added try catch to be safee

* updated db mocking for tests

* removed extra check to satisfy implementation tests

---------

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Brainslug <tim@brainslug.nl>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2024-06-04 17:00:17 +02:00
Pascal Jufer
ae9c64125c Specific error type mapping in isDirectusError function (#22346)
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2024-05-03 09:53:22 +02:00
Pascal Jufer
82b23a57d4 Add the Directus Marketplace (#21674)
* Add support for MIGRATIONS_PATH (#20627)

* Deprecate "local" extension types (#20624)

* No longer resolve local non-package extensions

* Remove local extensions from watcher

* Install fs-extra for app dev

* Replace usage of get-extensions

* Rename create functions to match behavior

* Drop extensions prefix from cli create

* Drop name requirement from cli add

* Drop extensions name regex checks

* Don't rely on regex for extension identification

* Add changeset

* Fix build for @directus/extensions

* Don't ensure nested dirs

* Load npm dependency by name prefix

* Throw more helpful error when pkg json is missing

* Fix dependency loading paths

* Only load local extensions if extensions path exists

* Fix merge conflict

* Remove unused fs-extra

* Only sync extensions if location is defined

* `extensions` -> `localExtensions`

* Fix resolving of package extensions

- consistent usage of term "package extensions"
- enhance error messages

* Fix syncing extension from storage

* Revert-revert change from #20627

---------

Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Add back `MIGRATIONS_PATH` to `@directus/env` (#21053)

* Setup browse endpoint for browsing Directus extensions (#21033)

* Rename tag directus-custom to directus-extension

* Devil's in the details

* [WIP] Start on search lib

* Setup extensions-registry package

* Update pnpm lock

* Move registry search fn to new package

* Squash bugs

* Remove unused deps

* Remove unused dep in `extensions`

* Add tests for validate-limit

* Finish test coverage

* Remove unused constant

* Add back lodash-es 😇

* Install extensions-registry

* Add default keywords

* Expose registry endpoint in rest api

* Add describe module

* Reinstal pnpm

* Drop reliance on author field

The problem is that there's no requirement for it to be a valid(ated) user. Publisher is the only field we can rely on being an existing NPM user.

* Update convertSearchResult test after author change

* Format

* Allow anything after the route

To support packages with scopes

* Add test for describe

* Add more tests

* Throw unprocessablecontenterror instead of error

* Install errors pkg

* Finish test coverage

* Don't require description

* Don't wrap individual keywords

* Run formatter

* Add changeset

* Update packages/extensions-registry/src/modules/search/utils/validate-text.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Return sandbox information

* Drop support for directus-custom keyword

* Set type to null for multiple tags

* Restart api for new registry

* Add version utility

* Add out-of-date error

* Add assertion for api versions

* Finish new list method

* Add describe

* Use updated registry functions

* Fix controller usage

* Return output in data flag to be consistent

* Add tests for the describe module

* Add tests for list module

* Finish tests for extensions-registry package

* Remove vscode shenanigans

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Project Forum: Browse UX (#21276)

* Render marketplace route in-app

* Update api list integration

* Add support for limit/offset in list

* Boilerplate route for drawer

* Render extension readme in drawer

* Render extension detail as separate page

* Setup markup for extension banner

* Render readme with markdown styling

* Boilerplate metadata block

* Use v-list for extension metadata metrics

* Render compatibility metric

* Show downloads and publish date

* Refactor to individual files

* Fix missing import

* Add size metric

* Show author on metadat

* Show verified badge

* Finish metadata

* Run formatter

* Style forum detail banner (#21290)

* Style banner contents

* Add background art

* Add banner styles to default dark mode theme

* Undo base style change to avoid conflict

* Add beta chip badge to settings nav (#21296)

* Add layout styling for detail page (#21297)

* Render metadata as grid if size allows

* Finish layout for detail page

* Fix alignment of count

* Fix search-input active state

* Finalize forum header

* Add marketplace banner

* Style registry filter

* Change v-list-item height to min-height

Allows rows for forum registry to grow

* Finalize styling for extension list item

* Finalize styling for registry

* Don't rely on format util

* Forum account detail page (#21299)

* Add support for author describe endpoint

* Boilerplate author page

* Add author endpoint

* Update naming to account

* Render github name on registry list

* Render user's name + avatar on detail

* Load account data on account page

* Restructure module

* Render account banner on account page

* Render account metadata

* Finish account detail page

* Run formatter

* Add sidebar info detail sections

* install extensions from registry (#21070)

* added an endpoint to install extensions from registries

* add changeset

* Revert format/lint scripts

* Update pnpm-lock

* Remove unused dependency

* Add download module to extensions-registry abstraction

* Download from extension registry instead of direct npm

* Throw on error

* Install based on version ID rather than name

* Install based on ID in body

Allows us to extend it in the future if need be

* Download to tmp path instead of extensions

* Save to extensions manager flattened

* Use const for package folder name

* Run more in try/catch to avoid server exit

* Remove changeset

* Undo docs updates

We'll do all docs in a single PR :)

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Manage extensions by directus_extensions primary key instead of npm package name (#21300)

* Add marketplace migration

* Include bundle info

* Rename name to folder

* Store name as folder

* [WIP] Change name to ID

* Update API to use UUIDs for extensions

* Update app to use extensions id

* Run formatter

* Add changeset

* Update generate-app-entrypoint to use updated format

* Fix graphql usage

* Format

* Add forum uninstall endpoint (#21301)

* Bootstrap uninstall

* Finish uninstall endpoint

* Use shared extensions store state for managing extensions in settings (#21317)

* Use extensionsstore for extensions management

* Use shared store for extensions state and modification

* Remove unused emit

* Adds install/uninstall GUI elements for project forum (#21319)

* Use warning color for incompatible

* Styling tweaks

* Transition optional background color

* Add keys for installing

* Add install button to extension detail

* Finish install button

* Add uninstall option to extensions item

* Empty local cached extensions on start

* Add (hopefully) temporary hack to await reload

* Add install / uninstall buttons

* Update lockfile

* Fix bundle entries not rendering properly

* Clean up spinner styling

* Properly wait for reload to be done

* Run formatter

* Fix registry name

* Fix local extension loading

* Render location/company in byline

* Rearchitect metadata to unstyled listitems

* Add max-width to page container

* Only allow admins to install/uninstall extensions

* Remove unused link

* Disable eslint for map use

* Run formatter

* Add no-results indicator (#21389)

* Add no-results indicator

* add error state as well

* Standardize trailing slash

* Standardize pk check

* Resolve unnecessary check

* Move all deps to dev deps in app

* Optimize exec order

* Pull data from top level package

* Redesign filter bar

* Reset page count on filter change

* Use default padding on top of page

Feels weird now, but there's a theme-update coming that'll make this make sense :)

* Show first/last in pagination

* Auto-format package name

* Use updated registry api return names

* Add sorting by total downloads

* Add sparkline for downloads

* Add a bit of padding

* Finalize sparkline styling

* Run formatterg

* Add sandbox flag in list call

* Add MARKETPLACE_TRUST env var

* Set sandbox requirement on download

* Render list item badge

* Fix padding

* Show license in gui

* Add support for multiple maintainers

* Add activity tracking for install

* Make incompatibility notice friendlier

* Tweak margin of sparkline

* Update default CSP headers to include github content

* Make extension readme selectable

* Consistent icon, add tooltips

* Reduce size of install button

* Run formatter

* Render readme anchors with target blank

* Add min width to v-select components

* Fix background on search input

* Reduce padding on marketplace pages

* Fix showing no results

* Be consistent

* Render type filter menu in full height

* Clean up compatibility notice

* Hide count result when search count is 0

* Use v-show instead

* Bump no results notice down a bit

* Cleanup size of install buttons

* Use buttons for install / author

* Fix sparkline rendering

* Add better loading state for extension detail page

* Add spinners

* Render proper error message in case readme is missing

* Use heart logo for marketplace

* Add image border shadow

* Add loading banner on account page

* Show reload warning as persistent notification

* Only show one notification

* Update snapshot

* Add padding to verified icon

* Base extensions id on package id, show installed badge

* Sync pagination / search / filter with URL

Special shoutout to Tim

* Next max ext (#21541)

* Add limit exceeded error

* Add limit exceeded translation

* Add extensions_limit env var

* Load extensions limit in app

* Don't allow installing unknown versions

* Don't require installed flag

* Return bundled entries in describe

* Update chip to match style

* Prevent install on limit reached

* Run formatter

* Fix bundle loading

* Resolve npm extensions by package manifest (#21478)

* Merge main into next (#21669)

* Clean-up deps

* Don't export type (as long as not used anywhere)

* Update semver to latest

* Relax compatibility notice

* Use more suitable icon for version

* Re-render download charts on color-scheme change

* Link extensions to marketplace

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: ian <licitdev@gmail.com>
Co-authored-by: Mahendra Kumar <22556323+mahendraHegde@users.noreply.github.com>
2024-03-04 14:34:59 -05:00
Pascal Jufer
81c9156508 Update Vitest to v1 (#20694) 2023-12-10 19:49:45 +01:00
Pascal Jufer
ba7fac402e Update Prettier v3.1.0 (#20481) 2023-11-20 16:23:22 +01:00
Boegie19
d82bb9c823 move api/errors to @directus/errors package (#20054)
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-10-18 18:19:22 +02:00
Rijk van Zanten
ca3e7f521f Upgrade previous "Extensions" system to new "Errors" model (#18797)
* Typecheck across packages that are built with esbuild

* Boilerplate new Errors package

* No need, tsup checks with --dts

* Switch to tsup

* Setup dev script

* Add readme

* More boilerplaty things

* Finish createError function

* Install @directus/random

* Downgrade node types

* Add utility function to check if an error is a DirectusError

* Use new is-error check

* Install errors package

* Add failed validation common error

* Export common errors

* Move joi convertion to utils

* Export failed validation

* Use new failed validation error in validate-batch

* Enhance typing output of createError

* Remove outdir (handled by tsup now)

* Replace Exception with Error

* Replace exception in test

* Remove exceptions from app

* Remove exceptions from app

* Remove failed validation exception from users service

* Remove old failed validation exception from shared

* Remove exceptions package in favor of errors

* Uninstall exceptions

* Replace baseexception check

* Migrate content too large error

* Critical detail

* Replace ForbiddenException

* WIP remove exceptions

* Add ForbiddenError to errors

* HitRateLimitError

* Move validation related error/helper to new validation package

* Add index

* Add docs

* Install random

* Convert TokenExpired

* Convert user-suspended

* Convert invalid-credentials

* Move UnsupportedMediaType

* Replace wrong imports for forbidden

* Convert invalid-ip

* Move invalid provider

* Move InvalidOtp

* Convert InvalidToken

* Move MethodNotAllowed

* Convert range not satisfiable

* Move unexpect response

* Move UnprocessableContent

* Move IllegalAssetTransformation

* Move RouteNotFound

* Finalize not found

* Various db errors

* Move value too long

* Move not null

* Move record-not-unique

* Move value out of range

* Finish db errors

* Service unavailable

* GQL errors

* Update packages/validation/src/errors/failed-validation.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update packages/validation/src/errors/failed-validation.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* InvalidQuery

* Add test for invalid query message constructor

* Invalid Payload

* Finalize exceptions move

* Improve type of isDirectusError

* Various fixes

* Fix build in api

* Update websocket exceptions use

* Allow optional reason for invalid config

* Update errors usage in utils

* Remove unused package from errors

* Update lockfile

* Update api/src/auth/drivers/ldap.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Update packages/validation/src/utils/joi-to-error-extensions.ts

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Put error codes in shared enum

* Replace instanceof checks in api

* Fix tests I think

* Tweak override names

* Fix linter warnings

* Set snapshots

* Start fixing BB tests

* Fix blackbox tests

* Add changeset

* Update changeset

* Update extension docs to use new createError abstraction

* 🙄

* Fix graphql validation error name

* 🥳

* use ErrorCode.Forbidden

* fix blackbox auth login test

* Add license files

* Rename preMutationException to preMutationError

* Remove unused ms dep & sort package.json

* Remove periods from error messages for consistency

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>

* Add optional code check

* Use updated error code checker

* Rename InvalidConfigError to InvalidProviderConfigError

---------

Co-authored-by: Azri Kahar <42867097+azrikahar@users.noreply.github.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: ian <licitdev@gmail.com>
2023-06-27 17:22:26 -04:00