Commit Graph

34 Commits

Author SHA1 Message Date
Rijk van Zanten
c45cd69dd7 v10.8.3 2023-12-21 14:19:08 -05:00
Pascal Jufer
98532788d5 Selective Patch Tuesday 🐸 (#20796) 2023-12-20 00:26:15 +01:00
Pascal Jufer
5e72ea1b08 Patch Tuesday 🐸 (#20714) 2023-12-12 04:36:50 +01:00
Pascal Jufer
1a07d7355d Update TypeScript to 5.3.3 (#20710) 2023-12-11 20:39:34 +01:00
Pascal Jufer
81c9156508 Update Vitest to v1 (#20694) 2023-12-10 19:49:45 +01:00
Nicola Krumschmidt
e57312a2f2 Add select node to abstract SQL that combines primitive and fn (#20654) 2023-12-06 17:29:00 +01:00
Nicola Krumschmidt
4f2a90c345 Split up abstract sql in root and sub query (#20638)
* Split up abstract sql in root and sub query

* Finish new root/sub query abstract sql mechanism

Co-authored-by: Jan Arends <jaads@users.noreply.github.com>

* playground

* Fix getMappedQueriesStream by closing controller

Co-authored-by: Jan Arends <jaads@users.noreply.github.com>

* Add double nested o2m example to playground

* adopted test to create o2m sub queries

* adopted more tests

* more test fixes

* removed playground again

* Fix tests

Co-authored-by: Jan Arends <jaads@users.noreply.github.com>

* Remove orm dir and move contents to utils

* Add tests for mapResult

* Remove orm export

---------

Co-authored-by: Jan Arends <jaads@users.noreply.github.com>
Co-authored-by: Jan Arends <jan.arends@mailbox.org>
2023-12-06 15:07:11 +00:00
Jan Arends
08bbe27794 Durus: nested type refactoring (#20553)
* initial thoughts

* optional meta data for a2o with json

* added test

* some docs

* Adjust abstract query to support single column any relations

* fix build

* added relational any option to nested one

* applied type refactoring in converter

* improved a2o relation definition

* no pk column name in any json column

* added place for a2o sub query generator

* docs on type

* deleted test

Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>

* docs

* reverted preps

* renamed meta to nesting

* removed reverted prep

* removed unnecessary check

* docs

* renamed meta

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>

---------

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>
2023-12-04 18:18:55 +00:00
Pascal Jufer
d30bb69490 Clean-up deps (#20581) 2023-11-30 00:46:09 +01:00
Pascal Jufer
a5215d674b Update TypeScript to 5.3.2 (#20571) 2023-11-29 13:54:56 +01:00
Pascal Jufer
ba7fac402e Update Prettier v3.1.0 (#20481) 2023-11-20 16:23:22 +01:00
Pascal Jufer
454ecdbefe Update Vitest to v0.34.6 (#20477) 2023-11-20 13:04:56 +01:00
Nicola Krumschmidt
d9bd5adeb1 Refactor the file structure in data-sql (#20433)
* Refactor data-sql types file structure

* used types dir directly

* target conversion outside of conditions, file for filter conversion result

* type import only via index file

* removed unused index files

* Refactor data-sql types file structure

* used types dir directly

* target conversion outside of conditions, file for filter conversion result

* type import only via index file

* removed unused index files

* fixed tests

* rename

---------

Co-authored-by: Jan Arends <jan.arends@mailbox.org>
2023-11-17 20:41:25 +01:00
Jan Arends
a920cd3978 Durus: Modifiers on o2m (#20411)
* docs

* added todo test

* refactoring

* refactoring

* convert filter

* conversion of all modifiers

* added more modifiers to test

* fixed tests

* test for various things

* Fix joins in nested many generation

---------

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
2023-11-16 10:01:23 +01:00
Nicola Krumschmidt
d5861365e9 Make modifiers and aliases on abstract queries required (#20393)
* Make modifiers on abstract queries required

Since the properties on the modifiers object are already optional, we can explicitly require the modifiers property

* Require at least one element if sort modifier is present

* Make aliases on abstract queries required

* Remove alias from abstract sql

* Always use aliases when generating alias mapping
2023-11-15 12:02:32 +01:00
Jan Arends
5c7c79dfc2 Durus sorting on an m2o field (#20357)
* Changed sort conversion to support nested targets

* Refactoring in sort test

* integrated new sort conversion

* formatting

* removed type errors

* test for sort in modifier conversion

* remove unneeded stuff from driver

* fixed tests

* removed unused type

* support functions for sort

* Remove not needed error handling

* Allow sorting by functions in query converter

---------

Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
2023-11-10 19:48:28 +00:00
Jan Arends
6f0205d054 Durus: Filtering on m2o relations (#20222)
* initial ideas

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>
Co-authored-by: ian <licitdev@users.noreply.github.com>

* removed function as target again

* converter test like the rest

* TDD test for converter

* TDD test for pg statement creation

* fix

* convert nested condition target

* test for function target conversion

* made condition nodes return join clause for nested filter

* handled the additional join for logical nodes

* adopted changes in filter test

* fixed test

* added join to modifiers

* Allow same target in all conditions

* Fix AbstractSqlQueryOrderNode type

* Fix deeply nested filter conversion

* Fix condition converters with nested target

* Make filter result clauses required

* Fix merging joins

* Fix convertLogical type

* Allow all SQL condition nodes to be functions

* fixed test

* rename

* new test, separated integration tests

* remove duplicated joins

* formatter

* fixed string condition

* adopted tests

* preparation for sort

* removed test complexity drastically

* clean up join duplicate removal

* unit test for modifier conversion

* reverted converter test

* removed join duplication removal

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>
Co-authored-by: ian <licitdev@users.noreply.github.com>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
2023-11-10 18:40:47 +01:00
Jan Arends
b2fe81b596 Durus o2m field selects for PostgreSQL (#20037)
* test to split up root query into sub queries

* moved splitter to driver

* moved splitter to data-sql

* filtered o2m within conversion

* removed splitter

* fix

* fix

* util to get root query

* refactoring, preparation for root and sub queries

* preparations for making separate sub queries

* refactoring

* implemented creation of sub query

* made store in relational metadata mandatory

* fixed export

* abstracted the database request

* get root query definition

* preparation to merge streams

* started converting nested many queries in converter

Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>

* refactoring

* at least two childnodes for logical operator

* refactoring missing module

* fixed tests for fields

* refactoring, mainly rename current to internal

* start integrating second approach

* getting closer but wip

* getting closer

* fixed naming

* merged streams into final stream

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* fixed returning array

Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>

* added generic type

* use existing many join node again

* fixed test

* made use of generic type

* changed wording for relational stuff

* composite keys

* new generator for sub query parameters

Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>

* worked on index test

* more specific return type for driver

* fixed index file tests

* removed approach on abstract query level

* worked on index test but wip

* simple test for nested many node

* continued on index test

* finally finished test

* randomized test values again

* moved common nested many logic up to data-sql

* remove closing controller

* typedocs and error handling

* single test for nested many querying logic

* refactor: added utility function for streams

* Remove unneeded type declaration

* Remove bind call

* Use AtLeastOneElement type alias

* Use unknown instead of any as value of Records

* Rename loadAllResultIntoMemory to readToEnd

* Fix o2m with compound keys

* Fix type error in createJoin

* Simplify AbstractSqlNestedMany type

* catch error of conversion in data-sql

* removed additional error handling in driver

* replaced table alias with actual table name

* Allow join field values to be numbers again

* Clarify property names on AbstractSqlNestedMany some more

* nested many node creation tests

* docs

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
2023-10-27 12:19:46 -04:00
Pascal Jufer
e987c1990e 10.6.4 (#20102) 2023-10-20 18:55:45 +00:00
Nicola Krumschmidt
2fed2670c8 Remove root property from AbstractQuery and minor cleanup (#19992)
* Move fields file out of fields folder

* Remove root property from AbstractQuery
2023-10-11 00:00:26 +02:00
Nicola Krumschmidt
6c45914b8a Split up related field node type in data abstraction layer (#19942)
* Fix casing in file names

* Refactor file structure

* Split up related field node

* formatter

* renamed fields-node.ts to fields.ts

* Small grammar fix in comment

Co-authored-by: Jan Arends <jan.arends@mailbox.org>

---------

Co-authored-by: Jan Arends <jan.arends@mailbox.org>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-10-09 10:12:42 -04:00
Jan Arends
d6e6208c26 Data abstraction: Field selection, modifiers and m2o for PostgreSQL driver (#19146)
* Naming typo

* First experiments

* Setup testing playground

* Fetch in client stream

* Add logging

* Remove unneeded stream to string util

* added logical operator to complex test and cleanup

* Improve typing

* started adding function support

* Support nested m2o in data-sql

* Implement join sql string creation

* It's alive!

* Remove unused aliases

* Add todo reminder

* fix build

* small improvements for fn conversion

* function conversion in pg driver

* more expressive typedocs

* toMatchObject in data sql to ignore alias and path

* added type to sort node

* moved and fixes tests for comparison

* moved condition tests

* separate file for functions

* test for function condition

* added proper args to function although not in use

* AT TIME ZONE 'UTC' when needed, proper arg value

* intersects support in pg driver

* convert geo value

* docs for intersects

* reworked column as function input

* support for functions in abstract select

* fixes tests

* count support in select

* refactoring: split up filter conversion

* starting every and some operators

Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>

* extracted variable in test

* in operator with sub query support

* split up conditions type into multiple types

* type for a single query parameter

* condition type

* intersects_bbox in pg driver

* finalized type declarations

* geo condition types

* pnpm lock update

* removed playground

* join

* fixed geo

* fixed types in test

* changeset

* made path prop required again

* geojson for intersects to the driver

* removed sub query from set-condition

* clean up form sub query removal

* added between support

* fixed geo condition test type

* formatting

* moved number operators to utils

* xy-condition to condition-xy

* remove between

* between clean up

* formatter

* refactoring: split up condition generation

* changeset

* split up type for geo condition

* refactoring: split up conditions converter

* fix formatter

* split up types

* split up abstract query into multiple files

* export fix

* split up condition builders in pg driver

* enabled all functions in select and nr condition

* adjusted join conversion and added test

* added test for node conversion

* moved index generator to converter dir

* docs

* format fix

* split up abstract query types further

* split up abstract sql types further

* split ups in converter and fine granular tests

* fix format

* proper index file in converter

* proper type for generator

* more docs on data-sql

* updated data readme

* readme tweaks

* removed circular dependency and added union type

* moved create-identifier function

* added index file in sql utils

* comment on converter usage

* removed circular deps in sql types

* last dependency fixes in data sql

* fixed tests

* cleanup up dependencies in overall abstract query

* removed base type to reduce noise

* reduce some index files in sql types declaration

* formatter

* more dependency tweaks

* formatter

* graphs for data and data-sql

* added dependency cruiser

* added lock file

* pnpm downgrade 8.6.0 to fixed lock file

* try fixing again with 8.6.2

* formatting

* formatting, now with correct formatter

* fixed dependencies

* formatter

* extracted variable for more readable code

Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>

* formatter

* fixed export of condition types for driver

* consistent file naming for conditions

* updated docs

* type fix

* removed todo

* another type fix

* split up expand func and added test

* todo test for driver class

* fixed unit test

* fixed typo

* removed todo, though about it, it's good as it is

* separated path map creation from query-converter

* renamed converter to query-converter

* user specific aliases for primitives

* fixed linter

* user specified alias for m2o

* documentation

* redesigned the function types and added alias support

* formatter

* unit test for pg driver index file

* moved call to source as class member

* moved alias map creation back to converter

* moved unique alias generation to ORM

* tsup update

* renamed nodes to fields

* redefined return type of query converter

* modifier conversion similar to fields conversion

* added response converter dir again

* aligned unit tests

* fixed circular dependency

* moved orm logic one level up again

* formatter

* removed SVGs

* Update pull_request_template.md

* Update pull_request_template.md

* Run formatter

* a bit renaming and restructuring

* formatter

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
2023-10-06 09:32:04 -04:00
Pascal Jufer
e1dd6fa037 Update TS related dependencies and configurations (#19866) 2023-09-29 14:45:47 +02:00
Pascal Jufer
c67775a9a0 Bundle packages (#19712)
* Bundle `@directus/composables`

* Bundle `@directus/constants`

* Align tsconfig in `data` packages

To prevent warnings like
> Cannot write file 'packages/data-sql/dist/index.d.ts' because it would overwrite input file

* Bundle `@directus/pressure`

* Align tsconfig in `@directus/random`

* Bundle `@directus/schema`

* Bundle `@directus/storage`

* Bundle `@directus/storage-driver-azure`

* Bundle `@directus/storage-driver-cloudinary`

* Bundle `@directus/storage-driver-gcs`

* Bundle `@directus/storage-driver-local`

* Bundle `@directus/storage-driver-s3`

* Align tsconfig in `@directus/themes`

* Bundle `@directus/update-check`

* Add changeset

* Bundle `storage-driver-supabase`

* Update changeset

* Remove dedicated export of 'types/overview'
2023-09-26 21:14:27 +08:00
rijkvanzanten
373cc9f4fa v10.5.0 2023-07-25 17:36:33 -04:00
Pascal Jufer
e583c6b6ea Add tsconfig for Vue & CJS (#19197)
* Add tsconfig for Vue

* Delete tsconfig.json

* Use vue tsconfig for composables

* Add changeset

* Update changeset

* Update changeset
2023-07-18 17:36:16 -04:00
rijkvanzanten
d3c9de9dfb v10.4.3 2023-07-11 16:00:17 +02:00
Jan Arends
84bd5d0ef3 Basic filters for PostgreSQL driver (#19005)
* conversion to abstract sql

* greater than for where clause in pg driver

* refactoring

* forward targets again

* more filter options and fixes

* added filters, moved operator mapping into driver

* removed trim

* add changeset

* removed first type approach

* enhanced abstract sql

* value and set comparison in abstract query

* split up where and intersect again

* preps for where comparison to a set

* typedocs

* fix linter

* The random things we did, who knows

* adding logical conditions (WIP)

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>

* Push values to parameters directly

* Fix convert test

* Fix convert filter tests

* Implement filter conversion for logical and negate nodes

* Fix where tests

* Add logical where stringification

* Negate comparison operators directly

* Remove some unneeded parentheses

* Improve parameter index generator type

* Fix formatter issues

* Add todo statements

* Remove unused node type

* Removed unneeded check

* One more unnecessary check

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>
Co-authored-by: Nicola Krumschmidt <nickrum@users.noreply.github.com>
Co-authored-by: Nicola Krumschmidt <nicola.krumschmidt@freenet.de>
2023-07-02 15:10:55 +02:00
Connor
92621429ee Linting and Formatting Upgrade (#18892)
* add docs to eslint

* update prettier ignore

* fix vitepress linting

* eslint ignore fixes

* prettier run

* update prettier ignore

* fix formatting

* enable linting of markdown files

* revert format command change

* fix irregular whitespace

* update dictionary

* (Changelog) Create four-boxes-shake.md

* Rework ESLint / Prettier setup

- Disable js/ts/vue files for Prettier to ensure linting/formatting is
  only happening via ESLint
- Rework formatting of code blocks in md files
  - Disable formatting of code blocks in md files under '/docs' by Prettier
  - Instead use "eslint-plugin-markdown" to format & __lint__ js*/ts*/vue such code blocks
  - Replace unmaintained "eslint-plugin-md" plugin by official "eslint-plugin-markdown" plugin
  - I'll check whether we can use this to format other code blocks
    (json, html, ...) as well
- Restructure, clean-up and apply some fixes to the ESLint config
  (Note: Not ready for flat config yet since not supported by
  vscode-eslint)
- Enable cache for ESLint / Prettier in scripts
- Clean-up ignore file
  - Explicit folder declaration (.../)
  - Don't ignore all 'extensions' folders in ESLint (only
    '/api/extensions/')
  - Enable formatting in '/.github' folder

* Fix all formatting issues with Prettier

* Update md files under /docs/.typedocs

* Fix lint issues in vue/js files

* ESLint / Prettier config revision v2

Enable Prettier for md code blocks, but only as warnings since it can
get into the way with Vitepress md extensions like '[!code ...]'
comments

* Remove prettier-ignore comments

* Make spellchecker happy

* Remove changeset

* Revert lint setup for code blocks

There are many cases in the docs where linting / formatting of code
blocks doesn't make
sense:
- Code block is only an excerpt - linter fails
- Code block contains special comments (e.g. markdown extensions) which
  needs to remain at the same place - formatting would break it
- ...

* Apply lint issues / formatting from temp lint setup

* Run formatter

* Fix merge failure

* Simplify & modernize ESLint / Prettier setup

No longer run Prettier via ESLint. Nowadays, this is the recommended
setup. There's no real need to run it this way, it's just an additional
layer.

Add VS Code settings to make the work with the new setup easier.

* Remove unused eslint disable directives

* Make editorconfig more useful

* Fix formatting issues reported by editorconfig

* Format files with Prettier

* Enable formatting of source translations file

* Format source translations file

* Remove unnecessary console error

* Remove unnecessary line

* Only ignore md files under .changeset

* Add CI reporter for Prettier

* Fail job on wrongly formatted files

* Fix format

* Test Prettier action on changed/added file

* Use simple CI format check for now & no cache

* Revert "Test Prettier action on changed/added file"

This reverts commit 4f7d8826ad.

* Introduce code blocks check for docs

* Fix code block issues

* Ignore auto-generated packages dir

* Fix comment position

* Also lint `/app/.storybook`

* Reformat modified files

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-06-29 11:54:01 +02:00
rijkvanzanten
2239ef8db7 v10.4.0 2023-06-27 17:43:26 -04:00
Jan Arends
0fea5ab016 converted abstract sort modifier (#18938)
* converted sort

* update comment

* changeset

* change wording

* renamed converter tests

* updated docs

* Change changeset description

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* docs

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
2023-06-20 10:36:44 -04:00
Jan Arends
1cbae4952c Limit and offset modifiers for Postgres driver (#18894)
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
2023-06-19 15:51:12 +02:00
rijkvanzanten
c0cd7860b1 v10.3.0 2023-06-13 12:52:12 -04:00
Rijk van Zanten
bc7e92e4b0 Add boilerplates for da project (#18642)
* Move data types to data package

* Setup boilerplate for driver

* Add basic data manager class

* Rename "datastore" to "store"

* Rename "fieldNodes" -> "nodes"

Opinions incoming!

* Setup manager basics

* fix pnpm-lock file

* Regenerate pnpm-lock file with pnpm v8.5

* add flowchart diagram

* revert flowchart diagram

* Add a basic data driver for PostgreSQL (#18808)

* add very basic query builder

* Start new data-sql package

* Start noodling

* Results of friday collab session

* add tests and separate function

* add flowchart diagram

* fix tests

* dependency diagram and fix

* add tsdocs

* fix query builder test

* small refactoring

* add underlying error in error message

* add tests for escape function

* add check if pool has a client to query the db

* revert tests, query func handles clients itself

* Update packages/data-sql/package.json

* Resolve comments

* Pass query

* Add tsup

* Remove compileroptions with tsup usage

* Split up converter

* Update graph

---------

Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>

* Improvement: Add albanian language translation (#18810)

* Add albanian to available languages

* Add me as contributor (CLA)

* Create fair-clouds-exercise.md

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Add extra type check in apply-diff (#18783)

* Fix content filter between (#18821)

* fixed incorrectly typecast value

* Create popular-toes-tap.md

* Created new contribution pages for non-code contributions (#18820)

* Community

* Fixed copy in community page

* Sponsrship and advocacy

* Clarified 80/20 rule in support page

* Reworked contribs intro

* Contributing sidebar

* Remove backing directus page

* Translations updates

* Spell check

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>

* Reuse S3 client and increase maxSockets (#18799)

* Reuse S3 client and increase maxSockets

* Update test to check for timeouts and non-2xx responses

* Create beige-olives-beam.md

* Fix unit tests

* Update packages/storage-driver-s3/src/index.ts

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Update .changeset/beige-olives-beam.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Switch to tsup builds

* Don't require connect/disconnect

* Add test for engine

* Fix tests

* Cleanup tests

* Update tests

* Add destroy method

* Call register on register

* Tweak readme to drop connect

* Add changeset

* Wrapper function for random identifiers (#18870)

* add util function in data for random identifiers

* move wrapper into random package

* fix docs typo

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

* Rename to identifier and add test

---------

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: rijkvanzanten <rijkvanzanten@me.com>

* Update packages/data/readme.md

Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>

---------

Co-authored-by: Jan Arends <jan.arends@mailbox.org>
Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch>
Co-authored-by: Nino Gjoni <49320058+ninogjoni@users.noreply.github.com>
Co-authored-by: Brainslug <br41nslug@users.noreply.github.com>
Co-authored-by: Kevin Lewis <kvn@lws.io>
Co-authored-by: ian <licitdev@gmail.com>
2023-06-13 11:10:47 -04:00