Add support for Geometry type, add Map Layout & Interface (#5684)

* Added map layout

* Cleanup and bug fixes

* Removed package-lock

* Cleanup and fixes

* Small fix

* Added back package-lock

* Saved camera, autofitting option, bug fixes

* Refactor and ui improvements

* Improvements

* Added seled mode

* Removed unused dependency

* Changed selection behaviour, cleanup.

* update import and dependencies

* make custom style into drawer

* remove unused imports

* use lodash functions

* add popups

* allow header to become small

* reorganize settings

* add styling to popup

* change default template

* add projection option

* add basic map interface

* finish simple map

* add mapbox style

* support more mapbox layouts

* add api key option

* add mapbox backgrounds to layout

* warn when no api key is set

* fix for latest version

* Improved map layout and interface, bug fixes, refactoring.

.

.

* Added postgis geometry format, added marker icon shadow

* Made map buttons bigger and their icons thinner. Added transition to header bar.

* Bug fixes and error handling in map interface.

* Moved box-select control out of the map component. Removed material icons sprite and use addImage for marker support.

* Handle MultiGeometry -> Geometry interface error.

* Removed hardcoded styles. Added migrations for basemap column. Lots of refactoring.

Removed hardcoded styles. Added migrations for basemap column. Lots of refactoring.

* Fixed style reloading error. Added translations.

* Moved worker code to lib.

* Removed worker code. Prevent Mapbox from removing access_token from the URL.

* Refactoring.

* Change basemap selection to in-map dropdown for layout and interface.

* Touchscreen selection support and small fixes.

* Small change.

* Fixed unused imports.

* Added support for PostgreSQL identity column

* Renamed migration. Added crs translation.

* Only show fields using the map interface in the map layout.

* Removed logging.

* Reverted Dockerfile change.

* Improved crs support.

* Fixed translations.

* Check for schema identity before updating it.

* Fixed popup not updating on feature hover.

* Added feature hover styling. Fixed layer customization input. Added out of bounds error handling.

* Added geometry type and support for database native geometries.

* Fixed linting.

* Fixed layout.

* Fixed layout.

* Actually fixed linting

* Full support for native geometries
Fixed basemap input
Improved feature popup on hover
Locked interfaced support

* Fixed geometryType option not updating

* Bug fixes in interface

* Fixed crash when empty basemap settings. Fixed fitBounds option not updating.

* Added back storage type option. Improved interface behaviour.

* Dropped wkb because of vendor inconsistency with binary data

* Updated layout to match new geometry type. Fixed geojson payload transform.

* Added missing geometry_format attributes to local types.

* Fixed typos & refactoring

* Removed dependency on proj4

* Fix error when empty map interface options

* Set geometry SRID to 4326 when inserting into the database

* Add support for selectMode

* Fix error on initial source load

* Added geocoder, use GeoJSON for api i/o, removed geometry_format option, refactoring

* Added geometry intersects filter. Created geometry helper class.

* Fix error when null geometryOptions, added mapbox_key setting.

* Moved all geometry parsing/serializing into processGeometries in `payload.ts`. Fixed type errors.

* Migrate to Vue 3

* Use wellknown instead of wkx

* Fixed basemap selection.

* Added available operator for geometry type

* Added nintersects filter, fixed map interface for filter input

* Added intersects_bbox filter & bug fixes.

* Fixed icons rendering

* Fixed cursor icon in select mode

* Added geometry aggregate function

* Fixed geometry processing bug when imported from relational field.

* Fixed error with geocoder instanciation

* Removed @types/maplibre-gl dependency

* Removed fitViewToData options

* Merge remote-tracking branch 'upstream/main' into map-layout

* Fixed style and geometryType in map interface options

* Fixed style change on map interface.

* Improved fitViewToData behaviour

* Fixed type imports and previous merge conflict

* Fixed linting

* Added available operators

* Fix and merge migrations

* Remove outdated p-queue dep

* Fix get-schema column extract

* Replace pg with postgis for local debugging

* Re-add missing import

* Add mapbox as a basemap when key exists

* Remove unused tz flag

* Process delta in payloadservice

* Set default map, add limit number styling

* Default display template to just PK

* Tweak styling of error dialog

* Fix method usage in helpers

* Move sdo_geo to oracle section

* Remove extensions from ts config exclude

* Move geo types to shared, remove _Geometry

* Remove unused type

* Tiny Tweaks

* Remove fit to bounds option in favor of on

* Validate incoming intersects query

* Deepmap filter values

* Add GraphQL support

* No defaultValue for geometryType

* Resolve c

* Fix translations

Co-authored-by: Nitwel <nitwel@arcor.de>
Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
This commit is contained in:
Oreille
2021-08-12 22:01:34 +02:00
committed by GitHub
parent 66d4c04121
commit 83e8814b2d
67 changed files with 4982 additions and 231 deletions

View File

@@ -0,0 +1,163 @@
import { Field, RawField } from '@directus/shared/types';
import { Knex } from 'knex';
import { stringify as geojsonToWKT, GeoJSONGeometry } from 'wellknown';
import getDatabase from '..';
let geometryHelper: KnexSpatial | undefined;
export function getGeometryHelper(): KnexSpatial {
if (!geometryHelper) {
const db = getDatabase();
const client = db.client.config.client as string;
const constructor = {
mysql: KnexSpatial_MySQL,
mariadb: KnexSpatial_MySQL,
sqlite3: KnexSpatial,
pg: KnexSpatial_PG,
redshift: KnexSpatial_Redshift,
mssql: KnexSpatial_MSSQL,
oracledb: KnexSpatial_Oracle,
}[client];
if (!constructor) {
throw new Error(`Geometry helper not implemented on ${client}.`);
}
geometryHelper = new constructor(db);
}
return geometryHelper;
}
class KnexSpatial {
constructor(protected knex: Knex) {}
isTrue(expression: Knex.Raw) {
return expression;
}
isFalse(expression: Knex.Raw) {
return expression.wrap('NOT ', '');
}
createColumn(table: Knex.CreateTableBuilder, field: RawField | Field) {
const type = field.schema?.geometry_type ?? 'geometry';
return table.specificType(field.field, type);
}
asText(table: string, column: string): Knex.Raw {
return this.knex.raw('st_astext(??.??) as ??', [table, column, column]);
}
fromText(text: string): Knex.Raw {
return this.knex.raw('st_geomfromtext(?, 4326)', text);
}
fromGeoJSON(geojson: GeoJSONGeometry): Knex.Raw {
return this.fromText(geojsonToWKT(geojson));
}
_intersects(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw('st_intersects(??, ?)', [key, geometry]);
}
intersects(key: string, geojson: GeoJSONGeometry): Knex.Raw {
return this.isTrue(this._intersects(key, geojson));
}
nintersects(key: string, geojson: GeoJSONGeometry): Knex.Raw {
return this.isFalse(this._intersects(key, geojson));
}
_intersects_bbox(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw('intersects(??, ?)', [key, geometry]);
}
intersects_bbox(key: string, geojson: GeoJSONGeometry): Knex.Raw {
return this.isTrue(this._intersects_bbox(key, geojson));
}
nintersects_bbox(key: string, geojson: GeoJSONGeometry): Knex.Raw {
return this.isFalse(this._intersects_bbox(key, geojson));
}
collect(table: string, column: string): Knex.Raw {
return this.knex.raw('st_astext(st_collect(??.??))', [table, column]);
}
}
class KnexSpatial_PG extends KnexSpatial {
createColumn(table: Knex.CreateTableBuilder, field: RawField | Field) {
const type = field.schema?.geometry_type ?? 'geometry';
return table.specificType(field.field, `geometry(${type})`);
}
_intersects_bbox(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw('?? && ?', [key, geometry]);
}
}
class KnexSpatial_MySQL extends KnexSpatial {
collect(table: string, column: string): Knex.Raw {
return this.knex.raw(
`concat('geometrycollection(', group_concat(? separator ', '), ')'`,
this.asText(table, column)
);
}
}
class KnexSpatial_Redshift extends KnexSpatial {
createColumn(table: Knex.CreateTableBuilder, field: RawField | Field) {
const type = field.schema?.geometry_type ?? 'geometry';
if (type !== 'geometry') field.meta!.special![1] = type;
return table.specificType(field.field, 'geometry');
}
}
class KnexSpatial_MSSQL extends KnexSpatial {
isTrue(expression: Knex.Raw) {
return expression.wrap(``, ` = 1`);
}
isFalse(expression: Knex.Raw) {
return expression.wrap(``, ` = 0`);
}
createColumn(table: Knex.CreateTableBuilder, field: RawField | Field) {
const type = field.schema?.geometry_type ?? 'geometry';
if (type !== 'geometry') field.meta!.special![1] = type;
return table.specificType(field.field, 'geometry');
}
asText(table: string, column: string): Knex.Raw {
return this.knex.raw('??.??.STAsText() as ??', [table, column, column]);
}
fromText(text: string): Knex.Raw {
return this.knex.raw('geometry::STGeomFromText(?, 4326)', text);
}
_intersects(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw('??.STIntersects(?)', [key, geometry]);
}
_intersects_bbox(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw('??.STEnvelope().STIntersects(?.STEnvelope())', [key, geometry]);
}
collect(table: string, column: string): Knex.Raw {
return this.knex.raw('geometry::CollectionAggregate(??.??).STAsText()', [table, column]);
}
}
class KnexSpatial_Oracle extends KnexSpatial {
isTrue(expression: Knex.Raw) {
return expression.wrap(``, ` = 'TRUE'`);
}
isFalse(expression: Knex.Raw) {
return expression.wrap(``, ` = 'FALSE'`);
}
createColumn(table: Knex.CreateTableBuilder, field: RawField | Field) {
const type = field.schema?.geometry_type ?? 'geometry';
if (type !== 'geometry') field.meta!.special![1] = type;
return table.specificType(field.field, 'sdo_geometry');
}
asText(table: string, column: string): Knex.Raw {
return this.knex.raw('sdo_util.from_wktgeometry(??.??) as ??', [table, column, column]);
}
fromText(text: string): Knex.Raw {
return this.knex.raw('sdo_geometry(?, 4326)', text);
}
_intersects(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw(`sdo_overlapbdyintersect(??, ?)`, [key, geometry]);
}
_intersects_bbox(key: string, geojson: GeoJSONGeometry): Knex.Raw {
const geometry = this.fromGeoJSON(geojson);
return this.knex.raw(`sdo_overlapbdyintersect(sdo_geom.sdo_mbr(??), sdo_geom.sdo_mbr(?))`, [key, geometry]);
}
collect(table: string, column: string): Knex.Raw {
return this.knex.raw(`concat('geometrycollection(', listagg(?, ', '), ')'`, this.asText(table, column));
}
}

View File

@@ -0,0 +1,15 @@
import { Knex } from 'knex';
export async function up(knex: Knex): Promise<void> {
await knex.schema.alterTable('directus_settings', (table) => {
table.json('basemaps');
table.string('mapbox_key');
});
}
export async function down(knex: Knex): Promise<void> {
await knex.schema.alterTable('directus_settings', (table) => {
table.dropColumn('basemaps');
table.dropColumn('mapbox_key');
});
}

View File

@@ -6,6 +6,8 @@ import { AST, FieldNode, NestedCollectionNode } from '../types/ast';
import applyQuery from '../utils/apply-query';
import { toArray } from '@directus/shared/utils';
import getDatabase from './index';
import { isNativeGeometry } from '../utils/geometry';
import { getGeometryHelper } from '../database/helpers/geometry';
type RunASTOptions = {
/**
@@ -143,6 +145,17 @@ async function parseCurrentLevel(
return { columnsToSelect, nestedCollectionNodes, primaryKeyField };
}
function getColumnPreprocessor(knex: Knex, schema: SchemaOverview, table: string) {
const helper = getGeometryHelper();
return function (column: string): Knex.Raw<string> {
const field = schema.collections[table].fields[column];
if (isNativeGeometry(field)) {
return helper.asText(table, column);
}
return knex.raw('??.??', [table, column]);
};
}
function getDBQuery(
schema: SchemaOverview,
knex: Knex,
@@ -151,8 +164,8 @@ function getDBQuery(
query: Query,
nested?: boolean
): Knex.QueryBuilder {
const dbQuery = knex.select(columns.map((column) => `${table}.${column}`)).from(table);
const preProcess = getColumnPreprocessor(knex, schema, table);
const dbQuery = knex.select(columns.map(preProcess)).from(table);
const queryCopy = clone(query);
queryCopy.limit = typeof queryCopy.limit === 'number' ? queryCopy.limit : 100;

View File

@@ -3,7 +3,8 @@ import yaml from 'js-yaml';
import { Knex } from 'knex';
import { isObject } from 'lodash';
import path from 'path';
import { Type } from '@directus/shared/types';
import { Type, Field } from '@directus/shared/types';
import { getGeometryHelper } from '../helpers/geometry';
type TableSeed = {
table: string;
@@ -55,6 +56,9 @@ export default async function runSeed(database: Knex): Promise<void> {
column = tableBuilder.string(columnName);
} else if (columnInfo.type === 'hash') {
column = tableBuilder.string(columnName, 255);
} else if (columnInfo.type === 'geometry') {
const helper = getGeometryHelper();
column = helper.createColumn(tableBuilder, { field: columnName } as Field);
} else {
column = tableBuilder[columnInfo.type!](columnName);
}

View File

@@ -264,3 +264,58 @@ fields:
language: css
lineNumber: true
width: full
- field: map_divider
interface: presentation-divider
options:
icon: map
title: $t:maps
special:
- alias
- no-data
width: full
- field: mapbox_key
interface: input
options:
icon: key
title: Mapbox Access Token
placeholder: pk.eyJ1Ijo.....
iconLeft: vpn_key
font: monospace
width: half
- field: basemaps
interface: list
special: json
options:
template: '{{name}}'
fields:
- field: name
name: $t:name
schema:
is_nullable: false
meta:
interface: text-input
options:
placeholder: Enter the basemap name...
- field: type
name: $t:type
meta:
interface: select-dropdown
options:
choices:
- value: raster
text: Raster
- value: tile
text: Raster TileJSON
- value: style
text: Mapbox Style
- field: url
name: $t:url
schema:
is_nullable: false
meta:
interface: text-input
options:
placeholder: http://{a-c}.tile.openstreetmap.org/{z}/{x}/{y}.png

View File

@@ -7,11 +7,11 @@ import { systemCollectionRows } from '../database/system-data/collections';
import env from '../env';
import { ForbiddenException, InvalidPayloadException } from '../exceptions';
import logger from '../logger';
import { FieldsService, RawField } from '../services/fields';
import { FieldsService } from '../services/fields';
import { ItemsService, MutationOptions } from '../services/items';
import Keyv from 'keyv';
import { AbstractServiceOptions, Collection, CollectionMeta, SchemaOverview } from '../types';
import { Accountability, FieldMeta } from '@directus/shared/types';
import { Accountability, FieldMeta, RawField } from '@directus/shared/types';
export type RawCollection = {
collection: string;

View File

@@ -13,16 +13,14 @@ import { ItemsService } from '../services/items';
import { PayloadService } from '../services/payload';
import { AbstractServiceOptions, SchemaOverview } from '../types';
import { Accountability } from '@directus/shared/types';
import { Field, FieldMeta, Type } from '@directus/shared/types';
import { Field, FieldMeta, RawField, Type } from '@directus/shared/types';
import getDefaultValue from '../utils/get-default-value';
import getLocalType from '../utils/get-local-type';
import { toArray } from '@directus/shared/utils';
import { isEqual, isNil } from 'lodash';
import { RelationsService } from './relations';
import { getGeometryHelper } from '../database/helpers/geometry';
import Keyv from 'keyv';
import { DeepPartial } from '@directus/shared/types';
export type RawField = DeepPartial<Field> & { field: string; type: Type };
export class FieldsService {
knex: Knex;
@@ -77,25 +75,22 @@ export class FieldsService {
fields.push(...systemFieldRows);
}
let columns = await this.schemaInspector.columnInfo(collection);
columns = columns.map((column) => {
return {
...column,
default_value: getDefaultValue(column),
};
});
const columns = (await this.schemaInspector.columnInfo(collection)).map((column) => ({
...column,
default_value: getDefaultValue(column),
}));
const columnsWithSystem = columns.map((column) => {
const field = fields.find((field) => {
return field.field === column.name && field.collection === column.table;
});
const { type = 'alias', ...info } = column ? getLocalType(column, field) : {};
const data = {
collection: column.table,
field: column.name,
type: column ? getLocalType(column, field) : 'alias',
schema: column,
type: type,
schema: { ...column, ...info },
meta: field || null,
};
@@ -202,12 +197,13 @@ export class FieldsService {
// Do nothing
}
const { type = 'alias', ...info } = column ? getLocalType(column, fieldInfo) : {};
const data = {
collection,
field,
type: column ? getLocalType(column, fieldInfo) : 'alias',
type,
meta: fieldInfo || null,
schema: column || null,
schema: type == 'alias' ? null : { ...column, ...info },
};
return data;
@@ -458,6 +454,9 @@ export class FieldsService {
column = table.dateTime(field.field, { useTz: false });
} else if (field.type === 'timestamp') {
column = table.timestamp(field.field, { useTz: true });
} else if (field.type === 'geometry') {
const helper = getGeometryHelper();
column = helper.createColumn(table, field);
} else {
column = table[field.type](field.field);
}

View File

@@ -1,4 +1,5 @@
import argon2 from 'argon2';
import { validateQuery } from '../utils/validate-query';
import {
ArgumentNode,
BooleanValueNode,
@@ -92,6 +93,12 @@ const GraphQLVoid = new GraphQLScalarType({
},
});
export const GraphQLGeoJSON = new GraphQLScalarType({
...GraphQLJSON,
name: 'GraphQLGeoJSON',
description: 'GeoJSON value',
});
export const GraphQLDate = new GraphQLScalarType({
...GraphQLString,
name: 'Date',
@@ -536,6 +543,30 @@ export class GraphQLService {
},
});
const GeometryFilterOperators = schemaComposer.createInputTC({
name: 'geometry_filter_operators',
fields: {
_eq: {
type: GraphQLGeoJSON,
},
_neq: {
type: GraphQLGeoJSON,
},
_intersects: {
type: GraphQLGeoJSON,
},
_nintersects: {
type: GraphQLGeoJSON,
},
_intersects_bbox: {
type: GraphQLGeoJSON,
},
_nintersects_bbox: {
type: GraphQLGeoJSON,
},
},
});
for (const collection of Object.values(schema.read.collections)) {
if (Object.keys(collection.fields).length === 0) continue;
if (SYSTEM_DENY_LIST.includes(collection.collection)) continue;
@@ -557,6 +588,9 @@ export class GraphQLService {
case GraphQLDate:
filterOperatorType = DateFilterOperators;
break;
case GraphQLGeoJSON:
filterOperatorType = GeometryFilterOperators;
break;
default:
filterOperatorType = StringFilterOperators;
}
@@ -1088,6 +1122,8 @@ export class GraphQLService {
query.fields = parseFields(selections);
validateQuery(query);
return query;
}

View File

@@ -172,8 +172,8 @@ export class ItemsService<Item extends AnyItem = AnyItem> implements AbstractSer
activity: activityID,
collection: this.collection,
item: primaryKey,
data: JSON.stringify(payload),
delta: JSON.stringify(payload),
data: await payloadService.prepareDelta(payload),
delta: await payloadService.prepareDelta(payload),
};
const revisionID = (await trx.insert(revisionRecord).into('directus_revisions').returning('id'))[0] as number;
@@ -480,14 +480,23 @@ export class ItemsService<Item extends AnyItem = AnyItem> implements AbstractSer
const snapshots = await itemsService.readMany(keys);
const revisionRecords = activityPrimaryKeys.map((key, index) => ({
activity: key,
collection: this.collection,
item: keys[index],
data:
snapshots && Array.isArray(snapshots) ? JSON.stringify(snapshots?.[index]) : JSON.stringify(snapshots),
delta: JSON.stringify(payloadWithTypeCasting),
}));
const revisionRecords: {
activity: PrimaryKey;
collection: string;
item: PrimaryKey;
data: string;
delta: string;
}[] = [];
for (let i = 0; i < activityPrimaryKeys.length; i++) {
revisionRecords.push({
activity: activityPrimaryKeys[i],
collection: this.collection,
item: keys[i],
data: snapshots && Array.isArray(snapshots) ? JSON.stringify(snapshots[i]) : JSON.stringify(snapshots),
delta: await payloadService.prepareDelta(payloadWithTypeCasting),
});
}
for (let i = 0; i < revisionRecords.length; i++) {
const revisionID = (

View File

@@ -10,6 +10,9 @@ import { AbstractServiceOptions, Item, PrimaryKey, Query, SchemaOverview, Altera
import { Accountability } from '@directus/shared/types';
import { toArray } from '@directus/shared/utils';
import { ItemsService } from './items';
import { isNativeGeometry } from '../utils/geometry';
import { getGeometryHelper } from '../database/helpers/geometry';
import { parse as wktToGeoJSON } from 'wellknown';
type Action = 'create' | 'read' | 'update';
@@ -19,6 +22,7 @@ type Transformers = {
value: any;
payload: Partial<Item>;
accountability: Accountability | null;
specials: string[];
}) => Promise<any>;
};
@@ -148,13 +152,16 @@ export class PayloadService {
})
);
await this.processDates(processedPayload, action);
this.processGeometries(processedPayload, action);
this.processDates(processedPayload, action);
if (['create', 'update'].includes(action)) {
processedPayload.forEach((record) => {
for (const [key, value] of Object.entries(record)) {
if (Array.isArray(value) || (typeof value === 'object' && value instanceof Date !== true && value !== null)) {
record[key] = JSON.stringify(value);
if (Array.isArray(value) || (typeof value === 'object' && !(value instanceof Date) && value !== null)) {
if (!value.isRawInstance) {
record[key] = JSON.stringify(value);
}
}
}
});
@@ -185,6 +192,7 @@ export class PayloadService {
value,
payload,
accountability,
specials: fieldSpecials,
});
}
}
@@ -192,14 +200,40 @@ export class PayloadService {
return value;
}
/**
* Native geometries are stored in custom binary format. We need to insert them with
* the function st_geomfromtext. For this to work, that function call must not be
* escaped. It's therefore placed as a Knex.Raw object in the payload. Thus the need
* to check if the value is a raw instance before stringifying it in the next step.
*/
processGeometries<T extends Partial<Record<string, any>>[]>(payloads: T, action: Action): T {
const helper = getGeometryHelper();
const process =
action == 'read'
? (value: any) => {
if (typeof value === 'string') return wktToGeoJSON(value);
}
: (value: any) => helper.fromGeoJSON(typeof value == 'string' ? JSON.parse(value) : value);
const fieldsInCollection = Object.entries(this.schema.collections[this.collection].fields);
const geometryColumns = fieldsInCollection.filter(([_, field]) => isNativeGeometry(field));
for (const [name] of geometryColumns) {
for (const payload of payloads) {
if (payload[name]) {
payload[name] = process(payload[name]);
}
}
}
return payloads;
}
/**
* Knex returns `datetime` and `date` columns as Date.. This is wrong for date / datetime, as those
* shouldn't return with time / timezone info respectively
*/
async processDates(
payloads: Partial<Record<string, any>>[],
action: Action
): Promise<Partial<Record<string, any>>[]> {
processDates(payloads: Partial<Record<string, any>>[], action: Action): Partial<Record<string, any>>[] {
const fieldsInCollection = Object.entries(this.schema.collections[this.collection].fields);
const dateColumns = fieldsInCollection.filter(([_name, field]) =>
@@ -638,4 +672,22 @@ export class PayloadService {
return { revisions };
}
/**
* Transforms the input partial payload to match the output structure, to have consistency
* between delta and data
*/
async prepareDelta(data: Partial<Item>): Promise<string> {
let payload = cloneDeep(data);
for (const key in payload) {
if (payload[key]?.isRawInstance) {
payload[key] = payload[key].bindings[0];
}
}
payload = await this.processValues('read', payload);
return JSON.stringify(payload);
}
}

View File

@@ -526,6 +526,9 @@ class OASSpecsService implements SpecificationSubService {
type: 'string',
format: 'uuid',
},
geometry: {
type: 'string',
},
};
}

View File

@@ -21,19 +21,3 @@ export type Sort = {
export type Filter = {
[keyOrOperator: string]: Filter | any;
};
export type FilterOperator =
| 'eq'
| 'neq'
| 'contains'
| 'ncontains'
| 'in'
| 'nin'
| 'gt'
| 'gte'
| 'lt'
| 'lte'
| 'null'
| 'nnull'
| 'empty'
| 'nempty';

View File

@@ -2,7 +2,20 @@ import { Type } from '@directus/shared/types';
import { Permission } from './permissions';
import { Relation } from './relation';
type CollectionsOverview = {
export type FieldOverview = {
field: string;
defaultValue: any;
nullable: boolean;
type: Type | 'unknown' | 'alias';
dbType: string | null;
precision: number | null;
scale: number | null;
special: string[];
note: string | null;
alias: boolean;
};
export type CollectionsOverview = {
[name: string]: {
collection: string;
primary: string;
@@ -11,18 +24,7 @@ type CollectionsOverview = {
note: string | null;
accountability: 'all' | 'activity' | null;
fields: {
[name: string]: {
field: string;
defaultValue: any;
nullable: boolean;
type: Type | 'unknown' | 'alias';
dbType: string | null;
precision: number | null;
scale: number | null;
special: string[];
note: string | null;
alias: boolean;
};
[name: string]: FieldOverview;
};
};
};

View File

@@ -5,6 +5,7 @@ import validate from 'uuid-validate';
import { InvalidQueryException } from '../exceptions';
import { Filter, Query, Relation, SchemaOverview } from '../types';
import { getRelationType } from './get-relation-type';
import { getGeometryHelper } from '../database/helpers/geometry';
const generateAlias = customAlphabet('abcdefghijklmnopqrstuvwxyz', 5);
@@ -86,6 +87,7 @@ export default function applyQuery(
* )
* ```
*/
export function applyFilter(
schema: SchemaOverview,
rootQuery: Knex.QueryBuilder,
@@ -403,6 +405,23 @@ export function applyFilter(
dbQuery[logical].whereNotBetween(key, value);
}
const geometryHelper = getGeometryHelper();
if (operator == '_intersects') {
dbQuery[logical].whereRaw(geometryHelper.intersects(key, compareValue));
}
if (operator == '_nintersects') {
dbQuery[logical].whereRaw(geometryHelper.nintersects(key, compareValue));
}
if (operator == '_intersects_bbox') {
dbQuery[logical].whereRaw(geometryHelper.intersects_bbox(key, compareValue));
}
if (operator == '_nintersects_bbox') {
dbQuery[logical].whereRaw(geometryHelper.nintersects_bbox(key, compareValue));
}
}
function getWhereColumn(path: string[], collection: string) {

18
api/src/utils/geometry.ts Normal file
View File

@@ -0,0 +1,18 @@
import { FieldOverview } from '../types';
const dbGeometricTypes = new Set([
'point',
'polygon',
'linestring',
'multipoint',
'multipolygon',
'multilinestring',
'geometry',
'geometrycollection',
'sdo_geometry',
'user-defined',
]);
export function isNativeGeometry(field: FieldOverview): boolean {
const { type, dbType } = field;
return type == 'geometry' && dbGeometricTypes.has(dbType!.toLowerCase());
}

View File

@@ -5,7 +5,7 @@ import getLocalType from './get-local-type';
export default function getDefaultValue(
column: SchemaOverview[string]['columns'][string] | Column
): string | boolean | null {
const type = getLocalType(column);
const { type } = getLocalType(column);
let defaultValue = column.default_value ?? null;
if (defaultValue === null) return null;

View File

@@ -8,7 +8,7 @@ import {
GraphQLType,
} from 'graphql';
import { GraphQLJSON } from 'graphql-compose';
import { GraphQLDate } from '../services/graphql';
import { GraphQLDate, GraphQLGeoJSON } from '../services/graphql';
import { Type } from '@directus/shared/types';
export function getGraphQLType(localType: Type | 'alias' | 'unknown'): GraphQLScalarType | GraphQLList<GraphQLType> {
@@ -25,6 +25,8 @@ export function getGraphQLType(localType: Type | 'alias' | 'unknown'): GraphQLSc
return new GraphQLList(GraphQLString);
case 'json':
return GraphQLJSON;
case 'geometry':
return GraphQLGeoJSON;
case 'timestamp':
case 'dateTime':
case 'date':

View File

@@ -3,7 +3,12 @@ import { Column } from 'knex-schema-inspector/dist/types/column';
import { FieldMeta, Type } from '@directus/shared/types';
import getDatabase from '../database';
const localTypeMap: Record<string, { type: Type; useTimezone?: boolean }> = {
type LocalTypeEntry = {
type: Type | 'unknown';
geometry_type?: 'Point' | 'LineString' | 'Polygon' | 'MultiPoint' | 'MultiLineString' | 'MultiPolygon';
};
const localTypeMap: Record<string, LocalTypeEntry> = {
// Shared
boolean: { type: 'boolean' },
tinyint: { type: 'integer' },
@@ -38,6 +43,15 @@ const localTypeMap: Record<string, { type: Type; useTimezone?: boolean }> = {
decimal: { type: 'decimal' },
numeric: { type: 'integer' },
// Geometries
point: { type: 'geometry', geometry_type: 'Point' },
linestring: { type: 'geometry', geometry_type: 'LineString' },
polygon: { type: 'geometry', geometry_type: 'Polygon' },
multipoint: { type: 'geometry', geometry_type: 'MultiPoint' },
multilinestring: { type: 'geometry', geometry_type: 'MultiLineString' },
multipolygon: { type: 'geometry', geometry_type: 'MultiPolygon' },
geometry: { type: 'geometry' },
// MySQL
string: { type: 'text' },
year: { type: 'integer' },
@@ -49,7 +63,7 @@ const localTypeMap: Record<string, { type: Type; useTimezone?: boolean }> = {
bit: { type: 'boolean' },
smallmoney: { type: 'float' },
money: { type: 'float' },
datetimeoffset: { type: 'timestamp', useTimezone: true },
datetimeoffset: { type: 'timestamp' },
datetime2: { type: 'dateTime' },
smalldatetime: { type: 'dateTime' },
nchar: { type: 'text' },
@@ -73,16 +87,17 @@ const localTypeMap: Record<string, { type: Type; useTimezone?: boolean }> = {
_varchar: { type: 'string' },
bpchar: { type: 'string' },
timestamptz: { type: 'timestamp' },
'timestamp with time zone': { type: 'timestamp', useTimezone: true },
'timestamp with time zone': { type: 'timestamp' },
'timestamp without time zone': { type: 'dateTime' },
timetz: { type: 'time' },
'time with time zone': { type: 'time', useTimezone: true },
'time with time zone': { type: 'time' },
'time without time zone': { type: 'time' },
float4: { type: 'float' },
float8: { type: 'float' },
// Oracle
number: { type: 'integer' },
sdo_geometry: { type: 'geometry' },
// SQLite
integerfirst: { type: 'integer' },
@@ -91,7 +106,7 @@ const localTypeMap: Record<string, { type: Type; useTimezone?: boolean }> = {
export default function getLocalType(
column: SchemaOverview[string]['columns'][string] | Column,
field?: { special?: FieldMeta['special'] }
): Type | 'unknown' {
): LocalTypeEntry {
const database = getDatabase();
const type = localTypeMap[column.data_type.toLowerCase().split('(')[0]];
@@ -99,10 +114,13 @@ export default function getLocalType(
const special = field?.special;
if (special) {
if (special.includes('json')) return 'json';
if (special.includes('hash')) return 'hash';
if (special.includes('csv')) return 'csv';
if (special.includes('uuid')) return 'uuid';
if (special.includes('json')) return { type: 'json' };
if (special.includes('hash')) return { type: 'hash' };
if (special.includes('csv')) return { type: 'csv' };
if (special.includes('uuid')) return { type: 'uuid' };
if (type?.type == 'geometry' && !type.geometry_type) {
type.geometry_type = special[1] as any;
}
}
/** Handle OracleDB timestamp with time zone */
@@ -111,30 +129,30 @@ export default function getLocalType(
if (type.startsWith('timestamp')) {
if (type.endsWith('with local time zone')) {
return 'timestamp';
return { type: 'timestamp' };
} else {
return 'dateTime';
return { type: 'dateTime' };
}
}
}
/** Handle Postgres numeric decimals */
if (column.data_type === 'numeric' && column.numeric_precision !== null && column.numeric_scale !== null) {
return 'decimal';
return { type: 'decimal' };
}
/** Handle MS SQL varchar(MAX) (eg TEXT) types */
if (column.data_type === 'nvarchar' && column.max_length === -1) {
return 'text';
return { type: 'text' };
}
/** Handle Boolean as TINYINT*/
if (column.data_type.toLowerCase() === 'tinyint(1)' || column.data_type.toLowerCase() === 'tinyint(0)') {
return 'boolean';
return { type: 'boolean' };
}
if (type) {
return type.type;
return type;
}
return 'unknown';
return { type: 'unknown' };
}

View File

@@ -140,7 +140,7 @@ async function getDatabaseSchema(
field: column.column_name,
defaultValue: getDefaultValue(column) ?? null,
nullable: column.is_nullable ?? true,
type: getLocalType(column) || 'alias',
type: column ? getLocalType(column).type : ('alias' as const),
dbType: column.data_type,
precision: column.numeric_precision || null,
scale: column.numeric_scale || null,
@@ -168,21 +168,19 @@ async function getDatabaseSchema(
if (!result.collections[field.collection]) continue;
const existing = result.collections[field.collection].fields[field.field];
const column = schemaOverview[field.collection].columns[field.field];
const special = field.special ? toArray(field.special) : [];
const { type = 'alias' } = existing && column ? getLocalType(column, { special }) : {};
result.collections[field.collection].fields[field.field] = {
field: field.field,
defaultValue: existing?.defaultValue ?? null,
nullable: existing?.nullable ?? true,
type:
existing && field.field in schemaOverview[field.collection].columns
? getLocalType(schemaOverview[field.collection].columns[field.field], {
special: field.special ? toArray(field.special) : [],
})
: 'alias',
type: type,
dbType: existing?.dbType || null,
precision: existing?.precision || null,
scale: existing?.scale || null,
special: field.special ? toArray(field.special) : [],
special: special,
note: field.note,
alias: existing?.alias ?? true,
};

View File

@@ -2,7 +2,7 @@ import { flatten, get, merge, set } from 'lodash';
import logger from '../logger';
import { Filter, Meta, Query, Sort } from '../types';
import { Accountability } from '@directus/shared/types';
import { parseFilter } from '@directus/shared/utils';
import { parseFilter, deepMap } from '@directus/shared/utils';
export function sanitizeQuery(rawQuery: Record<string, any>, accountability?: Accountability | null): Query {
const query: Query = {};
@@ -96,6 +96,14 @@ function sanitizeFilter(rawFilter: any, accountability: Accountability | null) {
}
}
filters = deepMap(filters, (val) => {
try {
return JSON.parse(val);
} catch {
return val;
}
});
filters = parseFilter(filters, accountability);
return filters;

View File

@@ -2,6 +2,7 @@ import Joi from 'joi';
import { isPlainObject } from 'lodash';
import { InvalidQueryException } from '../exceptions';
import { Query } from '../types';
import { stringify } from 'wellknown';
const querySchema = Joi.object({
fields: Joi.array().items(Joi.string()),
@@ -41,8 +42,6 @@ function validateFilter(filter: Query['filter']) {
for (const [key, nested] of Object.entries(filter)) {
if (key === '_and' || key === '_or') {
nested.forEach(validateFilter);
} else if (isPlainObject(nested)) {
validateFilter(nested);
} else if (key.startsWith('_')) {
const value = nested;
@@ -74,8 +73,17 @@ function validateFilter(filter: Query['filter']) {
case '_nempty':
validateBoolean(value, key);
break;
case '_intersects':
case '_nintersects':
case '_intersects_bbox':
case '_nintersects_bbox':
validateGeometry(value, key);
break;
}
} else if (isPlainObject(nested) === false && Array.isArray(nested) === false) {
} else if (isPlainObject(nested)) {
validateFilter(nested);
} else if (Array.isArray(nested) === false) {
validateFilterPrimitive(nested, '_eq');
} else {
validateFilter(nested);
@@ -121,3 +129,13 @@ function validateBoolean(value: any, key: string) {
return true;
}
function validateGeometry(value: any, key: string) {
try {
stringify(value);
} catch {
throw new InvalidQueryException(`"${key}" has to be a valid GeoJSON object`);
}
return true;
}