From bf85e6f2fcd93a6dc9c998bb0c9ef06f4b2a50db Mon Sep 17 00:00:00 2001 From: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com> Date: Thu, 31 Oct 2024 14:25:12 -0600 Subject: [PATCH] docs: add detailed documentation for sources (#37) Adds more detailed documentation for sources. --- README.md | 3 ++ docs/sources/README.md | 31 ++++++++++++++++ docs/sources/alloydb-pg.md | 70 ++++++++++++++++++++++++++++++++++++ docs/sources/cloud-sql-pg.md | 66 ++++++++++++++++++++++++++++++++++ docs/sources/postgres.md | 42 ++++++++++++++++++++++ 5 files changed, 212 insertions(+) create mode 100644 docs/sources/README.md create mode 100644 docs/sources/alloydb-pg.md create mode 100644 docs/sources/cloud-sql-pg.md create mode 100644 docs/sources/postgres.md diff --git a/README.md b/README.md index 5136f30d01..4225ca2b7e 100644 --- a/README.md +++ b/README.md @@ -133,6 +133,9 @@ sources: database: my_db ``` +For more details on configuring different types of sources, see the [Source +documentation.](docs/sources/README.md) + ### Tools diff --git a/docs/sources/README.md b/docs/sources/README.md new file mode 100644 index 0000000000..4ccb9a3a22 --- /dev/null +++ b/docs/sources/README.md @@ -0,0 +1,31 @@ +# Sources + +Sources represent a data source that a tool can interact with. You can define +Sources as a map in the `sources` section of your `tools.yaml` file. Typically, +a source configuration will contain any information needed to connect with and +interact with the database. + +```yaml +sources: + my-cloud-sql-source: + kind: cloud-sql-postgres + project: my-project-name + region: us-central1 + instance: my-instance-name + database: my_db + user: my-user + password: my-password +``` + +In implementation, each source is a different connection pool or client that used +to connect to the database and execute the tool. + +## Kinds of Sources + +We currently support the following types of kinds of sources: + +* [alloydb-postgres](./alloydb-pg.md) - Connect to an AlloyDB for PostgreSQL + cluster. +* [cloud-sql-postgres](./cloud-sql-pg.md) - Connect to a Cloud SQL for + PostgreSQL instance. +* [postgres](./postgres.md) - Connect to any PostgreSQL compatible database. diff --git a/docs/sources/alloydb-pg.md b/docs/sources/alloydb-pg.md new file mode 100644 index 0000000000..70126db7f5 --- /dev/null +++ b/docs/sources/alloydb-pg.md @@ -0,0 +1,70 @@ +# AlloyDB for PostgreSQL Source + +[AlloyDB for PostgreSQL][alloydb-docs] is a fully-managed, PostgreSQL-compatible +database for demanding transactional workloads. It provides enterprise-grade +performance and availability while maintaining 100% compatibility with +open-source PostgreSQL. + +[alloydb-docs]: https://cloud.google.com/alloydb/docs + +## Requirements + +### IAM Identity +By default, AlloyDB for PostgreSQL source uses the [AlloyDB Go +Connector][alloydb-go-conn] to authorize and establish mTLS connections to your +AlloyDB instance. The Go connector uses your [Application Default Credentials +(ADC)][adc] to authorize your connection to AlloyDB. + +In addition to [setting the ADC for your server][set-adc], you need to ensure the +IAM identity has been given the following IAM permissions: +- `roles/alloydb.client` +- `roles/serviceusage.serviceUsageConsumer` + +[alloydb-go-conn]: https://github.com/GoogleCloudPlatform/alloydb-go-connector +[adc]: https://cloud.google.com/docs/authentication#adc +[set-adc]: https://cloud.google.com/docs/authentication/provide-credentials-adc + +### Network Path + +Currently, this source only supports [connecting over Private +IP][private-ip]. Most notably, this means +you need to connect from a VPC that AlloyDB has been connected to. + +[private-ip]: https://cloud.google.com/alloydb/docs/private-ip + +### Database User + +Current, this source only uses standard authentication. You will need to [create a +PostreSQL user][alloydb-users] to login to the database with. + +[alloydb-users]: https://cloud.google.com/alloydb/docs/database-users/about + +## Example + +```yaml +sources: + my-alloydb-pg-source: + kind: "alloydb-pg-postgres" + project: "my-project-name" + region: "us-central1" + cluster: "my-cluster" + instance: "my-instance" + database: "my_db" + user: "my-user" + password: "my-password" +``` + +## Reference + +| **field** | **type** | **required** | **description** | +|-----------|:--------:|:------------:|------------------------------------------------------------------------------| +| kind | string | true | Must be "alloydb-postgres". | +| project | string | true | Name of the GCP project that the cluster was created in (e.g. "my-project"). | +| region | string | true | Name of the GCP region that the cluster was created in (e.g. "us-central1"). | +| cluster | string | true | Name of the AlloyDB cluster (e.g. "my-cluster"). | +| instance | string | true | Name of the AlloyDB instance within the cluser (e.g. "my-instance"). | +| database | string | true | Name of the Postgres database to connect to (e.g. "my_db"). | +| user | string | true | Name of the Postgres user to connect as (e.g. "my-pg-user"). | +| password | string | true | Password of the Postgres user (e.g. "my-password"). | + + diff --git a/docs/sources/cloud-sql-pg.md b/docs/sources/cloud-sql-pg.md new file mode 100644 index 0000000000..40d0e6738b --- /dev/null +++ b/docs/sources/cloud-sql-pg.md @@ -0,0 +1,66 @@ +# Cloud SQL for PostgreSQL Source + +[Cloud SQL for PostgreSQL][csql-pg-docs] is a fully-managed database service +that helps you set up, maintain, manage, and administer your PostgreSQL +relational databases on Google Cloud Platform. + +[csql-pg-docs]: https://cloud.google.com/sql/docs/postgres + +## Requirements + +### IAM Identity +By default, this source uses the [Cloud SQL Go Connector][csql-go-conn] to +authorize and establish mTLS connections to your Cloud SQL instance. The Go +connector uses your [Application Default Credentials (ADC)][adc] to authorize +your connection to Cloud SQL. + +In addition to [setting the ADC for your server][set-adc], you need to ensure the +IAM identity has been given the following IAM roles: +- `roles/cloudsql.client` + +[csql-go-conn]: https://github.com/GoogleCloudPlatform/cloud-sql-go-connector +[adc]: https://cloud.google.com/docs/authentication#adc +[set-adc]: https://cloud.google.com/docs/authentication/provide-credentials-adc + +### Network Path + +Currently, this source only supports [connecting over Public IP][public-ip]. +Because it uses the Go connector, is uses rotating client certificates to +establish a secure mTLS connection with the instance. + +[public-ip]: https://cloud.google.com/sql/docs/postgres/configure-ip + +### Database User + +Current, this source only uses standard authentication. You will need to [create a +PostreSQL user][cloud-sql-users] to login to the database with. + +[cloud-sql-users]: https://cloud.google.com/sql/docs/postgres/create-manage-users + +## Example + +```yaml +sources: + my-cloud-sql-pg-source: + kind: "cloud-sql-pg-postgres" + project: "my-project" + region: "us-central1" + instance: "my-instance" + database: "my_db" + user: "my-user" + password: "my-password" +``` + +## Reference + +| **field** | **type** | **required** | **description** | +|-----------|:--------:|:------------:|------------------------------------------------------------------------------| +| kind | string | true | Must be "cloud-sql-postgres". | +| project | string | true | Name of the GCP project that the cluster was created in (e.g. "my-project"). | +| region | string | true | Name of the GCP region that the cluster was created in (e.g. "us-central1"). | +| instance | string | true | Name of the Cloud SQL instance within the cluser (e.g. "my-instance"). | +| database | string | true | Name of the Postgres database to connect to (e.g. "my_db"). | +| user | string | true | Name of the Postgres user to connect as (e.g. "my-pg-user"). | +| password | string | true | Password of the Postgres user (e.g. "my-password"). | + + diff --git a/docs/sources/postgres.md b/docs/sources/postgres.md new file mode 100644 index 0000000000..c741d1a77e --- /dev/null +++ b/docs/sources/postgres.md @@ -0,0 +1,42 @@ +# PostgreSQL Source + +[PostgreSQL][pg-docs] is a powerful, open source object-relational database +system with over 35 years of active development that has earned it a strong +reputation for reliability, feature robustness, and performance. + +[pg-docs]: https://www.postgresql.org/ + +## Requirements + +### Database User + +This source only uses standard authentication. You will need to [create a +PostreSQL user][pg-users] to login to the database with. + +[pg-users]: https://www.postgresql.org/docs/current/sql-createuser.html + +## Example + +```yaml +sources: + my-pg-source: + kind: "postgres" + host: "127.0.0.1" + port: "5432" + database: "my_db" + user: "my-user" + password: "my-password" +``` + +## Reference + +| **field** | **type** | **required** | **description** | +|-----------|:--------:|:------------:|------------------------------------------------------------------------| +| kind | string | true | Must be "postgres". | +| host | string | true | IP address to connect to (e.g. "127.0.0.1") | +| port | string | true | Port to connect to (e.g. "5432") | +| database | string | true | Name of the Postgres database to connect to (e.g. "my_db"). | +| user | string | true | Name of the Postgres user to connect as (e.g. "my-pg-user"). | +| password | string | true | Password of the Postgres user (e.g. "my-password"). | + +